Latency and Cost of Proxied MCP Tool Calls

Measure The Whole Call Path

The honest latency model for a proxied MCP call starts with the protocol message, not with the infrastructure diagram. A client sends tools/call to the MCP server it knows about. With a proxy, that server authenticates the caller, maps the tool name to a source, forwards or performs the upstream work, shapes the result and returns a normal MCP tool result. The proxy is in the path, so it can add time. It is also the only place that can consistently enforce policy, measure duration and keep per-client setup from spreading across every agent.

{
  "jsonrpc": "2.0",
  "id": 42,
  "method": "tools/call",
  "params": {
    "name": "github/search_issues",
    "arguments": {
      "query": "repo:acme/api label:incident is:open"
    }
  }
}

A useful timing breakdown is client_to_proxy + proxy_policy + proxy_to_upstream + upstream_work + result_shaping + proxy_to_client. If the gateway is deployed far from the upstream API or the agent, the network pieces dominate. If the upstream API is slow, the proxy hop may be noise. If policy does database lookups on every call, proxy_policy becomes the part to fix. Do not collapse those terms into one average; the p95 for each term tells you where the user actually waits.

Discovery Can Cost More Than The Hop

The first hidden cost is often not execution latency. It is the context cost of discovery. MCP tool-capable servers answer tools/list with tool names, descriptions and input schemas. That payload may be inserted into the model context by the client, and large tool catalogs can make the model pay attention to irrelevant choices before it ever calls a tool. A proxy that exposes every upstream tool at once can make this worse. A proxy that reveals only the active source or task-specific toolset can make it better.

The 2026 MCP spec makes this easier to reason about because list results can carry ttlMs and cacheScope, and servers should return tools in deterministic order when the underlying set has not changed. Those two details matter for cost. Stable discovery helps clients cache the same tool list and avoid repeated prompt churn. A short list reduces the number of tool descriptions the model must inspect. The latency saved here is not measured in socket time alone; it appears as fewer model tokens and fewer wrong tool attempts.

For rollout, capture three numbers before and after proxying: bytes returned by tools/list, approximate tokens loaded into the agent from tool definitions, and how many tool names are visible during the task. If the proxy adds 30 ms but removes hundreds of unused tool descriptions from every turn, the end-to-end result may still be cheaper and more reliable.

Retries Need A Budget

Retry policy is where proxied tool calls can quietly become expensive. A direct client might retry once. An upstream SDK might retry twice. A proxy might add its own retry around the whole call. If those layers are not coordinated, one user request can fan out into several upstream requests, each with its own token output, rate-limit cost and waiting time.

Treat retries as a budget attached to the original tools/call. The proxy should know whether an upstream failure happened before work started, after a timeout, after a partial response or after the remote system accepted a write. Read-only calls can usually be retried more safely than write calls, but the proxy should still bound attempts and preserve a clear error when the budget is exhausted. For write-capable tools, the safer default is usually no automatic retry unless the upstream operation is explicitly idempotent or the connector can provide an idempotency key.

Fanout Changes The Math

Some proxy calls are one-to-one: one client call routes to one upstream tool. Others are fanout calls: the proxy checks several sources, merges results or asks a generated connector and a federated MCP server for adjacent data. Fanout is useful, but it changes both latency and cost. Sequential fanout adds the sum of upstream latencies. Parallel fanout usually waits for the slowest required source. Optional fanout can return partial results if a noncritical source misses its deadline.

Make that contract explicit in the tool shape. A tool named search_all should declare which sources it searches and whether it returns partial results. A narrower tool such as linear/search_issues should not unexpectedly call GitHub, Sentry and Stripe. Anthropic's tool guidance is relevant here: fewer, clearer tools and high-signal responses reduce mistakes, while pagination, filtering and truncation keep large responses from filling the context. In a proxy, that means the cheapest tool is often the one with the most precise source boundary.

A practical pattern is to keep aggregation out of the default path. Let the agent call a source-specific search first, then escalate to broader fanout when the user asks for cross-system evidence. This keeps the common path fast while preserving the option to spend more time and tokens when the question truly needs it.

Doing this with SPYN3

One measurement, not a benchmark: on 2026-10-14, from Germany to SPYN3's Amsterdam API, navigation calls such as list_toolsets and enable_toolset took about 0.3–0.8 s round trip. A DeepWiki question proxied through SPYN3 took 10.8 s, almost all of it upstream. On cost, only calls that reach a source count toward a plan. Navigation is free, and the Free plan includes 500 tool calls a month.

A Rollout Scorecard

Before moving an agent from direct MCP servers to a proxy, run one representative workflow both ways. Record tools/list size, visible tool count, first successful tool-call latency, p95 tool-call latency, upstream error rate, retry count and response token size. The goal is not to prove that proxying is always faster. The goal is to decide whether the added hop buys enough control to justify itself for this workflow.

1. Set a per-call deadline at the proxy and a shorter upstream deadline inside it, so the proxy has time to return a useful error.
2. Keep discovery deterministic and cacheable where the protocol allows it.
3. Prefer source-specific tools for the common path and reserve fanout for explicit cross-system questions.
4. Cap response size with pagination, filters or concise formats before the model sees the result.
5. Log latency by phase, not just one total duration.

If those numbers are visible, the proxy stops being a vague architecture tax. You can see whether it is acting as a slow middlebox, a useful control plane or both. Then you can tune the thing that actually costs you: network placement, upstream behavior, retries, tool discovery or result size.