Stopping Runaway Agents: Rate Limits and Quotas at the Gateway

Define what the loop is allowed to spend

An agent polling the same export job 10,000 times can send perfectly valid requests. Input validation will accept them; authorization may permit every one. The missing control is how often that authorized operation may run. The MCP tools specification explicitly requires servers to rate limit tool invocations, but the operator still has to choose the accounting unit and enforcement policy.

Start with a concrete failure scenario: a CI agent keeps checking an unchanged job status after losing its stopping condition. Record the tool, upstream requests per invocation, normal polling interval, and longest useful run. Set a per-run call budget alongside the gateway limits so that a patient loop eventually stops.

Count model tokens and upstream work separately. One tool invocation might fetch several pages or start an expensive database query. A tool-call quota bounds admitted invocations; it becomes a cost ceiling only if the work and price behind each invocation are also bounded.

Attach counters to authenticated agent identities

Give each independently operated agent a distinct gateway credential. Derive the counter identity from the authenticated principal and workspace, rather than a caller-supplied agent name. Ten CI workers behind one network address need separate budgets; a worker changing its claimed name must retain the same counter. The credential assignment pattern is covered in one scoped key per agent.

Layer a workspace ceiling over those individual counters. Otherwise, creating more agents multiplies the available throughput, and a fleet can exhaust a shared upstream allowance while every agent stays within its own limit. Where several workspaces share one upstream account, add a counter for that actual dependency too.

Treat credential rotation as an accounting decision: preserve the logical agent's consumption when replacing its key. Restarting a VM, reconnecting MCP, or rotating a credential should not accidentally buy the same job another monthly allowance. Document which identity survives each operation.

Separate sustained rate, burst capacity and concurrency

For a gateway you operate, an illustrative starting policy might allow one call per second on average, a five-call burst, two calls in flight, 300 calls per run, and 20,000 admitted calls per calendar month. These are example values to tune against measured workflows, not SPYN3 settings.

A token bucket separates refill rate from bucket capacity. With that example, an idle agent can spend five accumulated tokens immediately, then obtains another token each second. AWS documents this rate-and-burst model for API Gateway. A fixed minute counter behaves differently: traffic clustered around a reset can produce two adjacent bursts. State which algorithm your policy uses before interpreting a load test.

Limit in-flight work independently. Two slow exports can consume both slots even while the rate bucket has tokens. Bound upstream execution time and result size as well, and avoid an unlimited waiting queue that merely postpones the overload. A timeout releases the caller's wait; verify separately whether the upstream operation actually stopped.

Make the monthly quota an admission decision

A sustained allowance of 60 calls per minute permits 86,400 calls over 24 hours. Slowing the polling loop therefore does little to constrain a month of unattended execution. Define the monthly period, reset timezone, charged operation, and behavior at exhaustion before handing out credentials.

For a strict ceiling in your own gateway, reserve quota atomically before dispatch. The check and increment must share one concurrency-safe operation across replicas; separate reads and writes let parallel requests spend the last allowance repeatedly. Reject when no allowance remains. If the counter store is unavailable, reject quota-controlled dispatches until enforcement recovers. Keep the authoritative quota durable across gateway restarts.

Choose explicit accounting semantics. One defensible policy charges each admitted tool invocation, including upstream failures, because those attempts may already consume resources. Rejections never reach the upstream and do not consume that invocation quota. Gateway-generated upstream retries need their own bounded attempt budget. Keep attempted, admitted, rejected and completed counts distinct.

Read the enforcement guarantee, not just the configuration label. AWS explicitly says API Gateway usage-plan throttles and quotas are best effort and can be exceeded. Its documentation warns against relying on them to control costs. If overshoot is unacceptable, evaluate a separate strict admission mechanism.

Return a useful rejection without creating a retry loop

At an HTTP request gate, RFC 6585 defines 429 for excessive requests and permits a Retry-After header. RFC 9110 allows that header to contain a delay in seconds or an HTTP date. For a temporary rate rejection before dispatch, an illustrative HTTP response is:

HTTP/1.1 429 Too Many Requests
Content-Type: text/plain; charset=utf-8
Retry-After: 60
Cache-Control: no-store

Agent rate limit exceeded. This request was not dispatched.
Wait at least 60 seconds before another attempt.

This is an HTTP rejection example, not a standardized MCP tool-result body or a promise that every MCP client automatically waits. Test the actual client. Distinguish temporary throttling from monthly exhaustion in the error details; a monthly reset should terminate the current run rather than leave it sleeping and retrying indefinitely.

For retryable failures, assign one layer responsibility for retries, cap attempts and total elapsed time, and use exponential backoff with jitter. AWS's reliability guidance explains how retries across layers multiply load and how jitter spreads synchronized retries. An illustrative run policy permits at most two retries, always within its remaining deadline and call allowance.

Do not automatically replay an ambiguous write after a timeout. RFC 9110 restricts retries of non-idempotent requests; use a documented idempotency mechanism or reconcile the original operation first. A transport failure does not establish that the upstream did nothing.

Containing tool-call loops with SPYN3

SPYN3 puts one MCP endpoint per workspace in front of connected systems, with a per-agent rate limit of 120 calls per minute and a monthly tool-call quota per plan. Keys can be scoped to chosen sources, and writes need a separate grant. Every proxied tool call is audit-logged with source, tool, outcome and duration. If a loop keeps running, revoke its key or disconnect the source in the dashboard; the change takes effect on the next call. Pair these controls with a stopping condition in the agent run.

Prove that rejected calls never reach the upstream

Use a staging upstream that counts received requests, then exercise the gateway with the same client and parallelism as the intended agent. Client-visible errors alone cannot prove that enforcement happened before dispatch. Include these acceptance checks:

1. Send beyond the configured burst through multiple gateway replicas. Compare upstream receipts with admissions and check that restarting a replica does not restore spent allowance.
2. Leave one monthly admission available and issue simultaneous calls. For a strict quota, exactly one may be admitted; the others must produce no upstream requests.
3. Hold upstream responses open. Confirm the in-flight ceiling, bounded queue behavior, timeout handling and eventual slot release.
4. Inject repeated rate rejections and upstream failures. Confirm the client respects the wait hint where supported and exits when its retry or run budget ends.
5. Exhaust one agent's allowance, then call with another agent. Verify the intended individual isolation and shared workspace ceiling.
6. Revoke a looping agent's credential and try another call. Check the rejection and separately observe any work already in flight.

Monitor rejection reason, remaining quota, upstream attempts and repeated calls to the same tool. Alert while enough allowance remains for an operator to investigate. For the emergency action once a loop is identified, follow revoking an agent's access centrally. Raising the limit should follow an explanation of the repeated calls, rather than serve as the default response to exhaustion.