zero gravitySOLUTIONS
← All insightsCLOUD & OPERATIONS / CAPACITY PROTECTION GUIDE

Rate limits that protect both tenants and shared capacity.

QUICK ANSWER

Separate edge abuse control, tenant allowance, and shared execution capacity. API Gateway and WAF limits are useful but approximate; enforce business quotas through authenticated identity and protect expensive dependencies with explicit admission rules.

Inside this article 6 sections

Multi-tenant API rate limiting needs several layers: edge abuse protection, authenticated tenant admission, and limits on the shared resources that requests consume. A request-per-second cap alone cannot protect capacity when different requests perform very different amounts of work.

On AWS, API Gateway throttling and WAF rate-based rules are useful controls, but neither should be mistaken for a precise tenant spending limit or a substitute for application authorization. The application still needs to know which tenant is asking and what capacity the operation requires.

Separate three limiting decisions

The edge asks whether traffic resembles abuse or exceeds a broad arrival policy. The application asks whether the authenticated tenant may start another operation under its plan and current usage. The worker or dependency boundary asks whether enough shared capacity exists to execute the operation now.

These decisions can return different outcomes. An abusive request may be blocked. A tenant exceeding its short-term allowance may receive a retryable limit response. A valid large export may be durably queued and admitted to execution later. Define the behavior so clients know whether an operation was rejected or accepted.

Never derive the tenant budget solely from an arbitrary request header. Resolve tenant identity through the authentication and authorization path, then use the canonical account key for admission. Otherwise a caller can change the header and receive another allowance.

Original protection layers · illustrative SaaS export API
01
Edge traffic policy

WAF and gateway controls reduce abusive or excessive arrivals.

02
Authenticated tenant allowance

Account identity determines short-term tokens and product entitlement.

03
Execution capacity

Concurrent exports and downstream calls stay within a measured envelope.

Passing one layer does not bypass the next. Each protects a different boundary and produces a separately observable decision.

Understand the AWS controls and their limits

API Gateway REST API usage plans can associate API keys with throttling and quota targets. AWS documents these controls as best effort, not hard ceilings. API keys identify clients for usage plans; they should not be used as the authentication or authorization mechanism.

WAF rate-based rules aggregate traffic using configured keys and an evaluation window. They are designed for rate-based protection rather than exact per-request accounting, and enforcement can lag changes in traffic. An IP-based rule can also group many legitimate users behind one corporate network.

Use these controls to reduce unwanted load before it reaches expensive application paths. Keep precise contractual quotas, paid entitlements, or irreversible spend decisions in an application-controlled system with the required consistency. Do not promise a hard customer limit merely because a gateway dashboard displays a number.

Represent expensive operations in capacity units

For an illustrative export service, a small metadata lookup might cost one admission unit while a large export costs ten. These are internal scheduling weights, not prices. Choose weights from observed resource consumption and revise them when workloads change.

A token bucket can permit short bursts while controlling sustained admission. The following function demonstrates one local bucket decision with nonnegative inputs, elapsed time in seconds, and a positive cost. It is useful for arithmetic tests, not as a distributed limiter.

def admit(tokens, capacity, refill_per_second, elapsed, cost):
    available = min(capacity, tokens + refill_per_second * elapsed)
    if available < cost:
        return False, available
    return True, available - cost

assert admit(0, 20, 4, 2, 10) == (False, 8)
assert admit(0, 20, 4, 3, 10) == (True, 2)
assert admit(19, 20, 4, 10, 10) == (True, 10)

Across application instances, update the bucket atomically in an appropriate shared store or allocate controlled local allowances. A read-then-write race can admit more work than intended. Define clock handling, key expiry, failure behavior, and the acceptable approximation rather than treating the in-memory example as production infrastructure.

Protect concurrency as well as arrival rate

A tenant can stay below its arrival rate while holding many long-running operations open. Add concurrency limits around the resource that saturates: export slots, database connections, or partner calls. Release permits on every completion path and recover abandoned permits after process failure.

Combine tenant fairness with a global limit. In an illustrative pool of 24 export slots, allowing one account to occupy all 24 defeats isolation even if its request rate is modest. A policy could reserve headroom or cap a single tenant's concurrent work, but the exact numbers should follow measured workload and product promises.

A queue can defer accepted work, yet queueing is not free. Bound backlog age and storage, expose expected state, and keep one tenant's large burst from indefinitely delaying others. The capacity planning guide explains why a queue cannot repair a sustained arrival rate above completion capacity.

Make retry behavior part of the contract

Return a clear distinction between rejected admission and durable acceptance. For a rejected request, provide appropriate retry guidance and encourage jitter so clients do not synchronize at the next boundary. For an accepted asynchronous operation, return its operation ID and let clients query status.

Use idempotency to make retries safe, but decide whether repeated requests consume admission units. A request that merely retrieves an already accepted operation may deserve a cheaper path than a new expensive export. The tenant-aware idempotency guide covers the identity boundary needed for that distinction.

Keep limit responses separate from business errors in telemetry. A correctly enforced admission decision can still indicate that a tenant’s plan or workload needs attention. Review sustained rejection with product owners instead of automatically raising every limit until the shared system saturates.

Test noisy and quiet tenants together

Run a burst from one synthetic tenant while another sends a small steady workload. Observe per-tenant admission, completion latency, shared concurrency, and rejection reasons. Repeat with long-running requests, multiple API keys for one account, and several users behind one IP address.

Simulate limiter-store failure and worker crashes. The system should follow a deliberate availability policy and recover capacity permits without granting unlimited expensive work. Architecture design can connect edge rules, tenant identity, and dependency admission into limits that protect both customer fairness and shared capacity.

FROM INSIGHT TO ACTIONSoftware architecture consulting

Build rate limits around real workload costs.

We can align API Gateway and WAF policies with tenant-aware admission, dependency capacity, and noisy-neighbor tests.

Explore architecture consultingDiscuss your architecture

Start with your goals.
We’ll work through the next step together.