# name: limiting-request-rates description: Designs rate limiting and backpressure for APIs and services — token-bucket limits, limit keys, 429 responses with Retry-After, tiered and weighted limits, distributed enforcement, and load shedding. Use when the user asks to add rate limiting or throttling, protect an API from abuse or overload, return proper 429 responses, set per-user or per-API-key quotas, or handle traffic spikes with backpressure. Do not use for authorization and permissions (implementing-authorization) or capacity planning and autoscaling (scaling-backend-services).

# Limiting Request Rates

# When to use / when NOT to use

• Use for: request-rate limits, quotas, throttling, and load shedding on APIs and internal services.
• Do NOT use for: who-may-do-what decisions (→ implementing-authorization), fleet sizing (→ scaling-backend-services), or client-side retry logic in isolation.

# Core rules

1. Token bucket is the default algorithm: refill rate = sustained limit, bucket size = allowed burst (start: burst 2× the per-second rate). Fixed windows create boundary spikes; sliding-log costs memory — use them only with a measured reason.
2. Choose the limit key deliberately. Default: per API key/user ID (identity-based). Per-IP only as an anti-abuse fallback for unauthenticated routes — corporate NATs and CGNAT put thousands of users behind one IP.
   • ✅ ratelimit:{api_key}:{endpoint_class}
   • ❌ one global per-IP limit in front of a login page used by offices
3. A rejected request gets 429 + headers: Retry-After (seconds), X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset — on EVERY response, not just rejections, so clients can pace themselves before hitting the wall.
4. Weight endpoints by cost. One search = many token units, one health check = fewer. A single uniform limit either strangles cheap calls or lets expensive ones melt the backend (start: expensive endpoints 10× the base cost).
5. Distributed enforcement uses a central store (Redis) with a local fallback: on store failure, each node enforces limit / node_count locally rather than dropping enforcement entirely.
6. Fail-open by default, fail-closed on sensitive routes. If the limiter breaks, serve traffic (availability first) — EXCEPT login, signup, password reset, and payment endpoints, which fail-closed because unlimited tries there is the actual attack.
7. Load-shed before collapse: when saturated (queue depth/latency past threshold), reject early with 503 + Retry-After at the edge; a fast no is kinder than a timeout after 30 s of held resources.
8. Document the limits where API consumers read (limits, windows, headers, expected backoff). An undocumented limit is indistinguishable from an outage to the client.

# Workflow

1. Inventory endpoints; group into cost classes and mark sensitive routes (fail-closed set).
2. Set sustained rate + burst per class per key type; pick limit keys (rule 2).
3. Implement token bucket in Redis (atomic Lua/INCR+EXPIRE pattern) with the local fallback; emit the rule-3 headers everywhere.
4. Add metrics: rejections per key/class, top offenders, limiter latency.
5. Validate: hammer one key past its limit and confirm 429 + correct Retry-After while a second key sails through; kill Redis and confirm the fallback behavior matches rule 6 per route.

# Edge cases & failure modes

• Legitimate burst (batch import, retry storm after your own outage) → allow temporary overrides per key rather than raising the global limit.
• Clock skew across nodes breaks window math → keep all timing in the central store, not node clocks.
• 429 storms from misbehaving clients that don't back off → escalate: exponential penalty windows per repeat offender.
• Internal service-to-service calls need limits too (a runaway internal loop looks exactly like an attack) — but budget them separately from customer quotas.

# References

Redis token-bucket implementation and header helpers: see references/patterns.md.