spb/ultra-sharp-agent-skills Public

1---2name: scaling-backend-services3description: Scales backend services to handle load — bottleneck measurement first, stateless app tiers, connection pooling, load balancing, autoscaling on the saturating metric, queue-based load leveling, and read replicas. Use when the user asks how to scale a service or API, handle more traffic or concurrent users, fix connection exhaustion, size a connection pool, add a load balancer or autoscaling, or prepare for a traffic spike. Do not use for SQL query tuning (db-skills/optimizing-sql-performance), cache design (caching-strategies), or rate limiting (limiting-request-rates).4---56<!--7Author: Simon-Pierre Boucher8Contact: contact@spboucher.ai9-->1011# Scaling Backend Services1213## When to use / when NOT to use14- **Use for:** capacity planning, horizontal/vertical scaling decisions, pool sizing, load balancing, autoscaling, spike preparation.15- **Do NOT use for:** query-level tuning (db-skills/optimizing-sql-performance), caching design (caching-strategies), rate limiting (limiting-request-rates), or frontend performance.1617## Core rules18191. **Measure the bottleneck before scaling anything.** Load-test with realistic traffic shape and identify the saturating resource (CPU, memory, DB connections, IO, downstream API). Scaling the wrong tier spends money to move the queue.20   - ✅ "p99 collapses at 800 RPS; DB connections pinned at max" → fix pooling.21   - ❌ "It's slow, add more pods."222. **Vertical first when it's cheaper than complexity.** Doubling instance size is a config change; sharding is a project. Buy headroom while you build the durable fix.233. **Stateless app tier is the precondition for horizontal scaling.** Sessions, uploads, and locks live in external stores (Redis/object storage/DB) so any instance can serve any request and instances can die freely.244. **Pool every connection.** App→DB and app→downstream. DB pool starting point: `cores × 2` for CPU-bound work; more only for IO-wait-heavy loads. Many app replicas × pool size must stay under the DB's max — put a server-side pooler (pgbouncer) in front of Postgres once replicas multiply.255. **Load balance with health checks and connection draining.** Instances that fail readiness stop receiving traffic; deploys drain in-flight requests before termination — zero-downtime is a MUST-pass test, not a hope.266. **Autoscale on the saturating metric, not CPU by reflex.** Queue depth, p95 latency, or connections — whatever rule 1 found. Scale up fast, down slowly (thrash guard), and cap max instances below what the database can survive (rule 4 arithmetic).277. **Level spiky writes through a queue.** Accept fast (202 + job id), process at a sustainable rate, make consumers idempotent. The queue absorbs the spike; the worker pool sets the drain rate.288. **Read replicas for read-heavy loads — with lag eyes open.** Route reads that tolerate staleness to replicas; read-your-own-writes flows stay on the primary.2930## Workflow31321. Load-test to current breaking point; record the saturating metric and p95/p99 at each load step.332. Externalize any instance state found (rule 3).343. Size pools by rule 4 arithmetic across ALL replicas; add pgbouncer if the sum approaches DB max_connections.354. Configure LB health checks + draining; set autoscaling on the measured metric with up-fast/down-slow policies and a hard max.365. Queue spiky write paths (rule 7).376. Validate: re-run the load test — the previous breaking point passes; kill one instance at full load and confirm zero failed requests; verify autoscaler adds and (slowly) removes instances.3839## Edge cases & failure modes40- **Thundering herd after downtime** → LB slow-start / gradual traffic ramp for recovering instances; jittered client retries.41- **Autoscaler scales app until the DB dies** → rule 6 cap; the database's ceiling is the fleet's ceiling.42- **One hot key/tenant saturates a single shard/instance** → consistent hashing with hot-key detection; isolate the noisy tenant.43- **Long-lived connections (websockets) defeat draining** → set max connection age; force periodic reconnect so deploys can complete.4445## References46Pool math, autoscaling configs, load-test sketches: see [references/patterns.md](references/patterns.md).47