spb/ultra-sharp-agent-skills Public

1<!--2Author: Simon-Pierre Boucher3Contact: contact@spboucher.ai4-->56# Patterns — Scaling Backend Services78## Contents9- Connection-pool arithmetic10- pgbouncer minimal config11- Kubernetes HPA on a custom metric12- Load balancer health + draining (nginx)13- Queue-based load leveling sketch14- Load-test skeleton (k6)15- Gotchas1617## Connection-pool arithmetic1819```20per-instance pool  = cores × 2                  # CPU-bound starting point21fleet demand       = replicas × per-instance pool22must satisfy       : fleet demand < db max_connections − superuser_reserved2324Example: 12 replicas × 10 pool = 120 > Postgres default 100 → pgbouncer required.25```2627IO-wait-heavy workloads (slow downstreams inside transactions) justify larger28pools — but first shorten the transaction, don't widen the pool.2930## pgbouncer minimal config3132```ini33[databases]34app = host=10.0.0.5 port=5432 dbname=app3536[pgbouncer]37listen_port = 643238auth_type = scram-sha-25639pool_mode = transaction        ; multiplexes: many clients, few server conns40default_pool_size = 20         ; server-side connections per db/user pair41max_client_conn = 200042server_idle_timeout = 6043```4445`pool_mode = transaction` breaks session state (prepared statements, advisory46locks, `SET`) — verify the driver's compatibility mode before enabling.4748## Kubernetes HPA on a custom metric4950```yaml51apiVersion: autoscaling/v252kind: HorizontalPodAutoscaler53spec:54  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }55  minReplicas: 356  maxReplicas: 24            # capped by DB pool arithmetic, not by budget alone57  metrics:58    - type: Pods59      pods:60        metric: { name: http_p95_latency_ms }61        target: { type: AverageValue, averageValue: "250" }62  behavior:63    scaleUp:64      stabilizationWindowSeconds: 0      # react to spikes immediately65      policies: [{ type: Percent, value: 100, periodSeconds: 60 }]66    scaleDown:67      stabilizationWindowSeconds: 300    # shrink slowly — thrash guard68      policies: [{ type: Pods, value: 1, periodSeconds: 120 }]69```7071## Load balancer health + draining (nginx)7273```nginx74upstream api {75    least_conn;76    server 10.0.1.10:8000 max_fails=3 fail_timeout=10s;77    server 10.0.1.11:8000 max_fails=3 fail_timeout=10s;78    server 10.0.1.12:8000 slow_start=30s;   # ramp recovered instances gradually79}80```8182App side (draining): on SIGTERM, fail the readiness endpoint, keep serving83in-flight requests, exit after they finish or a 30 s deadline:8485```python86def handle_sigterm(*_):87    ready.set_unhealthy()        # LB stops sending new traffic88    server.shutdown(grace=30)    # finish in-flight, then exit89```9091## Queue-based load leveling sketch9293```python94# ingest: accept fast, defer work95@app.post("/imports")96def create_import(req):97    job_id = queue.enqueue("imports", req.body, idempotency_key=req.headers["Idempotency-Key"])98    return 202, {"job_id": job_id, "status_url": f"/imports/{job_id}"}99100# worker pool drains at a sustainable rate; consumers are idempotent101def worker():102    for job in queue.consume("imports", prefetch=1):103        if already_processed(job.idempotency_key):   # replay-safe104            job.ack(); continue105        process(job)106        mark_processed(job.idempotency_key)107        job.ack()108```109110Worker count — not producer rate — sets DB write pressure; scale workers only111while the DB stays under its ceiling.112113## Load-test skeleton (k6)114115```javascript116import http from 'k6/http';117export const options = {118  stages: [119    { duration: '2m', target: 200 },   // ramp120    { duration: '5m', target: 200 },   // steady — realistic think time below121    { duration: '2m', target: 800 },   // find the knee122  ],123  thresholds: { http_req_duration: ['p(95)<300'] },124};125export default function () {126  http.get('https://staging.example.com/api/orders');127}128```129130Test against staging with production-shaped data; an empty database lies.131132## Gotchas133134- **Sticky sessions are hidden state** — they break draining and uneven-load the fleet; externalize the session instead.135- **Autoscaling on CPU while blocked on IO** does nothing: instances idle at 20% CPU while every request waits on the pool. Scale on the metric that saturates.136- **`least_conn` beats round-robin** once request costs vary, but health checks matter more than the algorithm.137- **Replica lag is not constant** — it spikes exactly when you're overloaded, i.e., when you rerouted reads there. Monitor lag and fail reads back gracefully.138- **Local caches per instance multiply cold starts** during scale-up; a scale-out event can stampede the DB (see caching-strategies for stampede locks).139- **Max-instance caps get deleted in incidents** — document WHY the cap exists (DB arithmetic) next to the setting.140