---
name: handling-errors
description: Designs error handling for backend services — error taxonomy, HTTP problem+json responses (RFC 9457), retry semantics with backoff and jitter, circuit breakers, and fail-fast startup. Use when the user asks how to handle, structure, or standardize errors or exceptions in an API or service, design error responses, add retries or a circuit breaker, or fix swallowed/double-logged exceptions. Do not use for validating request input (validating-input) or for logging/metrics/tracing pipelines (instrumenting-observability).
---

<!--
Author: Simon-Pierre Boucher
Contact: contact@spboucher.ai
-->

# Handling Errors

## When to use / when NOT to use
- **Use for:** structuring errors and exceptions in backend code, API error responses, retry/circuit-breaker policy, startup failure behavior.
- **Do NOT use for:** request-validation rules and 400-level field errors (validating-input) or log/metric/trace plumbing (instrumenting-observability).

## Core rules

1. **Expected failures are values, exceptional failures are exceptions.** "User not found" is a normal outcome — return it. A broken DB socket is exceptional — raise it. Follow the language idiom (Result/Either in Rust/TS-fp, exceptions in Python/Java).
2. **Keep the taxonomy small.** 4–6 error classes mapped to HTTP: `ValidationError`→400, `AuthError`→401/403, `NotFound`→404, `Conflict`→409, `RateLimited`→429, everything else→500. New error types must argue their way in.
3. **API errors are problem+json (RFC 9457).**
   - ✅ `{"type":"https://api.example.com/errors/quota","title":"Quota exceeded","status":429,"detail":"Plan allows 100 reports/day.","instance":"/reports/123"}`
   - ❌ `{"error": "something went wrong"}` or a raw stack trace.
4. **Log or re-raise — never both.** Handling an exception twice produces double logs and masks the real failure point. Catch only where you can act; add context and re-raise otherwise.
   - ✅ `except PaymentError as e: raise OrderError(order_id=id) from e`
   - ❌ `except Exception: logger.error(e); raise` at every layer.
5. **User-facing messages say what to do next; internals stay in logs.** ✅ "Payment declined — try another card." ❌ "psycopg2.OperationalError: connection refused at 10.0.3.7:5432".
6. **Retry only idempotent operations,** with exponential backoff + full jitter (base 200 ms, factor 2, cap 30 s, max 5 attempts — bounded work, no thundering herd) and respect `Retry-After` when present.
7. **Wrap flaky dependencies in a circuit breaker** (open after 5 consecutive failures, half-open probe after 30 s) so one dead downstream doesn't exhaust your threads.
8. **Fail fast at startup.** Missing config, unreachable migrations, bad credentials → crash with a named cause before serving traffic. Never boot into a half-working state.

## Workflow

1. List the operation's failure modes; split expected vs exceptional.
2. Map each to the taxonomy (rule 2) and its HTTP status; add a new class only if none fits.
3. Implement handlers at the boundary layer only (HTTP middleware / job wrapper), converting taxonomy → problem+json.
4. Add retry/breaker policy for each external dependency (rules 6–7).
5. Validate: trigger each failure mode in a test — assert the status code, the problem+json shape, exactly one log entry per failure, and no stack trace in the response body.

## Edge cases & failure modes
- **Partial failure in a batch** → return 207-style per-item results or a summary object; never fail the whole batch silently.
- **Retryable error during a non-idempotent call** → do not retry; surface it. Make the call idempotent first (idempotency keys) if retries are required.
- **Error while handling an error** (e.g., logger down) → last-resort handler writes to stderr and returns a static 500 body; never raise from the handler.
- **Timeout vs failure ambiguity** → treat timeouts as unknown-outcome: only retry with an idempotency key.

## References
Deeper recipes and gotchas: see [references/patterns.md](references/patterns.md)