# Resilience patterns for OpenAI, Anthropic, xAI and Gemini clients

Status: DOCUMENTED (rules) · LIVE_VERIFIED (reference implementation exercised end-to-end via the four provider adapters: OpenAI/Anthropic 2026-09-18, xAI/Gemini 2026-09-19) · offline unit tests: tests/shared/test_resilient_client.py (46 tests incl. recorded xAI/Gemini error bodies) + resilientClient.ts --selftest Sources:

• https://developers.openai.com/api/docs/guides/error-codes · …/guides/rate-limits · …/guides/background (stream resume) · …/guides/production-best-practices · OpenAI OpenAPI spec (Idempotency-Key, sequence_number)
• https://platform.claude.com/docs/en/api/errors · …/api/rate-limits · …/build-with-claude/streaming (in-stream error) · …/manage-claude/compliance-errors (x-should-retry)
• xAI: https://docs.x.ai/developers/debugging · https://docs.x.ai/developers/rate-limits (exponential backoff 2**attempt, 5 retries; Batch API exempt) · generated/fragments/errors/xai-errors.json, headers/xai-headers.json (live 2026-09-19: 400 bad key, 422 string body, x-ratelimit-* headers)
• Gemini: https://ai.google.dev/gemini-api/docs/api-errors · …/troubleshooting · …/rate-limits · generated/fragments/errors/gemini-errors.json (live 429 body with QuotaFailure + RetryInfo), headers/gemini-headers.json Last verified: 2026-09-19

Implementation: examples/shared/resilient-client/resilient_client.py (stdlib) and resilientClient.ts (fetch). Both are transport-pluggable so the whole policy is unit-testable offline. ResilientClient(provider) for openai | anthropic | xai | gemini (env keys *_API_KEY, base URLs https://api.openai.com, https://api.anthropic.com, https://api.x.ai, https://generativelanguage.googleapis.com; auth Bearer / x-api-key / Bearer xai-… / x-goog-api-key).

# 1. Error envelopes (parse before classifying)

# 2. Error classification (what to retry)

Server hints that override everything:

• x-should-retry: false|true (Anthropic) — applied whenever present.
• Retry-After (OpenAI 429/503; Anthropic 429/529; absent on xAI and Gemini). Treat as a minimum, add jitter, cap it (max_retry_after_s, 60 s).
• Gemini: parse_gemini_retry_delay(body) prefers RetryInfo.retryDelay (protobuf Duration string "40s", or {seconds, nanos}), then the message regex Please retry in (\d+(\.\d+)?)s. Both were observed together in one body (40 s vs 54.22 s) — the structured value wins.
• Anthropic in-stream event: error (overloaded_error) after HTTP 200 → classify like the status. Gemini generateContent SSE has no documented mid-stream error event (a {"error": …} chunk is still handled); a stream that closes without finishReason is a failure. Gemini Interactions streams emit event: error.

# 3. Backoff

Full jitter: sleep = U(0, min(cap, base·2^(attempt-1))) (defaults base 0.5 s, cap 20 s), bounded by both max_attempts (4) and a wall-clock max_total_s (120 s). All four vendors recommend exponential backoff with jitter: OpenAI adds "limit both the number of attempts and the total time"; xAI's docs show 2**attempt with 5 retries and point bulk work to the Batch API (exempt from rate limits); Gemini's troubleshooting page says 429/408/5xx only and its SDK defaults to 4 attempts, ~1 s initial, 60 s max.

SDK interaction: openai 3.16.2 and anthropic 1.7.0 retry 2× by default (DEFAULT_MAX_RETRIES = 2); google-genai 2.24 retries 429/5xx via http_options.retry_options; the openai SDK against https://api.x.ai/v1 keeps OpenAI's defaults; xai-sdk (gRPC) surfaces grpc.StatusCode and the docs' sample loop catches RESOURCE_EXHAUSTED. When you own the policy, disable SDK retries (max_retries=0, retry_options=None) and feed exceptions to sdk_error_is_retryable(exc, provider=…) (pass provider="xai" explicitly for the OpenAI SDK-on-xAI case; gRPC errors are mapped by e.code()).

# 4. Timeouts and connection pooling

• Per-request: connect 5 s, read 600 s (OpenAI/Anthropic SDK defaults). xAI reasoning models can be slow: the docs use 3600 s client timeouts and xai-sdk defaults to 1620 s; Gemini's SDK sends its http_options.timeout as X-Server-Timeout and long Flex/Deep-Research jobs should use background: true (Interactions) or Batch rather than a long socket.
• Prefer streaming for long generations on all four providers (Anthropic's SDKs refuse non-streaming requests expected to exceed 10 min; Gemini 504 DEADLINE_EXCEEDED guidance says the same).
• Python stdlib urllib (reference client) opens a connection per request; production: http.client keep-alive, httpx.Client(limits=...), or the SDKs. Node fetch (undici) pools by default. xAI prompt caches are per server: keep a conversation on one host and send x-grok-conv-id (Chat) / prompt_cache_key (Responses) so retries land on the same cache.

# 5. Rate-limit headers (observed shape, parse defensively)

parse_rate_limit_headers() returns a RateLimitInfo; for Gemini it is intentionally empty except raw. Never generalise observed values into "documented limits" (xAI's header values are below the documented Tier 0 RPS and may be a new-team allowance).

# 6. Idempotency — facts

• OpenAI: Idempotency-Key documented on exactly one operation (POST /v1/agents/sessions/{id}/events). Anthropic: none. xAI: none documented on any endpoint; POST /v1/files/{id}/public-url is idempotent by behaviour (same URL returned). Gemini: none on generateContent/Interactions; files.name custom ids and File Search store names give create-once semantics (409 ALREADY_EXISTS). Webhook receivers dedupe on webhook-id (OpenAI, Anthropic, Gemini static webhooks).
• Consequence: a retried POST after a timeout may have succeeded server-side. Mitigations: stream; OpenAI background: true + resume; xAI store: true + GET /v1/responses/{id} (live: retrievable even with store:false) or deferred completions (deferred: true, result fetchable once within 24 h); Gemini Interactions background: true + GET /v1beta/interactions/{id}; client-side dedupe keyed on an id you also place in metadata (OpenAI/Anthropic) / labels (Gemini) — xAI Responses rejects metadata, so use safety_identifier/user or prompt_cache_key.

# 7. Stream reconnection

• OpenAI Responses: background: true, stream: true + sequence_number cursor → GET /v1/responses/{id}?stream=true&starting_after=N.
• xAI Responses: same event vocabulary with sequence_number, but background is rejected (400) → no resume; restart the request (keep the cursor for logs only). WebSocket mode (wss://api.x.ai/v1/responses) is the documented alternative for long agentic runs.
• Anthropic Messages: no resume → anthropic_stream_with_restart (synthetic restart marker; also on in-stream error).
• Gemini generateContent: no resume; every chunk carries running usageMetadata and the last chunk finishReason — a close without it means restart. Gemini Interactions: resumable SSE GET /v1beta/interactions/{id}?stream=true&last_event_id=… for background: true interactions; Live API: sessionResumption handles (valid 2 h) after goAway/connection reset (~10 min).
• All: keep the emitted text length to de-duplicate on resume/restart.

# 8. Circuit breaker

Per provider (optionally per model): closed → open after 5 consecutive failures → half-open after 30 s → one probe. Open circuits raise CircuitOpen immediately so fallback chains skip a degraded provider.

# 9. Fallbacks

call_with_fallback([Target…]) tries targets in order; Target.path may contain {model} (Gemini: /v1beta/models/{model}:generateContent).

• Same provider, other model on RetryExhausted, CircuitOpen, 404 model_not_found (xAI: 404 not-found for restricted models; Gemini: 404 "no longer available to new users"), transport errors, and Gemini 429 limit: 0 (the model is paid-only for this key → try a Flash/Lite model).
• Other provider — the request body must be rebuilt (Responses ≠ Messages ≠ generateContent); use the adapters. Only portable parameters survive; reasoning artefacts never do.
• Never fall back on auth/validation errors: OpenAI/Anthropic authentication_error/permission_error/invalid_request_error, xAI invalid-argument (incl. incorrect key)/unauthenticated*, Gemini INVALID_ARGUMENT/UNAUTHENTICATED/PERMISSION_DENIED/FAILED_PRECONDITION (AUTH_OR_VALIDATION_ERROR_TYPES) — the same error will recur and you would mask a real bug. Tested chain: Gemini limit: 0 → xAI (bad key stops the chain) / → Anthropic (succeeds).

# 10. Budget guard

BudgetGuard(max_usd, prices) converts usage into USD from any of the four shapes: OpenAI/xAI Responses (input_tokens, output_tokens, input_tokens_details.cached_tokens), OpenAI/xAI Chat (prompt_tokens, completion_tokens + completion_tokens_details.reasoning_tokens, prompt_tokens_details.cached_tokens), Anthropic (cache_read_input_tokens, cache_creation_input_tokens), Gemini usageMetadata (promptTokenCount, candidatesTokenCount + thoughtsTokenCount, cachedContentTokenCount). xAI also returns usage.cost_in_usd_ticks (1 tick = $1e-8) — the authoritative per-request cost when available. Prices come from generated/pricing.json; unknown models use a deliberately high default.

# 11. Checklist

• Parse the envelope first ({error:{}}, {code,error}, bare string, google.rpc, array-wrapped); classify by status and type/code/message.
• Never retry: quota/spend codes, Anthropic spend-cap 429, xAI 400 (incl. bad key) / 422, Gemini FAILED_PRECONDITION and limit: 0 429s.
• Honour Retry-After and x-should-retry; for Gemini read RetryInfo.retryDelay / "Please retry in Ns"; cap the hint; add jitter; xAI gets pure backoff.
• Bound retries by attempts and elapsed time; disable SDK retries when you own the loop; long client timeouts for reasoning models.
• Stream long generations; resume only where supported (OpenAI background, Gemini Interactions last_event_id, Live sessionResumption); restart elsewhere.
• Treat POST retries as non-idempotent: dedupe client-side (metadata / labels / your own store).
• Log x-request-id (OpenAI, xAI) / request-id (Anthropic) / body responseId (Gemini) with every failure.
• Circuit-break per provider; fall back only on capacity or model-availability errors.
• Guard spend with a price-table-driven budget (xAI: cross-check cost_in_usd_ticks).