# LLM enrichment — gateway, prompts, worker, budgets

LLMs are enrichment, never the crawler (spec §2.4, §24–25). Everything deterministic happens first (docs/EVENT-TAXONOMY.md); the model only sees bounded, already-diffed content, answers in strict JSON, and every output is validated, versioned and attributed.

# Configuration (config.Settings)

# Gateway (services/llm/gateway.py)

LLMProvider (abstract) → OpenAICompatibleProvider (httpx, /chat/completions, /models, /embeddings):

• complete_json(tier, system, user, schema, max_tokens) — response_format={"type": "json_object"}, the compact JSON schema of the pydantic model appended to the system prompt, <think> blocks / code fences / prose tolerated when extracting the object, strict pydantic validation, and one repair round-trip (the model sees its own answer plus the validation error). Still invalid → LLMValidationError (job failed, nothing stored).
• complete_text(...), embed(texts) (embedding tier; reserved for later search/clustering work), health() (model list + latency).
• Returns LLMResult(data, model, request_tokens, response_tokens, latency_ms, attempts, repaired); tokens go to llm_jobs and cost_ledger (dimension='llm', key = model, units = tokens).
• get_provider() / set_provider() for a process-wide instance or test doubles. The gateway is the only module allowed to call the endpoint.

Sticky model per stream: the worker drains one job kind at a time (classification → small model, summaries → medium model) so the on-demand server does not evict models between calls. Use catlas llm-test ["prompt"] [--tier medium] for a live health + round-trip check.

# Prompts (prompts/<task>/v<N>.md)

Loaded by services/llm/prompts.load_prompt(task, version=None) (latest vN by default); the leading <!-- … --> comment is metadata, the rest is the system prompt. The user message is always a compact JSON context — untrusted page text is never interpolated into instructions. events.prompt_version stores task/vN; bump the file (v2.md) when wording changes materially and keep v1 for reproducibility.

All schemas reject the forbidden wording (taxonomy.FORBIDDEN_WORDING: fired, laid off, shut down…) at validation time; prompts state the ban and the observational language ("detected", "no longer listed").

# Worker (services/llm/enrich.py)

run_llm_jobs(limit) — periodic llm-enrich every 15 s, catlas enrich [--limit] [--once]:

1. Budget check (finished jobs today < CA_LLM_DAILY_BUDGET), pick the sticky kind, claim jobs with FOR UPDATE SKIP LOCKED (attempts += 1).
2. Build the bounded context: company meta (name, domain, country, industries, description ≤ 400), surface, source URL, significance/kind, block counts, ≤ 12 diff blocks (before/after ≤ 3 kB each), structured deltas with lists trimmed to 10 items.
3. Call the gateway, validate, write:
   • classify_change → material & non-OTHER & importance ≥ 0.25 → new event origin='llm' (confidence capped at 0.9, status='review' + review_queue when < 0.5, dedupe sha(company, 'llm', change, subtype), event_sources, clustering, alerts). If a deterministic event with the same subtype already exists for that change, it is enriched instead (origin='hybrid', payload.llm_classification). changes.status='enriched'.
   • summarize_event → events.summary (previous kept in payload.summary_prev), origin='hybrid', model_provider/model_name/prompt_version/ schema_version, payload.llm (key points, or legal sections + materiality + tag materiality:<x>).
   • classify_industry → companies.source_meta.llm_industries (never overwrites registry industries).
4. llm_jobs gets status, model, prompt_version, tokens, latency, result JSON or error; retryable failures go back to pending until CA_LLM_JOB_MAX_ATTEMPTS, a retryable failure also stops the current batch (the server is unhealthy — the periodic task returns).

Enqueue side (services/events.py): classify_change for meaningful+ changes with no deterministic event or an ambiguous surface; summarize_event for legal / homepage / messaging / news / IR events that have diff content. Both only when settings.llm_configured, above CA_LLM_MIN_SIGNIFICANCE, within budget, and never twice for the same ref.

# /ask helper (services/llm/ask.py)

parse_question(q, industries=, countries=) is deterministic: countries (names/demonyms + registry table), industries (registry slugs/names), intents → event types/subtypes/tags (hiring, pricing ± increase/decrease, AI, launch, leadership, expansion, legal, developer, financing, M&A, communication), windows ("last 3 months", "this week", "since 2024", today/yesterday), quoted company names, answer style (list / count / compare / timeline / trend), min_importance for "major/important". route_question() adds the LLM refinement only when configured and only to tighten filters; on any failure the deterministic interpretation is returned. build_answer() phrases counts the API measured — it never invents results.

# Live check (2026-09-12)

catlas llm-test against https://www.llm-api.io/v1: health OK (15 models listed), small model round-trip 4.5 s, 96 prompt / 19 completion tokens, no repair needed. Live enrichment of factory changes: classify_change (small) 3.9 s; three summarize_event (medium, incl. legal-diff) 3.1–8.2 s, 0 failures.

# Retention & cost

llm_jobs finished > CA_RETENTION_LLM_JOBS_DAYS (60) are summarised into cost_ledger (archived:<model> tokens, archived-jobs:<model> count) and deleted (services/retention.py); events keep their model/prompt attribution forever.