# LLM enrichment — gateway, prompts, worker, budgets

LLMs are enrichment, never the crawler (spec §2.4, §24–25). Everything deterministic happens first (`docs/EVENT-TAXONOMY.md`); the model only
sees bounded, already-diffed content, answers in strict JSON, and every output is validated, versioned and attributed.

## Configuration (`config.Settings`)

| Setting | Default | Meaning |
|---|---|---|
| `CA_LLM_BASE_URL` / `CA_LLM_API_KEY` | `https://www.llm-api.io/v1` (`.env`) / key in `deploy/.llm-key` (git-ignored) | OpenAI-compatible endpoint; `settings.llm_configured` = enabled **and** URL set |
| `CA_LLM_ENABLED` | true | master switch — when off, jobs stay pending and the worker logs once |
| `CA_LLM_SMALL_MODEL` / `MEDIUM` / `LARGE` / `EMBEDDING` | `qwen3-4b-instruct-2507-4bit` / `qwen3.6-35b-a3b-4bit` / `qwen3.8-27b-4bit` / `qwen3-embedding-0.6b-8bit` | tiers used by tasks |
| `CA_LLM_DAILY_BUDGET` | 1500 | max jobs per UTC day (enqueue side counts created jobs, worker side counts finished jobs) |
| `CA_LLM_MIN_SIGNIFICANCE` | 0.40 | changes below this never reach a model |
| `CA_LLM_TIMEOUT` | 300 s | per request (the server loads models on demand: first call 30–60 s) |
| `CA_LLM_MAX_TRIES` / `BACKOFF_INITIAL_S` / `BACKOFF_MAX_S` | 6 / 15 / 120 | 429/503/5xx/timeouts: 15 → 30 → 60 → 120 → 120 s (Retry-After honoured) |
| `CA_LLM_JOB_MAX_ATTEMPTS` | 3 | retryable failures re-queue the job until this |
| `CA_LLM_CONTEXT_BLOCK_BYTES` | 3072 | before/after text per block sent to the model (≤ 12 blocks) |
| `CA_PROMPTS_DIR` | `<repo>/prompts` | prompt files location |

## Gateway (`services/llm/gateway.py`)

`LLMProvider` (abstract) → `OpenAICompatibleProvider` (httpx, `/chat/completions`, `/models`, `/embeddings`):

- `complete_json(tier, system, user, schema, max_tokens)` — `response_format={"type": "json_object"}`, the compact JSON schema of the pydantic model
  appended to the system prompt, `<think>` blocks / code fences / prose tolerated when extracting the object, **strict pydantic validation**, and
  **one repair round-trip** (the model sees its own answer plus the validation error). Still invalid → `LLMValidationError` (job failed, nothing stored).
- `complete_text(...)`, `embed(texts)` (embedding tier; reserved for later search/clustering work), `health()` (model list + latency).
- Returns `LLMResult(data, model, request_tokens, response_tokens, latency_ms, attempts, repaired)`; tokens go to `llm_jobs` and `cost_ledger`
  (`dimension='llm'`, key = model, units = tokens).
- `get_provider()` / `set_provider()` for a process-wide instance or test doubles. The gateway is the only module allowed to call the endpoint.

Sticky model per stream: the worker drains one job kind at a time (classification → small model, summaries → medium model) so the on-demand
server does not evict models between calls. Use `catlas llm-test ["prompt"] [--tier medium]` for a live health + round-trip check.

## Prompts (`prompts/<task>/v<N>.md`)

Loaded by `services/llm/prompts.load_prompt(task, version=None)` (latest `vN` by default); the leading `<!-- … -->` comment is metadata, the rest is
the **system** prompt. The user message is always a compact JSON context — untrusted page text is never interpolated into instructions.
`events.prompt_version` stores `task/vN`; bump the file (v2.md) when wording changes materially and keep v1 for reproducibility.

| Prompt | Tier | Schema (`services/llm/schemas.py`, version) | Used by |
|---|---|---|---|
| `change-classifier/v1` | small | `ChangeClassification` (`classify-v1`): subtype ∈ `EVENT_SUBTYPES` else OTHER, importance, confidence, title ≤ 120, summary ≤ 400, old/new value, entities, tags, language, `is_material` | `classify_change` jobs |
| `event-summarizer/v1` | medium | `EventSummary` (`summary-v1`): summary ≤ 400, key_points ≤ 5, confidence, language | `summarize_event` (non-legal) |
| `legal-diff/v1` | medium | `LegalDiffSummary` (`legal-v1`): sections_changed[{section, change}], materiality ∈ editorial/minor/material/unclear, summary, user_impact | `summarize_event` for LEGAL events |
| `industry-tagger/v1` | small | `IndustryTags` (`industry-v1`): slugs from the allowed list only, primary, keywords | `classify_industry` jobs → `companies.source_meta.llm_industries` (suggestion only) |
| `ask-router/v1` | small | `AskRoute` (`ask-v1`): intent, countries, industries, event types/subtypes, companies, window, keywords, answer_style | `services/llm/ask.route_question` (optional refinement of the deterministic parser) |

All schemas reject the forbidden wording (`taxonomy.FORBIDDEN_WORDING`: fired, laid off, shut down…) at validation time; prompts state the ban and
the observational language ("detected", "no longer listed").

## Worker (`services/llm/enrich.py`)

`run_llm_jobs(limit)` — periodic `llm-enrich` every 15 s, `catlas enrich [--limit] [--once]`:

1. Budget check (finished jobs today < `CA_LLM_DAILY_BUDGET`), pick the sticky kind, claim jobs with `FOR UPDATE SKIP LOCKED` (`attempts += 1`).
2. Build the bounded context: company meta (name, domain, country, industries, description ≤ 400), surface, source URL, significance/kind, block
   counts, ≤ 12 diff blocks (before/after ≤ 3 kB each), structured deltas with lists trimmed to 10 items.
3. Call the gateway, validate, write:
   - `classify_change` → material & non-OTHER & importance ≥ 0.25 → new event `origin='llm'` (confidence capped at 0.9, `status='review'` +
     review_queue when < 0.5, dedupe `sha(company, 'llm', change, subtype)`, event_sources, clustering, alerts). If a deterministic event with the
     same subtype already exists for that change, it is enriched instead (`origin='hybrid'`, `payload.llm_classification`). `changes.status='enriched'`.
   - `summarize_event` → `events.summary` (previous kept in `payload.summary_prev`), `origin='hybrid'`, `model_provider/model_name/prompt_version/
     schema_version`, `payload.llm` (key points, or legal sections + materiality + tag `materiality:<x>`).
   - `classify_industry` → `companies.source_meta.llm_industries` (never overwrites registry industries).
4. `llm_jobs` gets status, model, prompt_version, tokens, latency, `result` JSON or `error`; retryable failures go back to `pending` until
   `CA_LLM_JOB_MAX_ATTEMPTS`, a retryable failure also stops the current batch (the server is unhealthy — the periodic task returns).

Enqueue side (`services/events.py`): `classify_change` for meaningful+ changes with no deterministic event or an ambiguous surface;
`summarize_event` for legal / homepage / messaging / news / IR events that have diff content. Both only when `settings.llm_configured`, above
`CA_LLM_MIN_SIGNIFICANCE`, within budget, and never twice for the same ref.

## `/ask` helper (`services/llm/ask.py`)

`parse_question(q, industries=, countries=)` is deterministic: countries (names/demonyms + registry table), industries (registry slugs/names),
intents → event types/subtypes/tags (hiring, pricing ± increase/decrease, AI, launch, leadership, expansion, legal, developer, financing, M&A,
communication), windows ("last 3 months", "this week", "since 2024", today/yesterday), quoted company names, answer style (list / count / compare /
timeline / trend), `min_importance` for "major/important". `route_question()` adds the LLM refinement only when configured and only to tighten
filters; on any failure the deterministic interpretation is returned. `build_answer()` phrases counts the API measured — it never invents results.

## Live check (2026-09-12)

`catlas llm-test` against `https://www.llm-api.io/v1`: health OK (15 models listed), small model round-trip **4.5 s**, 96 prompt / 19 completion
tokens, no repair needed. Live enrichment of factory changes: `classify_change` (small) 3.9 s; three `summarize_event` (medium, incl. legal-diff)
3.1–8.2 s, 0 failures.

## Retention & cost

`llm_jobs` finished > `CA_RETENTION_LLM_JOBS_DAYS` (60) are summarised into `cost_ledger` (`archived:<model>` tokens, `archived-jobs:<model>` count)
and deleted (`services/retention.py`); events keep their model/prompt attribution forever.