Event taxonomy — deterministic rules, wording policy, importance & confidence
Owner: intelligence layer (services/events.py, services/clustering.py, rules version rules-v1, schema event-v1).
Vocabulary: taxonomy.EVENT_SUBTYPES (subtype → type + default importance). The LLM classifier may only pick from that list.
Pipeline
text
changes (status='pending', kind ≥ meaningful) ─▶ derive_events(change, company, sensor, baseline) pure rules, no I/O
─▶ persist_change_events() events + event_sources + clusters + review_queue + llm_jobs
─▶ changes.status = 'processed' · sensors.event_count · companies.last_event_at
─▶ alerts.evaluate_alerts(new_event_ids)catlas process-changes [--limit] [--loop]· periodicprocess-changesevery 20 s (SKIP LOCKED, batch 200).catlas reprocess-events --since 7d [--company]re-runs the rules on processed changes without refetching; dedupe keys make it idempotent, new rules simply add the events they now produce.- Noise / minor changes never produce events (they are archived if still pending). Meaningful (≥ 0.40), major (≥ 0.65), critical (≥ 0.85) do.
Rules → subtypes
| Family | Input (structured_delta) |
Subtype(s) | Title pattern |
|---|---|---|---|
| HIRING | jobs.added/removed/open_before/open_after |
JOB_COUNT_INCREASE when net > 0 (entities.jobs ≤ 50) |
12 new positions detected on careers page |
JOB_COUNT_DECREASE when net < 0 |
7 monitored job listings no longer visible on careers page | ||
AI keyword in added titles (taxonomy.AI_KEYWORDS) or is_ai |
AI_HIRING |
2 AI-related positions detected on careers page | |
| ≤ 5 jobs added | NEW_JOB per job |
New position listed: Senior ML Engineer (Toronto, CA) | |
added ≥ mean + 2σ of baselines.jobs_new_weekly (≥ 4 samples) — fallback ≥ 10 and ≥ 50 % of open_before |
HIRING_SURGE |
Hiring surge signal: 25 new positions detected in one observation (baseline ≈ 3.0 new/week) | |
| removed ≥ 10, ≥ 50 % of open_before and open_after ≤ 50 % | HIRING_FREEZE_SIGNAL (+ review unexpected_activity) |
Hiring slowdown signal: 30 of 40 monitored listings no longer visible | |
| PRICING | plans.price_changed |
PRICE_INCREASE / PRICE_DECREASE (old/new values, payload.pct) |
Pro plan price observed at $59 (was $49) |
plans.added |
NEW_PRICING_TIER (tag enterprise when contact-sales) |
New pricing tier listed: Enterprise (contact sales) | |
plans.removed |
PRICING_TIER_REMOVED |
Pricing tier no longer listed: Starter | |
| pricing surface, no typed delta | PRICING_CHANGE (text-diff) |
Pricing page materially updated (3 blocks changed) | |
| LEADERSHIP | people.added (executive) |
NEW_EXECUTIVE |
Jane Doe listed as Chief Financial Officer on leadership page |
people.removed (executive) |
EXECUTIVE_NO_LONGER_LISTED |
Jane Doe no longer listed on leadership page | |
people.title_changed |
EXECUTIVE_TITLE_CHANGE |
Ann Lee now listed as COO (was VP Operations) | |
| ≥ 3 people changes, or non-executive changes only | LEADERSHIP_CHANGE aggregate |
Leadership page updated: 2 added, 1 no longer listed | |
| PRODUCT | products.added / removed |
NEW_PRODUCT / PRODUCT_REMOVED |
New product listed: Atlas Pro · Product no longer listed: Atlas Lite |
| LOCATION | locations.added |
NEW_LOCATION (kind-aware label) |
New office listed: Toronto, CA |
locations.new_countries |
COUNTRY_EXPANSION |
New country presence listed: Japan (Tokyo) | |
locations.removed |
OFFICE_REMOVED |
Office no longer listed: Berlin, DE | |
| COMMUNICATION / IR / DEVELOPER | news.added (≤ 20 per change) |
NEWS_RELEASE · BLOG_POST · CHANGELOG_ENTRY · INVESTOR_UPDATE · EARNINGS_RELEASE (title heuristics: earnings/quarter/fiscal → earnings; investor/annual report/dividend → IR; category/surface otherwise) |
News release: </em></td> </tr> <tr> <td>DEVELOPER (text-diff)</td> <td>docs/developer · api · changelog</td> <td><code>DOC_CHANGE</code> · <code>API_CHANGE</code> · <code>CHANGELOG_ENTRY</code></td> <td><em>Documentation updated (5 sections changed)</em></td> </tr> <tr> <td>LEGAL (text-diff)</td> <td>legal_terms · legal_privacy · security</td> <td><code>TERMS_CHANGE</code> · <code>PRIVACY_POLICY_CHANGE</code> · <code>SECURITY_UPDATE</code> (+ review <code>legal_sensitive</code>)</td> <td><em>Terms of service page materially updated (3 sections changed)</em> — sections from block <code>path</code></td> </tr> <tr> <td>WEBSITE</td> <td>homepage meaningful · major+ · <code>meta.title_changed</code></td> <td><code>WEBSITE_CHANGE</code> · <code>HOMEPAGE_REDESIGN</code> · <code>MESSAGING_CHANGE</code></td> <td><em>Homepage materially redesigned (9 blocks changed, 62% of text)</em></td> </tr> <tr> <td>other surfaces (text-diff)</td> <td>products/services/solutions · investor_relations · sustainability · status · careers · locations · about</td> <td><code>PRODUCT_UPDATE</code> · <code>INVESTOR_UPDATE</code> · <code>SUSTAINABILITY_UPDATE</code> · <code>OPERATIONS_UPDATE</code> · <code>WEBSITE_CHANGE</code></td> <td><em><Surface> updated (n sections changed)</em></td> </tr> </tbody> </table> <p>Surfaces with no rule (<code>other</code>, <code>partners</code>, <code>customers</code>, <code>support</code>, …) and meaningful+ significance produce no deterministic event; the change is queued for the LLM classifier instead (<code>llm_jobs.kind = classify_change</code>), together with ambiguous surfaces (homepage/about/products text-only).</p> <h2 id="wording-policy-spec-167168" tabindex="-1"><a class="heading-anchor" href="#wording-policy-spec-167168" aria-hidden="true"><span class="anchor-icon" aria-hidden="true">#</span></a> Wording policy (spec §167–168)</h2> <ul> <li>Verbs: <strong>detected, observed, listed, now listed, no longer listed, no longer visible, appears, signal</strong>.</li> <li>Never: <em>fired, laid off, layoffs, shut down, bankrupt, collapsed</em> (<code>taxonomy.FORBIDDEN_WORDING</code>). <code>safe_wording()</code> rewrites them defensively; the LLM schemas reject them at validation time.</li> <li>Disappearance ≠ departure: <code>EXECUTIVE_NO_LONGER_LISTED</code> summaries say so explicitly; <code>HIRING_FREEZE_SIGNAL</code> / <code>hiring_freeze</code> are labelled <em>signal</em>.</li> <li>Titles ≤ 200 chars, summaries ≤ 600; entity lists bounded to 50; per-entity events capped (5 jobs, 10 people, 20 news items).</li> </ul> <h2 id="importance--confidence" tabindex="-1"><a class="heading-anchor" href="#importance--confidence" aria-hidden="true"><span class="anchor-icon" aria-hidden="true">#</span></a> Importance & confidence</h2> <ul> <li><code>importance = default(subtype) × (0.7 + 0.6·significance) × (1 + 0.3·magnitude)</code> clamped to [0.05, 1]. <code>magnitude</code> is rule-specific and 0–1: relative job delta (<code>n / max(open_before, 5)</code>), |price pct| / 50, headquarters vs office, country expansion 0.6, launch-tagged news 0.5, text ratio × 2 for page changes.</li> <li><code>confidence</code> = evidence quality (<code>taxonomy.EVIDENCE_CONFIDENCE</code>): structured ATS JSON <strong>0.95</strong> (jobs_board surface, Greenhouse/Lever/Ashby/ SmartRecruiters/Workday/JSON connectors), JSON-LD / feed <strong>0.9</strong>, HTML extraction <strong>0.8</strong>, text-diff-only <strong>0.7</strong>. <code>confidence_label</code> via <code>taxonomy.confidence_label</code> (VERIFIED ≥ 0.95, HIGH_CONFIDENCE ≥ 0.85, LIKELY ≥ 0.7, INFERRED ≥ 0.5, else LOW_CONFIDENCE).</li> <li>Corroboration: a second surface in the same cluster adds <strong>+0.03 per extra surface</strong>, capped at 0.99 (clustering below).</li> <li>Review queue: <code>major_event</code> for critical changes, <code>legal_sensitive</code> for legal inferences, <code>low_confidence</code> (< 0.5), <code>unexpected_activity</code> for freeze signals.</li> </ul> <h2 id="idempotency--clustering" tabindex="-1"><a class="heading-anchor" href="#idempotency--clustering" aria-hidden="true"><span class="anchor-icon" aria-hidden="true">#</span></a> Idempotency & clustering</h2> <ul> <li><code>events.dedupe_key = sha(company, subtype, normalised entity key, sensor, detection day)</code> → inserting the same change twice is a no-op (<code>on conflict do nothing</code>). Entity keys: job label, plan name, person name, product name, place, news title, <code>jobs:<before>><after></code> for aggregates, <code>page:<sensor></code> for page-level events.</li> <li><code>event_clusters.cluster_key = sha(company, subtype, normalised entity key, 7-day bucket)</code> — surface-independent, so the same press release seen on the newsroom <strong>and</strong> the feed, or the same executive on leadership <strong>and</strong> about, fold into one cluster. The first event is canonical; later ones are stored with <code>status='duplicate'</code> + <code>cluster_id</code>, the canonical event gains an <code>event_sources</code> row (<code>kind='corroboration'</code>), <code>payload.sources[]</code>, <code>payload.corroborations</code> and the confidence bump. Clusters track <code>source_count</code>, <code>surfaces[]</code>, <code>confidence</code>.</li> </ul> <h2 id="llm-enrichment-hooks" tabindex="-1"><a class="heading-anchor" href="#llm-enrichment-hooks" aria-hidden="true"><span class="anchor-icon" aria-hidden="true">#</span></a> LLM enrichment hooks</h2> <ul> <li><code>classify_change</code> (small model) when no deterministic event or ambiguous surface, significance ≥ <code>CA_LLM_MIN_SIGNIFICANCE</code>, within <code>CA_LLM_DAILY_BUDGET</code>, only when <code>settings.llm_configured</code>. Material classifications create <code>origin='llm'</code> events (dedupe <code>sha(company, 'llm', change, subtype)</code>); when a deterministic event of the same subtype already exists for the change, the classification enriches it instead (<code>origin='hybrid'</code>, <code>payload.llm_classification</code>).</li> <li><code>summarize_event</code> (medium model) for legal, homepage/messaging and news/IR events that carry diff content → <code>events.summary</code>, <code>origin='hybrid'</code>, <code>payload.llm</code> (key points or legal sections + materiality), model/prompt/schema versions recorded. See docs/LLM.md.</li> </ul> |