spb/firc0001_uqo

Thu, Nov 5, 2026 1

1. N8: rule-based baseline (snapshot rules.py) vs LLM on a stratified seeded sample of 60,000 ledger sections; rules-on-full vs rules-on-focused recall-loss bound

   …

   Items: FIRC0001-AI_C-19, THEO-09.
   30,000 productive + 30,000 zero-mention sections (inverse-probability weights; DuckDB USING SAMPLE applied in subqueries), 200 s, 4 workers.
   P(rules >= 1 | LLM >= 1) = 85.7% (10-K 97.4, 10-Q 94.4, 8-K 69.2); weighted P(LLM >= 1 | rules >= 1) = 32.5%; type overlap on co-detected sections 84.6% (M&A 96.8%);
   rules on _focus(text) recover 26.2% of the rules-on-full-text mentions in over-budget sections (10-K MD&A 20.8%, 8-K 84.2%).
   
   Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

   Simon-Pierre Boucher yesterday · 1 file +2
   b7e79df