spb/firc0001_uqo

1# Validation report — FIRC-0001, exhibit layer (protocol Part IX; Stage D laptop batches L1/L6)23Prepared 2026-11-05 by the laptop executor on the review worktree `worktrees/FIRC-0001` (branch `review-20261103`). The node-side report (DuckDB audits, regeneration of the 29 exports by N13) is `UQO-WP-REVIEW/reproducibility/FIRC-0001/validation_report_N.md`; this report covers what was re-run on the laptop and the resulting verdict for the exhibit layer.45## What was re-run, where, how long67| step | command | machine | wall time | log |8|---|---|---|---|---|9| exhibits | `OMP_NUM_THREADS=4 nice -n 10 python3 -I code/make_tables.py` | laptop (Python 3.14.4, matplotlib 3.10.9) | ≈ 3 s | `reproducibility/FIRC-0001/logs/L1_make_tables_*.log`, `L1_build_*.log` |10| paper | `latexmk -pdf -interaction=nonstopmode main.tex` (pdflatex + bibtex, `aer.bst`) | laptop (TeX Live 2026) | ≈ 40 s | `L1_build_20261105-142858.log` |11| tests | `python3 -I -m pytest -q tests` | laptop (pytest 9.1.1) | ≈ 1 s | `L6_pytest_*.log` |12| DOI resolution | `curl` × 24 against Crossref / DataCite, 0.3 s apart | laptop | ≈ 20 s | `L4_crossref_20261105-141223.json` |1314No LLM or embedding call was made; nothing was written on any node; `data/stats.json` and the 29 committed CSV exports were not modified by any laptop step (the generator no longer rewrites `stats.json`; `sha256(data/stats.json)` is asserted unchanged by `tests/test_tables.py::test_regeneration_reproduces_committed_tables`).1516## What matched1718- **Export reconciliation (21 assertions, all pass):** the marginal CSV tables sum to the headline counts of `stats.json` — ledger rows 1,123,098; stored mentions 308,065 (by form, by year, by confidence); projects 121,099 (by type, status, span, mention bucket, first-seen year, sector, index membership); filings 527,548 (by form, by year × form, by year, by parse status); sections 1,342,283; universe 2,564 = 491 + 9 + 2,064 = 2,499 + 65 = Σ sectors; 10-K and 10-Q item tables sum to the per-form ledger rows and mentions; amount-bearing projects 56,500 in `amount_dist.csv`, `projects_by_type.csv` and `stats.json`.19- **Regeneration:** `make_tables.py --out <scratch>` reproduces every committed `paper/tables/*.tex` byte for byte and the committed `results/inline_numbers.json` (207 macros). Figures are regenerated as PDF (vector; `CreationDate` suppressed) — compared visually, not byte-wise.20- **Build:** `latexmk` exit 0, 0 LaTeX errors, 0 undefined citations/references, bibtex 0 warnings, 39 pages. Overfull boxes: 17 (baseline 14); the three new ones are the wider Tables 3 and 5 whose notes still sit inside the `adjustbox` (item LITE-10, Stage E/L3 moves the notes outside), and the 835 pt box is the hand-typed regex of `appendix.tex:19` that Stage E replaces by `\input{tables/gate_regex}`.21- **Prompt and gate regex:** `paper/tables/prompt_verbatim.tex` and `gate_regex.tex` are now generated from the frozen snapshot (`code/audit/spid_snapshot/llm.py`, `taxonomy.py`, `rules.py`, SHA-256 in `SHA256SUMS`) by parsing the source with `ast`; the printed regex concatenates to the `_PROJECT_GATE` pattern of `rules.py` exactly (three source lines, 26 alternatives), and the prompt lists the 21 taxonomy labels in the order `TYPE_LABELS` sends them.22- **Literals removed from the generators:** `make_tables.py` — `2{,}587` (now `data/documented_inputs.json:iwv_equity_lines_reported`, labelled "count reported by the build log, source file not archived"), `(500 CIKs)` (now `stats.json:universe.n_sp500` / `sp500_ciks`), `2010-01-04`/`2026-10-27` (now `stats.json:filing_date_range` or min/max of `filings_by_form.csv`), `128.5` (now `stats.json:amount_total_median_musd` or `amount_dist.csv`), `2700` (y-limit now `universe.n × 1.05`), `300` (label offset now 0.7 % of the maximum), `170` (now `documented_inputs.json:snippet_chars`); `export_stats.py` — the `spid_runs` strings (`2026-06 (gpt-4o-mini, 16 workers, 153,894 sections)`, `… 48 workers, 968,134 sections`) are now read by regex from the two run logs and from the ledger (`None` when absent). `tests/test_tables.py::test_make_tables_has_no_data_literals` tokenises the generator and fails on any integer literal of three or more digits other than the scale `100` (`1e3/1e6/1e9` are exponent forms).2324## What did not match / is still open2526- `export_stats.py` was **edited but not run** on the laptop (it needs the two DuckDB files on M3U96b); the new keys it exports (`filing_date_range`, `amount_total_median_musd`, `amount_max`, `amount_ge_50b`, `amount_le_zero`, `ledger_10k_mdna_median_words`, `annual_panel_firms`, `sp500_ciks`, `gate_pass_total`, `calls_by_form`, `backfilled_rows`, `outage`, `conf_exact_default_count`, `amount_raw_deviations`, `fiscal_year_null_mentions`, structured `spid_runs`) and its three new dumps (`ledger_collisions.csv`, `cohort_by_run.csv`, `calls_by_form.csv`, plus the seeded bootstrap `projects_by_type_ci.csv`) appear in the exhibits only once N13 has re-run it and its 29-file diff against the committed exports is empty. Until then `make_tables.py` falls back to the committed exports and to the N1/N3 result files (`results/ledger/*.json`, `results/resolver/*`), and the "95 % CI of the median" column of Table 5 prints `--`.27- The figure PNGs under `paper/figures/` are produced by no script (DATA-15); they are kept for the Word export only.28- The release of the derived DuckDB tables (AD-1) and the licence of code and tables (AD-3) are author decisions; no `LICENSE` file is committed.2930## Reproducibility verdict3132- **Exhibit layer (tables, figures, inline numbers from `data/` and `results/`): R1** — reproduced from the documented analysis data; the raw filings (254.5 GB) and the two DuckDB files are too large to ship and the LLM extraction is not re-run.33- **Export layer (`data/*.csv`, `stats.json` from the DuckDB files): provisional R2** until N13 reports its diff; R1 if the 29 committed files are regenerated byte-identically on M3U96b, R4 if any headline count moves.34- **Extraction layer (LLM calls): not reproducible by design** — re-runnable and ledgered, with the determinism statement of `docs/DATA_AVAILABILITY.md`; stability under re-submission is unmeasured (AR-LLM-1 not granted).35