SPB Git
spboucher.ai

spb/firc0001_uqo

Public

Stage E (writing): prose wired to the self-contained generated floats (one \input per exhibit at its anchor; duplicates removed), widenotes/listings/makecell preamble, evidence register as longtable, audit-exhibit pointer appendix; docs: research_log review entries, data_dictionary/project_ontology/universe_definition/implementation_audit corrected to the audited facts (amount_raw formatted float, ledger semantics, 7 unmatched, coverage 85.3 %, WP No. 0001), README v1.1

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 5, 2026) parent 0d1f761 Browse files

23 changed files +152 −310

23 changed files
Unified Split
modified README.md +1 −1
@@ -1,6 +1,6 @@
1 1 # UQO Working Paper No. 0001 (FIRC-funded) — From Corporate Disclosures to a Russell 3000 Strategic Project Database
2 2
3 −*Identification, Measurement, Validation, and Research Applications* — Manel Kammoun, Djerrycharli Tandjambianda, Simon-Pierre Boucher (order to be confirmed; see `docs/contributions.md`). Version 1.0, October 28, 2026 — **working draft, research database under validation**; revision branch `review-20261103` (Stage D, November 2026). Research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO).
3 +*Identification, Measurement, Validation, and Research Applications* — Manel Kammoun, Djerrycharli Tandjambianda, Simon-Pierre Boucher (order to be confirmed; see `docs/contributions.md`). Version 1.1, November 2026 — **revised after peer review; working draft, research database under validation** (v1.0 of October 28, 2026 is the reviewed baseline, tag `review-baseline-20261103`); revision branch `review-20261103` (Stages D–E, November 2026). Research funded by the FIRC (Fonds institutionnel de recherche et de création, UQO).
4 4
5 5 > **TL;DR** — SPID turns 527,548 SEC filings (10-K/10-Q/8-K, 2010–Oct 2026) of 2,564 issuers (S&P 500 ∪ iShares IWV Russell 3000 proxy, 85.3 % of the index's 2,996 constituents) into 308,065 section-linked candidate project mentions and 121,099 candidate projects under the current resolution key (21-category taxonomy, status, years, locations, technologies, partners, amounts). The paper documents the universe, the distributed acquisition/parsing architecture, the LLM extraction contract (two runs under one prompt lineage), the resolution rule and its failure modes, descriptive patterns, and a validation protocol — stating explicitly which gates are met and which are not.
6 6
modified docs/data_dictionary.md +3 −3
@@ -13,13 +13,13 @@
13 13 ## Project layer — `project-db/data/db/spid.duckdb` (3.2 GB)
14 14 | Table | Key | Fields |
15 15 |---|---|---|
16 −| `extracted_sections` | `section_id` | accession_number, backend, n_mentions, extracted_at (ledger of every section submitted) |
16 +| `extracted_sections` | `section_id` | accession_number, backend, n_mentions, extracted_at — ledger of every section with ≥ 30 words (1,123,098 rows), **including** the 419,361 sections rejected by the lexical gate and never sent to the model (703,725 gate-passing calls); `backend` is the constant `llm` (no model id, prompt/taxonomy hash or call status); the 154,964 June rows were backfilled at one timestamp (2026-10-27 13:12:32) without the 30-word filter; a failed call is ledgered as `n_mentions = 0` |
17 17 | `spid_sections` | `section_id` | legacy 10-K items ingested in June 2026 (27,164 rows) |
18 −| `project_mentions` | `mention_id` | accession_number, cik, section_id, section_name, project_name, project_type, description, objective, amount_usd, amount_raw, locations[], technologies[], partners[], suppliers[], benefits[], risks[], years[], status, confidence, backend, ticker, company_name, form_type, filing_date, fiscal_year, sector |
18 +| `project_mentions` | `mention_id` (SHA-1 of accession, section_id, project_type and the first 40 characters of project_name; 412 intra-section collisions overwritten by INSERT OR REPLACE) | accession_number, cik, section_id, section_name, project_name, project_type, description, objective, amount_usd, amount_raw (**formatted copy of `amount_usd`, e.g. `$1,500,000,000`, not the source string; NULL when the amount is 0**), locations[], technologies[], partners[], suppliers[], benefits[], risks[], years[], status, confidence, backend, ticker, company_name, form_type, filing_date, fiscal_year, sector |
19 19 | `projects` | `project_id` | cik, ticker, company_name, sector, project_type, project_name, canonical_location, technologies[], status, total_amount_usd (max), n_mentions, n_filings, first_seen, last_seen, first_year, last_year, avg_confidence, description |
20 20 | `project_timeline` | (project_id, accession_number) | filing_date, form_type, status, amount_usd, confidence, snippet |
21 21 | `kg_nodes` / `kg_edges` | node_id / (src, rel, dst) | node_type ∈ {Company, Project, Location, Technology, Partner}; rel ∈ {owns, located_in, uses} |
22 22 | `project_embeddings` | `project_id` | model, dim, vector FLOAT[384] |
23 23 | views `src_filings`, `src_sections`, `src_companies`, `v_sections` | — | read-only views over the corpus Parquet (paths of the master node) |
24 24
25 −Null conventions: unknown amounts/locations are NULL / empty lists; `fiscal_year` is NULL for mentions of the June 2026 run.
25 +Null conventions: unknown amounts/locations are NULL / empty lists; `fiscal_year` is NULL for 68,827 mentions (all 39,930 June-run mentions and 28,897 October mentions whose filing carries no fiscal year); `years[]` and `technologies[]` may have been back-filled by regex from `description` when the model returned none (upper bounds 39 % / 46 %); a missing `confidence` is stored as 0.6 (3,278 mentions). Stage E, 2026-11-05: corrected after the Stage D audits (`results/ledger/`, `results/grounding/`).
modified docs/implementation_audit.md +6 −6
@@ -1,6 +1,6 @@
1 −# Implementation audit — UQO WP 28 (checked-out revision 2324b58, 2026-10-28)
1 +# Implementation audit — UQO Working Paper No. 0001 (checked-out revision 2324b58, 2026-10-28; annotated 2026-11-05 after peer review)
2 2
3 −Repository: `gitsrv:srv/git/uqo-research-db.git` (private UQO Git server; the public alias cited in the brief, `git.spboucher.ai/uqo-research-db.git`, points to the same server). The June 2026 methodological study examined `df7905d`; the paper's numbers come from the build at `2324b58`.
3 +Repository: `gitsrv:srv/git/uqo-research-db.git` (private UQO Git server; the public alias cited in the brief, `git.spboucher.ai/uqo-research-db.git`, points to the same server). The June 2026 methodological study cited `df7905d` as its revision, but that commit is dated 2026-10-27 and the repository's six commits are all of 2026-10-27/28: no June revision exists (file-system evidence only, `results/runs/git_evidence.md`). The paper's numbers come from the build at `2324b58`.
4 4
5 5 ## Inspected (paths as checked out)
6 6 - `src/acquisition/{universe,bulk_downloader,cik_mapper,sec_client}.py`, `scripts/{build_universe,download_filings,build_database,parse_filings,build_embeddings,build_factors,export_shard,merge_shards}.py`, `scripts/cluster/{orchestrate.py,rebalance.py,run_shard.sh,spid_run.sh}`
@@ -13,13 +13,13 @@ Repository: `gitsrv:srv/git/uqo-research-db.git` (private UQO Git server; the pu
13 13 |---|---|---|---|
14 14 | 1 | No human-labelled gold standard; precision/recall unknown | no `human_labels` table, no annotation files | High |
15 15 | 2 | Resolution key merges distinct phases / splits renamed projects; no link scores | `resolve.py` (MD5 of cik|type|loc/token); Cheniere example | High |
16 −| 3 | `extracted_sections` cannot distinguish "no project" from "API failure" | 149,352 zero-mention rows written during the credit outage, removed by timestamp | High |
17 −| 4 | Amounts untyped; project amount = max over mentions | `resolve.py`; `amount_raw` kept | High |
16 +| 3 | `extracted_sections` cannot distinguish "no project" from "API failure" (nor from a gate rejection: 419,361 rows never sent) | 149,352 zero-mention rows written during the credit outage, removed by timestamp (reconciled with the log, N1); 14 silent JSON-decode failures outside the outage; first post-outage batch (5,198 sections, 23 productive vs 536 expected) suspect — re-submission AR-LLM-4 not granted | High |
17 +| 4 | Amounts untyped; project amount = max over mentions; `amount_raw` is a formatted copy of the coerced float, not the source string | `resolve.py`; `llm.py:155`; 584 zero amounts with NULL `amount_raw`, 24 negative, 161 mentions ≥ 50 B USD (N1/N2) | High |
18 18 | 5 | No `project_events`; timeline = mention rows; cited years unordered | `project_timeline` schema | Medium |
19 19 | 6 | No prompt / taxonomy / schema hash per run | `extracted_sections.backend` = "llm" only | Medium |
20 20 | 7 | Evidence = model paraphrase; no character span of the supporting passage | `project_mentions.description` | Medium |
21 −| 8 | Focus windows truncate long sections (budget ≈ 9–12.6k chars) — recall on 10-K MD&A unknown | `llm.py::_focus` | Medium |
22 −| 9 | Universe: no file SHA-256, no match confidence, no membership dates, no security-vs-issuer table | `universe.json` | Medium |
21 +| 8 | Focus windows truncate long sections (budget ≈ 9–12.6k chars) — measured 2026-11-05: median 9.3 % of a 10-K MD&A shown, 62.7 % of mentions from sections shown < 25 %, rules-based recall bound 26.2 % | `llm.py::_focus`; `results/focus/`, `results/rules/` (N5, N8) | Medium |
22 +| 9 | Universe: derived artefacts now hashed (N10) but source holdings/ticker files not archived; no match confidence, no membership dates, no security-vs-issuer table | `universe.json`, `data/metadata/SHA256SUMS` | Medium |
23 23 | 10 | Form scope excludes 8-K exhibits (press releases) and 20-F/40-F (65 issuers) | `download_filings.py` | Medium |
24 24 | 11 | Sections lack offsets into raw HTML (chunks have offsets within sections) | `filing_sections` schema | Low |
25 25 | 12 | `pq.write_to_dataset(overwrite_or_ignore)` leaves stale partitions → duplicates in `filing_scores` until cleaned (done 2026-10-28) | `aggregation.py` | Low |
modified docs/project_ontology.md +5 −2
@@ -19,10 +19,13 @@ Proposed change: add `mention_label` (five values) and `evidence_span` (characte
19 19 21 categories, unchanged since June 2026 (`project-db/spid/taxonomy.py`): manufacturing_expansion, plant_construction, data_center, cloud_migration, ai_initiative, digital_transformation, erp_implementation, rd_program, product_launch, drug_pipeline, ma_integration, geographic_expansion, capex_program, supply_chain, automation, sustainability, energy_transition, infrastructure, cost_reduction, partnership, technology_deployment. Any extension must be versioned with a crosswalk (old code → new code).
20 20
21 21 ## Status vocabulary
22 −`planned | in_progress | completed | mentioned` (per mention; project status = latest mention by filing date). Proposed: dated `project_events` (announcement, filing/acceptance, expected start, observed start, expected completion, realized completion, postponement, cancellation).
22 +`planned | in_progress | completed | mentioned` (per mention; an unrecognised status is mapped to `mentioned` and an out-of-vocabulary type to `technology_deployment`, neither logged; project status = latest mention by filing date, ties on the latest date resolved in execution order for 972 projects). Proposed: dated `project_events` (announcement, filing/acceptance, expected start, observed start, expected completion, realized completion, postponement, cancellation).
23 23
24 24 ## Amounts
25 −`amount_usd` per mention (coerced float, raw string kept); project `total_amount_usd` = **max** over mentions. Never sum across projects; never read as realized expenditure. Proposed `project_amounts` with `amount_type` ∈ {announced_budget, estimated_cost, capex_incurred, transaction_value, contract_value, financing, other}, currency, scale, basis, estimate/realized flag.
25 +`amount_usd` per mention (coerced float; **`amount_raw` is a formatted copy of that float (`llm.py:155`), the source string, currency and scale are not stored** — corrected 2026-11-05, FIRC0001-EDIT-02/ADVE-03/DATA-03); project `total_amount_usd` = **max** over mentions. Never sum across projects; never read as realized expenditure. Proposed `project_amounts` with `amount_type` ∈ {announced_budget, estimated_cost, capex_incurred, transaction_value, contract_value, financing, other}, currency, scale, basis, estimate/realized flag.
26 +
27 +## Financing instruments (label added 2026-11-05, FIRC0001-EDIT-12 / AI_C-13 / THEO-08)
28 +Share-repurchase programs, dividends, credit facilities, term loans, note offerings, at-the-market programs and commercial paper are corporate actions, not projects; the taxonomy has no category for them and the gate word `program` admits them. Regex `repurchase|buy-?back|dividend|credit facility|revolving|term loan|notes? (due|offering)|at-the-market|senior notes|commercial paper` on `project_name || ' ' || description` matches 22,412 mentions (7.28 %; 54.5 % of 10-Q `unregistered_sales` mentions, 19.2 % of 8-K mentions; 52.6 % of `capex_program`, 16.4 % of `cost_reduction`, 7.0 % of `ma_integration`); `results/fp_classes/`. Proposed label `financing_instrument` in the six-way mention label (project, generic strategy, third-party, financing instrument, uncertain, none).
26 29
27 30 ## Known failure modes of the resolution key
28 31 False merges (same type + same location/token: multi-phase sites, serial acquirers in one city, drug programs sharing a token); false splits (renamed initiatives, location granularity, category change); no cross-issuer identity (JVs counted twice; inherited projects restart).
modified docs/research_log.md +7 −0
@@ -14,3 +14,10 @@
14 14 | open | Linkage audit; probabilistic linker | to assign | pending |
15 15 | open | Amount typing pass; 8-K exhibit ingestion; recall test on long MD&A | to assign | pending |
16 16 | open | Licensed Russell constituent history? | to assign | pending |
17 +| 2026-11-03 | Peer review of v1.0 opened (UQO-WP-REVIEW, branch `review-20261103`, baseline tag `review-baseline-20261103` = `678d233`); seven blinded referee reports, 125 items | review committee | done |
18 +| 2026-11-05 | Chief Editor consolidation: major revision; integrity PASS; clusters A–N; constraint: no new LLM call, DuckDB read-only | review committee | done |
19 +| 2026-11-05 | Stage D node audits on M3U96b (read-only, seed 20261105): ledger/calls reconciliation (N1), grounding of all 308,065 mentions (N2), resolver composition and specification curve (N3), two runs / two segmenters (N4), focus-window coverage (N5), financing false-positive classes (N6), 8-K item-code consistency (N7), rules-vs-LLM baseline (N8), time series with fixed cohort (N9), universe artefacts with SHA-256 (N10), annotation frame and protocol — no labels (N11), environment lock (N12), export regeneration (N13) | node executor | done |
20 +| 2026-11-05 | Stage D laptop batches: generators de-literalised and inline-number macros (L1), 24 DOI-verified references and `bloom2021` → `kalyani2025` (L4), Russell 3000 coverage 85.3 % (L5), replication package with tests, `run_all.sh`, DAS draft (L6) | laptop executor | done |
21 +| 2026-11-05 | Stage E: manuscript rewritten to the audited facts (v1.1): abstract, §1–§10 and appendices; `amount_raw`, ledger/calls, "schema-constrained", "not mechanically checkable", "reproducible end to end", "single extraction contract" corrected; project count reported as a range; cohort/run confound; gates (a) partial, (d) and (g) not met; exhibits rebuilt from `results/` | Stage E agents | done |
22 +| 2026-11-05 | **Decisions pending (authors):** AD-1 release of the derived tables; AD-2 human annotation of the frame (n = 1,677 + 385); AD-3 licence (MIT / CC BY 4.0 proposed); AR-LLM-1..3 stability / re-submission / amount-typing calls; AR-LLM-4 re-submission of the suspect post-outage batch (≈ 2,686 calls); deterministic tie-breaker in `resolve.py`; `extraction_runs` / `call_status` schema; confirm the name of the research-design specification ("the brief") | all | **pending** |
23 +
modified docs/universe_definition.md +6 −5
@@ -1,4 +1,4 @@
1 −# Universe definition (frozen for UQO WP 28, build of 2026-10-27/28)
1 +# Universe definition (frozen for UQO Working Paper No. 0001, build of 2026-10-27/28; reconciled 2026-11-05 after peer review)
2 2
3 3 **Estimand.** Current constituents at a snapshot date, approximated by fund holdings. Not historical point-in-time Russell 3000 membership (no licensed constituent history was available).
4 4
@@ -7,10 +7,11 @@
7 7 | Primary source | iShares Russell 3000 ETF (IWV) `latest-holdings.csv`, "Fund Holdings as of Oct 23, 2026", fetched 2026-10-27 (`data/metadata/russell3000_constituents.json`: `source`, `holdings_as_of`, `fetched_at`) |
8 8 | Secondary source | S&P 500 constituent table (Wikipedia), cached 2026-05-25 (`sp500_constituents.json`, 503 rows / 500 CIKs) |
9 9 | Identifier mapping | ticker → CIK via SEC `company_tickers.json` (fetched 2026-10-27); tickers normalised (`BRK B` → `BRK-B`) |
10 −| Exclusions | asset class ≠ Equity; CVR / escrow / rights lines; tickers with no SEC CIK (FDIC-filing banks). The 21 unmatched lines are listed in the `unmatched` field of the artifact. |
11 −| Result | 2,587 equity lines → 2,555 CIKs (share classes collapsed) → merged with S&P 500 → **2,564 issuers** (491 in both, 9 S&P-only, 2,064 Russell-only) |
10 +| Exclusions | asset class ≠ Equity; CVR / escrow / rights lines; tickers with no SEC CIK (FDIC-filing banks). **7** tickers reached the mapping step without a CIK and are listed in the `unmatched` field of the artifact: TOWN, FRBA, NBN, HIFS (FDIC-filing banks) and ADRO, THRD, HOLX (absent from `company_tickers.json` on 2026-10-27; reason not recorded). The holdings file and the ticker file were **not archived**, so the lines dropped before mapping (non-Equity, CVR / escrow / rights; 2,587 − 2,573 = 14 by subtraction from the build-log count) cannot be listed. Earlier versions of this document said 21; the only verifiable count is 7 (`stats.json:universe.russell_unmatched`). |
11 +| Result | 2,587 equity lines (build-log count) → 2,573 mapped to a CIK → 2,555 CIKs (18 dual-class registrants collapsed) → merged with S&P 500 → **2,564 issuers** (491 in both, 9 S&P-only, 2,064 Russell-only) |
12 +| Index coverage | Russell 3000 factsheet of 2026-09-30: 2,996 constituents → 2,555 / 2,996 = **85.3 %**; IWV uses representative sampling (Summary Prospectus, Prospectus dated 2026-07-31), so omitted members cannot be identified (`docs/universe_coverage.md`, `results/russell_coverage.json`) |
12 13 | Filing scope | Forms 10-K, 10-Q, 8-K (primary document only, no amendments, no exhibits), filed 2010-01-01 → 2026-10-27 |
13 14 | Observed | 2,499 issuers with ≥1 filing; 65 without (foreign private issuers filing 20-F/40-F; FDIC banks) — `data/issuers_without_filings.csv` |
14 −| Frozen artifact | `data/metadata/universe.json` (`built_at` 2026-10-27), copied to the paper folder under `data/` |
15 +| Frozen artifact | `data/metadata/universe.json` (`built_at` 2026-10-27; SHA-256 `3967b29b…`), `russell3000_constituents.json` (`758cbebf…`), `sp500_constituents.json` (`50840752…`), copied to the paper folder under `data/metadata/` with `SHA256SUMS` (N10) |
15 16
16 −**Open items (not implemented):** SHA-256 of the source files; per-row match confidence; membership dates; security-vs-issuer table (`universe_snapshots`); resolution of ticker changes and spin-offs over the sample period; a licensed historical constituent list.
17 +**Open items (not implemented):** SHA-256 of the *source* files (holdings CSV, `company_tickers.json` — not archived, not recoverable); per-row match confidence; membership dates; security-vs-issuer table (`universe_snapshots`); resolution of ticker changes and spin-offs over the sample period; a licensed historical constituent list.
modified paper/appendix/appendix.tex +43 −119
@@ -1,46 +1,49 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \begin{appendices}
3 5 % ============================================================================
4 6
5 7 \section{Taxonomy}\label{app:taxonomy}
6 8
7 −\begin{table}[H]
8 −\centering\small
9 −\caption{The \SPIDtaxonomyN-category SPID project taxonomy}\label{tab:taxonomy}
10 −\begin{threeparttable}
11 9 \input{tables/tab_taxonomy}
12 −\begin{tablenotes}\footnotesize
13 −\item Codes and labels are read with \texttt{ast} from \texttt{spid/taxonomy.py} in the hash-verified snapshot of revision \texttt{2324b58} (\texttt{code/audit/spid\_snapshot/}); the module's modification time predates the June 2026 run and no earlier revision exists, so the taxonomy is the same for both runs as far as the file system can show (\Cref{sec:versioning}). The taxonomy has no category for financing instruments (\Cref{sec:construct_def}); an out-of-vocabulary label returned by the model is mapped by substring matching and, failing that, to \texttt{technology\_deployment}.
14 −\end{tablenotes}
15 −\end{threeparttable}
16 −\end{table}
17 10
18 11 \section{Extraction contract}\label{app:prompt}
19 12
20 13 The system prompt below is reproduced from \texttt{spid/extract/llm.py} with the \SPIDtaxonomyN\ taxonomy codes and labels substituted exactly as the code does (generated by \texttt{code/make\_tables.py}; never retyped). The user message is the string ``\texttt{SECTION: <section\_name>}'' followed by the focused section text (\Cref{sec:focus}), cut at \SPIDuserMessageCap\ characters. Decoding parameters: temperature 0, JSON-object response format (valid JSON is guaranteed; the field list is not enforced by a schema), \SPIDcompletionTokens\ completion tokens, \SPIDretries\ attempts before an empty list is returned, \SPIDtimeoutSeconds-second timeout, no \texttt{seed}. The model identifier is not recorded in the ledger or the logs (\Cref{sec:versioning}). Straight quotes are rendered as sent.
21 14
22 −{\footnotesize\input{tables/prompt_verbatim}}
15 +\input{tables/prompt_verbatim}
23 16
24 17 \noindent The candidate gate is the case-insensitive regular expression below, reproduced from \texttt{spid/extract/rules.py} (\SPIDgateRegexAlternatives\ alternatives, \SPIDgateRegexLength\ characters, compiled with \texttt{\SPIDgateRegexFlags}); line breaks are inserted for display only. Sections without a match, or with fewer than \SPIDminWords\ words, are not submitted; sections with fewer than \SPIDminWords\ words are not ledgered either, except the \SPIDbackfilledUnderThirtyWords\ June rows backfilled without the filter.
25 18
26 −{\footnotesize\input{tables/gate_regex}}
19 +\input{tables/gate_regex}
27 20
28 21 \section{Data dictionary and data-model audit}\label{app:dictionary}
29 22
30 23 \Cref{tab:datamodel} lists the tables required by the research-design specification against the current implementation (generated from \texttt{data/datamodel\_audit.csv}). The full field-level dictionary of the current tables is maintained in \texttt{docs/data\_dictionary.md} of the working-paper folder and in the repository documentation; \texttt{amount\_raw} is documented there as a formatted copy of \texttt{amount\_usd}, not as a source string.
31 24
32 −{\footnotesize\input{tables/tab_datamodel}}
25 +% longtable generated by code/make_tables.py
26 +\input{tables/tab_datamodel}
33 27
34 28 \section{Evidence register}\label{app:evidence}
35 29
36 −\begin{table}[H]
37 −\centering\footnotesize
38 −\caption{Evidence register: status of the main claims of this paper at revision \texttt{2324b58} after the Stage D audits}\label{tab:evidence}
39 −\begin{threeparttable}
40 −\begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{5.9cm}@{}}
30 +{\footnotesize
31 +\begin{longtable}{@{}>{\raggedright\arraybackslash}p{9.5cm}>{\raggedright\arraybackslash}p{6.5cm}@{}}
32 +\caption{Evidence register: status of the main claims of this paper at revision \texttt{2324b58} after the Stage D audits}\label{tab:evidence}\\
41 33 \toprule
42 34 Claim & Status \\
43 35 \midrule
36 +\endfirsthead
37 +\multicolumn{2}{@{}l}{\footnotesize\itshape Evidence register (continued)}\\
38 +\toprule
39 +Claim & Status \\
40 +\midrule
41 +\endhead
42 +\midrule
43 +\multicolumn{2}{@{}r@{}}{\footnotesize\itshape continued on the next page}\\
44 +\endfoot
45 +\bottomrule
46 +\endlastfoot
44 47 Universe of \SPIDuniverseN\ issuers built from the IWV holdings file (\SPIDiwvHoldingsDate) and the S\&P~500 cache (\SPIDspWikipediaDate); covers \SPIDrussellProxyCoveragePct\% of the index constituent count & Implemented; derived artefacts verified and hashed; source holdings and ticker files not archived; index count read from the factsheet \\
45 48 \SPIDfilingsTotal\ primary documents of Forms 10-K/10-Q/8-K since 2010 downloaded; \SPIDfilingsNotParsed\ not parsed (\SPIDfilingsMissingOnDisk\ missing on disk) & Implemented, verified (database counts; accession list published) \\
46 49 Acquisition respected the SEC rate limit (aggregate token bucket; no 403 recorded) & Implemented; documented from run notes; orchestrator logs not archived \\
@@ -62,128 +65,49 @@ Amounts typed and separated from realized expenditure & Not implemented (propose
62 65 Dated project events and point-in-time revision flags & Not implemented (mention-level timeline only) \\
63 66 Release of the derived tables; licence & Author decisions pending \\
64 67 Author order, affiliations and CRediT contributions & Pending confirmation by the three coauthors \\
65 −\bottomrule
66 −\end{tabularx}
67 −\begin{tablenotes}\footnotesize
68 −\item ``Measured'' entries refer to the read-only audits of \texttt{code/audit/} whose outputs are in \texttt{results/} with a manifest (script hash, database sizes and modification times, row counts); none involved a model call. ``Verified'' means recomputed from the exported data or the databases; ``documented'' means stated by the code or the run notes without an independent check.
69 −\end{tablenotes}
70 −\end{threeparttable}
71 −\end{table}
68 +\end{longtable}
69 +
70 +\noindent\footnotesize Notes: ``Measured'' entries refer to the read-only audits of \texttt{code/audit/} whose outputs are in \texttt{results/} with a manifest (script hash, database sizes and modification times, row counts); none involved a model call. ``Verified'' means recomputed from the exported data or the databases; ``documented'' means stated by the code or the run notes without an independent check.\par}
71 +
72 72
73 73 \section{The two extraction runs}\label{app:runs}
74 74
75 75 \paragraph{Git and file-system evidence.} The repository holds \SPIDrepoCommits\ commits, all dated October~27--28, 2026 (\texttt{results/runs/git\_evidence.md}); \texttt{project-db/} enters at the first commit. The extractor modules carry modification times of \SPIDextractorMtimeDate\ and SHA-256 hashes recorded in \texttt{code/audit/spid\_snapshot/SHA256SUMS}; the driver files were edited on October~27, 2026 before the first commit. The diff between the revision the earlier version called ``the June revision'' (\texttt{df7905d}, dated October~27) and the build revision \texttt{2324b58} on the extractor, parser and driver files is the following change to \texttt{store.py}:
76 76
77 −{\scriptsize\verbatiminput{../results/runs/git_diff_df7905d_2324b58.patch}}
77 +\lstinputlisting{../results/runs/git_diff_df7905d_2324b58.patch}
78 78
79 79 \noindent The diff from the first commit (\texttt{0304133}) to \texttt{2324b58} on the same files is the following change to the corpus-layer \texttt{filing\_parser.py}:
80 80
81 −{\scriptsize\verbatiminput{../results/runs/git_diff_0304133_2324b58.patch}}
81 +\lstinputlisting{../results/runs/git_diff_0304133_2324b58.patch}
82 82
83 83 \paragraph{Mentions and projects by cohort, run and form.} \Cref{tab:june_october} reconciles the mentions of the S\&P~500 cohort (\texttt{in\_sp500} true) and of the Russell-only cohort by run and form; the June block is identified by the backfill timestamp. The S\&P~500 cohort's 10-Q mentions come from both runs for the same issuers, which permits a within-issuer comparison: on the \SPIDpairedIssuers\ issuers with 10-Qs in both runs, the productive share is \SPIDwithinProductiveJune\% in June and \SPIDwithinProductiveOctober\% in October (paired difference \SPIDpairedDeltaPp\ percentage points, standard error \SPIDpairedSePp), mentions per row \SPIDwithinMentionsPerRowJune\ and \SPIDwithinMentionsPerRowOctober, mean confidence \SPIDwithinConfJune\ and \SPIDwithinConfOctober, and the M\&A share \SPIDwithinMaShareJune\% and \SPIDwithinMaShareOctober\% (\texttt{results/runs/within\_sp500\_*.csv}). For 10-Qs, whose text went through the same parser in both runs, the two runs therefore behave alike; the cohort difference of \Cref{sec:time} comes from the volume of sections and from the 10-K segmenter, not from the extractor.
84 84
85 −\begin{table}[H]
86 −\centering\scriptsize
87 −\caption{Mentions, projects touched, filings and issuers by cohort, extraction run and form}\label{tab:june_october}
88 −\begin{threeparttable}
89 −\adjustbox{max width=\textwidth}{\input{tables/tab_june_october}}
90 −\begin{tablenotes}\footnotesize
91 −\item Source: \texttt{results/resolver/june\_october\_reconciliation.csv} (\texttt{code/audit/a03\_resolver\_speccurve.py}). \texttt{in\_sp500} is the cohort flag of the universe; \texttt{run} is ``backfilled\_june'' for rows stamped at the backfill timestamp and ``october'' otherwise; ``projects touched'' counts the projects of the current key that have at least one mention in the cell, so a project can be touched by both runs. The issuer-level regression of the project count on the cohort flag is in \texttt{results/runs/table6\_cohort\_regression.json} and summarized in \Cref{sec:time}.
92 −\end{tablenotes}
93 −\end{threeparttable}
94 −\end{table}
85 +\input{tables/tab_june_october}
95 86
96 87 \paragraph{Fiscal year.} \texttt{fiscal\_year} is empty for \SPIDfiscalYearNullMentions\ mentions because the corresponding \SPIDfiscalYearNullFilings\ filings carry no fiscal year in the corpus layer; a derived fill from the report or filing date is exported for \SPIDfiscalYearFillAccessions\ accessions (\texttt{results/runs/fiscal\_year\_fill\_by\_accession.csv}) and is not stored in the database.
97 88
98 89 \section{Additional tables}\label{app:tables}
99 90
100 −\begin{table}[H]
101 −\centering\footnotesize
102 −\caption{Coverage by filing year}\label{tab:coverage_year}
103 −\begin{threeparttable}
104 −\adjustbox{max width=\textwidth}{\input{tables/tab_coverage_year}}
105 −\begin{tablenotes}\footnotesize
106 −\item Source: \texttt{data/issuers\_by\_year.csv}, \texttt{data/filings\_by\_year\_form.csv}, \texttt{data/mentions\_by\_year.csv}, \texttt{data/projects\_by\_first\_seen\_year.csv}. ``Issuers'' counts issuers with at least one filing in the year; ``Projects first seen'' counts projects of the current key whose earliest mention is filed in the year, so 2010 is left-censored and 2026 (to \SPIDfilingDateLast) is partial. The fixed-cohort and windowed-key series are in \texttt{results/timeseries/first\_seen\_fixed\_cohort.csv} and \Cref{tab:timeseries_cohort}.
107 −\end{tablenotes}
108 −\end{threeparttable}
109 −\end{table}
110 −
111 −\begin{table}[H]
112 −\centering\scriptsize
113 −\caption{First-seen projects and mentions per issuer, all issuers and the fixed 2010 cohort, under the current key and under windowed keys}\label{tab:timeseries_cohort}
114 −\begin{threeparttable}
115 −\adjustbox{max width=\textwidth, max totalheight=0.82\textheight}{\input{tables/tab_timeseries_cohort}}
116 −\begin{tablenotes}\footnotesize
117 −\item Source: \texttt{results/timeseries/first\_seen\_fixed\_cohort.csv} (\texttt{code/audit/a09\_timeseries\_cohort.py}). ``fixed\_2010\_cohort'' restricts every series to the \SPIDfixedCohortIssuers\ issuers that file in every year from 2010 to 2025. ``first\_seen\_k3'' and ``first\_seen\_k5'' open a new episode when a project key is silent for more than three or five years (\SPIDepisodesKthree\ and \SPIDepisodesKfive\ episodes in total against \SPIDprojectCountCurrentKey\ projects). 2010 is left-censored and 2026 partial.
118 −\end{tablenotes}
119 −\end{threeparttable}
120 −\end{table}
121 −
122 −\begin{table}[H]
123 −\centering\small
124 −\caption{Issuers with the most resolved projects}\label{tab:top_issuers}
125 −\begin{threeparttable}
91 +\input{tables/tab_universe_artefacts}
92 +
93 +\input{tables/tab_coverage_year}
94 +
95 +\input{tables/tab_timeseries_cohort}
96 +
126 97 \input{tables/tab_top_issuers}
127 −\begin{tablenotes}\footnotesize
128 −\item Source: \texttt{data/top\_issuers.csv}. Counts are projects of the current key; rows with equal counts are ordered by ticker. The list is consistent with issuers whose business consists of enumerable sites, routes or wells; whether it also reflects over-splitting by the key has not been measured (\Cref{sec:resolution_rule}).
129 −\end{tablenotes}
130 −\end{threeparttable}
131 −\end{table}
132 −
133 −\begin{table}[H]
134 −\centering\small
135 −\caption{Self-reported confidence of mentions}\label{tab:confidence}
136 −\begin{threeparttable}
98 +
137 99 \input{tables/tab_confidence}
138 −\begin{tablenotes}\footnotesize
139 −\item Source: \texttt{data/confidence\_dist.csv}; confidence rounded to one decimal over the \SPIDmentions\ stored mentions. Mentions below \SPIDconfThreshold\ are not stored. The 0.6 bin includes \SPIDconfExactDefault\ mentions whose confidence was defaulted by the post-processing because the model returned none (\Cref{sec:contract}); the minimum stored value in both runs is 0.5. No calibration has been measured.
140 −\end{tablenotes}
141 −\end{threeparttable}
142 −\end{table}
143 −
144 −\begin{table}[H]
145 −\centering\small
146 −\caption{Knowledge graph composition}\label{tab:graph}
147 −\begin{threeparttable}
100 +
148 101 \input{tables/tab_graph}
149 −\begin{tablenotes}\footnotesize
150 −\item Source: \texttt{data/kg.csv}. Nodes are projected from the \texttt{projects} table of the current key (\texttt{owns}: company to project; \texttt{located\_in}: project to canonical location; \texttt{uses}: project to technology tag). Partner nodes are created without edges. Technology tags are free text and up to \SPIDtechRegexEqualPct\% of tag lists may have been back-filled from the model's description (\Cref{sec:contract}).
151 −\end{tablenotes}
152 −\end{threeparttable}
153 −\end{table}
154 −
155 −\begin{table}[H]
156 −\centering\scriptsize
157 −\caption{Most frequent name tokens of the resolution key among location-less mentions}\label{tab:top_name_tokens}
158 −\begin{threeparttable}
102 +
159 103 \input{tables/tab_top_name_tokens}
160 −\begin{tablenotes}\footnotesize
161 −\item Source: \texttt{results/resolver/top\_name\_tokens.csv} (\texttt{code/audit/a03\_resolver\_speccurve.py}), first twenty tokens. The token is the first word of at least four letters in the normalized project name of a mention without an extracted location (\Cref{sec:resolution_rule}); ``pct of token-keyed'' shares are over the \SPIDmentionsKeyedNameToken\ mentions and \SPIDprojectsKeyedNameToken\ projects keyed on a token.
162 −\end{tablenotes}
163 −\end{threeparttable}
164 −\end{table}
165 −
166 −\begin{table}[H]
167 −\centering\scriptsize
168 −\caption{Over-merge indicators of the resolution key, by category}\label{tab:overmerge}
169 −\begin{threeparttable}
170 −\adjustbox{max width=\textwidth}{\input{tables/tab_overmerge}}
171 −\begin{tablenotes}\footnotesize
172 −\item Source: \texttt{results/resolver/overmerge\_by\_type.csv}. ``pct\_ge3\_names'' (``pct\_ge2\_names'') is the share of projects of the category whose mentions carry three (two) or more distinct project names; ``pct\_ge3\_partners\_among\_with\_partner'' is the share, among projects with at least one partner, whose mentions name three or more distinct first partners; ``pct\_multi\_all\_names\_distinct'' is the share of multi-mention projects in which every mention carries a different name. Three or more distinct partners in one M\&A project is an over-merge by construction; distinct names may be renamings of one initiative or distinct initiatives, which only annotation can separate.
173 −\end{tablenotes}
174 −\end{threeparttable}
175 −\end{table}
176 −
177 −\begin{table}[H]
178 −\centering\scriptsize
179 −\caption{Field-level grounding of the structured fields in the source section, by form, item and field}\label{tab:grounding}
180 −\begin{threeparttable}
181 −\adjustbox{max width=\textwidth, max totalheight=0.84\textheight}{\input{tables/tab_grounding}}
182 −\begin{tablenotes}\footnotesize
183 −\item Source: \texttt{results/grounding/grounding\_by\_field\_form\_item.csv} (\texttt{code/audit/a02\_grounding.py}, all \SPIDmentions\ mentions joined to their stored section text; \SPIDgroundBootstrapReps\ bootstrap replicates clustered by section). A mention is ``grounded'' on a field when every element of the field is found in the section text after case folding (years as four-digit strings; locations, partners and technologies as literal strings; amounts in the usual renderings, e.g.\ ``1.5 billion'', ``1,500 million'', ``\$1.5B'', with one to three significant digits). Grounding measures transcription, not the correctness of the mention; the lower rates for locations and technologies reflect normalization by the model as well as invention; and for years and technologies the stored list may have been back-filled from the model's description (\Cref{sec:contract}).
184 −\end{tablenotes}
185 −\end{threeparttable}
186 −\end{table}
104 +
105 +\input{tables/tab_overmerge}
106 +
107 +
108 +\section{Stage D audit exhibits}\label{app:audit}
109 +
110 +The exhibits generated from the read-only audits of this revision (\texttt{code/audit/}, outputs in \texttt{results/}) are placed in the sections that discuss them: the ledger and outage reconciliation in \Cref{sec:ledger} (\Cref{tab:ledger_calls,tab:outage}), the focus-window coverage in \Cref{sec:focus} (\Cref{tab:focus,fig:focus}), the financing class in \Cref{sec:construct_def} (\Cref{tab:fp_classes}), the specification curve in \Cref{sec:speccurve} (\Cref{tab:spec_curve,fig:spec_curve}), the cohort regression in \Cref{sec:time} (\Cref{tab:cohort}), the grounding, 8-K consistency and rules-baseline results in \Cref{sec:val_mechanical} (\Cref{tab:grounding,fig:grounding,tab:benchmark_8k,tab:rules_agreement}), and the universe artefacts, cohort reconciliation, time series, name tokens and over-merge indicators in \Cref{app:runs,app:tables}.
187 111
188 112 \section{Replication}\label{app:replication}
189 113
modified paper/main.tex +8 −1
@@ -32,6 +32,7 @@
32 32 \usepackage{multirow}
33 33 \usepackage{threeparttable}
34 34 \usepackage{siunitx}
35 +\usepackage{makecell} % multi-line column headers of the generated tables (code/make_tables.py)
35 36 \sisetup{
36 37 input-symbols = {()},
37 38 table-format = -1.3,
@@ -92,8 +93,14 @@
92 93 \usetikzlibrary{positioning,arrows.meta,shapes.misc}
93 94 \usepackage{pdflscape}
94 95 \usepackage{adjustbox}
96 +\usepackage{makecell} % multi-line column headers of the generated tables (code/make_tables.py)
95 97 \usepackage{upquote} % straight quotes in verbatim (extraction contract, gate regex)
96 −\usepackage{verbatim} % \verbatiminput of the inter-revision patches (Appendix, results/runs/*.patch)
98 +\usepackage{listings} % \lstinputlisting (breaklines) of the inter-revision patches (Appendix, results/runs/*.patch)
99 +\lstset{basicstyle=\ttfamily\scriptsize, breaklines=true, breakatwhitespace=false, columns=fullflexible, keepspaces=true, frame=none, xleftmargin=0pt}
100 +% Notes under an adjustbox-scaled table: threeparttable measures the unscaled tabular, so its tablenotes overflow;
101 +% widenotes sets the notes in a \textwidth minipage and accepts the \item of the generated *_note.tex fragments.
102 +\newcommand{\widenotesitem}{\par}
103 +\newenvironment{widenotes}{\par\vspace{3pt}\begin{minipage}{\textwidth}\footnotesize\setlength{\parindent}{0pt}\let\item\widenotesitem}{\end{minipage}}
97 104 \input{tables/numbers} % generated inline-number macros (code/make_tables.py); never edit by hand
98 105 \input{sections/numbers_pending} % PROVISIONAL \providecommand macros for Stage D audit numbers not yet emitted by the generator (Agent 10); each line names its results/ source
99 106
modified paper/sections/applications.tex +2 −0
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Research Applications and Identification Limits}\label{sec:applications}
3 5 % ============================================================================
modified paper/sections/conclusion.tex +2 −0
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Limitations, Reproducibility and Conclusion}\label{sec:conclusion}
3 5 % ============================================================================
modified paper/sections/construct.tex +16 −4
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{The Project Construct and the Extraction Contract}\label{sec:construct}
3 5 % ============================================================================
@@ -15,7 +17,9 @@ The unit of measurement is a \emph{distinct, identifiable, issuer-associated str
15 17 \item \texttt{uncertain\_candidate}: a plausible initiative without enough identity evidence (no name, location, counterparty or amount) to be counted.
16 18 \end{itemize}
17 19
18 −The current build implements the first two tiers directly. The third is enforced only through the prompt and the confidence threshold; the fourth, fifth and sixth exist as instructions and as a confidence score, not as stored labels. The financing class was added in this revision after a regular-expression pass over the stored mentions (\texttt{repurchase|buy-?back|dividend|credit facility|revolving|term loan|notes? (due|offering)|at-the-market|senior notes|commercial paper} on the project name and description) matched \SPIDfinMentions\ mentions, \SPIDfinMentionsPct\% of the total, carrying \SPIDfinAmountB~B of the \SPIDfinAmountDenomB~B USD of mention amounts; the match concentrates in Part~II Item~2 of the 10-Q (\SPIDfinUnregPct\% of its mentions, where issuer repurchases are tabled) and in 8-K bodies (\SPIDfinEightKPct\%), and by category in \texttt{capex\_program} (\SPIDfinCapexPct\% of its mentions), \texttt{cost\_reduction} (\SPIDfinCostPct\%) and \texttt{ma\_integration} (\SPIDfinMaPct\%). \Cref{sec:results} reports the category counts with and without this class. Promoting the labels to explicit columns of the mention table, filled by annotators and eventually by a classifier, is the first schema change proposed in \texttt{docs/project\_ontology.md}.
20 +The current build implements the first two tiers directly. The third is enforced only through the prompt and the confidence threshold; the fourth, fifth and sixth exist as instructions and as a confidence score, not as stored labels. The financing class was added in this revision after a regular-expression pass over the stored mentions (\texttt{repurchase|buy-?back|dividend|credit facility|revolving|term loan|notes? (due|offering)|at-the-market|senior notes|commercial paper} on the project name and description) matched \SPIDfinMentions\ mentions, \SPIDfinMentionsPct\% of the total, carrying \SPIDfinAmountB~B of the \SPIDfinAmountDenomB~B USD of mention amounts; the match concentrates in Part~II Item~2 of the 10-Q (\SPIDfinUnregPct\% of its mentions, where issuer repurchases are tabled) and in 8-K bodies (\SPIDfinEightKPct\%), and by category in \texttt{capex\_program} (\SPIDfinCapexPct\% of its mentions), \texttt{cost\_reduction} (\SPIDfinCostPct\%) and \texttt{ma\_integration} (\SPIDfinMaPct\%). \Cref{sec:results} reports the category counts with and without this class, and \Cref{tab:fp_classes} gives the match by item, category and term. Promoting the labels to explicit columns of the mention table, filled by annotators and eventually by a classifier, is the first schema change proposed in \texttt{docs/project\_ontology.md}.
21 +
22 +\input{tables/tab_fp_classes}
19 23
20 24 \subsection{Taxonomy}
21 25
@@ -25,10 +29,14 @@ The SPID taxonomy has \SPIDtaxonomyN\ categories (\Cref{tab:taxonomy} in \Cref{a
25 29
26 30 A section is written to the extraction ledger if it contains at least \SPIDminWords\ words; it reaches the model only if it also matches a broad lexical gate (the case-insensitive regular expression reproduced from the code in \Cref{app:prompt}: project, initiative, program, invest, expan-, construct, build, launch, develop, deploy, implement, acqui-, partner, facilit-, capacity, capital, migrat-, transform, pipeline, plant, data center, automat-, sustainab-, renewable, restructur-). The gate is applied inside the extractor, which returns an empty list \emph{before any API call} when it fails, and the pipeline writes that result to the ledger as ``extracted with zero mentions'' exactly as it writes a model abstention. The ledger therefore counts \SPIDledgerTotal\ rows but only \SPIDgatePassTotal\ calls; \SPIDgateRejectedSections\ rows (\SPIDgateRejectedPct\%) were never sent, and no section with a stored mention fails the gate. The gate is deliberately permissive---it exists to avoid paying for sections that cannot contain a project---and its false-negative rate cannot be estimated from the ledger because gate rejections and abstentions are recorded identically; the annotation frame of \Cref{sec:validation} draws gate-rejected sections explicitly for that purpose.
27 31
28 −For sections longer than \SPIDfocusEmitBudget\ characters, the extractor collects windows of \SPIDfocusWindowBefore\ characters before and \SPIDfocusWindowAfter\ characters after each gate hit, in document order, merging overlaps, stops collecting at \SPIDfocusCollectBudget\ characters of source text, emits at most \SPIDfocusEmitBudget\ characters, and cuts the user message at \SPIDuserMessageCap\ characters (a limit never reached: \SPIDmsgTruncatedN\ messages were cut). The function is deterministic, so what the model was shown can be reconstructed for every ledgered section without a model call, and this revision does so (\texttt{results/focus/}). \SPIDfocusOverBudgetSections\ ledgered sections (\SPIDfocusOverBudgetSectionsPct\%) and \SPIDfocusOverBudgetCalls\ of the \SPIDgatePassTotal\ calls (\SPIDfocusOverBudgetCallsPct\%) exceed the budget; for them the mean fraction of the text shown is \SPIDfocusMeanShownOverBudgetPct\%. The submitted 10-K MD\&A sections (\SPIDmdnaN\ of them) have a median length of \SPIDmdnaMedianWords\ words or \SPIDmdnaMedianChars\ characters; the model saw a median \SPIDmdnaShownMedianPct\% of each (tenth percentile \SPIDmdnaShownPTenPct\%, ninetieth \SPIDmdnaShownPNinetyPct\%), the last character shown lies at \SPIDmdnaLastShownMedianPct\% of the text in the median section, \SPIDmdnaShownLtTwentyFivePct\% of these sections were shown for less than a quarter, and the median share of gate hits inside the shown windows is \SPIDmdnaHitsShownMedianPct\%. The medians are \SPIDbusinessShownMedianPct\% for the 10-K business item, \SPIDriskShownMedianPct\% for risk factors, \SPIDtenQmdnaShownMedianPct\% for the 10-Q MD\&A and \SPIDtenQfinShownMedianPct\% for the 10-Q financial-statement notes, whereas \SPIDeightKWholePct\% of 8-K bodies are shown whole. Because the windows are taken in document order and the budget is reached early, the liquidity and capital-resources discussion that closes a long MD\&A is structurally under-represented: only \SPIDcapexVocabShownMdnaPct\% of the occurrences of ``capital expenditure'', ``liquidity and capital resources'' or ``commitment'' in over-budget 10-K MD\&As fall inside a shown window (\SPIDcapexVocabShownBusinessPct\% in the business item, \SPIDcapexVocabShownTenQmdnaPct\% in the 10-Q MD\&A).
32 +For sections longer than \SPIDfocusEmitBudget\ characters, the extractor collects windows of \SPIDfocusWindowBefore\ characters before and \SPIDfocusWindowAfter\ characters after each gate hit, in document order, merging overlaps, stops collecting at \SPIDfocusCollectBudget\ characters of source text, emits at most \SPIDfocusEmitBudget\ characters, and cuts the user message at \SPIDuserMessageCap\ characters (a limit never reached: \SPIDmsgTruncatedN\ messages were cut). The function is deterministic, so what the model was shown can be reconstructed for every ledgered section without a model call, and this revision does so (\texttt{results/focus/}; \Cref{tab:focus}, \Cref{fig:focus}). \SPIDfocusOverBudgetSections\ ledgered sections (\SPIDfocusOverBudgetSectionsPct\%) and \SPIDfocusOverBudgetCalls\ of the \SPIDgatePassTotal\ calls (\SPIDfocusOverBudgetCallsPct\%) exceed the budget; for them the mean fraction of the text shown is \SPIDfocusMeanShownOverBudgetPct\%. The submitted 10-K MD\&A sections (\SPIDmdnaN\ of them) have a median length of \SPIDmdnaMedianWords\ words or \SPIDmdnaMedianChars\ characters; the model saw a median \SPIDmdnaShownMedianPct\% of each (tenth percentile \SPIDmdnaShownPTenPct\%, ninetieth \SPIDmdnaShownPNinetyPct\%), the last character shown lies at \SPIDmdnaLastShownMedianPct\% of the text in the median section, \SPIDmdnaShownLtTwentyFivePct\% of these sections were shown for less than a quarter, and the median share of gate hits inside the shown windows is \SPIDmdnaHitsShownMedianPct\%. The medians are \SPIDbusinessShownMedianPct\% for the 10-K business item, \SPIDriskShownMedianPct\% for risk factors, \SPIDtenQmdnaShownMedianPct\% for the 10-Q MD\&A and \SPIDtenQfinShownMedianPct\% for the 10-Q financial-statement notes, whereas \SPIDeightKWholePct\% of 8-K bodies are shown whole. Because the windows are taken in document order and the budget is reached early, the liquidity and capital-resources discussion that closes a long MD\&A is structurally under-represented: only \SPIDcapexVocabShownMdnaPct\% of the occurrences of ``capital expenditure'', ``liquidity and capital resources'' or ``commitment'' in over-budget 10-K MD\&As fall inside a shown window (\SPIDcapexVocabShownBusinessPct\% in the business item, \SPIDcapexVocabShownTenQmdnaPct\% in the 10-Q MD\&A).
29 33
30 34 Two consequences follow. First, \SPIDmentionsFromLtTwentyFive\ of the \SPIDledgerMentionsReturned\ mentions returned by the model (\SPIDmentionsFromLtTwentyFivePct\%) come from sections of which less than a quarter was shown---\SPIDmentionsFromLtTwentyFiveTenK\% of 10-K mentions, \SPIDmentionsFromLtTwentyFiveTenQ\% of 10-Q mentions and \SPIDmentionsFromLtTwentyFiveEightK\% of 8-K mentions---so most of the database rests on partial readings of its sources. Second, the yield is bounded by the budget rather than by the text: the number of mentions per 10-K MD\&A rises with section length from about \SPIDplateauLow\ to \SPIDplateauHigh\ up to roughly \SPIDplateauKchars\ characters and stays between \SPIDplateauRange\ beyond, about \SPIDmentionsPerTenKShownChars\ mentions per ten thousand shown characters regardless of length. A deterministic recall bound makes the same point from the other side: running the rule-based extractor of the code base on the full text and on the focused text of the over-budget sections, the focused text retains \SPIDrulesFocusRecovered\ of \SPIDrulesFocusTotal\ rule-based mentions (\SPIDrulesFocusRecoveredPct\%; \SPIDrulesFocusMdnaPct\% in the 10-K MD\&A, \SPIDrulesFocusBusinessPct\% in the business item, \SPIDrulesFocusTenQmdnaPct\% in the 10-Q MD\&A, \SPIDrulesFocusEightKPct\% in 8-K bodies), an upper bound on what any extractor can see through the windows (\Cref{sec:validation}). A separate concern raised in review---10-K MD\&As incorporated by reference and therefore too short to be submitted---turns out to be small: \SPIDshortMdnaN\ of \SPIDshortMdnaDenom\ parsed 10-K MD\&A sections (\SPIDshortMdnaPct\%) are under \SPIDminWords\ words, and \SPIDshortMdnaIncorporatedN\ are under 300 words with an incorporation-by-reference clause. Storing the focus spans per call is part of the proposed schema.
31 35
36 +\input{tables/tab_focus}
37 +
38 +\input{figures/fig_focus}
39 +
32 40 \subsection{The extraction contract as implemented}\label{sec:contract}
33 41
34 42 The LLM call is a chat completion with a fixed system prompt, the section name and the focused text as user message, temperature~0, the JSON-object response format and a \SPIDcompletionTokens-token completion limit. The response format guarantees syntactically valid JSON; it does not enforce a schema. The field list lives in the prompt (\Cref{app:prompt}): the model is instructed to act as a financial analyst extracting corporate projects, is given the \SPIDtaxonomyN\ allowed types, and is asked for a JSON object with one entry per project and the fields \texttt{project\_name}, \texttt{project\_type}, \texttt{description} (one sentence grounded in the text), \texttt{objective}, \texttt{status} (planned, in\_progress, completed, mentioned), \texttt{amount\_usd}, \texttt{locations}, \texttt{technologies}, \texttt{partners}, \texttt{benefits}, \texttt{risks}, \texttt{years} and \texttt{confidence}. Four rules in the prompt define the construct operationally: extract only real initiatives, not boilerplate or hypotheticals; return an empty list when no concrete project is described; do not invent facts---numbers and locations must appear in the text; prefer fewer high-quality projects. The output is then normalized deterministically, and several steps of that normalization were undocumented in the earlier version of this paper:
@@ -49,6 +57,10 @@ The instrument is re-runnable and ledgered, not deterministic. The calls used te
49 57
50 58 \subsection{Cost, throughput and the extraction ledger}\label{sec:ledger}
51 59
52 −Every ledgered section is written to the \texttt{extracted\_sections} table (section, accession, backend, number of mentions, timestamp), which makes the step resumable and prevents double billing. Restated on calls rather than rows, the October 2026 run wrote \SPIDoctoberRows\ rows of which \SPIDoctoberCalls\ were calls, and the June block \SPIDbackfilledRows\ rows of which \SPIDjuneCalls\ were calls; the model returned at least one project for \SPIDproductiveShareCalls\% of calls (\SPIDproductiveShareLedger\% of rows; \SPIDjuneProductiveShareCalls\% and \SPIDoctoberProductiveShareCalls\% by run), \SPIDmentionsPerCall\ mentions per call (\SPIDmentionsPerLedgerSection\ per row). At the authors' recorded total of about \SPIDtotalCostUsd~USD for the October run at the then-current \texttt{gpt-4o-mini} prices, the cost is about \SPIDusdPerCall~USD per call (\SPIDusdPerOctoberRow~USD per ledger row, the figure the earlier version reported ``per section''). The run was executed with \SPIDworkers\ concurrent workers; the ledger timestamps span \SPIDoctoberWallClockHours\ hours of wall-clock time (\SPIDoctoberFirstFlush\ to \SPIDoctoberLastFlush), of which \SPIDoutageHours\ hours are the largest gap, so the effective throughput over \SPIDoctoberEffectiveHours\ hours was \SPIDoctoberRowsPerSecondEffective\ rows or \SPIDoctoberCallsPerSecondEffective\ calls per second; the ``twelve hours'' and ``27--36 sections per second'' of the earlier version were mutually inconsistent and are withdrawn.
60 +Every ledgered section is written to the \texttt{extracted\_sections} table (section, accession, backend, number of mentions, timestamp), which makes the step resumable and prevents double billing. Restated on calls rather than rows, the October 2026 run wrote \SPIDoctoberRows\ rows of which \SPIDoctoberCalls\ were calls, and the June block \SPIDbackfilledRows\ rows of which \SPIDjuneCalls\ were calls; the model returned at least one project for \SPIDproductiveShareCalls\% of calls (\Cref{tab:ledger_calls}) (\SPIDproductiveShareLedger\% of rows; \SPIDjuneProductiveShareCalls\% and \SPIDoctoberProductiveShareCalls\% by run), \SPIDmentionsPerCall\ mentions per call (\SPIDmentionsPerLedgerSection\ per row). At the authors' recorded total of about \SPIDtotalCostUsd~USD for the October run at the then-current \texttt{gpt-4o-mini} prices, the cost is about \SPIDusdPerCall~USD per call (\SPIDusdPerOctoberRow~USD per ledger row, the figure the earlier version reported ``per section''). The run was executed with \SPIDworkers\ concurrent workers; the ledger timestamps span \SPIDoctoberWallClockHours\ hours of wall-clock time (\SPIDoctoberFirstFlush\ to \SPIDoctoberLastFlush), of which \SPIDoutageHours\ hours are the largest gap, so the effective throughput over \SPIDoctoberEffectiveHours\ hours was \SPIDoctoberRowsPerSecondEffective\ rows or \SPIDoctoberCallsPerSecondEffective\ calls per second; the ``twelve hours'' and ``27--36 sections per second'' of the earlier version were mutually inconsistent and are withdrawn.
61 +
62 +\input{tables/tab_ledger_calls}
63 +
64 +The ledger also records a failure mode encountered during the run, which this revision has reconciled against the run log rather than reported from memory (\Cref{tab:outage}). When the API key's credit balance was exhausted (log window \SPIDoutageWindowStart\ to \SPIDoutageWindowEnd), the extractor's retry logic returned empty results that were recorded as ``extracted with zero mentions'': the log carries \SPIDllmFailedRetryableHttpFourTwoNineFivexx\ retryable HTTP errors and \SPIDllmFailedConnectionError\ connection resets in that window, batches \SPIDoutageZeroBatchRange\ (\SPIDoutageZeroBatchWindow) returned no mention at all, and \SPIDoutageRowsRemoved\ rows were removed from the ledger---a number that reconciles exactly with the log as the sum of sections reported complete (\SPIDlogExtractionCompleteSum) minus the sections pending when the run was restarted (\SPIDlogPendingSecondStart). The final ledger has no row inside the largest gap (\SPIDoutageLargestGapStart\ to \SPIDoutageLargestGapEnd); \SPIDoutageSurvivingRows\ productive rows (\SPIDoutageSurvivingMentions\ mentions) stamped in the first quarter of an hour of the window survived because their calls had succeeded. Two residues remain. First, the extractor also returns an empty list after \SPIDretries\ failed attempts on any other error, and the log shows \SPIDllmFailedJsonDecodeError\ JSON-decode failures outside the outage (\SPIDjsonFailBeforeOutage\ before, \SPIDjsonFailAfterOutage\ after; \SPIDjuneLlmCallFailed\ in the June run), each ledgered as ``no project''; the earlier version reported one. Second, the first batch after the restart (\SPIDresumeBatchWindow; \SPIDresumeBatchSections\ sections of \SPIDresumeBatchCiks\ issuers, \SPIDresumeBatchCalls\ calls) returned \SPIDresumeBatchMentions\ mentions with no logged failure: \SPIDresumeBatchProductive\ productive sections against about \SPIDresumeBatchExpectedProductive\ expected at the form-by-item rates of the rest of the run, with \SPIDresumeBatchZeroSharePct\% zero-mention calls in its last minute. The most likely reading is a block of empty answers returned with a success status while the quota was being restored; those sections are ledgered as ``no project'' and were not re-run. A re-submission of the \SPIDresumeBatchCalls\ gate-passing sections of that batch (about \SPIDresumeBatchCalls\ calls) was requested during the review and is not granted in this revision, so the batch stays flagged as suspect in \Cref{tab:evidence} and the issuer list is published with the audit outputs. The general point stands: a ledger that cannot distinguish ``no project'' from ``call failed'' or ``empty answer'' is not sufficient, and an explicit \texttt{call\_status} column, together with the raw response, is part of the proposed schema.
53 65
54 −The ledger also records a failure mode encountered during the run, which this revision has reconciled against the run log rather than reported from memory. When the API key's credit balance was exhausted (log window \SPIDoutageWindowStart\ to \SPIDoutageWindowEnd), the extractor's retry logic returned empty results that were recorded as ``extracted with zero mentions'': the log carries \SPIDllmFailedRetryableHttpFourTwoNineFivexx\ retryable HTTP errors and \SPIDllmFailedConnectionError\ connection resets in that window, batches \SPIDoutageZeroBatchRange\ (\SPIDoutageZeroBatchWindow) returned no mention at all, and \SPIDoutageRowsRemoved\ rows were removed from the ledger---a number that reconciles exactly with the log as the sum of sections reported complete (\SPIDlogExtractionCompleteSum) minus the sections pending when the run was restarted (\SPIDlogPendingSecondStart). The final ledger has no row inside the largest gap (\SPIDoutageLargestGapStart\ to \SPIDoutageLargestGapEnd); \SPIDoutageSurvivingRows\ productive rows (\SPIDoutageSurvivingMentions\ mentions) stamped in the first quarter of an hour of the window survived because their calls had succeeded. Two residues remain. First, the extractor also returns an empty list after \SPIDretries\ failed attempts on any other error, and the log shows \SPIDllmFailedJsonDecodeError\ JSON-decode failures outside the outage (\SPIDjsonFailBeforeOutage\ before, \SPIDjsonFailAfterOutage\ after; \SPIDjuneLlmCallFailed\ in the June run), each ledgered as ``no project''; the earlier version reported one. Second, the first batch after the restart (\SPIDresumeBatchWindow; \SPIDresumeBatchSections\ sections of \SPIDresumeBatchCiks\ issuers, \SPIDresumeBatchCalls\ calls) returned \SPIDresumeBatchMentions\ mentions with no logged failure: \SPIDresumeBatchProductive\ productive sections against about \SPIDresumeBatchExpectedProductive\ expected at the form-by-item rates of the rest of the run, with \SPIDresumeBatchZeroSharePct\% zero-mention calls in its last minute. The most likely reading is a block of empty answers returned with a success status while the quota was being restored; those sections are ledgered as ``no project'' and were not re-run. A re-submission of the \SPIDresumeBatchCalls\ gate-passing sections of that batch (about \SPIDresumeBatchCalls\ calls) was requested during the review and is not granted in this revision, so the batch stays flagged as suspect in \Cref{tab:evidence} and the issuer list is published with the audit outputs. The general point stands: a ledger that cannot distinguish ``no project'' from ``call failed'' or ``empty answer'' is not sufficient, and an explicit \texttt{call\_status} column, together with the raw response, is part of the proposed schema.
66 +\input{tables/tab_outage}
modified paper/sections/data.tex +7 −24
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Sample: Issuer Universe and Filing Coverage}\label{sec:sample}
3 5 % ============================================================================
@@ -8,7 +10,7 @@ Two different populations can be labeled ``the Russell~3000''. The first is the
8 10
9 11 \subsection{Universe construction and reconciliation}
10 12
11 −The issuer universe is frozen in a single JSON artefact, \texttt{data/metadata/universe.json}, built by \texttt{scripts/build\_universe.py} and documented in \texttt{docs/universe\_definition.md}; the artefact and its two inputs are archived with the paper with SHA-256 checksums (\texttt{data/metadata/SHA256SUMS}: \texttt{\SPIDuniverseJsonSha\ldots}, \texttt{\SPIDrussellJsonSha\ldots}, \texttt{\SPIDspJsonSha\ldots}). It merges two sources on the SEC Central Index Key (CIK):
13 +The issuer universe is frozen in a single JSON artefact, \texttt{data/metadata/universe.json}, built by \texttt{scripts/build\_universe.py} and documented in \texttt{docs/universe\_definition.md}; the artefact and its two inputs are archived with the paper with SHA-256 checksums (\texttt{data/metadata/SHA256SUMS}; digests in \Cref{tab:universe_artefacts}). It merges two sources on the SEC Central Index Key (CIK):
12 14
13 15 \begin{enumerate}
14 16 \item the S\&P~500 constituent list scraped from the public Wikipedia table on \SPIDspWikipediaDate\ (\SPIDspWikipediaRows\ rows, \SPIDbackfilledCiks\ distinct CIKs, with GICS sector and sub-industry) \citep{wikipediasp500}, used for the first SPID release; and
@@ -17,16 +19,8 @@ The issuer universe is frozen in a single JSON artefact, \texttt{data/metadata/u
17 19
18 20 Tickers are normalized to the SEC convention (share-class separators become hyphens: ``BRK B'' $\to$ \texttt{BRK-B}) and mapped to CIKs through the SEC's public \texttt{company\_tickers.json} (\SPIDtickerMapSize\ entries at fetch time). \Cref{tab:reconciliation} gives the reconciliation from input securities to issuers with resolved projects, and two of its steps must be read with care. The first line, \SPIDiwvEquityLines\ equity lines with a ticker, is the count reported by the build log: neither the downloaded holdings file nor the ticker file was archived, the build script drops non-company lines (contingent value rights, escrow shares, rights when-issued) without recording them, and the difference between that count and the \SPIDrussellMappedLines\ mapped lines is therefore not reconstructible line by line. The \SPIDrussellUnmatched\ tickers that reached the mapping step without a CIK are listed in the artefact's \texttt{unmatched} field: four banks that file with the FDIC rather than the SEC and three tickers absent from the SEC ticker file on the download date, for which no reason is recorded. Earlier project documents gave other counts of excluded lines; the artefact count of \SPIDrussellUnmatched\ is the only one that can be verified and is the one used here. The \SPIDrussellMappedLines\ mapped lines collapse to \SPIDrussellDistinctCiks\ CIKs because \SPIDdualClassCiks\ registrants appear with two share classes. The merged universe of \SPIDuniverseN\ issuers is \SPIDbackfilledCiks\ S\&P~500 CIKs plus \SPIDrussellDistinctCiks\ proxy CIKs minus \SPIDuniverseBoth\ present in both lists; \SPIDuniverseSpOnly\ issuers are in the S\&P~500 snapshot but no longer in the IWV file (typically recently acquired) and are retained with an \texttt{in\_sp500} flag.
19 21
20 −\begin{table}[H]
21 −\centering\small
22 −\caption{Universe-to-dataset reconciliation}\label{tab:reconciliation}
23 −\begin{threeparttable}
22 +% exhibit generated by code/make_tables.py (self-contained float)
24 23 \input{tables/tab_reconciliation}
25 −\begin{tablenotes}\footnotesize
26 −\item Sources: \texttt{data/metadata/universe.json}, \texttt{russell3000\_constituents.json} and \texttt{sp500\_constituents.json} (archived with SHA-256 checksums), the merged DuckDB database, the IWV holdings file of \SPIDiwvHoldingsDate\ \citep{iwvholdings2026} and the Wikipedia S\&P~500 table of \SPIDspWikipediaDate\ \citep{wikipediasp500}. The first line is the count reported by the build log; the holdings file itself was not archived, so the lines dropped before the mapping step are not individually listed. The index held \SPIDrussellIndexConstituents\ constituents on \SPIDfactsheetDate\ \citep{ftserussell3000factsheet2026}; the proxy covers \SPIDrussellProxyCoveragePct\% of that count. The \SPIDissuersWithoutFilings\ issuers without filings are listed in \texttt{data/issuers\_without\_filings.csv}; most are foreign private issuers (e.g., Spotify, QIAGEN, Canadian Solar, Amdocs) that file annual reports on Form 20-F or 40-F, which are outside the current form scope, plus a few banks whose periodic reports are filed with the FDIC. \SPIDissuersFilingsNoProject\ issuers have filings but no resolved project.
27 −\end{tablenotes}
28 −\end{threeparttable}
29 −\end{table}
30 24
31 25 Several design decisions deserve to be stated plainly. The proxy universe counts \emph{issuers}, not securities. The universe table records, for each issuer, the SEC name, SIC code and description, state of incorporation, exchange and fiscal-year end from the SEC submissions header, the fund weight, and the cluster node that processed its filings (\texttt{shard\_node}). The research-design specification against which the build is audited in \texttt{docs/implementation\_audit.md} (hereafter ``the specification'': the list of required tables, keys and validation gates for a research-ready release) calls for a \texttt{universe\_snapshots} table with source checksums, membership dates and match confidence; this is only partially met. Snapshot date, source URL and match method are recorded and the derived artefacts are now hashed, but the source files are not archived and no per-row match confidence is stored, which is why gate~(a) of \Cref{tab:gates} is reported as partially met.
32 26
@@ -36,21 +30,10 @@ The June 2026 release remains a subset of the October build in the sense that ev
36 30
37 31 For every issuer the pipeline enumerates the SEC submissions index (\texttt{data.sec.gov/}\allowbreak\texttt{submissions/}\allowbreak\texttt{CIK\{cik\}.json}, including the paginated archive blocks) and retains the primary document of every Form 10-K, 10-Q and 8-K filed on or after January~1, 2010. Amendments (\texttt{/A}), exhibits other than the primary document, proxy statements, registration statements and foreign-issuer forms are out of scope. Within the retained documents, the extractor reads Items 1, 1A, 2, 7 and 7A of the 10-K, all seven segmented items of the 10-Q (Part~I Items 1--4, Part~II Items 1, 1A and 2) and the body of the 8-K; 10-K Items 3--6 and 8--9B are parsed but not submitted, and a 10-K in which the parser recognizes no Item heading at all is kept as one \texttt{full\_document} section that is outside the submitted scope (\SPIDfullDocumentTenK\ filings, of which \SPIDfullDocumentTenKLegacy\ were covered by the earlier S\&P~500 ingestion and \SPIDfullDocumentTenKInvisible\ are invisible to SPID; \Cref{sec:architecture}). The choice reflects where strategic initiatives are described in prose, and it is a defensible but not innocuous restriction: press releases attached as Exhibit~99 to an 8-K, which often carry the first and most detailed announcement of a project, are not read when they are not the primary document. Testing whether this scope omits important announcements is part of the validation roadmap.
38 32
39 −\begin{table}[H]
40 −\centering\small
41 −\caption{Filings in the corpus, by form}\label{tab:filings}
42 −\begin{threeparttable}
33 +% exhibit generated by code/make_tables.py
43 34 \input{tables/tab_filings}
44 −\begin{tablenotes}\footnotesize
45 −\item ``Not parsed'' counts filings whose primary document could not be cleaned (\SPIDparseErrors\ parser errors) or was logged as downloaded but not found on disk (\SPIDfilingsMissingOnDisk\ filings); the \SPIDfilingsNotParsed\ accession numbers are listed in \texttt{results/universe/filings\_status\_missing.csv}. Raw HTML size is the sum of primary-document sizes recorded at download time; no per-file checksum is stored.
46 −\end{tablenotes}
47 −\end{threeparttable}
48 −\end{table}
49 35
50 36 \Cref{tab:filings} summarizes the corpus: \SPIDfilingsTotal\ primary documents, \SPIDrawGb~GB of raw HTML, of which \SPIDfilingsNotParsed\ documents (\SPIDfilingsNotParsedPct\%) were not parsed. \Cref{fig:coverage} and \Cref{tab:coverage_year} in the appendix show the coverage by year. The number of issuers with at least one filing rises from \SPIDissuersFilingFirstYear\ in 2010 to \SPIDissuersFilingLastYear\ in 2026: the snapshot universe contains many companies that went public after 2010, so the early years of the panel are populated by the older and larger survivors. This is the concrete form taken by the survivorship caveat above.
51 37
52 −\begin{figure}[H]
53 −\centering
54 −\includegraphics[width=\textwidth]{figures/fig_coverage.pdf}
55 −\caption{Filings and issuers per year in the corpus. The 2026 bar covers January~1 to October~27, 2026. Issuers that went public after 2010 enter the panel at their first filing, so the number of observed issuers increases mechanically over time under the snapshot design. Source: \texttt{data/issuers\_by\_year.csv}, \texttt{data/filings\_by\_year\_form.csv}; generated by \texttt{code/make\_tables.py}.}\label{fig:coverage}
56 −\end{figure}
38 +% exhibit generated by code/make_tables.py
39 +\input{figures/fig_coverage}
modified paper/sections/introduction.tex +2 −0
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Introduction}
3 5 % ============================================================================
modified paper/sections/literature.tex +2 −0
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Related Literature}\label{sec:literature}
3 5 % ============================================================================
modified paper/sections/methodology.tex +5 −11
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Architecture: Acquisition, Parsing and the Distributed Build}\label{sec:architecture}
3 5 % ============================================================================
@@ -17,7 +19,7 @@
17 19 \node[box, right=of dl] (p) {Parsing\\ \scriptsize HTML $\to$ text, Item segmentation, chunks};
18 20 \node[box, right=of p] (e) {Embeddings \& factors\\ \scriptsize MiniLM \SPIDembeddingDim-d, \SPIDfactorAnchors\ anchors};
19 21 \node[art, below=of u] (ua) {\texttt{universe.json}\\ \texttt{shards.json}};
20 −\node[art, below=of dl] (da) {\texttt{download\_log.*.jsonl}\\ raw HTML on \SPIDacqNodes\ nodes};
22 +\node[art, below=of dl] (da) {\texttt{download\_log.jsonl}\\ raw HTML on \SPIDacqNodes\ nodes};
21 23 \node[art, below=of p] (pa) {\texttt{filings}, \texttt{filing\_sections}\\ \texttt{filing\_chunks} (Parquet)};
22 24 \node[art, below=of e] (ea) {\texttt{embeddings/}\\ \texttt{chunk\_scores}, panels};
23 25 \node[box, below=14mm of da, fill=red!5] (x) {Candidate gate + LLM extraction\\ \scriptsize regex gate, focus windows, JSON-object mode};
@@ -52,16 +54,8 @@ The 10-K items of the S\&P~500 block did not go through this path. They were pro
52 54
53 55 \Cref{tab:sections} reports the outcome by form. Two coverage statistics bound what the extractor could see: \SPIDfilingsNotParsedPct\% of filings were not parsed, and \SPIDfullDocumentTenKInvisible\ 10-K filings have neither recognized items nor a legacy row and are invisible to the project layer. The 10-K funnel reconciles exactly: \SPIDtenKparsedRows\ parsed 10-K rows minus \SPIDtenKoutOfScopeRows\ out-of-scope items, minus \SPIDtenKlegacyAccessionRows\ rows of filings covered by the legacy table, minus \SPIDtenKunderThirtyRows\ rows under \SPIDminWords\ words, gives \SPIDtenKparsedLedgered\ corpus-layer rows, which with the \SPIDlegacyTenKRows\ legacy rows equal the \SPIDledgerSectionsOneZeroK\ 10-K rows of the ledger.
54 56
55 −\begin{table}[H]
56 −\centering\small
57 −\caption{Parsed sections, ledgered sections, gate-passing calls and mentions, by form}\label{tab:sections}
58 −\begin{threeparttable}
59 −\adjustbox{max width=\textwidth}{\input{tables/tab_sections}}
60 −\begin{tablenotes}\footnotesize
61 −\input{tables/tab_sections_note}
62 −\end{tablenotes}
63 −\end{threeparttable}
64 −\end{table}
57 +% exhibit generated by code/make_tables.py (self-contained float)
58 +\input{tables/tab_sections}
65 59
66 60 \subsection{Distributed build and merge}
67 61
modified paper/sections/resolution.tex +7 −12
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Resolution, Timelines and Amounts}\label{sec:resolution}
3 5 % ============================================================================
@@ -20,18 +22,11 @@ The \SPIDsingleMentionShare\% of projects with a single mention are the populati
20 22
21 23 \subsection{Sensitivity of the project count to the key}\label{sec:speccurve}
22 24
23 −Because the key is a modelling choice, the project count is reported in this revision as a range. \Cref{tab:spec_curve} recomputes it for nine keys and five confidence thresholds from the same \SPIDmentions\ mentions. At the production threshold of \SPIDconfThreshold, grouping by issuer and type alone gives \SPIDprojectsKeyCikType\ projects; dropping the type from the current key gives \SPIDprojectsKeyTypeDropped; keying on the name token only gives \SPIDprojectsKeyNameOnly; the current key gives \SPIDprojectCountCurrentKey; normalizing city and state spellings gives \SPIDprojectsKeyCityNorm; removing stop-words before taking the token gives \SPIDprojectsKeyStopwords; keying on two salient tokens gives \SPIDprojectsKeyTwoTokens; a strict key on the full normalized name and location gives \SPIDprojectsKeyStrict; and treating every mention as its own project gives \SPIDprojectsKeyMentionLevel. Raising the threshold to 0.9 under the current key leaves \SPIDprojectsCurrentKeyAtNine\ projects. The single-mention share moves with the key from \SPIDsingleMentionShareCikType\% (issuer and type) to \SPIDsingleMentionShareStrict\% (strict), and the Cheniere cluster of \Cref{tab:provenance} has \SPIDchenierClusterCurrent\ mentions under the current key, \SPIDchenierClusterStrict\ under the strict key, \SPIDchenierClusterNameOnly\ under the name-only key and \SPIDchenierClusterCikType\ under issuer-and-type. The abstract and \Cref{sec:results} therefore quote \SPIDprojectCountCurrentKey\ ``under the current key'' and \SPIDprojectsKeyTypeDropped--\SPIDprojectsKeyStrict\ as the range over the alternative keys that keep a notion of initiative; none of these counts is a count of initiatives validated against human judgment.
24 −
25 −\begin{table}[H]
26 −\centering\scriptsize
27 −\caption{Project count, single-mention share and cluster statistics under alternative resolution keys and confidence thresholds (specification curve)}\label{tab:spec_curve}
28 −\begin{threeparttable}
29 −\adjustbox{max width=\textwidth}{\input{tables/tab_spec_curve}}
30 −\begin{tablenotes}\footnotesize
31 −\item Source: \texttt{results/resolver/spec\_curve.csv} (\texttt{code/audit/a03\_resolver\_speccurve.py}, read-only rebuild of the key from \texttt{project\_mentions} at revision \texttt{2324b58}). Keys: \texttt{k\_cik\_type} = issuer and type; \texttt{k\_type\_dropped} = issuer and location-or-token; \texttt{k\_current} = the production key (issuer, type, location-or-token); \texttt{k\_current\_stopwords} = current key with stop-words removed before the token; \texttt{k\_city\_normalised} = current key with city/state spellings normalized; \texttt{k\_name\_only} = issuer, type and token (location ignored); \texttt{k\_two\_tokens} = issuer, type and the first two salient tokens; \texttt{k\_strict} = issuer, type, full normalized name and location; \texttt{k\_mention\_level} = one project per mention. ``Threshold'' is the minimum mention confidence retained. ``Example cluster'' is the number of mentions in the Cheniere Corpus Christi cluster of \Cref{tab:provenance}. No key is validated against human labels; the table bounds the artefact of the key, it does not estimate the number of initiatives.
32 −\end{tablenotes}
33 −\end{threeparttable}
34 −\end{table}
25 +Because the key is a modelling choice, the project count is reported in this revision as a range. \Cref{tab:spec_curve} and \Cref{fig:spec_curve} recompute it for nine keys and five confidence thresholds from the same \SPIDmentions\ mentions. At the production threshold of \SPIDconfThreshold, grouping by issuer and type alone gives \SPIDprojectsKeyCikType\ projects; dropping the type from the current key gives \SPIDprojectsKeyTypeDropped; keying on the name token only gives \SPIDprojectsKeyNameOnly; the current key gives \SPIDprojectCountCurrentKey; normalizing city and state spellings gives \SPIDprojectsKeyCityNorm; removing stop-words before taking the token gives \SPIDprojectsKeyStopwords; keying on two salient tokens gives \SPIDprojectsKeyTwoTokens; a strict key on the full normalized name and location gives \SPIDprojectsKeyStrict; and treating every mention as its own project gives \SPIDprojectsKeyMentionLevel. Raising the threshold to 0.9 under the current key leaves \SPIDprojectsCurrentKeyAtNine\ projects. The single-mention share moves with the key from \SPIDsingleMentionShareCikType\% (issuer and type) to \SPIDsingleMentionShareStrict\% (strict), and the Cheniere cluster of \Cref{tab:provenance} has \SPIDchenierClusterCurrent\ mentions under the current key, \SPIDchenierClusterStrict\ under the strict key, \SPIDchenierClusterNameOnly\ under the name-only key and \SPIDchenierClusterCikType\ under issuer-and-type. The abstract and \Cref{sec:results} therefore quote \SPIDprojectCountCurrentKey\ ``under the current key'' and \SPIDprojectsKeyTypeDropped--\SPIDprojectsKeyStrict\ as the range over the alternative keys that keep a notion of initiative; none of these counts is a count of initiatives validated against human judgment.
26 +
27 +\input{tables/tab_spec_curve}
28 +
29 +\input{figures/fig_spec_curve}
35 30
36 31 \subsection{Timelines and point-in-time structure}
37 32
modified paper/sections/results.tex +16 −77
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Descriptive Patterns}\label{sec:results}
3 5 % ============================================================================
@@ -8,112 +10,49 @@ All tables, figures and inline numbers in this section are produced by \texttt{c
8 10
9 11 The extraction ledger contains \SPIDledgerTotal\ rows, of which \SPIDgatePassTotal\ passed the lexical gate and were submitted to the model (\Cref{sec:focus}). The model returned at least one project for \SPIDproductiveSections\ of them (\SPIDproductiveShareCalls\% of calls, \SPIDproductiveShareLedger\% of rows), yielding \SPIDledgerMentionsReturned\ mentions before identifier de-duplication and \SPIDmentions\ stored mentions---\SPIDmentionsPerCall\ per call and \SPIDmentionsPerProductiveSection\ per productive section. The density differs sharply by form and by item (\Cref{tab:items}): the MD\&A items of the 10-K and 10-Q produce \SPIDtenKmdnaMentions\ and \SPIDtenQmdnaMentions\ mentions respectively, Item~1 (Business) of the 10-K \SPIDtenKbusinessMentions, and the 10-Q financial-statement notes \SPIDtenQfinancialStatementsMentions, whereas risk-factor and market-risk items produce almost none despite their length---a pattern that \Cref{sec:focus} shows to be partly a property of the focus windows rather than of the text. The 8-K body yields \SPIDmentionsStoredEightK\ stored mentions (\SPIDmentionsLedgerEightK\ returned by the model) from \SPIDledgerSectionsEightK\ filings. Resolution reduces \SPIDmentions\ mentions to \SPIDprojectCountCurrentKey\ projects (\SPIDmentionsPerProject\ mentions per project); \SPIDsingleMentionShare\% of projects rest on a single mention, \SPIDtwoMentionShare\% on two, and \SPIDoverFiveMentionShare\% on more than five (\Cref{tab:status}).
10 12
11 −\begin{table}[H]
12 −\centering\small
13 −\caption{Ledgered sections, gate-passing calls and mentions obtained, by form and item}\label{tab:items}
14 −\begin{threeparttable}
15 13 \input{tables/tab_items}
16 −\begin{tablenotes}\footnotesize
17 −\item 8-K filings (\SPIDledgerSectionsEightK\ ledgered sections, \SPIDcallsEightK\ calls, \SPIDmentionsLedgerEightK\ mentions returned) are a single \texttt{full\_body} section each and are omitted from the table. ``Ledgered sections'' are rows of \texttt{extracted\_sections}; ``Gate passed'' recomputes the lexical gate of \Cref{app:prompt} over the ledgered text and counts the sections that reached the model. ``Mentions'' are the counts returned by the model as recorded in the ledger (\SPIDcollisionMentions\ fewer are stored; \Cref{sec:contract}). The 10-K \texttt{full\_body} rows are whole S\&P~500 10-Ks from the June 2026 ingestion in which no Item heading was found (\Cref{sec:segmenters}); the 10-Q \texttt{full\_document} rows are filings in which the corpus-layer parser found no heading. Two rows are dominated by the financing-instrument class of \Cref{sec:construct_def}: \SPIDfinUnregPct\% of the 10-Q \texttt{unregistered\_sales} mentions (Part~II Item~2, where issuer repurchases are tabled) and \SPIDfinEightKPct\% of 8-K mentions match the financing pattern.
18 −\end{tablenotes}
19 −\end{threeparttable}
20 −\end{table}
21 14
22 15 \subsection{Composition by category, sector and status}
23 16
24 17 \Cref{tab:types} and \Cref{fig:types} show the distribution over the \SPIDtaxonomyN\ categories. M\&A and post-merger integration dominate (\SPIDshareMaIntegration\% of projects), followed by capital-expenditure programs (\SPIDshareCapexProgram\%), strategic partnerships (\SPIDsharePartnership\%), geographic expansion (\SPIDshareGeographicExpansion\%) and plant or site construction (\SPIDsharePlantConstruction\%). These shares include the financing-instrument class: excluding projects whose representative name or description matches the financing pattern, \texttt{capex\_program} falls from \SPIDnCapexProgram\ to \SPIDcapexProjectsExcl\ projects (\SPIDshareCapexProgram\% to \SPIDcapexShareExcl\%), \texttt{cost\_reduction} from \SPIDnCostReduction\ to \SPIDcostProjectsExcl\ and \texttt{ma\_integration} from \SPIDnMaIntegration\ to \SPIDmaProjectsExcl\ (\texttt{results/fp\_classes/table5\_restated\_excl\_financing.csv}); \SPIDfinProjectsRep\ projects (\SPIDfinProjectsRepPct\%) match on their representative fields and \SPIDfinProjectsAny\ have at least one matching mention. The technology-related categories that motivated the first SPID release---data centers, AI initiatives, cloud migration, automation---are small in count (between \SPIDtechCategoriesCountRange\ projects each). Their disclosed amounts are reported in \Cref{tab:types} with the number of projects that carry one and a bootstrap interval for the median: the data-center median of \SPIDmedianAmountDataCenter~M USD rests on \SPIDwithAmountDataCenter\ projects with an amount and the AI median of \SPIDmedianAmountAiInitiative~M on \SPIDwithAmountAiInitiative, so the latter should not be quoted without its $N$. Two readings of the composition are possible and the data do not discriminate between them: either acquisitions are genuinely the most frequently disclosed discrete initiative in SEC filings, which is plausible given that every material acquisition triggers an 8-K and recurring 10-Q notes; or the extraction contract, which asks for ``concrete initiatives'', is more easily satisfied by acquisitions (named counterparty, date, price) than by internal programs. The 8-K item-code consistency check of \Cref{sec:validation} shows that the \texttt{ma\_integration} label co-occurs with the acquisition item at a rate far above its base rate, which supports the first reading for 8-Ks without settling the second. Disentangling the two is a question for the human-annotation study.
25 18
26 −\begin{table}[H]
27 −\centering\small
28 −\caption{Resolved projects by category, with disclosed amounts and an over-merge indicator}\label{tab:types}
29 −\begin{threeparttable}
30 −\adjustbox{max width=\textwidth}{\input{tables/tab_types}}
31 −\begin{tablenotes}\footnotesize
32 −\input{tables/tab_types_note}
33 −\end{tablenotes}
34 −\end{threeparttable}
35 −\end{table}
36 −
37 −\begin{figure}[H]
38 −\centering
39 −\includegraphics[width=0.95\textwidth]{figures/fig_types.pdf}
40 −\caption{Resolved projects by taxonomy category (\SPIDprojectCountCurrentKey\ projects under the current key; counts include the financing-instrument class of \Cref{sec:construct_def}). Source: \texttt{data/projects\_by\_type.csv}; generated by \texttt{code/make\_tables.py}.}\label{fig:types}
41 −\end{figure}
19 +% exhibit generated by code/make_tables.py (self-contained float)
20 +\input{tables/tab_types}
21 +
22 +\input{figures/fig_types}
42 23
43 24 By sector (\Cref{tab:sectors}), health care (\SPIDsectorProjectsHealthCare\ projects, \SPIDsectorIssuersHealthCare\ issuers) and industrials (\SPIDsectorProjectsIndustrials) contribute the most projects; \SPIDsectorTopFourNames\ have the most projects per issuer (\SPIDsectorTopFourPerIssuerRange), and financials the fewest (\SPIDsectorFinancialsPerIssuer), consistent with the intuition that asset-heavy, regulated or resource-extracting firms narrate discrete physical programs while banks describe fewer discrete initiatives---and also consistent with a key that creates one project per site. The list of the most ``productive'' issuers (\Cref{tab:top_issuers} in the appendix)---uranium and gold miners, airlines, a bitcoin miner, an oilfield-services company---is consistent with issuers whose business \emph{consists} of projects (mines, routes, wells, data-center sites) enumerating them one by one; whether it also reflects over-splitting by the key has not been measured.
44 25
45 −\begin{table}[H]
46 −\centering\small
47 −\caption{Issuers and projects by sector}\label{tab:sectors}
48 −\begin{threeparttable}
49 −\adjustbox{max width=\textwidth}{\input{tables/tab_sectors}}
50 −\begin{tablenotes}\footnotesize
51 −\item Sector labels come from the iShares holdings file (Russell proxy) or, for S\&P~500-only issuers, from the Wikipedia GICS table. ``Issuers in universe'' includes issuers without filings. Projects per issuer are computed over issuers with at least one project; the S\&P~500 issuers were extracted in two runs and with two 10-K segmenters, and their per-issuer counts are not comparable with those of Russell-only issuers without the adjustment of \Cref{sec:time}.
52 −\end{tablenotes}
53 −\end{threeparttable}
54 −\end{table}
26 +\input{tables/tab_sectors}
55 27
56 28 The latest status recorded for a project is \texttt{in\_progress} for \SPIDstatusShareInProgress\%, \texttt{completed} for \SPIDstatusShareCompleted\%, \texttt{planned} for \SPIDstatusSharePlanned\% and \texttt{mentioned} for \SPIDstatusShareMentioned\% (\Cref{tab:status}). Because status is taken from the most recent mention, it is a snapshot of the last thing the issuer said, not the terminal fate of the project: a project that was announced, mentioned once as in progress and then silently abandoned is recorded as in progress. The span between the earliest and latest year cited \emph{in the text} is unknown for \SPIDspanUnknownShareAll\% of projects (\SPIDspanUnknownN); among the \SPIDspanKnownN\ projects with cited years it is zero for \SPIDspanZeroShareKnown\% and exceeds five years for \SPIDspanOverFiveShareKnown\% (\SPIDspanZeroShareAll\% and \SPIDspanOverFiveShareAll\% of all projects). The specification's distinction between announcement, filing, expected start, observed start, expected and realized completion, and cancellation is not yet implemented as separate dated events (\Cref{sec:resolution}).
57 29
58 −\begin{table}[H]
59 −\centering\small
60 −\caption{Status, number of mentions and cited time span of resolved projects}\label{tab:status}
61 −\begin{threeparttable}
62 −\adjustbox{max width=\textwidth}{\input{tables/tab_status}}
63 −\begin{tablenotes}\footnotesize
64 −\item All three panels count the \SPIDprojectCountCurrentKey\ projects of the current key. ``Latest status'' is the status of the most recent mention (ties on the latest filing date, \SPIDprojectsStatusTie\ projects, resolved in execution order). ``Span of cited years'' is the difference between the largest and smallest four-digit year in the mentions' \texttt{years} lists; the \SPIDspanUnknownN\ projects without any cited year (\SPIDspanUnknownShareAll\%) are excluded from the span panel, so its shares are over \SPIDspanKnownN\ projects. Up to \SPIDyearsRegexEqualPct\% of year lists may have been back-filled from the model's description (\Cref{sec:contract}).
65 −\end{tablenotes}
66 −\end{threeparttable}
67 −\end{table}
30 +\input{tables/tab_status}
68 31
69 32 \subsection{Amounts, confidence and time}\label{sec:time}
70 33
71 34 A USD amount is attached to \SPIDamountMentions\ mentions (\SPIDamountMentionsShare\%) and to \SPIDamountProjects\ projects (\SPIDamountProjectsShare\%). \Cref{tab:amounts} reports the distribution of the largest amount cited per project: median \SPIDamountMedianMusd~M USD, interquartile range \SPIDamountPTwoFiveMusd--\SPIDamountPSevenFiveMusd~M, and a long right tail (P99 of \SPIDamountPNineNineBusd~B USD; maximum $\SPIDamountMaxSci$~USD; \SPIDamountGeFiftyBProjects\ projects at or above 50~B USD; \Cref{sec:amounts} on the grounding of the tail). The sum of these maxima across projects is \SPIDamountSumTusd~trillion USD, a number we report only to make the point that it must \emph{not} be interpreted: it adds transaction values, capex budgets, contract values, financing amounts (\SPIDfinAmountSharePct\% of mention amounts match the financing pattern) and estimated costs across projects and years, it double counts initiatives that the resolver split, and it treats a stated budget as if it were spent. \Cref{sec:amounts} explains how amounts are stored and what an \texttt{amount\_type} field would need to add.
72 35
73 −\begin{table}[H]
74 −\centering\small
75 −\caption{Distribution of disclosed amounts and of projects per issuer}\label{tab:amounts}
76 −\begin{threeparttable}
77 −\adjustbox{max width=\textwidth}{\input{tables/tab_amounts}}
78 −\begin{tablenotes}\footnotesize
79 −\input{tables/tab_amounts_note}
80 −\item Projects per issuer are counted over the \SPIDissuersWithProjects\ issuers with at least one project under the current key (maximum \SPIDprojectsPerIssuerMax); the S\&P~500 cohort was extracted in two runs (\Cref{sec:time}).
81 −\end{tablenotes}
82 −\end{threeparttable}
83 −\end{table}
36 +% exhibit generated by code/make_tables.py (self-contained float)
37 +\input{tables/tab_amounts}
84 38
85 39 The self-reported confidence of mentions is concentrated at the top of the scale (\Cref{fig:confidence}; \Cref{tab:confidence} in the appendix): \SPIDconfShareOne\% of mentions have confidence 1.0, \SPIDconfShareNine\% have 0.9, and only \SPIDconfShareLeSix\% fall at or below 0.6. Mentions below \SPIDconfThreshold\ are discarded at extraction time, and the 0.6 bin contains the \SPIDconfExactDefault\ mentions whose confidence the post-processing defaulted because the model returned none (\Cref{sec:contract}). The distribution says more about the model's reporting habits than about accuracy; the only accuracy-related quantity available so far moves the wrong way with confidence---the share of mentions whose structured fields are all found in the source text is \SPIDgroundAllTopBin\% in the 0.9--1.0 bin against \SPIDgroundAllLowerBinsRange\% in the lower bins (\Cref{sec:validation})---and whether a confidence of 0.7 corresponds to a lower precision than a confidence of 1.0 is precisely the calibration question that only labeled data can answer.
86 40
87 −\begin{figure}[H]
88 −\centering
89 −\includegraphics[width=0.85\textwidth]{figures/fig_confidence.pdf}
90 −\caption{Self-reported extraction confidence of the \SPIDmentions\ stored mentions (rounded to one decimal; mentions below \SPIDconfThreshold\ are not stored; the 0.6 bin includes \SPIDconfExactDefault\ defaulted values). Source: \texttt{data/confidence\_dist.csv}; generated by \texttt{code/make\_tables.py}.}\label{fig:confidence}
91 −\end{figure}
41 +\input{figures/fig_confidence}
92 42
93 43 Over time (\Cref{fig:mentions_year}; \Cref{tab:coverage_year}), mentions rise from \SPIDmentionsFirstYear\ in 2010 to \SPIDmentionsLastFullYear\ in 2025, almost in proportion to the number of issuers observed (\SPIDissuersWithMentionsFirstYear\ to \SPIDissuersWithMentionsLastFullYear\ with at least one mention; \SPIDmentionsPerIssuerFirstYear\ to \SPIDmentionsPerIssuerLastFullYear\ mentions per such issuer), while the number of projects first observed in a year lies between \SPIDfirstSeenRangeInner\ over \SPIDfirstSeenYearMin--\SPIDfirstSeenYearMax. The first and last years are excluded from that range on purpose: 2010 is left-censored, since every initiative already under way when the corpus starts is ``first seen'' then (\SPIDfirstSeenFirstYear\ projects), and 2026 is partial (\SPIDfirstSeenLastYear\ projects to October~27). The rise in mentions is consistent with a coverage effect of the snapshot universe rather than with an increase in disclosed activity: holding the panel fixed at the \SPIDfixedCohortIssuers\ issuers that file in every year from 2010 to 2025, mentions per issuer move from \SPIDfixedCohortMentionsPerIssuerFirst\ to \SPIDfixedCohortMentionsPerIssuerLast\ and newly seen projects per issuer from \SPIDfixedCohortNewPerIssuerFirst\ to \SPIDfixedCohortNewPerIssuerLast\ (\SPIDfixedCohortFirstSeenFirst\ to \SPIDfixedCohortFirstSeenLast\ projects; \texttt{results/timeseries/first\_seen\_fixed\_cohort.csv}). The decline in new projects per issuer is, at least in part, mechanical: under a time-invariant key, later mentions of a serial acquirer or of a multi-phase site attach to an existing project rather than opening a new one. A windowed variant that opens a new episode when a key is silent for more than three (five) years counts \SPIDepisodesKthree\ (\SPIDepisodesKfive) episodes instead of \SPIDprojectCountCurrentKey\ projects, and its first-seen series is reported alongside the current one in \Cref{tab:coverage_year}. Both readings remain testable with the panel; neither has been tested against labels.
94 44
95 −\begin{figure}[H]
96 −\centering
97 −\includegraphics[width=\textwidth]{figures/fig_mentions_year.pdf}
98 −\caption{Project mentions and projects first observed, by filing year, with the per-issuer series in the second panel. The y-axis starts at zero; 2010 (left-censored) and 2026 (January to October~27) are shaded. Source: \texttt{data/mentions\_by\_year.csv}, \texttt{data/projects\_by\_first\_seen\_year.csv}; generated by \texttt{code/make\_tables.py}.}\label{fig:mentions_year}
99 −\end{figure}
45 +\input{figures/fig_mentions_year}
46 +
47 +The two extraction runs add a confound of their own to any comparison between the S\&P~500 cohort and the rest of the universe. Issuers in both lists carry \SPIDcohortBothPerIssuerAll\ projects per issuer against \SPIDcohortRussellOnlyPerIssuer\ for Russell-only issuers, which the earlier version read as a difference in disclosure behaviour; counting only projects touched by an October-run mention, the same issuers carry \SPIDcohortBothPerIssuerOctober. In an issuer-level regression of the project count on an S\&P~500 indicator (\SPIDcohortOlsN\ issuers, HC1 standard errors), the coefficient is \SPIDcohortOlsAll\ (\SPIDcohortOlsAllSe) with all mentions, $\SPIDcohortOlsOctober$ (\SPIDcohortOlsOctoberSe) with October mentions only, and $\SPIDcohortOlsControlled$ (\SPIDcohortOlsControlledSe) once the logarithm of the issuer's ledgered sections is controlled for (\texttt{results/runs/table6\_cohort\_regression.json}). The cohort difference is a difference in run and volume, not evidence about behaviour, and \Cref{app:runs} gives the reconciliation by run and form (\Cref{tab:cohort}).
100 48
101 −The two extraction runs add a confound of their own to any comparison between the S\&P~500 cohort and the rest of the universe. Issuers in both lists carry \SPIDcohortBothPerIssuerAll\ projects per issuer against \SPIDcohortRussellOnlyPerIssuer\ for Russell-only issuers, which the earlier version read as a difference in disclosure behaviour; counting only projects touched by an October-run mention, the same issuers carry \SPIDcohortBothPerIssuerOctober. In an issuer-level regression of the project count on an S\&P~500 indicator (\SPIDcohortOlsN\ issuers, HC1 standard errors), the coefficient is \SPIDcohortOlsAll\ (\SPIDcohortOlsAllSe) with all mentions, $\SPIDcohortOlsOctober$ (\SPIDcohortOlsOctoberSe) with October mentions only, and $\SPIDcohortOlsControlled$ (\SPIDcohortOlsControlledSe) once the logarithm of the issuer's ledgered sections is controlled for (\texttt{results/runs/table6\_cohort\_regression.json}). The cohort difference is a difference in run and volume, not evidence about behaviour, and \Cref{app:runs} gives the reconciliation by run and form.
49 +\input{tables/tab_cohort}
102 50
103 51 \subsection{A provenance example}
104 52
105 53 \Cref{tab:provenance} follows one resolved project through its mentions: the Corpus Christi liquefaction trains of Cheniere Energy (ticker LNG), typed \texttt{plant\_construction}, with \SPIDprovenanceN\ mentions in \SPIDprovenanceForms\ filings between \SPIDprovenanceFirst\ and \SPIDprovenanceLast\ and a largest cited amount of \SPIDprovenanceMaxAmountBusd~B USD. Each row is traceable: the accession number identifies the document on EDGAR, the section identifier locates the passage in the parsed corpus, and the snippet is the model's one-sentence description, which the contract requires to be grounded in the passage but which is not itself checked (\Cref{sec:contract}). The example also exposes the resolver's coarseness. The project key for this issuer and type is the location string ``san patricio county, texas'', so the Stage~3 expansion and the earlier trains share one project identifier although Cheniere describes them as distinct phases; the representative name, ``Corpus Christi Liquefaction Stage~3 Project'', is that of the highest-confidence mention, filed in \SPIDprovenanceNameYear, and is applied retroactively to mentions from 2014 that describe the first trains; and that representative is one admissible choice among tied mentions (\Cref{sec:resolution_rule}). Under the strict key the same mentions form \SPIDchenierClusterStrict\ project each; under the issuer-and-type key they join \SPIDchenierClusterCikType\ mentions. A reviewer can see this because the mentions and their snippets are preserved; a user who consumes only the \texttt{projects} table cannot.
106 54
107 −\begin{table}[H]
108 −\centering\footnotesize
109 −\caption{Provenance of one resolved project: Cheniere Energy, Corpus Christi liquefaction (project \texttt{3e2705ba\ldots})}\label{tab:provenance}
110 −\begin{threeparttable}
111 −\adjustbox{max width=\textwidth}{\input{tables/tab_provenance}}
112 −\begin{tablenotes}\footnotesize
113 −\item All \SPIDprovenanceN\ rows are Form 8-K mentions (\texttt{data/example\_provenance.csv}). Snippets are the first characters of the model-generated description stored in \texttt{project\_timeline.snippet}; the full source section is retrievable through \texttt{project\_mentions.section\_id}. Amounts are shown only for mentions that cite one. The representative name shown in the caption of the project is the \SPIDprovenanceNameYear\ mention's name; it is not a label of the 2014 mentions.
114 −\end{tablenotes}
115 −\end{threeparttable}
116 −\end{table}
55 +\input{tables/tab_provenance}
117 56
118 57 \subsection{Knowledge graph and research-ready exports}
119 58
modified paper/sections/validation.tex +14 −3
@@ -1,3 +1,5 @@
1 +% Owner: Agent 10 (Stage E writing). Every generated exhibit is \input exactly once in the prose at the anchor
2 +% where it is discussed; please do not add a second \input of the same float (duplicate labels). — 2026-11-05
1 3 % ============================================================================
2 4 \section{Validation Framework and Current Audit}\label{sec:validation}
3 5 % ============================================================================
@@ -18,18 +20,27 @@ The aim of identifying ``all projects'' of an issuer is a \emph{coverage target}
18 20
19 21 \subsection{Mechanical accuracy audits (implemented in this revision)}\label{sec:val_mechanical}
20 22
21 −\paragraph{Field-level grounding.} The prompt requires that numbers and locations appear in the text, and the structured fields are literal tokens, so their containment in the stored section text can be tested for every mention. For all \SPIDmentions\ mentions joined to their section text (\texttt{results/grounding/}; \SPIDgroundBootstrapReps\ bootstrap replicates clustered by section), every listed year is found in \SPIDgroundYearsPct\% of mentions with years (\SPIDgroundYearsCiLow--\SPIDgroundYearsCiHigh), every listed partner in \SPIDgroundPartnersPct\%, the amount as a number (in any of the usual renderings) in \SPIDgroundAmountNumberPct\% and with a currency token within forty characters in \SPIDgroundAmountCurrencyPct\%, every listed location in \SPIDgroundLocationsPct\% (10-K \SPIDgroundLocationsTenK\%, 10-Q \SPIDgroundLocationsTenQ\%, 8-K \SPIDgroundLocationsEightK\%), and every technology tag in \SPIDgroundTechPct\%; at the level of individual elements the rates are \SPIDgroundElemYears\% for years, \SPIDgroundElemPartners\% for partners, \SPIDgroundElemLocations\% for locations and \SPIDgroundElemTech\% for technologies. All structured fields of a mention are found in \SPIDgroundAllPct\% of the \SPIDgroundAllN\ mentions that carry at least one (10-K \SPIDgroundAllTenK\%, 10-Q \SPIDgroundAllTenQ\%, 8-K \SPIDgroundAllEightK\%; June block \SPIDgroundAllJune\%, October \SPIDgroundAllOctober\%), with no difference between filings before and after the model's training cut-off beyond one percentage point. Three readings of these rates are needed. First, they are rates of \emph{transcription}, not of correctness: a year, partner or amount can be copied faithfully from a passage that describes a counterparty's initiative, a hypothetical, or a financing instrument, so grounding is necessary for a correct mention and not sufficient. Second, the lower rates for locations and technologies reflect normalization by the model (``Texas'' for ``Austin, TX''; ``cloud computing'' for ``AWS'') as much as invention; the element-level rate for locations is \SPIDgroundElemLocations\%. Third, \SPIDyearsAbsentFromText\ mentions carry a year absent from the section, and the technology and year lists equal the regex output on the model's own description for up to \SPIDtechRegexEqualPct\% and \SPIDyearsRegexEqualPct\% of mentions, so for those fields ``grounded'' may mean ``copied from the paraphrase'' (\Cref{sec:contract}). The full breakdown by field, form and item is in \Cref{tab:grounding}.
23 +\paragraph{Field-level grounding.} The prompt requires that numbers and locations appear in the text, and the structured fields are literal tokens, so their containment in the stored section text can be tested for every mention. For all \SPIDmentions\ mentions joined to their section text (\texttt{results/grounding/}; \SPIDgroundBootstrapReps\ bootstrap replicates clustered by section), every listed year is found in \SPIDgroundYearsPct\% of mentions with years (\SPIDgroundYearsCiLow--\SPIDgroundYearsCiHigh), every listed partner in \SPIDgroundPartnersPct\%, the amount as a number (in any of the usual renderings) in \SPIDgroundAmountNumberPct\% and with a currency token within forty characters in \SPIDgroundAmountCurrencyPct\%, every listed location in \SPIDgroundLocationsPct\% (10-K \SPIDgroundLocationsTenK\%, 10-Q \SPIDgroundLocationsTenQ\%, 8-K \SPIDgroundLocationsEightK\%), and every technology tag in \SPIDgroundTechPct\%; at the level of individual elements the rates are \SPIDgroundElemYears\% for years, \SPIDgroundElemPartners\% for partners, \SPIDgroundElemLocations\% for locations and \SPIDgroundElemTech\% for technologies. All structured fields of a mention are found in \SPIDgroundAllPct\% of the \SPIDgroundAllN\ mentions that carry at least one (10-K \SPIDgroundAllTenK\%, 10-Q \SPIDgroundAllTenQ\%, 8-K \SPIDgroundAllEightK\%; June block \SPIDgroundAllJune\%, October \SPIDgroundAllOctober\%), with no difference between filings before and after the model's training cut-off beyond one percentage point. Three readings of these rates are needed. First, they are rates of \emph{transcription}, not of correctness: a year, partner or amount can be copied faithfully from a passage that describes a counterparty's initiative, a hypothetical, or a financing instrument, so grounding is necessary for a correct mention and not sufficient. Second, the lower rates for locations and technologies reflect normalization by the model (``Texas'' for ``Austin, TX''; ``cloud computing'' for ``AWS'') as much as invention; the element-level rate for locations is \SPIDgroundElemLocations\%. Third, \SPIDyearsAbsentFromText\ mentions carry a year absent from the section, and the technology and year lists equal the regex output on the model's own description for up to \SPIDtechRegexEqualPct\% and \SPIDyearsRegexEqualPct\% of mentions, so for those fields ``grounded'' may mean ``copied from the paraphrase'' (\Cref{sec:contract}). The breakdown by field and form is in \Cref{tab:grounding} and \Cref{fig:grounding}; the full breakdown by item is in \texttt{results/grounding/grounding\_by\_field\_form\_item.csv}.
24 +
25 +\input{tables/tab_grounding}
26 +
27 +\input{figures/fig_grounding}
22 28
23 29 \paragraph{What the model was shown.} \Cref{sec:focus} reports the shown fraction of every ledgered section. The recall of the pipeline on long sections is bounded from above by what the windows contain: on the over-budget sections the rule-based extractor of the code base recovers \SPIDrulesFocusRecoveredPct\% of its full-text mentions from the focused text (\SPIDrulesFocusMdnaPct\% in the 10-K MD\&A). Gate~(e) is therefore reported as met at passage level, not at section level.
24 30
25 −\paragraph{External consistency with 8-K item codes.} Form 8-K bodies name the items they report under, and \SPIDeightKItemCodePct\% of the \SPIDledgerSectionsEightK\ 8-K sections carry at least one item code in their text, which gives an external classification at no cost (\texttt{results/benchmark\_8k/}). Among 8-Ks filed under Item~2.01 (completion of an acquisition or disposition; \SPIDbenchTwoOhOneN\ sections) the model returned at least one \texttt{ma\_integration} mention for \SPIDbenchMaGivenTwoOhOne\ (\SPIDbenchMaGivenTwoOhOneCiLow--\SPIDbenchMaGivenTwoOhOneCiHigh) of sections, against \SPIDbenchMaGivenEightOhOne\ under Item~8.01 (other events) and \SPIDbenchMaGivenFiveOhTwo\ under Item~5.02 (officer changes) and a base rate of \SPIDbenchMaBase; conversely \SPIDbenchItemsGivenMa\ of 8-K \texttt{ma\_integration} mentions sit in a filing under Item~2.01 or 1.01. Under Item~2.05 (exit or disposal costs; \SPIDbenchTwoOhFiveN\ sections) the model returned a \texttt{cost\_reduction} mention for \SPIDbenchCostGivenTwoOhFive\ (\SPIDbenchCostGivenTwoOhFiveCiLow--\SPIDbenchCostGivenTwoOhFiveCiHigh) of sections, a lift of about \SPIDbenchCostLift\ over the base rate, and \SPIDbenchTwoOhFiveGivenCost\ of \texttt{cost\_reduction} mentions sit under that item; Item~1.01 (material agreements) carries a \texttt{partnership} mention in \SPIDbenchPartnershipGivenOneOhOne\ of cases against a base rate of \SPIDbenchPartnershipBase. These are consistency rates with an administrative classification of the \emph{filing}, not precision or recall of the \emph{mention}: an Item~2.01 filing without an M\&A mention may describe a disposition, and an Item~8.01 filing with one may be a genuine acquisition announced under a general item. They show that the dominant category is not an artefact of the extractor's vocabulary; they do not measure its accuracy.
31 +\paragraph{External consistency with 8-K item codes.} Form 8-K bodies name the items they report under, and \SPIDeightKItemCodePct\% of the \SPIDledgerSectionsEightK\ 8-K sections carry at least one item code in their text, which gives an external classification at no cost (\texttt{results/benchmark\_8k/}). Among 8-Ks filed under Item~2.01 (completion of an acquisition or disposition; \SPIDbenchTwoOhOneN\ sections) the model returned at least one \texttt{ma\_integration} mention for \SPIDbenchMaGivenTwoOhOne\ (\SPIDbenchMaGivenTwoOhOneCiLow--\SPIDbenchMaGivenTwoOhOneCiHigh) of sections, against \SPIDbenchMaGivenEightOhOne\ under Item~8.01 (other events) and \SPIDbenchMaGivenFiveOhTwo\ under Item~5.02 (officer changes) and a base rate of \SPIDbenchMaBase; conversely \SPIDbenchItemsGivenMa\ of 8-K \texttt{ma\_integration} mentions sit in a filing under Item~2.01 or 1.01. Under Item~2.05 (exit or disposal costs; \SPIDbenchTwoOhFiveN\ sections) the model returned a \texttt{cost\_reduction} mention for \SPIDbenchCostGivenTwoOhFive\ (\SPIDbenchCostGivenTwoOhFiveCiLow--\SPIDbenchCostGivenTwoOhFiveCiHigh) of sections, a lift of about \SPIDbenchCostLift\ over the base rate, and \SPIDbenchTwoOhFiveGivenCost\ of \texttt{cost\_reduction} mentions sit under that item; Item~1.01 (material agreements) carries a \texttt{partnership} mention in \SPIDbenchPartnershipGivenOneOhOne\ of cases against a base rate of \SPIDbenchPartnershipBase. These are consistency rates with an administrative classification of the \emph{filing}, not precision or recall of the \emph{mention}: an Item~2.01 filing without an M\&A mention may describe a disposition, and an Item~8.01 filing with one may be a genuine acquisition announced under a general item. They show that the dominant category is not an artefact of the extractor's vocabulary; they do not measure its accuracy (\Cref{tab:benchmark_8k}).
32 +
33 +\input{tables/tab_benchmark_8k}
26 34
27 −\paragraph{Agreement with a rule-based baseline.} The code base contains a rule-based extractor (\texttt{rules.py}) that needs no model call. Run on a seeded, stratified sample of \SPIDrulesSampleProductive\ productive and \SPIDrulesSampleZero\ zero-mention ledger sections with inverse-probability weights (\texttt{results/rules/}), the rules flag at least one candidate in \SPIDrulesGivenLlmPct\% of the sections where the model returned a mention (10-K \SPIDrulesGivenLlmTenK\%, 10-Q \SPIDrulesGivenLlmTenQ\%, 8-K \SPIDrulesGivenLlmEightK\%; June \SPIDrulesGivenLlmJune\%, October \SPIDrulesGivenLlmOctober\%), while the model returns a mention in \SPIDllmGivenRulesPct\% of the sections the rules flag; the rules flag \SPIDrulesFlagPct\% of ledgered sections against \SPIDproductiveShareLedger\% for the model, section-level agreement is \SPIDrulesAgreementPct\% (model-only \SPIDrulesLlmOnlyPct\%, rules-only \SPIDrulesOnlyPct\%), and on co-detected sections the model's category is among the rule categories in \SPIDrulesTypeOverlapPct\% of cases (\texttt{ma\_integration} \SPIDrulesTypeMa\%, \texttt{drug\_pipeline} \SPIDrulesTypeDrug\%, \texttt{cost\_reduction} \SPIDrulesTypeCost\%, \texttt{capex\_program} \SPIDrulesTypeCapex\%, \texttt{plant\_construction} \SPIDrulesTypePlant\%, \texttt{geographic\_expansion} \SPIDrulesTypeGeo\%). Neither extractor is a ground truth, so this is a measure of the marginal information in the model step---it abstains on most of what the lexical rules flag and rarely fires where they do not---rather than of accuracy. A full-corpus run of the rules is deferred; the sample gives roughly half a percentage point of precision on the rates.
35 +\paragraph{Agreement with a rule-based baseline.} The code base contains a rule-based extractor (\texttt{rules.py}) that needs no model call. Run on a seeded, stratified sample of \SPIDrulesSampleProductive\ productive and \SPIDrulesSampleZero\ zero-mention ledger sections with inverse-probability weights (\texttt{results/rules/}), the rules flag at least one candidate in \SPIDrulesGivenLlmPct\% of the sections where the model returned a mention (10-K \SPIDrulesGivenLlmTenK\%, 10-Q \SPIDrulesGivenLlmTenQ\%, 8-K \SPIDrulesGivenLlmEightK\%; June \SPIDrulesGivenLlmJune\%, October \SPIDrulesGivenLlmOctober\%), while the model returns a mention in \SPIDllmGivenRulesPct\% of the sections the rules flag; the rules flag \SPIDrulesFlagPct\% of ledgered sections against \SPIDproductiveShareLedger\% for the model, section-level agreement is \SPIDrulesAgreementPct\% (model-only \SPIDrulesLlmOnlyPct\%, rules-only \SPIDrulesOnlyPct\%), and on co-detected sections the model's category is among the rule categories in \SPIDrulesTypeOverlapPct\% of cases (\texttt{ma\_integration} \SPIDrulesTypeMa\%, \texttt{drug\_pipeline} \SPIDrulesTypeDrug\%, \texttt{cost\_reduction} \SPIDrulesTypeCost\%, \texttt{capex\_program} \SPIDrulesTypeCapex\%, \texttt{plant\_construction} \SPIDrulesTypePlant\%, \texttt{geographic\_expansion} \SPIDrulesTypeGeo\%). Neither extractor is a ground truth, so this is a measure of the marginal information in the model step---it abstains on most of what the lexical rules flag and rarely fires where they do not---rather than of accuracy. A full-corpus run of the rules is deferred; the sample gives roughly half a percentage point of precision on the rates (\Cref{tab:rules_agreement}).
36 +
37 +\input{tables/tab_rules_agreement}
28 38
29 39 \paragraph{Known false-positive classes.} The financing-instrument pattern of \Cref{sec:construct_def} matches \SPIDfinMentionsPct\% of mentions, concentrated in Part~II Item~2 of the 10-Q and in 8-K bodies, and \Cref{sec:results} restates the category counts without them. A related parser check finds that the 10-Q Part~II Item~2 section (median \SPIDunregMedianWords\ words) is longer than \SPIDunregLongThreshold\ words in \SPIDunregGtThreeK\ of \SPIDitemTwoPairs\ filings (\SPIDunregGtThreeKPct\%), \SPIDswapCandidates\ of which have an MD\&A shorter than 500 words and are candidates for a Part~I/Part~II ``Item~2'' swap by the segmenter; \SPIDunregGtThreeKMentions\ mentions come from those sections. Both classes are detected by pattern, not by annotation, so their sizes are indicative, and the annotation protocol labels them explicitly.
30 40
31 41 \paragraph{Resolver sensitivity.} The dependence of the project count on the key and on the confidence threshold is reported in \Cref{sec:speccurve}; the over-merge indicators (distinct names and partners per project, amounts taken from a different counterparty) are in \Cref{sec:resolution_rule}.
32 42
43 +
33 44 \subsection{Accuracy evaluation against human labels (frame drawn, labels not produced)}\label{sec:val_human}
34 45
35 46 \paragraph{Sampling frame.} The unit is the ledgered section, that is, exactly the text block the extractor received a verdict for. The frame was drawn with a fixed seed (\SPIDannSeed) from the two populations of the ledger (\texttt{results/annotation\_frame/}; protocol in \texttt{docs/annotation\_protocol.md}). From the \SPIDannPopProductive\ productive sections, \SPIDannStrata\ strata are defined by item group, run (June block or October), the shown-fraction bin of \Cref{sec:focus}, the confidence bin of the section's best mention and the category group; allocation is proportional with a floor of \SPIDannFloor\ per stratum and a \SPIDannOversample-fold oversampling of the confidence bin at or below 0.7, which gives $n = \SPIDannNProductive$ (the floor dominates; a nested core subsample of \SPIDannCore\ drawn proportionally within the frame is flagged for a first pass). From the \SPIDannPopZero\ zero-mention sections, \SPIDannNZero\ are drawn with inclusion probability proportional to the gate verdict and to the logarithm of section length, so that gate-rejected sections are represented (\SPIDannZeroGatePass\ gate-passing, \SPIDannZeroGateReject\ gate-rejected) and false negatives can be estimated with inverse-probability weights; the earlier version's equal-count draw without weights would have biased recall upward. The frame is split into a development half (\SPIDannDev\ sections), on which prompt and threshold changes may be tuned, and a locked holdout half (\SPIDannHoldout), used once per release. The design effects are \SPIDannDeffProductive\ (productive) and \SPIDannDeffZero\ (zero-mention); the 95\% half-widths are $\pm$\SPIDannHalfWidthPrecision\ percentage points for a precision near 0.5 and $\pm$\SPIDannHalfWidthFn\ points for a false-negative rate near 5\%, and the minimum detectable difference in the false-negative rate between gate-passing and gate-rejected strata is \SPIDannMde\ points. The full-text frames, the text the model was shown, and the sealed model output are stored on the master node (\SPIDannFrameMb~MB); the worktree copies carry the text truncated to \SPIDannTextChars\ characters.
deleted paper/tables/prompt.tex +0 −30
@@ -1,30 +0,0 @@
1 −\begin{quote}\footnotesize\ttfamily\setlength{\parskip}{0pt}\obeylines
2 −You are a financial analyst extracting CORPORATE PROJECTS from SEC filing text.
3 −A ''project'' is any concrete initiative, investment, program, expansion, construction,
4 −product/R\&D effort, acquisition/integration, technology deployment, partnership, or
5 −strategic program the company is pursuing.
6 −
7 −Return STRICT JSON: \{''projects'': [ ... ]\}. Each project object has:
8 − project\_name : short descriptive name (use the official name if one is stated)
9 − project\_type : EXACTLY ONE of the allowed types below
10 − description : 1 sentence, grounded in the text (no speculation)
11 − objective : the goal/why (or '''')
12 − status : one of planned | in\_progress | completed | mentioned
13 − amount\_usd : numeric USD figure if stated (e.g. 1500000000), else null
14 − locations : array of places (city/state/country), else []
15 − technologies : array of technologies involved, else []
16 − partners : array of named partners/JV counterparties, else []
17 − benefits : array of expected benefits, else []
18 − risks : array of stated risks, else []
19 − years : array of 4-digit years mentioned, else []
20 − confidence : 0.0-1.0 (how clearly this is a real, concrete project)
21 −
22 −Allowed project\_type values:
23 − - <code>: <label> (the 21 taxonomy entries of Table A1)
24 −
25 −Rules:
26 −- Extract ONLY real initiatives, not generic risk-factor boilerplate or hypotheticals.
27 −- If the text describes no concrete project, return \{''projects'': []\}.
28 −- Do not invent facts. Numbers/locations must appear in the text.
29 −- Prefer fewer high-quality projects over many vague ones.
30 −\end{quote}
deleted paper/tables/tab_amounts_note.tex +0 −1
@@ -1 +0,0 @@
1 −\item Amounts are the largest USD figure cited across the mentions of a project; they mix transaction values, budgets, contract values and estimates and are never summed in the text.
deleted paper/tables/tab_sections_note.tex +0 −1
@@ -1 +0,0 @@
1 −\item ``Ledgered sections'' counts rows of the \texttt{extracted\_sections} ledger (sections with at least 30 words; for 10-Ks restricted to the items of Table~\ref{tab:items}); the ledger does not record whether the candidate gate passed or whether the API call succeeded, so this is not a count of calls; ``Gate-passing calls'' recomputes the candidate gate of Appendix~\ref{app:prompt} over the ledgered text (703{,}725 sections reached the model). ``Mentions returned by the model'' is $\sum$\texttt{n\_mentions} over the ledger (308{,}477); ``Mentions stored'' is the row count of \texttt{project\_mentions} (308{,}065). The difference of 412 (10-K 127, 10-Q 257, 8-K 28), spread over 304 sections, is the number of mentions overwritten by mention-identifier collisions before storage (Section~\ref{sec:construct}). $^{\dagger}$Issuers with at least one filing of any form, not a column sum.
deleted paper/tables/tab_table3_calls.tex +0 −9
@@ -1,9 +0,0 @@
1 −\begin{tabular}{lrrrrrrr}
2 −\toprule
3 −form\_type & ledger\_rows & calls & with\_mentions & returned\_by\_model\_before\_dedup & stored\_mentions & issuers & issuers\_with\_mentions \\
4 −\midrule
5 −10-K & 143{,}704 & 135{,}310 & 30{,}877 & 81{,}622 & 81{,}495 & 2{,}464 & 2{,}344 \\
6 −10-Q & 578{,}793 & 363{,}973 & 75{,}705 & 152{,}261 & 152{,}004 & 2{,}494 & 2{,}446 \\
7 −8-K & 400{,}601 & 204{,}442 & 61{,}648 & 74{,}594 & 74{,}566 & 2{,}499 & 2{,}462 \\
8 −\bottomrule
9 −\end{tabular}
deleted paper/tables/tab_types_note.tex +0 −1
@@ -1 +0,0 @@
1 −\item Shares are rounded to one decimal and sum to 100.1. ``With amount'' counts projects for which at least one mention carries a USD figure; ``median amount'' is the median across those projects of the largest amount cited (Section~\ref{sec:amounts} on why amounts are never summed). Medians printed in grey rest on fewer than 50 projects with an amount (\texttt{ai\_initiative}, \texttt{automation}, \texttt{cloud\_migration}) and should not be quoted without their $N$. The 95\% interval column is filled when \texttt{data/projects\_by\_type\_ci.csv} (bootstrap on the node) is present. ``$\geq 3$ distinct names'' is the share of projects of the category whose mentions carry three or more distinct project names (over-merge indicator of the resolution key, \texttt{results/resolver/overmerge\_by\_type.csv}).
2
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere