SPB Git

spb/firc0001_uqo

Public
8.0 KB · 36 lines latex
1% ============================================================================2\section{The Project Construct and the Extraction Contract}\label{sec:construct}3% ============================================================================45\subsection{What counts as a project}67The unit of measurement is a \emph{distinct, identifiable, issuer-associated strategic initiative} involving a planned, announced, ongoing, completed, postponed or cancelled action: a plant expansion, a research program, a technology deployment, an AI initiative, a data-center build, a supply-chain transformation, an acquisition and its integration, a renewable-energy investment, and the other categories of the taxonomy. The database stores two tiers and, in its intended final form, five labels:89\begin{itemize}10\item \texttt{project\_mention}: a particular sourced statement about an initiative, in one section of one filing, with the extracted fields and the model's one-sentence description;11\item \texttt{project}: a resolved initiative described by one or more mentions, carrying a stable identifier and a timeline;12\item \texttt{generic\_strategy}: non-specific aspirations (``we continue to invest in innovation'', ``we may pursue acquisitions'') that the contract instructs the model to exclude and that the resolver never sees unless the model fails to exclude them;13\item \texttt{historical\_or\_third\_party\_reference}: initiatives of a competitor, a customer or a former parent, mentioned in the issuer's filing but not attributable to the issuer;14\item \texttt{uncertain\_candidate}: a plausible initiative without enough identity evidence (no name, location, counterparty or amount) to be counted.15\end{itemize}1617The current build implements the first two tiers directly. The third is enforced only through the prompt and the confidence threshold; the fourth and fifth exist as instructions and as a confidence score, not as stored labels. Promoting them to explicit columns of the mention table, filled by annotators and eventually by a classifier, is the first schema change proposed in \texttt{docs/project\_ontology.md}.1819\subsection{Taxonomy}2021The SPID taxonomy has 21 categories (Table~\ref{tab:taxonomy} in Appendix~\ref{app:taxonomy}), defined in \texttt{spid/taxonomy.py} with a human-readable label and a set of lexical signals used by the rule-based extractor and by the candidate gate. The LLM extractor receives the 21 labels verbatim in its system prompt and must choose exactly one per project; a label outside the list is mapped back to the closest category by substring matching or, failing that, to \texttt{technology\_deployment}. The categories are not mutually exclusive in substance (an acquired plant is both \texttt{ma\_integration} and \texttt{plant\_construction}; a data center is also \texttt{infrastructure}), and the model's choice in such cases is a classification decision we have not yet audited. We document the taxonomy as it stands rather than redesign it: any extension would be versioned, with a crosswalk from old to new codes, so that counts published under the current version remain reproducible.2223\subsection{Candidate gating and focus windows}2425A section reaches the model only if it contains at least 30 words and matches a broad lexical gate (project, initiative, program, invest, expan-, construct, build, launch, develop, deploy, implement, acqui-, partner, facilit-, capacity, capital, migrat-, transform, pipeline, plant, data center, automat-, sustainab-, renewable, restructur-; see Appendix~\ref{app:prompt}). The gate is deliberately permissive---it exists to avoid paying for sections that cannot contain a project---and its false-negative rate is unknown; sampling rejected sections to estimate it is part of the validation plan. For sections longer than 9{,}000 characters, the extractor keeps windows of about 600 characters around each gate hit (merged when they overlap) up to a budget of roughly 9{,}000--12{,}600 characters, so that the cost per call is bounded regardless of section length. This ``focus'' step is the main source of \emph{silent truncation} in the pipeline: a project described far from any gate word, or beyond the budget in a very long MD\&A, is never shown to the model. The 10-K MD\&A sections submitted have a median length of 9{,}905 words (roughly 60{,}000 characters), so truncation is the rule rather than the exception for annual reports, and the recall of the pipeline on long sections is an open empirical question.2627\subsection{The extraction contract}2829The LLM call is a chat completion with a fixed system prompt, the section name and the focused text as user message, temperature~0, JSON-object output mode and a 2{,}000-token completion limit. The prompt (Appendix~\ref{app:prompt}) instructs the model to act as a financial analyst extracting corporate projects, lists the 21 allowed types, and returns a JSON object with one entry per project and the following fields: \texttt{project\_name}, \texttt{project\_type}, \texttt{description} (one sentence grounded in the text), \texttt{objective}, \texttt{status} (planned, in\_progress, completed, mentioned), \texttt{amount\_usd}, \texttt{locations}, \texttt{technologies}, \texttt{partners}, \texttt{benefits}, \texttt{risks}, \texttt{years} and \texttt{confidence}. Four rules in the prompt define the construct operationally: extract only real initiatives, not boilerplate or hypotheticals; return an empty list when no concrete project is described; do not invent facts---numbers and locations must appear in the text; prefer fewer high-quality projects. Output is post-processed deterministically: amounts are coerced to a float in USD (with the raw string retained in \texttt{amount\_raw}), list fields are truncated to eight items of at most 120 characters, years are parsed as four-digit integers, status is normalized, and mentions with confidence below 0.45 are dropped. The mention identifier is a SHA-1 of the accession number, section identifier, type and project name, so that re-running the extractor on the same section with the same output overwrites rather than duplicates.3031What the contract does \emph{not} guarantee should be equally explicit. The ``description'' is a model paraphrase, not a verbatim quote, and the character span of the supporting passage is not stored; grounding is therefore verifiable by a reader who opens the section, but not mechanically checkable. The model may extract a counterparty's initiative as the issuer's (the third-party label above). It may split one initiative into several entries when a section describes phases, or merge phases into one. It receives no information about the issuer other than the text, which protects against hallucinated background knowledge but also prevents it from recognizing a subsidiary name. And the same section is seen once; a project described across two sections of the same filing yields two mentions with distinct identifiers.3233\subsection{Cost, throughput and the extraction ledger}3435The October 2026 run submitted 968{,}134 sections in about twelve hours of effective extraction (48 concurrent calls, 27--36 sections per second), at a cost of roughly 775~USD for the full universe at current \texttt{gpt-4o-mini} prices (about 0.0008~USD per section). Every submitted section is written to an \texttt{extracted\_sections} ledger (section, accession, backend, number of mentions, timestamp), which makes the step resumable and prevents double billing. The ledger also protects against a failure mode encountered during the run: when the API key's credit balance was exhausted, the extractor's retry logic returned empty results that were recorded as ``extracted with zero mentions'' for 149{,}352 sections before the problem was detected. Those rows were removed from the ledger using the timestamp of the first quota error and the sections were re-submitted; the final ledger has no section processed during the outage. The episode is recorded in the research log because it illustrates a general point: a ledger that cannot distinguish ``no project'' from ``call failed'' is not sufficient, and an explicit \texttt{call\_status} column is now part of the proposed schema.36
Files · 6a7431b04424e703801360a0b98bd8984663df1b
Menu

Theme