SPB Git
spboucher.ai

spb/firc0001_uqo

Public
7.7 KB

# Git and file-system evidence for the two SPID extraction runs (N4a)

Generated 2026-11-05T14:24:12-05:00 on m3u96b from a read-only inspection of ~/UQO_RESEARCH_DB (clone of gitsrv:srv/git/uqo-research-db.git).

# Repository state

text
HEAD: d70086876dc850d656db6f2b5479285154352ae8
branch: main
tags: 
remote: origin	gitsrv:srv/git/uqo-research-db.git (fetch)
working tree (git status --short | head -20):

# All commits (there is no commit before 2026-10-27; the June 2026 run has no git revision)

text
d700868 2026-10-28 13:46:05 -0400 Simon-Pierre Boucher Rapport SPID : taille du corpus brut 255 Go (somme des tailles de fichiers)
2324b58 2026-10-28 12:35:01 -0400 Simon-Pierre Boucher Rapport SPID : taille de la base 3,2 Go
1a08d92 2026-10-28 12:33:13 -0400 Simon-Pierre Boucher Rapport SPID mis à jour pour l univers S&P 500 ∪ Russell 3000 (figures, tableaux, prose) ; make_figures lit le registre extracted_sections
e0483da 2026-10-28 12:28:05 -0400 Simon-Pierre Boucher SPID: registre extracted_sections, DuckDB 2.0, scripts de run/reprise (extraction Russell 3000 terminée 2026-10-28)
df7905d 2026-10-27 21:33:45 -0400 Simon-Pierre Boucher Russell 3000: correctifs du run distribué (section full_document, fusion append-only, décodage Arrow, rééquilibrage des files, clés de rapatriement)
0304133 2026-10-27 13:12:55 -0400 Simon-Pierre Boucher UQO EDGAR Research DB: extension S&P 500 -> Russell 3000 (pipeline scalable, exécution distribuée MacLustr)

# First commit of each extractor / parser file (git log --diff-filter=A)

text
project-db/spid/extract/llm.py                0304133 2026-10-27 13:12:55 -0400
project-db/spid/extract/rules.py              0304133 2026-10-27 13:12:55 -0400
project-db/spid/extract/base.py               0304133 2026-10-27 13:12:55 -0400
project-db/spid/taxonomy.py                   0304133 2026-10-27 13:12:55 -0400
project-db/spid/resolve.py                    0304133 2026-10-27 13:12:55 -0400
project-db/spid/ingest.py                     0304133 2026-10-27 13:12:55 -0400
project-db/spid/config.py                     0304133 2026-10-27 13:12:55 -0400
project-db/spid/pipeline.py                   0304133 2026-10-27 13:12:55 -0400
project-db/spid/store.py                      0304133 2026-10-27 13:12:55 -0400
project-db/spid/__main__.py                   0304133 2026-10-27 13:12:55 -0400
project-db/run_full_extract.py                0304133 2026-10-27 13:12:55 -0400
src/parsing/html_parser.py                    0304133 2026-10-27 13:12:55 -0400
src/parsing/section_extractor.py              0304133 2026-10-27 13:12:55 -0400
src/parsing/filing_parser.py                  0304133 2026-10-27 13:12:55 -0400
src/parsing/chunker.py                        0304133 2026-10-27 13:12:55 -0400
scripts/parse_filings.py                      0304133 2026-10-27 13:12:55 -0400

# Commits touching each file

text
project-db/spid/extract/llm.py                0304133 
project-db/spid/extract/rules.py              0304133 
project-db/spid/extract/base.py               0304133 
project-db/spid/taxonomy.py                   0304133 
project-db/spid/resolve.py                    0304133 
project-db/spid/ingest.py                     0304133 
project-db/spid/config.py                     0304133 
project-db/spid/pipeline.py                   0304133 
project-db/spid/store.py                      e0483da 0304133 
project-db/spid/__main__.py                   0304133 
project-db/run_full_extract.py                0304133 
src/parsing/html_parser.py                    0304133 
src/parsing/section_extractor.py              0304133 
src/parsing/filing_parser.py                  df7905d 0304133 
src/parsing/chunker.py                        0304133 
scripts/parse_filings.py                      0304133

# File modification times (file-system evidence; extractor modules untouched since 2026-06-17)

text
project-db/spid/extract/llm.py                2026-06-17 01:31:51  8007 bytes  sha256=ae7f3f61a8de3684
project-db/spid/extract/rules.py              2026-06-17 01:31:37  4554 bytes  sha256=7f191c18b0426d83
project-db/spid/extract/base.py               2026-06-17 01:31:29  2138 bytes  sha256=4ba1811f9513ab47
project-db/spid/taxonomy.py                   2026-06-17 01:31:17  12272 bytes  sha256=92ad69bcc4e448d0
project-db/spid/resolve.py                    2026-06-17 01:33:03  2786 bytes  sha256=925ab1258b2a364f
project-db/spid/ingest.py                     2026-06-17 01:35:03  4787 bytes  sha256=c53536becfe273f0
project-db/spid/config.py                     2026-10-27 12:34:12  4325 bytes  sha256=8c4a273631e00158
project-db/spid/pipeline.py                   2026-10-27 12:34:12  3832 bytes  sha256=6135f39adb9e78b7
project-db/spid/store.py                      2026-10-27 21:34:38  9888 bytes  sha256=1219c923fff72b99
project-db/spid/__main__.py                   2026-10-27 12:34:46  10512 bytes  sha256=8eccff89cf604005
project-db/run_full_extract.py                2026-10-27 12:34:46  2764 bytes  sha256=5f50795cc0ba49f0
src/parsing/html_parser.py                    2026-05-25 12:12:27  4656 bytes  sha256=61aca675ff4652e3
src/parsing/section_extractor.py              2026-05-25 12:12:36  6733 bytes  sha256=51970779101a4f34
src/parsing/filing_parser.py                  2026-10-27 19:54:51  7823 bytes  sha256=e17e6b0fc7bbaa25
src/parsing/chunker.py                        2026-10-27 12:13:39  2925 bytes  sha256=716a4013fb839a68
scripts/parse_filings.py                      2026-10-27 12:18:10  6467 bytes  sha256=35248a647e8d8dbf

# git diff --stat 0304133..2324b58 (first commit -> revision cited by the paper) for the extractor, parser and driver

text
 project-db/spid/store.py     |  4 +++-
 src/parsing/filing_parser.py | 20 +++++++++++++-------
 2 files changed, 16 insertions(+), 8 deletions(-)

# git diff --stat df7905d..2324b58 (the pair named in the paper)

text
 project-db/spid/store.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

# git diff --stat 2324b58..HEAD (after the paper's revision)

text

# June 2026 run parameters from project-db/logs/extract.log (first and last lines; model id absent)

text
08:28:02 INFO    spid.driver | CIKs total=500 done=0 todo=500 | batch=15 workers=16
08:28:03 INFO    spid.pipeline | Extracting from 4271 sections | backend=llm | workers=16
...
11:06:44 INFO    spid.driver | BATCH 34/34 done | batch_sec=116 batch_men=47 | CUM sections=153894 mentions=39761
11:06:44 INFO    spid.driver | FULL EXTRACT COMPLETE | cum_sections=153894 cum_mentions(this run)=39761 | TOTAL mentions in DB=39930
grep -c 'gpt' project-db/logs/extract.log = 0
grep -c 'gpt' data/logs/spid_extract.log = 0
project-db/logs mtimes:
  project-db/logs/extract_done_ciks.txt    2026-06-18 11:06:44
  project-db/logs/extract.log              2026-06-18 11:06:44
  project-db/logs/ingest.log               2026-06-18 08:27:06
  project-db/logs/post.log                 2026-06-18 11:20:12
project-db/.env present: yes (content never read)
project-db/.env.example model line: SPID_LLM_MODEL=gpt-4o-mini #   SPID_LLM_MODEL=maclustr-general

# Interpretation

  • The repository's first commit (0304133, 2026-10-27) already contains project-db/; no revision of the June 2026 code exists in git. The sentence '2324b58 supersedes df7905d examined in the June study' names two October commits.
  • The extractor modules (llm.py, rules.py, base.py, taxonomy.py, resolve.py, ingest.py) carry 2026-06-17 modification times: the prompt, the gate, the taxonomy and the resolver used in October are, by file-system evidence, the files that existed before the June run. The driver/config/store (config.py, pipeline.py, main.py: 2026-10-27 12:34; store.py: 2026-10-27 21:34) were edited before the first commit, so their June versions are not recoverable.
  • The only change to these files between df7905d and 2324b58 is the one-line DuckDB 2.0 compatibility fix in store.py (see patch).
Files · e80ef398ede4f5784e4d15d7b4453d5f38c765cc
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere