# PROGRESS — WP45 / JoF-250

Verified milestones only. Times are America/Toronto unless marked UTC. Code versions refer to `git rev-parse --short HEAD`.

## Amendments (decisions taken by the executor, to be ratified by the author)

| ID | Date | Decision | Rationale |
|---|---|---|---|
| A-001 | 2026-11-03 | API budget envelope fixed at **90 USD** (initially 60, raised before the annotation run once the per-document cost was measured at ≈0.09 USD on Opus 5.5) (config/models.yaml `budget_usd_cap`) for annotation + pilot experiments, `claude-sonnet-5-5`, key from `~/Desktop/Cluster/secrets/wp42/.env` | Mandate forbids spending without authorization; the author's standing practice on WP26/WP29/WP42/WP44 is an executor-fixed envelope logged as an amendment. Every call is logged in `data/annotations/api_ledger.jsonl`. |
| A-002 | 2026-11-03 | Corpus scope = the 4 volumes present in the folder (72–75, 2017–2020), 277 documents, not the « ~250 of 2018–2020 » of the mandate | The folder contains 2017 (downloaded on request the same day); all pre-ChatGPT; year is a stratification variable. |
| A-003 | 2026-11-03 | Global install of the Claude Code commands/skill/agents under `~/.claude/` (prefix `jof-`) | Explicit user request « accessible à toutes les sessions Claude Code ». |

## Phase 0 — Host audit (2026-11-03 17:30)
- macOS 27.0 (Darwin 27.0.0), Apple Silicon laptop; Claude Code **2.1.317**; Python 3.14 system, 3.11/3.12 via uv 0.11.7; PyMuPDF 1.27 available; `pdftotext` (poppler) present; GROBID not installed, Docker daemon not running → **PyMuPDF chosen** (local, no server); LaTeX: TeX Live (latexmk/pdflatex/biber).
- Global `~/.claude/commands` already holds `cluster-*` commands (untouched). `~/.claude/settings.json` has an auto-approve PreToolUse hook (not modified).
- Corpus folder `~/Desktop/Journal-of-Finance-2018-2020`: 277 PDF + `index.csv` + `_outils/` (download scripts). Original PDFs are never written to.

## Phase 1 — Corpus inventory (2026-11-03 17:36) ✅
- `jof250 corpus audit` → `data/manifests/{corpus_manifest.parquet,corpus_manifest.csv,corpus_audit.md,corpus_v1_frozen.json,crossref_cache.json,inventory_ledger.jsonl}`.
- Files scanned 278; **277 included** (2017: 63, 2018: 65, 2019: 74, 2020: 75); excluded 1 (`index.csv`, non-PDF); 0 SHA-256 duplicates; 0 DOI duplicates.
- Types: 268 articles (incl. presidential addresses), 8 replications/corrigenda, 1 retracted article (Rampini–Viswanathan–Vuillemey 2020, kept and flagged).
- **Crossref verification: 277/277** records status 200, container « The Journal of Finance », title similarity ≥ 85 (public API, polite pool).
- Pages: 11 775 total, median 43 per document; 230.6 MB.
- Fixes during the phase: ligature « fi » (U+FB01) in PDF text broke DOI regex → NFKC normalization; presidential address opens with a portrait page → validity checks the first three pages.

## Open blockers
- Human inter-rater audit of annotations and blind human evaluation (Phase 7) require qualified human reviewers: **not available in this session** → protocols and sheets prepared, results reported as model-judge only (see LIMITATIONS.md).

## Phase 2 — Structured extraction (2026-11-03 18:00) ✅
- `jof250 corpus parse` (parser `pymupdf-jof-heuristics-1.3`): 276/276 documents parsed, 0 failures, mean confidence 0.951; 3 documents < 0.8 (all corrigenda/notes, expected).
- Medians over the corpus: 43 pages, 10 965 body words, 5 numbered sections, 8 tables, 3 figures, 10 numbered equations, 38 footnotes, 52 references, 101-word abstract.
- Parser QA (`scripts/02_parser_qa.py`, 26-document stratified sample vs `pdftotext` regex): sections exact 85% (26/26 after adjudication of appendix sub-headings), tables within ±1 96%, figures exact 92%, references within 10% 92%, abstract presence 100%; parsed title vs Crossref title ≥ 90 similarity for 98.9% of all 276 documents. Equation counts are approximate (reference regex is noisy). Not a human gold standard (see LIMITATIONS.md).
- Fixes along the way: odd/even page margins for hanging-indent references; rotated (landscape) tables; figure captions whose first span only is bold; figure-internal text excluded from footnotes; DOI lines excluded from references; the AFA preliminary program (labelled ARTICLES in the TOC) excluded from the corpus (276 included).

## Phase 3 — Ontology annotation (started 2026-11-03 17:55)
- Schema v1.0 (`src/jof250/rhetoric.py`), prompt `document_annotation` v1.0, model claude-opus-5-5 effort medium; the API grammar for structured outputs rejected the full schema ("compiled grammar too large") → JSON requested in the prompt + strict local JSON-schema validation; invalid annotations are re-run.
- Pilot on 2 documents: ≈ 7.9k input / 2.3k output tokens, 0.08–0.10 USD per document.