# PROGRESS.md — WP39 wp39_uqo

# Phase 0 — Discovery and feasibility (2026-11-02) — GATE PASSED

• Inspected the UQO Research workspace (Recherche-UQO/, 39 papers, harmonisation standard); no wp39 existed; created finance/invisible-balance-sheet/ from the template; git initialised on a dedicated repository (no other paper touched).
• Probed FinData: /health ok, 64 datasets, OpenAPI 3.1 (45 routes), bars schema v1, fundamentals (67 metrics, 5,451 entities, 2012→2026), earnings events (Postgres), coverage endpoints, export throughput (≈ 1 s per symbol-decade; 12-year cap per request). Created least-privilege key wp39_uqo.
• Verified external public sources: SEC company_tickers.json + submissions (CIK/SIC), Kenneth French daily factors.
• Verified 94 bibliography entries on Crossref; wrote docs/NOVELTY_MEMO.md, literature/RELATED_WORK_MATRIX.csv.
• Outputs: DATA_AUDIT.md (viability = YES with survivorship + PIT caveats), configs/*.yaml, docs/API_INTEGRATION.md, docs/COMPUTE_BUDGET.md, docs/PIT_PROTOCOL.md, docs/RESEARCH_DESIGN.md (frozen), DECISIONS.md D-001…D-010.
• Owner instructions received during the session: (1) the fundamentals API must be used — it is the accounting source; (2) minimise AI usage — no LLM features, no neural models (D-006).
• Blockers: none. Next safe step: Phase 1 gate memo, then code + tests (Phase 2).

# Phase 1 — Research design and novelty — GATE (self-approved under the autonomous mandate, D-010)

• docs/RESEARCH_DESIGN.md frozen 2026-11-02 14:40 ET; hypotheses H1–H5 with null/alternative, primary endpoint (H1, h = 20, log loss, binary adverse operating-income surprise), decision rules, splits, inference, power plan.
• Owner review pending; nothing in Phases 2–5 depends on test-period data until the Phase 3 gate.

# Phase 2 — Data acquisition and verification — IN PROGRESS

• 2026-11-02 13:31 — 01a_extract_node_tables.sh: entities 5,451 · earnings_events 364,783 · corporate_actions 249,340 (raw format firstrate:stock:<SYM> / <SYM>_DIVS, loader adapted). 00_probe: health ok, 67 datasets, schema v1 ok, no earnings route.
• 13:33–13:50 — 01_audit_universe: 5,451 entities → 5,018 with ≥ 8 quarters → 4,568 eligible (1m coverage ≥ 250 sessions); CIK matched 5,073, SIC 5,029 (SEC submissions fetched with a 6-thread, 8 req/s limiter after a sequential first attempt at ~1 req/s was restarted); 2,522 firms have former names (ticker-reuse audit input). results/universe_audit.json.
• 13:50–14:01 — 02_build_pit_panel: first run hit the API rate limit (HTTP 429, 600 req/min) with 6 unthrottled threads → client now has a process-wide token bucket (WP39_RATE, default 6 req/s) and honours Retry-After; rerun succeeded: 13,052,739 facts (all revision 1), 194,817 firm-quarters, release match ok 172,328 (88.5 %), lag > 30 d 15,242, no release 6,020, filing before release 1,227; release timing after_close 76,752 / before_open 66,719 / unknown 51,346; base rates: adverse op-inc 31.2 %, adverse NI 32.7 %, ROA decline 29.4 %, OCF negative 25.9 %, liquidity 47.7 %, |S| ≥ 0.5 % 61.4 %; S_opinc quartiles −0.87 % / 0.00 % / +0.82 % (heavy tails, winsorised in continuous models). Design freeze hash written to artifacts/manifests/design_freeze.json before any model fit.
• 14:01 — 03_build_features --workers 8 (WP39_RATE=1 per process) running: streaming export → features → windows.
• 14:01–14:59 — 03_build_features: 4,568/4,568 symbols ok, 0 errors (58 min, 8 workers, ≈1.3 symbols/s); raw exports deleted after use; ~/wp39-cache/features 5.1 GB + windows 5.9 GB. 04_build_labels (28 s): event sample 481,159 observations (3,796 firms, 102,522 firm-quarters; h = 20: 97,788, base rate 30.1 %), calendar sample 258,480 firm-months; cascade in results/table_sample_cascade_event.csv. Phase 2 gate passed: zero known forward-looking features (tests + live as-of equivalence), coverage gaps explained (2013 start = 8 prior quarters required).

# Phase 3 — Descriptives and baselines (2026-11-02 15:00–15:01) — GATE PASSED

• 05_train_baselines (30 s): B0–B5 logistic (C tuned on validation only), ridge continuous target, firm hold-out, walk-forward OOF 2016–2026 (86,791 rows per set). Validation log loss h=20: B1 0.5938 → B2 0.5691 → B3 0.5695 / B4 0.5688 / B5 0.5706. Test set opened only after this gate (results in Phase 4/5).

# Phase 4 — Advanced models (15:01–15:30) — GATE PASSED

• 06_train_advanced (29 min): LightGBM, 24-trial equal budget per information set (LB1–LB5, LB5all), secondary targets, horizon ladder with a fixed configuration, firm hold-out, walk-forward OOF (event + calendar). Validation: LB1 0.5447 → LB2 0.5214 → LB3 0.5234 / LB4 0.5239 / LB5 0.5245 / all 0.5259.
• 07_evaluate: first run failed on a dtype mismatch (Int32/Int64 fold), fixed, rerun ok. Test period 2021–2026, 53,667 obs: Δ log loss B5−B2 = +0.0013 [−0.0013, +0.0041] (GW p = 0.52); LB5−LB2 = +0.0016 [+0.0010, +0.0023] (p = 0.0008, a deterioration); LB2−LB1 = −0.0068 [−0.0095, −0.0042]; AUC LB2 0.803, LB5 0.801. H1 and H2 rejected. H4: no horizon gradient (Spearman −0.5, p = 0.39/0.33). Secondary targets, firm hold-out, continuous target: same null. MDE (80 % power, 7 validation quarters): 0.0025.

# Phase 5 — Robustness and interpretation (15:30–15:58)

• 08_run_robustness: first run stuck 28 min in a row-wise pandas loop (CAR window) → vectorised in Polars, rerun in 25 s. Placebo (shuffled labels, 3 draws): AUC 0.50, Δ ≈ −0.0004 (noise floor). Subgroups: no gain anywhere; deterioration smallest for small/illiquid/loss firms. Panel: LFDS_B2 β = −0.0765 (t = −32.2); incremental d_lfds β = −0.0129 (t = −1.76); interactions opposite to H3 (less informative for small and low-leverage firms). H5: CAR on LFDS5 β ≈ 0 (t = 0.01); d_lfds +0.060 (t = 2.9, sign opposite to a deterioration signal); portfolios L/S alpha −0.59 %/mo (t = −1.85) gross, −0.62 % net for LB5; −0.49 % (t = −1.64) for LB2 → no economic value.

# Phase 6 — Manuscript (15:58–16:10)

• 09_make_manuscript: 15 LaTeX tables, 9 figures (validated palette), 206 macros in paper/results_macros.tex (fixes: NaN subgroup in figure, LaTeX escaping of math labels, Unicode in bib). paper/main.pdf compiled with latexmk on the laptop: 33 pages, 0 undefined references/citations.

# Typography and figure pass (2026-11-02 16:30–17:00)

• Paper: newtxtext + newtxmath (Times math with proper metrics), mathtools, bm, dsfont; notation macros (\E, \ind{}, \Info, \Acc, \Loss, \LFDS, \RV, \BPV, \RS, \TA, \dlog); displayed definitions for the surprise and its seasonal expectation (eq. 1), the log loss (eq. 2), the information set (eq. 3), the realized-measure families (eq. 4), the panel regression and the Giacomini–White statistic; author names normalised in the bibliography.
• Figures regenerated in journal style (src/wp39/figures.py): Times/STIX typography, monochrome marks distinguished by shape and line style with a single restrained accent for the 1-minute series, Panel A/B labels, inward ticks, no grid, journal widths (3.25 / 6.5 in), redesigned timeline schematic, ordinal horizon axis. Tables: grouped rules by family.
• paper/main.pdf: 34 pages, 0 undefined references.