TeX 53.6%
Python 46.2%
1# STATUS.md — Stages A–F2 complete; IRFA submission package assembled23**Date:** 2026-07-1345## Completed work671. **Project scaffold** per protocol §2: package (`src/intraday_fragility/`), configs,8 scripts, tests (7 passing), Makefile, `uv.lock`, docs, LaTeX tree.92. **Dataset probe (Stage A):** `mito0o852/OHLCV-1m` pinned at revision10 `776328445b7ac6e7815ef3a483e9c8ded1eb6d56`; inventory of 411 monthly Parquet11 files (87.7 GB), 1992-01→2026-03, **no missing months**12 (`data/manifests/repo_inventory.json`).133. **Pilot download** (1992-06, 2009-06, 2025-12; sha256 in14 `data/manifests/pilot_downloads.json`) and **pilot audit**15 (`scripts/01_pilot_audit.py`, JSON reports in `outputs/diagnostics/`,16 logs in `logs/audit/`).174. **Docs:** `DATA_AUDIT.md`, `LIMITATIONS.md`, `DECISION_LOG.md`,18 `DATA_DICTIONARY.md`, `COMPUTE_LOG.md`, `EMPIRICAL_DESIGN.md`,19 `REPRODUCIBILITY.md`, `LITERATURE_MAP.md` (candidates, none verified yet).205. **Pilot figures** (coverage, intraday seasonality; PDF+PNG; source data in21 `outputs/figures/data/`) and **summary-statistics LaTeX table**22 (`outputs/tables/tabP1_pilot_summary.{tex,csv}`).236. **Compiling LaTeX skeletons:** `paper/main.pdf` and24 `appendix/internet_appendix.pdf` build cleanly with the Data section drafted25 from verified audit facts only.267. **Stage B COMPLETE** (user instruction 2026-07-13: proceed with all data):27 all 411 monthly files downloaded (82 GB on disk), each verified against its28 LFS sha256 at download time; manifest `data/manifests/raw_files.{json,parquet,csv}`.29 Post-hoc full re-verification + per-file audit (row counts, duplicates,30 timestamp bounds) running (`verify-download` + `audit-raw`).318. **Literature review complete and massive:** 110 references verified at DOI32 level via OpenAlex/Crossref (two passes, 9 verification agents);33 `paper/sections/02_literature.tex` fully written (10 subsections organized34 around the contribution); documented novelty scan 2018–2026 supports35 positioning; manuscript compiles at 20 pages, 111 bib entries, 0 unresolved36 citations. See `docs/LITERATURE_MAP.md`.37389. **Stage C COMPLETE** (full-sample raw audit, 7.69B rows): perfect XNYS calendar39 match (0 missing/0 extra sessions, 0 weekend rows); DST confirmed clean (flat40 RTH share across calendar months); 100 OHLC violations total (1.3 per 10^8,41 concentrated 1993–95); 10,128 duplicate keys; 0 zero/negative volume. Daily42 ticker-day panel built (`data/metadata/daily/`, 411 files). 442k split-candidate43 ticker-days flagged (over-inclusive by design). 81,306 tickers, 221,957 spells44 (36% of tickers multi-spell → spell = identity unit). Coverage regime change in45 2020 (+10,965 entries, OTC expansion) identified as mandatory robustness axis.46 See DATA_AUDIT.md Stage C section, outputs/diagnostics/stage_c/.474810. **Stage D COMPLETE** (clean panel + universes + splits):49 * 6,944 high-confidence split events (4,819 fwd / 2,125 rev), ±1-day exclusion50 windows (20,662 ticker-days) — `outputs/diagnostics/stage_d/`;51 * monthly universes with strictly lagged screens (top500/1000/3000 by prior-month52 dollar volume; price/density screens; non-common ticker heuristic) —53 `data/metadata/universe_monthly.parquet` (4.44M rows);54 * clean minute panel `data/processed/minute/year=/month=/` (411 files, ~62 GB zstd):55 raw row total matches manifest EXACTLY (7,686,275,524); 10,128 duplicate rows56 merged per logged rule; sessions labelled with true XNYS closes (early-close57 truncation verified on 2025-12-24); flag_ohlc=100, split-window rows=4.30M;58 dedup verified (2020-03 spot check). RTH = 96.7% of rows.596011. **Stage E COMPLETE** (descriptive evidence): 9 JF-style figures (E1–E9, PDF+PNG,61 all with source CSVs in outputs/figures/data/) + Table I (sample construction by62 decade) + Table II (summary stats by decade, top-1000). Aggregates in63 outputs/diagnostics/stage_e/ (411 monthly minute-level aggregate files on the64 lagged top-1000 universe + Epps curves for 1996/2005/2015/2024).65 Headline descriptive facts (verified from saved outputs):66 * Extreme one-minute returns (|r|>1%) fell ~100× in frequency from the 1990s to67 the 2010s on the top-1000 universe, with sharp spikes in 2008 and 2020 —68 the paper's central tension in one figure (figE5).69 * The Epps effect has essentially disappeared: mean pairwise 1-minute correlation70 was ≈0.00 in 1996 (vs 0.16 at 30-min) but ≈0.21–0.30 in 2015/2024 with a nearly71 flat curve — cross-stock information incorporation is now sub-minute (figE8).72 * Intraday |return| and range shifted down decade by decade; volume concentrates73 increasingly at the close in recent decades (figE3).74 * CAVEAT for interpretation: pre-2001 minute returns embed 1/8–1/16 tick bounce;75 the 1990s "extreme" frequency partly reflects mechanical bid-ask bounce.76 Robustness planned: midpoint proxy, coarser sampling, range-based measures.777812. **Stage F baseline COMPLETE**: 1,854,384 real shock events (|z|≥6, isolated) +79 1,766,058 matched placebos, 410 months, lagged top-500 universe80 (outputs/estimates/events/, 410 files). Baseline result (Table IV, figs F1–F3,81 registry stage_f_baseline.json): **average abnormal propagation ≈ 0** —82 ΔΔbreadth between −1 and −3 ×10⁻⁴ across h∈{1..120} with day-block bootstrap83 SEs ≈ 1×10⁻⁵…1.5×10⁻⁴; no decade positive; weak insignificant positive gradient84 for |z|>12. Honest interpretation: isolated idiosyncratic shocks are absorbed.85 Falsification good news: pre-shock outcomes ≈ post placebo levels (no leakage).86 Next (protocol-specified): cumulative breadth through h, network-weighted87 receivers, and the common-shock (simultaneous-extreme) sample as separate object.888913. **Stage F2 COMPLETE — the paper's headline mechanism found**: neighbor−non-neighbor90 ΔΔ breadth = +64.0 (SE 1.9) ×10⁻⁴ at h=5 (t≈34), decaying to +49.6 at h=30;91 **tripled across eras** (+28 in 1992–99 / +30 in 2000–09 → +81 in 2010s / +90 in92 2020s). Market-wide cumulative remains null. Co-jump measure corrected to93 severe-synchronization (>10% of names): crisis spikes 2008/2020; severity 9.3%94 (2008) / 6.4% (2020) vs 3.5% calm. Placebo-network falsification passes (−2.7±2.8).9514. **Robustness battery (Table VIII)**: null + network channel stable across96 post-decimalization, mid-day, ex-announcement windows, calm/crisis, |z|≥8/12,97 ex-non-common, month-cluster bootstrap. Network diff declines with magnitude98 (+64→+34) — documented.9915. **IRFA submission package** (outputs/release/): manuscript_irfa.pdf (elsarticle,100 38 pp, 0 unresolved refs, elsarticle-harv bib), highlights.pdf, cover_letter.pdf,101 online_appendix.pdf, submission_checklist.md. All numbers macro-generated102 (48 macros). Provenance framed as Finnhub.io-sourced aggregation (user directive).103104## Key findings (verified)105106- Schema uniform: `timestamp` (ns, UTC, bar-start), OHLC (USD, **unadjusted**),107 `volume` (shares, integer-valued), `ticker`. No nulls.108- Timestamps are true UTC; DST handled correctly (09:30 ET session start maps to109 13:30 UTC in June, 14:30 UTC in December).110- Zero OHLC violations in 58.8M pilot bars; 14 benign duplicate pairs (OTC only).111- **No zero-volume bars: missing minute = no trade.** Trade-conditional panel.112- Liquid names complete (390 bars/day); sparse tail is huge in 2025 (23k tickers,113 median 5.5 bars/day — OTC expansion).114- Delisted securities present (only 22–26% of 1992 tickers persist).115- Early-close days handled by data; sessions must be truncated by exchange calendar.116- Unscreened minute-return SD is outlier-dominated (splits/bad prints) ⇒117 corporate-action screens are prerequisites for variance-based measures.118119## Quality control120121- `uv run pytest`: 7/7 pass.122- Both LaTeX documents compile (TeX Live 2026).123- Figures validated for CVD safety (palette validator) + grayscale line styles.124125## Unresolved issues / decisions requiring attention1261271. **Data license unspecified** on the HF card (Finnhub-sourced). Research use with128 citation documented; public redistribution of derived *data* deferred until129 clarified. (DATA_AUDIT §1)1302. Venue scope of bars (consolidated vs single venue) unknown — Stage C probe planned.1313. Security-type classification requires an external reference or suffix heuristics.1324. Literature map has candidates only; verification pass (DOI-level) still to do.133134## Next commands135136```bash137# when download finishes:138PYTHONPATH=src uv run python -m intraday_fragility.cli verify-download139PYTHONPATH=src uv run python -m intraday_fragility.cli audit-raw # fills manifest row counts140# then Stage C full raw audit scripts (04–07)141```142