# EXPERIMENTS — commands, versions, data snapshots, results, failures (WP36)

Environment: macOS 27.0 (Apple M5 Max, 18 cores, 48 GB), Python 3.12.13 via uv (`requirements-lock.txt`), R 4.6.1, TeX Live 2026, SQLite 3.54.

| ID | Date | Command | Data snapshot | Result / artefact | Outcome |
|---|---|---|---|---|---|
| E-000 | 2026-11-01 | `sqlite3 .backup` on M4M64b + drop users + VACUUM + SHA-256 + zstd + scp | `data/raw/autoka-snapshot-20261101.db` (1 348 825 088 bytes) | checksum OK | success |
| E-001 | 2026-11-01 | ad-hoc SQL audit queries (reports/data_access_audit.md) | snapshot 2026-11-01 | audit tables | success |
| E-002 | 2026-11-01 | `scripts/01_ingest.py` | snapshot sha256 65c3dad2… | `data/interim/*.parquet`, manifest (vehicles 273 265, price_log 428 484, sync_log 83 531; sync ts 2026-08-12 12:30:35 → 2026-11-01 06:49:42 UTC) | success (2.8 s) |
| E-003 | 2026-11-01 | `scripts/02_clean.py` | idem | `data/processed/listings.parquet`, `price_events.parquet` (entry 270 909, cut 76 261, rise 26 486, minor 27 915), `exclusions.csv`, `sample_construction.csv` (S0 273 265 → S1 136 393 → S2 114 767; S2L 128 091); region recovered 13 073; powertrain recovered 84 753; coords from dealer 5 582 | success (≈4 s) |
| E-004 | 2026-11-01 | `scripts/03_features.py` | idem | `data/processed/risk_set.parquet` 3 470 045 listing-days, 125 606 listings (S2L with coords), 2 377 make-model groups, exits 76 262, first cuts 34 103 | success (5.5 s) |
| E-005 | 2026-11-01 | risk-set sanity check | idem | duplicate listing rows (max 3 exits per listing_id) → id collision at 7 hex chars; fixed (D-012) and E-003/E-004 rerun | failure → fixed |
| E-006 | 2026-11-01 | `scripts/04_hedonic.py` | idem | `results/hedonic/`: A1–A5 fits (N 136 393; R² 0.923 → 0.977), Wald H1 nonlinear joint χ²(12)=664 p<1e-100, tensor χ²(49)=667, age×powertrain χ²(35)=312; H4 region joint χ²(16)=14.9 **p=0.53** (null); grouped-dealer CV RMSE: A1 0.1756, A2 0.1706, A2t 0.1724, A4 0.1614 (coverage 57 %), LightGBM 0.1588 | success (2 min 38 s) |
| E-007 | 2026-11-01 | D-014 fix: `02_clean` + `03_features` rerun | idem | S2 114 767 → **80 856** (start-up stock of otogo/autousagee, 33 224 listings, now left-censored); risk set 3 470 045 listing-days / 125 606 listings (S2L) | success |
| E-008 | 2026-11-01 | `scripts/06_hazards.py` (v3: no-competitor flag, Cox dealer-clustered SEs) | idem | B cloglog 318 323 listing-days / 15 692 events, age bins χ²(8)=170; C cloglog 1 663 764 / 43 311, χ²(8)=104; Cox concordance 0.575 (cut) / 0.648 (exit); PH violated for 5 / 9 covariates | success (≈12 min) |
| E-009 | 2026-11-01 | `scripts/07_competition.py` + `07b` | idem | main 25 km n=1 663 764, k=0 +0.89 pp; placebo null; dealer×weekday k=0 +0.55 pp | success |
| E-010 | 2026-11-01 | `scripts/08_prediction.py` (agent) | idem | metrics.json; chrono LightGBM log loss 0.405 / 0.585 | success (133 s) |
| E-011 | 2026-11-01 | `scripts/09_amli.py 200` (weeks 2–10, D-015) | idem | PCA 56 %; held-out slope 0.97 pp [0.69, 1.24]; 200 dealer-bootstrap reps | success (≈10 min) |
| E-012 | 2026-11-01 | `scripts/10_validation.py` | idem | exit_validation.json (94 % within 7 d; 10.2 % relisted) | success |
| E-013 | 2026-11-01 | first hazards run (v1/v2) | idem | v1 KeyError e_day (merge); v2 dropped listing-days without competitors (pricepos NaN) → B n_listings = n_events; both fixed in v3 | failures → fixed |
| E-014 | 2026-11-01 | first AMLI run (weeks 0–10) | idem | week-0 province components were start-up outliers → D-015 | superseded |
| E-015 | 2026-11-01 | `paper/`: latexmk | — | main.pdf 45 pp.; bib `$` escape fixed in busse2006cashback | success |
| E-016 | 2026-11-01 | robustness agent audit of Model B | idem | **BUG**: `risk_set.first_cut` was NULL (not 0) for never-cut listings → `hazards.cloglog` (dropna) and the first-cut event study were fitted on listings that eventually get a cut only (n_listings = n_events = 15 692). Fixed in `features.py` (fill_null 0) + guards in `hazards.py`/`competition.py`; regression test added; full pipeline rerun (E-017). Cox B, cut-size OLS, empirical hazards, Model C, AMLI and prediction were not affected. | failure → fixed |
| E-017 | 2026-11-01 | `make data && make models && make tables && make figures` (final end-to-end rerun after E-016) | snapshot sha256 65c3dad2… | all `results/`, `figures/`, `paper/tables/` regenerated from one run; key numbers checked against CLAIMS_LEDGER (hedonic, competition, AMLI, validation unchanged; Model B corrected: resid 0.744, pricepos 0.379, lkm +0.116, age bins χ²(8)=157; first-cut event study k=0 +0.73 pp vs base 1.14 %); 14 tests pass + 1 regression test | success |