EXPERIMENTS — commands, versions, data snapshots, results, failures (WP36)
Environment: macOS 27.0 (Apple M5 Max, 18 cores, 48 GB), Python 3.12.13 via uv (requirements-lock.txt), R 4.6.1, TeX Live 2026, SQLite 3.54.
| ID | Date | Command | Data snapshot | Result / artefact | Outcome |
|---|---|---|---|---|---|
| E-000 | 2026-11-01 | sqlite3 .backup on M4M64b + drop users + VACUUM + SHA-256 + zstd + scp |
data/raw/autoka-snapshot-20261101.db (1 348 825 088 bytes) |
checksum OK | success |
| E-001 | 2026-11-01 | ad-hoc SQL audit queries (reports/data_access_audit.md) | snapshot 2026-11-01 | audit tables | success |
| E-002 | 2026-11-01 | scripts/01_ingest.py |
snapshot sha256 65c3dad2… | data/interim/*.parquet, manifest (vehicles 273 265, price_log 428 484, sync_log 83 531; sync ts 2026-08-12 12:30:35 → 2026-11-01 06:49:42 UTC) |
success (2.8 s) |
| E-003 | 2026-11-01 | scripts/02_clean.py |
idem | data/processed/listings.parquet, price_events.parquet (entry 270 909, cut 76 261, rise 26 486, minor 27 915), exclusions.csv, sample_construction.csv (S0 273 265 → S1 136 393 → S2 114 767; S2L 128 091); region recovered 13 073; powertrain recovered 84 753; coords from dealer 5 582 |
success (≈4 s) |
| E-004 | 2026-11-01 | scripts/03_features.py |
idem | data/processed/risk_set.parquet 3 470 045 listing-days, 125 606 listings (S2L with coords), 2 377 make-model groups, exits 76 262, first cuts 34 103 |
success (5.5 s) |
| E-005 | 2026-11-01 | risk-set sanity check | idem | duplicate listing rows (max 3 exits per listing_id) → id collision at 7 hex chars; fixed (D-012) and E-003/E-004 rerun | failure → fixed |
| E-006 | 2026-11-01 | scripts/04_hedonic.py |
idem | results/hedonic/: A1–A5 fits (N 136 393; R² 0.923 → 0.977), Wald H1 nonlinear joint χ²(12)=664 p<1e-100, tensor χ²(49)=667, age×powertrain χ²(35)=312; H4 region joint χ²(16)=14.9 p=0.53 (null); grouped-dealer CV RMSE: A1 0.1756, A2 0.1706, A2t 0.1724, A4 0.1614 (coverage 57 %), LightGBM 0.1588 |
success (2 min 38 s) |
| E-007 | 2026-11-01 | D-014 fix: 02_clean + 03_features rerun |
idem | S2 114 767 → 80 856 (start-up stock of otogo/autousagee, 33 224 listings, now left-censored); risk set 3 470 045 listing-days / 125 606 listings (S2L) | success |
| E-008 | 2026-11-01 | scripts/06_hazards.py (v3: no-competitor flag, Cox dealer-clustered SEs) |
idem | B cloglog 318 323 listing-days / 15 692 events, age bins χ²(8)=170; C cloglog 1 663 764 / 43 311, χ²(8)=104; Cox concordance 0.575 (cut) / 0.648 (exit); PH violated for 5 / 9 covariates | success (≈12 min) |
| E-009 | 2026-11-01 | scripts/07_competition.py + 07b |
idem | main 25 km n=1 663 764, k=0 +0.89 pp; placebo null; dealer×weekday k=0 +0.55 pp | success |
| E-010 | 2026-11-01 | scripts/08_prediction.py (agent) |
idem | metrics.json; chrono LightGBM log loss 0.405 / 0.585 | success (133 s) |
| E-011 | 2026-11-01 | scripts/09_amli.py 200 (weeks 2–10, D-015) |
idem | PCA 56 %; held-out slope 0.97 pp [0.69, 1.24]; 200 dealer-bootstrap reps | success (≈10 min) |
| E-012 | 2026-11-01 | scripts/10_validation.py |
idem | exit_validation.json (94 % within 7 d; 10.2 % relisted) | success |
| E-013 | 2026-11-01 | first hazards run (v1/v2) | idem | v1 KeyError e_day (merge); v2 dropped listing-days without competitors (pricepos NaN) → B n_listings = n_events; both fixed in v3 | failures → fixed |
| E-014 | 2026-11-01 | first AMLI run (weeks 0–10) | idem | week-0 province components were start-up outliers → D-015 | superseded |
| E-015 | 2026-11-01 | paper/: latexmk |
— | main.pdf 45 pp.; bib $ escape fixed in busse2006cashback |
success |
| E-016 | 2026-11-01 | robustness agent audit of Model B | idem | BUG: risk_set.first_cut was NULL (not 0) for never-cut listings → hazards.cloglog (dropna) and the first-cut event study were fitted on listings that eventually get a cut only (n_listings = n_events = 15 692). Fixed in features.py (fill_null 0) + guards in hazards.py/competition.py; regression test added; full pipeline rerun (E-017). Cox B, cut-size OLS, empirical hazards, Model C, AMLI and prediction were not affected. |
failure → fixed |
| E-017 | 2026-11-01 | make data && make models && make tables && make figures (final end-to-end rerun after E-016) |
snapshot sha256 65c3dad2… | all results/, figures/, paper/tables/ regenerated from one run; key numbers checked against CLAIMS_LEDGER (hedonic, competition, AMLI, validation unchanged; Model B corrected: resid 0.744, pricepos 0.379, lkm +0.116, age bins χ²(8)=157; first-cut event study k=0 +0.73 pp vs base 1.14 %); 14 tests pass + 1 regression test |
success |