SPB Git

spb/wp29_uqo

Public
2.0 KB · 10 lines json
1{2  "verdict": "accept",3  "issues": [4    {"id": "R1", "severity": "minor", "location": "out/claims.json C5", "issue": "The no-look-ahead claim rests on a corruption test recorded in out/state/checks.md, which analysis.py does not reproduce. Reading the code does confirm the claim: training rows satisfy s+1<=t, and mu, sd and ybar are recomputed from those rows at every origin. The cited evidence, though, sits outside the reproducible pipeline.", "suggested_fix": "Cite the code-level guarantee (train_idx filter and per-origin statistics), or move the leakage test into a reproducible script."},5    {"id": "R2", "severity": "minor", "location": "out/report.md Limitations", "issue": "The report calls 2021-10…2026-09 a 'relatively calm post-COVID regime' without giving any statistic to support it.", "suggested_fix": "Report the evaluation-period SD of ip_growth, or soften the wording."},6    {"id": "R3", "severity": "minor", "location": "out/report.md Uncertainty", "issue": "The DM tests use a plain variance with no HAC term, which is acceptable for h=1. However, no interval is given for the relative improvements, which are the task's main estimands. This omission is acknowledged.", "suggested_fix": "Optionally add a block-bootstrap CI for 1 - RMSE_ridge/RMSE_baseline."}7  ],8  "summary": "All required keys are present. A scratch re-run of analysis.py reproduces results.json and forecasts.csv byte for byte, and an independent re-implementation of all three forecasts gives identical RMSE/MAE (naive 0.883/0.699, mean 0.544/0.429, ridge 0.561/0.451) with 413 training rows at the last origin. The code follows the information rules: per-origin expanding standardisation with ddof=1, an unpenalised intercept ybar, lambda=50, and labels dated <=t. Counts (417 rows, 180 forecast rows, 354/413 training rows), the exclusion ledger, the SHA-256 and the README discrepancy all check out. Claims are appropriately hedged: ridge beats naive, and ridge is not significantly different from the expanding mean (it is nominally worse). Only minor presentation issues remain."9}10
Files · main
Menu

Theme