SPB Git

Repository Public Research preview

DESIGN_RESPONSE.md 12.5 KB 121 lines markdown

1# Response to the design review (`docs/DESIGN_REVIEW.md`, 2026-11-09)23Written 2026-11-09 after the pipeline was run. Each item names the script and the metric file that implements the answer.4Numbers quoted are read from `results/metrics/*.json` as of this date. "Not done" items are listed at the end with reasons.567> **Post-revision note (2026-11-09, evening).** This response describes the state after the design review and the first pipeline8> run. The first complete draft was then reviewed by three referee-style agents, which led to a rebuild of the return panel from9> individual contracts, the cumulated-intraday and implied-yield fidelity, the removal of the short rate from the headline results10> and the dynamic analyses. Numbers quoted below (e.g. futures OOS R² 0.920 / 0.785 / 0.686; ETF correlations) are those of the11> earlier run; current values are in `docs/RESPONSE_TO_REFEREES.md` and `results/metrics/`. Threat 2's "telescoping" concern is now12> addressed directly by `fidelity_cumulated_intraday.json` (0.93 / 0.80 / 0.75 over all 3 963 days); the 15:00/16:00 timing13> sensitivity listed as "not run" in item 5 below was run in `04d` (level fidelity falls to 0.90 at either boundary).1415## Important coverage fact discovered while documenting (affects several answers)1617The quarterly weight estimation (`scripts/04_mimicking.py`) keeps a contract only if it has ≥ 100 minute returns on ≥ 85 % of the (rule relaxed on 2026-11-09 after this issue was found; the paragraph below describes the state BEFORE the fix)18training days. Read from `mimicking_weights_quarterly.json`, the contract sets actually used are: **ZF, ZN, US from 2011Q1 to192024Q1; ZT, ZF, ZN, US from 2024Q2** (ZT then reaches 97.5 % coverage); **TN and UB never enter** (TN has ≥ 300 minutes on2087–88 % of days, UB on 92–94 %, ZT on 88–94 % before 2024). The six-contract description in `CLAUDE.md` and `DESIGN.md` §2–§321therefore describes the data extracted, not the weights estimated. The six-root in-sample regression in `mimicking_fidelity.json`22(`insample_2016_2026_six_roots`, 2 185 days) is the only place where all six contracts are used together. This must be stated in23the paper, or the 95 % rule relaxed and the pipeline re-run. **Resolution (2026-11-09):** the rule was relaxed to ≥ 100 minutes on ≥ 85 % of days and scripts 04 → 11 were re-run; the contract sets are now ZT ZF ZN US (2011–2012), + UB (2013–2018), + TN (2019–). All numbers in the paper come from the re-run.2425## 1. The three threats2627### Threat 1 — the daily target is not spanned by the futures; H1 slope threshold infeasible2829Done.30- Cash-yield ceiling (review's "decisive diagnostic", adapted): `04c_targets_and_spanning.py` → `targets_and_spanning.json`,31  key `ceiling_cash_2_5_10_30_for_cmt`. Same rolling-quarterly OOS protocol as the futures, with the daily changes of the cash32  2/5/10/30y CMT yields as regressors: OOS R² full sample 0.986 (level), 0.845 (slope), 0.844 (curvature). The futures reach33  0.920 / 0.785 / 0.686 (`mimicking_fidelity.json`, `full_2011_2026`). The gap attributable to futures (basis, CTD, timing,34  missing contracts) is therefore about 0.07 / 0.06 / 0.16 of R²; the rest of the slope and curvature shortfall is the bill segment.35- Alternative targets: `04b_fidelity_gsw_and_sofr.py` → `mimicking_fidelity_robustness.json`: GSW 1–30y zero-curve factors (OOS36  R² 0.910 / 0.743 / 0.777) and CMT factors from maturities ≥ 1y (0.922 / 0.659 / 0.736). `04c` also tests the review's37  preferred CMT 2–30y target (`cmt_2y_plus`): it is ill-conditioned — daily-change SDs 5.8 / 15.5 / 28.0 bp, corr(ΔS, ΔC) = −0.75,38  fit RMSE 6.4 bp, OOS slope R² 0.533 (0.370 in 2022–2026) — and was **not adopted**. The pre-registered 11-maturity CMT target39  remains primary; the alternatives are reported as robustness.40- Short-rate instrument: `04b` adds the front 3-month SOFR future (SR3, extracted by `01b_extract_sr3.py`) from 2021Q141  (`cmt_with_sr3`): slope OOS R² 0.734 full (0.713 in 2022–2026) against 0.684 without SR3 in 2022–2026. Decided after seeing H1.42- Short rate y(0) = L + S is reported as a separate series (`dSR` in `07_events.py`) alongside implied 3m/2y/5y/10y/30y changes.43- H1 thresholds were **kept as frozen** and reported as passed/failed: level passes (≥ 0.90 in every sub-period), slope passes only44  in 2011–2015 (0.907), curvature passes (≥ 0.60) in 2011–2015 and 2022–2026 only. The §7 re-scoping trigger (< 0.6 for level or45  slope) is not reached. Thresholds relative to the ceiling (review item 2) were not re-written ex post; the ceiling is shown next46  to the estimates instead.4748### Threat 2 — H1 is a daily test; intraday validation and OOS coverage holes4950Done.51- Leave-one-contract-out spanning at minute frequency: `04c` → `targets_and_spanning.json`, key52  `leave_one_out_spanning_2019_2026`. Implementation differs from the review's proposal: a 3-component PCA of the other five53  contracts' returns predicts the held-out contract (in-sample OLS), at 1/5/30-minute sampling, 08:00–16:00 ET and all hours.54  Example (1 min, 08–16 ET): ZT 0.54, ZF 0.71, ZN 0.69, TN 0.80, US 0.72, UB 0.70; first three PCs explain 96.8 % of the six-contract55  variance (98.8 % at 5 min, 99.3 % at 30 min).56- Independent instrument: `06b_etf_crossval.py` → `etf_crossval.json`. Treasury ETFs SHY/IEI/IEF/TLH/TLT, 09:30–16:00 ET, with a57  construction that uses no daily regression (duration-based implied yield changes at five maturity points, NS OLS per minute).58  Correlation with the futures factors at 1/5/30 min: level 0.55 / 0.74 / 0.86; slope 0.10 / 0.29 / 0.49; curvature 0.15 / 0.32 /59  0.51; over 126 FOMC windows [−5, +25]: 0.86 / 0.73 / 0.85; regular-hours daily sums: 0.93 / 0.64 / 0.66. ETF slope and curvature60  are noisy (SD 4–5× the futures series), so the agreement is a lower bound.61- Coverage facts corrected in the documentation: four contracts in 2008–2009 (ZT, ZF, ZN, US), UB from 2010-01, TN from 2016-01;62  first OOS quarter 2011Q1; and, as stated above, the weights actually estimated use three contracts until 2024Q1 and four after.63- The telescoping accounting table (share of daily variance in dropped segments) was not produced; `returns_diagnostics.json`64  gives the number of segment breaks per contract (ZT 132 218, ZF 52 946, ZN 37 772, TN 40 062, US 43 538, UB 72 837 minutes).6566Not done: the external FOMC surprise benchmark (Bauer–Swanson / Swanson data files) — not downloaded; the paper keeps descriptive67language for event windows (review §2.12).6869### Threat 3 — microstructure noise7071Done.72- Noise-implied correlations: `06c_noise.py` → `noise_benchmark.json`. With independent rounding noise of variance 2·tick²/1273  per contract (current CME tick sizes) and the estimated quarterly weights: implied corr(L, S) = −0.945, corr(L, C) = −0.811,74  corr(S, C) = +0.669; implied noise SD 0.124 / 0.118 / 0.421 bp per minute. The observed 1-minute realized corr(L, S) is −0.9775  (`realized_correlations.json`), i.e. indistinguishable from the noise benchmark at 1 minute; at 30 minutes it is −0.99 and the76  daily CMT value is −0.89, so the sign is not a noise artifact but the 1-minute magnitude is uninformative.77- Volatility signature: `realized_correlations.json`, `rv_mean_bp2` — mean daily RV at 1/5/30 min: level 49.2 / 36.2 / 32.0 bp²,78  slope 49.8 / 37.8 / 33.8, curvature 404 / 227 / 175. Noise inflates 1-minute RV by roughly 50 % (level, slope) to 130 % (curvature).79- 5-minute sampling is the main frequency for H4 (reported alongside 1 and 30) and for the HAR target in H5 (`08_har.py` uses80  `RV_X_5m`). 1-minute is kept as robustness.81- Missing-minute rule made explicit (`06_intraday_factors.py`): a contract is "active" if it traded within the previous 1582  minutes; an active contract with no bar carries a zero return; an inactive one is NaN and the minute is dropped from the83  cross-section. Minutes with any required contract inactive are excluded.8485Not done: noise-robust multivariate estimators (realized kernel, pre-averaging, refresh-time), robust periodicity estimators86(WSD/FFF), Roll-type noise covariance from autocovariances, historical tick-size changes.8788## 2. Econometric issues (§2 of the review)8990| # | Issue | Response |91|---|---|---|92| 1 | CMT vs GSW; λ̂ | Both targets run (`04b`); CMT primary. λ̂ = 0.49 reported in `daily_factors_summary.json` as robustness only; λ = 0.7308 fixed everywhere else, so no look-ahead. |93| 2 | Level/slope separation | Short rate L + S, and implied yields at 3m/2y/5y/10y/30y, reported in `07_events.py` (`event_summary.json`, `var_dy_bp2`); the factor-unit variance mix (`var_mix_LSC`) is shown but is scale-dependent. Condition numbers and weight confidence bands not computed. |94| 3 | Bills stretch the fit | CMT ≥ 1y and ≥ 2y targets tested (`04b`, `04c`); not adopted (see Threat 1). |95| 4 | Basis, CTD, roll | Roll days identified by comparing the raw and ratio-adjusted daily continuous series (`returns.py: roll_days`; 67–75 roll days per contract) and the first return of the session on a roll day is dropped. The CTD-duration structural inversion was not promoted to co-primary. March 2020 and 2023 bill episodes are not excluded. |96| 5 | Timing | Yield day fixed at 15:30 → 15:30 ET (`returns.py: yield_day_bounds`). 15:00/16:00 sensitivity not run. Holidays: minutes fall into the next FRED yield day by construction. DST handled by tz-aware timestamps. |97| 6 | Bars open-to-open | Returns are close-to-close of consecutive bars (`minute_returns`); not previous-tick on a fixed grid. |98| 7 | Stale Asian prices | Active-contract rule (15 min); periodicity profile is the mean squared change by minute of day on all / event / non-event days (`intraday_periodicity.csv`), not a robust estimator. |99| 8 | OOS protocol | Frozen and implemented: quarterly, 3-year trailing window, ≥ 500 days, ≥ 85 % presence (relaxed from 95 %), no intercept at minute level; OOS R² = 1 − SSE/Σ(Δβ − mean)². Quarterly R² series available from `mimicking_oos_daily.parquet`. |100| 9 | Multiple testing | Holm correction applied to the Diebold–Mariano family in the paper tables (laptop table script); not applied to H1–H4 descriptive comparisons. |101| 10 | Variance-share inference | Event-level i.i.d. bootstrap (2 000 draws) replaces daily blocks; placebo windows at the same clock time on up to three matched non-event days (same weekday, ± 200 days) give the baseline and SD ratios with bootstrap CIs; mean and median shares reported; extra windows [−5, +15] and [−5, +90] (press conference) added. Unscheduled FOMC actions flagged (`events.py: UNSCHEDULED`) and excluded from the scheduled statistics. Statement times by rule (14:15 before 2011-04; 12:30 on press-conference days 2011–2012; 14:00 from 2013). Overlapping 08:30 releases not dropped. |102| 11 | H5 | Benchmarks as pre-registered (GARCH(1,1) on daily CMT changes, 22-day rolling variance); target is 5-minute RV only; DM with HAC (bandwidth 1.3 T^{1/2}, ≥ h−1 lags). Squared daily CMT change as second target, rotated-covariance HAR, single-contract HAR, Giacomini–White and MCS not implemented. |103| 12 | Language | Event windows described as variance realized at release times, not causal effects. |104105## 3. Positioning (§3 of the review)106107Adopted: the paper is framed as a measurement paper — spanning bounds by regime (ceiling vs futures), a validated 23-hour108factor series, and a variance accounting of the curve by clock time and event. Forecasting (H5) is kept as a pre-registered109hypothesis but presented after the measurement results. The NS-to-Swanson mapping was not done (no surprise series downloaded).110111## 4. Ranked changes (§4 of the review) — status1121131 not adopted (2y+ target ill-conditioned; GSW and ≥ 1y as robustness) · 2 partly (ceiling reported, thresholds unchanged) ·1143 done except the FOMC-surprise correlation · 4 partly (y(0) reported; bill/coupon exact decomposition not computed; SR3 added) ·1155 partly (5-min main; no noise-robust estimator) · 6 done (noise benchmark next to H4) · 7 done (explicit active rule) ·1168 not done · 9 done in documentation · 10 partly (DM/HAC; no second target, no GW/MCS) · 11 done (placebo baseline; ordering117test by paired bootstrap not done) · 12 done · 13 partly (rule-based times, [−5,+15]/[−5,+90]; not [−10,+20]/[−15,+105]) ·11814 partly (Holm on DM family) · 15 not done · 16 partly (roll days detected and first session return dropped; splice rule vs119first notice not audited) · 17 partly (15:30 fixed; no sensitivity) · 18 done · 19 adopted · 20 not done · 21 kept · 22 done in120the paper text (slope = short − long; the JSON key name is unchanged).121
Files main
Menu

Theme