# WP47: Referee review of the pre-registered design (before computation)

Reviewer role: econometric referee (JFE/JFEC standard) · Date: 2026-11-09 · Object: docs/DESIGN.md (frozen plan), docs/literature_search.md, src/wp47/ns.py, scripts/02_daily_factors.py, results/metrics/daily_factors_*.{csv,json}. No intraday result exists and none was inspected. Numbers marked [R] were computed for this review from the existing daily-factor files and the FRED/GSW inputs already in data/external/ (OLS on daily data, a few seconds of work). They are diagnostics for the design, not paper results. Everything else is quoted from daily_factors_summary.json or is an order-of-magnitude argument, labelled as such.

Sign convention (state it in DESIGN §3.1): the slope loading equals 1 at τ→0, so β2 ≈ short minus long (mean −1.99 pp). The summary key corr_slope_vs_spread10y3m actually uses 3m−10y. With this convention, a parallel rise with a pinned short end gives ΔL>0 and ΔS<0. That is why the correlation is negative.

# 1. The three most serious threats

# Threat 1: The daily target is largely not spanned by the futures (front end, ZLB collinearity). H1 as written is probably infeasible for slope.

For fixed λ and a fixed maturity set, the NS factors are an exact linear map of the 11 CMT yields: Δβ_t = A Δy_t with A = (X'X)⁻¹X'. Slope and curvature put large weights on 1m/3m/6m/1y. No futures root spans those maturities, and ZT (≈2y) is the shortest. Evidence:

• Cash-yield spanning ceiling [R]. Regress daily CMT factor changes (2008–2026) on the daily changes of the cash 2/5/10/30y CMT yields, i.e. the maturities the futures proxy, measured without basis or CTD error. In-sample R² = 0.983 (level), 0.737 (slope), 0.755 (curvature). The 1m/3m/6m changes alone give curvature R² = 0.31. H1 asks for an out-of-sample slope R² ≥ 0.90 from futures. That target sits well above what even perfect cash yields at those maturities deliver. H1 will probably fail on slope, and §7's re-scoping threshold (0.6) will probably not trigger. The design ends up in a grey zone it never planned for.
• CMT vs GSW. corr(ΔS) = 0.79 and corr(ΔC) = 0.70 (summary). Regressing ΔS_CMT on all three ΔGSW factors gives R² = 0.79 [R]. GSW is fitted to off-the-run coupons with maturities beyond about 1 year. The disagreement is therefore mostly the bill segment, which is exactly what futures cannot see.
• ZLB collinearity [R]. corr(ΔL, ΔS) by regime: 2008–09 −0.59; 2010–15 −0.974; 2016–19 −0.87; 2020–21 −0.91; 2022–26 −0.82. In 2010–15, sd(ΔL+ΔS) = 1.6 bp while sd(ΔL) = 6.4 and sd(ΔS) = 6.8 bp. The instantaneous short rate β1+β2 is pinned, so ΔS ≈ −ΔL. The level and slope mimicking portfolios are then two copies of one portfolio with opposite signs. Their separate identification rests on the residual variance of β1+β2, which is about 6% of the variance of ΔL in the ZLB and is unspanned by construction. The −0.82 full-sample correlation is mostly this mechanical regime effect, not an economic finding (relevant for H4).
• Curvature is driven by bill episodes [R]. The largest |ΔC| days are 2023-05-04 (195 bp), 2023-03-13 (174 bp), 2008-03-18, 2023-04-25, 2023-03-21 and 2008-03-25. These are debt-ceiling, SVB and Bear Stearns bill dislocations. The curvature change SD is 23 bp in 2022–26 against about 10 bp in 2010–21. A futures-based curvature cannot reproduce these days, and squared-error metrics will be dominated by them.

Decisive diagnostic (no new data). Split each daily factor change into a bill component (A columns for 1m–1y) and a coupon component (2–30y). Report each component's variance share and the R² of each component on futures returns, by regime. The coupon share is the attainable ceiling for H1.

Design fix (choose ex ante). (a) Make the primary target an NS fit on the coupon segment the futures span: CMT 2–30y, or GSW zero yields 1–30y. Keep the 11-maturity CMT fit as a secondary target, with its spanning ceiling reported. (b) Report the short rate y(0) = β1+β2 as an explicitly unspanned quantity. Alternatively, if FinData OS carries them, add a short-rate root (SR3 SOFR futures from 2018, GE Eurodollar to 2023, ZQ fed funds) and pre-register its use. (c) Re-state the H1 thresholds relative to the measured ceiling, e.g. OOS R² ≥ 0.9 × ceiling, rather than as absolute numbers.

# Threat 2: H1 does not validate the intraday series; it is a daily test in disguise. OOS coverage also has holes.

Log returns telescope. For each contract, the sum of within-session minute returns from 15:30 on t−1 to 15:30 on t equals the daily return, minus the returns dropped at the maintenance break, at splices and at gaps over 5 minutes. Hence Σ_m Ŵ r_{t,m} ≡ Ŵ r_t minus the dropped segments. The "cumulated intraday" R² in H1 equals the OOS R² of the daily regression plus an accounting error. It says nothing about whether minute-level factor changes are correctly measured: timing, noise, staleness, or the stability of the NS structure within the day. The H1 slope criterion in [0.9, 1.1] is likewise a daily weight-stability check.

OOS coverage problems in DESIGN §2–§3.2:

• UB starts in 2010-01, so K = 4 (ZT, ZF, ZN, US) in 2008–09, not 5.
• With a 3-year (≥500-day) trailing window re-estimated quarterly, the first OOS quarter is about 2010Q1–2011Q1. The 2008–09 "crisis" sub-period, which H1 itself mentions, has no OOS weights.
• K = 6 weights including TN are only OOS from about 2018Q1–2019Q1, not 2016.
• H5 says weights are "re-estimated monthly", which contradicts §3.2 (quarterly).

Decisive diagnostics/fixes.

1. Demote the telescoping check to an accounting table: the share of daily return variance in dropped segments, per contract and year. It should be a few percent. Report H1 as daily spanning.
2. Add a genuine intraday validation, pre-registered:
   • (i) Leave-one-contract-out. Build the factors from 5 roots, convert them to the implied yield change at the held-out root's CTD maturity via the NS loadings, and predict the held-out minute (and 5-minute) return. Report R² by session (Asia, Europe, US) and by event window. This tests the 3-factor NS structure at high frequency.
   • (ii) An independent instrument. Treasury ETFs on FinData (SHY/IEI/IEF/TLH/TLT; SHV/BIL for the front end, if present). Build ETF-based factors the same way and correlate 5- and 30-minute factor changes over 09:30–16:00 ET.
   • (iii) External FOMC benchmark. Correlate the 30-minute FOMC-window level and slope changes with a publicly posted high-frequency surprise series (e.g. Bauer–Swanson 2023 / Swanson 2021 data files), documented in data/external/.
3. State the effective OOS start per K. For 2008–09, either use an expanding window from 2007 (FRED CMT starts in 2007 in the current files) and flag that the window is short, or label 2008–09 results as in-sample.

# Threat 3: Microstructure noise and shared-contract noise mechanically generate H3/H4 patterns and bias H2 denominators.

Order of magnitude (current tick sizes; verify historical tick changes from CME notices). One tick in yield terms is about tick value / DV01:

• ZT: $7.81 on a DV01 of roughly $38
• ZF: $7.81 on roughly $43
• ZN: $15.63 on roughly $65
• TN: $15.63 on roughly $85
• US: $31.25 on roughly $150
• UB: $31.25 on roughly $220

That is ≈ 0.15–0.25 bp of yield per tick. If daily level variance (sd 6.17 bp) were spread evenly over 1 380 minutes, the efficient per-minute SD would be about 0.17 bp, and it is far smaller in the Asian session. Trade-based bars therefore carry bid–ask bounce of the same order as the signal at 1 minute, and larger than the signal overnight.

The mimicking portfolios make this worse:

• Collinear futures with a nearly mirror-image level/slope pair (Threat 1) produce large long–short weights. The noise variance of a factor is w'Ωw, with Ω the diagonal noise covariance, and grows with ‖w‖².
• Level and slope are built from the same contracts, so noise induces a mechanical covariance w_L'Ω w_S. That term is negative if the weights on a shared contract have opposite signs, which is likely here. H4's "negative at 1 minute, less negative at 5/30 minutes" can be produced by noise alone, independent of any Epps effect.
• The 08:30 and 14:00 spikes in H3 are robust, but the "minimum in the Asian session" and the curvature/level concentration ratio are sensitive to how stale minutes are treated. DESIGN §2 is internally inconsistent here. Missing minutes "carry a zero return", yet a minute "enters only if all roots trade". Dropping minutes loses the accumulated return. Carrying zeros and then releasing the accumulated move creates artificial spikes in thinly traded TN and UB overnight.
• H2 denominators (the yield day's RV over about 1 380 one-minute returns) accumulate noise linearly in the number of returns. This biases event-window variance shares downward, most for the factor with the largest ‖w‖, likely curvature or slope. The H2 ordering is therefore partly an artifact.

Decisive diagnostics/fixes.

1. Volatility-signature plots of RV^{(k)} at 1–60 minutes, per factor, regime and session.
2. Estimate the noise covariance from first-order autocovariances (Roll-type), or with a multivariate realized kernel or pre-averaging on the 6×6 futures covariance using refresh-time sampling. Then compute factor (co)variances as Ŵ Σ̂ Ŵ'. This is valid because each factor is a fixed linear combination.
3. Make 5-minute previous-tick, refresh-time sampling the primary frequency, with 1-minute as a robustness check. Use 15-minute sampling in the Asian session for H3.
4. Report the noise-implied correlation w_L'Ω̂ w_S / sqrt(…) next to the H4 estimates. H4 only counts if the noise-robust correlation shows the pattern.

# 2. Econometric issues (point by point)

1. Measurement error in daily factors (CMT vs GSW).

   • Error in the dependent variable does not bias w, but it caps R². Error-in-variables does arise on the regressor side (futures basis, timing; see 5–6).
   • CMT are bid-side, on-the-run par yields interpolated by Treasury. The method changed in Dec 2021 (to monotone convex), a structural break to test in the weights. They are read "near 3:30 pm".
   • GSW are off-the-run zero yields with essentially no sub-1y information.
   • Daily factor changes show no serial correlation [R] (AR(1) of ΔL, ΔS, ΔC: −0.02, 0.05, 0.00 for CMT; similar for GSW). Pure fitting noise in levels would make changes negatively autocorrelated. The CMT–GSW gap is therefore mostly a different object (front end, par vs zero, on- vs off-the-run), not transient noise.
   • Run H1–H2 on both targets and state which is primary. The ex-ante case for GSW (or CMT 2–30y) is that it matches the futures' maturity domain.
   • Report λ̂ = 0.49 (RMSE 9.4 vs 12.0 bp) as a robustness check. Daily-change moments barely move with λ̂ (ΔS sd 6.99 vs 7.27; corr(ΔL, ΔS) −0.83 vs −0.82 [R]), so λ is not first order. Do not estimate λ on 2008–2026 if it feeds OOS weights; that is look-ahead.

2. Level/slope separation (corr −0.82; −0.97 in the ZLB).

   • Report the condition number of the futures return covariance and the Var(β1+β2)/Var(β1) ratio per window.
   • Report weight paths with confidence bands, and the correlation between the fitted level and slope portfolios.
   • A cleaner parameterisation for interpretation is (short rate y(0) = β1+β2, level β1, curvature β3), or results stated in yield space (implied 2/5/10/30y minute changes), with NS factors as a summary.
   • In the ZLB, H2/H4 statements that compare "level" and "slope" are not separately identified. Pre-specify that ZLB results are reported for the coupon-segment factors only.

3. Short-end maturities (1m–6m) stretching the fit.

   • Fit RMSE by maturity [R]: 1m 15.0, 3m 5.1, 6m 11.2, 1y 15.0, 2y 9.3, 5y 13.9, 10y 10.5, 30y 14.4 bp. The fit is poor at both ends and at 5–7y, and the bills pull slope and curvature.
   • Daily residual-change SDs are largest at the bills (2.9, 2.3, 2.0 bp) [R].
   • Fix: as in Threat 1, a coupon-segment primary fit. A Svensson or second-curvature extension is not needed for this purpose.

4. Futures basis, CTD, roll.

   • (a) Duration drift. Price returns ≈ −D_CTD·Δy, and D_CTD varies with the yield level. Long contracts move materially between 0.5% and 5% yields, and a 3-year window spans regimes. The "structural inversion" alternative should become a co-primary construction: minute yield changes at 6 points via CTD duration, then a 3-parameter NS fit to those 6 points with fixed λ. That is transparent, has no estimated weights, and is less sensitive to collinearity.
   • (b) Basis dislocations. March 2020 (about 9–20 Mar, the basis-trade unwind) is a known episode in which futures and cash diverged. Pre-register it as an expected failure window and report fidelity with and without it.
   • (c) Roll rule. A "raw splice" at expiry leaves the series on the illiquid expiring contract after first notice. Check the v132 splice rule against volume and open interest. If it splices at expiry, switch to a volume-based roll, or drop minutes between first notice and the splice.
   • (d) Contract changes (TN launch 2016; UB basket) are breaks in w. Do not let the window straddle them without a dummy.
   • (e) Daily returns used to estimate w must be same-contract returns on roll days. The ratio-adjusted daily series gives this, but timestamps must be 15:30, not settlement.

5. Timing.

   • Treasury futures settle at 15:00 ET (14:00 CT), while CMT is read "near 15:30".
   • Sample futures at exactly 15:30 ET. Pre-register a sensitivity check at 15:00 and 16:00, and do not choose the timestamp by OOS fit.
   • Define yield days across holidays where CME trades but the cash market is closed or closes early (SIFMA early closes).
   • Use the CME session clock (CT) with a DST-consistent ET mapping, and condition H3 on US/Europe DST-mismatch weeks.

6. 1-minute microstructure. See Threat 3. Also:

   • Bars are open-to-open from trades. A minute's "open" is the first trade in the minute, so minutes are not synchronised across contracts.
   • Epps attenuation is therefore partly a bar-construction artifact. Previous-tick at the minute boundary (bar close) is preferable.

7. Stale prices in the Asian session.

   • Requiring all roots to trade biases the sample toward active minutes.
   • For H3, use per-contract refresh-time 5/15-minute returns and report the share of minutes with any missing root, by hour and year.
   • Estimate periodicity with a robust FFF or WSD estimator (Boudt–Croux–Laurent 2011) on volatility-standardised returns. Otherwise the 08:30 and 14:00 jump days dominate the profile.

8. OOS protocol.

   • Freeze now: window length, re-estimation frequency, K per period, timestamp, roll rule, λ (Diebold–Li fixed for the primary), target (Threat 1), and the definition of OOS R². State whether it is 1 − SSE/Σ(Δβ − mean_train)² or relative to zero; prefer the latter given near-zero means.
   • Report the OOS R² series per quarter, and the expanding-window and full-sample variants as bounds only.
   • Event days may stay in training windows, since they are prior to the forecast quarter.

9. Multiple testing.

   • Rough count: H1 3 factors × 2 criteria × 6 periods ≈ 36; H2 3 events × 3 factors × (ordering + 10× + regime contrast) ≈ 30+; H3 ≈ 6; H4 3 frequencies × 2 regimes ≈ 6–9; H5 3 factors × 3 horizons × 2 losses × 2 benchmarks = 36.
   • Pre-specify one primary test per hypothesis, e.g.:
     • H1: OOS R² on the primary target, full OOS sample.
     • H2: FOMC slope-share excess over a matched-clock baseline.
     • H4: noise-robust 5-minute level–slope correlation, ZLB vs 2022–26.
     • H5: HAR vs best benchmark for level, h = 1, QLIKE.
   • Apply Holm within each hypothesis family to the secondary tests. Use Romano–Wolf or Hansen–Lunde–Nason MCS for H5.

10. Inference for variance shares.

    • "Daily-block bootstrap" is not meaningful for non-contiguous event days. Resample events i.i.d., stratified by regime and meeting type (press conference or not). This gives about 150 scheduled FOMC statements and about 225 each of CPI and Employment over 2008–2026, but only about 48 FOMC in 2010–15.
    • Use the ratio of sums as the headline and the median per-event share as robustness. Shares are heavy-tailed and bounded.
    • Test the ordering with paired bootstrap differences within events.
    • Replace "≥10× 30/1380" with an excess over the same clock window on matched non-event days (same weekday, no other tier-1 release), tested by permutation over placebo dates. The 14:00–14:30 and 08:30–09:00 baselines are not 2.2%.
    • Denominators must be noise-robust (Threat 3).
    • Window: since 2011, and at every meeting since 2019, the press conference starts at 14:30. [−5, +25] cuts it off. Also report [−10, +20] (GSS) and a wide [−15, +105] window. Use per-meeting statement times (14:15 in 2008–2012; unscheduled actions: Oct 2008, Mar 2020).
    • CPI and Employment releases at 08:30 coincide with claims and other releases. Flag and drop overlap days in robustness.

11. H5 specifics.

    • Evaluating against RV from the same intraday data hands the result to HAR. Use two targets: noise-robust RV, and the squared daily CMT factor change. Both are admissible under Patton (2011) for MSE/QLIKE.
    • Compare with HAR on the realized futures covariance rotated by Ŵ, and with HAR on single-contract RV (e.g. ZN). Otherwise the NS rotation adds nothing testable.
    • Use Giacomini–White, since models are re-estimated on rolling windows. For overlapping h = 22, use HAC with ≥ h−1 lags.
    • Power is limited (about 115 non-overlapping 22-day blocks in 2016–2026).

12. Language. Event windows identify the variance realised at release times, not the "effect of monetary policy on slope". Without surprise measures, keep the language descriptive. Rigobon–Sack heteroskedasticity-based statements are possible if the matched-day baseline is in place.

# 3. Novelty: what a sceptical JFE/JFEC referee will say, and how to position

The sceptic's case.

• "Your intraday factors are three fixed linear combinations of six futures returns. Every object in H2–H5 is Ŵ Σ_t Ŵ', i.e. a rotation of the realized covariance of Treasury futures, with the NS label supplied by a daily regression."
• ABDV (2007) already study announcement responses of bond futures at 5-minute frequency with real-time expectations, across asset classes.
• Fleming–Remolona (1999, "The term structure of announcement effects") and Balduzzi–Elton–Green (2001) already measured intraday responses across the 3m–30y cash curve on GovPX. "Which part of the curve moves at 08:30" is known at the maturity level.
• GSS (2005) and Swanson (2021) extract target, path and LSAP factors from FOMC-window moves across maturities. Their factors are identified economic rotations (with surprise content). An NS rotation is statistically arbitrary and has no expectations measure.
• Hansen (2023, JFEC) already links high-frequency yield covariation to curve dynamics. Intraday periodicity in T-bond futures goes back to Bollerslev–Cai–Song (2000, JEmpFin).
• "The novelty is the label."

How to answer it. Three things can make this more than a label:

1. Spanning bounds as a result. Measure precisely which part of daily NS variation is tradable/spanned by futures, by regime. The coupon segment is nearly fully spanned; the bill segment, y(0) and much of curvature are not; the ZLB collapses the curve to effectively one spanned dimension. None of the papers above quantify this for NS factors, and it disciplines every high-frequency curve study that uses futures.
2. A validated 23-hour factor series (with the intraday validation of Threat 2), released as aggregates.
3. A variance accounting of the yield curve by clock time and event over 2008–2026: what share of daily level/slope/curvature variance arrives in FOMC/CPI/NFP windows, in US vs European vs Asian hours, and how that changed from ZLB to 2022–26. This is the piece ABDV and Fleming–Remolona cannot deliver: a single contract or maturity, a short sample, no factor decomposition, no overnight session.

Recommended positioning: a JFEC measurement paper with an economic variance-decomposition result. Drop "forecasting" as a headline contribution unless the rotated-covariance comparison (2.11) shows a gain. Add an explicit mapping (regression) of the FOMC-window NS factor changes on the Swanson/GSS factors. That shows what NS adds or loses relative to the identified rotation, and pre-empts the "arbitrary rotation" objection.

# 4. Ranked changes to DESIGN.md (keep / modify / add)

1. MODIFY §3.1/§4-H1: make the primary target an NS fit on the futures-spanned coupon segment (CMT 2–30y or GSW 1–30y), with the 11-maturity CMT fit secondary and its measured spanning ceiling reported.
2. MODIFY H1: state thresholds relative to the cash-yield spanning ceiling (in-sample about 0.98/0.74/0.76 on the 11-maturity CMT target [R]), and relabel H1 as daily spanning, since the intraday-sum test is an identity up to dropped segments.
3. ADD §3.4 intraday validation (pre-registered): leave-one-contract-out minute/5-minute prediction, an ETF-based independent construction, and a correlation with published FOMC surprise factors.
4. ADD §3.1: an exact bill vs coupon decomposition Δβ = A_bill Δy_bill + A_coup Δy_coup, with y(0) = β1+β2 reported as an unspanned quantity (or spanned by SR3/GE/ZQ if available in FinData OS).
5. MODIFY §3.3/§5: make 5-minute refresh-time previous-tick sampling primary, compute factor (co)variances as Ŵ Σ̂ Ŵ' with a noise-robust multivariate estimator, and keep 1-minute as robustness with signature plots.
6. ADD H4 guard: report the noise-implied level–slope correlation and accept H4 only if the noise-robust estimate shows the pattern, with ZLB results restricted to coupon-segment factors.
7. MODIFY §2: resolve the missing-minute contradiction (previous-tick price on the grid, no zero-return carrying, no dropped accumulated returns) and report the dropped-segment variance share.
8. MODIFY §3.2: promote the CTD-duration structural inversion (6-point NS fit each interval) to co-primary, with weight condition numbers and paths reported.
9. MODIFY §2/§3.2: correct K (4 in 2008–09, 5 in 2010–15, 6 from 2016) and state the effective OOS start per K (about 2011Q1 for K = 5 with 3-year windows; about 2019Q1 for K = 6), or adopt an explicit expanding/short-window rule for 2008–10.
10. MODIFY H5: align re-estimation frequency with §3.2, add the squared daily CMT factor change as a second target, add rotated-futures-covariance HAR and single-contract HAR as benchmarks, and use Giacomini–White and MCS.
11. MODIFY H2: replace the 10× uniform benchmark with an excess over a matched-clock non-event baseline (permutation test), and test the ordering by a paired event bootstrap.
12. MODIFY §5: replace the daily-block bootstrap for event shares with an event-level stratified bootstrap, and report ratio of sums plus median per-event shares.
13. ADD event windows: per-meeting statement times, [−10, +20] and [−15, +105] windows for press-conference meetings, and unscheduled actions flagged.
14. ADD §5 multiple testing: one primary test per hypothesis, Holm within families, Romano–Wolf/MCS for H5.
15. ADD §2 robustness: pre-register March 2020 (basis unwind) and the 2023 bill episodes (SVB, debt ceiling) as expected failure windows, reported with and without.
16. ADD §2: check the v132 splice rule against volume/first notice, and switch to a volume roll if it splices at expiry.
17. MODIFY §2 timing: fix the 15:30 ET snapshot, with 15:00/16:00 sensitivity not used for selection, plus explicit holiday, early-close and DST rules.
18. MODIFY §3.1: keep λ = 0.7308 fixed for the primary; use λ̂ only as robustness, estimated within training windows to avoid look-ahead.
19. ADD §1 positioning: spanning bounds plus clock/event variance accounting as the headline, an explicit NS-to-Swanson factor mapping, and descriptive (non-causal) language for event windows.
20. ADD H3: use robust periodicity estimation (WSD/FFF on standardised returns), 15-minute sampling overnight, and a DST-mismatch control.
21. KEEP: strictly prior rolling weights, the GSW robustness target, the sub-period scheme, two-sided 5% tests, and §7's falsification logic (with thresholds updated per item 2).
22. MODIFY §3.1 text: state the slope sign convention (β2 ≈ short − long) and rename corr_slope_vs_spread10y3m to reflect 3m−10y.