SPB Git

spb/wp29_uqo

Public

# Re-examination of the results memo: is the slope of y on x significant?

# Question

The memo (data/memo.md) ran OLS of y on x with an intercept on 400 monthly observations. It used conventional (iid) standard errors and concluded that the slope is significant at the 5% level. The disturbances are serially correlated and x is persistent, so the iid standard error is not valid. I re-estimate the slope β in y_t = α + β x_t + u_t and test H0: β = 0 with a Newey–West HAC standard error. Then I check whether the memo's claim still holds.

# Data and exclusions

  • data/series.csv has 400 rows (n_input_rows), running monthly from 1990-01 to 2023-04. There are no gaps, no duplicates and no missing values (out/state/data_audit.md). Its SHA-256 matches the bundle manifest.
  • I excluded no observations, so n_analysed = 400 (out/exclusions.csv: 400 − 0 = 400). data_issues is an empty list.
  • I used the memo only as something to check. None of its numbers feed into the computation.

# Method

  • Point estimate: OLS with an intercept on the full sample (beta_hat, alpha_hat).
  • Baseline (the memo's measure): the conventional iid OLS standard error (se_iid, baseline_t_iid, baseline_p_iid).
  • Residual persistence: rho_resid = Σû_t û_{t-1} / Σû_t².
  • HAC standard error (se_hac): Newey–West with a Bartlett kernel (weights 1 − j/(L+1)) and the small-sample factor T/(T−2). I computed it with statsmodels and checked it against my own numpy implementation (out/state/checks.md).
  • Bandwidth (hac_lags): a data-dependent rule:
    • First, the Andrews (1991) AR(1) plug-in for the Bartlett kernel, applied to the slope score v_t = (x_t − x̄)û_t. Its formula is S = 1.1447·(α̂(1)·T)^{1/3}, with α̂(1) = 4ρ_v²/((1−ρ_v)²(1+ρ_v)²). The score's estimated autocorrelation is ρ_v = rho_score = 0.401, which gives S = andrews_bandwidth_raw ≈ 8.2.
    • I rounded S up to 9 and kept the larger of that and the Newey–West (1994) rule of thumb floor(4(T/100)^{2/9}) = 5 (nw1994_rule_lags). That gives L = 9 months.
    • Why a plug-in rather than a fixed rule: with error autocorrelation near 0.7 and a persistent regressor, a fixed rule like 5 lags is likely to cut off autocorrelation that is still present.
  • Test and interval: t_hac = β̂/se_hac, a two-sided p-value from the standard normal (p_hac), and a 95% interval β̂ ± 1.96·se_hac.
  • Decision rule: memo_claim_supported = (p_hac < 0.05).
  • No random numbers are used (random_seed = null).

# Results

quantity value key
intercept 2.0925 alpha_hat
slope 0.1017 beta_hat
iid SE (memo) 0.0448 se_iid
iid t / p 2.27 / 0.0238 baseline_t_iid, baseline_p_iid
residual AR(1) coefficient 0.609 rho_resid
HAC lags 9 hac_lags
HAC SE 0.0654 se_hac
HAC SE / iid SE 1.46 se_ratio_hac_over_iid
HAC t 1.56 t_hac
HAC p (two-sided) 0.120 p_hac
95% CI (HAC) [−0.026, 0.230] beta_ci_low, beta_ci_high
memo claim supported false memo_claim_supported
  • My recomputation reproduces the memo's point estimates, SE, t and p to rounding. The memo's arithmetic is correct.
  • The residuals are strongly autocorrelated (rho_resid = 0.61).
  • The HAC standard error is 46% larger than the iid one (se_ratio_hac_over_iid). With it, the t-statistic falls to 1.56 (t_hac) and the p-value rises to 0.12 (p_hac). The 95% interval contains zero.
  • Verdict: the memo's claim that the slope is significant at the 5% level is not supported (memo_claim_supported = false).

# Uncertainty

  • Bandwidth: the verdict does not depend on the bandwidth choice. Across 5, 8, 9, 12, 18 and 24 lags, the HAC SE ranges from 0.0645 to 0.0663 and the p-value from 0.115 to 0.125 (hac_sensitivity_by_lag). Even the smallest bandwidth (the 1994 rule of thumb, 5 lags) gives p = 0.115.
  • Reference distribution: using a Student-t instead of the normal would only make the p-value larger.
  • Point estimate: β̂ = 0.10 is positive. The data cannot distinguish it from zero, or from values up to about 0.23, at the 95% level.

# Limitations and open issues

  • Known size distortion: Newey–West tests are known to over-reject in finite samples when regressors and errors are persistent. If anything, the true p-value is probably higher than 0.12, which reinforces the conclusion. I did not compute fixed-b (Kiefer–Vogelsang) critical values or a parametric GLS/Cochrane–Orcutt alternative because the task does not require them.
  • What the estimand means: β is a linear-projection coefficient, i.e. an association. The memo's wording that "x is a significant predictor" is also loose. The regression is contemporaneous, not out-of-sample prediction, and nothing here supports a causal reading.
  • Bandwidth convention: Andrews' S is the Bartlett truncation parameter (weight 1 − j/S), while statsmodels uses weights 1 − j/(L+1). Rounding S up therefore adds roughly one lag of extra conservatism. The sensitivity table shows this does not matter.
  • Simulated data: the data are synthetic, as stated in TASK.md, so the conclusions apply to this dataset only.
Files · main
Menu

Theme