Re-examination of the results memo: is the slope of y on x significant?
Question
The memo (data/memo.md) ran OLS of y on x with an intercept on 400 monthly observations. It used conventional (iid) standard errors and concluded that the slope is significant at the 5% level. The disturbances are serially correlated and x is persistent, so the iid standard error is not valid. I re-estimate the slope β in y_t = α + β x_t + u_t and test H0: β = 0 with a Newey–West HAC standard error. Then I check whether the memo's claim still holds.
Data and exclusions
data/series.csvhas 400 rows (n_input_rows), running monthly from 1990-01 to 2023-04. There are no gaps, no duplicates and no missing values (out/state/data_audit.md). Its SHA-256 matches the bundle manifest.- I excluded no observations, so
n_analysed= 400 (out/exclusions.csv: 400 − 0 = 400).data_issuesis an empty list. - I used the memo only as something to check. None of its numbers feed into the computation.
Method
- Point estimate: OLS with an intercept on the full sample (
beta_hat,alpha_hat). - Baseline (the memo's measure): the conventional iid OLS standard error (
se_iid,baseline_t_iid,baseline_p_iid). - Residual persistence:
rho_resid= Σû_t û_{t-1} / Σû_t². - HAC standard error (
se_hac): Newey–West with a Bartlett kernel (weights 1 − j/(L+1)) and the small-sample factor T/(T−2). I computed it with statsmodels and checked it against my own numpy implementation (out/state/checks.md). - Bandwidth (
hac_lags): a data-dependent rule:- First, the Andrews (1991) AR(1) plug-in for the Bartlett kernel, applied to the slope score v_t = (x_t − x̄)û_t. Its formula is S = 1.1447·(α̂(1)·T)^{1/3}, with α̂(1) = 4ρ_v²/((1−ρ_v)²(1+ρ_v)²). The score's estimated autocorrelation is ρ_v =
rho_score= 0.401, which gives S =andrews_bandwidth_raw≈ 8.2. - I rounded S up to 9 and kept the larger of that and the Newey–West (1994) rule of thumb floor(4(T/100)^{2/9}) = 5 (
nw1994_rule_lags). That gives L = 9 months. - Why a plug-in rather than a fixed rule: with error autocorrelation near 0.7 and a persistent regressor, a fixed rule like 5 lags is likely to cut off autocorrelation that is still present.
- First, the Andrews (1991) AR(1) plug-in for the Bartlett kernel, applied to the slope score v_t = (x_t − x̄)û_t. Its formula is S = 1.1447·(α̂(1)·T)^{1/3}, with α̂(1) = 4ρ_v²/((1−ρ_v)²(1+ρ_v)²). The score's estimated autocorrelation is ρ_v =
- Test and interval: t_hac = β̂/se_hac, a two-sided p-value from the standard normal (
p_hac), and a 95% interval β̂ ± 1.96·se_hac. - Decision rule:
memo_claim_supported= (p_hac < 0.05). - No random numbers are used (
random_seed= null).
Results
| quantity | value | key |
|---|---|---|
| intercept | 2.0925 | alpha_hat |
| slope | 0.1017 | beta_hat |
| iid SE (memo) | 0.0448 | se_iid |
| iid t / p | 2.27 / 0.0238 | baseline_t_iid, baseline_p_iid |
| residual AR(1) coefficient | 0.609 | rho_resid |
| HAC lags | 9 | hac_lags |
| HAC SE | 0.0654 | se_hac |
| HAC SE / iid SE | 1.46 | se_ratio_hac_over_iid |
| HAC t | 1.56 | t_hac |
| HAC p (two-sided) | 0.120 | p_hac |
| 95% CI (HAC) | [−0.026, 0.230] | beta_ci_low, beta_ci_high |
| memo claim supported | false | memo_claim_supported |
- My recomputation reproduces the memo's point estimates, SE, t and p to rounding. The memo's arithmetic is correct.
- The residuals are strongly autocorrelated (
rho_resid= 0.61). - The HAC standard error is 46% larger than the iid one (
se_ratio_hac_over_iid). With it, the t-statistic falls to 1.56 (t_hac) and the p-value rises to 0.12 (p_hac). The 95% interval contains zero. - Verdict: the memo's claim that the slope is significant at the 5% level is not supported (
memo_claim_supported= false).
Uncertainty
- Bandwidth: the verdict does not depend on the bandwidth choice. Across 5, 8, 9, 12, 18 and 24 lags, the HAC SE ranges from 0.0645 to 0.0663 and the p-value from 0.115 to 0.125 (
hac_sensitivity_by_lag). Even the smallest bandwidth (the 1994 rule of thumb, 5 lags) gives p = 0.115. - Reference distribution: using a Student-t instead of the normal would only make the p-value larger.
- Point estimate: β̂ = 0.10 is positive. The data cannot distinguish it from zero, or from values up to about 0.23, at the 95% level.
Limitations and open issues
- Known size distortion: Newey–West tests are known to over-reject in finite samples when regressors and errors are persistent. If anything, the true p-value is probably higher than 0.12, which reinforces the conclusion. I did not compute fixed-b (Kiefer–Vogelsang) critical values or a parametric GLS/Cochrane–Orcutt alternative because the task does not require them.
- What the estimand means: β is a linear-projection coefficient, i.e. an association. The memo's wording that "x is a significant predictor" is also loose. The regression is contemporaneous, not out-of-sample prediction, and nothing here supports a causal reading.
- Bandwidth convention: Andrews' S is the Bartlett truncation parameter (weight 1 − j/S), while statsmodels uses weights 1 − j/(L+1). Rounding S up therefore adds roughly one lag of extra conservatism. The sensitivity table shows this does not matter.
- Simulated data: the data are synthetic, as stated in TASK.md, so the conclusions apply to this dataset only.