F2-conf-03 — Coverage of OLS slope CIs in a small heteroskedastic sample
Question
For the fixed design in data/design.csv (n = 30, y = 1 + 0.5x + σ_i ε), does the nominal 95% t-interval for the slope reach 95% coverage? We compare the conventional SE with HC1, compute the exact SD of β̂, and compute the mean conventional SE.
Data and exclusions
I used one input, design.csv: 30 rows (n_input_rows), with no missing values and no duplicates. All σ are positive. No rows were excluded (out/exclusions.csv), and data_issues is empty. σ_i grows with |x_i|, so the high-leverage observations are also the noisiest.
Method
Each replication draws ε ~ N(0, I₃₀) and computes OLS β̂, se_conv (s² with divisor 28) and se_HC1 (HC0 × 30/28). An interval counts as covering when |β̂ − 0.5| ≤ t₀.₉₇₅(28)·se, with t = 2.0484 (t_critical_28). There are R = 2000 replications (n_replications) and the numpy default_rng seed is 20261030 (random_seed). The exact SD is sqrt(Σ w_i² σ_i²), where w is the slope row of (X'X)⁻¹X'. The undercoverage rule is coverage < 0.95 − 3·sqrt(0.95·0.05/2000) = 0.9354 (undercover_threshold). The binomial SE is evaluated at the nominal level.
Results
- Conventional-interval coverage: 0.844 (
coverage_conventional). HC1-interval coverage: 0.902 (coverage_hc1). - Exact SD(β̂) = 0.2680 (
sd_beta_exact). The MC SD is 0.2696 (sd_beta_mc) and the MC mean of β̂ is 0.4962 (mean_beta_mc). - Mean se_conv = 0.1903 (
mean_se_conventional), about 29% below the true SD. Mean se_HC1 = 0.2408 (mean_se_hc1), closer to the true SD but still too small. - The conventional interval does not attain nominal coverage (
conventional_ci_undercovers= true). HC1 improves coverage but also stays below 95%. Its gap of 0.048 is about 7 MCSE (coverage_hc1,mcse_coverage_hc1); the report applies no formal test to it.
Uncertainty
These are binomial Monte Carlo standard errors at R = 2000:
- Conventional: 0.0081 using the estimated p (
mcse_coverage_conventional). - HC1: 0.0066 (
mcse_coverage_hc1). - At p = 0.95: 0.0049 (
mcse_at_nominal_0.95).
The conventional shortfall (0.106) is about 13 MCSE under either SE definition, so the undercoverage conclusion does not depend on which one is used (undercovers_using_estimated_p_se = true). The sampling uncertainty in sd_beta_mc relative to the exact value is about 1.6%; the observed difference is 0.6%.
Limitations and open issues
- All results are conditional on this one synthetic design and normal errors. Other designs or error distributions will give different coverage.
- The task does not say whether the binomial SE in the undercoverage rule should use p = 0.95 or the estimated p. I used p = 0.95 and checked that the alternative gives the same conclusion.
- Monte Carlo estimates depend on the seed. Different seeds will move the coverages by roughly ±2 MCSE.
- Small-sample corrections stronger than HC1, such as HC2/HC3 or a Bell–McCaffrey degrees-of-freedom correction, were not evaluated because the task does not require them.