1# F2-conf-01 — School-level programme effect on student scores23## Question4A programme was assigned to whole schools. We estimate τ, the difference between the student-weighted mean score in treated and control schools (OLS slope of `score` on `treated` with an intercept), give an uncertainty measure that respects the school-level design, and test H0: τ = 0 at the 5% level (two-sided).56## Data and exclusions7Input: `data/schools.csv`, 1000 rows (`n_input_rows`), simulated data. Audit (`out/state/data_audit.md`): no missing values, no duplicates, unique student IDs, 40 schools (`n_schools`) of exactly 25 students (`school_size`), treatment constant within school, 20 treated schools (`n_treated_schools`). No rows excluded (`out/exclusions.csv`); `n_analysed` = 1000. `data_issues` is empty.89## Method10- Point estimate: OLS of score on treated with intercept (`tau_hat`).11- Naïve baseline: conventional homoskedastic iid OLS SE (`se_iid`).12- Main uncertainty: cluster-robust (Liang–Zeger, CR1 small-sample correction) SE clustered by `school_id` (`se_cluster`). Test statistic and 95% CI use a t distribution with G−1 = 39 df (`df_cluster`), which is appropriate with a moderate number of clusters.13- ICC: one-way random-effects ANOVA estimator on the OLS residuals, with G−2 between-school df (one lost to the treatment effect) (`icc_hat`).14- Sensitivity: two-sample t-test on the 40 school means (`sens_school_means_se`, `sens_school_means_p`).15- Deterministic; no randomness (`random_seed` = null).1617## Results18- `tau_hat` = 0.961 points (treated mean 51.28 `mean_treated` vs control 50.32 `mean_control`).19- `se_cluster` = 0.729; t = 1.32 (`t_stat_cluster`); `p_value` = 0.195.20- 95% CI: [`tau_ci_low` = −0.513, `tau_ci_high` = 2.436].21- `effect_significant_5pct` = false: the data do not support a non-zero effect at the 5% level.22- `icc_hat` = 0.240; implied design effect 6.77 (`design_effect_from_icc`).2324## Uncertainty25The naïve `se_iid` = 0.282 would give p = 0.0007 (`p_value_iid`) and a spurious "significant" result. Because the programme varies only between schools and students in a school share school-level shocks, the effective sample is closer to 40 schools than 1000 students: the clustered SE is 2.58× larger (`se_ratio_cluster_over_iid`), consistent with √(design effect) ≈ 2.60. The school-means t-test gives the same conclusion (SE 0.738, p = 0.201). Verification checks are listed in `out/state/checks.md`.2627## Limitations and open issues28- Not rejecting H0 does not mean the effect is zero; the CI is compatible with effects from about −0.5 to +2.4 points.29- With 40 clusters, CR1 with t(39) is a standard and reasonably accurate choice; wild-cluster bootstrap was not used (not required; balanced design makes CR1 ≈ school-means t-test, which was checked).30- Causal interpretation relies on the school assignment being as-good-as-random, as in the simulated design; no covariates were available.31- The data are synthetic.32