1# F2-conf-01 — School-level programme effect23## Question4Estimate the average effect τ of a programme assigned at school level on student test scores (difference in student-weighted mean score between treated and control schools = OLS slope of `score` on `treated`), with uncertainty that respects the clustered design, and test H0: τ = 0 at the 5% level (two-sided).56## Data and exclusions7`data/schools.csv` (synthetic): 1000 rows read (`n_input_rows`), 40 schools (`n_schools`) of 25 students each (`students_per_school`), 20 treated (`n_treated_schools`). No missing values, no duplicate students, treatment constant within school; no rows excluded (`out/exclusions.csv`), so `n_analysed` = 1000. `data_issues` is empty.89## Method10OLS of score on an intercept and `treated`. Naïve baseline: conventional homoskedastic SE (`se_iid`). Main uncertainty measure: cluster-robust (CR1, school clusters, small-sample correction G/(G−1)·(N−1)/(N−K)) SE (`se_cluster`), with inference from a t distribution with G−1 = 39 degrees of freedom (`cluster_df`). The residual intraclass correlation (`icc_hat`) uses the one-way ANOVA estimator on OLS residuals grouped by school. Cross-check: a two-sample t-test on the 40 school means. No randomness (`random_seed` = null).1112## Results13- Estimated effect `tau_hat` = 0.961 points.14- Cluster-robust SE `se_cluster` = 0.729; t = 1.32 (`t_stat_cluster`); `p_value` = 0.195.15- 95% CI [`tau_ci_low`, `tau_ci_high`] = [−0.513, 2.436].16- `effect_significant_5pct` = false: at the 5% level the data do not support a non-zero effect.17- `icc_hat` = 0.235.1819## Uncertainty20The naïve SE `se_iid` = 0.282 would give `p_value_iid` = 0.0007, which looks highly significant. It treats 1000 students as independent, but treatment varies only across 40 schools and residuals are correlated within school (`icc_hat` = 0.235). The cluster SE is 2.58 times larger (`se_ratio_cluster_to_iid`), which agrees with the Moulton design effect 1+(25−1)·ICC = 6.64 (`design_effect_from_icc`; √6.64 ≈ 2.58). The school-means analysis gives the same answer (SE `check_se_school_means` = 0.738, p `check_p_school_means_pooled_t` = 0.201).2122## Limitations and open issues23- With 40 clusters (20 per arm, balanced) CR1 with t(39) should be reliable. Other reasonable choices (school-means t-test) give similar p-values (≈0.20), so the conclusion does not depend on which one is used.24- Reading τ̂ as causal assumes schools were assigned to treatment as if at random. The data are simulated and that assignment was not verified.25- Not rejecting H0 does not show the effect is zero: the CI includes effects up to about 2.4 points.26