1# F2-conf-01 — School-level programme effect on student scores23## Question4A programme was assigned to whole schools. What is its effect on student test scores, τ = treated-school minus control-school student-weighted mean score (OLS slope of `score` on `treated`), with uncertainty that reflects school-level assignment? Is τ ≠ 0 at the 5% level (two-sided)?56## Data and exclusions7`data/schools.csv` (synthetic): 1000 rows (`n_input_rows`), no missing values, no duplicate students, treatment constant within each school. There are 40 schools (`n_schools`) with 25 students each; 20 are treated (`n_treated_schools`). No rows were excluded (`out/exclusions.csv`), so `n_analysed` = 1000. No data issues (`data_issues` = []).89## Method10- Point estimate: OLS of score on treated, with an intercept (`tau_hat`).11- Naive baseline: conventional homoskedastic iid OLS standard error (`se_iid`).12- Main uncertainty: cluster-robust (CR1) standard error clustered by school, using statsmodels' small-sample correction (`se_cluster`). Tests and the 95% CI use a t distribution with G−1 = 39 degrees of freedom (`aux_df_cluster`), which is conservative with 40 clusters.13- ICC: one-way ANOVA variance-component estimator applied to the OLS residuals (`icc_hat`).14- Cross-check: Welch two-sample t-test on the 40 school means (`aux_school_means_welch_p`).15- The method is deterministic and uses no random seed (`random_seed` = null).1617## Results18- `tau_hat` = 0.961 points.19- `se_cluster` = 0.729, t = 1.32 (`aux_t_stat_cluster`), `p_value` = 0.195.20- 95% CI: [`tau_ci_low`, `tau_ci_high`] = [−0.513, 2.436].21- `effect_significant_5pct` = false. At the 5% level, the data do not support a non-zero effect.22- `icc_hat` = 0.235.2324## Uncertainty25The naive `se_iid` = 0.282 would give p = 0.0007 (`aux_p_value_iid`), which looks highly significant. It ignores the fact that students in the same school are correlated. The cluster SE is 2.58 times larger (`aux_se_ratio_cluster_over_iid`). This matches the Moulton factor √(1+(25−1)·0.235) = √6.64 (`aux_design_effect_moulton`). The school-means Welch test reaches the same conclusion (p = 0.201, `aux_school_means_welch_p`). Because treatment varies only between schools, the effective sample size is close to 40 schools, not 1000 students.2627## Limitations and open issues28- A non-significant result does not show that the effect is zero. The CI is consistent with effects from about −0.5 to +2.4 points.29- Reading τ causally requires that schools were assigned to treatment as if at random. The task does not document the assignment mechanism.30- Cluster-robust inference relies on asymptotics in the number of clusters. With 40 balanced clusters this is generally adequate, and the t(39) reference plus the school-means cross-check supports it.31- The ICC is estimated with noise from 40 schools.32