# F2-conf-01 — School-level programme effect on student scores

## Question
A programme was assigned to whole schools. We estimate τ, the difference between treated- and control-school mean student scores (OLS slope of `score` on `treated` with intercept). We also give a standard error that reflects the clustered design, where treatment varies only between schools and students in the same school share school-level shocks. Finally, we test H0: τ = 0 at the 5% level (two-sided).

## Data and exclusions
- `data/schools.csv` (synthetic). 1000 rows read (`n_input_rows`). The file has no missing values, no duplicate rows and no duplicate student ids. Its SHA-256 and the full audit are in `out/state/data_audit.md`.
- `n_schools` = 40 schools of 25 students each (`students_per_school`). `n_treated_schools` = 20. Treatment is constant within every school.
- No exclusions (`out/exclusions.csv`): `n_analysed` = 1000. `data_issues` is empty.

## Method
- Point estimate: OLS `score ~ treated` (statsmodels).
- Naïve baseline: conventional iid/homoskedastic OLS SE (`se_iid`).
- Main uncertainty measure: cluster-robust SE with school as the cluster (CR1: statsmodels' default small-sample factor G/(G−1)·(N−1)/(N−K)) (`se_cluster`). The p-value and 95% CI use a t distribution with G−1 = 39 degrees of freedom (`t_df`). This is a common conservative reference when there are few clusters.
- Sensitivity check: a two-sample t-test on the 40 school means. Because every school has the same size, this targets the same τ (`school_mean_ttest_t`, `school_mean_ttest_p`).
- ICC: a one-way ANOVA (method-of-moments) estimator applied to the OLS residuals grouped by school, so the treatment effect is removed first (`icc_hat`). Implied design effect = 1 + (m−1)·ICC (`design_effect_from_icc`).
- The method is deterministic, so no seed is used (`random_seed` = null).

## Results
- `tau_hat` = 0.961 points: treated-school students score about 0.96 points higher on average.
- `se_iid` = 0.282 versus `se_cluster` = 0.729. The clustered SE is 2.58 times larger (`se_ratio_cluster_to_iid`). That ratio fits the estimated `icc_hat` = 0.235, which implies a design effect of 6.64 (`design_effect_from_icc`) and √6.64 ≈ 2.58.
- Clustered test: t = 1.32 (`t_stat_cluster`), `p_value` = 0.195. 95% CI: [`tau_ci_low`, `tau_ci_high`] = [−0.51, 2.44].
- `effect_significant_5pct` = false: with design-valid uncertainty, the data **do not** support a non-zero effect at the 5% level.
- The naïve iid analysis would give p = 0.0007 (`p_value_iid`) and falsely declare significance. This shows why the clustered SE matters.

## Uncertainty
The CI runs from about −0.5 to +2.4 points. So the data are consistent with no effect and also with a positive effect of up to about 2.4 points. The school-means t-test gives the same conclusion (p = 0.201, `school_mean_ttest_p`). Inference rests on 40 independent schools, the effective number of independent units, not on 1000 students.

## Limitations and open issues
- The data are synthetic. Reading τ as a causal effect assumes schools were assigned to treatment as-good-as-randomly, which cannot be checked from these data.
- With G = 40 clusters, cluster-robust inference is generally regarded as reasonable but not exact. CR1 with t(39) and the school-means t-test agree here. Wild cluster bootstrap and CR2 were not computed because the task does not require them.
- `icc_hat` is a moment estimator and is noisy with 40 clusters. It is a descriptive quantity and does not drive the decision.
- Not significant does not mean no effect. The study cannot rule out effects of up to about 2.4 points.