# WP35 — Final research audit (checklist)

Date: 2026-11-01. Executor: Claude Code (claude-fable-5-1), autonomous, under the PI's mandate (`docs/MANDATE_VERBATIM.txt`).
Status codes: **DONE** (code ran, output exists, source traceable, limitations documented), **PARTIAL**, **FAILED**,
**BLOCKED** (needs PI action or data that does not exist), **NOT EXECUTED** (planned, not run).

## Phase 0 — Environment and feasibility
| Task | Status | Evidence |
|---|---|---|
| Hardware/tools/LaTeX/connectivity audit | DONE | `results/logs/feasibility_report.md` |
| Real sample downloads with hashes | DONE | `data/metadata/source_registry.json` (9 sources, SHA-256, retrieval times) |
| LLM API access check | DONE (none used) | no keys configured; no paid calls; `anthropic` MCP failed to connect this session |

## Phase 1 — Literature and pre-registration
| Task | Status | Evidence |
|---|---|---|
| Reference verification against Crossref | DONE | `research/prior_art.csv` (145/173 verified by query; corrections and books in `bib_overrides.csv`; 182 entries in `paper/references.bib`) |
| Novelty matrix | DONE | `research/literature_review.md` §2; `docs/novelty_search_report.md`; paper Table 1 |
| Pre-registration frozen before outcome associations | DONE | `research/preregistration.md`, SHA-256 `ecda8804…` in `research/decisions.md` D-003 |

## Phase 2 — Data engineering
| Task | Status | Evidence |
|---|---|---|
| Seshat Polaris-2026 parser (codes, ranges, dates, flags, pre-crisis aggregates, CrisisDB, succession) | DONE | `src/wp35/harmonization/seshat.py`; `data/processed/seshat_*.parquet`; 132 unparsed values logged |
| D-PLACE EA v3.3 + Glottolog families | DONE | `dplace.py`; `dplace_ea_societies.parquet`; code midpoints logged |
| V-Dem v16 + Maddison 2023 crosswalk and events | DONE | `modern.py`; `modern_panel.parquet`; `crosswalk_vdem_maddison.csv` (39 unmatched) |
| Coverage tables, temporal figures, missingness diagnostics, data dictionary | DONE | `results/diagnostics/*`, `paper/figures/fig_seshat_*`, `data/metadata/data_dictionary.md` |
| Equinox-2020 alternative release parsed | NOT EXECUTED | listed in pre-registration §6 item 8; raw file downloaded; parser not written (time) |

## Phase 3 — Theory and measurement
| Task | Status | Evidence |
|---|---|---|
| Formal framework: 5 theorems with proofs, 3 conjectures, mapping to tests | DONE | `research/framework.md`, paper §3 |
| D, K, P indices; IRT alternative; PC alternative; bounds; invariance; leave-one-block-out | DONE | `src/wp35/measurement/build_indices.py`; `results/empirical/measurement_*` |
| A-priori power (MDE) | DONE | `power_h2a_binary.csv`, `power_h2a_fractional.csv`; Y1b declared underpowered a priori |
| Measurement report | DONE | `research/measurement.md` |

## Phase 4 — Historical econometrics
| Task | Status | Evidence |
|---|---|---|
| H1 (Seshat OLS/mixed; EA Conley + ordered logit; modern within/differences) | DONE | `h1_seshat.csv`, `h1_ea.csv`, `h1_modern.csv` |
| H2-A primary (fractional logit), baselines, placebo, robustness set, LORO + 10×5 CV, 80% rule | DONE | `h2_seshat_primary.csv`, `_robustness.csv`, `_cv_*.csv`; fold-internal leakage check `h2_seshat_cv_noleakage.csv` (A-001) |
| H2-A binary (Y1b) | DONE (underpowered; quasi-separation → region FE/HC1) | `h2_seshat_primary.csv` rows A1b–A3b |
| Y2 Cox, Y3 succession | DONE | `h2_seshat_y2_y3.csv` |
| Dated-only "strict" pre-crisis sensitivity | FAILED (not estimable) | too few dated facts; see `research/limitations.md` A.2 |
| H2-C modern: pooled/conditional/LPM, baselines, placebo, negative control, forward chaining, LORO, calibration, Oster/Cinelli–Hazlett | DONE | `h2_modern_primary.csv`, `_robustness.csv`, `_cv_*`, `h2_modern_cv_predictions.parquet` |
| H4 (hinge/spline OOS) both panels | DONE | `h4_seshat_summary.json`, `h4_modern_summary.json`, `h4_modern_curve.csv` |
| H5 (held-out blocks) both panels | DONE | `h5_seshat_blocks.csv`, `h5_modern_blocks.csv` |
| Holm correction across the three primary tests | DONE | `results/empirical/holm_primary_tests.csv` (all adjusted p < 0.01; the H2 tests nevertheless fail the pre-registered out-of-sample rule) |
| Identification note | DONE | `research/identification.md`, paper §5.5 |

## Phase 5 — Simulations
| Task | Status | Evidence |
|---|---|---|
| ABM with heterogeneous agents, networks, bandwidth, shocks, five imposed regimes | DONE | `src/wp35/simulations/model.py` |
| Factorial design (6,048 runs, 8 seeds) + ablations (584) | DONE | `results/simulated/runs_main.parquet`, `runs_ablation.parquet` |
| Predefined metrics incl. bits per success, overhead, Gini, latency; Shannon quantities where defined | DONE | `simulation_summary.json`, `cells_main.csv` |
| Fragility search (cells where institutions lose) | DONE | `institution_loss_shares.csv`, `cells_institution_vs_none.csv` |
| Theorem checks in silico | DONE | `simulation_summary.json` → `theorem_checks` |

## Phase 6 — Integration
| Task | Status | Evidence |
|---|---|---|
| Observable vs non-observable mechanisms; rivals; consistency; scope conditions | DONE | `research/integration.md`, paper §9 |

## Phase 7 — Figures, tables, reproducibility
| Task | Status | Evidence |
|---|---|---|
| All tables/figures generated by script | DONE | `src/wp35/visualization/make_all.py` (26 fragments, 15 figures) |
| Tests | DONE | `tests/` — 11 passed after revision |
| Clean-environment full reproduction | PARTIAL | every target was run in sequence from the venv in this session; a from-scratch `make all` on a second machine was not run |

## Phase 8 — Paper
| Task | Status | Evidence |
|---|---|---|
| LaTeX manuscript compiled, references resolved, figures present | DONE | `paper/main.pdf` 80 pp. after revision; 0 undefined citations/references; 8 small overfull boxes |
| Abstract written after analyses | DONE | — |

## Phase 9 — Adversarial review
| Task | Status | Evidence |
|---|---|---|
| Three independent reviews (institutional economist, historical anthropologist, computational social scientist) | DONE — all three: major revision | `docs/reviews/review_*.md` |
| Response to reviewers and implemented corrections | DONE — 25 points fixed with reruns (A-002…A-007), 6 accepted as unfixable and stated, 3 declined with reasons | `docs/reviews/response_to_reviewers.md`; `research/decisions.md` |
| Defects found by review that violated repository rule 1 (text–results mismatch) | FIXED | five robustness variants never computed → computed; placebo tautology → respecified; failed CV folds misreported → repaired |

## Blocked / requires PI
- ~~Creation of the spbgit remote `wp35_uqo.git` and `git push`~~ — done 2026-11-01 on PI instruction ("push").
- Any external publication (SSRN) and the website entry (`spboucher.ai`).
- Equinox-2020 replication (not blocked, simply not executed).

## Headline verdict (for the record, after revision)
H1 supported as association (all datasets). H2 **not supported** beyond baselines in either panel (pre-registered rule);
the restriction β_D+β_K=0 is rejected in the modern panel; the Seshat placebo is not distinguishable from P; higher K
predicts shorter polity durations (against the hypothesis). H3 supported in operating-cost sense (SIMULATED; success
effects are parameters), refuted in total-cost sense at short horizons; central hierarchies fragile at equal budgets.
H4 rejected in Seshat; not rejected but small in the modern panel. H5: sign stable in every held-out block (regions, eras,
cultural spheres, periods). Universal framing abandoned per pre-registration §7.