# WP35 — Final research audit (checklist)
Date: 2026-11-01. Executor: Claude Code (claude-fable-5-1), autonomous, under the PI's mandate (docs/MANDATE_VERBATIM.txt).
Status codes: DONE (code ran, output exists, source traceable, limitations documented), PARTIAL, FAILED,
BLOCKED (needs PI action or data that does not exist), NOT EXECUTED (planned, not run).
# Phase 0 — Environment and feasibility
| Task |
Status |
Evidence |
| Hardware/tools/LaTeX/connectivity audit |
DONE |
results/logs/feasibility_report.md |
| Real sample downloads with hashes |
DONE |
data/metadata/source_registry.json (9 sources, SHA-256, retrieval times) |
| LLM API access check |
DONE (none used) |
no keys configured; no paid calls; anthropic MCP failed to connect this session |
# Phase 1 — Literature and pre-registration
| Task |
Status |
Evidence |
| Reference verification against Crossref |
DONE |
research/prior_art.csv (145/173 verified by query; corrections and books in bib_overrides.csv; 182 entries in paper/references.bib) |
| Novelty matrix |
DONE |
research/literature_review.md §2; docs/novelty_search_report.md; paper Table 1 |
| Pre-registration frozen before outcome associations |
DONE |
research/preregistration.md, SHA-256 ecda8804… in research/decisions.md D-003 |
# Phase 2 — Data engineering
| Task |
Status |
Evidence |
| Seshat Polaris-2026 parser (codes, ranges, dates, flags, pre-crisis aggregates, CrisisDB, succession) |
DONE |
src/wp35/harmonization/seshat.py; data/processed/seshat_*.parquet; 132 unparsed values logged |
| D-PLACE EA v3.3 + Glottolog families |
DONE |
dplace.py; dplace_ea_societies.parquet; code midpoints logged |
| V-Dem v16 + Maddison 2023 crosswalk and events |
DONE |
modern.py; modern_panel.parquet; crosswalk_vdem_maddison.csv (39 unmatched) |
| Coverage tables, temporal figures, missingness diagnostics, data dictionary |
DONE |
results/diagnostics/*, paper/figures/fig_seshat_*, data/metadata/data_dictionary.md |
| Equinox-2020 alternative release parsed |
NOT EXECUTED |
listed in pre-registration §6 item 8; raw file downloaded; parser not written (time) |
# Phase 3 — Theory and measurement
| Task |
Status |
Evidence |
| Formal framework: 5 theorems with proofs, 3 conjectures, mapping to tests |
DONE |
research/framework.md, paper §3 |
| D, K, P indices; IRT alternative; PC alternative; bounds; invariance; leave-one-block-out |
DONE |
src/wp35/measurement/build_indices.py; results/empirical/measurement_* |
| A-priori power (MDE) |
DONE |
power_h2a_binary.csv, power_h2a_fractional.csv; Y1b declared underpowered a priori |
| Measurement report |
DONE |
research/measurement.md |
# Phase 4 — Historical econometrics
| Task |
Status |
Evidence |
| H1 (Seshat OLS/mixed; EA Conley + ordered logit; modern within/differences) |
DONE |
h1_seshat.csv, h1_ea.csv, h1_modern.csv |
| H2-A primary (fractional logit), baselines, placebo, robustness set, LORO + 10×5 CV, 80% rule |
DONE |
h2_seshat_primary.csv, _robustness.csv, _cv_*.csv; fold-internal leakage check h2_seshat_cv_noleakage.csv (A-001) |
| H2-A binary (Y1b) |
DONE (underpowered; quasi-separation → region FE/HC1) |
h2_seshat_primary.csv rows A1b–A3b |
| Y2 Cox, Y3 succession |
DONE |
h2_seshat_y2_y3.csv |
| Dated-only "strict" pre-crisis sensitivity |
FAILED (not estimable) |
too few dated facts; see research/limitations.md A.2 |
| H2-C modern: pooled/conditional/LPM, baselines, placebo, negative control, forward chaining, LORO, calibration, Oster/Cinelli–Hazlett |
DONE |
h2_modern_primary.csv, _robustness.csv, _cv_*, h2_modern_cv_predictions.parquet |
| H4 (hinge/spline OOS) both panels |
DONE |
h4_seshat_summary.json, h4_modern_summary.json, h4_modern_curve.csv |
| H5 (held-out blocks) both panels |
DONE |
h5_seshat_blocks.csv, h5_modern_blocks.csv |
| Holm correction across the three primary tests |
DONE |
results/empirical/holm_primary_tests.csv (all adjusted p < 0.01; the H2 tests nevertheless fail the pre-registered out-of-sample rule) |
| Identification note |
DONE |
research/identification.md, paper §5.5 |
# Phase 5 — Simulations
| Task |
Status |
Evidence |
| ABM with heterogeneous agents, networks, bandwidth, shocks, five imposed regimes |
DONE |
src/wp35/simulations/model.py |
| Factorial design (6,048 runs, 8 seeds) + ablations (584) |
DONE |
results/simulated/runs_main.parquet, runs_ablation.parquet |
| Predefined metrics incl. bits per success, overhead, Gini, latency; Shannon quantities where defined |
DONE |
simulation_summary.json, cells_main.csv |
| Fragility search (cells where institutions lose) |
DONE |
institution_loss_shares.csv, cells_institution_vs_none.csv |
| Theorem checks in silico |
DONE |
simulation_summary.json → theorem_checks |
# Phase 6 — Integration
| Task |
Status |
Evidence |
| Observable vs non-observable mechanisms; rivals; consistency; scope conditions |
DONE |
research/integration.md, paper §9 |
| Task |
Status |
Evidence |
| All tables/figures generated by script |
DONE |
src/wp35/visualization/make_all.py (26 fragments, 15 figures) |
| Tests |
DONE |
tests/ — 11 passed after revision |
| Clean-environment full reproduction |
PARTIAL |
every target was run in sequence from the venv in this session; a from-scratch make all on a second machine was not run |
# Phase 8 — Paper
| Task |
Status |
Evidence |
| LaTeX manuscript compiled, references resolved, figures present |
DONE |
paper/main.pdf 80 pp. after revision; 0 undefined citations/references; 8 small overfull boxes |
| Abstract written after analyses |
DONE |
— |
# Phase 9 — Adversarial review
| Task |
Status |
Evidence |
| Three independent reviews (institutional economist, historical anthropologist, computational social scientist) |
DONE — all three: major revision |
docs/reviews/review_*.md |
| Response to reviewers and implemented corrections |
DONE — 25 points fixed with reruns (A-002…A-007), 6 accepted as unfixable and stated, 3 declined with reasons |
docs/reviews/response_to_reviewers.md; research/decisions.md |
| Defects found by review that violated repository rule 1 (text–results mismatch) |
FIXED |
five robustness variants never computed → computed; placebo tautology → respecified; failed CV folds misreported → repaired |
# Blocked / requires PI
- Creation of the spbgit remote
wp35_uqo.git and git push (authorization).
- Any external publication (SSRN) and the website entry (
spboucher.ai).
- Equinox-2020 replication (not blocked, simply not executed).
# Headline verdict (for the record, after revision)
H1 supported as association (all datasets). H2 not supported beyond baselines in either panel (pre-registered rule);
the restriction β_D+β_K=0 is rejected in the modern panel; the Seshat placebo is not distinguishable from P; higher K
predicts shorter polity durations (against the hypothesis). H3 supported in operating-cost sense (SIMULATED; success
effects are parameters), refuted in total-cost sense at short horizons; central hierarchies fragile at equal budgets.
H4 rejected in Seshat; not rejected but small in the modern panel. H5: sign stable in every held-out block (regions, eras,
cultural spheres, periods). Universal framing abandoned per pre-registration §7.