# 🧭 The Social Compression Hypothesis **UQO Working Paper No. 35** — *A pre-registered test of coordination demand, institutional capacity, and instability in historical and modern societies, with a formal framework and agent-based experiments* [-b31b1b?logo=latex&logoColor=white)](paper/main.pdf) [](pyproject.toml) [](data/raw/README.md) [](Makefile) [](research/preregistration.md) [](https://uqo.ca) [](mailto:simon-pierre.boucher@uqo.ca) > **TL;DR** — Can the mismatch between a society's coordination *demand* (scale) and its coordination *capacity* (administration, standardized rules and money, communication infrastructure, record-keeping) predict instability? Under a pre-registration frozen before any outcome was examined, we built the indices from Seshat Polaris-2026 (626 polities, 127 CrisisDB crises with pre-crisis predictors), the Ethnographic Atlas (1,291 societies) and a V-Dem/Maddison panel (14,637 country-years, 2,692 events). Capacity and scale are positively associated everywhere. The mismatch index is positively associated with crisis severity (0.20 log-odds/SD) and modern instability onsets (0.40/SD), stable across regions and eras — **but fails the pre-registered test**: it does not beat the scale-only baseline (Seshat) or the capacity-only baseline (modern) out of sample, a placebo does as well, past events load on it as strongly as future ones, and an Oster adjustment flips its sign. Five theorems make "institutions as codes" precise; 6,632 agent-based runs confirm them in silico and locate where institutions hurt (unpaid set-up, saturated hierarchies, unstructured task spaces). The universal theory is not supported; the narrower paper is what we wrote. --- ## 📌 Status | Item | Value | |---|---| | Version | 1.0 (November 2026) | | SSRN | not yet submitted | | Journal | — | | Code & data | fully reproducible from the raw downloads (`make all`); raw files not redistributed (D-PLACE CC BY-NC, V-Dem terms); SHA-256 registry included | | Process | executed autonomously by Claude Code (Fable 5.1) under the PI's mandate (`docs/MANDATE_VERBATIM.txt`); PI review pending | ## 🎯 What this paper does The Social Compression Hypothesis reads institutions as devices that lower the communication cost per coordinated task. The paper (i) states that reading as five theorems (coding gain, break-even scale, stale-code penalty, hierarchical routing, backlog dynamics) and three explicit conjectures; (ii) pre-registers and runs the first out-of-sample test of the demand/capacity mismatch conjecture of Shin et al. (2020) on real historical and modern data; (iii) runs factorial agent-based experiments with five imposed institutional regimes where messages, bits, queues and overhead are measured directly. ``` P_it = z(log D_it) - z(K_it) # mismatch index (K standardized, not logged: bounded share) E[Y1] = Λ(α + β P + γ'X + μ_region + τ_era) # Seshat crisis severity (fractional logit) Pr(Y456 in (t,t+5]) = Λ(α + β P_it + γ'X_it + μ_r + τ_t) # modern onsets (logit, country-clustered) ``` ## 🏆 Headline results | Object | Estimate | |---|---| | corr(z(D), z(K)) across Seshat polities | 0.84 — P retains 17% of the joint variance | | H1 Seshat: D on K (region + era FE) | 0.75 SD per SD of K (SE-based 95% CI excludes 0); EA: 0.70 log community size per jurisdictional level; modern within-country 0.15 | | H2-A Seshat crisis severity, β_P | 0.196 (SE 0.056, n = 127); with covariates 0.204 (0.085, n = 87); scale-only 0.257 (0.138); placebo 0.257 (0.138) | | H2-A out of sample | beats FE-only / K-only / complexity in ≥ 80% of folds; **not** scale-only (75%), persistence (69%), D+K (75%); loses to scale alone in 7/10 leave-region-out folds | | H2-C modern Y456, β_P | 0.398 (0.089, n = 14,637); capacity-only −0.632 (0.126); negative control (past events) 0.436 vs forward 0.475; Oster-adjusted −0.024; RV = 0.07 | | H2-C out of sample | never beats capacity-only (forward chaining 0/2; LORO 1/6 log-lik) | | H4 threshold | Seshat: hinge/spline beat linear in 10%/8% of folds; modern: 80%/87% but +0.001 nats (negligible) | | H5 | β_P positive in every held-out macro-region, era, region and period | | Y2 polity hazard / Y3 succession | no association with P (HR 1.00–1.03) | | H3 (SIMULATED) | norms/records lower operating bits per success in 99.7% / 71% of paired runs and raise success in 99.3% / 100%; central hierarchy at half capacity: success −0.44; a 200-bit code is repaid only with s = 256, Zipf 1.5, T = 160; stale-code penalty 1.90 bits vs KL 1.98 | ## 🗂 Repository layout ``` social-compression-hypothesis/ ├── CLAUDE.md, docs/MANDATE_VERBATIM.txt # mandate and binding rules ├── research/ # preregistration (frozen), literature_review, framework (theorems), │ │ # measurement, identification, integration, limitations, decisions, │ │ # prior_art.csv (Crossref-verified), bib_overrides.csv ├── configs/sources.yaml, regions.yaml # source registry inputs; macro-regions and eras ├── data/{raw,interim,processed,metadata}/ # raw (not committed; SHA-256 in metadata/source_registry.json) ├── src/wp35/ │ ├── acquisition/fetch.py # downloads + hashes │ ├── harmonization/{seshat,dplace,modern,build_panels}.py │ ├── measurement/{irt,build_indices}.py # D, K, P, 2PL IRT, invariance, power │ ├── econometrics/{common,h1,h2_seshat,h2_modern,run_all}.py │ ├── simulations/{model,run_experiments,analyze}.py # SIMULATED │ ├── visualization/make_all.py # every table fragment and figure │ └── validation/{verify_bib,build_bib}.py # Crossref verification → references.bib ├── results/{empirical,simulated,diagnostics,logs}/ # machine-readable outputs, execution log, audit ├── tests/ # 11 tests (parsers, IRT, CV utilities, simulation invariants) ├── paper/ # main.tex, sections/, appendix/, tables/, figures/, references.bib → main.pdf └── replication/ # replication guide ``` ## 🚀 Quick start ```bash uv venv -p 3.12 .venv && uv pip install -p .venv/bin/python -e . make fetch && make data && make measure && make econ && make sim && make figures && make paper make test ``` About 25 minutes end-to-end on an Apple-silicon laptop (the modern-panel estimation dominates); the simulations take about one minute on 12 cores. ## 🗃 Data | Source | Release obtained (2026-11-01) | Use | |---|---|---| | Seshat Global History Databank | Polaris-2026 (GitHub `build_polaris_dataset`, commit 55ca5fbc2b, MIT) + Equinox-2020 spreadsheet + Zenodo 8120128 | historical polities, CrisisDB crisis consequences | | D-PLACE Ethnographic Atlas | v3.3 (CC BY-NC 4.0) + Glottolog 5.3 families | ethnographic cross-section (H1 only) | | V-Dem | v16 via the official `vdemdata` R package (commit f4dd26922e) | modern institutions and outcomes | | Maddison Project Database | 2023 (dataverse.nl 421302/421303) | modern population and GDP per capita | Coverage, missingness and the data dictionary: `results/diagnostics/`, `data/metadata/data_dictionary.md`. Missing is never zero; nothing is forward-filled; no outcome is imputed. ## 🔬 Methodology 1. `acquisition/fetch.py` — registry-driven downloads with SHA-256. 2. `harmonization/*` — long facts table with explicit uncertainty; polity, society and country-year panels; outcomes. 3. `measurement/build_indices.py` — D (PC1 of logged scale), K (four disjoint blocks; IRT alternative), P; invariance; bounds; a-priori power. 4. `econometrics/h1.py` — OLS/mixed (Seshat), Conley + ordered logit (EA), within/differences (modern). 5. `econometrics/h2_seshat.py` — fractional logit, baselines, placebo, 10×5 CV + leave-region-out, hinge/spline, H5 blocks, Cox, succession. 6. `econometrics/h2_modern.py` — pooled/conditional logit and LPM, forward chaining, LORO, negative control, placebo, Oster/Cinelli–Hazlett. 7. `simulations/*` — ABM with five imposed regimes; factorial 6,048 + 584 ablation runs; Shannon quantities where defined. 8. `visualization/make_all.py` — all 26 table fragments and 15 figures from results. ## 🖼 Figures & tables inventory | Figure | Content | |---|---| | `fig_seshat_temporal_coverage`, `fig_seshat_missingness` | polity spans, crisis years, coded/missing indicators | | `fig_measurement_DK`, `fig_irt_invariance` | D vs K, block vs latent K, item difficulties, Old/New World invariance | | `fig_h2_coefplot`, `fig_h5_forest`, `fig_h4_curve`, `fig_calibration_modern`, `fig_trajectories` | main estimates, held-out blocks, shape, OOS calibration, P over time | | `fig_sim_scaling`, `fig_sim_phase`, `fig_sim_breakeven`, `fig_sim_shocks` | SIMULATED scaling, saturation, break-even, shock responses | ## ⚠️ Limitations Crisis-selected outcome; 94% of Seshat facts undated; presence codes; D is scale only; corr(D,K) = 0.84; modern K correlates 0.75 with democracy and is expert-coded ex post; no causal identification; Equinox-2020 replication not executed. Full list: `research/limitations.md`. ## 📚 Citation ```bibtex @techreport{boucher2026wp35, author = {Boucher, Simon-Pierre}, title = {The Social Compression Hypothesis: A Pre-Registered Test of Coordination Demand, Institutional Capacity, and Instability in Historical and Modern Societies}, institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives}, type = {UQO Working Paper}, number = {35}, year = {2026}, month = {November} } ``` ## 👤 Author & contact Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca — Département des sciences administratives, UQO. ## 📝 Notes Numbering: WP 35 (next free number after WP 34). Category folder `economie/` created for this paper. Adversarial reviews and responses: `docs/reviews/`. Final audit: `results/logs/final_research_audit.md`.