# 🧭 The Social Compression Hypothesis

UQO Working Paper No. 35 — A pre-registered test of coordination demand, institutional capacity, and instability in historical and modern societies, with a formal framework and agent-based experiments

TL;DR — Can the mismatch between a society's coordination demand (scale) and its coordination capacity (administration, standardized rules and money, communication infrastructure, record-keeping) predict instability? Under a pre-registration frozen before any outcome was examined, we built the indices from Seshat Polaris-2026 (626 polities, 127 CrisisDB crises with pre-crisis predictors), the Ethnographic Atlas (1,291 societies) and a V-Dem/Maddison panel (14,637 country-years, 2,692 events). Capacity and scale are positively associated everywhere. The mismatch index is positively associated with crisis severity (0.20 log-odds/SD) and modern instability onsets (0.39/SD), stable across regions, eras and cultural spheres — but fails the pre-registered test: it does not beat the scale-only baseline (Seshat) or the capacity-only baseline (modern) out of sample, the equal-and-opposite restriction it imposes is rejected in the modern panel, a placebo is not distinguishable from it in Seshat, past events load on it nearly as strongly as future ones, and it is not robust to an Oster-type adjustment; higher capacity even predicts shorter polity durations in Seshat. Five theorems make "institutions as codes" precise; 6,632 agent-based runs confirm them in silico and locate where institutions hurt (unpaid set-up, saturated hierarchies, unstructured task spaces). The universal theory is not supported; the narrower paper is what we wrote.

# 📌 Status

# 🎯 What this paper does

The Social Compression Hypothesis reads institutions as devices that lower the communication cost per coordinated task. The paper (i) states that reading as five theorems (coding gain, break-even scale, stale-code penalty, hierarchical routing, backlog dynamics) and three explicit conjectures; (ii) pre-registers and runs the first out-of-sample test of the demand/capacity mismatch conjecture of Shin et al. (2020) on real historical and modern data; (iii) runs factorial agent-based experiments with five imposed institutional regimes where messages, bits, queues and overhead are measured directly.

P_it = z(log D_it) - z(K_it)        # mismatch index (K standardized, not logged: bounded share)
E[Y1] = Λ(α + β P + γ'X + μ_region + τ_era)             # Seshat crisis severity (fractional logit)
Pr(Y456 in (t,t+5]) = Λ(α + β P_it + γ'X_it + μ_r + τ_t)  # modern onsets (logit, country-clustered)

# 🏆 Headline results

# 🗂 Repository layout

social-compression-hypothesis/
├── CLAUDE.md, docs/MANDATE_VERBATIM.txt     # mandate and binding rules
├── research/                                # preregistration (frozen), literature_review, framework (theorems),
│   │                                        # measurement, identification, integration, limitations, decisions,
│   │                                        # prior_art.csv (Crossref-verified), bib_overrides.csv
├── configs/sources.yaml, regions.yaml       # source registry inputs; macro-regions and eras
├── data/{raw,interim,processed,metadata}/   # raw (not committed; SHA-256 in metadata/source_registry.json)
├── src/wp35/
│   ├── acquisition/fetch.py                 # downloads + hashes
│   ├── harmonization/{seshat,dplace,modern,build_panels}.py
│   ├── measurement/{irt,build_indices}.py   # D, K, P, 2PL IRT, invariance, power
│   ├── econometrics/{common,h1,h2_seshat,h2_modern,run_all}.py
│   ├── simulations/{model,run_experiments,analyze}.py   # SIMULATED
│   ├── visualization/make_all.py            # every table fragment and figure
│   └── validation/{verify_bib,build_bib}.py # Crossref verification → references.bib
├── results/{empirical,simulated,diagnostics,logs}/      # machine-readable outputs, execution log, audit
├── tests/                                   # 11 tests (parsers, IRT, CV utilities, simulation invariants)
├── paper/                                   # main.tex, sections/, appendix/, tables/, figures/, references.bib → main.pdf
└── replication/                             # replication guide

# 🚀 Quick start

uv venv -p 3.12 .venv && uv pip install -p .venv/bin/python -e .
make fetch && make data && make measure && make econ && make sim && make figures && make paper
make test

About 25 minutes end-to-end on an Apple-silicon laptop (the modern-panel estimation dominates); the simulations take about one minute on 12 cores.

# 🗃 Data

Coverage, missingness and the data dictionary: results/diagnostics/, data/metadata/data_dictionary.md. Missing is never zero; nothing is forward-filled; no outcome is imputed.

# 🔬 Methodology

1. acquisition/fetch.py — registry-driven downloads with SHA-256.
2. harmonization/* — long facts table with explicit uncertainty; polity, society and country-year panels; outcomes.
3. measurement/build_indices.py — D (PC1 of logged scale), K (four disjoint blocks; IRT alternative), P; invariance; bounds; a-priori power.
4. econometrics/h1.py — OLS/mixed (Seshat), Conley + ordered logit (EA), within/differences (modern).
5. econometrics/h2_seshat.py — fractional logit, baselines, placebo, 10×5 CV + leave-region-out, hinge/spline, H5 blocks, Cox, succession.
6. econometrics/h2_modern.py — pooled/conditional logit and LPM, forward chaining, LORO, negative control, placebo, Oster/Cinelli–Hazlett.
7. simulations/* — ABM with five imposed regimes; factorial 6,048 + 584 ablation runs; Shannon quantities where defined.
8. visualization/make_all.py — all 26 table fragments and 15 figures from results.

# 🖼 Figures & tables inventory

# ⚠️ Limitations

Crisis-selected outcome; 94% of Seshat facts undated; presence codes; D is scale only; corr(D,K) = 0.84; modern K correlates 0.75 with democracy and is expert-coded ex post; no causal identification; Equinox-2020 replication not executed. Full list: research/limitations.md.

# 📚 Citation

@techreport{boucher2026wp35,
  author      = {Boucher, Simon-Pierre},
  title       = {The Social Compression Hypothesis: A Pre-Registered Test of Coordination Demand, Institutional Capacity, and Instability in Historical and Modern Societies},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {UQO Working Paper},
  number      = {35},
  year        = {2026},
  month       = {November}
}

# 👤 Author & contact

Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca — Département des sciences administratives, UQO.

# 📝 Notes

Numbering: WP 35 (next free number after WP 34). Category folder economie/ created for this paper. Adversarial reviews (institutional economist, historical anthropologist, computational social scientist) and the response: docs/reviews/ — they found real defects (a tautological placebo, silently failed CV folds, mislabelled rows, un-run robustness variants, simulation accounting) that were corrected (amendments A-002 to A-007 in research/decisions.md). Final audit: results/logs/final_research_audit.md.