1# Referee report — UQO WP 35, "The Social Compression Hypothesis" (v1.0, Nov. 2026)23**Reviewer persona:** senior institutional economist (transaction costs, state capacity, political economy of institutions, applied econometrics).4**Materials read:** `paper/main.pdf` (abstract, §1, §3–§7, §9–§11); `research/preregistration.md`, `measurement.md`, `identification.md`, `decisions.md`; the five result files named in the brief plus `h2_modern_cv_folds.csv`, `h2_seshat_cv_folds.csv`, `h2_seshat_cv_noleakage.csv`; `src/wp35/measurement/build_indices.py`, `src/wp35/econometrics/common.py`, `h2_seshat.py`, `h2_modern.py`, `leakage_check.py`.56## 1. Summary78The paper reads institutions as devices that compress the communication cost of coordination, formalises this in five standard information-theoretic results plus three conjectures (§3), and pre-registers a test of the empirical corollary: a mismatch index P = z(log D) − z(K) between coordination demand (scale) and coordination capacity (administration, rules/money, communication infrastructure, records) should predict instability beyond scale-only, capacity-only, complexity-only and persistence baselines, judged out of sample by an 80%-of-folds rule. On Seshat Polaris-2026 (127 CrisisDB records) and a V-Dem–Maddison panel (14,637 country-years), P is positively associated with crisis severity (0.20/SD, Table 9) and with five-year instability onsets (0.40/SD, Table 14), but fails the out-of-sample rule against scale alone (Seshat) and capacity alone (modern); a placebo performs comparably, the negative control loads as strongly as the forward test, and an Oster adjustment crosses zero. The paper concludes the universal framing is unsupported and keeps only "scale matters, capacity matters." The candour and pre-registration are welcome; the problems below concern whether the tests test what they claim.910## 2. Major issues1112**M1. D and K are not substantively disjoint, and K is size-biased by construction.** The sets are *nominally* disjoint (prereg §3.1–3.2), but `settlement_hierarchy` (in D) and `administrative_level` (in K) are both hierarchy depth, and most K items are polity-level *presence-anywhere* codes (a road, a court, a coin "exists somewhere"). The probability that at least one instance exists rises mechanically with territory and population, so K is a saturating function of D before any economics enters. This, not only co-evolution, explains much of corr(D,K) = 0.84 (Table 5), makes the H1 "primary test" unfailable, and contaminates P at the measurement stage. The paper notes presence ≠ reach (p. 53) but not this implication. Fix: normalise presence items by log territory (item-level logit with a scale offset) or restrict K to intensity items, and re-run H1 with hierarchy items removed from both sides. K–Y overlap is also real in the modern panel: `v2svstterr` (territorial control) is close to a contemporaneous measure of civil war, and V-Dem coders rate K knowing what followed (acknowledged p. 53, not addressed).1314**M2. The Seshat placebo is a tautology.** `h2_seshat.py` line 37 defines `P_placebo = z(D) − z(X_prod_infra)`; line 54 fits it with `XS = X_warfare + X_prod_infra + epidemic_v` as controls. With `X_prod_infra` both inside the placebo and a free regressor, the model is an exact reparametrisation of B1 (D only + XS). `h2_seshat_primary.csv` confirms it: both coefficients equal 0.25723724…, with identical SE, n = 94 and log-likelihood −37.6295. The sentence "the placebo … is associated with severity at 0.26 — the same as scale alone" (p. 33) and the abstract's "a placebo index performs comparably" carry zero evidential content. Conceptually it is also poorly chosen: irrigation, markets and food storage are coordination institutions (Ostrom). The modern placebo (polyarchy for K, run without the polyarchy control, line 72) is valid.1516**M3. P is not a coherent economic object, and the paper already contains the better test.** ρ = D/K needs common units (Def. 4); P imposes the exchange rate 1 SD of D = 1 SD of K, which has no theoretical basis (the "Limitation" box, p. 13, concedes this). The empirical content of P is the restriction β_D = −β_K in z-units. B5 reports the unrestricted coefficients: Seshat 0.34 (SE 0.16) and −0.21 (SE 0.19); modern 0.17 (SE 0.07) and −0.71 (SE 0.14). In Seshat the restriction is *not* rejected and capacity has the predicted sign; in the modern panel it is rejected because capacity dominates. A Wald test of β_D + β_K = 0, or a capacity-shortfall index K − E[K | D] (orthogonal to scale by construction), would replace the fold-counting apparatus with one interpretable number. Also, C3 "P given K" is algebraically B5 (same log-likelihood −5627.075 in `h2_modern_primary.csv`) yet p. 38 presents it as a separate finding, and Table 9's row "A3: Y1 ~ P + D + K + X + FE" is mislabelled — the code (`h2_seshat.py` line 48) conditions on `complexity_pc1_z`; P + D + K would be perfectly collinear.1718**M4. Rival mechanisms are weakly represented.** The "warfare technology index" is a technology-presence share, not military competition (wars fought, neighbours' power) in the Tilly/Gennaioli–Voth sense; elite overproduction is absent (acknowledged); fiscal capacity is absent from Seshat K but inside modern K (`v2stfisccap`), so that rival is inseparable in both panels. Worse, `epidemic_v` is itself a CrisisDB consequence item coded with the outcome (`h2_seshat.py` line 26), so A2 conditions on a component of the outcome vector. Adding X drops 40 of 127 records (A2 n = 87) while B1–B4 use n = 93–100, so p. 28's "fitted on the same observations" holds in CV but not in Table 9.1920**M5. K–democracy collinearity, negative control and Oster: wrong lessons drawn.** corr(K, polyarchy) = 0.75; `v2elembcap`, `v2cltrnslw`, `v2clrspct` are components of V-Dem's liberal/electoral indices, so "controlling for polyarchy" partials out a near-component of K (C2 0.72 → C1 0.40). The negative control was pre-registered as "must not show a *stronger* association than the forward spec" (prereg §5.4). It did not: 0.44 (SE 0.10) past vs 0.48 (SE 0.10) future. By the authors' own rule it *passed*; p. 39's claim that "the pre-registration said in advance that a negative control of this kind would be read that way" is inaccurate — amend the rule (logged) or report the pass. The control is also uninformative on its own terms: instability destroys capacity (§3.8), so a backward association is expected under the hypothesis and under confounding alike. The Oster exercise treats the drop from adding polyarchy/GDP as proportional selection (δ = 1), implausible when the key observable is quasi-collinear with the regressor; the abstract's "an Oster adjustment reverses its sign" overclaims an assumption-laden point estimate. The Cinelli–Hazlett RV = 0.072 is twelve times P's partial R² of 0.006 and should be benchmarked against observed covariates.2122**M6. The 80% rule is a sign test with granularity problems, and the reported fold counts are wrong.** The rule ignores magnitude (mean Δ log-lik per observation +0.002 to +0.011 in Seshat, Table 11). With 10 LORO folds it means 8/10; with 6 modern regions 5/6 passes and 4/6 fails; with forward chaining it means "win both" — and the first forward split (train ≤ 1945) **failed for every model** with a statsmodels predict error (`h2_modern_cv_folds.csv`; `folds = 2` in `h2_modern_cv_wins.csv`), yet p. 39 says "in any of the three temporal splits" and Table 16's caption says "three temporal splits". Seshat's "fifty cross-validation folds" (p. 33) are 48 (rep0_fold1 and rep9_fold1 errored). Further: B5 (D + K) *nests* P, so requiring P to beat it out of sample asks a one-parameter restriction to beat its unrestricted version — P winning 75% of folds against B5 is mild evidence *for* the restriction; and random 5-fold CV is not blocked by polity although 16 polities contribute multiple records with identical undated whole-polity P (`load_crisis`), so LORO is the only clean scheme. Use paired fold-difference tests plus the Wald test (M3), repair the failed folds, and correct the text.2324**M7. Holm and the "three primary tests".** H1-Seshat cannot realistically fail (collinearity 0.84 was known from Turchin et al. 2017), so one of the three Holm-corrected primaries is a free pass. More seriously, the Holm correction is never computed: `common.py` defines `holm()` but no script calls it and no result file contains it; p. 32's "survives the Holm correction" is asserted, not produced. Since the binding criterion is the out-of-sample rule (no p-value), the multiple-testing apparatus is decorative. H4-modern met the pre-registered rule (hinge 12/15 = 0.80, spline 0.87, `h4_modern_summary.json`) but pp. 42–43 reinterpret it as "negligible" — a post-hoc deviation not logged in `decisions.md` (only A-001 is).2526**M8. Small-sample inference.** A2 fits about 20 parameters (10 macro-regions, 6 eras, 3 covariates) on 87 fractional observations with 31 clusters; no wild-cluster bootstrap or randomisation inference; Y1b needed era effects dropped for separation; the fractional logit does not weight by the number of coded items (3-item and 14-item records count equally); H5 within-block estimates on 5–13 records (Table 12) are noise.2728**M9. Leakage in cross-validation: real but immaterial.** `build_indices.py` fits the D loadings (`pc1_pairwise`, line 65), the z-scores of D and K (lines 88–89), the IRT item parameters (line 80) and the complexity PC1 (line 123) on the full polity sample before any fold is drawn; `cv_compare` in `common.py` then uses these fixed columns. This is transductive preprocessing, but it is outcome-blind, an affine rescaling of one regressor is absorbed exactly by the GLM slope and intercept, it is shared by every compared model, and the authors' own fold-internal check (`leakage_check.py`; `h2_seshat_cv_noleakage.csv`: P beats D-only in 2/10 vs 3/10) confirms the verdict is unchanged. The same logic covers modern forward chaining; say so once. The CV problems that matter are in M6.2930**M10. Over- and under-claiming.** Overclaims: the placebo sentence (M2), "reverses its sign" (M5), "three temporal splits"/"fifty folds" (M6), "survives the Holm correction" (M7), and "reproduces every number … with one command" (p. 55) while two folds and one split silently error. Underclaim: the interesting narrow result — in Seshat capacity given scale has the predicted sign and the restriction survives; in the modern panel capacity dominates and it fails — is buried under a fold-counting verdict.3132## 3. Minor issues33341. Table 9 row A3 label vs code (M3); Table 16 caption "three temporal splits"; p. 33 "fifty folds".352. `t_since` for countries with no observed prior event is set to the sample maximum (`h2_modern.py` line 52); the comment says "flagged in the paper" — I could not find the flag.363. `K_modern` averages whichever of six items are coded (≥ 4); item composition changes over time, so K is not era-comparable without an item-availability control.374. The Seshat persistence baseline (`prior_crisis_nga`) is built from the crisis-selected sample itself.385. Whole-polity territory and population are means over facts, so fragmentation or conquest (Y1 items) pulls D down — circularity in the other direction.396. Conjecture 1's falsifier promises the share of polities whose P flips sign under alternative K weightings; it is not reported.407. Bibliography: ALL-CAPS author names (WRIGHT and JOHNSON, STASAVAGE, FEARON and LAITIN, CEDERMAN); Table 3 cells truncated; p. 3 blank.418. The "Process" paragraph (p. 54: PI has not reviewed code or text) belongs on the title page.4243## 4. Questions to the authors44451. Re-run the Seshat placebo without `X_prod_infra` in the controls: coefficient and LORO win share?462. Wald p-value of β_D + β_K = 0 in B5, each panel, pooled and under LORO?473. Why did the train ≤ 1945 forward split fail, and what are the results once fixed?484. Where is the Holm computation, and what are the three adjusted p-values?495. With K items normalised by log territory (or intensity-only), what is corr(D, K) and does H1 still pass?506. Why is H4-modern reported as rejected when the pre-registered rule was met?5152## 5. Recommendation: **major revision**5354The project is well-conceived and unusually honest, and the negative conclusion may well survive, but several headline "falsifications" are implementation artefacts (M2, M6, M7) or misreadings of the paper's own rules (M5, M7), and P is tested through a restriction the paper never states as such (M3). Mandatory changes:55561. Remove or correctly re-estimate the Seshat placebo and rewrite every sentence that relies on it (abstract, pp. 33, 51–52).572. Make the unrestricted D + K model the primary object, with a Wald test of β_D + β_K = 0 and paired fold-difference tests; repair the failed CV folds and the forward split, block k-fold by polity, and correct all fold counts.583. Compute and report the Holm correction or delete the claim; report the negative control and H4-modern against the pre-registered rules as written, logging any reinterpretation as a numbered amendment.594. Address the size bias of presence-anywhere K items (M1) with at least one normalised or intensity-only K, and discuss the D/K hierarchy overlap.605. Fix Table 9's A3 label, present C3 as the reparametrisation of B5 it is, fit the Table 9 baselines on the common sample, and remove `epidemic_v` from the controls (or justify conditioning on a consequence item).61