# WP35 data dictionary (processed files)

Built by `python -m wp35.harmonization.build_panels` (2026-11-01). Raw provenance and SHA-256: `source_registry.json`.
Counts below are copied from `results/diagnostics/coverage_summary.json` and the coverage CSVs produced by the same
run; they are not typed by hand elsewhere. Missing values are `NaN` everywhere; no zero stands for "unknown".

## A. Seshat Polaris-2026 (historical polities)

### `seshat_facts.parquet` — one row per fact (56,274 rows; 2,503 dated; 132 unparsed values logged)
| column | meaning |
|---|---|
| sheet, section | Polaris sheet (Social complexity / Warfare / Religion / General) and section code (sc, wf, rt, general) |
| seshat_region, polity_number, polity_id | Seshat identifiers (42 regions, 626 polities) |
| variable | original Polaris `variable_name` |
| value | presence code mapped P/IP→1, A/IA→0, disputed (A~P)→0.5, SU/U/unknown→NaN; numeric variables → midpoint (geometric for population/territory/largest settlement; arithmetic for levels) |
| value_lo, value_hi | numeric bounds from `value_from`/`value_to` (ranges kept) |
| text | raw text for General-sheet categorical facts |
| year_from, year_to, dated | fact date interval; `dated=False` ⇒ fact describes the whole polity period (24,401 of 26,033 SC facts) |
| inferred, disputed, unparsed | IP/IA flag; A~P-type flag; value not in the code vocabulary (logged to `results/diagnostics/seshat_unparsed_values.csv`) |
| is_uncertain, is_disputed | Polaris flags |

### `seshat_polity_agg.parquet` — polity × variable aggregates (all facts)
`value_mean` (mean of coded values across facts), `value_min/max`, `logv_mean` (mean log midpoint for geometric
variables), `lo_min`, `hi_max`, `n_facts`, `n_dated`, `n_undated`, `n_inferred`, `n_disputed`, `n_flag_uncertain`.
`seshat_polity_wide.parquet` = pivot of `value_mean`. `seshat_precrisis_agg.parquet` = same aggregates computed per
CrisisDB record using only facts with `year_to < crisis_start` or undated facts (`crisis_id` column).

### `seshat_polity_meta.parquet` — one row per polity (626)
`long_name, seshat_region, homeNGA, start_year, end_year, macro_region` (10, `configs/regions.yaml`), `midpoint`,
`era` (6 blocks), `duration`, `censored` (end_year ≥ 1900: 30 polities), `centralization`
(`polity_degree_of_centralization`), `relationship_to_preceding`, `preceding_entity`, `succeeding_entity`,
`suprapolity_relations`, `utm_zone`, `language_family`, `successor_id` (matched for 285 polities),
`successor_relationship`, `y3_discontinuous` (coded for 207 polities; 37 = 1).

### `seshat_crisis.parquet` — CrisisDB "Crisis consequences" (166 records, 150 polities)
`crisis_id, polity_id, other_polity_id, crisis_start, crisis_end` (= `year_from`, `year_to`), item values `<item>_v`
(same code mapping), `y1_n_coded`, `y1_n_present`, `y1_severity` (= present / coded over the 14 instability items
listed in the pre-registration; coded for all 166), `y1b_severe` (any of collapse, fragmentation, century_plus,
conquest, civil_war present; defined when all five coded: 158 records, 122 positive), `y_reform`, `epidemic_v`.
Codes in Polaris: P, A, IP, IA, SU, DIS (treated as missing).

Coverage of the pre-registered indicators (share of the 626 polities missing): `results/diagnostics/seshat_missingness_by_indicator.csv`;
e.g. written_record 12%, settlement_hierarchy 16%, administrative_level 21%, polity_population 52%, measurement systems 82–86%.
Polities with ≥ 50% of the K indicators coded: 376; with any D scale variable: 561; with population: 299; with territory: 355.

## B. D-PLACE Ethnographic Atlas v3.3 — `dplace_ea_societies.parquet` (1,291 societies)
`soc_id, Name, Latitude, Longitude, Glottocode, region` (50), `main_focal_year` (−2000…1965), `xd_id`,
`Family_ID` (Glottolog 5.3 family; isolates keep their own glottocode; 173 families), `Macroarea`,
`EA030, EA032, EA033, EA034, EA042, EA066, EA067, EA068, EA070, EA072, EA073, EA090, EA005, EA028, EA113` (ordinal code
numbers from `codes.csv` `ord`; "Missing data" → NaN), `ea031_code`, `EA031_mid` (geometric midpoint of the category
range parsed from the code label; open categories use the bound, see `results/diagnostics/dplace_code_midpoints.csv`),
`EA202_mid` = `EA202` (continuous population size, 953 coded), `log_community_size` (611 coded), `log_population`,
`jurisdictional_hierarchy_beyond` (= EA033 ord − 1 : 0 acephalous … 4 four levels; 1,155 coded), `jurisdictional_hierarchy_local`.

## C. Modern panel — `modern_panel.parquet` (28,092 country-years, 202 countries, 1789–2025)
V-Dem v16 columns kept verbatim (`country_name, country_text_id, country_id, year, e_regionpol_6C, e_regionpol,
v2x_polyarchy, e_civil_war, e_miinterc, e_coups, e_pt_coup, v2regendtype, v2regint, v2regdur, v2regidnr, e_pop, e_area,
e_gdppc, v2x_regime, v2x_rule`, K items `v2clrspct, v2stcritrecadm, v2cltrnslw, v2stfisccap, v2svstterr, v2elembcap`).
Maddison 2023 merged by ISO3 (`country_text_id` = `countrycode`; 163 matched, 39 V-Dem countries unmatched, list in
`crosswalk_vdem_maddison.csv`): `maddison_pop` (thousands; 15,821 country-years, from 1820), `maddison_gdppc`.
Derived: `log_pop`, `log_area`, `log_gdppc`, `y4_civilwar_onset` (e_civil_war 0→1; 176 onsets / 13,608 coded,
1817–2006), `y5_coup` (e_coups ≥ 1; 599 / 13,813, 1789–2008), `regime_end_year` (last year of a `v2regidnr`),
`y6_irregular_regime_end` (regime end year with `v2regendtype` ∈ {0 military coup, 1 other coup, 2 self-coup,
3 assassination, 5 loss in civil war, 6 loss in inter-state war, 7 foreign intervention, 8 popular uprising};
1,041 / 27,846). Codebook readings are in `research/measurement.md`.