WP35 data dictionary (processed files)
Built by python -m wp35.harmonization.build_panels (2026-11-01). Raw provenance and SHA-256: source_registry.json.
Counts below are copied from results/diagnostics/coverage_summary.json and the coverage CSVs produced by the same
run; they are not typed by hand elsewhere. Missing values are NaN everywhere; no zero stands for "unknown".
A. Seshat Polaris-2026 (historical polities)
seshat_facts.parquet — one row per fact (56,274 rows; 2,503 dated; 132 unparsed values logged)
| column | meaning |
|---|---|
| sheet, section | Polaris sheet (Social complexity / Warfare / Religion / General) and section code (sc, wf, rt, general) |
| seshat_region, polity_number, polity_id | Seshat identifiers (42 regions, 626 polities) |
| variable | original Polaris variable_name |
| value | presence code mapped P/IP→1, A/IA→0, disputed (A~P)→0.5, SU/U/unknown→NaN; numeric variables → midpoint (geometric for population/territory/largest settlement; arithmetic for levels) |
| value_lo, value_hi | numeric bounds from value_from/value_to (ranges kept) |
| text | raw text for General-sheet categorical facts |
| year_from, year_to, dated | fact date interval; dated=False ⇒ fact describes the whole polity period (24,401 of 26,033 SC facts) |
| inferred, disputed, unparsed | IP/IA flag; A~P-type flag; value not in the code vocabulary (logged to results/diagnostics/seshat_unparsed_values.csv) |
| is_uncertain, is_disputed | Polaris flags |
seshat_polity_agg.parquet — polity × variable aggregates (all facts)
value_mean (mean of coded values across facts), value_min/max, logv_mean (mean log midpoint for geometric
variables), lo_min, hi_max, n_facts, n_dated, n_undated, n_inferred, n_disputed, n_flag_uncertain.
seshat_polity_wide.parquet = pivot of value_mean. seshat_precrisis_agg.parquet = same aggregates computed per
CrisisDB record using only facts with year_to < crisis_start or undated facts (crisis_id column).
seshat_polity_meta.parquet — one row per polity (626)
long_name, seshat_region, homeNGA, start_year, end_year, macro_region (10, configs/regions.yaml), midpoint,
era (6 blocks), duration, censored (end_year ≥ 1900: 30 polities), centralization
(polity_degree_of_centralization), relationship_to_preceding, preceding_entity, succeeding_entity,
suprapolity_relations, utm_zone, language_family, successor_id (matched for 285 polities),
successor_relationship, y3_discontinuous (coded for 207 polities; 37 = 1).
seshat_crisis.parquet — CrisisDB "Crisis consequences" (166 records, 150 polities)
crisis_id, polity_id, other_polity_id, crisis_start, crisis_end (= year_from, year_to), item values <item>_v
(same code mapping), y1_n_coded, y1_n_present, y1_severity (= present / coded over the 14 instability items
listed in the pre-registration; coded for all 166), y1b_severe (any of collapse, fragmentation, century_plus,
conquest, civil_war present; defined when all five coded: 158 records, 122 positive), y_reform, epidemic_v.
Codes in Polaris: P, A, IP, IA, SU, DIS (treated as missing).
Coverage of the pre-registered indicators (share of the 626 polities missing): results/diagnostics/seshat_missingness_by_indicator.csv;
e.g. written_record 12%, settlement_hierarchy 16%, administrative_level 21%, polity_population 52%, measurement systems 82–86%.
Polities with ≥ 50% of the K indicators coded: 376; with any D scale variable: 561; with population: 299; with territory: 355.
B. D-PLACE Ethnographic Atlas v3.3 — dplace_ea_societies.parquet (1,291 societies)
soc_id, Name, Latitude, Longitude, Glottocode, region (50), main_focal_year (−2000…1965), xd_id,
Family_ID (Glottolog 5.3 family; isolates keep their own glottocode; 173 families), Macroarea,
EA030, EA032, EA033, EA034, EA042, EA066, EA067, EA068, EA070, EA072, EA073, EA090, EA005, EA028, EA113 (ordinal code
numbers from codes.csv ord; "Missing data" → NaN), ea031_code, EA031_mid (geometric midpoint of the category
range parsed from the code label; open categories use the bound, see results/diagnostics/dplace_code_midpoints.csv),
EA202_mid = EA202 (continuous population size, 953 coded), log_community_size (611 coded), log_population,
jurisdictional_hierarchy_beyond (= EA033 ord − 1 : 0 acephalous … 4 four levels; 1,155 coded), jurisdictional_hierarchy_local.
C. Modern panel — modern_panel.parquet (28,092 country-years, 202 countries, 1789–2025)
V-Dem v16 columns kept verbatim (country_name, country_text_id, country_id, year, e_regionpol_6C, e_regionpol, v2x_polyarchy, e_civil_war, e_miinterc, e_coups, e_pt_coup, v2regendtype, v2regint, v2regdur, v2regidnr, e_pop, e_area, e_gdppc, v2x_regime, v2x_rule, K items v2clrspct, v2stcritrecadm, v2cltrnslw, v2stfisccap, v2svstterr, v2elembcap).
Maddison 2023 merged by ISO3 (country_text_id = countrycode; 163 matched, 39 V-Dem countries unmatched, list in
crosswalk_vdem_maddison.csv): maddison_pop (thousands; 15,821 country-years, from 1820), maddison_gdppc.
Derived: log_pop, log_area, log_gdppc, y4_civilwar_onset (e_civil_war 0→1; 176 onsets / 13,608 coded,
1817–2006), y5_coup (e_coups ≥ 1; 599 / 13,813, 1789–2008), regime_end_year (last year of a v2regidnr),
y6_irregular_regime_end (regime end year with v2regendtype ∈ {0 military coup, 1 other coup, 2 self-coup,
3 assassination, 5 loss in civil war, 6 loss in inter-state war, 7 foreign intervention, 8 popular uprising};
1,041 / 27,846). Codebook readings are in research/measurement.md.