SPB Git

spb/wp35_uqo

Public
9.3 KB · 130 lines latex
1\section{Results: modern states, 1820--2025}2\label{sec:modern}34The modern panel is the form of the test that the mandate's event-risk model has in mind: a country-year panel with lagged predictors, fixed effects, forward-chaining validation and held-out regions. All estimates are \EMP\ and associational.56\subsection{H2: instability events in the next five years}7\label{sec:modern-h2}89In the primary specification (Table~\ref{tab:h2-modern}), a one-standard-deviation higher mismatch is associated with $0.40$ higher log-odds of at least one civil-war onset, coup or irregular regime end in the following five years (SE 0.09; 14,637 country-years, 2,692 event windows), conditional on development, electoral democracy, time since the last event, region and year. The coefficient is $0.72$ without covariates, $1.31$ in a conditional logit with country and decade effects (which uses only countries that experience at least one event window), and $0.11$ in probability units in a linear model with country and year effects. It is positive for every horizon (0.33 at one year, 0.39 at three, 0.45 at ten), largest for civil-war onsets (1.01, SE 0.20), small and imprecise for coups (0.20, SE 0.14) and intermediate for irregular regime ends (0.25, SE 0.09); it does not depend on the capacity aggregation, on standardizing within years, on adding land area to demand, on dropping the nineteenth century, on removing the democracy control or on adding the persistence control (Table~\ref{tab:h2-modern-robust}). Every held-out region and period gives a pooled coefficient between $0.29$ and $0.47$, and every within-block estimate is positive (Table~\ref{tab:h5-modern}, Figure~\ref{fig:forest}). The second H2 primary test also passes the conventional bar.1011\begin{table}[H]12\centering\footnotesize13\caption{H2 in the modern panel: any instability event in $(t,t+5]$. Logit with region and year effects and country-clustered standard errors unless stated; $X$ = log GDP per capita, electoral democracy, cubic in years since the last event. ``Events'' counts country-years whose five-year window contains an event.}14\label{tab:h2-modern}15\begin{tabular}{L{8.4cm} l r r r r}16\toprule17Specification & Term & Coef. & SE & $n$ & Events \\18\midrule19\input tables/tab_h2_modern 20\bottomrule21\end{tabular}22\end{table}2324The baselines, placebo and negative control tell the rest (Table~\ref{tab:h2-modern}, Figure~\ref{fig:coefplot}). Population alone is weakly associated with instability ($0.13$, SE 0.07); capacity alone is strongly and negatively associated ($-0.63$, SE 0.13), as is development ($-0.56$ per log unit of GDP per capita). With population and capacity entered separately, population carries $0.17$ (SE 0.07) and capacity $-0.71$ (SE 0.14); conditional on capacity, the mismatch adds $0.23$ (SE 0.09). The signal in $P$ is, here, overwhelmingly the capacity component with the sign reversed. The placebo that replaces capacity with electoral democracy is associated at $0.20$ (SE 0.06) against $0.45$ (SE 0.08) for $P$ with the same controls, so the capacity items do carry information beyond regime type, which is reassuring for the measurement model but not for the hypothesis. The negative control is not reassuring: past events, in the five years before $t$, are associated with $P_t$ at $0.44$ (SE 0.10), against $0.48$ (SE 0.10) for future events with the same controls. A predictor that is as strongly related to what has already happened as to what will happen is a marker of chronically unstable countries, not a leading indicator, and the pre-registration said in advance that a negative control of this kind would be read that way.2526\begin{table}[H]27\centering\footnotesize28\caption{Pre-registered robustness set for the modern primary coefficient (same specification as the primary row of Table~\ref{tab:h2-modern} unless stated).}29\label{tab:h2-modern-robust}30\begin{tabular}{L{6.8cm} l r r r r}31\toprule32Variant & Term & Coef. & SE & $n$ & Events \\33\midrule34\input tables/tab_h2_modern_robust 35\bottomrule36\end{tabular}37\end{table}3839Out of sample (Table~\ref{tab:h2-modern-cv}, Figure~\ref{fig:calibration}) the verdict is the same as in Seshat with the roles of the components exchanged. Under forward chaining, the model with $P$ has a slightly higher area under the curve than the fixed-effects model (0.773 against 0.768) and than population alone, but a lower one than capacity alone (0.777) and than population and capacity together (0.779), and it never beats the capacity-only model on log-likelihood or discrimination in any of the three temporal splits; with regions held out it beats capacity alone in one of six folds on log-likelihood and three of six on discrimination. All models are well calibrated under forward chaining (slopes 0.95--0.99) and all are badly calibrated when a whole region is held out (slopes about 0.43), which is itself a finding: the level of instability risk is regional, and no index built from scale and capacity transports it. Under the pre-registered 80\% rule, $P$ fails against the capacity-only and the scale-plus-capacity baselines in both schemes, and the second H2 primary test is therefore not passed.4041\begin{table}[H]42\centering\footnotesize43\caption{Out-of-sample validation in the modern panel. Top: share of folds in which the model with $P$ beats the baseline on the metric; the rule requires 0.80. Bottom: mean metrics by scheme and model (forward chaining: three temporal splits; LORO: six held-out regions).}44\label{tab:h2-modern-cv}45\begin{tabular}{l l l r r r}46\toprule47Baseline & Scheme & Metric & Folds & Win share & Mean $\Delta$ \\48\midrule49\input tables/tab_h2_modern_cv 50\bottomrule51\end{tabular}\\[3mm]52\begin{tabular}{l l r r r r}53\toprule54Scheme & Model & OOS log-lik. & AUC & Brier & Calibration slope \\55\midrule56\input tables/tab_h2_modern_cv_summary 57\bottomrule58\end{tabular}59\end{table}6061\begin{figure}[H]62\centering63\includegraphics[width=\textwidth]{figures/fig_calibration_modern.pdf}64\caption{Out-of-sample calibration in the modern panel (decile means of predicted probability against observed event shares) for the model with $P$, the capacity-only baseline and the fixed-effects baseline, under forward chaining (left) and leave-one-region-out (right).}65\label{fig:calibration}66\end{figure}6768\begin{table}[H]69\centering\footnotesize70\caption{H5 in the modern panel: $\beta_P$ with one region or period held out (pooled), and estimated within the held-out block.}71\label{tab:h5-modern}72\begin{tabular}{L{5.0cm} r r r r l}73\toprule74Held-out block & Pooled $\beta_P$ & SE & $n$ & Within $n$ & Within-block (SE) \\75\midrule76\input tables/tab_h5_modern_blocks 77\bottomrule78\end{tabular}79\end{table}8081\begin{figure}[H]82\centering83\includegraphics[width=\textwidth]{figures/fig_h5_forest.pdf}84\caption{H5: the mismatch coefficient with each macro-region or era (Seshat, left) and each region or period (modern panel, right) held out, and the coefficient estimated inside the held-out block where possible.}85\label{fig:forest}86\end{figure}8788\subsection{Sensitivity to unobservables}89\label{sec:modern-sens}9091Table~\ref{tab:sensitivity} applies Oster's and Cinelli--Hazlett's diagnostics to the linear-probability version of the primary model. Adding the controls moves the coefficient from $0.082$ to $0.038$ while raising $R^2$ from 0.127 to 0.161; the bias-adjusted coefficient at $R_{\max}=1.3R^2$ and $\delta=1$ is $-0.024$, on the other side of zero. The partial $R^2$ of $P$ is 0.006 and the robustness value is 0.072: an unobserved confounder that explained seven percent of the residual variance of both the mismatch and the outcome would suffice to remove the association. Under the conventions of the sensitivity literature this is a fragile coefficient.9293\begin{table}[H]94\centering\footnotesize95\caption{Sensitivity of the modern primary coefficient to unobservables (linear probability model with the primary controls).}96\label{tab:sensitivity}97\begin{tabular}{L{10cm} r}98\toprule99Quantity & Value \\100\midrule101\input tables/tab_sensitivity 102\bottomrule103\end{tabular}104\end{table}105106\subsection{H4: a nonlinearity of negligible size}107\label{sec:modern-h4}108109With country-blocked folds, the hinge improves on the linear form in 12 of 15 folds and the spline in 13 (Table~\ref{tab:h4}), which meets the pre-registered 80\% rule by one fold. The improvement is 0.001 nats per observation; the in-sample curve (Figure~\ref{fig:h4curve}) is a mild convexity around the median rather than a step, and the hinge knot is at $-0.08$ standard deviations, i.e.\ at the centre of the distribution. We report this as a marginal nonlinearity consistent with the smooth convexity of Theorem~\ref{thm:backlog} and not as a threshold; the pre-registration's falsifier for H4 was a hinge or step model that \emph{outperforms} the smooth one, and a spline is a smooth model.110111\begin{table}[H]112\centering\footnotesize113\caption{H4: linear against hinge (knot chosen on the training fold) and natural cubic spline, out of sample.}114\label{tab:h4}115\begin{tabular}{L{5.4cm} r r r r r r}116\toprule117Panel & Folds & Hinge wins & Spline wins & Log-lik.\ linear & hinge & spline \\118\midrule119\input tables/tab_h4 120\bottomrule121\end{tabular}122\end{table}123124\begin{figure}[H]125\centering126\includegraphics[width=0.6\textwidth]{figures/fig_h4_curve.pdf}127\caption{Shape of the relation between $P$ and the five-year event probability in the modern panel, at covariate means, linear and spline fits.}128\label{fig:h4curve}129\end{figure}130
Files · 39ef4252c6e9baf1f0a3a4809ca1ab5579d2895d
Menu

Theme