1\section{The economics of synchronization}\label{sec:cost}23The frontier of \cref{tab:exponents} makes $(2,1)$, $(1,2)$ and $(4,0)$ statistically interchangeable at the level of exponents. Choosing among them is a question of cost, and the cost model below is proposed, calibrated and tested, not proved. Let $T_{\mathrm{fit}}(n,p)$ be the elapsed time of one local MLE fit on $n$ observations, $T_{\mathrm{der}}(m,p)$ that of one local score-and-Hessian evaluation, $\lambda$ the latency of one synchronized exchange (broadcast plus the slowest worker's reply, excluding payload), $B$ the effective uplink bandwidth at the coordinator, $b$ the bytes per transmitted number and $T_{\mathrm{solve}}(p)$ the coordinator's linear solve. For the doubling pattern the local phase of order $r$ performs $2^{r}-1$ fits touching every local observation $r$ times, so that with a fit cost $T_{\mathrm{fit}}(n,p) \approx c_0(p) + c_1(p)\,n$ (fixed overhead plus a linear pass, the empirical shape of Newton fits with a bounded number of iterations),4\begin{equation}\label{eq:cost}5\begin{aligned}6T(r,q) \;\approx\;& \underbrace{(2^r - 1)\,c_0 + r\,c_1\, m}_{\text{local phase (slowest worker)}}7\;+\; q\,\Big[\underbrace{\lambda + T_{\mathrm{der}}(m,p) + \frac{K\,b\,(p + p(p+1)/2)}{B} + T_{\mathrm{solve}}(p)}_{\text{one synchronized derivative phase}}\Big] \\8&+\; T_{\mathrm{sched}}(K) + T_{\mathrm{straggle}}(K),9\end{aligned}10\end{equation}11where the last two terms collect worker start-up and the excess of the slowest worker over the mean. Sums are critical-path sums: the local fits of different workers overlap, the $q$ derivative phases do not.1213\begin{proposition}[Exchange rate between a local level and a round, under \eqref{eq:cost}]\label{prop:cost}14Hold the effective depth $D = 2^q r$ fixed. Moving from $(r,q)$ to $(2r, q-1)$ removes one derivative phase and adds the local levels $r+1,\dots,2r$. Under \eqref{eq:cost} the round is cheaper iff15\begin{equation}\label{eq:rule}16\lambda + T_{\mathrm{der}}(m,p) + \frac{Kb(p + p(p+1)/2)}{B} + T_{\mathrm{solve}}(p) \;<\; (2^{2r} - 2^{r})\,c_0 + r\,c_1 m .17\end{equation}18The right-hand side grows linearly in $m$ and exponentially in $r$; the left-hand side grows linearly in $K$ and quadratically in $p$. For fixed $N$ and small $p$ the rule favours local levels when $K$ is large and $m$ is small, and favours rounds when $m$ is large. Holding statistical accuracy fixed in finite samples requires, in addition, the variance inflation of higher local orders (\cref{sec:mc}) to be accounted for.19\end{proposition}2021The proposition is a tautology given \eqref{eq:cost}; its content is the calibration. \Cref{sec:bench} measures every term of \eqref{eq:cost} on the cluster---fit and derivative times by $m$ and $p$, per-phase latency percentiles, bytes per phase, start-up and straggler excess---and reports where \eqref{eq:rule} lands for the designs run. We do not model overlapping of computation and communication, asynchronous updates (which change the estimator) or energy; these are listed as open in \cref{sec:discussion}.22