1\section{Measured distributed benchmarks}\label{sec:bench}23\subsection{Setup}4The protocol of \cref{alg:protocol} was run as written---a coordinator process and $K$ worker processes exchanging length-prefixed messages over TCP---on the MacLustr cluster of Apple-silicon Macs connected by a wired local network (mixed 1 and 10~GbE through one switch), with the coordinator on a Mac Studio (M4 Max, 16 cores, 64 GB) and workers on up to sixteen distinct hosts (Mac Studio M3 Ultra, M2 Ultra, M4 Max, M5 Max; Mac mini M4, M4 Pro, M5 Pro, M6; MacBook Pro M4 Max; the coordinator host also hosts one worker in the sixteen-host designs). Workers run Python 3.9 or 3.14 with NumPy 2.0--2.5; each simulates its own block of a logistic regression with $N = 2^{22}$ observations from a seed, so that no data move. The coordinator logs the wall-clock of each stage, every worker's local fit and derivative times, the reply latency of every worker in every phase (so that the slowest worker and the 99th percentile are known), and the bytes of every message including frame headers and serialization. Every design was repeated three times, the host-scaling endpoints six times (\benchRuns{} runs in total); \cref{tab:bench} reports medians and \cref{fig:bench} the decomposition of the elapsed time. Worker start-up over \texttt{ssh} and the connection phase (about \benchConnect{} seconds) are excluded from the protocol times and reported separately.56\begin{figure}[t]7\centering8\includegraphics[width=\textwidth]{fig_bench_exchange.pdf}9\caption{Measured elapsed time of the protocol by variant (medians over three repeats). First three panels: local phase (slowest worker) and synchronized derivative phases at equal exponent $7/8$ for $(4,0)$, $(3,1)$, $(2,1)$, $(1,2)$ and the stale and single-machine curvature variants of $(2,1)$, logistic regression with $p = 8$, $32$ and $64$, $N = 2^{22}$, $K = 16$ workers on 16 hosts. Right: $(2,1)$ with $K = 16$ workers spread over 1, 2, 4, 8 and 16 hosts.}\label{fig:bench}10\end{figure}1112\subsection{What the measurements show}13Three facts emerge, all specific to this hardware and implementation. First, the synchronized derivative phase is cheap: with $p = 8$ it takes \benchRoundTwoOne{} ms end to end on sixteen hosts, of which the slowest worker's score-and-Hessian evaluation accounts for \benchDerMaxPeight{} ms and the network round trip for about \benchLatencyExcess{} ms; the uplink of \benchUpTwoOne{} kB is irrelevant at these sizes. With $p = 64$ the phase takes \benchRoundPsixtyfourTwoOne{} s, almost all of it the slowest worker's computation of the $64 \times 64$ Hessian on $2^{18}$ observations (\benchDerMaxPsixtyfour{} s), with an uplink of \benchUpPsixtyfourTwoOne{} kB. At the intermediate dimension $p = 32$ the derivative phase takes \benchRoundPthirtytwoTwoOne{} s and the local phase \benchLocalPthirtytwoOneTwo{} s for one fit, \benchLocalPthirtytwoTwoOne{} s for the half-block pattern and \benchLocalPthirtytwoThreeOne{} s for the quarter-block pattern, so that $(1,2)$ costs $\benchLocalPthirtytwoOneTwo + \benchRoundsPthirtytwoOneTwo$ s against $\benchLocalPthirtytwoTwoOne + \benchRoundPthirtytwoTwoOne$ s for $(2,1)$. Second, the local phase dominates and grows with the order: at $p = 8$ the slowest worker needs \benchLocalOneTwo{} s for one fit, \benchLocalTwoOne{} s for the three fits of the half-block pattern, \benchLocalThreeOne{} s for the seven fits of the quarter-block pattern and \benchLocalFourZero{} s for the fifteen fits of order four; at $p = 64$ the half-block pattern costs \benchLocalPsixtyfourTwoOne{} s against \benchLocalPsixtyfourOneTwo{} s for one fit. Third, the exchange rate of \cref{prop:cost} therefore favours rounds on this network: at equal exponent $7/8$, $(1,2)$ is the fastest protocol at both dimensions ($\benchLocalOneTwo + \benchRoundsOneTwoS$ s against $\benchLocalTwoOne + \benchRoundTwoOneS$ s for $(2,1)$ at $p = 8$; $\benchLocalPsixtyfourOneTwo + \benchRoundsPsixtyfourOneTwo$ s against $\benchLocalPsixtyfourTwoOne + \benchRoundPsixtyfourTwoOne$ s at $p = 64$), and it is also the more accurate one (\cref{sec:mc}). The stale and single-machine variants save nothing here, because the derivative phase they shorten was already short; their statistical cost (\cref{thm:inexact}) is paid for no gain. In terms of \eqref{eq:rule}, the measured $\lambda + T_{\mathrm{der}} + Kb(p + p(p+1)/2)/B + T_{\mathrm{solve}}$ is two to ten times smaller than $r\,c_1 m$ at every $(r, p)$ tested, so the inequality holds and one more round beats one more level of local fits.1415\subsection{Hosts, processes and stragglers}16Spreading $K = 16$ workers from one host to sixteen (\cref{fig:bench}, right) does not speed up the local phase: its wall-clock is the slowest worker's, and the slowest of sixteen heterogeneous machines (a Mac mini) is slower than the slowest of sixteen processes on one Mac Studio (\benchLocalHostsSixteen{} s against \benchLocalHostsOne{} s), while the mean worker fit time is about the same (\benchFitMeanHostsSixteen{} s). The derivative phase roughly doubles from one host to sixteen (\benchRoundHostsOne{} ms to \benchRoundTwoOne{} ms), again through the slowest worker rather than through the network. With $K = 64$, $128$ and $256$ workers on sixteen hosts (four, eight and sixteen processes per host, $m = 2^{16}$, $2^{15}$ and $2^{14}$) the local phase falls to \benchLocalKonetwentyeight{} s and \benchLocalKtwofiftysix{} s for $(2,1)$ and the derivative phase to \benchRoundKonetwentyeight{} ms and \benchRoundKtwofiftysix{} ms (99th-percentile reply latency \benchLatNinetyNineKtwofiftysix{} ms with 256 connections): more partitions mean smaller blocks and faster local fits, but by \cref{thm:frontier} they also move the estimator towards the frontier, and the trade is statistical, not computational. The straggler excess, the gap between the slowest and the mean worker, is the quantity that an implementation should attack first; in these runs it comes from hardware heterogeneity rather than from the network.1718\subsection{Limits of the benchmark}19The measurements are for one cluster, one Python/NumPy implementation with pickle framing, one network, and $K \le 128$; they do not establish a general speed advantage of any protocol. On a network with latency of tens of milliseconds, or with bandwidth-limited uplinks of dense Hessians at large $p$, the left-hand side of \eqref{eq:rule} grows and the balance can tip towards local levels; the cost model is provided so that \eqref{eq:rule} can be evaluated with the parameters of a given installation. No energy or monetary cost was measured, and none is claimed.2021\input{tables/bench}22\FloatBarrier23