SPB Git

spb/wp28_uqo

Public
6.2 KB · 65 lines latex
1\section{Main results: the order--round frontier}\label{sec:results}23\subsection{One Newton step contracts quadratically}4The global correction is a classical one-step estimator \citep{lecam1956asymptotic,bickel1975one,vandervaart1998asymptotic}, with the difference that the comparison point is $\thetaN$ rather than $\theta_0$. Under \cref{ass:newton}, for any $\vartheta_N \pto \theta_0$ the Newton map $\mathcal N_N(\vartheta) := \vartheta - H_N(\vartheta)^{-1}g_N(\vartheta)$ satisfies5\begin{equation}\label{eq:contraction}6\norm{\mathcal N_N(\vartheta_N) - \thetaN} \le \tfrac12 M_N\Lambda_N \norm{\vartheta_N - \thetaN}^27\end{equation}8on an event of probability tending to one (\cref{lem:newton}), and $q$ iterates satisfy $\norm{\iter{}{q} - \thetaN} \le (\tfrac12 M_N\Lambda_N)^{2^q-1}\norm{\iter{}{0} - \thetaN}^{2^q}$ (\cref{lem:qsteps}). Applied to the pilot bound \eqref{eq:pilot}, repeated squaring acts on all three terms:9\begin{equation}\label{eq:qbound}10\norm{\iter{r}{q} - \thetaN} = \op\!\Big(\Big[m^{-r} + \frac{1}{m\sqrt K} + \frac1N\Big]^{2^q}\Big).11\end{equation}1213\subsection{The frontier}14\begin{theorem}[Order--round frontier]\label{thm:frontier}15Let \cref{ass:lin,ass:bias,ass:newton} hold with order $r \ge 1$ and fixed $q \ge 0$, let $m \to \infty$ and16\[17K = o\big(N^{\alphafr{r}{q}}\big), \qquad \alphafr{r}{q} = 1 - \frac{1}{2^{q+1} r}.18\]19Then $\sqrt N\,(\iter{r}{q} - \thetaN) \pto 0$, and consequently $\sqrt N(\iter{r}{q} - \theta_0) \dto \mathcal N(0,\Sigma_\psi)$.20\end{theorem}2122\begin{proof}[Proof sketch]23By \eqref{eq:qbound}, $\sqrt N\norm{\iter{r}{q} - \thetaN}$ is bounded by a sum of three terms, which with $m = N/K$ read24\begin{equation}\label{eq:three-terms}25\sqrt N\, m^{-r2^q} = N^{1/2}\Big(\frac KN\Big)^{r2^q}, \qquad26\sqrt N\Big(\frac{\sqrt K}{N}\Big)^{2^q} = N^{1/2}\,K^{2^{q-1}}N^{-2^q}, \qquad \sqrt N\, N^{-2^q}.27\end{equation}28The first vanishes if and only if $K^{r2^q} = o(N^{r2^q - 1/2})$, that is $K = o(N^{\alphafr{r}{q}})$. The second equals $\sqrt{K/N}$ for $q = 0$ and vanishes because $\alphafr{r}{0} < 1$; for $q \ge 1$ it is at most $N^{1/2 - 2^{q-1}} \le N^{-1/2}$ whenever $K \le N$. The third always vanishes. Consistency of the pilot, needed to apply \eqref{eq:contraction}, follows from \eqref{eq:pilot} and $m \to \infty$. The limit law follows from \cref{ass:newton} and Slutsky's lemma. \Cref{app:proofs} gives the details, including the event on which all iterates stay in $U$.29\end{proof}3031\begin{table}[t]32\centering\small33\begin{threeparttable}34\caption{Sufficient exponents $\alphafr{r}{q}$ in $K = o(N^{\alpha})$ (\cref{thm:frontier}), by local order $r$ and number of aggregated Newton corrections $q$.}\label{tab:exponents}35\begin{tabular}{@{}lcccc@{}}36\toprule37 & $q = 0$ & $q = 1$ & $q = 2$ & $q = 3$ \\38\midrule39$r = 1$ (plain local MLE) & $1/2$ & $3/4$ & $7/8$ & $15/16$ \\40$r = 2$ (half-block jackknife) & $3/4$ & $7/8$ & $15/16$ & $31/32$ \\41$r = 3$ (quarter-block, or $(1,2,3)$) & $5/6$ & $11/12$ & $23/24$ & $47/48$ \\42$r = 4$ (eighth-block, or $(1,2,3,4)$) & $7/8$ & $15/16$ & $31/32$ & $63/64$ \\43\bottomrule44\end{tabular}45\begin{tablenotes}\footnotesize46\item The exponent depends on $(r,q)$ only through $D = 2^q r$; cells on an anti-diagonal of constant $D$ coincide. These are sufficient conditions under fixed $p$; they are not machine counts for a given $N$, not minimax statements and not evidence of speed.47\end{tablenotes}48\end{threeparttable}49\end{table}5051\Cref{tab:exponents} displays the frontier. Three readings are useful. First, the cells $(1,0)$, $(1,1)$ and $(1,1)$-with-local-curvature are the regimes of \citet{zhang2013communication}, \citet{huang2019distributed} (whose stated condition is the weaker $K = O(N^{1/2})$, while the $k^4N^{-4}$ remainder of their MSE bound already implies $3/4$) and \citet{zhu2021least}; the table extends them in both directions. Second, the exponent is symmetric in the exchange ``one more round $\leftrightarrow$ double the local order'': $(2,1)$, $(1,2)$ and $(4,0)$ all give $7/8$. Which of the three is cheaper is a question about the hardware, not about statistics, and is the subject of \cref{sec:cost,sec:bench}. Third, the dominant restriction is always the deterministic bias term; the random remainder and the full-sample term are never binding for $q \ge 1$ and bind only through $K = o(N)$ for $q = 0$. This is a statement about rates, not about finite samples: in \cref{sec:mc} the random term is visible as variance inflation at moderate $m$.5253\subsection{Inference}54\begin{corollary}[Inferential validity]\label{cor:inference}55Under the conditions of \cref{thm:frontier}, let $\widehat\Sigma_N \pto \Sigma_\psi$. Then for every coordinate $j$ and level $1-\tau$ the interval56\begin{equation}\label{eq:wald}57\iter{r}{q,j} \;\pm\; z_{1-\tau/2}\,\sqrt{\widehat\Sigma_{N,jj}/N}58\end{equation}59has asymptotic coverage $1 - \tau$. Under correct specification $\widehat\Sigma_N = -H_N(\iter{r}{q})^{-1}$, which is available at no extra communication from the last derivative phase (for $q \ge 1$); under misspecification the sandwich $H_N^{-1}\,\widehat J_N\,H_N^{-1}$ with $\widehat J_N = K^{-1}\sum_k\abs{D_k}^{-1}\sum_{i\in D_k}\nabla\log f(X_i;\theta)\nabla\log f(X_i;\theta)^\top$ requires one additional $p(p+1)/2$ upload per worker in that phase.60\end{corollary}61The corollary says nothing about estimators outside the frontier: when $K/N^{\alphafr{r}{q}}$ does not vanish, the point estimator has a non-centered limit (\cref{thm:exponential}) and the interval under-covers, as the coverage tables of \cref{sec:mc} show for the plain average and the one-step estimator at small $m$.6263\subsection{What the theorem does not say}64The theorem is a sufficient statement under high-level assumptions; it gives neither the finite-sample behaviour (the constants $C$, $C_b$, $M_N$, $\Lambda_N$, $\Amp$ and $\betar$ all matter at moderate $m$) nor any comparison between protocols at equal cost. The exponent cannot be improved \emph{for this construction} (\cref{thm:exponential}), but other constructions can do better: in exponential families the pooled sufficient statistic reaches $\thetaN$ exactly for every $K \le N$, and multi-round schemes with $q \to \infty$ remove the restriction on $K$ entirely at the price of unbounded synchronization \citep{jordan2019communication,fan2023communication}.65
Files · 960926063a6ca5ed53de00e9616d11db178d3b22
Menu

Theme