SPB Git

spb/wp28_uqo

Public
6.0 KB · 56 lines latex
1\section{Main results: the order--round frontier}\label{sec:results}23\subsection{One Newton step contracts quadratically}4The global correction is a classical one-step estimator \citep{lecam1956asymptotic,bickel1975one,vandervaart1998asymptotic}, with the difference that the comparison point is $\thetaN$ rather than $\theta_0$. Under \cref{ass:newton}, for any $\vartheta_N \pto \theta_0$ the Newton map $\mathcal N_N(\vartheta) := \vartheta - H_N(\vartheta)^{-1}g_N(\vartheta)$ satisfies5\begin{equation}\label{eq:contraction}6\norm{\mathcal N_N(\vartheta_N) - \thetaN} \le \tfrac12 M_N\Lambda_N \norm{\vartheta_N - \thetaN}^27\end{equation}8on an event of probability tending to one (\cref{lem:newton}), and $q$ iterates satisfy $\norm{\iter{}{q} - \thetaN} \le (\tfrac12 M_N\Lambda_N)^{2^q-1}\norm{\iter{}{0} - \thetaN}^{2^q}$ (\cref{lem:qsteps}). Applied to the pilot bound \eqref{eq:pilot}, repeated squaring acts on all three terms:9\begin{equation}\label{eq:qbound}10\norm{\iter{r}{q} - \thetaN} = \op\!\Big(\Big[m^{-r} + \frac{1}{m\sqrt K} + \frac1N\Big]^{2^q}\Big).11\end{equation}1213\subsection{The frontier}14\begin{theorem}[Order--round frontier]\label{thm:frontier}15Let \cref{ass:lin,ass:bias,ass:newton} hold with order $r \ge 1$ and fixed $q \ge 0$, let $m \to \infty$ and16\[17K = o\big(N^{\alphafr{r}{q}}\big), \qquad \alphafr{r}{q} = 1 - \frac{1}{2^{q+1} r}.18\]19Then $\sqrt N\,(\iter{r}{q} - \thetaN) \pto 0$, and consequently $\sqrt N(\iter{r}{q} - \theta_0) \dto \mathcal N(0,\Sigma_\psi)$.20\end{theorem}2122\begin{proof}[Proof sketch]23By \eqref{eq:qbound}, $\sqrt N\norm{\iter{r}{q} - \thetaN}$ is bounded by a sum of three terms. The bias term is $\sqrt N\,m^{-r2^q} = N^{1/2}(K/N)^{r2^q}$, which vanishes iff $K^{r2^q} = o(N^{r2^q - 1/2})$, i.e.\ iff $K = o(N^{\alphafr{r}{q}})$. The random term is $\sqrt N(\sqrt K/N)^{2^q}$: for $q = 0$ it equals $\sqrt{K/N}$ and vanishes because $\alphafr{r}{0} < 1$; for $q \ge 1$ it is at most $N^{1/2 - 2^{q-1}} \le N^{-1/2}$ whenever $K \le N$. The full-sample term $\sqrt N N^{-2^q}$ vanishes. Consistency of the pilot, needed to apply \eqref{eq:contraction}, follows from \eqref{eq:pilot} and $m \to \infty$. The limit law follows from \cref{ass:newton} and Slutsky's lemma. \Cref{app:proofs} gives the details, including the event on which all iterates stay in $U$.24\end{proof}2526\begin{table}[t]27\centering\small28\begin{threeparttable}29\caption{Sufficient exponents $\alphafr{r}{q}$ in $K = o(N^{\alpha})$ (\cref{thm:frontier}), by local order $r$ and number of aggregated Newton corrections $q$.}\label{tab:exponents}30\begin{tabular}{@{}lcccc@{}}31\toprule32 & $q = 0$ & $q = 1$ & $q = 2$ & $q = 3$ \\33\midrule34$r = 1$ (plain local MLE) & $1/2$ & $3/4$ & $7/8$ & $15/16$ \\35$r = 2$ (half-block jackknife) & $3/4$ & $7/8$ & $15/16$ & $31/32$ \\36$r = 3$ (quarter-block, or $(1,2,3)$) & $5/6$ & $11/12$ & $23/24$ & $47/48$ \\37$r = 4$ (eighth-block, or $(1,2,3,4)$) & $7/8$ & $15/16$ & $31/32$ & $63/64$ \\38\bottomrule39\end{tabular}40\begin{tablenotes}\footnotesize41\item The exponent depends on $(r,q)$ only through $D = 2^q r$; cells on an anti-diagonal of constant $D$ coincide. These are sufficient conditions under fixed $p$; they are not machine counts for a given $N$, not minimax statements and not evidence of speed.42\end{tablenotes}43\end{threeparttable}44\end{table}4546\Cref{tab:exponents} displays the frontier. Three readings are useful. First, the cells $(1,0)$, $(1,1)$ and $(1,1)$-with-local-curvature are the regimes of \citet{zhang2013communication}, \citet{huang2019distributed} (whose stated condition is the weaker $K = O(N^{1/2})$, while the $k^4N^{-4}$ remainder of their MSE bound already implies $3/4$) and \citet{zhu2021least}; the table extends them in both directions. Second, the exponent is symmetric in the exchange ``one more round $\leftrightarrow$ double the local order'': $(2,1)$, $(1,2)$ and $(4,0)$ all give $7/8$. Which of the three is cheaper is a question about the hardware, not about statistics, and is the subject of \cref{sec:cost,sec:bench}. Third, the dominant restriction is always the deterministic bias term; the random remainder and the full-sample term are never binding for $q \ge 1$ and bind only through $K = o(N)$ for $q = 0$. This is a statement about rates, not about finite samples: in \cref{sec:mc} the random term is visible as variance inflation at moderate $m$.4748\subsection{Inference}49\begin{corollary}[Inferential validity]\label{cor:inference}50Under the conditions of \cref{thm:frontier}, let $\widehat\Sigma_N \pto \Sigma_\psi$. Then for every $j$ and level $1-\tau$ the interval $\iter{r}{q,j} \pm z_{1-\tau/2}\,(\widehat\Sigma_{N,jj}/N)^{1/2}$ has asymptotic coverage $1 - \tau$. Under correct specification $\widehat\Sigma_N = -H_N(\iter{r}{q})^{-1}$, which is available at no extra communication from the last derivative phase (for $q \ge 1$); under misspecification the sandwich $H_N^{-1}\,\widehat J_N\,H_N^{-1}$ with $\widehat J_N = K^{-1}\sum_k\abs{D_k}^{-1}\sum_{i\in D_k}\nabla\log f(X_i;\theta)\nabla\log f(X_i;\theta)^\top$ requires one additional $p(p+1)/2$ upload per worker in that phase.51\end{corollary}52The corollary says nothing about estimators outside the frontier: when $K/N^{\alphafr{r}{q}}$ does not vanish, the point estimator has a non-centered limit (\cref{thm:exponential}) and the interval under-covers, as the coverage tables of \cref{sec:mc} show for the plain average and the one-step estimator at small $m$.5354\subsection{What the theorem does not say}55The theorem is a sufficient statement under high-level assumptions; it gives neither the finite-sample behaviour (the constants $C$, $C_b$, $M_N$, $\Lambda_N$, $\Amp$ and $\betar$ all matter at moderate $m$) nor any comparison between protocols at equal cost. The exponent cannot be improved \emph{for this construction} (\cref{thm:exponential}), but other constructions can do better: in exponential families the pooled sufficient statistic reaches $\thetaN$ exactly for every $K \le N$, and multi-round schemes with $q \to \infty$ remove the restriction on $K$ entirely at the price of unbounded synchronization \citep{jordan2019communication,fan2023communication}.56
Files · 1ae646dc989aa524829d12cef2c3022d9e30261f
Menu

Theme