citation_key,authors,year,title,venue,doi_or_url,verified_full_text,assumptions,number_of_rounds,upstream_bytes,local_computation,statistical_rate,proof_type,applicable_models,overlap_with_WP28,novelty_threat,notes,page_numbers_checked
zhang2013communication,"Zhang, Yuchen; Duchi, John C.; Wainwright, Martin J.",2013,"Communication-Efficient Algorithms for Statistical Optimization","Journal of Machine Learning Research 14:3321-3363",https://www.jmlr.org/papers/v14/zhang13b.html (arXiv:1209.4129),"yes (official JMLR PDF)","A1 compact convex Theta with theta* interior; A2 local strong convexity of population risk at theta*; A3 8th moments of gradient and Hessian deviation + Hessian L(x)-Lipschitz with 8th moments; A4 (SAVGM only) third derivatives Lipschitz with E[M^8]<inf","1","AVGM: one d-vector per machine; SAVGM: corrected d-vector (two local fits)","one ERM on n points (+ one on rn points for SAVGM)","Thm 1 (AVGM): MSE = O(1/N + K^2/N^2); optimal rate iff K <~ sqrt(N) (eq.10). Thm 4 (SAVGM, (theta1 - r theta2)/(1-r)): MSE <= ((2+3r)/(1-r)^2)(1/N)E||H^-1 grad||^2 + c n^-3/(r(1-r)^2) + O((1-r)^-2 m^-1 n^-2); optimal rate iff K <~ N^(2/3) (eq.11)","non-asymptotic MSE bounds via Taylor expansion of FOC + 8th-moment concentration","general smooth M-estimation / ERM, fixed d","(i) partial: SAVGM is a single-ratio subsampling bias correction with inflated leading variance constant, not a jackknife; (ii) none; (iii) none","medium for (i); low for (ii)/(iii)","No Richardson/jackknife; multiple subsamples left open (p.3330); Newton only as local solver","3321-3332, 3337, 3340-3345"
huang2019distributed,"Huang, Cheng; Huo, Xiaoming",2019,"A distributed one-step estimator","Mathematical Programming 174(1-2):41-76",https://doi.org/10.1007/s10107-019-01369-0 (arXiv:1511.01443v2),"arXiv v2 yes; Springer text NOT VERIFIED","compact convex Theta; population Hessian negative definite at theta0; 8th moments; Hessian Lipschitz; concave criterion; iid equal split; fixed d","2","round 1: d-vector; round 2: d gradient + d^2 Hessian at the broadcast average","one local M-estimation; one gradient+Hessian at theta^(0); centre one d x d solve","Thm 3.4: one-step from the plain average is sqrt(N)-normal and equivalent to the centralized M-estimator when k = O(sqrt N). Thm 3.5: MSE <= 2Tr(Sigma)/N + O(N^-2) + O(k^4 N^-4). Lemma B.5: local bias C/n","asymptotic normality + non-asymptotic MSE bound","M-estimators and MLE, fixed d","(ii) direct: one global Newton step already proved equivalent, but only under k = O(sqrt N) and from an uncorrected average; (i) none; (iii) none (q=1 only)","high for (ii) unless positioned as bias-corrected input + enlarged frontier","Hessian = average of local Hessians at the averaged estimate; sharper k=o(N^(3/4)) implied by k^4N^-4 not stated by the authors","arXiv v2 pp. 1-3, 5-8, 11, 15-16, 24-26"
lin2011aggregated,"Lin, Nan; Xi, Ruibin",2011,"Aggregated estimating equation estimation","Statistics and Its Interface 4(1):73-83",https://doi.org/10.4310/SII.2011.v4.n1.a8,"yes (publisher reprint)","(C1)-(C6): smooth psi, p.d. Jacobians, strong consistency, bounded n^-1 A_k, bounded second derivatives, tail bound with alpha in (1/4,1/2); fixed p","1","p-vector + p x p local Jacobian at the LOCAL estimate per block","one EE solve per block; centre one p x p solve","AEE = (sum A_k)^-1 sum A_k beta_k. Thm 4.1: ||AEE - EE_full|| <= C(||worst local error||^2 + ...). Thm 4.2: sqrt(N)||AEE - EE_full|| -> 0 if K = O(n^gamma), gamma < 1/3 (K << N^(1/4))","asymptotic (Taylor expansion of aggregated score + tail bounds)","estimating equations, GLM/QLE, fixed p","none for (i)-(iii); Hessian-weighted one-shot baseline with restrictive K growth","low","No variance estimator; OLAP motivation","73-76, 78-79"
rosenblatt2016optimality,"Rosenblatt, Jonathan D.; Nadler, Boaz",2016,"On the optimality of averaging in distributed statistical learning","Information and Inference: A Journal of the IMA 5(4):379-404",https://doi.org/10.1093/imaiai/iaw013 (arXiv:1407.2724v2),"arXiv v2 yes; OUP pagination NOT VERIFIED","Set 1: van der Vaart 5.23-type; Set 2: Rilstone-Srivastava-Ullah derivatives to order 3/4; Set 3: Gaussian design p/n -> kappa; m FIXED in fixed-p results","1","one p-vector per machine","one local M-estimation","Thm 1 first-order equivalence (m fixed); Thm 2 B2(theta_bar) = delta/n = m B2(theta_hat_N); Thm 3 second-order MSE; HD Lemma 1: MSE ratio 1 + kappa(r2/r1)(1-1/m)","asymptotic stochastic expansions","M-estimators; Gaussian linear models (HD)","(i) gives the target of the correction (bias = m x centralized bias) and warns about variance; no correction proposed; (ii)/(iii) none","low (supporting)","Remark 2 lists bias-reduction routes","arXiv v2 pp. 1-12, 15-18"
jordan2019csl,"Jordan, Michael I.; Lee, Jason D.; Yang, Yun",2019,Communication-Efficient Distributed Statistical Inference,"JASA 114(526):668-681",10.1080/01621459.2018.1429274,"full text arXiv 1605.07689v3 (JASA typeset 403; published theorem numbers NOT VERIFIED)","PA compact convex Theta; PB mu- I <= Hessian(theta*) <= mu+ I; PC identifiability; PD 16th-moment bounds and Lipschitz Hessian; initial estimator O_p(n^-1/2)","1 round per iteration; ceil(log k/log n) iterations; 1 round suffices if ||theta_bar-theta*||=o_P(sqrt(n/N)) (Cor 3.4)","d numbers per machine per round (gradient); no Hessian","machine 1 minimizes the surrogate or takes one quadratic step with its own Hessian","||theta_tilde-theta_hat|| <= C(||theta_bar-theta_hat||^2 + (N^-1/2 + n^-1/2)||theta_bar-theta_hat||) (Thm 3.1, 3.3); contraction C/sqrt(n) per round; translation: K=o(N^1/2) local-MLE start, K=o(N^2/3) average start","finite-sample high-probability bounds + asymptotic normality corollary","regular M-estimators / MLE; l1; quasi-posterior","single-machine Hessian; no bias correction; precedent for the inexact-Hessian factor n^-1/2 (claim iv)",moderate,"JASA numbering unknown","arXiv pp. 7-17"
fan2023cease,"Fan, Jianqing; Guo, Yongyi; Wang, Kaizheng",2023,Communication-Efficient Accurate Statistical Estimation,"JASA 118(542):1000-1010",10.1080/01621459.2021.1969238,"full text arXiv 1906.04870v2 + PMC10281708 (numbering identical); JASA typeset 403","rho-strong convexity on a ball; Hessian homogeneity delta; Lipschitz Hessian; GLM canonical link, sub-Gaussian covariates, n >= c p","1 round per iteration (Alg 3) or 2 (Alg 4); 1 step from the plain average suffices iff n^3 >> N(kappa^2 p log N)(p+kappa^2 log p) (p16)","p numbers per machine per round; no Hessian","gradient-enhanced proximal local minimization on machine 1 (Alg 3) or on all machines + averaging (Alg 4)","||theta_t-theta_hat||=O_P(eta^{t/2}||theta_0-theta_hat||), eta=kappa^2 p log N/n (Thm 4.1); linear model: averaged step p^2/n^2 => K=o(N^3/4); single machine p^3/2/n^3/2 => K=o(N^2/3)","deterministic contraction + Hessian concentration","regularized ERM, GLM, linear regression","explicit one-step exponents 2/3 and 3/4 (local / averaged local Hessians) from the plain average; no bias correction; no jackknife comparison","moderate-high for claim (ii) architecture; none for jackknife/7/8","Alg 4 = DANE extension; proximal point view","arXiv pp. 4-18; PMC lines 230-262"
chen2022fone,"Chen, Xi; Liu, Weidong; Zhang, Yichen",2022,First-Order Newton-Type Estimator for Distributed Estimation and Inference,"JASA 117(540):1858-1874",10.1080/01621459.2021.1891925,"full text arXiv 1811.11368v2; JASA typeset 403","p=O(n^kappa1); batch >> p log n; local strong convexity at theta*; N=O(n^A); initial O_P(n^-delta1); loss may be non-smooth","K rounds, K constant depending on delta's (Thm 3.3)","O(p) per machine per round; no Hessian ever","machine 1 runs mini-batch first-order iterations approximating the Newton step","DC-SGD needs L=O(sqrt(N/p)) (Thm 3.1, necessary on examples); Dist-FONE: per-round factor n^-delta0 (Thm 3.3)","asymptotic O_P rates by induction over rounds","convex possibly non-smooth losses; diverging p","no bias correction; no K-exponent for fixed rounds; citable sqrt(N) barrier statement",low,"computation on largest machine","arXiv pp. 3, 7-17"
shamir2014dane,"Shamir, Ohad; Srebro, Nathan; Zhang, Tong",2014,Communication-Efficient Distributed Optimization using an Approximate Newton-type Method,"Proc. 31st ICML, PMLR 32(2):1000-1008",https://proceedings.mlr.press/v32/shamir14.html,"full text PMLR PDF + arXiv 1312.7853v4","quadratic losses lambda <= Hess <= L; matrix concentration of local Hessians; strongly convex smooth local objectives (Thm 4)","2 averaging communications per iteration; t=O((L/lambda)^2 log(dm)/n log(1/eps)) iterations (Thm 3)","d numbers per machine per averaging; no Hessian","local proximal subproblem with gradient correction","||w^t-w_hat|| <= ||I - H_tilde^-1 H||^t ||w^0-w_hat||, ||I-H_tilde^-1 H|| <= 4 beta^2/lambda^2, beta ~ sqrt(log(dm)/n) (Thm 2, Lemmas 1-2); Thm 1 one-shot averaging fails when lambda <= O(1/sqrt n)","optimization (eps-suboptimality); mirror descent view","smooth strongly convex ERM; quadratics","averaged local Hessians cost ~1/n per round (claim iv precedent); no statistical K-exponent; no bias correction",low,"Thm 5 single-machine variant = CSL precursor; App. A remark on a bias-corrected average","PMLR pp. 1003-1006; arXiv pp. 5-10, 16"
wang2017edsl,"Wang, Jialei; Kolar, Mladen; Srebro, Nathan; Zhang, Tong",2017,Efficient Distributed Learning with Sparsity,"Proc. 34th ICML, PMLR 70:3636-3645",https://proceedings.mlr.press/v70/wang17f.html,"full text PMLR PDF + arXiv 1605.07991v1 (numbering differs)","L-smooth loss; restricted strong convexity on machine 1; sub-Gaussian designs; n >~ s^2 log p","1 round per iteration; 1 round matches the lasso iff n >~ m s^2 log p; O(log m) rounds otherwise","p numbers per machine per round","master solves the shifted l1 problem (= subsampled Newton when lambda=0, quadratic)","contraction a_n ~ s sqrt(log p/n) per round (Thm 6); geometric after t rounds (Cor 7)","high-probability recursive bounds","sparse linear/logistic, graphical models","single-machine Hessian; l1; no bias correction; K <~ sqrt(N) for one round",low,"concurrent with CSL","PMLR pp. 3637-3641; arXiv pp. 2-7"
battey2018distributed,"Battey, Heather; Fan, Jianqing; Liu, Han; Lu, Junwei; Zhu, Ziwei",2018,Distributed testing and estimation under sparse high dimensional models,"Annals of Statistics 46(3):1352-1382",10.1214/17-AOS1587,"full text arXiv 1509.05457v1 + PMC6051757 (low-dim theorems moved to supplement); journal page mapping NOT VERIFIED","lasso/SCAD/MCP error rates; bounded eigenvalues, sub-Gaussian rows; GLM analogues; finite fourth moments","1 round (one-shot)","d numbers per subsample","local lasso + debiasing (nodewise lassos); low-dim local OLS/GLM","Wald normality k=o((s log d)^-1 sqrt n) (Thm 3.3); lasso rate retained iff k=O(sqrt(n/(s^2 log d))) (Thm 4.3); low-dim GLM k=o(sqrt n/(d v log n)) (arXiv Thm 4.13)","high-probability decomposition + CLT","sparse high-dim linear & GLM; low-dim linear & GLM","no bias correction beyond lasso debiasing; establishes the sqrt(N) baseline (r=1,q=0); Remark 4.5 = same leading term, remainders differ",low,"n = total sample size in their notation","arXiv pp. 1-2, 8, 13-17; PMC lines 92-99, 368-442, 531-537"
volgushev2019quantile,"Volgushev, Stanislav; Chao, Shih-Kang; Cheng, Guang",2019,Distributed inference for quantile regression processes,"Annals of Statistics 47(3):1634-1662",10.1214/18-AOS1730,"full text arXiv 1701.06088v3 + NSF-PAR accepted manuscript (numbering identical)","bounded design with eigenvalue bounds; conditional density bounded with bounded derivative; Holder smoothness of tau -> beta(tau)","1 round (one-shot)","m numbers per machine per quantile level","local quantile regression fits","sup_tau ||beta_bar - beta_or|| = O_P(S log N/N + S^1/4 (log N)^7/4/N^3/4) + o_P(N^-1/2) (Thm 3.1); lower bound >= C S/N for some model (Thm 3.2); sufficient S=o(N^1/2/log N), NECESSARY S=o(N^1/2) (Cor 3.3)","new Bahadur representation; explicit worst-case construction","linear quantile regression; series models; conditional distribution functions","template for sharp (necessary + sufficient) partition-count conditions; no correction; no Newton",low,"necessity is worst-case over a model class","arXiv pp. 1, 6, 9-11"
shang2017limits,"Shang, Zuofeng; Cheng, Guang",2017,Computational Limits of A Distributed Algorithm for Smoothing Spline,"JMLR 18(108):1-37",https://jmlr.org/papers/v18/16-289.html,"full text JMLR PDF","periodic m-th order smoothing spline; evenly spaced design; iid errors; Sobolev ball","1 round (one-shot averaging)","one function estimate per machine","local smoothing spline with common lambda","minimax optimal iff s=O(N^2m/(2m+1)) (Thm 3.1); worst-case squared bias >= C(a_m n^-2m - 8N^-1) (Thm 3.2) => sharp; testing s=O(N^(4m-1)/(4m+1)) sharp","non-asymptotic bounds in eigen-basis; phase transition","nonparametric regression; testing","sharp maximal number of machines set by local bias vs 1/N variance; same mechanism, nonparametric",low,"improves ZDW bound","JMLR pp. 1, 5-7, 9-10"