1citation_key,authors,year,title,venue,doi_or_url,verified_full_text,assumptions,number_of_rounds,upstream_bytes,local_computation,statistical_rate,proof_type,applicable_models,overlap_with_WP28,novelty_threat,notes,page_numbers_checked2zhang2013communication,"Zhang, Yuchen; Duchi, John C.; Wainwright, Martin J.",2013,"Communication-Efficient Algorithms for Statistical Optimization","Journal of Machine Learning Research 14:3321-3363",https://www.jmlr.org/papers/v14/zhang13b.html (arXiv:1209.4129),"yes (official JMLR PDF)","A1 compact convex Theta with theta* interior; A2 local strong convexity of population risk at theta*; A3 8th moments of gradient and Hessian deviation + Hessian L(x)-Lipschitz with 8th moments; A4 (SAVGM only) third derivatives Lipschitz with E[M^8]<inf","1","AVGM: one d-vector per machine; SAVGM: corrected d-vector (two local fits)","one ERM on n points (+ one on rn points for SAVGM)","Thm 1 (AVGM): MSE = O(1/N + K^2/N^2); optimal rate iff K <~ sqrt(N) (eq.10). Thm 4 (SAVGM, (theta1 - r theta2)/(1-r)): MSE <= ((2+3r)/(1-r)^2)(1/N)E||H^-1 grad||^2 + c n^-3/(r(1-r)^2) + O((1-r)^-2 m^-1 n^-2); optimal rate iff K <~ N^(2/3) (eq.11)","non-asymptotic MSE bounds via Taylor expansion of FOC + 8th-moment concentration","general smooth M-estimation / ERM, fixed d","(i) partial: SAVGM is a single-ratio subsampling bias correction with inflated leading variance constant, not a jackknife; (ii) none; (iii) none","medium for (i); low for (ii)/(iii)","No Richardson/jackknife; multiple subsamples left open (p.3330); Newton only as local solver","3321-3332, 3337, 3340-3345"3huang2019distributed,"Huang, Cheng; Huo, Xiaoming",2019,"A distributed one-step estimator","Mathematical Programming 174(1-2):41-76",https://doi.org/10.1007/s10107-019-01369-0 (arXiv:1511.01443v2),"arXiv v2 yes; Springer text NOT VERIFIED","compact convex Theta; population Hessian negative definite at theta0; 8th moments; Hessian Lipschitz; concave criterion; iid equal split; fixed d","2","round 1: d-vector; round 2: d gradient + d^2 Hessian at the broadcast average","one local M-estimation; one gradient+Hessian at theta^(0); centre one d x d solve","Thm 3.4: one-step from the plain average is sqrt(N)-normal and equivalent to the centralized M-estimator when k = O(sqrt N). Thm 3.5: MSE <= 2Tr(Sigma)/N + O(N^-2) + O(k^4 N^-4). Lemma B.5: local bias C/n","asymptotic normality + non-asymptotic MSE bound","M-estimators and MLE, fixed d","(ii) direct: one global Newton step already proved equivalent, but only under k = O(sqrt N) and from an uncorrected average; (i) none; (iii) none (q=1 only)","high for (ii) unless positioned as bias-corrected input + enlarged frontier","Hessian = average of local Hessians at the averaged estimate; sharper k=o(N^(3/4)) implied by k^4N^-4 not stated by the authors","arXiv v2 pp. 1-3, 5-8, 11, 15-16, 24-26"4lin2011aggregated,"Lin, Nan; Xi, Ruibin",2011,"Aggregated estimating equation estimation","Statistics and Its Interface 4(1):73-83",https://doi.org/10.4310/SII.2011.v4.n1.a8,"yes (publisher reprint)","(C1)-(C6): smooth psi, p.d. Jacobians, strong consistency, bounded n^-1 A_k, bounded second derivatives, tail bound with alpha in (1/4,1/2); fixed p","1","p-vector + p x p local Jacobian at the LOCAL estimate per block","one EE solve per block; centre one p x p solve","AEE = (sum A_k)^-1 sum A_k beta_k. Thm 4.1: ||AEE - EE_full|| <= C(||worst local error||^2 + ...). Thm 4.2: sqrt(N)||AEE - EE_full|| -> 0 if K = O(n^gamma), gamma < 1/3 (K << N^(1/4))","asymptotic (Taylor expansion of aggregated score + tail bounds)","estimating equations, GLM/QLE, fixed p","none for (i)-(iii); Hessian-weighted one-shot baseline with restrictive K growth","low","No variance estimator; OLAP motivation","73-76, 78-79"5rosenblatt2016optimality,"Rosenblatt, Jonathan D.; Nadler, Boaz",2016,"On the optimality of averaging in distributed statistical learning","Information and Inference: A Journal of the IMA 5(4):379-404",https://doi.org/10.1093/imaiai/iaw013 (arXiv:1407.2724v2),"arXiv v2 yes; OUP pagination NOT VERIFIED","Set 1: van der Vaart 5.23-type; Set 2: Rilstone-Srivastava-Ullah derivatives to order 3/4; Set 3: Gaussian design p/n -> kappa; m FIXED in fixed-p results","1","one p-vector per machine","one local M-estimation","Thm 1 first-order equivalence (m fixed); Thm 2 B2(theta_bar) = delta/n = m B2(theta_hat_N); Thm 3 second-order MSE; HD Lemma 1: MSE ratio 1 + kappa(r2/r1)(1-1/m)","asymptotic stochastic expansions","M-estimators; Gaussian linear models (HD)","(i) gives the target of the correction (bias = m x centralized bias) and warns about variance; no correction proposed; (ii)/(iii) none","low (supporting)","Remark 2 lists bias-reduction routes","arXiv v2 pp. 1-12, 15-18"6jordan2019csl,"Jordan, Michael I.; Lee, Jason D.; Yang, Yun",2019,Communication-Efficient Distributed Statistical Inference,"JASA 114(526):668-681",10.1080/01621459.2018.1429274,"full text arXiv 1605.07689v3 (JASA typeset 403; published theorem numbers NOT VERIFIED)","PA compact convex Theta; PB mu- I <= Hessian(theta*) <= mu+ I; PC identifiability; PD 16th-moment bounds and Lipschitz Hessian; initial estimator O_p(n^-1/2)","1 round per iteration; ceil(log k/log n) iterations; 1 round suffices if ||theta_bar-theta*||=o_P(sqrt(n/N)) (Cor 3.4)","d numbers per machine per round (gradient); no Hessian","machine 1 minimizes the surrogate or takes one quadratic step with its own Hessian","||theta_tilde-theta_hat|| <= C(||theta_bar-theta_hat||^2 + (N^-1/2 + n^-1/2)||theta_bar-theta_hat||) (Thm 3.1, 3.3); contraction C/sqrt(n) per round; translation: K=o(N^1/2) local-MLE start, K=o(N^2/3) average start","finite-sample high-probability bounds + asymptotic normality corollary","regular M-estimators / MLE; l1; quasi-posterior","single-machine Hessian; no bias correction; precedent for the inexact-Hessian factor n^-1/2 (claim iv)",moderate,"JASA numbering unknown","arXiv pp. 7-17"7fan2023cease,"Fan, Jianqing; Guo, Yongyi; Wang, Kaizheng",2023,Communication-Efficient Accurate Statistical Estimation,"JASA 118(542):1000-1010",10.1080/01621459.2021.1969238,"full text arXiv 1906.04870v2 + PMC10281708 (numbering identical); JASA typeset 403","rho-strong convexity on a ball; Hessian homogeneity delta; Lipschitz Hessian; GLM canonical link, sub-Gaussian covariates, n >= c p","1 round per iteration (Alg 3) or 2 (Alg 4); 1 step from the plain average suffices iff n^3 >> N(kappa^2 p log N)(p+kappa^2 log p) (p16)","p numbers per machine per round; no Hessian","gradient-enhanced proximal local minimization on machine 1 (Alg 3) or on all machines + averaging (Alg 4)","||theta_t-theta_hat||=O_P(eta^{t/2}||theta_0-theta_hat||), eta=kappa^2 p log N/n (Thm 4.1); linear model: averaged step p^2/n^2 => K=o(N^3/4); single machine p^3/2/n^3/2 => K=o(N^2/3)","deterministic contraction + Hessian concentration","regularized ERM, GLM, linear regression","explicit one-step exponents 2/3 and 3/4 (local / averaged local Hessians) from the plain average; no bias correction; no jackknife comparison","moderate-high for claim (ii) architecture; none for jackknife/7/8","Alg 4 = DANE extension; proximal point view","arXiv pp. 4-18; PMC lines 230-262"8chen2022fone,"Chen, Xi; Liu, Weidong; Zhang, Yichen",2022,First-Order Newton-Type Estimator for Distributed Estimation and Inference,"JASA 117(540):1858-1874",10.1080/01621459.2021.1891925,"full text arXiv 1811.11368v2; JASA typeset 403","p=O(n^kappa1); batch >> p log n; local strong convexity at theta*; N=O(n^A); initial O_P(n^-delta1); loss may be non-smooth","K rounds, K constant depending on delta's (Thm 3.3)","O(p) per machine per round; no Hessian ever","machine 1 runs mini-batch first-order iterations approximating the Newton step","DC-SGD needs L=O(sqrt(N/p)) (Thm 3.1, necessary on examples); Dist-FONE: per-round factor n^-delta0 (Thm 3.3)","asymptotic O_P rates by induction over rounds","convex possibly non-smooth losses; diverging p","no bias correction; no K-exponent for fixed rounds; citable sqrt(N) barrier statement",low,"computation on largest machine","arXiv pp. 3, 7-17"9shamir2014dane,"Shamir, Ohad; Srebro, Nathan; Zhang, Tong",2014,Communication-Efficient Distributed Optimization using an Approximate Newton-type Method,"Proc. 31st ICML, PMLR 32(2):1000-1008",https://proceedings.mlr.press/v32/shamir14.html,"full text PMLR PDF + arXiv 1312.7853v4","quadratic losses lambda <= Hess <= L; matrix concentration of local Hessians; strongly convex smooth local objectives (Thm 4)","2 averaging communications per iteration; t=O((L/lambda)^2 log(dm)/n log(1/eps)) iterations (Thm 3)","d numbers per machine per averaging; no Hessian","local proximal subproblem with gradient correction","||w^t-w_hat|| <= ||I - H_tilde^-1 H||^t ||w^0-w_hat||, ||I-H_tilde^-1 H|| <= 4 beta^2/lambda^2, beta ~ sqrt(log(dm)/n) (Thm 2, Lemmas 1-2); Thm 1 one-shot averaging fails when lambda <= O(1/sqrt n)","optimization (eps-suboptimality); mirror descent view","smooth strongly convex ERM; quadratics","averaged local Hessians cost ~1/n per round (claim iv precedent); no statistical K-exponent; no bias correction",low,"Thm 5 single-machine variant = CSL precursor; App. A remark on a bias-corrected average","PMLR pp. 1003-1006; arXiv pp. 5-10, 16"10wang2017edsl,"Wang, Jialei; Kolar, Mladen; Srebro, Nathan; Zhang, Tong",2017,Efficient Distributed Learning with Sparsity,"Proc. 34th ICML, PMLR 70:3636-3645",https://proceedings.mlr.press/v70/wang17f.html,"full text PMLR PDF + arXiv 1605.07991v1 (numbering differs)","L-smooth loss; restricted strong convexity on machine 1; sub-Gaussian designs; n >~ s^2 log p","1 round per iteration; 1 round matches the lasso iff n >~ m s^2 log p; O(log m) rounds otherwise","p numbers per machine per round","master solves the shifted l1 problem (= subsampled Newton when lambda=0, quadratic)","contraction a_n ~ s sqrt(log p/n) per round (Thm 6); geometric after t rounds (Cor 7)","high-probability recursive bounds","sparse linear/logistic, graphical models","single-machine Hessian; l1; no bias correction; K <~ sqrt(N) for one round",low,"concurrent with CSL","PMLR pp. 3637-3641; arXiv pp. 2-7"11battey2018distributed,"Battey, Heather; Fan, Jianqing; Liu, Han; Lu, Junwei; Zhu, Ziwei",2018,Distributed testing and estimation under sparse high dimensional models,"Annals of Statistics 46(3):1352-1382",10.1214/17-AOS1587,"full text arXiv 1509.05457v1 + PMC6051757 (low-dim theorems moved to supplement); journal page mapping NOT VERIFIED","lasso/SCAD/MCP error rates; bounded eigenvalues, sub-Gaussian rows; GLM analogues; finite fourth moments","1 round (one-shot)","d numbers per subsample","local lasso + debiasing (nodewise lassos); low-dim local OLS/GLM","Wald normality k=o((s log d)^-1 sqrt n) (Thm 3.3); lasso rate retained iff k=O(sqrt(n/(s^2 log d))) (Thm 4.3); low-dim GLM k=o(sqrt n/(d v log n)) (arXiv Thm 4.13)","high-probability decomposition + CLT","sparse high-dim linear & GLM; low-dim linear & GLM","no bias correction beyond lasso debiasing; establishes the sqrt(N) baseline (r=1,q=0); Remark 4.5 = same leading term, remainders differ",low,"n = total sample size in their notation","arXiv pp. 1-2, 8, 13-17; PMC lines 92-99, 368-442, 531-537"12volgushev2019quantile,"Volgushev, Stanislav; Chao, Shih-Kang; Cheng, Guang",2019,Distributed inference for quantile regression processes,"Annals of Statistics 47(3):1634-1662",10.1214/18-AOS1730,"full text arXiv 1701.06088v3 + NSF-PAR accepted manuscript (numbering identical)","bounded design with eigenvalue bounds; conditional density bounded with bounded derivative; Holder smoothness of tau -> beta(tau)","1 round (one-shot)","m numbers per machine per quantile level","local quantile regression fits","sup_tau ||beta_bar - beta_or|| = O_P(S log N/N + S^1/4 (log N)^7/4/N^3/4) + o_P(N^-1/2) (Thm 3.1); lower bound >= C S/N for some model (Thm 3.2); sufficient S=o(N^1/2/log N), NECESSARY S=o(N^1/2) (Cor 3.3)","new Bahadur representation; explicit worst-case construction","linear quantile regression; series models; conditional distribution functions","template for sharp (necessary + sufficient) partition-count conditions; no correction; no Newton",low,"necessity is worst-case over a model class","arXiv pp. 1, 6, 9-11"13shang2017limits,"Shang, Zuofeng; Cheng, Guang",2017,Computational Limits of A Distributed Algorithm for Smoothing Spline,"JMLR 18(108):1-37",https://jmlr.org/papers/v18/16-289.html,"full text JMLR PDF","periodic m-th order smoothing spline; evenly spaced design; iid errors; Sobolev ball","1 round (one-shot averaging)","one function estimate per machine","local smoothing spline with common lambda","minimax optimal iff s=O(N^2m/(2m+1)) (Thm 3.1); worst-case squared bias >= C(a_m n^-2m - 8N^-1) (Thm 3.2) => sharp; testing s=O(N^(4m-1)/(4m+1)) sharp","non-asymptotic bounds in eigen-basis; phase transition","nonparametric regression; testing","sharp maximal number of machines set by local bias vs 1/N variance; same mechanism, nonparametric",low,"improves ZDW bound","JMLR pp. 1, 5-7, 9-10"14dhaene2015split,"Dhaene, Geert; Jochmans, Koen",2015,"Split-panel Jackknife Estimation of Fixed-effect Models","Review of Economic Studies 82(3):991-1030",10.1093/restud/rdv007,"yes (accepted manuscript)","alpha-mixing stationary panel; influence-function representation; bias expansion B1/T + ... + Bk/T^k; N/T -> rho",NA,NA,NA,"bias o(T^-h) with h-th order jackknife; sqrt(NT) normal with variance Sigma^-1 unchanged (first order)","asymptotic rectangular-array","nonlinear fixed-effect panel MLE","same extrapolation algebra (weights from a Vandermonde-type system in 1/T); bias-optimal partitions {2,3,...,h+1}, not {2,4,...}; higher-order variance left open","high for the extrapolation step; no distributed/Newton component","Supplement Thms S.2.1, S.2.2, S.3.1 cited from the manuscript","manuscript pp. 7-10, 24-26, 33"15hahn2024efficient,"Hahn, Jinyong; Hughes, David W.; Kuersteiner, Guido; Newey, Whitney K.",2024,"Efficient bias correction for cross-section and panel data","Quantitative Economics 15(3):783-816",10.3982/QE2350,"yes","iid; third-order stochastic expansion; regularity App. B",NA,NA,NA,"split-sample bias estimate inconsistent (O_p(1)); higher-order variance of split-sample corrections = 2x leave-one-out (Props 5-7)","asymptotic expansions","MLE cross-section; FE panel","variance cost of split-sample corrections",high,"analytical, bootstrap and leave-one-out share the same higher-order variance (Thms 1-2)","pp. 783-787, 797-802, 806-807"16quenouille1956notes,"Quenouille, M. H.",1956,"Notes on bias in estimation","Biometrika 43(3/4):353-360",10.1093/biomet/43.3-4.353,no,"bias expansion in powers of 1/n (per Kosmidis 2014)",NA,NA,NA,"bias O(n^-2) after jackknife",NOT VERIFIED,"general estimators","origin of the jackknife bias cancellation",low,"text not read",none17quenouille1949approximate,"Quenouille, M. H.",1949,"Approximate tests of correlation in time-series","JRSS B 11(1):68-84",10.1111/j.2517-6161.1949.tb00023.x,no,NOT VERIFIED,NA,NA,NA,NOT VERIFIED,NOT VERIFIED,"time series","split-half bias cancellation (attribution per secondary sources)",low,"text behind paywall",none18tukey1958bias,"Tukey, J. W.",1958,"Bias and confidence in not-quite large samples (abstract)","Annals of Mathematical Statistics 29(2):614",10.1214/aoms/1177706647,no,NA,NA,NA,NA,NA,"abstract","general","names the jackknife",low,"DOI covers Abstracts of Papers pp. 614-623",none19schucany1971bias,"Schucany, W. R.; Gray, H. L.; Owen, D. B.",1971,"On bias reduction in estimation","JASA 66(335):524-533",10.1080/01621459.1971.10482296,no,"known ratio of bias functions (per secondary)",NA,NA,NA,"removes k bias terms with k+1 estimators (per secondary)",NOT VERIFIED,"general estimators","generalized-jackknife weights = same linear-system idea",medium,"brief's DOI was wrong",none20sharot1976generalized,"Sharot, Trevor",1976,"The generalized jackknife: finite samples and subsample sizes","JASA 71(354):451-454",10.1080/01621459.1976.10480367,no,NOT VERIFIED,NA,NA,NA,NOT VERIFIED,NOT VERIFIED,"general","subsample-size choice",medium,"no text available",none21efron1982jackknife,"Efron, Bradley",1982,"The Jackknife, the Bootstrap and Other Resampling Plans","SIAM CBMS-NSF 38",10.1137/1.9781611970319,no,NA,NA,NA,NA,NA,"monograph","general","jackknife as linear extrapolation in 1/n (sec 2.3 per Kosmidis)",low,"",none22cox1968general,"Cox, D. R.; Snell, E. J.",1968,"A general definition of residuals","JRSS B 30(2):248-265",10.1111/j.2517-6161.1968.tb00724.x,no,"regular likelihood; cumulants O(n)",NA,NA,NA,"O(1/n) bias formula","formal expansion","parametric MLE","analytic first-order bias targeted by the extrapolation",low,"formula per Giles 2011",none23cordeiro1991bias,"Cordeiro, Gauss M.; McCullagh, Peter",1991,"Bias correction in generalized linear models","JRSS B 53(3):629-643",10.1111/j.2517-6161.1991.tb01852.x,"abstract","GLM with known/estimated dispersion",NA,NA,NA,"O(1/n) bias removed","formal expansion","GLMs","closed-form analytic bias (benchmark for the jackknife)",low,"formula per Kosmidis et al. 2018",none24firth1993bias,"Firth, David",1993,"Bias reduction of maximum likelihood estimates","Biometrika 80(1):27-38",10.1093/biomet/80.1.27,"abstract","regular parametric problems",NA,NA,NA,"first-order bias removed via score modification","asymptotic","parametric MLE, GLMs","implicit alternative to the jackknife",low,"",none25rilstone1996second,"Rilstone, Paul; Srivastava, V. K.; Ullah, Aman",1996,"The second-order bias and mean squared error of nonlinear estimators","Journal of Econometrics 75(2):369-395",10.1016/0304-4076(96)89457-7,no,NOT VERIFIED,NA,NA,NA,"O(1/n) bias and MSE expansions",NOT VERIFIED,"M-estimators","analytic second-order bias of M-estimators",low,"corrigendum 2005 124(1):203-204",none26hahn2004jackknife,"Hahn, Jinyong; Newey, Whitney",2004,"Jackknife and analytical bias reduction for nonlinear panel models","Econometrica 72(4):1295-1319",10.1111/j.1468-0262.2004.00533.x,"abstract + 2002 WP","static nonlinear FE panel; expansion theta[T] = theta + beta/T + gamma/T^2 + O(T^-3)",NA,NA,NA,"bias O(T^-2)","asymptotic","static nonlinear panel MLE","delete-one jackknife vs analytic correction",low-medium,"",none27lecam1956asymptotic,"Le Cam, Lucien",1956,"On the asymptotic theory of estimation and testing hypotheses","Proc. Third Berkeley Symp. 1:129-156",10.1525/9780520313880-014,no,NOT VERIFIED,NA,NA,NA,NOT VERIFIED,NOT VERIFIED,"parametric","one-step efficiency origin",low,"",none28bickel1975one,"Bickel, P. J.",1975,"One-step Huber estimates in the linear model","JASA 70(350):428-434",10.1080/01621459.1975.10479884,"abstract","mild regularity conditions",NA,NA,NA,"one-step asymptotically equivalent to the full M-estimate",NOT VERIFIED,"linear model M-estimators","one-step Newton from a preliminary estimator",low,"",none29vandervaart1998asymptotic,"van der Vaart, A. W.",1998,"Asymptotic Statistics","Cambridge University Press",10.1017/CBO9780511802256,"TOC",NA,NA,NA,NA,NA,"textbook","general","Sec. 5.7 One-Step Estimators p. 71",low,"",TOC30bhattacharya1978validity,"Bhattacharya, R. N.; Ghosh, J. K.",1978,"On the validity of the formal Edgeworth expansion","Annals of Statistics 6(2):434-451",10.1214/aos/1176344134,"abstract","iid, smooth H, moment + Cramer conditions",NA,NA,NA,"valid Edgeworth expansion to arbitrary order","rigorous","MLE, minimum contrast","justifies the bias expansion in 1/n",low,"",none31bach2021richardson,"Bach, Francis",2021,"On the effectiveness of Richardson extrapolation in data science","SIAM J. Math. Data Sci. 3(4):1251-1277",10.1137/21M1397349,yes,"power-series dependence in the hyperparameter",NA,NA,NA,"O(1/k) -> O(1/k^2) in iterations","asymptotic","optimisation, ridge, smoothing","Richardson in iterations/regularisation, not sample size",low,"",arXiv32lin2024jackknife,"Lin, Licong; Su, Fangzhou; Mou, Wenlong; Ding, Peng; Wainwright, Martin J.",2024,"When is it worthwhile to jackknife? Breaking the quadratic barrier for Z-estimators","arXiv 2411.02909",https://arxiv.org/abs/2411.02909,"abstract","high-dim Z-estimators",NA,NA,NA,"sqrt(n)-consistent when n >> d^{3/2}","non-asymptotic","Z-estimators, GLMs","leave-one-out jackknife only",low-medium,"",abstract33wu2023subsampling,"Wu, Shuyuan; Zhu, Xuening; Wang, Hansheng",2023,"Subsampling and jackknifing: a practically convenient solution for large data analysis with limited computational resources","Statistica Sinica 33:2041-2064",10.5705/ss.202021.0257,"abstract","mild conditions",NA,NA,NA,"consistent, asymptotically normal","asymptotic","general estimators","jackknife debiasing within subsamples; no nested sizes",low,"",abstract34chudik2018half,"Chudik, Alexander; Pesaran, M. Hashem; Yang, Jui-Chung",2018,"Half-panel jackknife fixed-effects estimation of linear panels with weakly exogenous regressors","Journal of Applied Econometrics 33(6):816-836",10.1002/jae.2623,"abstract","weakly exogenous regressors",NA,NA,NA,"bias O(T^-2); needs N/T^3 -> 0","asymptotic","linear dynamic panels","half-panel jackknife application",low,"",none35zhu2021lsa,"Zhu, Xuening; Li, Feng; Wang, Hansheng",2021,"Least-Square Approximation for a Distributed System","Journal of Computational and Graphical Statistics 30(4):1004-1018",10.1080/10618600.2021.1923517 (arXiv:1908.04904v4),"yes (arXiv v4)","iid C1-C5; c1 <= n_k/n <= c2","1 (WLSE) / 2 (TWLSE) / m (iterated)","K(p + p^2) per round","local M-estimation; TWLSE adds one LOCAL Newton step per machine before Hessian-weighted averaging","Prop 1 bias O_p(K/sqrt N); Thm 1 efficient if n/N^1/2 -> inf; Thm 2 TWLSE bias O_p(n^-2 sqrt N), efficient if n/N^1/4 -> inf (K = O(N^3/4)); Prop 2 m-WLSE with m/log N -> inf rounds allows n = O(1)","asymptotic normality","M-estimators","closest to 'raw average + one step' with local Hessians at local points; unequal sizes","STRONG for (r=1,q=1) exponent 3/4 and for the stale/local-Hessian variant; no bias correction; no joint frontier","trade-off remark p.12","pp. 6, 10-12"36gu2023heterogeneity,"Gu, Jia; Chen, Songxi",2023,"Distributed statistical inference under heterogeneity","Journal of Machine Learning Research 24(387):1-56",https://jmlr.org/papers/v24/22-1274.html (arXiv:2209.06482v1),"yes (arXiv v1)","heterogeneous blocks; Assumptions 1-7; moments v >= 4; unequal n_k","1","K(p + p^2)","local M-estimation + analytic leading-bias estimate B_k with sample splitting","WD: K = o(N^1/2) (Thms 2-3); debiased dWD/dSaC: MSE <= C1/(nK) + C2/(n^2 K) + C3/n^3, chi^2 limit if K = o(n^2) = o(N^2/3) (Thms 4-6); Cor 1 beats the SAVGM constant","MSE bounds + chi^2 limits","likelihood and quasi-likelihood (sandwich weights)","analytic local debiasing before weighted averaging (our analytic Cox-Snell baseline is this idea); unequal sizes; heterogeneity","STRONG for 'debias then average' (analytic, not split-sample); none for Newton rounds / frontier","SM 1.10 compares with Huang-Huo","pp. 3, 17, 20-24"37wu2023subsampling_jds,"Wu, Shuyuan; Zhu, Xuening; Wang, Hansheng",2023,"Subsampling and jackknifing: a practically convenient solution for large data analysis with limited computational resources","Statistica Sinica 33:2041-2064",10.5705/ss.202021.0257 (arXiv:2304.06231v1),"yes (arXiv v1)","sub-Gaussian; g eight times differentiable; N = o(n^4); log K = o(sqrt n); with-replacement subsamples","1","K estimates","delete-one jackknife (n refits) per subsample, then average","Thm 2: bias O(n^-2) + O(N^-1) vs tau_2/n for plain averaging; Thm 4 normal under N = o(n^4)","expansion + CLT","smooth functions of means","jackknife-debiased local estimates then averaged (delete-one, not split-sample; n refits vs 2^r - 1 nested fits)","STRONG for 'jackknife then average' (different jackknife); none for Newton / frontier","Li et al. 2024 review extends to distributed M-estimators","pp. 10-16"38yin2022newton,"Yin, Zhuoran",2022,"Communication-efficient distributed Newton-like optimization with gradients and M-estimators","arXiv 2207.06253 (Temple PhD thesis 2022 NOT VERIFIED)",https://arxiv.org/abs/2207.06253,"yes (arXiv v1)","iid; third-order smooth loss; sqrt(n)-consistent local M-estimators","2 / T","M-estimators + gradients + local second/third-derivative summaries","local M-estimation + derivative summaries","Fisher information from gradients and M-estimators with error O_p(n^-1 + m^-1); Thm 3.2 bias-adjusted one-step error O_p(n^-3/2 + m^-1 n^-1/2), efficient iff m = o(n^2) (K = o(N^2/3))","expansion bounds","M-estimators","analytic bias adjustment of the one-step estimator from the plain average","PARTIAL/STRONG for 'debiased averaging + Newton' (analytic, 2/3); none for split-sample / frontier","thesis may contain a non-normal limit for many centres (NOT VERIFIED)","pp. 4, 10-11"39gao2022review,"Gao, Yuan; Liu, Weidong; Wang, Hansheng; Wang, Xiaozhou; Yan, Yibo; Zhang, Riquan",2022,"A review of distributed statistical inference","Statistical Theory and Related Fields 6(2):89-99",10.1080/24754269.2021.1974158 (arXiv:2304.06245),"yes","review","-","-","-","averaging MSE C1/N + C2/n^2; one-step C1/N + O(n^-4 + N^-2); K = o(sqrt N) necessary (citing Volgushev)","review","-","confirms no (order, rounds) frontier as of 2022; open problem: unbalanced sizes",NONE,"p. 20 open problem","pp. 4-7, 11, 20"40chen2024binary,"Chen, Xi; Jing, Wenbo; Liu, Weidong; Zhang, Yichen",2024,"Distributed estimation and inference for semiparametric binary response models","Annals of Statistics 52(3):922-947",10.1214/24-AOS2376 (arXiv:2210.08393v4),"yes","smoothed max-score kernel order alpha; heterogeneous sizes (Sec 4)","1 / T","L(p + p^2)","smoothed local max score","Thm 3.1 one-shot L condition; Thms 3.3-3.4 multi-round removes it; Thm 4.1 weighted average with unequal m_l","nonsmooth asymptotics","binary choice","multi-round removes the machine-count condition; unequal sizes","PARTIAL","","pp. 5-6, 18-24, 30-32"41wang2021osup,"Wang, Fei; Zhu, Yuan; Huang, Danyang; Qi, Haobo; Wang, Hansheng",2021,"Distributed one-step upgraded estimation for non-uniformly and non-randomly distributed data","Computational Statistics & Data Analysis 162:107265",10.1016/j.csda.2021.107265,"abstract","non-random unequal partitions; pilot sample","2","pilot + derivatives","one Newton step from a pilot","as efficient as the full MLE","asymptotic normality","MLE / GLM","pilot + one step; unequal non-random partitions","PARTIAL (C2); STRONG (C5)","","abstract"42zhou2020glm,"Zhou, Ping; Yu, Zhen; Ma, Jingyi; Tian, Maozai; Fan, Ye",2020,"Communication-efficient distributed estimator for generalized linear models with a diverging number of covariates","arXiv 2001.06194v2 (journal NOT VERIFIED)",https://arxiv.org/abs/2001.06194,"yes","GLM, p_n = o(n^alpha); max n_k / min n_k = O(1)","2","K(p + p^2)","weighted average + one Fisher-scoring step","AEE needs K = o(sqrt n/p_n) (Thm 3); weighted average + one step needs K = O(sqrt(n/p_n)) (Thm 4)","efficiency","GLM","one-step from a weighted average; unequal sizes","PARTIAL","","pp. 4, 11, 13"43duan2022heterogeneity,"Duan, Rui; Ning, Yang; Chen, Yong",2022,"Heterogeneity-aware and communication-efficient distributed statistical inference","Biometrika 109(1):67-83",10.1093/biomet/asab007,"abstract","heterogeneous sites; density-ratio tilting","1-2","O(p + p^2)","local MLE + score","Cramer-Rao bound attained if the number of sites is smaller in rate than n","two-index asymptotics","parametric likelihood","heterogeneity-aware surrogate","PARTIAL (C5/C4)","","abstract"44chen2019quantile,"Chen, Xi; Liu, Weidong; Zhang, Yichen",2019,"Quantile regression under memory constraint","Annals of Statistics 47(6):3244-3273",10.1214/18-AOS1777,"abstract","QR; batch size m","few","p^2 per round","smoothed scores","naive DC needs n = o(m^2); multi-round removes","Bahadur","quantile regression","multi-round removes the constraint","PARTIAL (C2)","","abstract"45chen2014split,"Chen, Xueying; Xie, Min-ge",2014,"A split-and-conquer approach for analysis of extraordinarily large data","Statistica Sinica 24:1655-1684",10.5705/ss.2013.088,"abstract","penalized GLM; K subsets","1","K(p + p^2)","penalized GLM","equivalence when K is controlled (condition NOT VERIFIED)","asymptotic normality","penalized GLM","weighted combination with local information","PARTIAL (C5)","","abstract"46dobriban2021averaging,"Dobriban, Edgar; Sheng, Yue",2021,"Distributed linear regression by averaging","Annals of Statistics 49(2):918-943",10.1214/20-AOS1984,"abstract","linear regression; random-matrix asymptotics","1","Kp","OLS","exact efficiency loss of weighted averaging","random matrix theory","linear regression","efficiency loss of averaging","low","","abstract"47szabo2020adaptive,"Szabo, Botond; van Zanten, Harry",2020,"Adaptive distributed methods under communication constraints","Annals of Statistics 48(4):2347-2380",10.1214/19-AOS1890,"abstract","nonparametric; bit budget","1-2","bits","-","minimax bounds under communication constraints","information-theoretic","nonparametric","bit accounting precedent","low","","abstract"48cai2022gaussian,"Cai, T. Tony; Wei, Hongji",2022,"Distributed Gaussian mean estimation under communication constraints: optimal rates and communication-efficient algorithms","Annals of Statistics 50(4):1992-2020",10.1214/21-AOS2167,"abstract","Gaussian mean; bit budget","1","bits","-","optimal rates (localization + refinement)","information-theoretic","Gaussian mean","bit accounting precedent","low","","abstract"49duchi2014optimality,"Duchi, John C.; Jordan, Michael I.; Wainwright, Martin J.; Zhang, Yuchen",2014,"Optimality guarantees for distributed statistical estimation","arXiv 1405.0782 (NeurIPS 2013 version)",https://arxiv.org/abs/1405.0782,"abstract","bit budgets","-","bits","-","communication lower bounds","information-theoretic","location / regression","lower bounds (universal, not algorithm-family)","low","Garg-Ma-Nguyen 2014 and Braverman et al. 2016 NOT VERIFIED","abstract"50kal2025adaptive,"Kal, Dennis; Szabo, Botond; Guhaniyogi, Rajarshi; Pillai, Natesh; Pati, Debdeep",2025,"Adaptive divide and conquer with two rounds of communication","arXiv 2508.17073",https://arxiv.org/abs/2508.17073,"abstract/intro","white noise model","2","(log2 n)^2 bits + optimal bits","local summaries","rate-optimal adaptive","minimax","nonparametric","bit accounting with two rounds","low","","pp. 1-2"51banerjee2019divide,"Banerjee, Moulinath; Durot, Cecile; Sen, Bodhisattva",2019,"Divide and conquer in nonstandard problems and the super-efficiency phenomenon","Annals of Statistics 47(2):720-757",10.1214/17-AOS1633,"abstract","cube-root problems","1","Kp","isotonic","super-efficiency","nonstandard","isotonic regression","none","NONE","","abstract"52minsker2019distributed,"Minsker, Stanislav",2019,"Distributed statistical estimation and rates of convergence in normal approximation","Electronic Journal of Statistics 13:5213-5252",10.1214/19-EJS1647,"abstract","robust aggregation","1","Kp","local","same rate if K <= O(sqrt N)","Berry-Esseen","M-estimators / MLE","none","NONE","","abstract"53luo2020renewable,"Luo, Lan; Song, Peter X.-K.",2020,"Renewable estimation and incremental inference in generalized linear models with streaming data sets","Journal of the Royal Statistical Society Series B 82(1):69-97",10.1111/rssb.12352,"abstract","streaming data","sequential","summaries","incremental Newton","consistency + normality","asymptotic","GLM","none","NONE","","abstract"54tan2022quantile,"Tan, Kean Ming; Battey, Heather; Zhou, Wen-Xin",2022,"Communication-constrained distributed quantile regression with optimal statistical guarantees","Journal of Machine Learning Research 23(272):1-61",https://jmlr.org/papers/v23/21-1269.html,"abstract","double-smoothed QR","near-constant","gradients / Hessians","smoothed objectives","communication-vs-error trade-off","finite-sample","quantile regression","trade-off (no Hessian compression)","low","","abstract"55alimisis2021quantized,"Alimisis, Foivos; Davies, Peter; Alistarh, Dan",2021,"Communication-efficient distributed optimization with quantized preconditioners","Proc. 38th ICML (arXiv 2102.07214)",https://arxiv.org/abs/2102.07214,"abstract","strongly convex; lattice quantization","iterative","quantized Hessians","local Hessians","communication complexity (optimization only)","optimization","convex ERM","quantized-Hessian algorithm without statistical K-exponent","low (no statistics)","","abstract"56lin2023proforma,"Lin, Lu; Li, Feng",2023,"Global debiased DC estimations for biased estimators via pro forma regression","TEST 32(2):726-758",10.1007/s11749-023-00850-5 (arXiv:1904.07477v2),"yes","non-identically distributed batches","1","batches x (estimate + covariate)","local estimator + structural covariate","Thms 3.3-3.4 sqrt(n)-consistent / normal for any N, m","regression-based","biased estimators","not block-size extrapolation","NONE / weak","","pp. 10-15"57li2024selective,"Li, Xuetong; Gao, Yuan; Chang, Hong; Huang, Danyang; Ma, Yingying; Pan, Rui; Qi, Haobo; Wang, Feifei; Wu, Shuyuan; Xu, Ke; Zhou, Jing; Zhu, Xuening; Zhu, Yingqiu; Wang, Hansheng",2024,"A selective review on statistical methods for massive data computation","arXiv 2403.11163 (journal NOT VERIFIED)",https://arxiv.org/abs/2403.11163,"yes","review","-","-","-","JDS bias O(1/n^2) + O(1/N); per-machine delete-one M-estimators","review","M-estimators","JDS 'readily applicable to distributed computing'","PARTIAL (C1)","","pp. 15-16"58