SPB Git

spb/wp28_uqo

Public
5.3 KB

# Literature audit A — ZDW 2013, Huang–Huo 2019, Lin–Xi 2011, Rosenblatt–Nadler 2016 (theorem level)

Method (verification task, 2026-10-28 18:47–18:53): full-text PDFs read (JMLR official PDF; arXiv 1511.01443v2; arXiv 1407.2724v2; Chalmers-hosted reprint of Lin & Xi with running header "Statistics and Its Interface Volume 4 (2011) 73–83"). Bibliographic records cross-checked with CrossRef. Springer's page for Huang–Huo is bot-blocked: theorem numbers for that paper refer to arXiv v2 (Nov 2015); the published text (36 pp. vs 31 pp.) is NOT VERIFIED.

# 1. Zhang, Duchi & Wainwright (2013), JMLR 14:3321–3363 (arXiv 1209.4129), full text read

Notation: N = mn samples, m machines (= our K), n = N/m (= our m). θ̄₁ = AVGM (eq. 2, p. 3324).

  • Assumptions (Sec. 2.1, 3.1, pp. 3324–3326): A1 compact convex Θ, θ* interior; A2 local strong convexity of population risk at θ*; A3 8th moments of gradient and Hessian deviation, Hessian L(x)-Lipschitz with 8th moments; A4 (SAVGM only) third-derivative Lipschitz with E[M⁸] < ∞. Example 1 (p. 3326): without Hessian Lipschitz continuity the local bias is Ω(n^{-1/2}).
  • Theorem 1 (AVGM, p. 3327): E‖θ̄−θ*‖² ≤ (2/(nm))E‖H^{-1}∇f‖² + (c/(λ²n²))[H² log d + (L²G²/λ²)E‖H^{-1}∇f‖²] + O(m^{-1}n^{-2}) + O(n^{-3}); i.e. O(1/N + K²/N²). Corollary 3 compares with the Hájek–Le Cam bound tr I^{-1}/N. The n^{-2} term is attributed to the O(1/n) local bias (p. 3329).
  • SAVGM (Sec. 2.2.2, Theorem 4, p. 3329): subsample of size ⌈rn⌉ per machine, θ_SAVGM = (θ̄₁ − rθ̄₂)/(1−r); MSE ≤ ((2+3r)/(1−r)²)(1/(nm))E‖H^{-1}∇f‖² + c[…]n^{-3}/(r(1−r)²) + O((1−r)^{-2}m^{-1}n^{-2}). Leading constant inflated by (2+3r)/(1−r)²; r → 0 recommended (r ∝ m^{3/2}/N).
  • K-vs-N conditions (p. 3330, eqs. 10–11): AVGM optimal rate iff m ≲ √N; SAVGM iff m ≲ N^{2/3}. These are MSE-rate statements, not first-order equivalence to the full MLE.
  • No Richardson / higher-order extrapolation, no jackknife (words absent); multiple subsamples left as an open question about variance (p. 3330). No one-step Newton comparison (Newton appears only as the LIBSVM local solver, p. 3337). Section 3.5 is an SGD variant.
  • Rounds: 1; uplink: one d-vector (two local fits for SAVGM).

# 2. Huang & Huo (2019), Math. Programming 174(1–2):41–76, DOI 10.1007/s10107-019-01369-0 (arXiv 1511.01443v2 read)

  • One-step estimator (Sec. 2.4, eq. 2.9): θ^(0) = simple average of local M-estimators; θ^(1) = θ^(0) − [M̈(θ^(0))]^{-1} Ṁ(θ^(0)) with M the GLOBAL empirical criterion: machines compute local gradient and local Hessian AT THE BROADCAST AVERAGE and send both (d + d² numbers); two rounds.
  • Assumptions (Sec. 3.1) copied from ZDW (compactness, negative-definite population Hessian, 8th moments, Hessian Lipschitz, concavity); no third-derivative condition.
  • Theorem 3.4: when k = O(√N), √N(θ^(1) − θ0) ⇒ N(0, Σ): "asymptotically equivalent to the centralized M-estimator". Remark: proof only needs a √N-consistent start.
  • Theorem 3.5: E‖θ^(1) − θ0‖² ≤ 2 Tr Σ/N + O(N^{-2}) + O(k⁴N^{-4}). Corollary 3.6: MLE efficiency. Lemma B.5: local bias ≤ C̃/n + O(n^{-2}); Lemma C.1 needs k = O(n) = O(√N) for √N-consistency of the average.
  • No bias correction, no jackknife (SAVGM only as a simulation competitor, Sec. 4). q = 1 only, no repeated rounds. Authors claim priority for the distributed one-step idea.
  • Note (auditor's derivation, not stated by the authors): the k⁴N^{-4} remainder is o(N^{-1}) whenever k = o(N^{3/4}).

# 3. Lin & Xi (2011), Statistics and Its Interface 4(1):73–83, DOI 10.4310/SII.2011.v4.n1.a8, full text read

  • AEE: A_k = −Σ_i ∂ψ(z_ki, β̂_nk)/∂β (local Jacobian at the LOCAL estimate), β̃ = (Σ A_k)^{-1} Σ A_k β̂_nk (eq. 4). One round, p + p² numbers per block. Motivated by OLAP/data streams.
  • Theorem 4.1 (K bounded): ‖AEE − EE_full‖ ≤ C₁(‖worst local error‖² + ‖full error‖²). Theorem 4.2: √N‖AEE − EE_full‖ → 0 if K = O(n^γ), 0 < γ < min{1−2α, 4α−1} ≤ 1/3 (⇒ K ≪ N^{1/4}; conversion is the auditor's). Theorem 5.2: GLM/QLE normality under the same condition.
  • No bias expansion, no correction, no Newton step, no variance estimator.

# 4. Rosenblatt & Nadler (2016), Information and Inference 5(4):379–404, DOI 10.1093/imaiai/iaw013 (arXiv 1407.2724v2 read)

  • Fixed p, m FIXED: Theorem 1 first-order equivalence of the average to the centralized estimator; Theorem 2 (second-order bias, Rilstone–Srivastava–Ullah expansions): B₂(θ̄) = δ/n = m·B₂(θ̂_N); Theorem 3 second-order MSE with excess ((m−1)/m)δδᵀ/n² + …; Remark 2 lists bias-reduction routes (Kim 2006, Liu & Ihler 2014, SAVGM, DDPCA, DANE) and warns bias is traded for variance. High-dimensional regime p/n → κ: Lemma 1 first-order loss linear in m.
  • No jackknife, no Newton, no K-vs-N growth condition.

# Summary of stated K-vs-N conditions

ZDW13: √N (AVGM), N^{2/3} (SAVGM), MSE-rate sense. HH19: k = O(√N) for the one-step estimator (Thm 3.4). LX11: K ≪ N^{1/4}. RN16: m fixed. None states a jackknife of local MLEs, a Newton step from a bias-corrected average, an N^{7/8} threshold, or an (r, q) frontier.

Local artefacts kept by the verification task: /tmp/wp28lit/{zdw2013,hh2019,rn2016}.{pdf,txt}, /tmp/wp28lit/linxi/.

Files · 0a1c470d4ca522b2576819112daf78116d0c1d99
Menu

Theme