# 🧮 Beyond One-Step Distributed Maximum Likelihood

UQO Working Paper No. 28 — Local bias cancellation, global synchronization, and the price of inexact curvature: a joint order–round frontier, its sharpness in an exact model, and measured communication costs

TL;DR — Each of $K$ workers debiases its local MLE by an order-$r$ split-sample extrapolation over nested sub-blocks; the coordinator averages and applies $q$ aggregated Newton steps. For fixed-dimensional regular likelihoods the result is first-order equivalent to the centralized MLE whenever $K = o(N^{1-1/(2^{q+1}r)})$ (half-block jackknife + one step: $K = o(N^{7/8})$ against the $K = O(N^{1/2})$ stated for the plain-average one-step estimator). The exponent is sharp for the construction in the exponential model (boundary shift $-\theta_0(\beta_r c^r)^{2^q}$, e.g. $-4\theta_0c^4$), and a perturbation theorem gives the exponent for stale, single-machine and $b$-bit quantized Hessians. Monte Carlo (exponential, logistic, Poisson; paired replications; $N$ up to $2^{22}$), an exact boundary study, ablations, and timed coordinator–worker benchmarks on the MacLustr cluster quantify the finite-sample costs. Components are known (Quenouille/Dhaene–Jochmans jackknife; Huang–Huo one-step); the joint frontier, its sharpness and the bit-precision result are new. No communication lower bound is claimed.

# 📌 Status

# 🎯 What this paper does

Distributed MLE with $N$ observations on $K$ workers of $m = N/K$ observations: averaging local MLEs keeps the $O(1/m)$ local bias, so plain averaging is equivalent to the centralized MLE only for $K = o(N^{1/2})$. Two known remedies — local debiasing and global Newton corrections — are analysed jointly. The local correction is the generalized (split-sample) jackknife with Vandermonde weights over nested sub-blocks of sizes $m/s_j$,

J_{r,k} = Σ_j w_j T_{s_j,k},    Σ_j w_j = 1,  Σ_j w_j s_j^a = 0 (a = 1..r-1)
θ^(0)   = K^{-1} Σ_k J_{r,k};   θ ← θ − H_N(θ)^{-1} g_N(θ)   (q times, aggregated score and Hessian)

Main results: Theorem 1 (frontier $K = o(N^{1-1/(2^{q+1}r)})$ with the centered random remainder of order $(m\sqrt K)^{-1}$ kept), Theorem 2 (exact exponential identities, boundary limits for every $(r,q)$, Newton basin $(0, 2\hat\theta_N)$, necessity), Theorem 3 (inexact curvature: stale averaged local Hessians $\Rightarrow 1-1/(2(r+q))$; single-machine Hessian $\Rightarrow 1-1/(2r+q)$, recovering CEASE's $(t+1)/(t+2)$ and CSL's $2/3$; quantization to $b \ge (\tfrac12 - r(1-\alpha))\log_2 N$ bits), Proposition 4 (unequal sizes, misspecification), Proposition 5 (cost model and exchange rate between a local level and a round).

# 🏆 Headline results

# 🗂 Repository layout

distributed-mle-synchronization/
├── CLAUDE.md                      # the research mandate (verbatim)
├── PROJECT_MAP.md · SOURCE_PROVENANCE.md · PHASE_STATUS.md · research_log.md
├── LIMITATIONS_AND_CLAIMS.md · FINAL_REVIEW.md · FINAL_DELIVERY.md
├── references/initial_pamle/      # frozen prior draft + archive + SHA256SUMS.txt (read-only)
├── literature/                    # audits A–D, PRIOR_ART_MATRIX.csv (56 rows), PRIORITY_AND_GAP.md
├── theory/                        # ASSUMPTIONS.md, THEOREM_AUDIT.md (proof ledger), symbolic_checks.py (SymPy)
├── src/pamle/                     # library: jackknife, partitioning, likelihoods, algorithms, communication, diagnostics, protocol, worker
├── experiments/                   # run_monte_carlo.py (+configs/), scatter_cluster.py, run_boundary.py, run_ablations.py, run_skewed.py,
│                                  #   run_benchmarks.py, process_results.py, make_tables.py, EXPERIMENT_MANIFEST.csv
├── tests/                         # pytest suite (weights, exact identities, derivatives, estimators, accounting, reproducibility, protocol)
├── results/{raw,processed,tables,figures,resource_profiles}
├── paper/                         # main.tex, sections/, appendices/, tables/ (generated), figures/, references.bib → main.pdf
├── scripts/                       # compile_paper.sh, validate_pdf.sh, reproduce_all.sh
└── docs/                          # cluster inventory, misc

# 🚀 Quick start

python3 -m pip install -r requirements.txt
pytest -q                                   # 30 tests (≈ 10 s)
bash scripts/check_lean.sh                  # Lean 4 + Mathlib: 27 theorems (needs elan; first build downloads the Mathlib cache)
python3 theory/symbolic_checks.py           # SymPy verification of every identity (≈ 1 min)
python3 experiments/run_monte_carlo.py --config experiments/configs/main_exponential.yaml   # laptop, ≈ 2 h
python3 experiments/run_boundary.py; python3 experiments/run_ablations.py --model logistic; python3 experiments/run_skewed.py
python3 experiments/scatter_cluster.py --configs experiments/configs/main_logistic.yaml experiments/configs/main_poisson.yaml experiments/configs/robustness.yaml --launch   # cluster
python3 experiments/run_benchmarks.py --config experiments/configs/benchmarks.yaml --out results/raw/benchmarks.jsonl               # cluster
python3 experiments/process_results.py --figures && python3 experiments/make_tables.py
bash scripts/compile_paper.sh && bash scripts/validate_pdf.sh

Runtime: the exponential Monte Carlo (40 cells × 2000 replications) takes about two hours on an Apple M5 Max laptop; the logistic/Poisson/robustness grids (136 cells) were spread over 18 MacLustr nodes (several hours); the benchmarks need the cluster.

# 🗃 Data

All data are synthetic and seeded (seed_base = 20261028 plus a hash of the cell description); nothing proprietary. Raw per-replication outputs (results/raw/*.csv) carry a JSON manifest (git commit, Python/NumPy versions, host, wall time, SHA-256); experiments/EXPERIMENT_MANIFEST.csv collects them.

# 🔬 Methodology

1. Local split fits on nested sub-blocks, batched Newton with step halving and status codes — src/pamle/likelihoods.py, algorithms.local_split_fits.
2. Vandermonde weights, pilot — jackknife.py, algorithms.worker_corrected_estimates, pilot_estimator.
3. Global corrections (exact / stale / single-machine / quantized curvature), CSL, AEE, analytic Cox–Snell — algorithms.global_newton, csl, aee_lin_xi, analytic_bias_correction.
4. Communication accounting — communication.py; real protocol — protocol.py, worker.py.
5. Experiments and processing — experiments/ (every table/figure regenerated from results/).

# 🖼 Figures & tables inventory

Tables (paper/tables/*.tex, generated by experiments/make_tables.py): mc_<design>.tex, mc_rates.tex, boundary.tex, abl_exponential.tex, abl_logistic.tex, skew.tex, bench.tex, numbers.tex.

# ⚠️ Limitations

See LIMITATIONS_AND_CLAIMS.md. Fixed $p$ and i.i.d. data only; sufficient exponents, sharp only for the construction in the exponential model; the boundary regime $m \asymp N^{1/(2^{q+1}r)}$ is reached only at very large $N$; higher orders inflate variance and can fail at small $m$; benchmarks are specific to one cluster and one Python/TCP implementation; no energy or cost measurements; no lower bounds.

# 📝 Citation

@techreport{boucher2026distributedmle,
  author      = {Boucher, Simon-Pierre},
  title       = {Beyond One-Step Distributed Maximum Likelihood: Local Bias Cancellation, Global Synchronization, and the Price of Inexact Curvature},
  institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives},
  type        = {Working Paper},
  number      = {28},
  year        = {2026},
  month       = {October}
}

# 👤 Author & contact

Simon-Pierre Boucher — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 simon-pierre.boucher@uqo.ca

# 🗒 Notes

• Numbering: No. 28 is the next free number of the UQO series (Recherche-UQO/README.md stops at 27). The SPID database paper in finance/russell3000-strategic-project-database carries FIRC series No. 0001, not 28; see SOURCE_PROVENANCE.md.
• Authorship: the title page lists Simon-Pierre Boucher only; a provenance and AI-use disclosure follows the abstract (the prior note's AI co-author designation was removed at the author's request on 2026-10-28).
• Deviation from the template: paper/main.tex adds amsthm, algorithmic and TikZ packages; sections/titlepage.tex is the exact template copy.