# 🧮 Beyond One-Step Distributed Maximum Likelihood **UQO Working Paper No. 28** — *Local bias cancellation, global synchronization, and the price of inexact curvature: a joint order–round frontier, its sharpness in an exact model, and measured communication costs* [](paper/main.pdf) [](requirements.txt) [](results/) [](experiments/) [](https://uqo.ca) [](mailto:simon-pierre.boucher@uqo.ca) > **TL;DR** — Each of $K$ workers debiases its local MLE by an order-$r$ split-sample extrapolation over nested sub-blocks; the coordinator averages and applies $q$ aggregated Newton steps. For fixed-dimensional regular likelihoods the result is first-order equivalent to the centralized MLE whenever $K = o(N^{1-1/(2^{q+1}r)})$ (half-block jackknife + one step: $K = o(N^{7/8})$ against the $K = O(N^{1/2})$ stated for the plain-average one-step estimator). The exponent is sharp for the construction in the exponential model (boundary shift $-\theta_0(\beta_r c^r)^{2^q}$, e.g. $-4\theta_0c^4$), and a perturbation theorem gives the exponent for stale, single-machine and $b$-bit quantized Hessians. Monte Carlo (exponential, logistic, Poisson; paired replications; $N$ up to $2^{22}$), an exact boundary study, ablations, and timed coordinator–worker benchmarks on the MacLustr cluster quantify the finite-sample costs. Components are known (Quenouille/Dhaene–Jochmans jackknife; Huang–Huo one-step); the joint frontier, its sharpness and the bit-precision result are new. No communication lower bound is claimed. --- ## 📌 Status | Item | Value | |---|---| | Version | 1.0 (October 2026) — working draft under internal review; see `FINAL_REVIEW.md` | | SSRN | not yet submitted | | Journal | — (candidates: Journal of Econometrics, Econometric Theory, JASA T&M, JCGS) | | Code & data | fully reproducible from `experiments/` (synthetic data only); raw Monte Carlo outputs committed in `results/raw/` with manifests | | Provenance | develops the exploratory note "Local Bias Correction versus Global Synchronization in Distributed Maximum Likelihood" (Boucher & "GPT-6.1", 2026-10-28), frozen in `references/initial_pamle/`; see `SOURCE_PROVENANCE.md` and the disclosure following the abstract | ## 🎯 What this paper does Distributed MLE with $N$ observations on $K$ workers of $m = N/K$ observations: averaging local MLEs keeps the $O(1/m)$ local bias, so plain averaging is equivalent to the centralized MLE only for $K = o(N^{1/2})$. Two known remedies — local debiasing and global Newton corrections — are analysed **jointly**. The local correction is the generalized (split-sample) jackknife with Vandermonde weights over nested sub-blocks of sizes $m/s_j$, ``` J_{r,k} = Σ_j w_j T_{s_j,k}, Σ_j w_j = 1, Σ_j w_j s_j^a = 0 (a = 1..r-1) θ^(0) = K^{-1} Σ_k J_{r,k}; θ ← θ − H_N(θ)^{-1} g_N(θ) (q times, aggregated score and Hessian) ``` Main results: Theorem 1 (frontier $K = o(N^{1-1/(2^{q+1}r)})$ with the centered random remainder of order $(m\sqrt K)^{-1}$ kept), Theorem 2 (exact exponential identities, boundary limits for every $(r,q)$, Newton basin $(0, 2\hat\theta_N)$, necessity), Theorem 3 (inexact curvature: stale averaged local Hessians $\Rightarrow 1-1/(2(r+q))$; single-machine Hessian $\Rightarrow 1-1/(2r+q)$, recovering CEASE's $(t+1)/(t+2)$ and CSL's $2/3$; quantization to $b \ge (\tfrac12 - r(1-\alpha))\log_2 N$ bits), Proposition 4 (unequal sizes, misspecification), Proposition 5 (cost model and exchange rate between a local level and a round). ## 🏆 Headline results | Object | Result | |---|---| | Sufficient exponent, half-block jackknife + 1 Newton step | $K = o(N^{7/8})$ (plain average + 1 step: $3/4$; quarter-block + 1 step: $11/12$) | | Exponent structure | depends on $(r,q)$ only through $D = 2^q r$: one round ⇔ doubling the local order | | Sharpness (exponential model) | $\sqrt N(\theta^{(1)}_2 - \hat\theta_N) \to -4\theta_0c^4$ when $K \sim cN^{7/8}$; divergence beyond | | Inexact curvature | stale: $1-1/(2(r+q))$; single machine: $1-1/(2r+q)$; bits for quantized derivatives: $(\tfrac12 - r(1-\alpha))\log_2 N$ | | Exact variance cost of the half-block correction | $\mathrm{Var}\,J_2 = \theta_0^2 m^2(m^2-6m+17)/((m-4)(m-2)^2(m-1)^2) = \theta_0^2(m^{-1}+4m^{-2}+O(m^{-3}))$, same first two terms as the raw MLE | | Monte Carlo and benchmarks | see `paper/tables/` (generated), `results/processed/summary.csv`, `results/processed/benchmarks_summary.csv` | ## 🗂 Repository layout ``` distributed-mle-synchronization/ ├── CLAUDE.md # the research mandate (verbatim) ├── PROJECT_MAP.md · SOURCE_PROVENANCE.md · PHASE_STATUS.md · research_log.md ├── LIMITATIONS_AND_CLAIMS.md · FINAL_REVIEW.md · FINAL_DELIVERY.md ├── references/initial_pamle/ # frozen prior draft + archive + SHA256SUMS.txt (read-only) ├── literature/ # audits A–D, PRIOR_ART_MATRIX.csv (56 rows), PRIORITY_AND_GAP.md ├── theory/ # ASSUMPTIONS.md, THEOREM_AUDIT.md (proof ledger), symbolic_checks.py (SymPy) ├── src/pamle/ # library: jackknife, partitioning, likelihoods, algorithms, communication, diagnostics, protocol, worker ├── experiments/ # run_monte_carlo.py (+configs/), scatter_cluster.py, run_boundary.py, run_ablations.py, run_skewed.py, │ # run_benchmarks.py, process_results.py, make_tables.py, EXPERIMENT_MANIFEST.csv ├── tests/ # pytest suite (weights, exact identities, derivatives, estimators, accounting, reproducibility, protocol) ├── results/{raw,processed,tables,figures,resource_profiles} ├── paper/ # main.tex, sections/, appendices/, tables/ (generated), figures/, references.bib → main.pdf ├── scripts/ # compile_paper.sh, validate_pdf.sh, reproduce_all.sh └── docs/ # cluster inventory, misc ``` ## 🚀 Quick start ```bash python3 -m pip install -r requirements.txt pytest -q # 30 tests (≈ 10 s) python3 theory/symbolic_checks.py # SymPy verification of every identity (≈ 1 min) python3 experiments/run_monte_carlo.py --config experiments/configs/main_exponential.yaml # laptop, ≈ 2 h python3 experiments/run_boundary.py; python3 experiments/run_ablations.py --model logistic; python3 experiments/run_skewed.py python3 experiments/scatter_cluster.py --configs experiments/configs/main_logistic.yaml experiments/configs/main_poisson.yaml experiments/configs/robustness.yaml --launch # cluster python3 experiments/run_benchmarks.py --config experiments/configs/benchmarks.yaml --out results/raw/benchmarks.jsonl # cluster python3 experiments/process_results.py --figures && python3 experiments/make_tables.py bash scripts/compile_paper.sh && bash scripts/validate_pdf.sh ``` Runtime: the exponential Monte Carlo (40 cells × 2000 replications) takes about two hours on an Apple M5 Max laptop; the logistic/Poisson/robustness grids (136 cells) were spread over 18 MacLustr nodes (several hours); the benchmarks need the cluster. ## 🗃 Data All data are synthetic and seeded (`seed_base = 20261028` plus a hash of the cell description); nothing proprietary. Raw per-replication outputs (`results/raw/*.csv`) carry a JSON manifest (git commit, Python/NumPy versions, host, wall time, SHA-256); `experiments/EXPERIMENT_MANIFEST.csv` collects them. ## 🔬 Methodology 1. Local split fits on nested sub-blocks, batched Newton with step halving and status codes — `src/pamle/likelihoods.py`, `algorithms.local_split_fits`. 2. Vandermonde weights, pilot — `jackknife.py`, `algorithms.worker_corrected_estimates`, `pilot_estimator`. 3. Global corrections (exact / stale / single-machine / quantized curvature), CSL, AEE, analytic Cox–Snell — `algorithms.global_newton`, `csl`, `aee_lin_xi`, `analytic_bias_correction`. 4. Communication accounting — `communication.py`; real protocol — `protocol.py`, `worker.py`. 5. Experiments and processing — `experiments/` (every table/figure regenerated from `results/`). ## 🖼 Figures & tables inventory | Figure | Content | |---|---| | `fig_rms_gap_vs_m_` | RMS scaled gap to the centralized MLE vs $m$ for all estimators | | `fig_rmse_theta0_vs_m_` | scaled RMSE vs $\theta_0$: bias–variance trade-off across orders | | `fig_gap_collapse_` | gaps against $K/N^{\alpha(r,q)}$ for several $N$ (collapse predicted by Theorem 1) | | `fig_coverage95_` | empirical 95 % coverage vs $m$ | | `fig_boundary_exponential` | boundary study: gap / asymptotic limit vs $m$ with finite-$m$ prediction | Tables (`paper/tables/*.tex`, generated by `experiments/make_tables.py`): `mc_.tex`, `mc_rates.tex`, `boundary.tex`, `abl_exponential.tex`, `abl_logistic.tex`, `skew.tex`, `bench.tex`, `numbers.tex`. ## ⚠️ Limitations See `LIMITATIONS_AND_CLAIMS.md`. Fixed $p$ and i.i.d. data only; sufficient exponents, sharp only for the construction in the exponential model; the boundary regime $m \asymp N^{1/(2^{q+1}r)}$ is reached only at very large $N$; higher orders inflate variance and can fail at small $m$; benchmarks are specific to one cluster and one Python/TCP implementation; no energy or cost measurements; no lower bounds. ## 📝 Citation ```bibtex @techreport{boucher2026distributedmle, author = {Boucher, Simon-Pierre}, title = {Beyond One-Step Distributed Maximum Likelihood: Local Bias Cancellation, Global Synchronization, and the Price of Inexact Curvature}, institution = {Universit\'e du Qu\'ebec en Outaouais, D\'epartement des sciences administratives}, type = {Working Paper}, number = {28}, year = {2026}, month = {October} } ``` ## 👤 Author & contact **Simon-Pierre Boucher** — Département des sciences administratives, Université du Québec en Outaouais (UQO), Gatineau, QC. 📧 [simon-pierre.boucher@uqo.ca](mailto:simon-pierre.boucher@uqo.ca) ## 🗒 Notes - Numbering: No. 28 is the next free number of the UQO series (`Recherche-UQO/README.md` stops at 27). The SPID database paper in `finance/russell3000-strategic-project-database` carries FIRC series No. 0001, not 28; see `SOURCE_PROVENANCE.md`. - Authorship: the title page lists Simon-Pierre Boucher only; a provenance and AI-use disclosure follows the abstract (the prior note's AI co-author designation was removed at the author's request on 2026-10-28). - Deviation from the template: `paper/main.tex` adds amsthm, algorithmic and TikZ packages; `sections/titlepage.tex` is the exact template copy.