WP28: phase 0 — workspace, frozen prior draft, replication lock
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
24 changed files +12,650 −0
added
.gitignore
+35 −0
@@ -0,0 +1,35 @@ | ||
| 1 | +# Raw & processed data — never committed (see data/raw/README.md for provenance) | |
| 2 | +data/raw/* | |
| 3 | +!data/raw/README.md | |
| 4 | +data/processed/* | |
| 5 | +!data/processed/.gitkeep | |
| 6 | + | |
| 7 | +# Python | |
| 8 | +__pycache__/ | |
| 9 | +*.pyc | |
| 10 | +.pytest_cache/ | |
| 11 | +.venv/ | |
| 12 | +*.egg-info/ | |
| 13 | + | |
| 14 | +# LaTeX build artifacts (paper/main.pdf IS committed) | |
| 15 | +paper/*.aux | |
| 16 | +paper/*.log | |
| 17 | +paper/*.out | |
| 18 | +paper/*.toc | |
| 19 | +paper/*.lof | |
| 20 | +paper/*.lot | |
| 21 | +paper/*.fls | |
| 22 | +paper/*.fdb_latexmk | |
| 23 | +paper/*.bbl | |
| 24 | +paper/*.blg | |
| 25 | +paper/*.spl | |
| 26 | +paper/*.synctex.gz | |
| 27 | +paper/*.run.xml | |
| 28 | +paper/*.bcf | |
| 29 | + | |
| 30 | +# Logs | |
| 31 | +logs/ | |
| 32 | +*.log | |
| 33 | + | |
| 34 | +# macOS | |
| 35 | +.DS_Store | |
added
CLAUDE.md
+1055 −0
@@ -0,0 +1,1055 @@ | ||
| 1 | +# CLAUDE.md — UQO RESEARCH WORKING PAPER 28 | |
| 2 | +## Research execution mandate: Distributed Maximum Likelihood, Local Bias Correction, and the Economics of Synchronization | |
| 3 | + | |
| 4 | +> **Mission type:** autonomous, multi-phase research program for Claude Code. | |
| 5 | +> | |
| 6 | +> **Required working paper identifier:** `WP28` / `WP0028` (adopt the exact naming convention found in the UQO research repository; do not create duplicate identifiers). | |
| 7 | +> | |
| 8 | +> **Principal researcher:** Simon-Pierre Boucher — Université du Québec en Outaouais (UQO). | |
| 9 | +> | |
| 10 | +> **Prior exploratory draft credits:** Simon-Pierre Boucher and GPT-6.1. Preserve this provenance and the user's requested GPT-6.1 attribution in the working draft. Confirm the applicable journal/venue authorship and AI-disclosure policy before a submission version; never falsely represent an AI system as a legally accountable human author. | |
| 11 | +> | |
| 12 | +> **Language:** English, throughout manuscript, proofs, code comments, figures, documentation and supplementary material. | |
| 13 | +> | |
| 14 | +> **Research domain:** theoretical econometrics, statistical optimization, distributed maximum likelihood, communication complexity, high-order bias correction, numerical statistics, heterogeneous computing. | |
| 15 | +> | |
| 16 | +> **Current date for literature cutoff:** 2026-10-28; search current material rather than assuming bibliographic currency. | |
| 17 | +> | |
| 18 | +> **Quality aspiration:** a genuinely original, technically rigorous, reproducible article potentially suitable for leading econometrics / statistics / statistical-computing journals. Quality is measured by new and correct results, not page count or grandiose claims. | |
| 19 | + | |
| 20 | +--- | |
| 21 | + | |
| 22 | +## 0. Non-negotiable operating instructions | |
| 23 | + | |
| 24 | +You are Claude Code acting as a research mathematician, econometrician, algorithm designer, literature reviewer, scientific software engineer, adversarial referee, and academic typesetter. This is NOT an instruction to paraphrase the exploratory note. It is a research mandate to independently validate, challenge, extend, and—if justified—replace its central ideas with a stronger, defensible contribution. | |
| 25 | + | |
| 26 | +**Execute in order, completing one phase's acceptance tests before starting the next.** Make substantive progress autonomously; do not stop after writing a research plan, a few claims, or a skeleton PDF. Use tools to actually inspect the repository, read the reference source, derive proofs, implement estimators, run tests and simulations, compile LaTeX, review output, and write reproducibility documentation. When a proof or experiment fails, report the failure and adapt the design rather than hiding it. Continue to a publishable-quality, internally consistent draft within the resources and access actually available. Never assert results from jobs that have not finished. | |
| 27 | + | |
| 28 | +### 0.1 Truthfulness and scientific integrity | |
| 29 | + | |
| 30 | +- No invented theorem, proof, assumption, dataset, benchmark, reference, DOI, parameter estimate, speedup, significance level, or journal acceptance. | |
| 31 | +- Do not claim that distributed gradients, one-step Newton, split-sample jackknife, Richardson extrapolation, `K = o(N^{7/8})`, or `K = o(N^{11/12})` are intrinsically novel. Investigate who has already obtained each idea or bound. | |
| 32 | +- Explicitly differentiate **proved results**, **numerically supported observations**, **heuristics**, **conditional conjectures**, and **future research**. | |
| 33 | +- An asymptotic partition-rate theorem is *not* a wall-clock acceleration theorem. A low communication-round count is *not* the same as a low total number of transmitted bytes, and neither is proof of a favorable energy or financial cost. | |
| 34 | +- Do not conflate model-correct i.i.d. likelihood inference with misspecified, dependent, heterogeneous or growing-dimensional settings. | |
| 35 | +- Every estimator must be reproducibly defined; every theorem must give precise assumptions; every proof must close all essential steps. | |
| 36 | +- A dead end with a mathematical explanation is better than a fabricated breakthrough. A strong negative result or counterexample may itself become the contribution. | |
| 37 | +- Treat `GPT-6.1` authorship as provenance from the existing draft and user instruction, not a substitute for human responsibility or journal compliance. | |
| 38 | + | |
| 39 | +### 0.2 Research process and files | |
| 40 | + | |
| 41 | +1. Inspect the current repository before creating files. Read its root `CLAUDE.md`, README, WP index, templates, `wp27` conventions, the existing WP28 slot if any, scripts and build infrastructure. | |
| 42 | +2. Respect local conventions; use `WP28` as the canonical ID in metadata, paper front matter, output names and progress logs. | |
| 43 | +3. Never overwrite an unrelated existing WP28 without first recording the collision. If WP28 is already occupied, preserve it and create a conflict report / safe working branch; do not silently renumber without authorization. | |
| 44 | +4. Keep all work self-contained under the repository's conventional working-paper directory. If no convention exists, use `working-papers/WP28-distributed-mle/`. | |
| 45 | +5. Keep `research_log.md` updated with dated decisions, theorem changes, computations, citations verified, failures and exact reproducibility commands. | |
| 46 | +6. Use the researcher's available CPU/GPU/cluster systems only when provisioned and authorized. Benchmark actual hardware; never assume the existence, credentials, health, bandwidth, or availability of a MacLustr node. | |
| 47 | +7. Do not expose API keys, SSH keys, passwords, endpoint secrets or private datasets in source files, logs, paper, artifacts or commits. | |
| 48 | +8. Avoid destructive commands, forced pushes and unrelated repository modifications. Do not publish a preprint or submit to a journal without explicit instruction. | |
| 49 | +9. Complete the project from initial inspection through PDF validation; stop only when a real blocker makes an indispensable step impossible. Document blocked items, and continue with whatever can be completed. | |
| 50 | + | |
| 51 | +--- | |
| 52 | + | |
| 53 | +## 1. Research brief and target positioning | |
| 54 | + | |
| 55 | +### 1.1 Working title | |
| 56 | + | |
| 57 | +**Beyond One-Step Distributed Maximum Likelihood: Higher-Order Bias Cancellation, Communication–Computation Frontiers, and Adaptive Econometric Inference** | |
| 58 | + | |
| 59 | +Alternative title if contributions warrant it: | |
| 60 | + | |
| 61 | +**The Price of Synchronization: Sharp Statistical and Computational Frontiers for Distributed Maximum Likelihood** | |
| 62 | + | |
| 63 | +Use a sober title supported by the final actual contribution. Keep the initial draft title, *Local Bias Correction versus Global Synchronization in Distributed Maximum Likelihood*, as the starting point, not as a fixed requirement. | |
| 64 | + | |
| 65 | +### 1.2 Central research question | |
| 66 | + | |
| 67 | +> For regular nonlinear likelihood models with `N` observations and `K` distributed partitions, how can additional local computation replace global network synchronization while preserving the first-order inferential properties of centralized maximum likelihood? What is the optimal tradeoff between local bias order, number of global Newton-like corrections, numerical conditioning, communicated bytes, elapsed time, and interval coverage? | |
| 68 | + | |
| 69 | +The **research novelty target** is not merely a fractional exponent. Seek a mathematically precise, technically defensible **statistical–communication–computation frontier** and a practical policy for choosing local correction depth and global synchronization rounds under measurable machine and network constraints. Determine whether such a contribution is already established and formulate the exact gap before claiming novelty. | |
| 70 | + | |
| 71 | +### 1.3 Required outputs (not aspirational placeholders) | |
| 72 | + | |
| 73 | +- A fully sourced, original English research article, preferably approximately 25–40 substantive pages excluding appendices *if the research supports this length*; no padding. | |
| 74 | +- A mathematical appendix with complete proofs, exact regularity conditions, auxiliary lemmas, failure cases and counterexamples. | |
| 75 | +- Well-tested, documented, executable reference implementations of all estimators. | |
| 76 | +- Reproducible Monte Carlo experiments and, when feasible, actual distributed / multi-process / multi-node benchmarks. | |
| 77 | +- Clear publication-grade tables, vector figures, formulas and algorithm pseudocode. | |
| 78 | +- A verified `.bib` bibliography and a literature/priority audit. | |
| 79 | +- A deterministic one-command or documented small-command build producing the final PDF. | |
| 80 | +- Code, environments, checksums, seeds, raw and processed results, resources and run manifests sufficient for independent reproduction. | |
| 81 | +- A candid paper-specific `LIMITATIONS_AND_CLAIMS.md`, `THEOREM_AUDIT.md`, `PRIOR_ART_MATRIX.csv`, `EXPERIMENT_MANIFEST.csv` and `FINAL_REVIEW.md`. | |
| 82 | +- If a stronger mathematical contribution cannot be validated, deliver a rigorous and accurately framed working paper, not an inflated method paper. | |
| 83 | + | |
| 84 | +--- | |
| 85 | + | |
| 86 | +## 2. Source-of-truth: inspect the existing PAMLE paper and data | |
| 87 | + | |
| 88 | +The user has already prepared an exploratory paper and reproducibility archive. If supplied in the Claude Code environment, read them in full before modifying any mathematics: | |
| 89 | + | |
| 90 | +- `PAMLE_Boucher_GPT-6.1.pdf` | |
| 91 | +- `PAMLE_Boucher_GPT-6.1_LaTeX_and_Reproducibility.zip` | |
| 92 | + | |
| 93 | +The source archive is expected to contain files such as `pamle.tex`, `references.bib`, `monte_carlo.py`, `simulation_summary.csv`, `simulation_replicates.csv`, and a figure. These names describe the handoff; resolve actual paths instead of assuming they are mounted under the same directory. If missing, recover from the project context or the article text provided by the researcher; never invent missing numeric data. | |
| 94 | + | |
| 95 | +**Source chronology:** prior study dated 2026-10-28, by Simon-Pierre Boucher and GPT-6.1; a 16-page introductory research note. The existing Monte Carlo used independent exact-Gamma simulation for an exponential MLE, `N = 1,048,576`, local sizes `m in {256, 64, 16}`, seed `20261028`, and **500** replicates per design. Do not silently describe these as 1,000 replications or as physical-cluster experiments. Verify all numbers directly against the actual CSV and code before citing them. | |
| 96 | + | |
| 97 | +### 2.1 Baseline formulas to check, not blindly trust | |
| 98 | + | |
| 99 | +Use `N` for sample size, `K` for number of worker partitions, `m = N/K`, fixed parameter dimension `p`, truth `theta_0`, centralized MLE `theta_hat_N`: | |
| 100 | + | |
| 101 | +```text | |
| 102 | +L_N(theta) = N^{-1} sum_i log f(X_i; theta) | |
| 103 | +g_N(theta) = grad L_N(theta) | |
| 104 | +H_N(theta) = Hessian L_N(theta) | |
| 105 | + | |
| 106 | +local_mle[k] = argmax_{theta} m^{-1} sum_{i in block k} log f(X_i; theta) | |
| 107 | +local_jack2[k] = 2 * local_mle[k] - (half_A_mle[k] + half_B_mle[k]) / 2 | |
| 108 | +pilot = K^{-1} sum_k local_jack2[k] | |
| 109 | +final = pilot - inv(H_N(pilot)) @ g_N(pilot) | |
| 110 | +``` | |
| 111 | + | |
| 112 | +**Never explicitly invert the Hessian in production code:** use a stable linear solve / trust-region / damped step with condition checks. The displayed inverse is mathematical notation only. | |
| 113 | + | |
| 114 | +Previously derived sufficient rates (conditional, not novelty claims): | |
| 115 | + | |
| 116 | +| Initialization | Global Newton corrections | Sufficient `K` regime for first-order centralized equivalence | | |
| 117 | +|---|---:|---| | |
| 118 | +| Average ordinary local MLEs | 0 | `K = o(N^(1/2))` | | |
| 119 | +| Average ordinary local MLEs | 1 | `K = o(N^(3/4))` | | |
| 120 | +| Half-block jackknife, cancels `m^{-1}` bias | 1 | `K = o(N^(7/8))` | | |
| 121 | +| Quarter-block higher-order extrapolation, cancels `m^{-1}, m^{-2}` bias | 1 | `K = o(N^(11/12))` | | |
| 122 | +| Average ordinary local MLEs | 2 | `K = o(N^(7/8))` | | |
| 123 | + | |
| 124 | +These are **sufficient asymptotic restrictions under nontrivial assumptions**, NOT maximal possible machine counts, NOT minimax lower bounds, NOT evidence of speedups, and NOT automatically valid for growing `p` or heterogeneous workers. | |
| 125 | + | |
| 126 | +### 2.2 Essential stochastic decomposition | |
| 127 | + | |
| 128 | +Require a common asymptotic influence function and a local remainder: | |
| 129 | + | |
| 130 | +```text | |
| 131 | +theta_hat_n - theta_0 = (1/n) sum_i psi(X_i) + R_n, | |
| 132 | +E[psi(X)] = 0, | |
| 133 | +E||R_n||^2 <= C / n^2, | |
| 134 | +E[R_n] = b_1/n + b_2/n^2 + O(n^{-3}). | |
| 135 | +``` | |
| 136 | + | |
| 137 | +For the half-block jackknife, the influence term is preserved *exactly*. The corrected local remainder has a centered stochastic contribution of order `O_p(1 / (m * sqrt(K)))` after worker averaging; this must not be omitted: | |
| 138 | + | |
| 139 | +```text | |
| 140 | +pilot - theta_hat_N = O_p( m^{-2} + (m*sqrt(K))^{-1} + N^{-1} ). | |
| 141 | +``` | |
| 142 | + | |
| 143 | +Under suitable local Lipschitz Hessian and inverse Hessian control: | |
| 144 | + | |
| 145 | +```text | |
| 146 | +||final - theta_hat_N|| = O_p( ||pilot - theta_hat_N||^2 ). | |
| 147 | +``` | |
| 148 | + | |
| 149 | +Consequently: | |
| 150 | + | |
| 151 | +```text | |
| 152 | +sqrt(N) ||final - theta_hat_N|| | |
| 153 | + = O_p( K^4/N^(7/2) + K/N^(3/2) + N^(-3/2) ). | |
| 154 | +``` | |
| 155 | + | |
| 156 | +This supports `K = o(N^(7/8))`, provided consistency and all conditions hold uniformly along the relevant triangular array. Every `O_p` statement must be checked carefully, including dependence across half-sample and full-sample estimators and possible `p`-dependence in constants. | |
| 157 | + | |
| 158 | +### 2.3 Higher-order extrapolation to scrutinize | |
| 159 | + | |
| 160 | +For local whole-block, half-block and quarter-block averages `T_1, T_2, T_4`, respectively: | |
| 161 | + | |
| 162 | +```text | |
| 163 | +J_3 = (8/3) T_1 - 2 T_2 + (1/3) T_4. | |
| 164 | +``` | |
| 165 | + | |
| 166 | +Check the Vandermonde moment equations: | |
| 167 | + | |
| 168 | +```text | |
| 169 | +sum w_j = 1, | |
| 170 | +sum w_j * s_j = 0, | |
| 171 | +sum w_j * s_j^2 = 0, | |
| 172 | +``` | |
| 173 | + | |
| 174 | +where `s_j` is the partition multiplier. This cancels the first two expected bias terms **if an appropriate expansion holds**. Quantify finite-sample variance amplification and coefficient growth. If high-order corrections produce unstable or non-feasible parameter vectors, address it explicitly. Do not claim higher order is always better. | |
| 175 | + | |
| 176 | +### 2.4 Exact exponential example that must be independently rederived | |
| 177 | + | |
| 178 | +For `X_i ~ Exp(rate = theta_0)`: | |
| 179 | + | |
| 180 | +```text | |
| 181 | +theta_hat_n = 1 / Xbar_n | |
| 182 | +E[theta_hat_n] = theta_0 * n/(n-1), n > 1 | |
| 183 | +Bias(jack2) = -2 theta_0 / [(m-1)(m-2)], m > 2 | |
| 184 | +Newton(x) = 2x - x^2 Xbar_N | |
| 185 | +Newton(x) - theta_hat_N = -(x-theta_hat_N)^2 / theta_hat_N | |
| 186 | +``` | |
| 187 | + | |
| 188 | +At the boundary `K/N^(7/8) -> c > 0`, the previous paper derived: | |
| 189 | + | |
| 190 | +```text | |
| 191 | +sqrt(N) (theta_one_step_jack2 - theta_hat_N) | |
| 192 | + ->p -4 theta_0 c^4. | |
| 193 | +``` | |
| 194 | + | |
| 195 | +Reproduce and verify this limit, including centering, constants and distributional conditions. Important caveat: the exponential family has a trivially aggregable finite-dimensional sufficient statistic, so it is NOT evidence of communication optimality or a need for approximate MLE. Use it as a mathematical sanity test and exact counterexample laboratory. | |
| 196 | + | |
| 197 | +--- | |
| 198 | + | |
| 199 | +## 3. PHASE 0 — Repository reconnaissance and reproducibility lock | |
| 200 | + | |
| 201 | +**Actions** | |
| 202 | + | |
| 203 | +1. Detect repository root and project structure; inventory other working papers, especially `WP27`, the registry/index, document preamble, citation style, quality requirements, and naming patterns. | |
| 204 | +2. Search for prior paper, archive and relevant internal data, but never scrape or transmit private material to external services without authorization. | |
| 205 | +3. Extract the existing archive to an immutable `references/initial_pamle/` directory if available; record checksums. | |
| 206 | +4. Read `pamle.tex` line by line for definitions, theorem statements, source provenance, experimental settings and any mathematical defects. | |
| 207 | +5. Run the existing simulation at its original seed and parameters. Verify row counts, method names, table numbers, plots and deterministic outputs (allowing documented floating-point tolerance). | |
| 208 | +6. Compile the old PDF separately if toolchain permits; identify existing warnings and shortcomings. | |
| 209 | +7. Create the target WP28 directory using repository conventions. Do not rename other WP IDs. | |
| 210 | +8. Write `PROJECT_MAP.md`, `SOURCE_PROVENANCE.md`, `PHASE_STATUS.md`, and `research_log.md`. | |
| 211 | + | |
| 212 | +**Gate 0: pass only when** all old claims and artifacts are inventoried, every existing number is tagged as replicated / not replicated / unavailable, and a WP28 workspace exists without clobbering other work. | |
| 213 | + | |
| 214 | +--- | |
| 215 | + | |
| 216 | +## 4. PHASE 1 — Exhaustive state-of-the-art / priority investigation | |
| 217 | + | |
| 218 | +Before deciding the main theorem is new, research the following intersecting bodies of literature through publisher pages, journal archives, arXiv, RePEc, Crossref/OpenAlex where accessible, and backward/forward citations. Do not trust search snippets or automated summaries in place of reading the relevant proofs. Record publication and preprint dates to evaluate priority. Search from foundational literature through **2026-10-28**. | |
| 219 | + | |
| 220 | +### 4.1 Minimum anchor references to verify | |
| 221 | + | |
| 222 | +1. Zhang, Duchi and Wainwright (2013), *Communication-Efficient Algorithms for Statistical Optimization*, Journal of Machine Learning Research 14:3321–3363; https://www.jmlr.org/papers/v14/zhang13b.html . Check averaging, subsampling bias correction, rate claims and communication accounting. | |
| 223 | +2. Huang and Huo (2019), *A Distributed One-Step Estimator*, Mathematical Programming 174:41–76; DOI `10.1007/s10107-019-01369-0`; preprint https://arxiv.org/abs/1511.01443 . Check the exact initialization assumptions, local/global curvature and one-step proof. | |
| 224 | +3. Dhaene and Jochmans (2015), *Split-panel Jackknife Estimation of Fixed-effect Models*, Review of Economic Studies 82(3):991–1030; DOI `10.1093/restud/rdv007`. Check split-sample theory, higher-order extrapolation and limitations in dependent panels. | |
| 225 | +4. Jordan, Lee and Yang (2019), *Communication-Efficient Distributed Statistical Inference*, JASA 114(526):668–681; DOI `10.1080/01621459.2018.1429274`. Study surrogate likelihood / CSL as an important competing communication-reduction strategy. | |
| 226 | +5. Fan, Guo and Wang, *Communication-Efficient Accurate Statistical Estimation*, verify current journal and bibliographic details. Read the full algorithm and convergence results, particularly multi-round corrections. | |
| 227 | +6. Rosenblatt and Nadler (2016), *On the optimality of averaging in distributed statistical learning*, Information and Inference; DOI `10.1093/imaiai/iaw013`. | |
| 228 | +7. Related work on distributed debiasing, bias-corrected one-shot estimation, Richardson extrapolation, high-order influence functions, federated M-estimation, communication lower bounds, Hessian sketching, K-step one-step estimators, and heterogeneity. | |
| 229 | + | |
| 230 | +### 4.2 Specific novelty threats to investigate | |
| 231 | + | |
| 232 | +- The Zhang–Duchi–Wainwright *subsampling/bootstrapping* approach already improves communication-efficient statistical error rates. Determine whether their techniques implicitly or explicitly subsume our bias cancellation. | |
| 233 | +- Some one-shot or multi-round distributed estimators may already attain the same `K` regime, with weaker assumptions or better time complexity. | |
| 234 | +- Split-sample / higher-order jackknife corrections are known. The weights alone are not publishable novelty. | |
| 235 | +- Multi-round Newton may replicate higher-order bias advantages; quantify this with matched communication budgets. | |
| 236 | +- A distributed *one-step* method still requires an uplink of local estimates, a broadcast of the common pilot, and an uplink of derivatives. Carefully define whether this is “two communication phases,” “one additional round,” or multiple network exchanges. Never advertise it as a literal zero- or one-message protocol. | |
| 237 | +- In regular parametric exponential families, sufficient statistics yield exact distributed MLE; compare against this trivial baseline and acknowledge the limitation. | |
| 238 | +- Finite sample `m`, increasing `K`, and high dimensional `p` may break second-order expansions or stable Hessian inversion. | |
| 239 | + | |
| 240 | +### 4.3 Required literature outputs | |
| 241 | + | |
| 242 | +`literature/PRIOR_ART_MATRIX.csv` with at least: | |
| 243 | + | |
| 244 | +```text | |
| 245 | +citation_key,authors,year,title,venue,doi_or_url,verified_full_text, | |
| 246 | +assumptions,number_of_rounds,upstream_bytes,local_computation, | |
| 247 | +statistical_rate,proof_type,applicable_models,overlap_with_WP28, | |
| 248 | +novelty_threat,notes,page_numbers_checked | |
| 249 | +``` | |
| 250 | + | |
| 251 | +Plus `literature/PRIORITY_AND_GAP.md` with: | |
| 252 | + | |
| 253 | +- 20–40 **verified relevant references as a search target**, not fabricated filler; include additional sources only if material. | |
| 254 | +- At least five papers compared at theorem/algorithm level, not only abstract level. | |
| 255 | +- A *claim-by-claim* map: known, possibly new, demonstrably new with specified proof, or undecided. | |
| 256 | +- One precisely stated **Candidate Contribution A** (strongest), **B** (feasible backup) and **C** (negative result / optimality counterexample). | |
| 257 | +- An explicit GO / REVISE / DROP decision for each proposed claim. | |
| 258 | + | |
| 259 | +**Gate 1:** no unqualified novelty language is allowed until the closest competing theorems have been inspected and cited. | |
| 260 | + | |
| 261 | +--- | |
| 262 | + | |
| 263 | +## 5. PHASE 2 — Build a mathematical specification and attack the assumptions | |
| 264 | + | |
| 265 | +### 5.1 Baseline model | |
| 266 | + | |
| 267 | +Start with i.i.d. observations and a correctly specified fixed-dimensional regular likelihood: | |
| 268 | + | |
| 269 | +- `X_1,...,X_N ~ P_{theta_0}` independent; | |
| 270 | +- `theta_0` interior to an open parameter set in `R^p`; | |
| 271 | +- information matrix `I(theta_0)` positive definite; | |
| 272 | +- enough differentiability and envelope/moment conditions for the score/Hessian expansions; | |
| 273 | +- local estimators exist and are measurable and consistently select the correct branch; | |
| 274 | +- local and global Hessian inverses are uniformly controlled with high probability; | |
| 275 | +- partitioning is random or deterministic independent of the observations, with equal block sizes for the first theorem; | |
| 276 | +- integers `K = K_N`, `m = N/K`, and `m -> infinity` along the triangular array. | |
| 277 | + | |
| 278 | +Separate assumptions for stochastic linearization, bias expansion, Hessian smoothness, central-limit theorem, moments and parameter-domain feasibility. State precisely which theorem uses which assumption. | |
| 279 | + | |
| 280 | +### 5.2 Definitions and proof checks | |
| 281 | + | |
| 282 | +Derive, with all terms shown: | |
| 283 | + | |
| 284 | +1. Influence expansion of MLE local estimates, including a mean-square bound on nonlinear remainder. | |
| 285 | +2. Bias expansions `b_j / m^j` uniform enough to handle multiple partition sizes. | |
| 286 | +3. Exact cancellation of the first-order stochastic term under the linear combination of whole-block and sub-block estimators. | |
| 287 | +4. Bounds for centered corrections and the covariances between nested local samples. | |
| 288 | +5. Bounds for differences relative to the *centralized MLE*, not just relative to `theta_0`. | |
| 289 | +6. The quadratic Newton remainder from integral Taylor expansion, including Hessian invertibility at both the pilot and points along the path. | |
| 290 | +7. Feasibility and positivity of parameters after extrapolation, including constrained or reparameterized MLEs. | |
| 291 | +8. Consequences for feasible variance estimation and confidence-interval coverage. | |
| 292 | +9. Critical boundaries and local alternatives, distinguishing sufficient, necessary-for-this-construction, and truly minimax-optimal results. | |
| 293 | + | |
| 294 | +### 5.3 Required edge cases / mathematical traps | |
| 295 | + | |
| 296 | +- Bias coefficient cancellations do not automatically imply a high-order `L2` remainder. | |
| 297 | +- `O_p` constants must not secretly diverge with `p`, partition count or bias-correction order. | |
| 298 | +- Asymptotic exponents require the small-o condition, not merely `K <= N^alpha` with a fixed nonvanishing proportionality. | |
| 299 | +- A shared global influence term can cancel; if local distributions differ, that identity may fail. | |
| 300 | +- For local MLE non-existence (e.g. complete separation in logit), document exact finite-sample safeguards and their statistical implications. | |
| 301 | +- Jackknife weights can be negative, so a rate, variance or probability estimate may leave its parameter space. | |
| 302 | +- A multi-step method can diverge if its initial point lies outside the attraction basin; use cautious local statements or explicit globalization. | |
| 303 | +- Distinguish observed Hessian, expected Fisher information, and sandwich uncertainty under misspecification. | |
| 304 | +- Do not use exact sufficient-statistic examples to make universal computational lower-bound claims. | |
| 305 | + | |
| 306 | +### 5.4 Mathematical validation support | |
| 307 | + | |
| 308 | +Use symbolic algebra and high-precision numeric checks to verify coefficients, moment formulas, Taylor identities and boundary constants. Mathematica/SymPy/Sage/Lean may supplement proofs, but machine-checked simplification is not a substitute for a mathematical justification. If Lean is installed, formalize at least one deterministic algebraic lemma as a bonus, not as a blocker. | |
| 309 | + | |
| 310 | +**Gate 2:** every proposed theorem has assumptions, conclusion, proof strategy, test cases and a `proved / conditional / open / false` label in `THEOREM_AUDIT.md`. | |
| 311 | + | |
| 312 | +--- | |
| 313 | + | |
| 314 | +## 6. PHASE 3 — Derive a GENERAL HIGH-ORDER BIAS-CORRECTION FAMILY | |
| 315 | + | |
| 316 | +This is a core mathematical task; derive the construction rather than copying the `r=2,3` formulas. | |
| 317 | + | |
| 318 | +### 6.1 Split-granularity extrapolation | |
| 319 | + | |
| 320 | +For positive integers `s_1,...,s_r` dividing `m`, define `T_{s_j,k}` as the average of the `s_j` local MLEs computed on equal disjoint sub-blocks of size `m/s_j`. For `s_1 = 1`, it is the full-block MLE. | |
| 321 | + | |
| 322 | +Assume for a sub-block size `n`: | |
| 323 | + | |
| 324 | +```text | |
| 325 | +E(theta_hat_n) = theta_0 + b_1/n + ... + b_{r-1}/n^(r-1) + O(n^(-r)). | |
| 326 | +``` | |
| 327 | + | |
| 328 | +Let `w_1,...,w_r` solve: | |
| 329 | + | |
| 330 | +```text | |
| 331 | +sum_j w_j = 1, | |
| 332 | +sum_j w_j s_j^a = 0 for a=1,...,r-1. | |
| 333 | +``` | |
| 334 | + | |
| 335 | +Define the local corrected estimator: | |
| 336 | + | |
| 337 | +```text | |
| 338 | +J_{r,k} = sum_j w_j T_{s_j,k}. | |
| 339 | +``` | |
| 340 | + | |
| 341 | +Prove, subject to precise assumptions: | |
| 342 | + | |
| 343 | +- the `r-1` leading expected bias coefficients cancel; | |
| 344 | +- the common first-order influence function is preserved; | |
| 345 | +- centered correction remainder has `L2` norm at most `O(m^{-1})` for **fixed `r` and controlled weights**; | |
| 346 | +- worker averaging yields `O_p((m sqrt(K))^{-1})` centered remainder; | |
| 347 | +- mean bias is `O(m^{-r})`. | |
| 348 | + | |
| 349 | +A required leading bound is: | |
| 350 | + | |
| 351 | +```text | |
| 352 | +||pilot_r - theta_hat_N|| | |
| 353 | + = O_p( m^(-r) + 1/(m sqrt(K)) + N^(-1) ). | |
| 354 | +``` | |
| 355 | + | |
| 356 | +Do not claim this bound if its proof fails for a considered model. | |
| 357 | + | |
| 358 | +### 6.2 Numerical conditioning and high-order weights | |
| 359 | + | |
| 360 | +Study weight explosion, conditioning of Vandermonde systems, finite precision error, and variance inflation as `r` grows. Evaluate partition patterns including `s={1,2,4,...}` and less aggressive alternatives, integer divisibility issues, and Richardson-like extrapolation with uneven sub-blocks. Optimize the choice of `s_j` under a prescribed variance-inflation or computation budget, if possible. | |
| 361 | + | |
| 362 | +Derive exact or bounded covariance between nested MLEs where tractable. Test coefficient condition numbers and the effective sample size of high-order corrected estimators. | |
| 363 | + | |
| 364 | +### 6.3 Order selection | |
| 365 | + | |
| 366 | +Explore a **data-adaptive choice of local bias order `r`**, but do not assert that choosing `r` from the data preserves the fixed-order theorem. If selection changes the estimator's asymptotic distribution, provide sample-splitting, independent pilot selection, uniform stochastic bounds, or an honest restricted claim. Treat the adaptive theory as an open research target until proved. | |
| 367 | + | |
| 368 | +**Gate 3:** derive weights from the linear system programmatically, verify moment constraints, and present a formal theorem plus numerical conditioning analysis. | |
| 369 | + | |
| 370 | +--- | |
| 371 | + | |
| 372 | +## 7. PHASE 4 — Prove the statistical–synchronization frontier | |
| 373 | + | |
| 374 | +### 7.1 Core general theorem (candidate) | |
| 375 | + | |
| 376 | +Let `r >= 1` be the effective local residual bias order and `q >= 0` be the number of globally aggregated Newton corrections. With fixed `r`, `q` and controlled constants, and with the pilot bound from Phase 3, derive a sufficient regime: | |
| 377 | + | |
| 378 | +```text | |
| 379 | +K = o( N^[ 1 - 1 / ( 2^(q+1) * r ) ] ). | |
| 380 | +``` | |
| 381 | + | |
| 382 | +The derivation should state explicitly how repeated quadratic convergence acts on BOTH: | |
| 383 | + | |
| 384 | +- deterministic bias `m^{-r}`; | |
| 385 | +- stochastic remainder `1/(m sqrt(K))`; | |
| 386 | +- global remainder `N^{-1}`. | |
| 387 | + | |
| 388 | +Show exactly when the bias restriction is dominant. Do not hide alternative restrictions that arise from nuisance parameters, approximation error, numerical tolerance, increasing dimension or nonuniform constants. | |
| 389 | + | |
| 390 | +### 7.2 Strengthening the contribution | |
| 391 | + | |
| 392 | +Go beyond a sufficient exponent. Investigate at least THREE of the following, with proofs if feasible: | |
| 393 | + | |
| 394 | +**A. Boundary distributions.** For `K ~ c N^alpha`, obtain centered or shifted limiting distributions, including explicit leading bias constants, for several nontrivial likelihood models. | |
| 395 | + | |
| 396 | +**B. Necessity / sharpness for a specified estimator family.** Construct examples showing the exponent cannot be relaxed *for the particular fixed `(r,q)` construction*, even though other estimators may do better. | |
| 397 | + | |
| 398 | +**C. Communication-aware decision theorem.** Given local fitting cost `T_fit(m,p)`, network latency `lambda`, bandwidth `B`, derivative transfer size and correction depth, characterize conditions under which `(r+1, q)` beats `(r, q+1)` in elapsed time at equal asymptotic precision. | |
| 399 | + | |
| 400 | +**D. Adaptive frontier.** Derive an oracle or regret-style bound for choosing `(r,q)` from pilot measurements, controlling selection-induced statistical error and network uncertainty. | |
| 401 | + | |
| 402 | +**E. Heterogeneous partitions.** Extend to unequal `m_k` and weighted aggregation, showing how bias corrections and local information geometry must change. | |
| 403 | + | |
| 404 | +**F. Robustness to model misspecification.** Establish asymptotic equivalence to centralized pseudo-MLE with sandwich covariance under defensible conditions, and distinguish this from Fisher-efficiency claims. | |
| 405 | + | |
| 406 | +**G. Approximate derivatives.** Bound how Hessian approximation, quantization, compression, inexact linear solves, or straggler failures perturb first-order equivalence. | |
| 407 | + | |
| 408 | +**H. Increasing dimension.** If realistic, derive explicit `p_N` dependencies in remainder and Hessian bounds. Do not relabel a fixed-`p` theorem as a high-dimensional theorem. | |
| 409 | + | |
| 410 | +### 7.3 Proof depth requirement | |
| 411 | + | |
| 412 | +Every theorem must have: | |
| 413 | + | |
| 414 | +- a complete statement and correctly scoped assumptions; | |
| 415 | +- an informative lemma sequence; | |
| 416 | +- no magical `o_p(1)` steps; | |
| 417 | +- all triangular-array / uniformity issues handled; | |
| 418 | +- moment and bias constants traced through the proof; | |
| 419 | +- a discussion of necessity versus sufficiency; | |
| 420 | +- a counterexample or violation for at least one essential assumption; | |
| 421 | +- a direct comparison with the closest prior theorem. | |
| 422 | + | |
| 423 | +**Gate 4:** at least one nontrivial theorem beyond a rephrasing of known one-step-plus-jackknife results is fully proved, **or** the report explicitly concludes that novelty is not yet proved and reframes the paper around validated insights and counterexamples. | |
| 424 | + | |
| 425 | +--- | |
| 426 | + | |
| 427 | +## 8. PHASE 5 — Exact analytic models and sharp counterexamples | |
| 428 | + | |
| 429 | +### 8.1 Exponential(rate) benchmark: full analytic treatment | |
| 430 | + | |
| 431 | +Prove using Gamma moments: | |
| 432 | + | |
| 433 | +```text | |
| 434 | +E[1 / Xbar_n], Var[1 / Xbar_n], | |
| 435 | +E[local jack2], Var[local jack2], | |
| 436 | +E[local jack3] (if integrability allows), | |
| 437 | +exact Newton identity, | |
| 438 | +finite-sample error identities where possible, | |
| 439 | +boundary distribution constants. | |
| 440 | +``` | |
| 441 | + | |
| 442 | +Derive from the gamma density rather than just reporting formulas. Check the exact conditions on `m` needed for moments to exist; distinguish bias existence, variance existence and higher-order integrability. Verify that the `jack3` formula yields feasible positive-rate estimates often enough for Newton and document when it does not. | |
| 443 | + | |
| 444 | +### 8.2 At least two additional nonlinear likelihoods | |
| 445 | + | |
| 446 | +Try and prioritize: | |
| 447 | + | |
| 448 | +1. Logistic regression with controlled predictors and nonseparation; include `p > 1`, conditioning, Newton numerical stability and exact/approximate centralized solutions. | |
| 449 | +2. Gamma GLM, Poisson regression, negative binomial regression, Weibull survival, or beta regression where likelihood nonlinearity makes bias correction meaningful. | |
| 450 | +3. A heteroskedastic nonlinear regression or simple volatility likelihood (e.g. GARCH) as a challenging extension, with dependence theory kept separate. | |
| 451 | + | |
| 452 | +For each: | |
| 453 | + | |
| 454 | +- write the log-likelihood, score and observed Hessian; | |
| 455 | +- specify data generating process and identification; | |
| 456 | +- establish which assumptions are theoretically justified versus numerically plausible; | |
| 457 | +- seek an explicit second-order bias coefficient, exact finite-sample identity, or asymptotic counterexample; | |
| 458 | +- report failures, singularities, boundary fits and weak identification; | |
| 459 | +- include a case where jackknife+Newton loses to another method. | |
| 460 | + | |
| 461 | +### 8.3 Adversarial configurations | |
| 462 | + | |
| 463 | +Build an explicit failure gallery: | |
| 464 | + | |
| 465 | +- `m` too small for a local MLE to exist; | |
| 466 | +- nearly singular information matrices; | |
| 467 | +- distributed covariate shift / non-i.i.d. local partitioning; | |
| 468 | +- outlier contamination and model misspecification; | |
| 469 | +- large `p/m` and separation; | |
| 470 | +- Hessian inexactness / quantization; | |
| 471 | +- jackknife noise amplification at high order; | |
| 472 | +- extreme network latency or slowest-node bottlenecks. | |
| 473 | + | |
| 474 | +**Gate 5:** exact exponential results verified by symbolic algebra and independent numerical checks; at least two other likelihoods implemented, with honest assumptions. | |
| 475 | + | |
| 476 | +--- | |
| 477 | + | |
| 478 | +## 9. PHASE 6 — Implement a research-grade benchmark library | |
| 479 | + | |
| 480 | +### 9.1 Technology | |
| 481 | + | |
| 482 | +Preferred stack: Python 3.11+ with NumPy, SciPy, pandas, matplotlib and pytest, plus JAX/PyTorch if useful for verified automatic differentiation. Optional Rust/C++ numerical kernels and MPI/Ray/Dask for genuine distributed tests only when justified by the environment. Pin dependencies and exact versions. No hard requirement that a GPU be present. | |
| 483 | + | |
| 484 | +Suggested project tree (adapt to existing UQO layout): | |
| 485 | + | |
| 486 | +```text | |
| 487 | +WP28-distributed-mle/ | |
| 488 | +├── CLAUDE.md # copy/symlink this mandate, if appropriate | |
| 489 | +├── README.md | |
| 490 | +├── PHASE_STATUS.md | |
| 491 | +├── research_log.md | |
| 492 | +├── paper/ | |
| 493 | +│ ├── main.tex | |
| 494 | +│ ├── preamble.tex | |
| 495 | +│ ├── sections/ | |
| 496 | +│ ├── appendices/ | |
| 497 | +│ ├── references.bib | |
| 498 | +│ └── figures/ | |
| 499 | +├── literature/ | |
| 500 | +│ ├── PRIOR_ART_MATRIX.csv | |
| 501 | +│ └── PRIORITY_AND_GAP.md | |
| 502 | +├── theory/ | |
| 503 | +│ ├── THEOREM_AUDIT.md | |
| 504 | +│ ├── ASSUMPTIONS.md | |
| 505 | +│ ├── proofs_check.py | |
| 506 | +│ └── symbolic_checks.py | |
| 507 | +├── src/pamle/ | |
| 508 | +│ ├── likelihoods.py | |
| 509 | +│ ├── local_fit.py | |
| 510 | +│ ├── partitioning.py | |
| 511 | +│ ├── jackknife.py | |
| 512 | +│ ├── derivatives.py | |
| 513 | +│ ├── algorithms.py | |
| 514 | +│ ├── aggregation.py | |
| 515 | +│ ├── communication.py | |
| 516 | +│ ├── diagnostics.py | |
| 517 | +│ └── runtime.py | |
| 518 | +├── experiments/ | |
| 519 | +│ ├── configs/ | |
| 520 | +│ ├── run_monte_carlo.py | |
| 521 | +│ ├── run_benchmarks.py | |
| 522 | +│ ├── run_ablations.py | |
| 523 | +│ └── EXPERIMENT_MANIFEST.csv | |
| 524 | +├── tests/ | |
| 525 | +│ ├── test_math_identities.py | |
| 526 | +│ ├── test_jackknife_weights.py | |
| 527 | +│ ├── test_models_and_gradients.py | |
| 528 | +│ ├── test_estimators.py | |
| 529 | +│ ├── test_asymptotic_rates.py | |
| 530 | +│ ├── test_communication_accounting.py | |
| 531 | +│ └── test_reproducibility.py | |
| 532 | +├── data/ # synthetic outputs + explicitly permitted data | |
| 533 | +├── results/ | |
| 534 | +│ ├── raw/ | |
| 535 | +│ ├── processed/ | |
| 536 | +│ ├── tables/ | |
| 537 | +│ ├── figures/ | |
| 538 | +│ └── resource_profiles/ | |
| 539 | +├── scripts/ | |
| 540 | +│ ├── setup.sh | |
| 541 | +│ ├── reproduce_all.sh | |
| 542 | +│ ├── compile_paper.sh | |
| 543 | +│ └── validate_pdf.sh | |
| 544 | +├── LIMITATIONS_AND_CLAIMS.md | |
| 545 | +├── FINAL_REVIEW.md | |
| 546 | +└── FINAL_DELIVERY.md | |
| 547 | +``` | |
| 548 | + | |
| 549 | +### 9.2 Mandatory estimator implementations | |
| 550 | + | |
| 551 | +Implement clean, common-interface versions of: | |
| 552 | + | |
| 553 | +- Full centralized maximum likelihood, using a trusted optimizer and tight tolerances. | |
| 554 | +- Average of independently fitted local MLEs. | |
| 555 | +- Average local MLE + **one** global Newton correction. | |
| 556 | +- Average local MLE + **two** global Newton corrections. | |
| 557 | +- Half-block jackknife initial estimator, without global correction. | |
| 558 | +- Half-block jackknife + **one** global Newton correction. | |
| 559 | +- Higher-order (quarter-block) jackknife, without global correction. | |
| 560 | +- Higher-order jackknife + **one** global Newton correction. | |
| 561 | +- Generalized `r`-th order split estimator + `q` global corrections. | |
| 562 | +- Credible implementations of at least two strong literature baselines where practical, such as surrogate likelihood/CSL and another relevant distributed estimator. If an exact faithful implementation is infeasible, identify the limitation and do not use a misnamed approximation. | |
| 563 | +- Optional bias-corrected local information-weighted averaging and adaptive method if justified by theory. | |
| 564 | + | |
| 565 | +### 9.3 Correctness before optimization | |
| 566 | + | |
| 567 | +- Unit tests: scalar and multivariate gradients/Hessians against high-accuracy finite differences and/or independent autodiff. | |
| 568 | +- Test the jackknife weights satisfy their moment equations up to floating-point tolerance. | |
| 569 | +- Test worker repartitioning, deterministic seeds, missing partitions, small `m`, singular Hessians and parameters near boundaries. | |
| 570 | +- For exponential likelihood, test exact Gamma moments and exact Newton identity over numerous random inputs. | |
| 571 | +- For logit, guard against separation, singularity and ill-conditioned curvature; distinguish exact MLE from ridge-penalized substitute. | |
| 572 | +- Check all algorithms agree with the centralized solution to specified tolerance when theory predicts it. | |
| 573 | +- Ensure `np.linalg.inv` is NOT used for production Newton steps. Use `solve`, regularization if reported, and diagnostics. | |
| 574 | +- Track optimization failures rather than discard failed replications silently. | |
| 575 | +- Capture and report both estimator validity and computational success rate. | |
| 576 | +- If implementing damped Newton or projection for robustness, distinguish its theoretical algorithm from the idealized undamped formula. | |
| 577 | + | |
| 578 | +**Gate 6:** all critical tests pass and the code reconstructs original-paper figures within stated tolerances before large experiments begin. | |
| 579 | + | |
| 580 | +--- | |
| 581 | + | |
| 582 | +## 10. PHASE 7 — Statistical Monte Carlo: exhaustive design, not a decorative chart | |
| 583 | + | |
| 584 | +### 10.1 Design matrix | |
| 585 | + | |
| 586 | +Run a factorial/sequential design covering: | |
| 587 | + | |
| 588 | +- `N`: logarithmic sequence spanning small, moderate and large sample sizes, chosen based on available RAM/compute; at least four distinct `N` values in the main rate study. | |
| 589 | +- `K`: logarithmic grid including values below, near and above `N^(1/2)`, `N^(3/4)`, `N^(7/8)`, `N^(11/12)` when integer block sizes and computation permit. | |
| 590 | +- `m`: explicitly report local sample sizes; do not quote exponents when the corresponding `m` fails to grow. | |
| 591 | +- `p`: small scalar case, moderate `p`, and a stress test for `p/m` approaching a problematic regime. | |
| 592 | +- Models: exponential(rate), logistic regression, plus at least one additional regular nonlinear likelihood. | |
| 593 | +- DGP conditions: well specified, misspecified (clearly labeled), varying Fisher condition numbers, balanced and deliberately skewed partitioning. | |
| 594 | +- Correction methods: raw averaging, one-/two-step Newton, jack2, jack3, and new method if supported. | |
| 595 | +- Network budgets: ideal communication-free statistical experiment, high-latency/low-bandwidth emulation, actual distributed test when possible. | |
| 596 | +- Repetitions: target at least 1,000 per **main statistical design** where practical; adaptively allocate computation and report actual replications and Monte Carlo standard errors. Do not confuse this target with previously executed 500-replication pilot. | |
| 597 | + | |
| 598 | +### 10.2 Required metrics | |
| 599 | + | |
| 600 | +Measure all of the following, as applicable: | |
| 601 | + | |
| 602 | +```text | |
| 603 | +bias, standard deviation, MSE, RMSE, | |
| 604 | +N-scaled and sqrt(N)-scaled errors vs theta_0, | |
| 605 | +sqrt(N)-scaled estimator gap vs centralized MLE, | |
| 606 | +coverage and length of 90%, 95%, 99% confidence intervals, | |
| 607 | +condition number, Newton residual norm, convergence rate, | |
| 608 | +optimization failure / nonexistence / infeasible parameter rate, | |
| 609 | +realized worker fits, actual floating-point runtime, memory, | |
| 610 | +communication rounds, downlink + uplink bytes, p50/p90/p99 latency, | |
| 611 | +energy if actually measured and instrumented. | |
| 612 | +``` | |
| 613 | + | |
| 614 | +Use paired replications where comparing estimators: same synthetic sample or sufficient statistics, same split, same DGP and random stream across methods. Publish Monte Carlo standard errors or confidence bands and record simulation seeds. | |
| 615 | + | |
| 616 | +### 10.3 Statistical diagnostics | |
| 617 | + | |
| 618 | +- Plot `log RMSE` against `log N` and investigate observed slopes; compare with theoretical slopes only when regimes are sufficiently large. | |
| 619 | +- Plot scaled estimator gaps as a function of `K/N^alpha`, not only raw `K`. | |
| 620 | +- Verify exact boundary-shift constants in exponential likelihood using several `c` values and increasing `N`. | |
| 621 | +- Plot model-specific bias–variance tradeoffs for jackknife correction order. | |
| 622 | +- For interval inference, compare nominal versus empirical coverage and uncertainty estimator consistency. | |
| 623 | +- Run sensitivity analyses for unusually large jackknife weights, nonlinear curvature and near-boundary estimates. | |
| 624 | +- Label every figure with the estimation procedure, DGP, sample size, number of partitions, replication count, units and uncertainty quantification. | |
| 625 | +- Report finite-sample failures rather than condition on the subset of successful trials without noting selection. | |
| 626 | +- No fabricated or manually inserted results. All paper numbers and figures must be regenerated from committed CSV/Parquet outputs. | |
| 627 | + | |
| 628 | +### 10.4 Reproducibility manifest | |
| 629 | + | |
| 630 | +For each completed run write: | |
| 631 | + | |
| 632 | +```text | |
| 633 | +run_id,git_commit,python_version,dependency_lock_sha256, | |
| 634 | +seed,N,K,m,p,DGP,method,optimizer,tolerance,replicates, | |
| 635 | +parallel_backend,node_count,hardware,OS,wall_seconds, | |
| 636 | +communicated_bytes,rounds,status,raw_output_sha256 | |
| 637 | +``` | |
| 638 | + | |
| 639 | +**Gate 7:** full tables exist, figures directly follow data, and results do not claim asymptotic behavior unsupported by the tested regime. | |
| 640 | + | |
| 641 | +--- | |
| 642 | + | |
| 643 | +## 11. PHASE 8 — REAL communication and compute benchmarks | |
| 644 | + | |
| 645 | +Do not call an in-memory mathematical partition a distributed cluster benchmark. Execute independent workers/processes or actual machines, record network traffic and synchronization overhead, and include an empirical systems section only if there is genuine evidence. | |
| 646 | + | |
| 647 | +### 11.1 Benchmark architecture | |
| 648 | + | |
| 649 | +Implement a coordinator/worker protocol with explicit events: | |
| 650 | + | |
| 651 | +```text | |
| 652 | +Stage A workers fit local MLE / split MLEs in parallel; | |
| 653 | +Stage B workers send corrected local parameter vectors; | |
| 654 | +Stage C coordinator aggregates and broadcasts theta^(0); | |
| 655 | +Stage D workers evaluate score + Hessian at global theta^(0); | |
| 656 | +Stage E workers send derivatives; | |
| 657 | +Stage F coordinator aggregates derivatives and solves Newton system; | |
| 658 | +Stage G additional correction steps repeat D-F, if requested. | |
| 659 | +``` | |
| 660 | + | |
| 661 | +Account for every broadcast, parameter vector, gradient vector, Hessian, message header, serialization cost, worker startup, scheduler delay, data movement and retry. Distinguish *network message count*, *communication rounds*, *transferred floating-point values* and *actual byte totals*. Specify the communication model: star coordinator vs all-reduce vs decentralized. For dense second derivatives, expect `O(K p^2)` uplink numeric payloads per correction, with additional downlink payloads and overhead. | |
| 662 | + | |
| 663 | +### 11.2 Hardware scope | |
| 664 | + | |
| 665 | +The researcher may have access to a MacLustr collection of Mac Studio / Mac mini nodes and other compute. **Inventory live, permitted resources**, do not hard-code model counts from stale descriptions. If available, benchmark up to available levels such as 1, 2, 4, 8 and 16 workers and more only when practical. If not available, perform a verified single-host multi-process baseline and clearly state its limits. | |
| 666 | + | |
| 667 | +Capture: | |
| 668 | + | |
| 669 | +- processor/model, CPU architecture, number of cores, RAM, accelerator availability; | |
| 670 | +- OS, BLAS backend, Python/library versions; | |
| 671 | +- network interface/link rate, observed bandwidth, latency and topology; | |
| 672 | +- worker startup and orchestration overhead; | |
| 673 | +- local-fit time distribution and straggler delay; | |
| 674 | +- derivative compute time and Hessian solve time; | |
| 675 | +- coordinator CPU/RAM burden; | |
| 676 | +- speedup, parallel efficiency and resource utilization; | |
| 677 | +- where measured, electricity and dollar-cost assumptions. | |
| 678 | + | |
| 679 | +### 11.3 The economics of synchronization | |
| 680 | + | |
| 681 | +Derive and empirically calibrate a cost model, such as: | |
| 682 | + | |
| 683 | +```text | |
| 684 | +T_total(r,q,K,p,N) | |
| 685 | + = T_local(r, N/K, p, worker_resources) | |
| 686 | + + T_schedule(K) | |
| 687 | + + T_aggregate_pilot(K,p) | |
| 688 | + + q*[T_broadcast(K,p) + T_derivatives(N/K,p) | |
| 689 | + + T_collect(K,p^2) + T_linear_solve(p)] | |
| 690 | + + T_stragglers(K) + T_failures(K). | |
| 691 | +``` | |
| 692 | + | |
| 693 | +Treat this as a **proposed model**, not a proved identity. Measure the terms. Model overlapping compute/communication and asynchronous variants if supported, and distinguish sums from critical paths in parallel execution. | |
| 694 | + | |
| 695 | +Study the crossover at equal statistical accuracy: | |
| 696 | + | |
| 697 | +- jack2 + 1 global step versus ordinary pilot + 2 global steps; | |
| 698 | +- jack3 + 1 global step versus jack2 + more rounds; | |
| 699 | +- exact/sufficient-statistic pooling, where applicable; | |
| 700 | +- CSL/surrogate likelihood and other strong baselines; | |
| 701 | +- dense Hessian communication versus Hessian-vector products or approximate derivatives. | |
| 702 | + | |
| 703 | +Derive a decision boundary involving latency, bandwidth, worker compute speed, `m`, and `p`. If it is empirical rather than general, call it empirical. | |
| 704 | + | |
| 705 | +### 11.4 Acknowledge benchmark limitations | |
| 706 | + | |
| 707 | +- Single-host processes do not validate cross-machine network scaling. | |
| 708 | +- Local wall-clock speedups do not imply total cost benefits. | |
| 709 | +- `K` partitions need not equal `K` physical hosts. | |
| 710 | +- Large `p` may make Hessian communication dominant; do not extrapolate scalar examples. | |
| 711 | +- Asynchronous updates change the mathematical estimator and may not inherit exact-theorem guarantees. | |
| 712 | + | |
| 713 | +**Gate 8:** any hardware claims have run logs, profiler measurements and manifests. Otherwise explicitly state that real distributed performance is not established. | |
| 714 | + | |
| 715 | +--- | |
| 716 | + | |
| 717 | +## 12. PHASE 9 — Stronger directions / actual discovery search | |
| 718 | + | |
| 719 | +Select the strongest viable research contribution based on proofs + novelty audit + experiments, rather than trying to force all directions into one unfocused manuscript. | |
| 720 | + | |
| 721 | +### Track A — Optimal local correction versus global synchronization | |
| 722 | + | |
| 723 | +Mathematically characterize when one extra local extrapolation stage replaces one global Newton synchronization, holding inferential accuracy fixed. Pursue sharp upper/lower bounds under a *specified* family of communication/computation models. Provide an experimentally calibrated phase diagram. | |
| 724 | + | |
| 725 | +### Track B — Adaptive bias order under latency constraints | |
| 726 | + | |
| 727 | +Design a runtime selection policy for `(r,q)` based on locally estimable bias, variance, Hessian conditioning and network cost. Prove nonpathological selection consistency, oracle-rate inequality or finite-sample dominance within a stated model, if achievable. Use valid independence/selection handling. | |
| 728 | + | |
| 729 | +### Track C — Statistically efficient inexact derivatives | |
| 730 | + | |
| 731 | +Study compressed or sketched Hessians/gradients. Characterize derivative error tolerances, e.g., sufficient `||delta_g||` and `||delta_H||` orders for first-order equivalent inference. Combine this with bias cancellation to reduce *bytes*, not just rounds. Establish all matrix perturbation steps. | |
| 732 | + | |
| 733 | +### Track D — Heterogeneous / federated econometrics | |
| 734 | + | |
| 735 | +Generalize the scheme to uneven worker counts, unequal sample sizes, covariate distribution shifts, model misspecification, and possibly overlapping samples; derive weighted bias corrections and robust variance. Prove exact scope rather than handwave a heterogeneous CLT. | |
| 736 | + | |
| 737 | +### Track E — Lower bounds, counterexamples and impossibility | |
| 738 | + | |
| 739 | +If speed/novelty claims do not survive scrutiny, identify an informative hard instance or impossibility for a given class of local-and-round-limited protocols; distinguish algorithm-family lower bounds from universal distributed estimation lower bounds. This can anchor a strong theory paper on its own. | |
| 740 | + | |
| 741 | +### Explicit selection rule | |
| 742 | + | |
| 743 | +At the end of this phase, record: | |
| 744 | + | |
| 745 | +```text | |
| 746 | +Primary theorem claim: | |
| 747 | +Nearest existing theorem(s): | |
| 748 | +Precise difference in assumptions/conclusion: | |
| 749 | +Complete proof available (Y/N): | |
| 750 | +Counterexample search completed (Y/N): | |
| 751 | +Numerical supporting evidence available (Y/N): | |
| 752 | +Communication benefit established (mathematical/empirical/neither): | |
| 753 | +Retain, weaken, replace, or drop: | |
| 754 | +``` | |
| 755 | + | |
| 756 | +**Gate 9:** one coherent article thesis, not six unrelated novelty claims. | |
| 757 | + | |
| 758 | +--- | |
| 759 | + | |
| 760 | +## 13. PHASE 10 — Write the paper as a serious journal submission | |
| 761 | + | |
| 762 | +### 13.1 Mandatory manuscript structure | |
| 763 | + | |
| 764 | +Adapt chapter allocation to findings; recommended: | |
| 765 | + | |
| 766 | +1. **Introduction.** The problem, economic/statistical motivation, exact literature gap, clearly delimited contributions, and why the tradeoff matters. | |
| 767 | +2. **Related literature.** A substantive, fair comparison of one-shot and multi-round distributed MLE, bias correction, surrogate likelihood and communication-limited inference. | |
| 768 | +3. **Econometric environment and distributed protocol.** Likelihood, partitions, assumptions, local estimators, messages, computational model. | |
| 769 | +4. **General bias-correction construction.** Weight system, common influence preservation, variance cost, numerical conditioning. | |
| 770 | +5. **Main theoretical results.** Theorem statements, interpretation, sufficient/frontier regimes, proof sketch with complete appendix. | |
| 771 | +6. **Exact models and boundaries.** Exponential exact identities and nontrivial nonlinear examples, including failure regimes. | |
| 772 | +7. **Computation–communication frontier.** Model, tradeoff theorems or conditional predictions, protocol comparison. | |
| 773 | +8. **Monte Carlo evidence.** Complete design, replication count, Monte Carlo uncertainty, bias/MSE/coverage/failure tables and figures. | |
| 774 | +9. **Real distributed benchmarks (only if actually run).** Hardware, environment, transfer accounting, elapsed-time and crossover analysis. | |
| 775 | +10. **Robustness / extensions.** Misspecification, heterogeneous partitions, inexact Hessians, high dimension or dependency as appropriately substantiated. | |
| 776 | +11. **Discussion and limitations.** Known methods, non-novel pieces, remaining open issues, domain restrictions. | |
| 777 | +12. **Conclusion.** Short, proportional to evidence, no marketing language. | |
| 778 | +13. **References.** Verified, machine-readable metadata. | |
| 779 | +14. **Technical appendices.** Every proof, asymptotic expansion, supplementary models, algorithms, additional sensitivity tables, replication instructions. | |
| 780 | + | |
| 781 | +### 13.2 Writing quality and tone | |
| 782 | + | |
| 783 | +Write in precise, natural, restrained academic English (top econometrics / theoretical statistics style). Avoid generic AI prose and unsupported superlatives: “groundbreaking,” “revolutionary,” “paradigm shift,” “legendary,” “remarkable,” “unprecedented,” “significantly better” without formal statistical support. Favor concise argumentation, strong results, intuitive transitions and properly qualified claims. | |
| 784 | + | |
| 785 | +- State exactly what is new versus already known. | |
| 786 | +- Make propositions, lemmas, theorems, assumptions, definitions, remarks and corollaries internally consistent. | |
| 787 | +- Use stable notation `N, K, m, p, r, q` throughout. | |
| 788 | +- Clearly distinguish *inference equivalence* to a centralized estimator from *optimality* or *exact equality*. | |
| 789 | +- Place technical rates and conditions adjacent to the statements they qualify. | |
| 790 | +- Treat normal confidence intervals as requiring a valid variance estimator; mere point-estimator convergence is not enough to claim valid coverage. | |
| 791 | +- Tables and plots must have units, populations, seeds/replicates when relevant, and intelligible notes. | |
| 792 | +- Do not claim runtime improvements based on FLOP counts without actual measurement or justified analytic model. | |
| 793 | +- Do not cite a source for a theorem it does not actually establish. | |
| 794 | + | |
| 795 | +### 13.3 Authorship, affiliations and funding | |
| 796 | + | |
| 797 | +Working manuscript attribution should preserve: | |
| 798 | + | |
| 799 | +```text | |
| 800 | +Simon-Pierre Boucher | |
| 801 | +Université du Québec en Outaouais | |
| 802 | + | |
| 803 | +GPT-6.1 | |
| 804 | +AI contribution / provenance of prior theoretical draft | |
| 805 | +``` | |
| 806 | + | |
| 807 | +If the existing UQO research series has a formal metadata / disclosure template, comply with that template and document any change to author display. The journal submission version may need GPT-6.1 removed from the formal author list and placed in transparent AI-use disclosure; **flag this for the human author rather than concealing the difference**. Never invent a model email address, institutional affiliation or ORCID. | |
| 808 | + | |
| 809 | +Research program funding line to include **only if confirmed for this particular WP28 project**: | |
| 810 | + | |
| 811 | +> Research funded by the FIRC (Fonds institutionnel de recherche et de création), Université du Québec en Outaouais. | |
| 812 | + | |
| 813 | +Because this line has been used for other UQO working papers, verify that its use on WP28 is appropriate rather than automatically claiming funding. | |
| 814 | + | |
| 815 | +### 13.4 Style / LaTeX quality | |
| 816 | + | |
| 817 | +- Follow native UQO working-paper style if one exists. Otherwise use a clean, sophisticated, predominantly white journal design with readable text and restrained accent color; no dark cover or promotional brochure aesthetics. | |
| 818 | +- 11pt or similarly comfortable font; professional math typesetting; linked PDF contents, hyperlinked references, booktabs tables, float control, clever cross-references and consistent equation numbering. | |
| 819 | +- Use `amsmath`, `amsthm`, `mathtools`, `booktabs`, `microtype`, `hyperref`, `cleveref`, `natbib`/`biblatex` as appropriate and available. | |
| 820 | +- Prefer vector figures (PDF/SVG converted properly) and generated TikZ for simple diagrams; no rasterized text. | |
| 821 | +- Centralize macros and theorem styles in the preamble; compile without undefined references or citations. | |
| 822 | +- Make the title page include WP28, date, authors, affiliations/disclosure, abstract, keywords and JEL codes (`C13`, `C15`, `C55`, `C63` subject to fit). | |
| 823 | +- Include algorithm pseudocode that accurately reflects communications and synchronized derivative evaluations. | |
| 824 | +- Check all math fits the page; no overfull/underfull warnings that materially harm appearance. | |
| 825 | +- Do not artificially inflate the page count. | |
| 826 | + | |
| 827 | +### 13.5 Bibliography rules | |
| 828 | + | |
| 829 | +- Source all bibliographic entries from verifiable journal DOI or publisher pages, supplemented by preprint dates where needed. | |
| 830 | +- Verify author order, title spelling, publication year, volume, pages and DOI. | |
| 831 | +- Do not cite unpublished manuscripts as peer-reviewed unless they actually are. | |
| 832 | +- Use primary papers for technical claims and compare the exact theorem statement/assumption set. | |
| 833 | +- Cite priority-threatening work prominently, not buried in generic related work. | |
| 834 | +- Include only works genuinely read and relevant; retain a wider research bibliography separately if helpful. | |
| 835 | + | |
| 836 | +**Gate 10:** complete meaningful article, full appendix, linked tables/figures, a coherent and verified novelty claim, and no unsupported quantitative assertions. | |
| 837 | + | |
| 838 | +--- | |
| 839 | + | |
| 840 | +## 14. PHASE 11 — Adversarial referee and proof audit | |
| 841 | + | |
| 842 | +Simulate at least three independent review perspectives; log objections and responses: | |
| 843 | + | |
| 844 | +**Referee A: theoretical econometrician.** Challenges stochastic expansions, uniformity, bias cancellation, influence-function equivalence, Fisher/sandwich covariance, CLT, conditioning, and necessity of rate restrictions. | |
| 845 | + | |
| 846 | +**Referee B: statistical optimization specialist.** Challenges novelty versus Huang–Huo, Zhang–Duchi–Wainwright, CSL and known higher-order expansions; examines Hessian solves, alternative algorithms, strength of numerical baseline and edge cases. | |
| 847 | + | |
| 848 | +**Referee C: distributed-systems/reproducibility specialist.** Challenges synchronization definitions, true worker independence, network accounting, memory, speedups, end-to-end experiments, stragglers and raw-data reproducibility. | |
| 849 | + | |
| 850 | +For every objection, respond with (a) exact location in paper/code, (b) correction made, (c) new proof/test if needed, and (d) residual limitation. Record unresolved issues candidly; do not have the same model merely declare its own work accepted. | |
| 851 | + | |
| 852 | +### Formal audit checklist | |
| 853 | + | |
| 854 | +- Every theorem's assumptions are enumerated. | |
| 855 | +- No proof relies on an unproved stronger bound in disguise. | |
| 856 | +- Expansions used under local sample size `m/2` or `m/4` are valid for the stated `m` range. | |
| 857 | +- `K=o(N^alpha)` is always interpreted as a triangular-array rate, not a fixed-N machine limit. | |
| 858 | +- Boundary constants have been independently checked. | |
| 859 | +- Every table value has a traceable input/output path. | |
| 860 | +- New method comparisons include ordinary local averaging + two Newton rounds. | |
| 861 | +- A baseline with exact sufficient-statistic aggregation is shown for applicable families. | |
| 862 | +- Communication savings distinguish actual bytes and rounds. | |
| 863 | +- The paper openly states if zero new theorem or no actual cluster benchmarks were achieved. | |
| 864 | +- AI contribution disclosure and human review requirements are explicit. | |
| 865 | +- Paper/source sync passes and all PDF page elements render cleanly. | |
| 866 | + | |
| 867 | +**Gate 11:** produce `FINAL_REVIEW.md` with a final verdict `ready for human review / theoretically incomplete / novelty unresolved / computational evidence insufficient`, plus actionable issues ranked by severity. | |
| 868 | + | |
| 869 | +--- | |
| 870 | + | |
| 871 | +## 15. PHASE 12 — Final build, packaging, and delivery | |
| 872 | + | |
| 873 | +### 15.1 Build and verification | |
| 874 | + | |
| 875 | +Provide an executable workflow, ideally: | |
| 876 | + | |
| 877 | +```bash | |
| 878 | +python -m pip install -r requirements.txt | |
| 879 | +pytest -q | |
| 880 | +python experiments/run_monte_carlo.py --config experiments/configs/reproduce_main.yaml | |
| 881 | +python experiments/run_benchmarks.py --config experiments/configs/main.yaml # only when permitted | |
| 882 | +bash scripts/compile_paper.sh | |
| 883 | +bash scripts/validate_pdf.sh | |
| 884 | +``` | |
| 885 | + | |
| 886 | +Update commands to match reality. Do not add fake scripts or claim benchmark success when the environment cannot execute them. | |
| 887 | + | |
| 888 | +Verify: | |
| 889 | + | |
| 890 | +- PDF actually exists, is nonempty, and its text is searchable. | |
| 891 | +- First page, sample theorem page, main table/figure pages and final appendix render correctly. | |
| 892 | +- No missing reference/citation/equation cross-reference, broken figure, clipped formula or watermark. | |
| 893 | +- Main tables are generated programmatically from data files. | |
| 894 | +- Source compiles on a clean environment or with clearly documented toolchain assumptions. | |
| 895 | +- Project checksums and commit state are captured. | |
| 896 | +- The article is named consistently with WP28, not WP27 or an unrelated previous series. | |
| 897 | + | |
| 898 | +### 15.2 Delivery artifact checklist | |
| 899 | + | |
| 900 | +At completion, create (adapt actual filenames to series conventions): | |
| 901 | + | |
| 902 | +```text | |
| 903 | +WP28_Distributed_MLE_Boucher.pdf | |
| 904 | +WP28_Distributed_MLE_Boucher.tex or paper/main.tex | |
| 905 | +WP28_Distributed_MLE_Source_and_Reproducibility.zip | |
| 906 | +WP28_Research_Summary.md | |
| 907 | +WP28_Theorem_Audit.md | |
| 908 | +WP28_Prior_Art_and_Novelty.md | |
| 909 | +WP28_Experiment_Results.csv | |
| 910 | +WP28_Final_Review.md | |
| 911 | +``` | |
| 912 | + | |
| 913 | +Give the researcher a closing report specifying: | |
| 914 | + | |
| 915 | +1. Full article title, exact authorship/disclosure representation, WP28 identifier and verified page count. | |
| 916 | +2. Strongest proved theorem, assumptions and exact rate/bound—not just intuition. | |
| 917 | +3. What is established as new versus known or not checked. | |
| 918 | +4. Number of simulations actually run, whether they completed, datasets, seed, uncertainty and code path. | |
| 919 | +5. Whether actual distributed compute benchmarks were performed, on which verified machines, and measured results. | |
| 920 | +6. Main caveats and unresolved mathematical/computational issues. | |
| 921 | +7. Exact paths to PDF, LaTeX, source archive, bibliography and output data. | |
| 922 | +8. Reproduction and compilation commands. | |
| 923 | +9. Suggested next proof or experiment that would most improve publishability. | |
| 924 | + | |
| 925 | +**Gate 12:** deliver a buildable manuscript and research artifacts in the actual repository. Do not end with a roadmap in place of results. | |
| 926 | + | |
| 927 | +--- | |
| 928 | + | |
| 929 | +## 16. Scientific acceptance standards: pass/fail rubric | |
| 930 | + | |
| 931 | +The project is **not complete** merely because a PDF compiles. Audit every category: | |
| 932 | + | |
| 933 | +| Category | Required evidence | Failure condition | | |
| 934 | +|---|---|---| | |
| 935 | +| Mathematical correctness | Full proofs, explicit assumptions, verified stochastic orders | Missing random remainder, unjustified asymptotics | | |
| 936 | +| Research contribution | Precise difference from closest literature and substantive new result or defensible negative discovery | Claim of novelty based only on jackknife+Newton combination | | |
| 937 | +| Econometric validity | Appropriate variance, coverage, fixed-`p` scope and misspecification warnings | Fisher efficiency claimed without correct model conditions | | |
| 938 | +| Algorithmic correctness | Unit tests, stable solves, failure handling, derivative checks | Unchecked Hessian inverses or silent fit failures | | |
| 939 | +| Statistical experiments | Raw results, seeds, uncertainty, proper baselines, honest N/K/m | Hypothetical speedup marketed as measured | | |
| 940 | +| Systems validation | Real messages and timing logs for any physical-cluster claims | Synthetic partitions passed off as multi-host deployment | | |
| 941 | +| Literature integrity | Verified DOI/metadata and theorem-level comparisons | Fabricated citations or priority omission | | |
| 942 | +| Manuscript quality | Complete argument, journal-style typography, clean figures | Decorative length, overclaiming, repetitive prose | | |
| 943 | +| Reproducibility | Pinned deps, scripts, results manifest, exact commands | Non-reconstructible reported numbers | | |
| 944 | +| Attribution | Simon-Pierre Boucher properly named; AI role transparently recorded | Invented AI credential/affiliation or hidden assistance | | |
| 945 | + | |
| 946 | +--- | |
| 947 | + | |
| 948 | +## 17. Mandatory mathematical appendix tasks (detailed proof queue) | |
| 949 | + | |
| 950 | +Complete or disprove these in order, and keep a numbered ledger. | |
| 951 | + | |
| 952 | +**Lemma 1 — Common influence preservation.** For each split granularity `s`, prove the average of the `s` disjoint block influence sums equals the original full-block influence sum. Extend to arbitrary normalized extrapolation weights. | |
| 953 | + | |
| 954 | +**Lemma 2 — High-order bias cancellation.** Solve the Vandermonde system and show corrected bias `O(m^{-r})`. State how unknown bias coefficients cancel under equal-distribution assumptions. | |
| 955 | + | |
| 956 | +**Lemma 3 — Local stochastic remainder.** Under uniform `L2` controls, show `E||R^J_{k,r}||^2 <= C_r/m^2`, with `C_r` depending explicitly on the weights and split sizes. | |
| 957 | + | |
| 958 | +**Lemma 4 — Averaged random remainder.** For independent blocks, prove the variance/second-moment inequality giving `O_p(1/(m sqrt K))` after centering. Do not assume nested within-worker fits are independent. | |
| 959 | + | |
| 960 | +**Lemma 5 — Central-MLE comparison.** Cancel the common global influence term and prove `O_p(m^{-r}+(m sqrt K)^{-1}+N^{-1})` relative to centralized MLE. | |
| 961 | + | |
| 962 | +**Lemma 6 — One-step Newton quadratic remainder.** Prove the integral identity and uniform Hessian bound on a path in the attraction region. | |
| 963 | + | |
| 964 | +**Lemma 7 — q-step propagation.** With fixed `q`, establish the repeated-squaring bound, including the randomness of the constant and consistency of intermediate iterates. | |
| 965 | + | |
| 966 | +**Theorem 1 — First-order equivalence.** Under the stated complete assumptions, derive a sufficient `K` rate and the resulting influence-function CLT / covariance. | |
| 967 | + | |
| 968 | +**Theorem 2 — Generic (r,q) frontier.** Prove the advertised sufficient exponent or replace it with the correct bound if supplementary terms dominate. | |
| 969 | + | |
| 970 | +**Theorem 3 — Exact exponential boundary.** Derive all Gamma moments needed and establish the nonzero limit `-4 theta_0 c^4` for half-block jackknife + one global Newton step when `K ~ c N^(7/8)`. | |
| 971 | + | |
| 972 | +**Theorem 4 — Distinct stronger contribution.** Prove an additional relevant result not simply implicit in the first three (e.g., adaptive crossover, byte/error bound, inexact Newton perturbation theorem, heterogeneity regime or constrained optimality result), subject to literature priority verification. | |
| 973 | + | |
| 974 | +**Proposition — Failure/counterexample.** Demonstrate a setting where higher-order correction harms MSE, invalidates parameter feasibility, or violates a theorem assumption. Make it useful for method selection. | |
| 975 | + | |
| 976 | +**Corollary — Inferential validity.** Specify the conditions under which confidence intervals using consistently estimated centralized Fisher/sandwich covariance remain valid for the distributed estimator. | |
| 977 | + | |
| 978 | +**Proof ledger** must contain: | |
| 979 | + | |
| 980 | +```text | |
| 981 | +statement_id | status | assumptions | source_or_new | proof_file | | |
| 982 | +independent_check | counterexample_attempt | paper_location | unresolved_issue | |
| 983 | +``` | |
| 984 | + | |
| 985 | +--- | |
| 986 | + | |
| 987 | +## 18. Mandatory experimental ablations and comparisons | |
| 988 | + | |
| 989 | +Every estimator comparison must control for both **statistics** and **resources**. At minimum: | |
| 990 | + | |
| 991 | +1. **Fixed total N, varying K:** how accuracy deteriorates as local sample sizes shrink. | |
| 992 | +2. **Fixed K, varying N:** whether empirical scaling agrees with asymptotic claims. | |
| 993 | +3. **Fixed communication rounds:** compare improved local corrections at same number of derivative synchronizations. | |
| 994 | +4. **Fixed computational budget:** compare ordinary and higher-order local fitting using elapsed time, not raw count of method names. | |
| 995 | +5. **Fixed target accuracy:** minimize wall-clock time subject to a common RMSE or inferential-coverage threshold. | |
| 996 | +6. **Fixed network bandwidth / latency:** vary local compute speed and evaluate synchronization crossover. | |
| 997 | +7. **Jackknife variance inflation:** compare error variance and failure rates across `r=1,2,3,4` as feasible. | |
| 998 | +8. **Global Newton rounds:** compare `q=0,1,2,3` and show when extra rounds dominate high-order bias cancellation. | |
| 999 | +9. **Hessian communication:** dense exact Hessian versus candidate approximations, with exact byte accounting and accuracy degradation. | |
| 1000 | +10. **Parameter dimension:** evaluate how `p` changes curvature, communication size and fitted sample requirements. | |
| 1001 | +11. **Imbalanced worker sizes:** equal versus unequal `m_k`, weighted versus naive aggregation. | |
| 1002 | +12. **Near-degenerate cases:** weak signal, almost singular Fisher information, separation, contamination, heavy tails and worker fit failures. | |
| 1003 | + | |
| 1004 | +A figure is mandatory only if backed by actual computed data. For any unattained experiment, document a precise blocker and do not fabricate a placeholder plot. | |
| 1005 | + | |
| 1006 | +--- | |
| 1007 | + | |
| 1008 | +## 19. Concrete prospective figure and table program | |
| 1009 | + | |
| 1010 | +### Figures | |
| 1011 | + | |
| 1012 | +- **Figure 1:** coordinator/worker protocol with exact uplink, downlink and synchronization stages. | |
| 1013 | +- **Figure 2:** theoretical admissible partition exponent `alpha(r,q)` and explicit caveat: sufficient rates only, fixed `r,q,p`. | |
| 1014 | +- **Figure 3:** exact Gamma-model boundary shift versus `c`, Monte Carlo intervals and analytic `-4 theta_0 c^4` curve. | |
| 1015 | +- **Figure 4:** RMSE and bias of raw, jack2, jack3, multi-round Newton vs `m`, emphasizing finite-sample variance amplification. | |
| 1016 | +- **Figure 5:** observed empirical probability of valid local fits and conditioning diagnostics. | |
| 1017 | +- **Figure 6:** measured real/simulated-with-clear-label communication-vs-computation phase diagram. | |
| 1018 | +- **Figure 7:** latency/bandwidth crossover map comparing jack2+1 with ordinary+2 under equal target accuracy. | |
| 1019 | +- **Figure 8:** actual multi-node speedup and parallel efficiency *only if measured*. | |
| 1020 | +- **Figure A1:** nontrivial likelihood counterexample illustrating limits of a general claim. | |
| 1021 | + | |
| 1022 | +### Tables | |
| 1023 | + | |
| 1024 | +- **Table 1:** literature/priority comparison with the closest relevant work. | |
| 1025 | +- **Table 2:** estimator definitions, local fit counts, global corrections, network phases, bytes and proved statistical regime. | |
| 1026 | +- **Table 3:** theorem assumptions and applicability by model class. | |
| 1027 | +- **Table 4:** Monte Carlo bias, SD, RMSE, coverage, solver failures and Monte Carlo standard errors. | |
| 1028 | +- **Table 5:** computational time and messages by worker count, machine class and network profile. | |
| 1029 | +- **Table 6:** ablation of order `r`, steps `q`, Hessian approximation and stability. | |
| 1030 | + | |
| 1031 | +Do not force a figure or table to exist if its substantive data or mathematical content is unavailable. Prioritize interpretability and exact replication. | |
| 1032 | + | |
| 1033 | +--- | |
| 1034 | + | |
| 1035 | +## 20. Autonomy and phase continuation protocol | |
| 1036 | + | |
| 1037 | +After completing each phase, append to `PHASE_STATUS.md`: | |
| 1038 | + | |
| 1039 | +```text | |
| 1040 | +Phase: | |
| 1041 | +Start/finish timestamp: | |
| 1042 | +What was actually done: | |
| 1043 | +Artifacts created: | |
| 1044 | +Tests passed: | |
| 1045 | +Tests failed: | |
| 1046 | +Unproved statements or unresolved citations: | |
| 1047 | +Decision and justification: | |
| 1048 | +Next phase action: | |
| 1049 | +``` | |
| 1050 | + | |
| 1051 | +If the phase is unsuccessful, do not declare victory. Diagnose whether the blocker is (i) mathematical falsehood, (ii) missing assumptions, (iii) missing data/library access, (iv) exhausted compute budget, (v) conflicts in the repository or (vi) publication-priority overlap. Handle the issue using a scientifically defensible fallback, record the limits and continue. Do not allow a failed ambitious theorem to prevent producing the strongest valid WP28 manuscript attainable. | |
| 1052 | + | |
| 1053 | +### Final instruction to Claude Code | |
| 1054 | + | |
| 1055 | +**Start immediately.** Inspect the UQO research repository and the supplied PAMLE prior draft, preserve the WP28 identity, reproduce the original math and Monte Carlo, perform a serious priority audit, derive and verify the strongest provable extension, implement fair competing estimators, execute actual experiments, and write a polished English academic paper with full proofs and transparent caveats. Compile and visually validate the final PDF. Keep scientific honesty above ambition, and give Simon-Pierre Boucher the complete, reproducible WP28 package—not a promise, outline, or fictional result. | |
added
PHASE_STATUS.md
+17 −0
@@ -0,0 +1,17 @@ | ||
| 1 | +# PHASE_STATUS — WP28 | |
| 2 | + | |
| 3 | +Phase: 0 — Repository reconnaissance and reproducibility lock | |
| 4 | +Start/finish timestamp: 2026-10-28 18:33 → 18:50 (America/Toronto) | |
| 5 | +What was actually done: inspected `Recherche-UQO` (README, HARMONISATION, template, WP 25/27 conventions); located the prior draft in | |
| 6 | +`~/Downloads/WP28_UQO_Claude_Research_Brief/`; froze it with SHA256 under `references/initial_pamle/`; read the 16-page PDF and the | |
| 7 | +source archive in full; re-ran `monte_carlo.py` at seed 20261028 (identical outputs to ≤ 5e-13); compiled `pamle.tex` (16 pp.); | |
| 8 | +inventoried the live MacLustr cluster (22/29 nodes reachable; see `docs/cluster_inventory_2026-10-28.md`); created the WP28 workspace | |
| 9 | +from the UQO template. | |
| 10 | +Artifacts created: PROJECT_MAP.md, SOURCE_PROVENANCE.md, PHASE_STATUS.md, research_log.md, docs/cluster_inventory_2026-10-28.md, | |
| 11 | +references/initial_pamle/*, paper/{Makefile,.latexmkrc,uq_logo.jpg,sections/titlepage*.tex}, .gitignore. | |
| 12 | +Tests passed: replication of the prior Monte Carlo (summary + 10,500 replicate rows). | |
| 13 | +Tests failed: none. | |
| 14 | +Unproved statements or unresolved citations: all theorems of the prior draft are tagged "to re-derive" until Phase 2; the 5 prior | |
| 15 | +references are being verified in Phase 1. | |
| 16 | +Decision and justification: keep identifier WP28 (free in the series; collision note recorded). Proceed to Phase 1. | |
| 17 | +Next phase action: literature/priority audit (four parallel verification tasks launched 18:47). | |
added
PROJECT_MAP.md
+31 −0
@@ -0,0 +1,31 @@ | ||
| 1 | +# PROJECT_MAP — UQO Working Paper No. 28 (distributed MLE) | |
| 2 | + | |
| 3 | +Repository root: `~/Desktop/Recherche-UQO` (index `README.md`, form standard `HARMONISATION.md`). | |
| 4 | +This paper: `finance/distributed-mle-synchronization/` (series convention: `<domain>/<slug>/`, metadata macro `\WPnumber{28}`, | |
| 5 | +spbgit remote `gitsrv:srv/git/wp28_uqo.git` to be created at delivery). | |
| 6 | + | |
| 7 | +Conventions observed (2026-10-28): | |
| 8 | +- Template `_modele-working-paper/` (12pt letterpaper article, mathptmx, natbib authoryear + `aer` style, `titlepage.tex` copied verbatim, | |
| 9 | + macros `\WPnumber … \WPaddress`, `\WPdate` = "Month YYYY", `\WPversion` = number only). Theoretical papers (WP 19, 24, 25) add | |
| 10 | + amsthm environments and `mathtools`; appendices live in `paper/appendices/`. | |
| 11 | +- Highest existing number: 27 (`finance/invisible-hand-of-machines`). No `\WPnumber{28}` exists anywhere in the series | |
| 12 | + (see `SOURCE_PROVENANCE.md` for the informal "WP 28" label on the FIRC No. 0001 database paper). | |
| 13 | +- Per-paper layout: `README.md`, `.gitignore`, `requirements.txt`, `paper/`, `src/`, `scripts/` or `experiments/`, `data/`, | |
| 14 | + `results/`, `figures/`, `docs/`; `paper/main.pdf` committed; raw data not committed. | |
| 15 | + | |
| 16 | +This paper's tree (mandate §9.1, adapted to the series): | |
| 17 | +``` | |
| 18 | +CLAUDE.md research mandate (verbatim copy of the brief) | |
| 19 | +PROJECT_MAP.md SOURCE_PROVENANCE.md PHASE_STATUS.md research_log.md | |
| 20 | +LIMITATIONS_AND_CLAIMS.md FINAL_REVIEW.md FINAL_DELIVERY.md (written at the end of the corresponding phases) | |
| 21 | +references/initial_pamle/ immutable copy of the prior draft (PDF, zip, unpacked sources, SHA256SUMS.txt) | |
| 22 | +literature/ PRIOR_ART_MATRIX.csv, PRIORITY_AND_GAP.md | |
| 23 | +theory/ THEOREM_AUDIT.md, ASSUMPTIONS.md, symbolic_checks.py, proofs_check.py | |
| 24 | +src/pamle/ reference implementations (likelihoods, splits, jackknife weights, Newton, protocol, accounting) | |
| 25 | +experiments/ run_* scripts, configs/, EXPERIMENT_MANIFEST.csv | |
| 26 | +tests/ pytest suite | |
| 27 | +results/{raw,processed,tables,figures,resource_profiles} | |
| 28 | +paper/{main.tex,sections,appendices,figures,tables,references.bib} | |
| 29 | +scripts/{setup.sh,reproduce_all.sh,compile_paper.sh,validate_pdf.sh} | |
| 30 | +docs/ cluster inventory, referee reports, misc audits | |
| 31 | +``` | |
added
SOURCE_PROVENANCE.md
+33 −0
@@ -0,0 +1,33 @@ | ||
| 1 | +# SOURCE_PROVENANCE — prior draft and identifier | |
| 2 | + | |
| 3 | +## Prior exploratory draft (handoff, dated 2026-10-28) | |
| 4 | +Supplied in `~/Downloads/WP28_UQO_Claude_Research_Brief/` (mandate `CLAUDE.md`, identical to `~/Downloads/CLAUDE-6.md`): | |
| 5 | +- `references/PAMLE_Boucher_GPT-6.1.pdf` — 16 pages, "Local Bias Correction versus Global Synchronization in Distributed Maximum | |
| 6 | + Likelihood", authors listed as Simon-Pierre Boucher (UQO) and "GPT-6.1" (AI co-author designation). SHA256 | |
| 7 | + `e81198b7…18cd`. | |
| 8 | +- `references/PAMLE_Boucher_GPT-6.1_LaTeX_and_Reproducibility.zip` — `pamle.tex`, `references.bib` (5 entries), `monte_carlo.py`, | |
| 9 | + `simulation_summary.csv` (21 rows), `simulation_replicates.csv` (10,500 rows = 3 designs × 500 replicates × 7 estimators), | |
| 10 | + `figures/mc_comparison.{pdf,png}`, `README.md`. SHA256 `613c6f8d…f850`. | |
| 11 | +Frozen read-only copy with per-file checksums: `references/initial_pamle/` (`SHA256SUMS.txt`). Never edited. | |
| 12 | + | |
| 13 | +## Replication status of every number in the prior draft (Gate 0) | |
| 14 | +| Item | Status | | |
| 15 | +|---|---| | |
| 16 | +| Monte Carlo (seed 20261028, N = 2^20, m ∈ {256, 64, 16}, B = 500, θ0 = 1) | **replicated** on the laptop (Python 3.14.4 / NumPy 2.4.4, 3.4 s); summary and all 10,500 replicate rows agree to ≤ 5e-13 absolute | | |
| 17 | +| Table 2 of the draft (mean scaled gaps) | **replicated** (identical to six decimals) | | |
| 18 | +| "≈ 0" entry for raw + 2 Newton at m = 256 | actual value −2.42e-07 (CSV); the draft rounds it to ≈ 0 — recorded, not an error | | |
| 19 | +| Figure 2 | regenerated by the same script (vector + png) | | |
| 20 | +| Exact Gamma moment identities (23)–(27), (28), (30), (41) | re-derived symbolically in Phase 2 (`theory/symbolic_checks.py`) | | |
| 21 | +| Boundary constant −4 θ0 c^4 | re-derived in Phase 2/5 and checked numerically | | |
| 22 | +| Compilation of `pamle.tex` | compiles with pdfLaTeX (16 pp.); 1 overfull box; duplicate hyperref destinations for tables/figures (cleveref + manual labels) | | |
| 23 | +| Replicate count | the draft says 500 — correct; the mandate warns not to call it 1,000 | | |
| 24 | +| Physical cluster benchmark | **none in the prior draft** (the README says so explicitly) | | |
| 25 | + | |
| 26 | +## Identifier: WP28 | |
| 27 | +- The series index (`Recherche-UQO/README.md`, `HARMONISATION.md`) stops at No. 27; no `paper/main.tex` in the series declares | |
| 28 | + `\WPnumber{28}`. | |
| 29 | +- **Collision note.** The database paper `finance/russell3000-strategic-project-database` was informally called "WP 28 (next after 27)" | |
| 30 | + in the assistant's session memory on 2026-10-28, but its manuscript and README carry the FIRC series number **0001**, not 28. | |
| 31 | + Nothing was renumbered or overwritten. If the principal researcher prefers to reserve No. 28 for the database paper, this | |
| 32 | + manuscript can be renumbered by changing `\WPnumber`, the README, the BibTeX `number` field and the spbgit remote name | |
| 33 | + (`wp28_uqo.git` → `wp29_uqo.git`); all internal file names use the slug, not the number. | |
added
data/processed/.gitkeep
+0 −0
added
docs/cluster_inventory_2026-10-28.md
+35 −0
@@ -0,0 +1,35 @@ | ||
| 1 | +# MacLustr live inventory — 2026-10-28 18:45 (ssh from laptop, default login python3) | |
| 2 | + | |
| 3 | +| alias | hostname | cores | RAM | chip | python3 | numpy/scipy | LAN IP | load 1/5/15 | | |
| 4 | +|---|---|---|---|---|---|---|---|---| | |
| 5 | +| M3U96a | MacStudnpierre2 | 28 | 96GB | Apple M3 Ultra | Python 3.14.5 | 2.4.4 1.17.1 | 192.168.2.80 | 1,53 1,68 1,76 | | |
| 6 | +| M3U96b | m3u96b | 28 | 96GB | Apple M3 Ultra | Python 3.14.4 | 2.4.4 1.17.1 | 192.168.2.85 | 2,02 1,93 1,91 | | |
| 7 | +| M2U64 | MacStudonpierre | 24 | 64GB | Apple M2 Ultra | Python 3.9.6 | 2.0.2 1.13.1 | 192.168.2.61 | 0,94 1,06 1,07 | | |
| 8 | +| M4M64a | MacStudnpierre6 | 16 | 64GB | Apple M4 Max | Python 3.14.4 | 2.4.4 1.17.1 | 192.168.2.70 | 1,49 1,56 1,60 | | |
| 9 | +| M4M64b | MacStudnpierre4 | 16 | 64GB | Apple M4 Max | Python 3.14.4 | ModuleNotFoundError: No module named 'scipy' | 192.168.2.77 | 1,23 1,19 1,18 | | |
| 10 | +| M4M64c | M4M64c | 16 | 64GB | Apple M4 Max | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.103 | 1,92 1,73 1,75 | | |
| 11 | +| M5M36 | M5M36 | 18 | 36GB | Apple M5 Max | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.104 | 1,02 1,11 1,09 | | |
| 12 | +| M4BP48 | simonpirresMBP2 | 16 | 48GB | Apple M4 Max | Python 3.9.6 | ModuleNotFoundError: No module named 'scipy' | | 0,94 0,89 0,77 | | |
| 13 | +| M4BP36 | simonpierresMBP | 14 | 36GB | Apple M4 Max | Python 3.9.6 | ModuleNotFoundError: No module named 'scipy' | | 1,17 1,18 1,19 | | |
| 14 | +| M4M36 | UNREACHABLE | | |
| 15 | +| M2M32 | UNREACHABLE | | |
| 16 | +| M2M32b | UNREACHABLE | | |
| 17 | +| M2M32c | MacStudnpierre7 | 12 | 32GB | Apple M2 Max | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.76 | 1,05 1,05 1,06 | | |
| 18 | +| m4mc | UNREACHABLE | | |
| 19 | +| M1M32 | MacStudnpierre8 | 10 | 32GB | Apple M1 Max | Python 3.9.6 | 2.0.2 1.13.1 | 192.168.2.72 | 1,41 1,28 1,20 | | |
| 20 | +| m4ma | UNREACHABLE | | |
| 21 | +| m4mb | UNREACHABLE | | |
| 22 | +| m4md | Macmininpierre5 | 10 | 16GB | Apple M4 | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.62 | 1,77 1,57 1,45 | | |
| 23 | +| m4ml | m4ml | 10 | 16GB | Apple M4 | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.63 | 1,33 1,56 1,52 | | |
| 24 | +| m4mm | m4mm | 12 | 48GB | Apple M4 Pro | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.92 | 1,73 1,56 1,42 | | |
| 25 | +| m4mn | m4mn | 12 | 48GB | Apple M4 Pro | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.93 | 1,37 1,41 1,39 | | |
| 26 | +| m6ma | m6ma | 12 | 24GB | Apple M6 | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.105 | 1,45 1,27 1,18 | | |
| 27 | +| m6mb | m6mb | 12 | 24GB | Apple M6 | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.106 | 1,15 1,13 1,10 | | |
| 28 | +| m5ma | m5ma | 15 | 24GB | Apple M5 Pro | Python 3.9.6 | ModuleNotFoundError: No module named 'numpy' | 192.168.2.112 | 0,88 1,07 1,06 | | |
| 29 | +| m2m16 | UNREACHABLE | | |
| 30 | +| M3BA24 | MacBookonpierre | 8 | 24GB | Apple M3 | Python 3.9.6 | ModuleNotFoundError: No module named 'scipy' | | 1,15 1,23 1,23 | | |
| 31 | +| M3BA16 | MacBooknpierre2 | 8 | 16GB | Apple M3 | Python 3.9.6 | ModuleNotFoundError: No module named 'scipy' | | 1,35 1,22 1,25 | | |
| 32 | +| m2m8a | Minidesonpierre | 8 | 8GB | Apple M2 | Python 3.9.6 | ModuleNotFoundError: No module named 'scipy' | 192.168.2.66 | 1,74 1,68 1,61 | | |
| 33 | +| m2m8b | Minidesnpierre2 | 8 | 8GB | Apple M2 | xcode-select: note: No developer tools were found, requesting install. | | |
| 34 | + | |
| 35 | +Unreachable at inventory time: M4M36, M2M32, M2M32b, m4mc, m4ma, m4mb, m2m16 (7 of 29). Laptop: Apple M5 Max, 18 cores, 48 GB, Python 3.14.4, NumPy 2.4.4, SciPy 1.17.1. | |
added
paper/.latexmkrc
+5 −0
@@ -0,0 +1,5 @@ | ||
| 1 | +$pdf_mode = 1; | |
| 2 | +$pdflatex = 'pdflatex -interaction=nonstopmode -halt-on-error -synctex=1 %O %S'; | |
| 3 | +$bibtex_use = 2; | |
| 4 | +$clean_ext = 'synctex.gz run.xml bbl bcf fdb_latexmk fls log aux out toc lof lot blg'; | |
| 5 | +@default_files = ('main.tex'); | |
added
paper/Makefile
+11 −0
@@ -0,0 +1,11 @@ | ||
| 1 | +.PHONY: all clean distclean | |
| 2 | + | |
| 3 | +all: | |
| 4 | + latexmk | |
| 5 | + | |
| 6 | +clean: | |
| 7 | + latexmk -c | |
| 8 | + | |
| 9 | +distclean: | |
| 10 | + latexmk -C | |
| 11 | + rm -f *.synctex.gz *.run.xml *.bbl *.bcf | |
added
paper/sections/titlepage.tex
+77 −0
@@ -0,0 +1,77 @@ | ||
| 1 | +% ============================================================================ | |
| 2 | +% Title Page | |
| 3 | +% ============================================================================ | |
| 4 | +\thispagestyle{empty} | |
| 5 | + | |
| 6 | +\begin{center} | |
| 7 | + | |
| 8 | +% --- Logo --- | |
| 9 | +\includegraphics[width=3.5cm]{uq_logo.jpg} | |
| 10 | + | |
| 11 | +\vspace{0.6cm} | |
| 12 | + | |
| 13 | +{\footnotesize\textsc{Université du Québec en Outaouais}}\\[0.15cm] | |
| 14 | +{\footnotesize\textsc{Département des sciences administratives}} | |
| 15 | + | |
| 16 | +\vspace{0.8cm} | |
| 17 | + | |
| 18 | +{\footnotesize\textsc{Working Paper No.~\WPnumber}} | |
| 19 | + | |
| 20 | +\vspace{1.2cm} | |
| 21 | + | |
| 22 | +% --- Title --- | |
| 23 | +{\LARGE\bfseries \WPtitle\par} | |
| 24 | + | |
| 25 | +\ifx\WPsubtitle\empty\else | |
| 26 | + \vspace{0.3cm} | |
| 27 | + {\large\itshape \WPsubtitle\par} | |
| 28 | +\fi | |
| 29 | + | |
| 30 | +\vspace{1.2cm} | |
| 31 | + | |
| 32 | +% --- Author --- | |
| 33 | +{\large \WPauthor}\\[0.3cm] | |
| 34 | +{\normalsize \WPaffiliation}\\[0.15cm] | |
| 35 | +{\normalsize \href{mailto:\WPemail}{\WPemail}}\\[0.15cm] | |
| 36 | +{\small \WPaddress} | |
| 37 | + | |
| 38 | +\vspace{0.8cm} | |
| 39 | + | |
| 40 | +% --- Date & Version --- | |
| 41 | +{\normalsize \WPdate}\\[0.1cm] | |
| 42 | +{\small Version~\WPversion} | |
| 43 | + | |
| 44 | +\end{center} | |
| 45 | + | |
| 46 | +\vfill | |
| 47 | + | |
| 48 | +\newpage | |
| 49 | + | |
| 50 | +% ============================================================================ | |
| 51 | +% Abstract Page | |
| 52 | +% ============================================================================ | |
| 53 | +\thispagestyle{empty} | |
| 54 | + | |
| 55 | +\vspace*{1cm} | |
| 56 | + | |
| 57 | +\noindent\rule{\textwidth}{0.4pt} | |
| 58 | +\vspace{0.3cm} | |
| 59 | + | |
| 60 | +\noindent\textbf{Abstract} | |
| 61 | + | |
| 62 | +\vspace{0.15cm} | |
| 63 | + | |
| 64 | +\noindent\WPabstract | |
| 65 | + | |
| 66 | +\vspace{0.4cm} | |
| 67 | + | |
| 68 | +\noindent\textbf{Keywords:} \WPkeywords | |
| 69 | + | |
| 70 | +\vspace{0.15cm} | |
| 71 | + | |
| 72 | +\noindent\textbf{JEL Classification:} \WPjel | |
| 73 | + | |
| 74 | +\vspace{0.3cm} | |
| 75 | +\noindent\rule{\textwidth}{0.4pt} | |
| 76 | + | |
| 77 | +\newpage | |
added
paper/sections/titlepage_anon.tex
+53 −0
@@ -0,0 +1,53 @@ | ||
| 1 | +% ============================================================================ | |
| 2 | +% ANONYMOUS TITLE PAGE — for double-anonymized journal submission | |
| 3 | +% Used by paper/main_anon.tex (same macros as main.tex, no author block, | |
| 4 | +% no institution, no logo). Abstract/keywords/JEL page identical to titlepage.tex. | |
| 5 | +% ============================================================================ | |
| 6 | +\thispagestyle{empty} | |
| 7 | +\begin{center} | |
| 8 | + | |
| 9 | +\vspace*{3cm} | |
| 10 | + | |
| 11 | +{\LARGE\bfseries \WPtitle\par} | |
| 12 | + | |
| 13 | +\ifx\WPsubtitle\empty\else | |
| 14 | + \vspace{0.3cm} | |
| 15 | + {\large\itshape \WPsubtitle\par} | |
| 16 | +\fi | |
| 17 | + | |
| 18 | +\vspace{1.5cm} | |
| 19 | + | |
| 20 | +{\normalsize [Author identity withheld for review]} | |
| 21 | + | |
| 22 | +\vspace{0.8cm} | |
| 23 | + | |
| 24 | +{\normalsize \WPdate} | |
| 25 | + | |
| 26 | +\end{center} | |
| 27 | + | |
| 28 | +\vfill | |
| 29 | + | |
| 30 | +\newpage | |
| 31 | +\thispagestyle{empty} | |
| 32 | +\vspace*{1cm} | |
| 33 | + | |
| 34 | +\noindent\rule{\textwidth}{0.4pt} | |
| 35 | +\vspace{0.3cm} | |
| 36 | + | |
| 37 | +\noindent\textbf{Abstract} | |
| 38 | +\vspace{0.15cm} | |
| 39 | + | |
| 40 | +\noindent\WPabstract | |
| 41 | + | |
| 42 | +\vspace{0.4cm} | |
| 43 | + | |
| 44 | +\noindent\textbf{Keywords:} \WPkeywords | |
| 45 | + | |
| 46 | +\vspace{0.15cm} | |
| 47 | + | |
| 48 | +\noindent\textbf{JEL Classification:} \WPjel | |
| 49 | + | |
| 50 | +\vspace{0.3cm} | |
| 51 | +\noindent\rule{\textwidth}{0.4pt} | |
| 52 | + | |
| 53 | +\newpage | |
added
paper/uq_logo.jpg
+0 −0
Binary file not shown.
added
references/initial_pamle/PAMLE_Boucher_GPT-6.1.pdf
+0 −0
Binary file not shown.
added
references/initial_pamle/PAMLE_Boucher_GPT-6.1_LaTeX_and_Reproducibility.zip
+0 −0
Binary file not shown.
added
references/initial_pamle/SHA256SUMS.txt
+10 −0
@@ -0,0 +1,10 @@ | ||
| 1 | +e81198b7c81363c9e0645742e9a44b544efa5a093de28f0e89c89acd58fa18cd PAMLE_Boucher_GPT-6.1.pdf | |
| 2 | +613c6f8d3b54cf55b661e1051ae8d35ab2e9ed2c42fde49e5fdcee8cea3bf850 PAMLE_Boucher_GPT-6.1_LaTeX_and_Reproducibility.zip | |
| 3 | +f1b8a0f2437cf5ed98a8eab8c0a903a8d17a73a75e1124de0c4b790e3ec75b9f ./figures/mc_comparison.pdf | |
| 4 | +321fbae65a07db11b3771fef77eb09e8064289c8a3c90b49678f9db60d4a863b ./figures/mc_comparison.png | |
| 5 | +3fd8b7513654c3450f6cca470e19c442fb19229ab9119975eff8102114b60e09 ./monte_carlo.py | |
| 6 | +3e89f5d4eea05534b2fa0e62f0fd5074c6f9825cf6729daded4f9f0152a21da3 ./pamle.tex | |
| 7 | +be107589e0eba0e5bae3469a023e41751e926ca3a2b358d6c8eb13049ac7c5d3 ./README.md | |
| 8 | +cb9baeb771e1431345323ff2e3d0a006789ddb37fc3b1f45886706c3603d4b83 ./references.bib | |
| 9 | +85b8e4a31f8b1d7d59c92b5832dfca733cdca4f5d4d8b01cde49b18bc0b2e67a ./simulation_replicates.csv | |
| 10 | +c3c9ff9c90be5156b90bf61baca2c67d4ce33cbfc73f626d63a4a89c7a911500 ./simulation_summary.csv | |
added
references/initial_pamle/unpacked/README.md
+35 −0
@@ -0,0 +1,35 @@ | ||
| 1 | +# Local Bias Correction versus Global Synchronization in Distributed Maximum Likelihood | |
| 2 | + | |
| 3 | +Authors listed in the working paper: **Simon-Pierre Boucher and GPT-6.1** (AI designation requested by the user). | |
| 4 | + | |
| 5 | +This is a *working research note*, not a peer-reviewed or priority-validated paper. The theorem assumptions, limits of the exponential illustration, and computational scope are explained inside the PDF. | |
| 6 | + | |
| 7 | +## Files | |
| 8 | + | |
| 9 | +- `pamle.tex` - LaTeX source (16-page article) | |
| 10 | +- `references.bib` - machine-readable bibliography (the article also contains manual bibliography entries, so BibTeX is **not** required for compilation) | |
| 11 | +- `figures/mc_comparison.pdf` - vector chart included in the article | |
| 12 | +- `figures/mc_comparison.png` - preview version of the chart | |
| 13 | +- `monte_carlo.py` - exact-Gamma reproducible simulation | |
| 14 | +- `simulation_summary.csv` - reported statistics for 500 replications per partition count | |
| 15 | +- `simulation_replicates.csv` - 10,500 method/replication rows, across 3 designs x 500 replications x 7 estimators | |
| 16 | + | |
| 17 | +## Build the LaTeX article | |
| 18 | + | |
| 19 | +```bash | |
| 20 | +pdflatex -interaction=nonstopmode pamle.tex | |
| 21 | +pdflatex -interaction=nonstopmode pamle.tex | |
| 22 | +``` | |
| 23 | + | |
| 24 | +Requirements: pdfLaTeX and standard LaTeX packages (amsmath, amsthm, microtype, hyperref, cleveref, natbib, tikz, listings, etc.). The project uses relative paths, so compile from this directory. | |
| 25 | + | |
| 26 | +## Reproduce Monte Carlo | |
| 27 | + | |
| 28 | +```bash | |
| 29 | +pip install numpy matplotlib | |
| 30 | +python monte_carlo.py | |
| 31 | +``` | |
| 32 | + | |
| 33 | +Seed: 20261028. Total observations in each replication: 1,048,576. Partition sizes: m=256, m=64, m=16. Repetitions: 500 per design. Exponential rate=1. The script draws exact block-wise Gamma sufficient statistics rather than materializing every individual observation. | |
| 34 | + | |
| 35 | +**No physical cluster was benchmarked** and no numerical speedup is claimed. All simulations are synthetic and measure scaled estimator gaps relative to the centralized MLE. | |
added
references/initial_pamle/unpacked/figures/mc_comparison.pdf
+0 −0
Binary file not shown.
added
references/initial_pamle/unpacked/figures/mc_comparison.png
+0 −0
Binary file not shown.
added
references/initial_pamle/unpacked/monte_carlo.py
+82 −0
@@ -0,0 +1,82 @@ | ||
| 1 | +#!/usr/bin/env python3 | |
| 2 | +"""Reproducible exact-sufficient-statistics Monte Carlo for rate-parameter exponential MLE. | |
| 3 | +Each worker is partitioned into four quarters; sums are sampled from their exact Gamma law. | |
| 4 | +No data movement/communication/network benchmark is claimed. | |
| 5 | +""" | |
| 6 | +import csv, numpy as np | |
| 7 | +from pathlib import Path | |
| 8 | +import matplotlib; matplotlib.use('Agg') | |
| 9 | +import matplotlib.pyplot as plt | |
| 10 | +ROOT=Path(__file__).parent | |
| 11 | +SEED=20261028 | |
| 12 | +N=2**20 | |
| 13 | +THETA=1.0 | |
| 14 | +REPS=500 | |
| 15 | +M_LIST=(256,64,16) | |
| 16 | +rng=np.random.default_rng(SEED) | |
| 17 | + | |
| 18 | +def newton(x, mle): | |
| 19 | + return 2.0*x-x*x/mle | |
| 20 | + | |
| 21 | +rows=[]; summary=[] | |
| 22 | +for m in M_LIST: | |
| 23 | + K=N//m | |
| 24 | + obs={key: [] for key in ('raw','one_step_raw','two_step_raw','jack2','one_step_jack2','jack3','one_step_jack3')} | |
| 25 | + for rep in range(REPS): | |
| 26 | + # Gamma is a *sum* of m/4 exponential(rate=theta) observations. | |
| 27 | + quarters=rng.gamma(shape=m//4, scale=1/THETA,size=(K,4)) | |
| 28 | + halves=quarters.reshape(K,2,2).sum(axis=2) | |
| 29 | + totals=halves.sum(axis=1) | |
| 30 | + full_mle=N/np.sum(totals) | |
| 31 | + local_full=m/totals | |
| 32 | + local_halves=(m/2)/halves | |
| 33 | + local_quarts=(m/4)/quarters | |
| 34 | + raw=float(local_full.mean()) | |
| 35 | + J2=float((2*local_full - local_halves.mean(axis=1)).mean()) | |
| 36 | + J3=float(((8/3)*local_full -2*local_halves.mean(axis=1) +(1/3)*local_quarts.mean(axis=1)).mean()) | |
| 37 | + values={'raw':raw, | |
| 38 | + 'one_step_raw':newton(raw,full_mle), | |
| 39 | + 'two_step_raw':newton(newton(raw,full_mle),full_mle), | |
| 40 | + 'jack2':J2,'one_step_jack2':newton(J2,full_mle), | |
| 41 | + 'jack3':J3,'one_step_jack3':newton(J3,full_mle)} | |
| 42 | + for key,val in values.items(): | |
| 43 | + delta=np.sqrt(N)*(val-full_mle) | |
| 44 | + obs[key].append(delta) | |
| 45 | + rows.append((m,K,rep,key,delta)) | |
| 46 | + for name, data in obs.items(): | |
| 47 | + a=np.array(data) | |
| 48 | + summary.append(dict(m=m,K=K,method=name,mean_scaled_error=float(a.mean()), | |
| 49 | + mc_standard_error=float(a.std(ddof=1)/np.sqrt(REPS)), | |
| 50 | + rmse_scaled_error=float(np.sqrt(np.mean(a*a))), | |
| 51 | + p95_abs_scaled_error=float(np.quantile(np.abs(a),.95)))) | |
| 52 | + print('m',m,'K',K,[(k,round(np.mean(v),6)) for k,v in obs.items()],flush=True) | |
| 53 | +with open(ROOT/'simulation_replicates.csv','w',newline='') as f: | |
| 54 | + w=csv.writer(f); w.writerow(['m','K','rep','method','scaled_error_relative_to_centralized_mle']); w.writerows(rows) | |
| 55 | +with open(ROOT/'simulation_summary.csv','w',newline='') as f: | |
| 56 | + w=csv.DictWriter(f,fieldnames=list(summary[0]));w.writeheader();w.writerows(summary) | |
| 57 | + | |
| 58 | +palette={'one_step_raw':'#A55040','two_step_raw':'#9B7C5A','one_step_jack2':'#176F86','one_step_jack3':'#407B58'} | |
| 59 | +labels={'one_step_raw':'Raw averaging + 1 Newton','two_step_raw':'Raw averaging + 2 Newton','one_step_jack2':'Half-block jackknife + 1 Newton','one_step_jack3':'Quarter-block jackknife + 1 Newton'} | |
| 60 | +fig,axs=plt.subplots(1,2,figsize=(11.5,4.0),layout='constrained') | |
| 61 | +ks=np.array([N//m for m in M_LIST]) | |
| 62 | +for method in palette: | |
| 63 | + vals=[next(r for r in summary if r['K']==int(k) and r['method']==method) for k in ks] | |
| 64 | + axs[0].plot(ks,[abs(x['mean_scaled_error']) for x in vals],'-o',lw=2,ms=5, | |
| 65 | + color=palette[method],label=labels[method]) | |
| 66 | + axs[1].plot(ks,[x['rmse_scaled_error'] for x in vals],'-o',lw=2,ms=5,color=palette[method]) | |
| 67 | +for ax in axs: | |
| 68 | + ax.set_xscale('log',base=2) | |
| 69 | + ax.set_yscale('log') | |
| 70 | + ax.set_xticks(ks) | |
| 71 | + ax.set_xticklabels([f'{k:,}' for k in ks]) | |
| 72 | + ax.grid(alpha=.2,which='both') | |
| 73 | + ax.set_xlabel('Number of partitions K') | |
| 74 | +axs[0].set_ylabel('Absolute mean scaled gap') | |
| 75 | +axs[1].set_ylabel('RMSE of scaled gap') | |
| 76 | +axs[0].set_title('Bias relative to centralized MLE') | |
| 77 | +axs[1].set_title('Error relative to centralized MLE') | |
| 78 | +fig.legend(loc='outside lower center',bbox_to_anchor=(.5,-.09),ncol=2,frameon=False,fontsize=9) | |
| 79 | +fig.savefig(ROOT/'figures'/'mc_comparison.pdf',bbox_inches='tight') | |
| 80 | +fig.savefig(ROOT/'figures'/'mc_comparison.png',dpi=170,bbox_inches='tight') | |
| 81 | +plt.close(fig) | |
| 82 | +print('wrote outputs') | |
added
references/initial_pamle/unpacked/pamle.tex
+603 −0
@@ -0,0 +1,603 @@ | ||
| 1 | +\documentclass[11pt,letterpaper]{article} | |
| 2 | +\usepackage[T1]{fontenc} | |
| 3 | +\usepackage[utf8]{inputenc} | |
| 4 | +\usepackage{lmodern} | |
| 5 | +\usepackage{microtype} | |
| 6 | +\usepackage[margin=1.08in,headheight=23pt]{geometry} | |
| 7 | +\usepackage{amsmath,amssymb,amsthm,mathtools,bm} | |
| 8 | +\usepackage{booktabs,tabularx,array,multirow} | |
| 9 | +\usepackage{graphicx} | |
| 10 | +\usepackage{xcolor} | |
| 11 | +\usepackage{fancyhdr} | |
| 12 | +\usepackage{titlesec} | |
| 13 | +\usepackage{enumitem} | |
| 14 | +\usepackage{natbib} | |
| 15 | +\usepackage{hyperref} | |
| 16 | +\usepackage[nameinlink,noabbrev,capitalise]{cleveref} | |
| 17 | +\usepackage{listings} | |
| 18 | +\usepackage{float} | |
| 19 | +\usepackage{tikz} | |
| 20 | +\usetikzlibrary{arrows.meta,positioning,calc} | |
| 21 | +\definecolor{navy}{HTML}{17304A} | |
| 22 | +\definecolor{teal}{HTML}{216C78} | |
| 23 | +\definecolor{charcoal}{HTML}{242E35} | |
| 24 | +\definecolor{grayblue}{HTML}{728291} | |
| 25 | +\definecolor{pale}{HTML}{E8F0F2} | |
| 26 | +\definecolor{rulegray}{HTML}{D6E0E5} | |
| 27 | +\hypersetup{colorlinks=true,linkcolor=navy,citecolor=teal,urlcolor=teal,pdftitle={Local Bias Correction versus Global Synchronization in Distributed Maximum Likelihood},pdfauthor={Simon-Pierre Boucher and GPT-6.1}} | |
| 28 | +\titleformat{\section}{\large\bfseries\color{navy}}{\thesection.}{0.6em}{} | |
| 29 | +\titleformat{\subsection}{\normalsize\bfseries\color{navy}}{\thesubsection}{0.6em}{} | |
| 30 | +\titleformat{\subsubsection}{\normalsize\bfseries\color{charcoal}}{\thesubsubsection}{0.6em}{} | |
| 31 | +\titlespacing*{\section}{0pt}{19pt}{8pt} | |
| 32 | +\titlespacing*{\subsection}{0pt}{12pt}{5pt} | |
| 33 | +\setlength{\parskip}{4.2pt} | |
| 34 | +\setcounter{tocdepth}{1} | |
| 35 | +\setlength{\parindent}{0pt} | |
| 36 | +\setlength{\emergencystretch}{3em} | |
| 37 | +\setlist[itemize]{topsep=3pt,itemsep=2pt,leftmargin=1.4em} | |
| 38 | +\setlist[enumerate]{topsep=3pt,itemsep=3pt,leftmargin=1.6em} | |
| 39 | +\pagestyle{fancy} | |
| 40 | +\fancyhf{} | |
| 41 | +\fancyhead[L]{\small\color{grayblue}\textsc{Boucher \& GPT-6.1}} | |
| 42 | +\fancyhead[R]{\small\color{grayblue}\textsc{Distributed Maximum Likelihood}} | |
| 43 | +\fancyfoot[C]{\color{grayblue}\thepage} | |
| 44 | +\renewcommand{\headrulewidth}{0.35pt} | |
| 45 | +\renewcommand{\headrule}{\hbox to\headwidth{\color{rulegray}\leaders\hrule height \headrulewidth\hfill}} | |
| 46 | +\newtheorem{theorem}{Theorem}[section] | |
| 47 | +\newtheorem{proposition}[theorem]{Proposition} | |
| 48 | +\newtheorem{corollary}[theorem]{Corollary} | |
| 49 | +\newtheorem{lemma}[theorem]{Lemma} | |
| 50 | +\theoremstyle{definition} | |
| 51 | +\newtheorem{assumption}[theorem]{Assumption} | |
| 52 | +\newtheorem{remark}[theorem]{Remark} | |
| 53 | +\newtheorem{example}[theorem]{Example} | |
| 54 | +\DeclareMathOperator{\Var}{Var} | |
| 55 | +\DeclareMathOperator{\Bias}{Bias} | |
| 56 | +\DeclareMathOperator{\MSE}{MSE} | |
| 57 | +\DeclareMathOperator{\RMSE}{RMSE} | |
| 58 | +\newcommand{\EE}{\mathbb E} | |
| 59 | +\newcommand{\PP}{\mathbb P} | |
| 60 | +\newcommand{\RR}{\mathbb R} | |
| 61 | +\newcommand{\op}{O_{\!p}} | |
| 62 | +\newcommand{\smallop}{o_{\!p}} | |
| 63 | +\newcommand{\thet}{\theta_0} | |
| 64 | +\newcommand{\thn}{\widehat\theta_N} | |
| 65 | +\newcommand{\score}{s} | |
| 66 | +\newcommand{\I}{\mathcal I} | |
| 67 | +\newcommand{\D}{\mathcal D} | |
| 68 | +\lstdefinestyle{clean}{basicstyle=\ttfamily\footnotesize,backgroundcolor=\color{pale!25},frame=single,rulecolor=\color{rulegray},keywordstyle=\color{navy}\bfseries,commentstyle=\color{grayblue},showstringspaces=false,breaklines=true,tabsize=2,xleftmargin=1em,xrightmargin=1em} | |
| 69 | +\lstset{style=clean} | |
| 70 | +\begin{document} | |
| 71 | +\thispagestyle{empty} | |
| 72 | +\vspace*{0.23in} | |
| 73 | +{\color{teal}\rule{1.35in}{3pt}}\\[22pt] | |
| 74 | +{\LARGE\bfseries\color{navy} Local Bias Correction versus Global Synchronization in Distributed Maximum Likelihood\par} | |
| 75 | +\vspace{11pt} | |
| 76 | +{\Large\color{charcoal} Rate Frontiers, Exact Exponential Analysis, and Reproducible Monte Carlo Evidence\par} | |
| 77 | +\vspace{17pt} | |
| 78 | +{\large\bfseries Simon-Pierre Boucher\textsuperscript{1} \quad and \quad GPT-6.1\textsuperscript{2}\par} | |
| 79 | +\vspace{7pt} | |
| 80 | +{\small\color{grayblue}\textsuperscript{1} Universit\'e du Qu\'ebec en Outaouais \qquad \textsuperscript{2} AI co-author designation\par} | |
| 81 | +\vspace{10pt} | |
| 82 | +{\small\color{grayblue}Working paper \;\textbullet\; October 28, 2026 \;\textbullet\; Theoretical and numerical research note\par} | |
| 83 | +\vspace{18pt} | |
| 84 | +{\color{rulegray}\hrule} | |
| 85 | +\vspace{12pt} | |
| 86 | +{\bfseries\color{navy} Abstract}\par | |
| 87 | +Distributed maximum likelihood estimation (MLE) trades statistical efficiency against network communication and local computation. We analyze a two-phase construction that combines a within-worker split-sample jackknife with a single globally aggregated Newton step. Under fixed-dimensional asymptotic linearity, a second-order bias expansion and local Hessian regularity, we prove first-order equivalence to centralized MLE for $K=o(N^{7/8})$ partitions; the proof explicitly retains the jackknife's centered random remainder. A higher-order local extrapolation yields $K=o(N^{11/12})$ under stronger assumptions. For exponential observations, exact bias formulas and a Newton identity establish a nonzero limiting shift at the $N^{7/8}$ boundary of the specified algorithm. Reproducible exact-Gamma Monte Carlo experiments confirm the qualitative pattern. We account for derivative-communication costs and compare extra local estimation with an additional Newton synchronization. These results provide a transparent theoretical and numerical working note, not an established claim of methodological priority or improved wall-clock performance.\par | |
| 88 | +\vspace{13pt} | |
| 89 | +{\small\textbf{Keywords:} maximum likelihood; distributed econometrics; split-sample jackknife; one-step Newton; asymptotic equivalence; communication complexity; Monte Carlo.}\par | |
| 90 | +{\small\textbf{JEL codes:} C13, C15, C55, C63.}\par | |
| 91 | +\vspace{10pt} | |
| 92 | +{\color{rulegray}\hrule} | |
| 93 | +\vspace{10pt} | |
| 94 | +{\small\color{grayblue}\textbf{Disclosure.} The second author label is an AI designation supplied for this draft. The mathematics is conditional on explicit assumptions, and the numerical experiment is executable and independently reproducible. Researcher review, literature-priority checks, and empirical performance benchmarking remain necessary prior to submission.}\par | |
| 95 | +\clearpage | |
| 96 | +\tableofcontents | |
| 97 | +\clearpage | |
| 98 | + | |
| 99 | +\section{Introduction} | |
| 100 | +Massive datasets distributed across geographically or operationally separate devices create a tension between three objectives: statistically efficient estimation, low network communication, and computational scalability. For a regular parametric model the centralized maximum likelihood estimator can attain the inverse Fisher-information asymptotic variance, but collecting all observations on one machine may be expensive or prohibited. An alternative is to fit the model independently on $K$ data partitions and average the local estimators. This does not automatically preserve centralized statistical efficiency: nonlinear estimation induces a local bias that does not disappear merely by averaging independent estimates. | |
| 101 | + | |
| 102 | +Early systematic analyses of divide-and-conquer statistical optimization include \citet{zhang2013}, who study averaging and communication-efficient alternatives, while \citet{huang2019} develop a distributed one-step estimator. Multi-round and adaptive distributed procedures are represented by \citet{fan2023}. Separately, jackknife extrapolation has a long history in bias correction; \citet{dhaene2015} treat split-panel corrections, including higher-order variants, in nonlinear fixed-effects models. Our starting point is to combine these familiar elements and quantify the \emph{exchange rate} between extra local statistical work and an additional global synchronization. | |
| 103 | + | |
| 104 | +We focus on a sharply defined question: for fixed parameter dimension $p$, how fast may the partition count $K$ grow with the global sample size $N$ while a locally bias-corrected, globally one-step estimator remains first-order equivalent to the centralized MLE? For the half-block jackknife, we establish $K=o(N^{7/8})$ under explicit regularity conditions. Our proof takes special care to retain a random term of order $1/(m\sqrt K)$, where $m=N/K$ is the local sample size. Omitting that term would produce an incomplete argument, even if the final exponent happened to be correct. | |
| 105 | + | |
| 106 | +Our contributions are deliberately bounded. We give a transparent sufficient-rate theorem, an exact verification and boundary distribution for an exponential model, a higher-order generalization, and a reproducible computational illustration. We do \emph{not} establish that this procedure dominates the best available distributed estimators, nor do we prove a universal communication lower bound. The asymptotic regime is fixed-$p$, i.i.d., correctly specified, and locally regular. The exact exponential model actually admits trivial sufficient-statistic aggregation; it is therefore a proof-of-concept and counterexample laboratory, not a favorable application of distributed approximate MLE. | |
| 107 | + | |
| 108 | +\section{Model, estimators, and information exchange} | |
| 109 | +\subsection{Likelihood and partitioning} | |
| 110 | +Let $X_1,\ldots,X_N$ be i.i.d. observations with density $f(x;\theta)$, $\theta\in\Theta\subseteq\RR^p$, and true value $\thet$ in the interior of $\Theta$. Let $K$ divide $N$, set $m=N/K$, and partition the index set into disjoint blocks $\D_1,\ldots,\D_K$, each of size $m$. The average log-likelihood and centralized MLE are | |
| 111 | +\begin{align} | |
| 112 | + L_N(\theta)&=\frac1N\sum_{i=1}^N\log f(X_i;\theta), | |
| 113 | + &\thn&\in\arg\max_{\theta\in\Theta}L_N(\theta).\label{eq:global} | |
| 114 | +\end{align} | |
| 115 | +Worker $k$ computes a local MLE $\widehat\theta_{k,m}$ based on $\D_k$. In the baseline construction $m$ is even and each block has two disjoint halves $A_k,B_k$, with half-sample MLEs $\widehat\theta_{k,A}$ and $\widehat\theta_{k,B}$. Write | |
| 116 | +\begin{align} | |
| 117 | + \bar\theta_{\mathrm{raw}}&=K^{-1}\sum_{k=1}^K\widehat\theta_{k,m},\\ | |
| 118 | + J_k&=2\widehat\theta_{k,m}-\tfrac12\bigl(\widehat\theta_{k,A}+\widehat\theta_{k,B}\bigr),\label{eq:jack}\\ | |
| 119 | + \theta^{(0)}&=K^{-1}\sum_{k=1}^KJ_k.\label{eq:init} | |
| 120 | +\end{align} | |
| 121 | +This local operation needs three likelihood fits per block but, because all three fits access locally stored observations, introduces no within-cluster data transmission beyond the final parameter vector. | |
| 122 | + | |
| 123 | +Define the full score and Hessian by $g_N(\theta)=\nabla_\theta L_N(\theta)$ and $H_N(\theta)=\nabla_\theta^2L_N(\theta)$. After the coordinator broadcasts $\theta^{(0)}$, each worker sends its contributions to $g_N(\theta^{(0)})$ and $H_N(\theta^{(0)})$. The coordinator computes | |
| 124 | +\begin{equation} | |
| 125 | + \boxed{\quad\theta^{(1)}=\theta^{(0)}-H_N(\theta^{(0)})^{-1}g_N(\theta^{(0)})\quad.}\label{eq:newton} | |
| 126 | +\end{equation} | |
| 127 | +An invertible Hessian and well-defined, consistent local solutions are assumed throughout; a production implementation may require regularization and step damping, neither of which is covered by the present theory. | |
| 128 | + | |
| 129 | +\begin{figure}[ht] | |
| 130 | +\centering | |
| 131 | +\begin{tikzpicture}[node distance=8mm and 7mm,>=Latex,scale=.95,transform shape, | |
| 132 | + box/.style={draw=rulegray,rounded corners=3pt,fill=white,align=center,minimum height=0.65cm,font=\small,inner sep=5pt}, | |
| 133 | + cent/.style={box,fill=pale!75},arr/.style={->,thick,draw=teal}] | |
| 134 | +\node[box,text width=10.5cm] (data) {Partitioned observations: $N=Km$ and fixed $p$}; | |
| 135 | +\node[box,below left=of data,xshift=17mm,text width=3.1cm] (w1) {Worker $1$\\full / half-sample MLEs}; | |
| 136 | +\node[box,right=of w1,text width=3.1cm] (w2) {Worker $2$\\full / half-sample MLEs}; | |
| 137 | +\node[box,right=of w2,text width=3.1cm] (wk) {Worker $K$\\full / half-sample MLEs}; | |
| 138 | +\node[cent,below=13mm of w2,text width=8.2cm] (avg) {Coordinator: average jackknifed estimates $\theta^{(0)}$}; | |
| 139 | +\node[cent,below=10mm of avg,text width=8.2cm] (agg) {Broadcast $\theta^{(0)}$; aggregate scores and Hessians}; | |
| 140 | +\node[box,below=9mm of agg,text width=8.2cm] (out) {One Newton correction: $\theta^{(1)}$}; | |
| 141 | +\draw[arr] (data.south) -- (w1.north);\draw[arr] (data.south) -- (w2.north);\draw[arr] (data.south) -- (wk.north); | |
| 142 | +\draw[arr] (w1.south) -- (avg.north west);\draw[arr] (w2.south) -- (avg.north);\draw[arr] (wk.south) -- (avg.north east); | |
| 143 | +\draw[arr] (avg) -- (agg);\draw[arr] (agg) -- (out); | |
| 144 | +\end{tikzpicture} | |
| 145 | +\caption{Local bias correction followed by one globally aggregated Newton step. Arrows indicate logical phases, not measured network latency.}\label{fig:architecture} | |
| 146 | +\end{figure} | |
| 147 | + | |
| 148 | +\subsection{Regularity and the benchmark MLE} | |
| 149 | +Our central statistical result is conditional upon the following strong but explicit assumptions, stated for a \emph{sequence} with $N\to\infty$ and $K=K_N\to\infty$ permitted. | |
| 150 | + | |
| 151 | +\begin{assumption}[Uniform asymptotic linear expansion]\label{ass:lin} | |
| 152 | +For every relevant sub-sample size $n\in\{m,m/2,N\}$, its MLE has the representation | |
| 153 | +\begin{equation} | |
| 154 | + \widehat\theta_n-\thet=\frac1n\sum_{i=1}^n\psi(X_i)+R_n,\label{eq:lin} | |
| 155 | +\end{equation} | |
| 156 | +where $\EE\psi(X_i)=0$, $\EE\|\psi(X_i)\|^2<\infty$, $\Var(\psi(X_i))=\I(\thet)^{-1}$ for a positive-definite Fisher information matrix, and a common constant $C<\infty$ satisfies | |
| 157 | +\begin{equation} | |
| 158 | + \EE\|R_n\|^2\le Cn^{-2}.\label{eq:r2} | |
| 159 | +\end{equation} | |
| 160 | +For correctly specified, regular parametric MLE, $\psi(x)=\I(\thet)^{-1}\nabla_\theta\log f(x;\thet)$. The expansion is required jointly with the same $\psi$ on all nested subsamples. | |
| 161 | +\end{assumption} | |
| 162 | + | |
| 163 | +\begin{assumption}[Second-order bias expansion]\label{ass:bias} | |
| 164 | +There exist vectors $b_1,b_2$ and a constant $C_b$, not depending on $n$, for which | |
| 165 | +\begin{equation} | |
| 166 | + \EE R_n=\frac{b_1}{n}+\frac{b_2}{n^2}+\rho_n, | |
| 167 | + \qquad\|\rho_n\|\le C_b n^{-3},\label{eq:biasexp} | |
| 168 | +\end{equation} | |
| 169 | +for the sample sizes involved. The existence of this expansion is not implied by mere consistency or asymptotic normality. | |
| 170 | +\end{assumption} | |
| 171 | + | |
| 172 | +\begin{assumption}[Local Newton regularity]\label{ass:hess} | |
| 173 | +With probability tending to one, $\widehat\theta_N$ is a consistent interior stationary maximizer, $g_N(\thn)=0$, and there is a fixed convex neighborhood $\mathcal U$ of $\thet$ such that | |
| 174 | +\begin{equation} | |
| 175 | + \sup_{\theta\in\mathcal U}\|H_N(\theta)^{-1}\|=\op(1),\qquad | |
| 176 | + \sup_{u\ne v\in\mathcal U}\frac{\|H_N(u)-H_N(v)\|}{\|u-v\|}=\op(1).\label{eq:localreg} | |
| 177 | +\end{equation} | |
| 178 | +The probability that both $\thn$ and the initializer lie in $\mathcal U$ tends to one. The centralized MLE obeys $\sqrt N(\thn-\thet)\Rightarrow\mathcal N(0,\I(\thet)^{-1})$. | |
| 179 | +\end{assumption} | |
| 180 | + | |
| 181 | +\begin{remark} | |
| 182 | +The assumptions deliberately fix $p$ and require i.i.d. partitions, finite moments of inverse-information-type quantities, and a well-behaved local optimizer. Weak identification, incidental parameters, dependent samples, and growing $p$ require different analysis. The theorem does not apply merely because code can execute on multiple machines. | |
| 183 | +\end{remark} | |
| 184 | + | |
| 185 | +\section{An exact decomposition of local bias correction} | |
| 186 | +\begin{proposition}[Half-block jackknife initializer]\label{prop:init} | |
| 187 | +Under \cref{ass:lin,ass:bias}, with independent blocks and $m=N/K$ even, | |
| 188 | +\begin{align} | |
| 189 | + \EE[J_k]-\thet&=-\frac{2b_2}{m^2}+O(m^{-3}),\label{eq:jackbias}\\ | |
| 190 | + \theta^{(0)}-\thn&= | |
| 191 | + \op\left(m^{-2}+\frac1{m\sqrt K}+N^{-1}\right)\\ | |
| 192 | + &=\op\left(\frac{K^2}{N^2}+\frac{\sqrt K}{N}+\frac1N\right).\label{eq:initrate} | |
| 193 | +\end{align} | |
| 194 | +\end{proposition} | |
| 195 | +\begin{proof} | |
| 196 | +Write $U_k=m^{-1}\sum_{i\in\D_k}\psi(X_i)$ and define $U_{k,A}$ and $U_{k,B}$ analogously. Because the halves partition $\D_k$ exactly, | |
| 197 | +\begin{equation} | |
| 198 | + \tfrac12(U_{k,A}+U_{k,B})=U_k.\label{eq:influencecancel} | |
| 199 | +\end{equation} | |
| 200 | +Applying \cref{eq:lin} to the three local estimators yields | |
| 201 | +\begin{equation} | |
| 202 | + J_k-\thet=U_k+R_k^J,\qquad | |
| 203 | + R_k^J=2R_{k,m}-\tfrac12(R_{k,A,m/2}+R_{k,B,m/2}).\label{eq:rj} | |
| 204 | +\end{equation} | |
| 205 | +Using \cref{eq:biasexp} at $m$ and $m/2$ gives | |
| 206 | +\begin{align*} | |
| 207 | + \EE R_k^J | |
| 208 | + &=2\left(\frac{b_1}{m}+\frac{b_2}{m^2}\right) | |
| 209 | + -\left(\frac{2b_1}{m}+\frac{4b_2}{m^2}\right)+O(m^{-3})\\ | |
| 210 | + &=-\frac{2b_2}{m^2}+O(m^{-3}). | |
| 211 | +\end{align*} | |
| 212 | +The $L^2$ triangle inequality and \cref{eq:r2} imply | |
| 213 | +\begin{align*} | |
| 214 | + (\EE\|R_k^J\|^2)^{1/2} | |
| 215 | + &\le 2\sqrt C/m+\frac12\bigl(2\sqrt C/m+2\sqrt C/m\bigr)\\ | |
| 216 | + &=4\sqrt C/m. | |
| 217 | +\end{align*} | |
| 218 | +Hence $\EE\|R_k^J\|^2\le16C/m^2$. Independence of blocks eliminates cross terms for centered remainders: | |
| 219 | +\begin{equation} | |
| 220 | + \EE\left\|K^{-1}\sum_{k=1}^K(R_k^J-\EE R_k^J)\right\|^2 | |
| 221 | + \le\frac{16C}{Km^2}. | |
| 222 | +\end{equation} | |
| 223 | +Thus, by Markov's inequality, | |
| 224 | +\begin{equation} | |
| 225 | + \theta^{(0)}-\thet | |
| 226 | + =\frac1N\sum_{i=1}^N\psi(X_i) | |
| 227 | + -\frac{2b_2}{m^2}+O(m^{-3}) | |
| 228 | + +\op\left(\frac1{m\sqrt K}\right). | |
| 229 | +\end{equation} | |
| 230 | +The full-sample expansion gives $\thn-\thet=N^{-1}\sum_i\psi(X_i)+\op(N^{-1})$. Subtraction cancels the leading influence term \emph{exactly}, proving the second assertion.\end{proof} | |
| 231 | + | |
| 232 | +\begin{remark}[Why the random remainder matters]\label{rem:noise} | |
| 233 | +A small \emph{mean} remainder does not imply a small realized remainder. The term $1/(m\sqrt K)=\sqrt K/N$ in \cref{eq:initrate} describes variation of centered local second-order terms. Bias cancellation alone would not justify removing it. Averaging over workers does remove part of its magnitude, but only after an explicit second-moment argument. | |
| 234 | +\end{remark} | |
| 235 | + | |
| 236 | +\section{One Newton step and the $7/8$ frontier} | |
| 237 | +\begin{lemma}[Quadratic local error contraction]\label{lem:newton} | |
| 238 | +Under \cref{ass:hess}, for any initializer $\vartheta_N$ satisfying $\vartheta_N\to_p\thet$ and lying with $\thn$ in the regularity neighborhood, its undamped Newton update $\mathcal N_N(\vartheta_N)=\vartheta_N-H_N(\vartheta_N)^{-1}g_N(\vartheta_N)$ obeys | |
| 239 | +\begin{equation} | |
| 240 | + \|\mathcal N_N(\vartheta_N)-\thn\| | |
| 241 | + =\op(\|\vartheta_N-\thn\|^2).\label{eq:quadratic} | |
| 242 | +\end{equation} | |
| 243 | +\end{lemma} | |
| 244 | +\begin{proof} | |
| 245 | +Let $e=\vartheta_N-\thn$. By the fundamental theorem of calculus, $g_N(\vartheta_N)=\int_0^1 H_N(\thn+te)e\,dt$. Since $g_N(\thn)=0$, one obtains | |
| 246 | +\begin{align*} | |
| 247 | + \mathcal N_N(\vartheta_N)-\thn | |
| 248 | + &=H_N(\vartheta_N)^{-1}\int_0^1 | |
| 249 | + [H_N(\vartheta_N)-H_N(\thn+te)]e\,dt. | |
| 250 | +\end{align*} | |
| 251 | +The local Lipschitz property bounds the integrand by $\Lambda_N(1-t)\|e\|^2$. Taking norms and integrating yields | |
| 252 | +\begin{equation*} | |
| 253 | + \|\mathcal N_N(\vartheta_N)-\thn\| | |
| 254 | + \le\tfrac12\|H_N(\vartheta_N)^{-1}\|\Lambda_N\|e\|^2, | |
| 255 | +\end{equation*} | |
| 256 | +where the multiplicative factor is $\op(1)$ by \cref{ass:hess}.\end{proof} | |
| 257 | + | |
| 258 | +\begin{theorem}[Jackknife--Newton equivalence]\label{thm:seven} | |
| 259 | +Under \cref{ass:lin,ass:bias,ass:hess}, with $K=o(N^{7/8})$ and $m=N/K$ even, the estimator in \cref{eq:jack,eq:init,eq:newton} satisfies | |
| 260 | +\begin{equation} | |
| 261 | + \boxed{\quad\sqrt N\bigl(\theta^{(1)}-\thn\bigr)\xrightarrow{p}0.\quad}\label{eq:equiv} | |
| 262 | +\end{equation} | |
| 263 | +Consequently, | |
| 264 | +\begin{equation} | |
| 265 | + \sqrt N(\theta^{(1)}-\thet)\Rightarrow | |
| 266 | + \mathcal N\left(0,\I(\thet)^{-1}\right). | |
| 267 | +\end{equation} | |
| 268 | +\end{theorem} | |
| 269 | +\begin{proof} | |
| 270 | +By \cref{prop:init}, the starting discrepancy is | |
| 271 | +\begin{equation*} | |
| 272 | + e_0=\theta^{(0)}-\thn | |
| 273 | + =\op(m^{-2}+m^{-1}K^{-1/2}+N^{-1}). | |
| 274 | +\end{equation*} | |
| 275 | +The assumed $K$ regime implies $m\to\infty$ and hence consistency of the initializer, permitting \cref{lem:newton}. Then | |
| 276 | +\begin{align} | |
| 277 | + \|\theta^{(1)}-\thn\| | |
| 278 | + &=\op\left(m^{-4}+\frac1{Km^2}+N^{-2}\right)\nonumber\\ | |
| 279 | + &=\op\left(\frac{K^4}{N^4}+\frac K{N^2}+\frac1{N^2}\right).\label{eq:newtonrate} | |
| 280 | +\end{align} | |
| 281 | +Multiplication by $\sqrt N$ gives | |
| 282 | +\begin{equation} | |
| 283 | + \sqrt N\|\theta^{(1)}-\thn\| | |
| 284 | + =\op\left(\frac{K^4}{N^{7/2}}+\frac K{N^{3/2}}+N^{-3/2}\right). | |
| 285 | +\end{equation} | |
| 286 | +All three deterministic orders vanish whenever $K=o(N^{7/8})$. The first result follows; the limiting distribution then follows from the centralized MLE central limit theorem and Slutsky's lemma.\end{proof} | |
| 287 | + | |
| 288 | +\begin{corollary}[Comparison with an uncorrected start]\label{cor:raw} | |
| 289 | +If the ordinary local average instead has bias $\op(m^{-1})$ and a linear expansion of the form used above, then one global Newton step has a sufficient statistical-equivalence regime $K=o(N^{3/4})$. Two global Newton steps, started from the same raw average, have a sufficient regime $K=o(N^{7/8})$ under analogous local regularity at successive steps. | |
| 290 | +\end{corollary} | |
| 291 | +\begin{proof} | |
| 292 | +The raw starting discrepancy is $\op(m^{-1}+m^{-1}K^{-1/2}+N^{-1})$ and is dominated by $m^{-1}$ in the relevant regime. One squaring requires $\sqrt N m^{-2}\to0$, or $K=o(N^{3/4})$; two squarings require $\sqrt Nm^{-4}\to0$, or $K=o(N^{7/8})$. All stochastic remainders are negligible under these regimes.\end{proof} | |
| 293 | + | |
| 294 | +\begin{remark}[Interpretation rather than priority claim] | |
| 295 | +\cref{thm:seven} does not by itself establish a new statistical estimator relative to all prior literature. The well-studied ingredients are local estimator combination, one-step distributed optimization, and split-sample bias correction \citep{zhang2013,huang2019,dhaene2015}. Its role here is to establish a self-contained rate calculation and a specific communication--computation comparison suitable for further research. | |
| 296 | +\end{remark} | |
| 297 | + | |
| 298 | +\section{Exact exponential-model analysis} | |
| 299 | +\subsection{An analytically tractable likelihood} | |
| 300 | +Suppose $X_i\stackrel{\mathrm{iid}}\sim\operatorname{Exp}(\theta_0)$ using the \emph{rate} parameterization. The average log-likelihood is | |
| 301 | +\begin{equation} | |
| 302 | + L_n(\theta)=\log\theta-\theta\bar X_n, | |
| 303 | + \qquad\widehat\theta_n=\bar X_n^{-1}. | |
| 304 | +\end{equation} | |
| 305 | +Let $S_n=\sum_{i=1}^n X_i$, so $S_n\sim\operatorname{Gamma}(n,\text{rate}=\theta_0)$. The Gamma inverse moments give, for $n>2$, | |
| 306 | +\begin{align} | |
| 307 | + \EE[\widehat\theta_n]&=\theta_0\frac n{n-1},\label{eq:expmom1}\\ | |
| 308 | + \Var(\widehat\theta_n)&=\theta_0^2\frac{n^2}{(n-1)^2(n-2)}.\label{eq:expmom2} | |
| 309 | +\end{align} | |
| 310 | +Here $\psi(X)=\theta_0-\theta_0^2X$. The remainder has the \emph{exact} form | |
| 311 | +\begin{equation} | |
| 312 | + R_n=\widehat\theta_n-\theta_0-\frac1n\sum_i\psi(X_i) | |
| 313 | + =\frac{(1-\theta_0\bar X_n)^2}{\bar X_n},\label{eq:expremainder} | |
| 314 | +\end{equation} | |
| 315 | +whose moments are | |
| 316 | +\begin{align} | |
| 317 | + \EE R_n&=\frac{\theta_0}{n-1}=\theta_0(n^{-1}+n^{-2}+O(n^{-3})),\\ | |
| 318 | + \EE R_n^2&=\theta_0^2\frac{3n+2}{n(n-1)(n-2)}=O(n^{-2}).\label{eq:expR2} | |
| 319 | +\end{align} | |
| 320 | +These identities verify the key strong mean-square and bias assumptions for this example, with $b_1=b_2=\theta_0$. The score and Hessian are $g_N(\theta)=1/\theta-\bar X_N$ and $H_N(\theta)=-1/\theta^2$; local Newton regularity holds around every fixed $\theta_0>0$. | |
| 321 | + | |
| 322 | +\subsection{The exact jackknife bias} | |
| 323 | +The local full and half estimators have means $\theta_0m/(m-1)$ and $\theta_0m/(m-2)$ respectively. Therefore | |
| 324 | +\begin{align} | |
| 325 | + \EE J_k&=2\theta_0\frac m{m-1}-\theta_0\frac m{m-2},\nonumber\\ | |
| 326 | + \boxed{\;\EE J_k-\theta_0=-\frac{2\theta_0}{(m-1)(m-2)}.\;}\label{eq:exactbias2} | |
| 327 | +\end{align} | |
| 328 | +The identity confirms the $m^{-2}$ order without relying on an asymptotic expansion. The split estimates are dependent because they share observations with the full estimate; accounting for this dependence is essential when evaluating variance. Nevertheless, \cref{eq:expremainder} and the decomposition in \cref{eq:rj} ensure $\Var(J_k)=\theta_0^2/m+O(m^{-2})$ as $m\to\infty$. | |
| 329 | + | |
| 330 | +\subsection{Exact Newton identity and boundary distribution} | |
| 331 | +A Newton step started from $x$ simplifies to | |
| 332 | +\begin{equation} | |
| 333 | + \mathcal N_N(x)=2x-x^2\bar X_N. | |
| 334 | +\end{equation} | |
| 335 | +Because $\thn=1/\bar X_N$, the following error formula holds \emph{exactly}: | |
| 336 | +\begin{equation} | |
| 337 | + \boxed{\quad\mathcal N_N(x)-\thn | |
| 338 | + =-\frac{(x-\thn)^2}{\thn}.\quad}\label{eq:exactnewton} | |
| 339 | +\end{equation} | |
| 340 | + | |
| 341 | +\begin{proposition}[Sharp boundary for the specified exponential construction]\label{prop:boundary} | |
| 342 | +Suppose $N\to\infty$, $m=N/K$ is even and $K/N^{7/8}\to c\in(0,\infty)$. For the half-block jackknife average and one Newton update, | |
| 343 | +\begin{equation} | |
| 344 | + \sqrt N\bigl(\theta^{(1)}-\thn\bigr) | |
| 345 | + \xrightarrow{p}-4\theta_0c^4,\label{eq:frontiergap} | |
| 346 | +\end{equation} | |
| 347 | +and hence | |
| 348 | +\begin{equation} | |
| 349 | + \sqrt N(\theta^{(1)}-\theta_0) | |
| 350 | + \Rightarrow\mathcal N(-4\theta_0c^4,\theta_0^2).\label{eq:frontierdist} | |
| 351 | +\end{equation} | |
| 352 | +\end{proposition} | |
| 353 | +\begin{proof} | |
| 354 | +From \cref{eq:exactbias2}, the jackknife average has bias asymptotic to $-2\theta_0m^{-2}$. Since $m=N/K\sim N^{1/8}/c$, | |
| 355 | +\begin{equation} | |
| 356 | + N^{1/4}(\EE\theta^{(0)}-\theta_0)\to-2\theta_0c^2. | |
| 357 | +\end{equation} | |
| 358 | +The centered average fluctuates on scale $N^{-1/2}$, since its leading influence function averages all $N$ observations and the remaining centered term has standard deviation $O((m\sqrt K)^{-1})=o(N^{-1/4})$. Likewise $\thn-\theta_0=O_p(N^{-1/2})$. Therefore | |
| 359 | +\begin{equation} | |
| 360 | + N^{1/4}(\theta^{(0)}-\thn)\xrightarrow{p}-2\theta_0c^2. | |
| 361 | +\end{equation} | |
| 362 | +Use the exact identity \cref{eq:exactnewton}, $\thn\to_p\theta_0$, and the continuous mapping theorem to obtain \cref{eq:frontiergap}. The centralized MLE satisfies $\sqrt N(\thn-\theta_0)\Rightarrow\mathcal N(0,\theta_0^2)$, proving \cref{eq:frontierdist}.\end{proof} | |
| 363 | + | |
| 364 | +\begin{remark}[Not a lower bound for all distributed estimators] | |
| 365 | +In the exponential model, each worker can transmit $\sum_{i\in\D_k}X_i$. Summing these $K$ scalars recovers $\thn=N/\sum_i X_i$ \emph{exactly}. Thus the sharp frontier in \cref{prop:boundary} concerns this particular jackknife-plus-one-step algorithm; it is not an information-theoretic limitation of distributed likelihood estimation. | |
| 366 | +\end{remark} | |
| 367 | + | |
| 368 | +\section{Higher-order local extrapolation} | |
| 369 | +\subsection{Cancelling two bias terms with four quarter-block fits} | |
| 370 | +Assume $m$ is divisible by four. Let $A_{k,2}$ denote the average of MLEs on the two disjoint halves and $A_{k,4}$ the corresponding average on four disjoint quarter-blocks. Define | |
| 371 | +\begin{equation} | |
| 372 | + \boxed{\quad J_k^{[3]}=\frac83\widehat\theta_{k,m}-2A_{k,2}+\frac13 A_{k,4}.\quad}\label{eq:jack3} | |
| 373 | +\end{equation} | |
| 374 | +The coefficients solve | |
| 375 | +\begin{align} | |
| 376 | + \tfrac83-2+\tfrac13&=1,\\ | |
| 377 | + \tfrac83-2(2)+\tfrac13(4)&=0,\\ | |
| 378 | + \tfrac83-2(2^2)+\tfrac13(4^2)&=0. | |
| 379 | +\end{align} | |
| 380 | +The first identity preserves the leading influence term; the others cancel $n^{-1}$ and $n^{-2}$ terms of a common bias expansion. With a third-order bias expansion $\EE R_n=b_1n^{-1}+b_2n^{-2}+b_3n^{-3}+O(n^{-4})$, the local bias is | |
| 381 | +\begin{equation} | |
| 382 | + \EE[J_k^{[3]}]-\theta_0=8b_3m^{-3}+O(m^{-4}).\label{eq:bias3} | |
| 383 | +\end{equation} | |
| 384 | +The analog of the $L^2$ triangle inequality establishes a centered remainder of order $\op((m\sqrt K)^{-1})$ after averaging workers, despite sharing nested data. Consequently | |
| 385 | +\begin{equation} | |
| 386 | + \theta^{(0,[3])}-\thn | |
| 387 | + =\op\left(m^{-3}+\frac1{m\sqrt K}+N^{-1}\right).\label{eq:init3} | |
| 388 | +\end{equation} | |
| 389 | + | |
| 390 | +\begin{corollary}[A sufficient $11/12$ regime]\label{cor:eleven} | |
| 391 | +Suppose \cref{ass:lin,ass:hess} hold for $n\in\{m,m/2,m/4,N\}$, and the bias expansion through $n^{-3}$ holds with $O(n^{-4})$ remainder. For $m$ divisible by four and $K=o(N^{11/12})$, one global Newton step starting from $K^{-1}\sum_kJ_k^{[3]}$ is first-order equivalent to centralized MLE. | |
| 392 | +\end{corollary} | |
| 393 | +\begin{proof} | |
| 394 | +From \cref{eq:init3} and \cref{lem:newton}, | |
| 395 | +\begin{equation*} | |
| 396 | + \sqrt N\|\theta^{(1,[3])}-\thn\| | |
| 397 | + =\op\left(\frac{K^6}{N^{11/2}}+\frac K{N^{3/2}}+N^{-3/2}\right)=o_p(1). | |
| 398 | +\end{equation*} | |
| 399 | +The first expression tends to zero if $K=o(N^{11/12})$; the others do so automatically.\end{proof} | |
| 400 | + | |
| 401 | +\subsection{Exact higher-order bias in the exponential model} | |
| 402 | +Using the exact mean in \cref{eq:expmom1}, for $m>4$, | |
| 403 | +\begin{equation} | |
| 404 | + \boxed{\quad\EE[J_k^{[3]}]-\theta_0 | |
| 405 | + =\frac{8\theta_0}{(m-1)(m-2)(m-4)}.\quad}\label{eq:exactbias3} | |
| 406 | +\end{equation} | |
| 407 | +The bias magnitude is indeed $O(m^{-3})$. However, higher-order jackknife coefficients can amplify the finite-sample variance; the correction also requires local fits on quarter-blocks, which can have unstable or nonexistent moments for very small $m$. This is a reason to compare bias, variance, wall-clock time, and convergence failures rather than optimizing the asymptotic exponent alone. | |
| 408 | + | |
| 409 | +\subsection{A general bias--synchronization tradeoff} | |
| 410 | +Consider a fixed bias-removal order $r\ge1$. Suppose a local combination retains the same influence function and achieves | |
| 411 | +\begin{equation} | |
| 412 | + \theta_r^{(0)}-\thn | |
| 413 | + =\op\left(m^{-r}+\frac1{m\sqrt K}+N^{-1}\right),\label{eq:generalinit} | |
| 414 | +\end{equation} | |
| 415 | +with the requisite uniform moment bounds. After $q\ge0$ locally regular global Newton updates, repeated quadratic error contraction yields | |
| 416 | +\begin{equation} | |
| 417 | + \|\theta_r^{(q)}-\thn\| | |
| 418 | + =\op\left[ | |
| 419 | + \left(m^{-r}+\frac1{m\sqrt K}+N^{-1}\right)^{2^q} | |
| 420 | + \right].\label{eq:qsteps} | |
| 421 | +\end{equation} | |
| 422 | +\begin{proposition}[Sufficient rate envelope]\label{prop:envelope} | |
| 423 | +For fixed integers $r\ge1$ and $q\ge0$, under \cref{eq:generalinit} and the required local quadratic contraction at each update, a sufficient condition for $\sqrt N(\theta_r^{(q)}-\thn)\to_p0$ is | |
| 424 | +\begin{equation} | |
| 425 | + \boxed{\quad K=o\left(N^{\,1-1/(2^{q+1}r)}\right).\quad}\label{eq:envelope} | |
| 426 | +\end{equation} | |
| 427 | +\end{proposition} | |
| 428 | +\begin{proof} | |
| 429 | +With $m=N/K$, the bias component of the scaled remainder is $\sqrt N m^{-r2^q}=N^{1/2}(K/N)^{r2^q}$, which vanishes precisely under the stated sufficient regime. The centered-remainder component is $\sqrt N(\sqrt K/N)^{2^q}$. For $q=0$ it equals $\sqrt{K/N}$, which vanishes since the displayed exponent is less than one; for $q\ge1$ it is at most of order $N^{1/2-2^{q-1}}$ when $K\le N$, hence vanishes. The full-sample term $\sqrt N(N^{-1})^{2^q}$ also vanishes. The sum of finitely many terms may be bounded by the corresponding sum of powers, finishing the proof.\end{proof} | |
| 430 | + | |
| 431 | +\begin{table}[ht] | |
| 432 | +\centering\small | |
| 433 | +\caption{Sufficient partition-growth exponents under fixed-dimensional regularity. These are \emph{not} hardware speedups or universal lower bounds.}\label{tab:frontiers} | |
| 434 | +\begin{tabularx}{\linewidth}{@{}Xccc@{}} | |
| 435 | +\toprule | |
| 436 | +Initialization & Local bias order $r$ & Global Newton steps $q$ & $K=o(N^\alpha)$\\ | |
| 437 | +\midrule | |
| 438 | +Raw local mean & 1 & 0 & $\alpha=1/2$\\ | |
| 439 | +Raw local mean & 1 & 1 & $\alpha=3/4$\\ | |
| 440 | +Raw local mean & 1 & 2 & $\alpha=7/8$\\ | |
| 441 | +Half-block jackknife & 2 & 0 & $\alpha=3/4$\\ | |
| 442 | +Half-block jackknife & 2 & 1 & $\alpha=7/8$\\ | |
| 443 | +Quarter-block extrapolation & 3 & 0 & $\alpha=5/6$\\ | |
| 444 | +Quarter-block extrapolation & 3 & 1 & $\alpha=11/12$\\ | |
| 445 | +\bottomrule | |
| 446 | +\end{tabularx} | |
| 447 | +\end{table} | |
| 448 | + | |
| 449 | +\begin{remark} | |
| 450 | +\Cref{prop:envelope} is a rate-accounting result, conditional on an initializer satisfying \cref{eq:generalinit}; it is \emph{not} a proof that such an estimator exists for every $r$ or every econometric model. Constructing stable higher-order initializers in heterogeneous or high-dimensional settings remains an open implementation and theory problem. | |
| 451 | +\end{remark} | |
| 452 | + | |
| 453 | +\section{Numerical experiment: exact Gamma simulation} | |
| 454 | +\subsection{Design and reproducibility} | |
| 455 | +To isolate estimator behavior from optimization and simulation discretization, we draw the sufficient statistics of exponential observations exactly. For each worker $k$, we generate four independent Gamma variates, each representing the sum of $m/4$ independent exponential observations; their sum is the block sufficient statistic. From these sums we evaluate local full, half, and quarter MLEs; the centralized MLE follows from the sum over all workers. | |
| 456 | + | |
| 457 | +The common sample size is $N=2^{20}=1{,}048{,}576$, the true rate is $\theta_0=1$, and the partition sizes are $m\in\{256,64,16\}$, corresponding to $K\in\{4{,}096;16{,}384;65{,}536\}$. We run $B=500$ independent Monte Carlo replications in each design with NumPy's default random generator initialized at seed \texttt{20261028}. These are \emph{simulated data partitions, not physical computers}. We compare raw averaging with one or two Newton steps and both jackknife corrections with one Newton step. The estimand for each replication is the gap to the \emph{same-sample centralized MLE}, scaled by $\sqrt N$: | |
| 458 | +\begin{equation} | |
| 459 | + \Delta_b=\sqrt N(\widehat\theta_b^{\mathrm{dist}}-\widehat\theta_{N,b}).\label{eq:mcgap} | |
| 460 | +\end{equation} | |
| 461 | +A gap to centralized MLE directly measures the numerical-statistical approximation, not the sampling error $\widehat\theta_{N,b}-\theta_0$. The exact code and the full 500-replication outputs are included in the accompanying reproducibility bundle. | |
| 462 | + | |
| 463 | +\subsection{Results} | |
| 464 | +\begin{table}[ht] | |
| 465 | +\centering\small | |
| 466 | +\caption{Mean scaled gap $\overline\Delta$ across 500 independent Monte Carlo replications. More negative numbers indicate a larger downward difference relative to centralized MLE. Rounding to six decimal places.}\label{tab:simulation} | |
| 467 | +\begin{tabular}{@{}lrrr@{}} | |
| 468 | +\toprule | |
| 469 | +Estimator & $m=256$ & $m=64$ & $m=16$\\ | |
| 470 | +\midrule | |
| 471 | +Raw mean + 1 Newton & $-0.015728$ & $-0.258367$ & $-4.549613$\\ | |
| 472 | +Raw mean + 2 Newton & $\approx 0$ & $-0.000065$ & $-0.020216$\\ | |
| 473 | +Half-block jackknife + 1 Newton & $-0.000016$ & $-0.000331$ & $-0.093720$\\ | |
| 474 | +Quarter-block jackknife + 1 Newton & $-0.000030$ & $-0.000147$ & $-0.011034$\\ | |
| 475 | +\bottomrule | |
| 476 | +\end{tabular} | |
| 477 | +\end{table} | |
| 478 | + | |
| 479 | +\begin{figure}[ht] | |
| 480 | +\centering | |
| 481 | +\includegraphics[width=\linewidth]{figures/mc_comparison.pdf} | |
| 482 | +\caption{Absolute mean and root mean squared scaled gaps to the centralized MLE, calculated from the reproducibility script. Logarithmic axes make both small and large discrepancies visible. For the quarter-block method, its increased finite-sample variability can sometimes offset reduced bias.}\label{fig:mc} | |
| 483 | +\end{figure} | |
| 484 | + | |
| 485 | +The half-block correction substantially reduces the one-step gap for large local samples, consistent with \cref{thm:seven}. At $m=16$, higher-order correction improves the mean one-step gap still further, but also increases local fitting requirements and possible sensitivity to extreme quarter-sample MLEs. With two Newton steps, uncorrected averaging performs competitively: this is expected from \cref{cor:raw}, not evidence against the theory. The simulation is not a runtime evaluation and provides no evidence that one strategy is faster on actual distributed nodes. | |
| 486 | + | |
| 487 | +\subsection{What the simulation does not establish} | |
| 488 | +Three caveats are material. First, $N$ is fixed, whereas \cref{thm:seven,cor:eleven} describe triangular-array limits. Second, the Monte Carlo experiment uses exact aggregate statistics, hence omits network latency, local iterative optimizer cost, load imbalance, failure recovery, and floating-point differences across devices. Third, because the Gamma model has an elementary sufficient statistic, any empirical advantage here is a property of the estimator \emph{approximation}, not an argument against exact aggregation. | |
| 489 | + | |
| 490 | +\section{Communication and computation accounting} | |
| 491 | +\subsection{Payload and synchronization rounds} | |
| 492 | +For $p$ parameters and dense symmetric Hessians, each worker's initial jackknife estimate is a $p$-vector. One subsequent derivative evaluation requires one $p$-vector and $p(p+1)/2$ unique Hessian entries per worker. Excluding protocol headers, numerical precision and broadcasts, the total amount of worker-to-coordinator data is on the order of | |
| 493 | +\begin{equation} | |
| 494 | + \boxed{\quad K\left[2p+\frac{p(p+1)}2\right]\quad\text{numbers}.}\label{eq:comm} | |
| 495 | +\end{equation} | |
| 496 | +The two communication \emph{phases} are not reducible to one if the second-phase score and Hessian are evaluated at the common, data-dependent $\theta^{(0)}$. One can transmit asynchronous or compressed derivatives, but then the current mathematical guarantee does not automatically apply. Dense-Hessian payload and coordinator matrix inversion may become prohibitive when $p$ is large. | |
| 497 | + | |
| 498 | +\begin{table}[ht] | |
| 499 | +\centering\small | |
| 500 | +\caption{Logical comparison. A ``derivative phase'' is one globally synchronized derivative evaluation after the initial communication of local estimates. Local-fit counts assume direct refitting on nested partitions.}\label{tab:comm} | |
| 501 | +\begin{tabularx}{\linewidth}{@{}Xccc@{}} | |
| 502 | +\toprule | |
| 503 | +Method & Local MLE fits / worker & Derivative phases & Sufficient exponent\\ | |
| 504 | +\midrule | |
| 505 | +Raw mean + 1 Newton & 1 & 1 & $3/4$\\ | |
| 506 | +Raw mean + 2 Newton & 1 & 2 & $7/8$\\ | |
| 507 | +Half-block jackknife + 1 Newton & 3 & 1 & $7/8$\\ | |
| 508 | +Quarter-block extrapolation + 1 Newton & 7 & 1 & $11/12$\\ | |
| 509 | +\bottomrule | |
| 510 | +\end{tabularx} | |
| 511 | +\end{table} | |
| 512 | + | |
| 513 | +The meaningful systems-level claim is conditional: if an additional full-cluster synchronization has greater cost than two extra local likelihood fits, half-block jackknifing may be favorable; if local fits dominate and network latency is low, extra Newton steps may be faster. The cut-off depends on model dimension, optimizer complexity, scheduling, bandwidth, and numerical conditioning. | |
| 514 | + | |
| 515 | +\subsection{A cost model for an empirical follow-up} | |
| 516 | +Let $T_{\mathrm{fit}}(m,p)$ denote the elapsed time for one local MLE fit, $T_{\mathrm{der}}(m,p)$ for one local derivative evaluation, $L$ the per-phase synchronization latency, and $B$ the effective worker-to-coordinator bandwidth. A first-order approximation is | |
| 517 | +\begin{align} | |
| 518 | + T_{\mathrm{half+N}}&\approx 3T_{\mathrm{fit}}(m,p)+T_{\mathrm{der}}(m,p)+2L+\frac{bK[2p+p(p+1)/2]}{B},\\ | |
| 519 | + T_{\mathrm{raw+2N}}&\approx T_{\mathrm{fit}}(m,p)+2T_{\mathrm{der}}(m,p)+3L+\frac{bK[3p+p(p+1)]}{B}, | |
| 520 | +\end{align} | |
| 521 | +where $b$ is bytes per transmitted scalar and the expression deliberately suppresses overlapping parallelism, coordinator inversion, broadcasts, network contention, and data loading. These are accounting illustrations, \emph{not} validated time-complexity theorems. An actual benchmark must record energy, cold starts, memory use, fit failures, $p$, hardware topology, and communication bytes independently of statistical accuracy. | |
| 522 | + | |
| 523 | +\section{Limitations and research agenda} | |
| 524 | +\subsection{Identification, misspecification, and dependence} | |
| 525 | +The Fisher-information variance in the main theorem presumes a correctly specified regular model. Under misspecification, the asymptotic covariance is generally the sandwich form and the common influence function must be modified. If observations exhibit cross-node dependence or unequal distributions, the independent-block variance calculation in \cref{prop:init} changes. Fixed effects with incidental parameters can have nonstandard bias expansions; indeed, that setting is one of the motivations for \citet{dhaene2015}. Neither validity nor exact exponents should be assumed to transfer without new proofs. | |
| 526 | + | |
| 527 | +\subsection{High-dimensional Hessians and nonconvex likelihoods} | |
| 528 | +Our proof assumes fixed $p$. When $p$ grows with $N$, the norm of the Fisher inverse and the constants in the high-moment expansion can depend on $p$, and the $K$ exponent may weaken. In nonconvex likelihoods, local workers may converge to different stationary points, while a full Newton step can escape the basin in which the quadratic bound holds. Regularization, trust regions and generalized linear-system solvers may mitigate these issues but require separate theory. Stronger generality without control of such effects would overstate the current result. | |
| 529 | + | |
| 530 | +\subsection{Statistical versus computational optimality} | |
| 531 | +We have not proved that the number of communications can be independent of $N$ in a minimax sense, nor that this particular scheme achieves a lower bound over all algorithms. Indeed, exact sufficient-statistic aggregation defeats any supposed general lower bound in the exponential model. The appropriate open question is narrower: \emph{over a specified class of nonlinear likelihoods, and under a model of allowable local computation and communication, what Pareto frontier links local refits, derivative synchronizations, transferred bytes, and statistical precision?} | |
| 532 | + | |
| 533 | +A useful research program would (i) characterize the finite-sample risk of high-order jackknife starts; (ii) derive feasible model-specific analytical bias corrections that use fewer local optimizations; (iii) compare with modern multi-round techniques such as CEASE \citep{fan2023}; (iv) extend the expansions to clustered or time-dependent panels; and (v) implement a wall-clock benchmark across physical nodes, varying the cost ratio of fitting to synchronization. | |
| 534 | + | |
| 535 | +\section{Conclusion} | |
| 536 | +This note develops and verifies a local-bias-correction-plus-global-Newton construction for distributed maximum likelihood estimation. Under explicit uniform mean-square remainders, second-order bias expansions, and local Hessian regularity, a half-block jackknife initializer followed by one global Newton step is first-order equivalent to centralized MLE when $K=o(N^{7/8})$. The stochastic component of the jackknife remainder has been kept throughout the proof. A third-order local extrapolation yields the sufficient exponent $11/12$ under stronger assumptions. Exact Gamma calculations establish the rate and a nonzero boundary shift in the exponential model, while a reproducible Monte Carlo experiment confirms the expected qualitative pattern. | |
| 537 | + | |
| 538 | +The scientific interpretation is appropriately limited. Related distributed one-step estimation and higher-order jackknife ideas are established, and the method has not yet been benchmarked in a real networked environment. The potentially productive direction is the \emph{quantitative exchange between local correction and global synchronization}, rather than an unsupported claim to have surpassed all existing maximum likelihood algorithms. | |
| 539 | + | |
| 540 | +\section*{Statements and reproducibility} | |
| 541 | +\addcontentsline{toc}{section}{Statements and reproducibility} | |
| 542 | +\textbf{Code and data.} The accompanying package contains \texttt{monte\_carlo.py}, the simulation CSV outputs, and figure sources. The simulation requires Python, NumPy, and Matplotlib. All generated observations are synthetic; no proprietary data or user data were used.\\ | |
| 543 | +\textbf{Authorship.} The designation ``GPT-6.1'' is included at the request of the user preparing this draft. It refers to an AI-assistance authorship label, rather than a verified human contributor. Responsibility for scientific validation and compliance with any target journal's authorship policy rests with the submitting researcher.\\ | |
| 544 | +\textbf{Research status.} Unsubmitted methodological working paper. Theoretical claims are explicitly assumption-dependent. Priority and novelty have not been exhaustively established. | |
| 545 | + | |
| 546 | +\begin{thebibliography}{99} | |
| 547 | +\bibitem[Zhang et~al.(2013)Zhang, Duchi, and Wainwright]{zhang2013} | |
| 548 | +Zhang, Y., Duchi, J.~C., and Wainwright, M.~J. (2013). | |
| 549 | +Communication-efficient algorithms for statistical optimization. | |
| 550 | +\emph{Journal of Machine Learning Research}, 14:3321--3363. | |
| 551 | +\url{https://www.jmlr.org/papers/v14/zhang13b.html}. | |
| 552 | + | |
| 553 | +\bibitem[Huang and Huo(2019)]{huang2019} | |
| 554 | +Huang, C. and Huo, X. (2019). | |
| 555 | +A distributed one-step estimator. | |
| 556 | +\emph{Mathematical Programming}, 174(1--2):41--76. | |
| 557 | +\url{https://doi.org/10.1007/s10107-019-01369-0}. | |
| 558 | + | |
| 559 | +\bibitem[Dhaene and Jochmans(2015)]{dhaene2015} | |
| 560 | +Dhaene, G. and Jochmans, K. (2015). | |
| 561 | +Split-panel jackknife estimation of fixed-effect models. | |
| 562 | +\emph{The Review of Economic Studies}, 82(3):991--1030. | |
| 563 | +\url{https://doi.org/10.1093/restud/rdv007}. | |
| 564 | + | |
| 565 | +\bibitem[Fan et~al.(2023)Fan, Guo, and Wang]{fan2023} | |
| 566 | +Fan, J., Guo, Y., and Wang, K. (2023). | |
| 567 | +Communication-efficient accurate statistical estimation. | |
| 568 | +\emph{Journal of the American Statistical Association}, 118(542):1000--1010. | |
| 569 | +\url{https://doi.org/10.1080/01621459.2021.1969238}. | |
| 570 | + | |
| 571 | +\bibitem[Efron(1982)]{efron1982} | |
| 572 | +Efron, B. (1982). \emph{The Jackknife, the Bootstrap and Other Resampling Plans}. | |
| 573 | +SIAM, Philadelphia. | |
| 574 | +\url{https://doi.org/10.1137/1.9781611970319}. | |
| 575 | +\end{thebibliography} | |
| 576 | + | |
| 577 | +\appendix | |
| 578 | +\section{Proof details for the exponential remainder} | |
| 579 | +Let $Y=\theta_0\bar X_n$, so $nY\sim\operatorname{Gamma}(n,\text{rate}=1)$. We have $\widehat\theta_n=\theta_0/Y$ and $R_n=\theta_0(1-Y)^2/Y=\theta_0(Y^{-1}-2+Y)$. The requisite moments for $n>2$ are | |
| 580 | +\begin{equation} | |
| 581 | + \EE Y=1,\quad\EE Y^2=1+1/n,\quad | |
| 582 | + \EE Y^{-1}=\frac n{n-1},\quad | |
| 583 | + \EE Y^{-2}=\frac{n^2}{(n-1)(n-2)}. | |
| 584 | +\end{equation} | |
| 585 | +Therefore | |
| 586 | +\begin{align*} | |
| 587 | + \EE R_n&=\theta_0\left(\frac n{n-1}-2+1\right)=\frac{\theta_0}{n-1},\\ | |
| 588 | + \EE R_n^2&=\theta_0^2\EE(Y^{-2}-4Y^{-1}+6-4Y+Y^2)\\ | |
| 589 | + &=\theta_0^2\left[\frac{n^2}{(n-1)(n-2)}-\frac{4n}{n-1}+3+\frac1n\right]\\ | |
| 590 | + &=\theta_0^2\frac{3n+2}{n(n-1)(n-2)}. | |
| 591 | +\end{align*} | |
| 592 | +This supplies an elementary verification of \cref{eq:expR2}. For nested subsamples, the same $\psi$ is used for each sample, which is why the linear term cancels in the jackknife identity. | |
| 593 | + | |
| 594 | +\section{Exact bias of the quarter-block extrapolation} | |
| 595 | +At full, half and quarter-block sizes, the rate MLE has expectations $\theta_0m/(m-1)$, $\theta_0m/(m-2)$ and $\theta_0m/(m-4)$ respectively. Inserting these values into \cref{eq:jack3} yields | |
| 596 | +\begin{align*} | |
| 597 | + \EE J_k^{[3]}-\theta_0 | |
| 598 | + &=\theta_0\left[\frac{8/3}{m-1}-\frac4{m-2}+\frac{4/3}{m-4}\right]\\ | |
| 599 | + &=\frac{8\theta_0}{(m-1)(m-2)(m-4)}, | |
| 600 | +\end{align*} | |
| 601 | +proving \cref{eq:exactbias3}. The high-order bias coefficients are positive in this particular model, whereas the half-block bias in \cref{eq:exactbias2} is negative. Such alternating signs are one reason direct extrapolation is not necessarily monotone in finite-sample accuracy. | |
| 602 | + | |
| 603 | +\end{document} | |
added
references/initial_pamle/unpacked/references.bib
+34 −0
Large diff (34 lines) is not loaded by default to keep the page fast.Load diff
added
references/initial_pamle/unpacked/simulation_replicates.csv
+10501 −0
Large diff (10,501 lines) is not loaded by default to keep the page fast.Load diff
added
references/initial_pamle/unpacked/simulation_summary.csv
+22 −0
Large diff (22 lines) is not loaded by default to keep the page fast.Load diff
added
research_log.md
+11 −0
Large diff (12 lines) is not loaded by default to keep the page fast.Load diff