SPB Git

spb/wp28_uqo

Public
11.2 KB · 91 lines markdown
1# PHASE_STATUS — WP2823Phase: 0 — Repository reconnaissance and reproducibility lock4Start/finish timestamp: 2026-10-28 18:33 → 18:50 (America/Toronto)5What was actually done: inspected `Recherche-UQO` (README, HARMONISATION, template, WP 25/27 conventions); located the prior draft in6`~/Downloads/WP28_UQO_Claude_Research_Brief/`; froze it with SHA256 under `references/initial_pamle/`; read the 16-page PDF and the7source archive in full; re-ran `monte_carlo.py` at seed 20261028 (identical outputs to ≤ 5e-13); compiled `pamle.tex` (16 pp.);8inventoried the live MacLustr cluster (22/29 nodes reachable; see `docs/cluster_inventory_2026-10-28.md`); created the WP28 workspace9from the UQO template.10Artifacts created: PROJECT_MAP.md, SOURCE_PROVENANCE.md, PHASE_STATUS.md, research_log.md, docs/cluster_inventory_2026-10-28.md,11references/initial_pamle/*, paper/{Makefile,.latexmkrc,uq_logo.jpg,sections/titlepage*.tex}, .gitignore.12Tests passed: replication of the prior Monte Carlo (summary + 10,500 replicate rows).13Tests failed: none.14Unproved statements or unresolved citations: all theorems of the prior draft are tagged "to re-derive" until Phase 2; the 5 prior15references are being verified in Phase 1.16Decision and justification: keep identifier WP28 (free in the series; collision note recorded). Proceed to Phase 1.17Next phase action: literature/priority audit (four parallel verification tasks launched 18:47).1819Phase: 1 — Exhaustive state-of-the-art / priority investigation20Start/finish timestamp: 2026-10-28 18:47 → 19:1221What was actually done: four parallel verification tasks read 24 papers in full text (arXiv/accepted manuscripts where publisher pages were bot-blocked) and verified 56 bibliographic records via Crossref; theorem-level comparisons recorded in literature/audit_A..D.md; PRIOR_ART_MATRIX.csv (56 rows); PRIORITY_AND_GAP.md with claim-by-claim map, candidates A/B/C and GO/REVISE/DROP decisions; two wrong DOIs in the brief corrected.22Artifacts created: literature/audit_A_*.md, audit_B_*.md, audit_C_*.md, audit_D_*.md, PRIOR_ART_MATRIX.csv, PRIORITY_AND_GAP.md.23Tests passed: n/a. Tests failed: n/a.24Unproved statements or unresolved citations: texts behind paywalls (Quenouille 1949/1956, Schucany et al. 1971, Sharot 1976, Rilstone et al. 1996, Le Cam 1956, Bickel 1975 technical content, Cordeiro–McCullagh original notation, published JASA numbering of JLY/CEASE/FONE, Springer text of Huang–Huo, Yin's 2022 thesis) — flagged in PRIORITY_AND_GAP.md §5.25Decision and justification: GO for the joint (r,q) frontier, sharpness, inexact-curvature/quantization theorem; DROP novelty claims on the components; REVISE unequal sizes/misspecification to extensions; cost model as conditional proposition.26Next phase action: mathematical specification (Phase 2).2728Phase: 2 — Mathematical specification and attack on the assumptions29Start/finish timestamp: 2026-10-28 18:55 → 19:4030What was actually done: assumptions A1–A5 written (theory/ASSUMPTIONS.md); every identity of the prior draft re-derived symbolically (theory/symbolic_checks.py S1–S8: weights, Gamma moments, exact biases, Newton identity, q-fold identity and boundary constants, exact Var(J2) via the Beta decomposition, Newton basin, amplification constants, exponent table); sign error found and fixed in the analytic Cox–Snell correction implementation (b1 = +½ I^{-1} T; checked b1 = θ0 in the exponential model).31Artifacts created: theory/ASSUMPTIONS.md, theory/THEOREM_AUDIT.md (proof ledger), theory/symbolic_checks.py.32Tests passed: all symbolic checks; 29 unit tests. Tests failed: none.33Unproved statements: adaptive order selection (H1) left open by design.34Decision: proceed with the general r-th order family and the (r,q) theorem.3536Phase: 3 — General high-order bias-correction family37Start/finish timestamp: 2026-10-28 19:00 → 19:4538What was actually done: Vandermonde weights for arbitrary patterns computed in exact rational arithmetic (src/pamle/jackknife.py), constants β_r and A_r derived, Lemmas 1–5 proved (paper App. A), conditioning analysis of 11 patterns (results/tables/weights_analysis.csv: κ(V) from 6.9 for (1,2) to 4.8e5 for (1,2,4,8,16); A_r = 4, 8, 80/7 for the doubling pattern; (1,2,3) has smaller |β_3| = 6 but larger A_3 = 12).39Artifacts: src/pamle/jackknife.py, experiments/run_weights_analysis.py, results/tables/weights_analysis.csv, paper/sections/construction.tex.40Tests passed: tests/test_jackknife_weights.py. Decision: order selection from the data stated as open (no theorem).4142Phase: 4 — Statistical–synchronization frontier43Start/finish timestamp: 2026-10-28 19:30 → 20:3044What was actually done: Theorem 1 (joint (r,q) frontier) proved with the stochastic remainder tracked (App. A); Theorem 3 (inexact curvature: stale averaged local Hessians ⇒ 1−1/(2(r+q)); single-machine ⇒ 1−1/(2r+q), recovering CEASE's (t+1)/(t+2) and JLY's 2/3; relative quantization ⇒ bit requirement (½ − r(1−α)) log2 N) proved; Proposition 4 (unequal sizes with weights m_k/N, equal-weight counterexample; misspecification) proved; Proposition 5 (cost model) stated as conditional.45Artifacts: paper/sections/results.tex, inexact.tex, costmodel.tex, robustness.tex, appendices/proofs.tex.46Tests: exponent formulas tested (tests/test_communication_accounting.py). Unresolved: Theorem 3(a) needs the fourth-moment block condition (A4) — stated explicitly.4748Phase: 5 — Exact analytic models and counterexamples49Start/finish timestamp: 2026-10-28 19:00 → 20:4050What was actually done: Theorem 2 (exact exponential identities (i)–(iv), boundary limit (v) for all (r,q), necessity (vi)) proved (App. B) and checked symbolically and numerically; failure gallery (Newton basin (0, 2θ̂_N), negative pilots, sign alternation β_2 < 0 < β_3); logistic and Poisson likelihoods implemented with batched Newton, separation safeguard, finite-difference-checked derivatives, analytic Cox–Snell baseline; boundary study run (results/processed/boundary_summary*.csv; finite-m prediction matches Monte Carlo means to 3 digits; the (3,1) boundary regime m ≍ N^{1/24} is unreachable).51Artifacts: paper/sections/exactmodel.tex, appendices/exponential.tex, experiments/run_boundary.py, experiments/boundary_pred.py.52Tests passed: tests/test_math_identities.py (6 tests). Decision: exponential model used as a laboratory only; no computational lower-bound claim.5354Phase: 6 — Research-grade benchmark library55Start/finish timestamp: 2026-10-28 18:58 → 21:0056What was actually done: package src/pamle (jackknife, partitioning, likelihoods, algorithms incl. CSL/AEE/Cox–Snell/stale/single/quantized Newton, communication accounting, diagnostics, TCP protocol + worker); 30 tests incl. reproduction of the prior draft's Monte Carlo summary to 1e-9 relative and an end-to-end protocol test with worker subprocesses; np.linalg.inv is used only for the diagnostic Cox–Snell bias (p×p), never in a Newton step.57Tests passed: 30/30 (laptop, Python 3.14); 29/29 on a Python 3.9 node (the 30th needs the frozen references directory).58Decision: Gate 6 passed; large experiments launched.5960Phase: 7 — Statistical Monte Carlo61Start/finish timestamp: 2026-10-28 19:20 → 2026-10-29 01:30 (cluster + laptop)62What was actually done: 40 exponential cells (2000 reps; 500 at N = 2^22, m = 8 dropped), 136 logistic/Poisson/robustness cells on 18 cluster nodes (1000 reps; 500 for p = 20 and ill-conditioned designs), boundary study (2000/1000/400 reps), ablations (1000 exponential, 400 logistic; one cell at 300), unequal-size experiment (1000/300 reps); failure-affected cells (32) rerun with valid-worker-only variants (laptop for N ≤ 2^18, cluster at 500 reps for N = 2^20); processing and tables regenerated from raw CSVs; pseudo-true target used for misspecified designs.63Artifacts created: results/raw/*.csv + manifests, results/processed/{summary,boundary_summary*,ablation_*_summary,skew_summary}.csv, results/figures/*.pdf, paper/tables/*.tex, experiments/EXPERIMENT_MANIFEST.csv.64Tests passed: 30/30. Tests failed: none.65Unproved statements: none new. Observation recorded: once the bias is cancelled, the one-step gap decays like (m√N)⁻¹ at fixed N (squared centered remainder), so empirical slopes are shallower than the bias exponents.66Decision: report RMS and median gaps, failure fractions and valid-worker-only rows; no replication discarded.6768Phase: 8 — Real communication and compute benchmarks69Start/finish timestamp: 2026-10-29 00:05 → 01:1070What was actually done: 23 designs (75 runs) of the TCP protocol on MacLustr (supplementary p = 32 and K = 256 designs added 2026-10-29 15:12 at the author's request; K = 256 needed a raised file-descriptor limit) (coordinator M4M64a; workers on up to 16 distinct hosts; K = 16, 64, 128; p = 8 and 64; exact/stale/single curvature; (r,q) ∈ {(4,0),(3,1),(2,1),(1,2)}); per-stage wall-clock, worker times, latency percentiles, bytes logged (results/raw/benchmarks.jsonl). Fixes along the way: workers must stay children of the ssh session; JSON serialization of NumPy types; M4M64b blocked by macOS local-network privacy → coordinator host used as 16th worker host.71Artifacts: results/raw/benchmarks.jsonl, results/processed/benchmarks_summary.csv, paper/tables/bench.tex, results/figures/fig_bench_exchange.pdf.72Tests passed: tests/test_protocol.py (laptop and M3U96a).73Decision: §10 reports measured quantities only; exchange rate favours rounds on this LAN; no general speed claim.7475Phase: 9 — Stronger directions / selection76Primary theorem claim: joint (r,q) frontier K = o(N^{1−1/(2^{q+1}r)}) with sharpness in the exponential model and the inexact-curvature frontier (stale 1−1/(2(r+q)); single 1−1/(2r+q); bits (½−r(1−α))log₂N).77Nearest existing theorem(s): Huang–Huo 2019 Thm 3.4 (k = O(√N), plain average + 1 step); Zhu–Li–Wang 2021 Thm 2 (3/4 with local steps); CEASE remark after Thm 4.1 ((t+1)/(t+2), single-machine curvature); JLY Thm 3.1 (m^{-1/2} factor); Gu–Chen 2023 / Yin 2022 (2/3 with analytic debiasing).78Precise difference: extrapolated local start of any order r combined with q aggregated-curvature rounds; exponent depends on 2^q r; sharpness for a corrected construction; inexact-curvature cases expressed as exponents with bit counts.79Complete proof available: Y (App. A–B; algebraic core machine-checked in Lean).80Counterexample search completed: Y (Theorem 2(iv)–(vi); exact pooling; equal-weight unequal blocks; higher-order failure at small m).81Numerical supporting evidence available: Y (Monte Carlo, boundary study, ablations).82Communication benefit established: empirical only (measured on one LAN; rounds cheaper than local levels there).83Retain, weaken, replace, or drop: RETAIN theorems; WEAKEN wall-clock claims to measurements; DROP adaptive-order theorem (open).8485Phase: 10 — Manuscript86Start/finish: 2026-10-28 20:30 → 2026-10-29 01:40. 53-page UQO working paper (12pt letter) with abstract, provenance note, 12 sections, bibliography (40 verified entries), 4 appendices; all tables/figures generated from results/; numbers quoted in prose are LaTeX macros produced by experiments/make_tables.py.8788Phase: 11 — Adversarial review: FINAL_REVIEW.md (referees A, B, C; verdict: ready for human review with 7 ranked issues).8990Phase: 12 — Final build and delivery: scripts/compile_paper.sh + scripts/validate_pdf.sh + scripts/check_lean.sh; FINAL_DELIVERY.md; push to spbgit wp28_uqo.git.91
Files · main
Menu

Theme