spb/localvm-research Public License

1---2project: localvm-research3document: research/candidate_ranking4author: Simon-Pierre Boucher5contact: contact@spboucher.ai6created: 2026-08-127status: draft8---910# Candidate Ranking: Phase 4 Deliverable1112Ranks all 24 gaps from `research/research_gaps.md` on the ten charter §7 axes, scores 1–10:13**N** Novelty · **P** Technical plausibility · **Mem** memory reduction · **BW** bandwidth14reduction · **Q** quality retention · **IC** implementation complexity (10 = simplest) ·15**HW** Apple Silicon practicality (Metal/MLX kernel feasibility, unified-memory fit) ·16**PT** post-training compatibility · **Sig** research significance · **Risk** (10 = lowest).17Totals are unweighted sums for orientation only; selection (§3) weights significance, cluster18synergy, and the measured substrate. Contrarian gaps G22–G24 are scored **as experiments**19(cheap falsifiers): their Mem/BW are honest 1s; their value shows in Sig/IC/Risk. Binding20expH facts used throughout: **≤650 MB/token SSD budget at 5 tok/s** (≤1.3 GB/token21theoretical), the **≥256 KiB @ QD≥4–8, 16 KB-aligned fetch contract** (4 KiB QD1 = 67 MB/s,22a 200× cliff), the **shift/mask-cheap decode rule** (i-quant LUT penalty), free compute/I-O23overlap (<5% contention), and the ~35:1 RAM:SSD ratio.2425## 1. Scorecards2627### A. Progressive representations and refinement28#### G01 — ResidualPager (total 69)29- **N 8** — Paging residual planes from NVMe is SOTA §7.3.1's verified-unoccupied intersection on any OS; the encodings themselves (Any-Precision, BitStack, RRQ) are prior art, which caps this below 9.30- **P 6** — Successive-refinement theory and BitDelta's ~1-bit corrections are solid ground, but the load-bearing assumption — refined-set temporal stability — is unmeasured (expB), and BitStack itself warns that reconstruction overhead slows inference even fully resident.31- **Mem 9** — A 2–3-bit base of a 70B model is 17.5–26 GB, resident on the 48 GB target; RAM bounds *base* size, not model size — the strongest direct residency decoupling of any non-contrarian gap.32- **BW 7** — Bytes/token = base RAM reads + fetched residuals; good only if the refined set is stable and reused across tokens (the 1–2% non-overlapped ceiling), with no cross-token amortization of the kind G17 gets.33- **Q 7** — Full quality is recoverable in principle (planes sum to the original), but the practical operating point is whatever residual budget fits under 650 MB/token — quality is hostage to expD's recovery curve.34- **IC 4** — Needs a progressive encoder, block index, demand pager, and refinement application path — substantial new machinery before a single kernel is written.35- **HW 6** — Zero-copy unified memory and free prefetch overlap are genuine Apple-specific advantages, but bitplane decode is gated by G05; the affine-delta fallback is materially weaker.36- **PT 9** — Pure PTQ plus calibration; no retraining anywhere.37- **Sig 9** — Directly instantiates the charter §2 decoupling and criteria B/C/E; if it works it is the project's canonical representation.38- **Risk 4** — Dies cleanly if expD+expB show >20% of residual bytes touched per token with <50% temporal reuse; the 35:1 ratio gives little forgiveness.3940#### G02 — MarginGate (total 77)41- **N 9** — SOTA §7.3.2's narrow-but-real gap after excluding QSpec/BiLD/CALM near-misses: nobody gates *storage-tier* residual fetch on decision stability with a correctness story, and no paper in §5 or §6 optimizes or even reports bytes/token.42- **P 7** — Margins demonstrably carry signal (the entire early-exit line; QuantSpec >90% acceptance; ρ=0.82 learned error features), but the confidently-wrong quadrant (G23) is exactly the unmeasured joint distribution that could void it.43- **Mem 8** — Inherits G01's resident low-bit base and adds only a probe; slightly below G01 because it does not itself define the representation.44- **BW 9** — Bytes/token collapses to base-read + (escalation rate × residual fetch): at the measured 5–15% knife-edge prior, the largest expected per-token byte cut of any per-token scheme in the document.45- **Q 7** — CALM-style calibrated risk or exact rejection variants exist and are proven, but the lossy-gated form ships precisely the errors that slip the gate — quality rides on the AUROC.46- **IC 5** — The gate itself is trivial; calibration machinery, rollback, and integration with a pager that must exist first make it moderate.47- **HW 8** — Prototype needs only MLX quantized casts, logging, and control flow — no new kernels before the pager stage.48- **PT 9** — Calibration-only; a small learned probe is explicitly charter-permitted (§14).49- **Sig 10** — If the gate works, the optimization objective of local inference changes from resident-size to decision-gated bytes/token — this is the thesis of the project stated as a mechanism.50- **Risk 5** — expG is cheap and decisive before any engineering; the failure mode is well-defined (AUROC <0.8 or escalation >40% on any major domain) rather than diffuse.5152#### G03 — DeltaKernel (total 67)53- **N 8** — ΔY = (ΔW)X against cached activations is an unpublished primitive (SOTA §7.3.3): GMRES-IR transposed to transformer inference, never written down.54- **P 7** — Exact for linear ops by linearity; the two known threats (attention nonlinearity, Metal gather cost) are bounded and enumerable rather than open-ended.55- **Mem 4** — Saves no residency itself; the activation cache is tens of MB — this is an enabling multiplier, not a reduction mechanism.56- **BW 7** — Makes refinement cost proportional to residual size instead of layer size; without it every escalation or verification doubles compute and halves the whole family's economics.57- **Q 9** — Exact per linear layer by construction; downstream nonlinearity handling is a mapped recompute (expF), not an approximation of unknown size.58- **IC 5** — One fused Metal kernel plus activation-caching hooks — contained, but real kernel work.59- **HW 5** — Block-sparse gather matvec is exactly the access pattern Apple GPUs punish (Endor / i-quant lesson transposed); may need the G20 CPU escape hatch to survive.60- **PT 10** — A pure runtime primitive; the model is untouched.61- **Sig 7** — An enabling primitive, not a regime; its significance is inherited from G02/G17.62- **Risk 5** — Kill is crisp and cheap (expE_delta: ≥2× over recompute at 10% density, ≤1e-3 relative error), and failure has a named fallback (recompute + G20).6364#### G04 — RDLayout (total 72)65- **N 6** — EBCOT-style Lagrangian truncation is 30-year-old machinery and BitStack is adjacent; the novelty is compiling it against logit-KL distortion and the *measured* SSD contract, which is real but incremental.66- **P 8** — Sensitivity-weighted allocation beats uniform everywhere it has been tried (SqueezeLLM, imatrix, LASER's improve-when-truncated matrices); the optimizer is mature and the charter's compile stage budgets unlimited offline time.67- **Mem 7** — A static-but-nonuniform operating point strictly better than uniform bpw at matched bytes, and it is the substrate every dynamic gap pages from.68- **BW 6** — Compile-time only: bytes at matched quality improve, but every loaded byte is still read each token unless a dynamic gap sits on top.69- **Q 8** — Better-than-uniform at matched bytes is the design objective, and per-block KL attribution errors degrade the gain, not correctness.70- **IC 6** — An offline compiler with no runtime component; the work is bounded and parallelizable.71- **HW 8** — The output *is* the 16 KB-aligned, ≥256 KiB-bundled layout the contract demands; the decode path can stay affine (shift/mask-cheap) if G05 fails.72- **PT 10** — Calibration-set only.73- **Sig 6** — Real but possibly incremental: if imatrix-style mixing already captures most of the nonuniform gain, this is a <15%-relative-KL footnote.74- **Risk 7** — Worst case is a null result at modest cost; nothing downstream breaks since pagers fall back to existing formats.7576#### G05 — MetalTrellis (total 63)77- **N 7** — Zero Metal implementations of bitplane, MSB-sliced, or trellis decode exist (SOTA §7.3.5); the formats themselves are published CUDA work.78- **P 6** — Shift/mask-cheap by construction (passes the decode rule on paper), but bit-transpose gather coalescing and trellis sequentiality vs SIMD-group width are genuine unknowns — which is why it is an experiment.79- **Mem 6** — Unlocks the 2–2.5 bpw frontier on Metal (~8× vs FP16) as a component of G01/G04, never alone.80- **BW 7** — QTIP proves >80% of peak bandwidth is reachable with ~2 instr/weight on CUDA; success here means byte savings finally translate to tokens/s on Apple GPUs.81- **Q 7** — Format-neutral: quality equals the underlying encodings' published quality iff the kernel runs at speed.82- **IC 4** — Three hand-written Metal matvecs with correctness and throughput accounting — the most kernel-intensive micro in the plan.83- **HW 5** — HW practicality is the axis *under test*; the i-quant precedent is explicitly unfavorable and the honest prior is uncertain.84- **PT 10** — Kernels only; nothing touches the model.85- **Sig 7** — A fan-out dependency: gates whether G01/G04 use bitplanes or degrade to affine-delta residuals.86- **Risk 4** — Binary outcome; a loss permanently narrows the representation design space (though the affine fallback exists).8788### B. Sparsity as a paging policy89#### G06 — TealPager (total 51)90- **N 6** — TEAL/CATS/GRIFFIN exist with published numbers; the paging framing and the bytes/token measurement are new, the mechanism is not.91- **P 6** — Thresholds are calibration-only and GRIFFIN's per-sequence granularity is inherently prefetch-friendly, but SwiGLU mask working-set statistics are completely unmeasured (all published stats are ReLU-era).92- **Mem 5** — 40–50% model-wide sparsity is ~2× — rescues a 20–30% RAM deficit, nothing more.93- **BW 5** — The 2× FFN cut erodes further if masks churn: Jaccard(t,t+1) <0.6 makes paging traffic exceed the dense-read baseline.94- **Q 4** — Sirius showed contextual sparsity specifically damages reasoning/GSM8K; the 99%-retention claims are perplexity mirages, and this gap starts from that liability.95- **IC 4** — Metal gather kernel + threshold calibration + paging integration; three subsystems for a 2× ceiling.96- **HW 4** — A 50%-sparse gather matvec on a GPU with no sparse tensor cores may run *slower* than dense — a kill condition entirely independent of quality.97- **PT 8** — Calibration thresholds only, but applicability is limited to SwiGLU dense models with enough safe sparsity.98- **Sig 6** — The only training-free selectivity mechanism for dense stock models — important if true, but with a modest ceiling even then.99- **Risk 3** — Two independent cliffs (reasoning quality; gather kernel), either one fatal.100101#### G07 — WandaTiers (total 68)102- **N 7** — Importance-scores-as-reversible-residency with dense correction is SOTA §7.3.6, unoccupied; every ingredient (Wanda, Endor format, Sirius loop) is published separately.103- **P 7** — Demoted weights are by construction those whose absence changes outputs least, and Sirius bounds the correction bill at ~11% of tokens.104- **Mem 6** — 20–50% demotion runs 1.3–1.5× over-RAM dense models; not a 10× mechanism, but the cheapest real one.105- **BW 6** — Bytes/token = cold-block demand, favorable only if demand is rare rather than diffuse; diffuse demand lands in the 67 MB/s regime and kills it.106- **Q 6** — Global-average scores can miss load-bearing weights ("Accuracy is Not All You Need": 25%+ flips for pruning-type interventions); the correction loop mitigates but does not erase this.107- **IC 7** — The cheapest oversubscription route in the document: no re-encoding, no predictors, no new format — placement plus a dense-readable bitmap.108- **HW 8** — Endor-style dense-readable demoted blocks avoid the gather penalty; serve-miss-as-zero is trivial on unified memory.109- **PT 10** — One-shot post-training statistics; fully reversible.110- **Sig 6** — A working version is a solid systems result and a useful floor, not a new regime.111- **Risk 5** — Single crisp kill (cold-block demand >5% of demoted bytes/token at 35% demotion), measurable in expA-blocked before any runtime exists.112113### C. Cache, replacement, and residency policy114#### G08 — MRURing (total 66)115- **N 6** — MRU-for-cyclic-scans is DBMIN 1985; never applied to the dense-decode weight scan, so the novelty is purely in the (real) application.116- **P 8** — The access pattern is fully known (static layer order), the math is 40 years old, and the 0.025 tok/s MLX-mmap disaster is the measured counterfactual it fixes.117- **Mem 4** — Rescues only slightly-over-budget models: the re-read floor caps useful overflow at ~650 MB (5 tok/s) to ~6.5 GB (0.5 tok/s).118- **BW 6** — Achieves the theoretical floor — re-read exactly the overflow X GB/token — where kernel LRU re-reads everything.119- **Q 10** — Bit-exact; a pure replacement-policy change.120- **IC 5** — The simulator is trivial; the real ring needs explicitly managed wired buffers under the iogpu limit, i.e., engineering against the kernel-panic cliff.121- **HW 6** — macOS gives no UBC eviction control, forcing the wired-arena route; expH confirms the re-read stream itself runs at ~13 GB/s with free overlap.122- **PT 10** — No model change of any kind.123- **Sig 5** — Narrow applicability caps significance, but it is also the enabling floor under G18's sweeps.124- **Risk 6** — The simulator answers most of the question for free; kill is a measured <1.5× gain over kernel-LRU mmap at 10–20% overflow.125126#### G09 — ArcExperts (total 75)127- **N 5** — Every ingredient is published or an open llama.cpp feature request (#20757); composition novelty only, and upstream may merge layout+QD first.128- **P 9** — 2.23× from layout alone is already measured on a Mac (#18758), MoE temporal locality is confirmed, and ARC's superiority on mixed workloads is textbook storage systems.129- **Mem 6** — Runs Qwen3-30B/80B-class MoE on 16–48 GB Macs — capacity via the selectivity the router provides for free.130- **BW 7** — Reads/token already shown to drop 1418→370 from layout; ARC + QD≥8 explicit reads + purgeable-warm demotion stack multiplicatively on that.131- **Q 10** — Exact: misses block briefly, nothing is approximated.132- **IC 6** — Cache + bundle compiler + explicit reads inside an existing runtime; zero new math.133- **HW 9** — Expert bundles (MBs) natively satisfy the ≥256 KiB QD≥8 contract; the zero-copy Metal path ships today.134- **PT 10** — Stock checkpoints, stock router.135- **Sig 5** — High deployment value with a real "engineering, not research" risk if policy headroom over LRU is <5 hit-rate points at every cache size.136- **Risk 8** — Worst case still yields a useful runtime plus a publishable routing-trace study.137138#### G10 — PurgeableWarm (total 57)139- **N 5** — Purgeable memory is a shipped, decade-old macOS/iOS API; pointing it at weights is new but small.140- **P 6** — The semantics fit (a purge is exactly a cache miss), but per-access volatile-state transitions may be unaffordable and the OS purge order is model-blind.141- **Mem 2** — Robustness and coexistence with user apps, not reduction.142- **BW 2** — None directly.143- **Q 9** — Purge → miss → refetch at 13 GB/s; quality untouched, latency jitter only.144- **IC 6** — An MTLHeap wrapper plus pressure testing; contained.145- **HW 7** — A macOS-unique mechanism — one of the few gaps discrete-GPU prior work literally cannot express.146- **PT 10** — Runtime only.147- **Sig 4** — Valuable hygiene for every caching gap (converts panic risk into graceful degradation), not a result by itself.148- **Risk 6** — expJ is a cheap probe; failure just means fixed self-managed budgets.149150#### G11 — NeverBlock (total 55)151- **N 8** — Anti-caching's "never stall on a miss, repair later" has never been formulated as an architectural invariant for neural weights.152- **P 5** — QEP's near-exponential error compounding says a layer-10 miss contaminates the whole downstream token; repair-later may collapse into deferred blocking with extra bookkeeping.153- **Mem 3** — No direct reduction; it buys decoupling of token cadence from storage latency.154- **BW 4** — Converts I/O jitter into bounded quality jitter; average bytes unchanged.155- **Q 5** — Real, bounded quality noise under miss bursts, entirely dependent on expF's layer-tolerance map.156- **IC 4** — Correct deferred-repair bookkeeping across layers and in-flight tokens is subtle, and it presupposes a G01-style representation for graceful miss values.157- **HW 7** — Unified memory makes the substitute value (resident base plane) free to reach; the repair path is G03's kernel.158- **PT 9** — A runtime invariant over an already-permitted representation.159- **Sig 7** — P99, not mean, is the real interactive constraint (PowerInfer-2: +40.9% P99); a definitive answer matters for any shipped runtime.160- **Risk 3** — Plausibly self-killing via error compounding; mercifully, the expF injection test is cheap.161162### D. Prefetch and prediction163#### G12 — HiddenPrefetch (total 57)164- **N 6** — Eliseev–Mazur lookahead and ShadowLLM early probes exist; extending to N-layer-ahead SSD prefetch with wasted-byte penalties is incremental-but-real (Pythia's objective, never for weights).165- **P 6** — One early probe demonstrably shadows all layers, but the blocks that matter for prefetch (non-resident ones) are by definition the distribution's tail, where prediction is hardest.166- **Mem 2** — None; it *spends* RAM on apparatus (PowerInfer-2's predictors cost 2.6 GB at 47B — the cautionary number).167- **BW 6** — Does not reduce bytes; it hides latency — which expH says is fully hideable given ~1–5 ms of lead time.168- **Q 8** — Mispredictions cost bandwidth, never correctness.169- **IC 4** — Trace pipeline + probe training + integration into a pager that does not exist yet.170- **HW 7** — Logistic probes are microseconds on AMX and overlap is measured-free; footprint is the only HW concern.171- **PT 8** — Small trained probes are charter-permitted but do require a training loop and per-model calibration.172- **Sig 6** — The enabler that determines whether G01/G06/G07 meet their overlap assumptions; inherits significance from them.173- **Risk 4** — Recall <70% at ≤2× overfetch on the non-resident subset is a live possibility, and the probe+metadata budget cap (5% of RAM) is tight.174175#### G13 — DomainCache (total 66)176- **N 7** — Session-granular weight working sets are unmeasured and unbuilt; adjacent evidence (domain-correlated experts, LoRA subspaces) is suggestive, not demonstrative.177- **P 5** — The rival hypothesis — token-level syntactic dominance with a thin idiosyncratic tail — is equally plausible a priori; expC exists precisely because nobody knows.178- **Mem 6** — A domain profile bounds the resident set for an entire session if the structure exists.179- **BW 8** — Session-granularity amortizes fetches over thousands of tokens, making the 35:1 ratio irrelevant — the most favorable bytes/token geometry in the document, conditional on locality being real.180- **Q 7** — Cache *warming* is quality-neutral; quality is at stake only in exclusionary (G07-style) use.181- **IC 6** — Offline profiles + session-start warming (seconds of sequential 13 GB/s reads); light runtime.182- **HW 8** — Sequential warm reads are the SSD's best case, and it composes directly with the G08/G09 cache machinery.183- **PT 9** — Profiles are offline statistics over a stock model.184- **Sig 7** — Would establish semantic structure in weight usage — a scientifically interesting result independent of its systems value.185- **Risk 3** — One unmeasured premise carries everything; a within-vs-cross-domain Jaccard delta <0.10 kills it outright.186187### E. Decomposition and sharing188#### G14 — TrunkDelta (total 56)189- **N 7** — Assigning a sharing decomposition to the memory hierarchy (trunk wired, deltas streamed in layer order) is unoccupied; the decompositions themselves are published.190- **P 4** — Every published sharing result needs light retraining, and BitDelta's 1-bit evidence is for finetune-vs-base pairs, not layer-vs-layer within one model — the load-bearing transfer is unproven.191- **Mem 7** — The trunk amortizes across L layers (read L times, stored once); if deltas quantize to 1–2 bits, resident cost collapses.192- **BW 7** — Delta streaming is perfectly sequential in static layer order — the one access pattern the SSD serves at full 13 GB/s with zero prediction machinery.193- **Q 4** — Calibration-only trunk extraction plausibly degrades enough that deltas must carry most of the information, at which point nothing is saved.194- **IC 4** — ALS trunk fitting + delta encoder + streaming runtime; moderate-heavy across offline and online components.195- **HW 7** — Dense small GEMMs plus sign-mask decode are Metal-friendly and shift/mask-cheap.196- **PT 6** — The charter permits calibration passes, but all published quality points used 30–40M-token retraining; the honest score reflects that dependence.197- **Sig 7** — Layer-vs-layer delta compressibility is a real scientific question whose answer is valuable either way.198- **Risk 3** — The expK kill (deltas ≥3 bits/param at a 30% trunk) is quite plausible given prior art's retraining dependence.199200#### G15 — SpectralSplit (total 63)201- **N 6** — CALDERA and the SVD family are directly adjacent; tiering the two terms (resident low-rank, paged residual) is a genuinely new but small step.202- **P 5** — Weights are not globally low-rank (SOTA §2.8 headline); the base may be so poor that residual demand is near-universal per token.203- **Mem 6** — A rank-256 hot path is ~12% of dense bytes at 4096², plus whatever residual tier is resident.204- **BW 6** — Same demand structure as G01 but with a worse-conditioned base and double the kernel launches.205- **Q 5** — The split is exact by construction, but the operating curve is plausibly dominated by G01's precision split at equal bytes — the explicit kill condition.206- **IC 7** — Both components run on existing fast MLX kernels today; no custom Metal work gates a prototype — the cheapest Group A/E prototype to stand up.207- **HW 9** — Dense small GEMMs are the one decomposition Metal already runs at full speed.208- **PT 9** — SVD plus quantization, calibration-only.209- **Sig 6** — Mostly valuable as the controlled head-to-head that decides precision-split vs spectral-split for the progressive family.210- **Risk 4** — The kill likely triggers on the base-alone-KL arm (>1.0 nats), but the experiment is nearly free given the expD ladder.211212#### G16 — SeedSynth (total 62)213- **N 7** — SeedLM exists (Apple-authored, FPGA-validated); the Metal question and the seeds-as-nearly-free-prefetch composition are unexplored.214- **P 5** — LFSR generation is shift/XOR-only, but QTIP's ~2 instr/weight ceiling is demanding, and compile-time coefficient search at 70B scale is heavy.215- **Mem 7** — Seeds + coefficients shrink resident and stored bytes together, ~10× per block against 4-bit.216- **BW 9** — Attacks bytes/token *below* the information-theoretic floor of stored-bits formats — the strongest pure-bandwidth story in the document if the kernel wins.217- **Q 4** — Hard-capped at 4-bit parity with no path to exactness: it can never serve as a verification tier, only as a base.218- **IC 5** — One fused generate+matvec Metal kernel for the probe; the full compiler is heavy but deferrable.219- **HW 6** — Targets the real ALU:BW imbalance at batch-1 decode, but the i-quant precedent shows Apple GPUs punish decode work; genuinely uncertain.220- **PT 9** — Data-free re-encoding of a stock checkpoint.221- **Sig 7** — "Weights need not be stored, only recoverable" is the project's thesis in its purest form; even a negative Metal result is a useful boundary.222- **Risk 3** — Single-kernel kill at <80% of the 4-bit baseline, plus reproduction risk of SeedLM's parity claims on SwiGLU models.223224### F. Scheduling, speculation, and amortization225#### G17 — BlockVerify (total 76)226- **N 8** — SpecExec's economics applied to residual planes with rollback is exactly the "block-deferred verification policy" SOTA §8 lists as open; QuantSpec verifies every block from *resident* 16-bit weights, never from storage.227- **P 7** — The rejection-sampling proof carries over unchanged (same support, base=draft), and 4-bit acceptance priors are measured (>90%); acceptance at the 2–3-bit bases real oversubscription requires is the open coin-flip.228- **Mem 8** — Same resident low-bit base as G01; residuals never resident, read once per accepted block.229- **BW 9** — Divides residual bytes/token by the acceptance length, relaxing the brutal 1–2% per-token ceiling to 10–20% of weight bytes per sweep — and sweeps are sequential-by-layer, the SSD's best pattern.230- **Q 9** — Exact by construction under the standard rejection rule; quality risk lives only in throughput, not in outputs.231- **IC 5** — Drafting loop + rollback + residual-sweep orchestration; mitigated by MLX already shipping speculative verification kernels (ReDrafter) on Metal.232- **HW 7** — Sequential residual sweeps at QD≥8 are contract-native; the open cost is Metal batch-verification width sharing unified-memory bandwidth.233- **PT 9** — Base and residual encodings are PTQ; no training anywhere.234- **Sig 9** — If mean acceptance ≥4–7 at a 3-bit base, this makes 1.5–2× oversubscription interactive *with exact outputs* — criteria B and D simultaneously.235- **Risk 5** — Acceptance at low-bit bases is unmeasured (the G24 exposure), but the expG acceptance-length variant measures it before any engineering; rollback economics degrade it gracefully.236237#### G18 — SpecSweep (total 78)238- **N 6** — A port of SpecExec to unified memory + NVMe; the platform mechanics (zero-copy Metal buffers, no PCIe staging, 13 GB/s overflow sweeps) are new, the idea is not.239- **P 8** — The arithmetic uses only measured numbers (expH ceiling × SpecExec acceptance), and speculation already ships on Metal in both llama.cpp and MLX.240- **Mem 7** — Dense *stock* models beyond RAM with zero model transformation: 20 GB of overflow sweeps in ~1.5 s; ~1.4× oversubscription plausibly interactive.241- **BW 8** — Overflow bytes ÷ ~20 accepted tokens/sweep; the amortization factor is measured elsewhere, not hypothesized.242- **Q 10** — Exact rejection sampling; the target distribution is untouched by construction.243- **IC 7** — No new representation and no new kernels: a draft model, tree verification, and an overflow sweep (plus G08's ring for sweep stability).244- **HW 9** — Everything runs on shipped Metal paths today; the one unknown is RAM-vs-RAM contention between batch verification and the concurrent sweep — the contention expH did not measure.245- **PT 10** — Stock checkpoints, stock draft model.246- **Sig 6** — Doubles as the honest baseline every representation-heavy gap must beat at equal oversubscription; even its negative result is load-bearing for the project.247- **Risk 7** — Acceptance shortfall degrades throughput gracefully rather than killing the mechanism; the kill (accepted length <5, or <2 tok/s compound at 1.4×) still leaves a working exact runtime.248249#### G19 — RouterBias (total 66)250- **N 6** — Trained cache-conditional experts exist; the zero-shot measurement is genuinely missing but conceptually tiny.251- **P 6** — Near-tied gates are common under load-balancing pressure, yet the trained systems exist precisely because zero-shot biasing may not preserve quality.252- **Mem 4** — Raises effective cache capacity on over-budget MoE; no direct reduction.253- **BW 6** — A 10–15-point hit-rate lift multiplies directly into tokens/s where misses dominate token time.254- **Q 4** — Systematically starving rarely-resident experts risks distribution drift over long generations — a failure aggregate PPL won't show but flips will.255- **IC 10** — ~5 lines in mlx-lm plus a λ sweep; the cheapest intervention in the entire document.256- **HW 10** — Pure logit arithmetic; no kernels, no I/O, no formats.257- **PT 10** — A runtime knob on a stock router.258- **Sig 4** — A knob, not a regime: highest value-per-effort in the document, lowest ceiling.259- **Risk 6** — Cheap to test, but the kill (every useful λ costs >2% flips or >1 pt GSM8K) is quite plausible given why the trained variants exist.260261#### G20 — AmxSidecar (total 61)262- **N 6** — Kernel-affinity splitting on unified memory is unexplored precisely because capacity splitting is meaningless there — nobody looked; still an infrastructure idea, not a mechanism.263- **P 7** — P-cores idle during decode, AMX sustains ~100+ GB/s, copies are zero by construction; the unmeasured hole is RAM-bandwidth contention (expH tested SSD-vs-GPU, not CPU-vs-GPU).264- **Mem 2** — None.265- **BW 3** — No byte reduction; it reroutes irregular work (gathers, ΔY, probes) off the GPU's weakest path.266- **Q 9** — Numerically identical work on a different engine.267- **IC 6** — Accelerate GEMV + shared-event sync; the microbench is a day, production scheduling more.268- **HW 8** — Exploits Apple-specific silicon (AMX/Accelerate) that every CUDA-derived design structurally ignores.269- **PT 10** — Pure runtime.270- **Sig 4** — A contingency: the escape hatch for G03/G06/G15 gather-kernel failures, not a result in itself.271- **Risk 6** — expM is cheap; failure merely closes the escape hatch (kill: CPU <50 GB/s under GPU saturation, GPU loses >15%, or sync >50 µs/layer).272273### G. Attention-side working set274#### G21 — KVBudget (total 69)275- **N 4** — SparQ, KIVI, KVQuant are all published and KV quantization on Metal is "solved-enough"; the Metal SparQ port and joint weight+KV accounting are engineering with a thin research veneer.276- **P 9** — All ingredient algorithms are training-free and validated; mlx-lm's quantized-KV plumbing exists to extend.277- **Mem 5** — Frees gigabytes of residency at long context — a direct multiplier on every weight-side cache's budget.278- **BW 6** — Up to 8× attention-transfer cut where context is long; possibly <10% of per-token bytes at ≤32k — the self-mooting risk.279- **Q 9** — Published methods are near-lossless at 3–4-bit KV.280- **IC 7** — An extension of existing quantized-KV paths plus one gather kernel.281- **HW 8** — RAM-resident gathers only — the 35:1 SSD ratio never applies; the Metal gather cost question is the sole concern.282- **PT 10** — A runtime cache format.283- **Sig 3** — Explicitly self-demoting: expN_kvshare (one afternoon) may strike it to Phase-10 engineering; its main role is to protect other gaps' accounting.284- **Risk 8** — Nothing fails; the worst case is "not a priority," established almost for free.285286### H. Contrarian gaps (scored as experiments — cheap falsifiers, not architectures)287#### G22 — WarmEnough (total 71)288- **N 6** — Warm steady-state fault accounting for selective workloads on macOS is unpublished; the mechanism under test (UBC + kernel LRU) is ancient.289- **P 8** — llama.cpp + dtrace + wired ballast yields a decisive number in days; every dramatic prior number (0.025 tok/s, 2.23×) is cold-start or dense-cyclic, so the warm question is genuinely open.290- **Mem 1** — Reduces nothing; it can only *delete engineering* (Group C) from the plan.291- **BW 1** — None; it measures whether the kernel already delivers the bandwidth story.292- **Q 10** — A measurement; quality untouched.293- **IC 8** — Scripts around existing tools; no new code paths.294- **HW 10** — Runs on the shipped stack as-is.295- **PT 10** — Nothing touches any model.296- **Sig 8** — A <10% warm fault-time share at 75% residency demotes ~40% of the document's engineering (G08–G11); >25% validates Group C — enormous information either way.297- **Risk 9** — Cannot fail as an experiment; both outcomes are directly actionable.298299#### G23 — ConfidentlyWrong (total 76)300- **N 8** — P(disagree | margin) for quantized bases — the joint distribution, not the marginals — has never been published (SOTA §6.7), despite a crowded escalation literature built on its assumed shape.301- **P 9** — Pure logging and analysis over the expG run matrix; the measurement itself cannot fail.302- **Mem 1** — None; it gates the gaps that claim reductions.303- **BW 1** — None.304- **Q 10** — A measurement.305- **IC 9** — Shares the expG run with G02, G17, and G24 — near-zero marginal cost.306- **HW 10** — MLX quantized casts plus logging; no kernels.307- **PT 10** — Nothing modified.308- **Sig 9** — Single-handedly decides whether the escalation family is the project's center (AUROC >0.85) or dead (<0.65 at a 3-bit base) — the highest information-per-compute-hour in the plan; the gaps doc orders it first for this reason.309- **Risk 9** — Decisive either way; both outcomes are publishable guidance for the whole cheap-first field.310311#### G24 — BasinCollapse (total 74)312- **N 7** — Agreement-with-full-model and recovery-smoothness-per-residual-byte at 2–3-bit PTQ bases are unreported; the ParetoQ transition it probes is published but measured only as standalone quality, never as draft quality.313- **P 9** — MLX quantization at four bitrates plus reference logging; entirely mechanical on the shared ladder.314- **Mem 1** — None directly; it sets the resident-base floor every progressive gap must respect.315- **BW 1** — None.316- **Q 10** — A measurement.317- **IC 8** — The quantize/refine ladder it needs is shared infrastructure for eight other gaps.318- **HW 10** — Stock MLX.319- **PT 10** — Nothing modified.320- **Sig 9** — Fixes the project's oversubscription ceiling: 3-bit agreement <75% caps the whole progressive family at ~1.5–1.6× and reframes the charter §15 stretch target as unreachable by this route — the single number that stretch goal lives or dies on.321- **Risk 9** — Any outcome is a measured ceiling worth recording (charter §17 explicitly values this).322323## 2. Summary table324325| Gap | Name | N | P | Mem | BW | Q | IC | HW | PT | Sig | Risk | Total |326|-----|------------------|---|---|-----|----|----|----|----|----|-----|------|-------|327| G18 | SpecSweep | 6 | 8 | 7 | 8 | 10 | 7 | 9 | 10 | 6 | 7 | **78** |328| G02 | MarginGate | 9 | 7 | 8 | 9 | 7 | 5 | 8 | 9 | 10 | 5 | **77** |329| G17 | BlockVerify | 8 | 7 | 8 | 9 | 9 | 5 | 7 | 9 | 9 | 5 | **76** |330| G23 | ConfidentlyWrong | 8 | 9 | 1 | 1 | 10 | 9 | 10 | 10 | 9 | 9 | **76** |331| G09 | ArcExperts | 5 | 9 | 6 | 7 | 10 | 6 | 9 | 10 | 5 | 8 | **75** |332| G24 | BasinCollapse | 7 | 9 | 1 | 1 | 10 | 8 | 10 | 10 | 9 | 9 | **74** |333| G04 | RDLayout | 6 | 8 | 7 | 6 | 8 | 6 | 8 | 10 | 6 | 7 | **72** |334| G22 | WarmEnough | 6 | 8 | 1 | 1 | 10 | 8 | 10 | 10 | 8 | 9 | **71** |335| G01 | ResidualPager | 8 | 6 | 9 | 7 | 7 | 4 | 6 | 9 | 9 | 4 | **69** |336| G21 | KVBudget | 4 | 9 | 5 | 6 | 9 | 7 | 8 | 10 | 3 | 8 | **69** |337| G07 | WandaTiers | 7 | 7 | 6 | 6 | 6 | 7 | 8 | 10 | 6 | 5 | **68** |338| G03 | DeltaKernel | 8 | 7 | 4 | 7 | 9 | 5 | 5 | 10 | 7 | 5 | **67** |339| G08 | MRURing | 6 | 8 | 4 | 6 | 10 | 5 | 6 | 10 | 5 | 6 | **66** |340| G13 | DomainCache | 7 | 5 | 6 | 8 | 7 | 6 | 8 | 9 | 7 | 3 | **66** |341| G19 | RouterBias | 6 | 6 | 4 | 6 | 4 | 10 | 10 | 10 | 4 | 6 | **66** |342| G05 | MetalTrellis | 7 | 6 | 6 | 7 | 7 | 4 | 5 | 10 | 7 | 4 | **63** |343| G15 | SpectralSplit | 6 | 5 | 6 | 6 | 5 | 7 | 9 | 9 | 6 | 4 | **63** |344| G16 | SeedSynth | 7 | 5 | 7 | 9 | 4 | 5 | 6 | 9 | 7 | 3 | **62** |345| G20 | AmxSidecar | 6 | 7 | 2 | 3 | 9 | 6 | 8 | 10 | 4 | 6 | **61** |346| G10 | PurgeableWarm | 5 | 6 | 2 | 2 | 9 | 6 | 7 | 10 | 4 | 6 | **57** |347| G12 | HiddenPrefetch | 6 | 6 | 2 | 6 | 8 | 4 | 7 | 8 | 6 | 4 | **57** |348| G14 | TrunkDelta | 7 | 4 | 7 | 7 | 4 | 4 | 7 | 6 | 7 | 3 | **56** |349| G11 | NeverBlock | 8 | 5 | 3 | 4 | 5 | 4 | 7 | 9 | 7 | 3 | **55** |350| G06 | TealPager | 6 | 6 | 5 | 5 | 4 | 4 | 4 | 8 | 6 | 3 | **51** |351352Totals span 51–78. Unweighted totals reward safety: the falsifiers and G18 rank high partly353because they cannot fail badly, while G01 (the project's canonical representation) is dragged354down by IC 4 / Risk 4 — selection below therefore weights Sig, cluster synergy, and the355substrate contract, not raw totals. G21 (69) is unselected as self-demoting pending expN.356357## 3. Selected candidates for prototyping358359Four candidates: three architecture compositions (following the natural clusters named in360`research_gaps.md`) plus one falsifier battery that gates them; all share the build-once361infrastructure listed there.362363### Candidate 1 — Progressive-Residual Runtime (G01 + G02 + G04, with G03 kernel; gated by G05)364**Selection rationale.** The composition the entire SOTA analysis points at: a 2–3-bit base365resident in unified memory (17.5–26 GB for a 70B model on the 48 GB target); residual planes366compiled by G04 into KL-optimal, 16 KB-aligned, ≥256 KiB embedded bundles on NVMe; fetched367only when G02's margin/learned-feature gate says the token decision is not yet stable; applied368via G03's ΔY kernel at cost proportional to residual size. It attacks criteria **B**369(bytes/token ≪ compressed checkpoint: base RAM reads + 5–15% escalation × residual fetch),370**C** (token-dependent fraction of model information), **D** (progressive/conditional371execution), and **E** (a post-training representation with a qualitatively better372storage/RAM/quality tradeoff than fixed quantization), and satisfies the substrate contract373by construction: escalation on 5–15% of tokens fits under 650 MB/token, prefetch overlap is374free, G04/G05 keep decode shift/mask-cheap. Highest-Sig cluster (10 + 9 + 6) despite mid-pack375totals — exactly where totals mislead.376377**Promote / kill (expG, expD, expB, expF, expE variants).**378- PROMOTE: expG AUROC ≥0.8 for margin(+learned features) predicting base-vs-reference disagreement, with escalation ≤40% at 99% effective agreement on every major domain; expD reaching KL ≤0.05 to FP16 touching ≤20% of residual bytes/token; expB temporal reuse of the refined set ≥50%; expE_decode progressive variants ≥70% of affine-baseline throughput; expE_delta ΔY ≥2× over full recompute at 10% density.379- KILL: expG AUROC <0.8 or escalation >40% on any major domain (G02 dies; fall back to Candidate 2's ungated verification); expD >20% residual bytes AND expB reuse <50% (steady-state traffic >650 MB/token — G01 dies at 70B scale); expE_decode both variants <70% (G05 dies; degrade to affine-delta residuals and re-score); G24's 3-bit agreement <75% on chat/code or non-smooth recovery (family capped at ~1.5–1.6× oversubscription — record the ceiling, pivot).380381### Candidate 2 — Amortized Verification Sweeps (G17 + G18, on a G08 MRU floor)382**Selection rationale.** The speculation cluster converts the SSD's one great strength —383sequential 13 GB/s sweeps — into exact over-budget decoding by dividing swept bytes across384accepted token blocks. G18 is buildable today with zero model transformation (stock GGUF/MLX385checkpoints, shipped Metal speculation): a 20 GB overflow sweeps in ~1.5 s, so ~20 accepted386tokens/sweep gives an I/O ceiling near 13 tok/s at 1.4× over-RAM — criterion **A** (a model387larger than unified memory at interactive latency) with exact outputs, plus **F** (speculative388sweep scheduling materially changes out-of-core economics). G17 is the research upgrade: the389same amortization applied to residual planes, relaxing Candidate 1's per-token traffic ceiling390by the acceptance length while keeping rejection-sampling exactness. G18 is also the charter391§8.4-honest baseline Candidate 1 must beat at equal oversubscription — funding it is392epistemically mandatory, and its worst case is still a working exact runtime.393394**Promote / kill (expL_specsweep, expG acceptance variant, expI_cachesim).**395- PROMOTE: expL accepted length ≥5 with a 1B draft against 70B-Q4 at tree width 16–64, and Metal batch verification sustaining ≥2 tok/s compound at 1.4× over-RAM under a concurrent 13 GB/s stream; expG-variant mean accepted block length ≥4 at a 3-bit base on chat/code (G17 economics hold).396- KILL: expL accepted length <5 or verification-under-streaming drops the compound ceiling below 2 tok/s (G18 demoted to baseline-only); expG-variant accepted length <4 at 3-bit base (G17 dies — sweep bytes ÷ 4 exceeds the per-token budget for 1.5× over-RAM models); expI/expO showing the swept overflow set unstable under MRU (<1.5× over kernel-LRU) removes the G08 floor.397398### Candidate 3 — MoE Residency Engine (G09 + G12 + G19, with G10 warm tier and G04-style bundle layout)399**Selection rationale.** The cache/prefetch cluster is the nearest-term deployable win and the400only candidate whose per-token selectivity is *free* (the router computes it): compiled40116 KB-aligned expert bundles (natively meeting the ≥256 KiB QD≥8 contract), an ARC ghost-list402cache, hidden-state prefetch N layers ahead (lead time ≥ fetch latency, which expH makes403sufficient), zero-shot router bias as a cheap multiplier, purgeable-warm demotion for graceful404pressure behavior. Quality is exact, every ingredient has isolated measured evidence (2.23×405from layout alone on a Mac), and it serves criteria **A** (Qwen3-30B/80B-class MoE on 16–48 GB406Macs, interactive) and **F** (cache/prediction/scheduling changing out-of-core economics). Its407known weakness is the "engineering, not research" risk — precisely what its kill thresholds408test, and why it runs on shared infrastructure rather than monopolizing effort.409410**Promote / kill (expB MoE-trace variant, expO_warmsteady, expJ_purgeable, G19 sweep).**411- PROMOTE: ARC beats LRU by ≥5 hit-rate points at some cache size in trace replay (25/50/75% of expert pool); prefetch probe recall ≥70% at ≤2× overfetch for non-resident experts; router bias lifting hit rate ≥10 points at ≤2% flips and ≤1 pt GSM8K.412- KILL: ARC <5 points over LRU at every cache size AND expO warm fault-time share <10% at 75% residency (policy headroom ≈ 0 and the kernel already suffices — demote to llama.cpp-upstream engineering, keep only the layout compiler); expJ transition overhead >10 µs/block/token kills the purgeable tier only; G19's kill (every useful λ costs >2% flips or >1 pt GSM8K) strikes the bias knob only.413414### Candidate 4 — Gate Zero: the contrarian falsifier battery (G23 + G24 + G22, plus the G21 expN probe)415**Selection rationale.** Selected *as experiments*, per the gaps doc's own framing: the416cheapest instruments in the project, and they arbitrate everything above before engineering417begins. G23/G24 ride the single expG run matrix (which also serves G02/G17) and decide whether418the escalation/progressive families are the project's center or dead; G22 (llama.cpp + dtrace,419days) decides whether Group C custom residency is research or redundant; expN (an afternoon)420decides whether KV growth silently invalidates every weight-side budget. Charter §17 makes421concluding "it doesn't work" a first-class deliverable, and the fan-in analysis shows these422four measurements gate the priority or design of seventeen of the twenty-four gaps. If all423contrarians "win," the documented salvage architecture (G18 + G09 + G04: speculative sweeps424over a well-laid-out, kernel-cached, R-D-compiled model) is itself unoccupied territory — even425total premise failure yields Candidates 2 and 3 plus a publishable negative map.426427**Decision thresholds (these ARE the promote/kill evidence for the whole portfolio).**428- G23: AUROC <0.65 (margin+entropy+learned features, 3-bit base) ⇒ G02 and G17's gated forms die; block-level rejection sampling (G17/G18 ungated) survives. AUROC >0.85 ⇒ the escalation family becomes the project's center.429- G24: 3-bit same-top-token <75% on chat/code, or KL plateaus until >50% of residual bytes ⇒ the progressive family is capped at ~1.5–1.6× oversubscription; record the measured ceiling; Candidate 1 is demoted and Candidate 2 promoted to primary.430- G22: warm fault-time share <10% of token time at 75% residency ⇒ G08–G11 demoted to engineering polish (Candidate 3 loses its cache-policy arm, keeps layout+prefetch); >25% ⇒ Group C stands as designed.431- expN: KV share <10% of per-token bytes at ≤32k for 8B–30B models ⇒ strike G21 from the candidate list to Phase-10 engineering; otherwise every candidate's RAM budget is re-derived with KV co-modeled.432433## 4. Preferred candidate434435If one candidate defines the project, it is **Candidate 1, the Progressive-Residual Runtime436(G01+G02+G04+G03)** — the only one that rewrites the relationship charter §20 names as the437ideal outcome. Candidate 2 amortizes reads and Candidate 3 exploits existing routing, but both438leave hardware coupled to model size (the sweep scales with overflow bytes; the expert pool439sets tier sizes). Candidate 1 alone makes required RAM a function of the *base representation*440(a compile-time constant chosen to fit the machine) and per-token cost a function of *decision441uncertainty* (escalation rate × residual fetch) — additional model size lands on NVMe as442embedded residual planes that cost latency only when, and in proportion to how often, the443token decision actually needs them. That is `instantaneous useful working set → hardware444required` in mechanism form, on the one platform (unified memory, zero-copy Metal buffers,44513 GB/s NVMe with free overlap) where its mechanics are measured viable, while keeping the446original pretrained weights as the sole source of intelligence. It is also the portfolio's447riskiest member — hence Gate Zero runs first, Candidate 2 stands as its honest baseline and448fallback, and its kill thresholds are numeric and scheduled before any runtime engineering.449Next (Phase 5/6): build shared infrastructure in the gaps doc's order — expG run matrix first,450then the expA/B/C trace campaign, then the expD/expE ladder — before candidate-specific code.451