1# Related work and novelty matrix (Gate A, 2026-11-01)23Sources: `research/prior_art.csv` (59 works, each verified on its arXiv / publisher / GitHub page on 2026-11-01 by a4web-research pass; evidence URL per row) and a direct code/doc audit of the four reference repositories shallow-cloned in5`third_party/` (hashes below). Venue fields left empty in the CSV were not verifiable (ACM DL blocked fetches); they are6cited as arXiv preprints until re-verified at Gate G.78| repo | commit | date | what we read |9|---|---|---|---|10| JustVugg/colibri | `0a5504a1dc64f86fcd0fe7db298b0448c730e7e6` | 2026-10-31 | README (How it works, Benchmarks), docs/cluster.md, docs/CACHE_ROUTE.md, docs/metal.md, docs/METAL-M5MAX-PERF-REPORT.md, docs/routing-telemetry.md (titles) |11| ml-explore/mlx | `bd63d3803008edda03ce74080955d9879e2ef1eb` (version.h 0.32.5) | 2026-11-01 | mlx/distributed/{ring,mpi,jaccl,nccl}, docs/src/usage/distributed.rst (JACCL = RDMA over Thunderbolt 5, macOS ≥ 26.2, `rdma_ctl enable`), ops `gather_qmm`/`gather_mm` |12| ml-explore/mlx-lm | `736ab0d8c1d0edc79c3d798044fe6904e2fa23be` | 2026-10-30 | models/switch_layers.py (SwitchGLU → `gather_qmm`, sort when ≥ 64 rows), models/olmoe.py, qwen3_moe.py (PipelineMixin), generate.py (wired limit, async eval), 50+ MoE architectures |13| ggml-org/llama.cpp | `a91eab67d8389d979d29fdf6633b2edd0718eb4b` | 2026-10-31 | common/arg.cpp (`--cpu-moe`, `--n-cpu-moe`, `--override-tensor`, `--rpc`), tools/rpc, ggml-metal backend |1415## 1. What is already known (and therefore not claimed)1617**Expert offloading and SSD streaming.** Per-expert offloading with an LRU cache was established by Eliseev & Mazur (2023) and18MoE-Infinity (Xue et al. 2024); SSD-tier streaming is demonstrated in Colibri (C engine: per-layer LRU + learned pinned hot set19`.coli_usage`, one `pread` per expert, O_DIRECT, loader pool, Metal backend opt-in, 2.24 tok/s GLM-5.2 int4 on an M5 Max 128 GB),20SSD-LLaMA (Liang et al. 2026) and FlashMoE (Kim et al. 2026, learned replacement beating LRU/LFU). Colibri explicitly states the21design principle we adopt: *placement only decides speed, never which experts run*.2223**Expert prediction.** Pre-gated MoE (Hwang et al., ISCA 2024) and Read-ME (Cai et al. 2024) change the architecture to route one24layer ahead; Fate (Fang et al. 2025), ProMoE, ExpertFlow (He et al. 2026), LayerScope, DuoServe, SpecPrefetch and SeqMoE25(2025–2026) learn cross-layer predictors on GPUs/NPUs; Colibri's `PILOT` prefetches the next layer from the router lookahead26(GLM-5.2: 71.6 % predictable one layer ahead, per README). Liang et al. (ICLR 2026) quantify routing locality and warn that27not all models suit offloading. Cache-aware *routing* (Skliar et al. 2025, arXiv 2412.00099; Colibri `CACHE_ROUTE`) changes the28expert choice and is therefore excluded from TITAN-MoE by construction (correctness constraint).2930**Distributed inference on Macs.** MLX ships ring (TCP/Thunderbolt), MPI and JACCL (RDMA over Thunderbolt 5, macOS ≥ 26.2,31Mac Studio); Apple reports 816B-parameter MoE inference across 4 × M5 Ultra (tensor/pipeline sharding of whole layers). exo,32prima.cpp, Petals and llama.cpp RPC shard layers or tensors, not experts. Colibri's cluster mode delegates routed experts to33remote "expert workers" over persistent TCP with a deterministic weighted hash (weights ∝ measured disk speed), batch-union34requests, q8 activations (not token-exact) and timeouts; it has no topology or latency model and no replication.3536**Apple Silicon characterization.** powermetrics-based energy per token for dense models (GreenBench, Silicon Showdown),37MoE on M-series with all experts resident (NPUMoE/ANE, BaseRT Metal, Alfarizy et al.). Silicon Showdown reports MoE on Apple38Silicon is latency-bound rather than bandwidth-bound when resident — a finding our Gate B baseline is consistent with39(OLMoE 4-bit: 494 tok/s decode on M5 Max with 432 MiB of expert weights touched per token, i.e. ≈ 0.2 TB/s effective).4041## 2. Novelty matrix (hypotheses vs prior art)4243| TITAN-MoE element | closest prior art | already shown there? | what remains open (our claim, to be tested) |44|---|---|---|---|45| H1 causal cross-layer/cross-token prefetch on Apple Silicon with an SSD tier | Colibri PILOT (1 layer ahead), Fate, SeqMoE, FlashMoE (GPU/x86) | prediction works on GPUs; Colibri has a single heuristic | head-to-head of no-prefetch / LRU / LFU / transition / learned predictors under identical Metal kernels and SSD, net of predictor overhead, on held-out prompt families |46| H2 topology-aware expert placement on heterogeneous Macs | ExFlow (GPU affinity), OrderMoE (edge servers), Colibri cluster (hash by disk speed) | datacenter GPUs only; Colibri has no topology model | cost-model placement using measured per-node compute, RAM, SSD, link latency/bandwidth; comparison with static/uniform/frequency/capacity placement |47| H3 remote RAM vs local SSD as the second tier | MLX JACCL, exo, llama.cpp RPC (whole shards) ; SSD-LLaMA, Klotski (local SSD) | never compared as *expert cache tiers* | measured tier model (10 GbE / Thunderbolt vs NVMe) and a coordinated 3-tier policy under fixed budgets |48| H4 fused Metal kernels for quantized grouped MoE | MLX `gather_qmm`, llama.cpp `mul_mat_id`, BaseRT, Colibri Metal | strong stock kernels exist | gains only if profiling shows a bottleneck; end-to-end (not kernel-only) comparison with explicit tolerances |49| H5 latency × throughput × energy Pareto with adaptive orchestration | GreenBench, Silicon Showdown (dense, single node) | no energy for offloaded/distributed MoE on Macs | powermetrics-based J/token across budgets, nodes and policies |5051Honest expectations recorded before experiments: on a 48–96 GB node a *small* MoE is fully resident and no cache policy can52beat the all-resident baseline (H1/H3 only matter when the expert set exceeds the budget); distribution over Ethernet adds53≥ 100 µs per layer of RTT, so a 16-layer model would lose ≥ 1.6 ms per token versus 2 ms per token local decode — H2 can only54win at scales where a single node cannot hold the model or where SSD streaming is slower than the link. These are the regimes55the protocol targets (medium/large tiers, memory budgets < 100 %).5657## 3. Baselines and comparability (from the audit)58- mlx-lm: same weights, same kernels → exact comparison (Gate B shows bit-identical logits).59- llama.cpp Metal: different quantization (K-quants) → speed reported with a precision flag + quality check.60- Colibri on macOS: OLMoE engine is CPU int8 (Metal backend covers GLM/Inkling/Kimi engines only) → reference only.61