1# Decision record (WP33 TITAN-MoE)23Format: D-NNN · date · decision · rationale · alternatives considered · consequences.45## D-001 · 2026-11-01 · Project location and numbering6Decision: the project is UQO Working Paper No. 33, stored at `Recherche-UQO/ia/titan-moe/` (category `ia/`, after WP30/31),7future bare repo `gitsrv:srv/git/wp33_uqo.git`. Inside, the directory structure follows the mandate (`titan/`, `metal/`,8`transport/`, `adapters/`, `tests/`, `benchmarks/`, `research/`, `figures/`, `paper/`, `artifacts/`, `scripts/`).9Rationale: one numbered folder per paper is the series standard (`HARMONISATION.md`); the mandate's `wp33_titan_moe/` name is10honoured as the repository name. Deviation noted in README "Notes".1112## D-002 · 2026-11-01 · Reference MLX backend first; third-party code as shallow git-ignored clones13Decision: Gate B is a pure-MLX reference implementation on top of `mlx.core` ops (`gather_qmm`, `gather_mm`), not a fork of14mlx-lm. Colibri, mlx, mlx-lm and llama.cpp are shallow-cloned under `third_party/` for audit only; hashes are recorded.15Rationale: mandate ("reference MLX backend first; optimized C++/Metal/Swift only where justified"); keeps the repo small.1617## D-003 · 2026-11-01 · First model: OLMoE-1B-7B-0125-Instruct, 4-bit MLX conversion18Decision: `mlx-community/OLMoE-1B-7B-0125-Instruct-4bit` (rev `e2384419`, Apache-2.0, 16 layers, 64 experts, top-8, no shared expert).19Rationale: small (3.9 GB), fully open licence, supported by mlx-lm (`olmoe.py`), llama.cpp (GGUF) and Colibri (OLMoE engine),20so three baselines can be compared on the same architecture; 64 experts × 16 layers gives a non-trivial routing/caching problem.21Alternatives: Qwen1.5-MoE-A2.7B (non-Apache licence), gpt-oss-20b (MXFP4, larger). Medium/large candidates (Qwen3-30B-A3B,22gpt-oss-120b, Qwen3-235B) are deferred to Gate C/F and need an explicit download authorization above 20 GB.2324## D-004 · 2026-11-01 · Unreachable nodes are excluded, not assumed25Decision: nodes that did not answer SSH within 15 s at inventory time are excluded from WP33 experiments until re-probed;26the paper's hardware table only lists machines whose specifications were read live (`sysctl`, `system_profiler`).2728## D-005 · 2026-11-01 · Slot-table execution with `gather_qmm` on stacked slot buffers29Decision: each layer owns a `SlotTable` of `capacity` resident experts in stacked MLX buffers; expert ids are remapped to30slot ids and executed with the same `mx.gather_qmm` kernel mlx-lm uses, so the compute path is identical for resident and31just-loaded experts. Loading = in-place slice assignment (`buf[slot] = tensor`), measured at 0.305 ms for the 9 tensors of32one OLMoE expert into 216 MB buffers (`benchmarks/raw/gateB/design/slot_update_microbench_laptop.txt`), i.e. MLX reuses the33buffer rather than copying it. Prefill rows whose unique-expert union exceeds the capacity are chunked greedily by rows (exact).34Alternatives measured: per-expert `quantized_matmul` loop 0.43 ms/layer vs 0.26 ms for gather_qmm (1 token, top-8);35re-stacking 8 experts per call 0.50 ms/layer. Consequence: the host must read the router's choice (`tolist()`) before each36layer's expert kernels — the unavoidable synchronization of demand loading, shared with Colibri; its cost is measured in Gate C.3738## D-006 · 2026-11-01 · Activation parity39Decision: use mlx-lm's compiled `swiglu` (silu(gate)·up in one fused kernel). Rationale: an unfused fp16 `up·sigmoid(gate)·gate`40gave max |Δlogit| = 0.0195 vs mlx-lm; the fused version gives 0.0 (bit-identical) — see `benchmarks/raw/tests/numerical_*.json`.4142## D-007 · 2026-11-01 · Lazy slot installs, batched demand misses43Decision: `SlotTable.ensure()` reads all missing experts of a layer, installs them as lazy slice assignments and never calls44`mx.eval` itself; the layer's `gather_qmm` consumes the updated buffers in the same graph. Rationale: the first version evaluated45the whole table after every miss (one GPU sync per missed expert): at 16 slots decode went from 28/41 tok/s to 51/73 tok/s46(general/code, held-out) once removed (`benchmarks/raw/gateC/prefetch_online_laptop/smoke*.jsonl`). Per-miss cost on the laptop47with a page-cached file is now 0.21 ms (`benchmarks/raw/gateC/design/miss_cost_breakdown_laptop.txt`: pread 0.13 + numpy/MLX48copy 0.06 + install) against 0.43 ms from a cold SSD and ≈ 28 ms over 1 GbE.4950## D-008 · 2026-11-01 · Medium model download (17.2 GB) under the standing ≤ 20 GB rule51Decision: download `mlx-community/Qwen3-30B-A3B-Instruct-2507-4bit` (rev `e9675aa3`, Apache-2.0, 48 layers × 128 experts, top-8)52on M3U96a (cluster coordinator for Gates E/F), then rsync to worker nodes over the LAN rather than re-downloading. Rationale:53OLMoE (3.6 GB of experts) fits in every node's RAM, so the memory-budget regimes of H1/H3 need a model whose experts exceed54the budget of the smaller nodes (36–64 GB) and whose expert count (6 144) gives a realistic routing problem. gpt-oss-120b (62 GB)55is deferred until explicitly authorized.5657## D-009 · 2026-11-01 · Expert-contiguous storage layout for the SSD tier58Decision: the SSD tier reads from a TITAN blob (`experts.titan`, one 4 KiB-aligned contiguous record per expert, index JSON)59generated once from the checkpoint by `scripts/convert_layout.py`; the source checkpoint is never modified and remains the60source of dense weights. Rationale: measured 9 scattered reads per expert in the HF safetensors (expert span 1.5 GB) → cold6116-slot decode 8–10 tok/s; with parallel reads 18; with the contiguous layout 36 tok/s (4.5× over v1). Colibri performs an62equivalent conversion to its own container; we keep safetensors for everything else so mlx-lm remains the exact reference.6364## D-010 · 2026-11-01 · H1 verdict for the small model and the remote tier as capacity only65Decision: report H1 as *not supported* for OLMoE on the laptop NVMe tier (transition prefetch at depth 2: +0.14 hit rate, −4 to −15 %66throughput; mechanism = install cost on the critical path × 50 % precision) and keep the prefetcher as an instrumented component;67report the remote tier as a capacity mechanism (2.5 ms per layer over 1 GbE vs 1.2 ms per layer of cold-NVMe misses). Both verdicts68are limited to the measured regime (small model, experts fit on local NVMe, 1 GbE) and the paper says so.6970## D-011 · 2026-11-01 · No custom Metal kernels in v0.1 (profiling-driven)71Decision: Gate D stops at profiling plus the fused up+gate option (kept, off by default). Rationale: at batch 1 the decode step is72bound by dispatch/synchronisation latency (gather_qmm takes the same 0.17 ms for 1–16 experts; in-graph ops cost 5–7 µs; the step73moves 452 MB in 2.99 ms = 151 GB/s on an M4 Max), so fusing kernels cannot buy more than a few percent; the measured fusion gives74−0.2 % / +2.6 %. The mandate's rule "profile before optimization; report end-to-end benefit" is applied literally: H4 is reported as75not supported in the measured regime, and token batching is identified as the lever that would change the regime.76