\section{Results}
\label{sec:results}

Every number below has a raw log in \code{benchmarks/raw/}; the path is given with each result. Results marked
\textsc{pending} have a protocol but no log yet in this version.

\subsection{Correctness}
\label{sec:results-correctness}
On four prompts (general, code, reasoning, French) \titan{} reproduces mlx-lm's logits with $\max|\Delta| = 0.0$ (float16) at
capacities 64, 32, 16 and 8 slots per layer, with identical argmax at every position and identical greedy continuations of
32 tokens at 16 slots (\code{benchmarks/raw/tests/numerical\_*.json}, \code{greedy\_*.json}, \code{pytest\_gateB\_*.txt},
7 tests passed). The registry's byte ranges read with \code{pread} are byte-equal to \code{mx.load} for sampled experts of both the
per-expert layout (OLMoE) and the stacked layout (Qwen3-30B-A3B, on M3U96a). Qwen3-30B-A3B logit parity:
\textsc{pending} (test launched on M3U96a; result to be inserted from its log).

\subsection{All-resident control and demand loading}
\label{sec:results-capacity}
Table~\ref{tab:capacity} summarises the laptop capacity sweep (64-token greedy generations, general and code prompts,
page-cached checkpoint, \code{benchmarks/raw/gateB/capacity\_sweep\_laptop/sweep.jsonl} and
\code{benchmarks/raw/gateC/prefetch\_online\_laptop/smoke\_lazyinstall.jsonl}).

\begin{table}[t]
\centering\small
\caption{OLMoE-1B-7B 4-bit on the M5 Max laptop: decode throughput vs residency (2 repeats after warm-up; ranges over prompts).}
\label{tab:capacity}
\begin{tabular}{llrrr}
\toprule
Backend & Residency & decode \tps{} & hit rate & stall / 64 tokens \\
\midrule
mlx-lm & all & 491--496 & -- & -- \\
\titan{} & 64 slots, preloaded & 489 & 1.00 & 0 \\
\titan{} & 64 slots, not preloaded (sync path) & 201--203 & 1.00 & 0 \\
\titan{} & 32 slots, LRU, demand (v1 installs) & 56--80 & 0.73--0.81 & 0.7--1.0 s \\
\titan{} & 16 slots, LRU, demand (v1 installs) & 36--45 & 0.48--0.60 & 1.7--2.0 s \\
\titan{} & 16 slots, LRU, demand (lazy installs) & 51--73 & 0.48--0.67 & 0.6--0.9 s \\
\titan{} & 16 slots, LRU + transition prefetch ($d{=}1$, $m{=}8$) & 35--56 & 0.63--0.73 & 0.6--1.1 s \\
\bottomrule
\end{tabular}
\end{table}

Two observations. The all-resident control matches mlx-lm within one percent only when every layer's table is complete;
otherwise each layer pays a host synchronisation, which alone costs 2.4$\times$ in throughput on this model and is the floor
of any demand-loading design, including colibri's. And the demand-loading rows are the H1 \emph{control}: at 16 slots the
model does 53 misses per token (Prop.~\ref{prop:capacity}), i.e.\ about 11~ms of fetch per 2~ms of compute.

\subsection{Routing locality}
\label{sec:results-locality}
From the v1 traces (18 prompts, 3\,686 tokens), per-layer LRU hit rates are 0.57 at 16 slots, 0.80 at 32 and 0.93 at 48 (LFU
within 0.01), nearly identical on fit and held-out prompts (\code{benchmarks/processed/gateC/sim\_olmoe\_laptop\_ssd\_v1.json});
a one-layer-ahead transition predictor has recall@8 of 0.50 on held-out prompts. On the v2 traces (65 prompts, 6 families,
\code{benchmarks/processed/gateC/}) the transition predictor keeps almost all of its accuracy when it looks further ahead
(recall@8 0.534 / 0.525 / 0.517 at depth 1 / 2 / 3), and a per-layer logistic probe on the previous layer's router
log-probabilities reaches recall@8 of \textbf{0.761} held-out (0.779 fit) with prefetch precision 0.74--0.83, lifting the simulated
hit rate at 16 slots from 0.58 (LRU) to 0.81 (oracle 0.94). A first version of the probe scored 0.16 because of a label-alignment
bug across traces; the fix is recorded in the log and the buggy rows are excluded from Table~\ref{tab:sim} and Figure~\ref{fig:sim}.
The rank-based previous-token predictor hurts at every capacity (precision 0.18--0.47, it evicts useful experts).

\input{tables/sim}

\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{sim_hit_vs_capacity.pdf}
\caption{Trace-driven simulation on the v2 held-out prompts: per-layer hit rate (left) and modelled stall per token (right)
by slot capacity and predictor. Generated by \code{scripts/make\_figures.py} from \code{benchmarks/processed/gateC/}.}
\label{fig:sim}
\end{figure}

\subsection{Prefetching (H1)}
\label{sec:results-prefetch}
\paragraph{Page-cache tier, one-layer horizon.} The online prefetcher raised the hit rate from 0.48 to 0.63 (general) and from
0.67 to 0.73 (code) but reduced throughput from 51 to 35 and from 73 to 56~\tps{} (Table~\ref{tab:capacity}): the lead time
$d\,\tau_\ell = 0.13$~ms is shorter than the 0.21~ms fetch, so predicted experts arrive late, and the 50\,\% of wrong predictions
add traffic. This is the regime excluded by condition~\eqref{eq:condition}.

\paragraph{Cold-NVMe tier.} The SSD tier itself had to be engineered before the question could be asked: in the published
OLMoE checkpoint the nine tensors of one expert are scattered over a 1.5~GB span of the file (hash order of keys), so each miss
cost nine random reads and 16-slot decoding ran at 8--10~\tps{}; parallel demand reads raised it to 18 and the expert-contiguous
layout of Section~\ref{sec:architecture} (one 4~KiB-aligned record per expert, one \code{pread}) to 36~\tps{}
(\code{benchmarks/raw/gateC/cold\_nvme\_laptop/}). Table~\ref{tab:cold} then compares predictors on that tier with the page cache
purged before every configuration: at 16 slots the depth-2 transition predictor cuts the stall by a third (0.77 vs 1.15~s per 64
tokens) and ties with demand loading (35.0 vs 36.4~\tps{}); every other predictor, including depth 1 and 3 and the rank-based one,
loses; at 32 slots all of them lose to demand loading (41--42 vs 50.6~\tps{}).

\paragraph{Attribution.} Figure~\ref{fig:attribution} and Table~\ref{tab:attr} split the host time per token into the wait for
the GPU graph (which includes the slice writes that install experts), prefetch bookkeeping and late waits, and demand reads.
The probe raises the hit rate to 0.78 and cuts demand reads from 19.9 to 12.8~ms per token, but the GPU wait grows from 14.7
to 19.7~ms and late waits add 3~ms: each installed expert costs nine lazy slice writes (about 0.13~ms of graph work on the
critical path), and the prefetcher installs more experts than it saves reads for. Confidence thresholds and deeper horizons
change the mix, not the sum. \textbf{H1 is therefore not supported on this model and tier}: the best predictor ties with no
prefetch. The attribution is positive knowledge: the bottleneck is the install path, not prediction accuracy (the probe is
already within 0.16 of the oracle's hit rate at 16 slots), so a single-copy install (one record per expert written in one
operation) is the change that would make the probe's 7~ms of saved reads count, and it is the first item of version~0.2.

\input{tables/attr}

\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{attribution.pdf}
\caption{Host time per token on the cold-NVMe tier (OLMoE, 16 slots, held-out general prompts): GPU wait including installs,
prefetch bookkeeping and late waits, demand reads. Labels give the median decode throughput and hit rate.}
\label{fig:attribution}
\end{figure}

\input{tables/cold}

\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{capacity_sweep.pdf}
\caption{Decode throughput of every single-node configuration measured on the laptop (median over repeats, bar = range):
mlx-lm, \titan{} all-resident with and without the synchronisation-free path, bounded residency on the page-cache tier, and the
cold-NVMe tier with and without prefetch. Source files are named in brackets.}
\label{fig:capacity}
\end{figure}

\subsection{Medium model (Qwen3-30B-A3B)}
On M3U96a (M3 Ultra, 96~GB) the stacked-layout registry reads byte-equal slices, logits are bit-identical to mlx-lm at 128, 64,
32 and 16 slots per layer and greedy decoding is token-identical at 32 slots
(\code{benchmarks/raw/tests/m3u96a/}). Table~\ref{tab:qwen3} gives the baselines: mlx-lm 121.4~\tps{} and \titan{} all-resident
121.7~\tps{} (medians of 3 repeats), i.e.\ parity on a model whose decode step touches 0.97~GiB of expert weights.

\input{tables/qwen3}

\subsection{Two-node execution over gigabit Ethernet (H2, H3: first evidence)}
\label{sec:results-twonode}
With M3U96a as coordinator and M3U96b as worker (OLMoE, 32 slots, page-cached checkpoint on both, held-out general and code
prompts, \code{benchmarks/raw/gateE/two\_node\_m3u96a\_m3u96b.jsonl}), executing the odd-numbered experts of every layer on the
worker keeps the logits bit-identical to mlx-lm but reduces decode throughput from 70 / 87~\tps{} (local control) to
15 / 15~\tps{} (Figure~\ref{fig:twonode}). The remote wait is 2.5~s per 64 tokens, i.e.\ about 2.5~ms per layer: a 0.5--0.65~ms TCP
round trip plus the worker's array creation, kernel dispatch, evaluation and serialisation, paid 16 times per token. This is
larger than a cold-NVMe demand miss at 16 slots (about 1.2~ms per layer) and twenty times the local compute per layer. On this
topology the remote tier is therefore a capacity mechanism, not a latency one, exactly as Section~\ref{sec:prefetch-condition}
anticipates; H2 can only be tested in the regime where a node cannot hold the experts it needs locally.

\begin{figure}[t]
\centering
\includegraphics[width=0.6\textwidth]{two_node.pdf}
\caption{Two Mac Studios over 1~GbE: local bounded residency vs remote execution of half the experts (medians).}
\label{fig:twonode}
\end{figure}

The round trip was measured directly (\code{benchmarks/raw/gateE/remote\_cost\_breakdown\_m3u96a\_m3u96b.txt}): application-level
ping 0.74--1.17~ms; one layer's request (one row, eight pairs) 2.59~ms; eight rows with 64 pairs 7.3~ms; 32 rows with 256 pairs
19.1~ms, of which about 9~ms is the 1~MB reply on the wire. The worker therefore adds 1.2--1.7~ms per call to the network
latency, which bounds what a faster transport could recover without also changing the worker's execution path.

\subsection{Placement on the medium model (Gate F, H2)}
\label{sec:results-gateF}
The same two Studios ran Qwen3-30B-A3B with the coordinator's residency bounded to 32 or 64 of the 128 experts per layer and its
page cache purged before every configuration, so that a locally executed miss is a cold-NVMe read (\code{benchmarks/raw/gateF/},
16 runs per cell, 64 tokens, held-out general and code prompts, logits bit-identical to mlx-lm in every cell). Three placements were
compared: everything local, the odd-numbered half of every layer's experts on the worker (static), and the least popular half on the
worker with popularity estimated from the fit traces (the frequency-based placement of Algorithm~\ref{alg:placement} with one
worker). Table~\ref{tab:gateF} and Figure~\ref{fig:gateF} give the result: local demand loading decodes at 8.8~\tps{} with 32 slots
(66~ms of stall per token) and 16.3 with 64; frequency-based placement beats static placement at both budgets (5.1 vs 4.2 and 6.0 vs
5.2~\tps{}, 35\,\% fewer remote pairs, remote wait 67 vs 112~ms per token), but both lose to the local cold tier ($-3.6$ [$-4.4$, $-2.9$]
and $-10.2$ [$-13.6$, $-8.8$]~\tps{}, Table~\ref{tab:audit}). The ordering is the one the cost model predicts from the measured constants:
a remote layer costs 1.5--3~ms over gigabit Ethernet, a cold miss on this NVMe about 0.9~ms per expert in parallel, and the medium model has
48 layers. \textbf{H2 is supported in its weak form} (cost-aware placement beats static placement) and \textbf{not in its strong form}
(remote execution does not beat local NVMe) on this topology.

\input{tables/gateF}

\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{gateF_qwen3.pdf}
\caption{Qwen3-30B-A3B on two Mac Studios over 1~GbE: decode throughput (left) and per-token waiting (right: local stall, light;
remote wait, dark) for three placements at two local budgets.}
\label{fig:gateF}
\end{figure}

\subsection{Metal kernels (H4)}
\label{sec:results-metal}
Profiling on M4M64a (\code{benchmarks/raw/gateD/}) shows that a decode step with all experts resident takes 2.99~ms
(334~\tps{} synchronous) and moves 452~MB of expert weights, i.e.\ 151~GB/s or about 28\,\% of the chip's nominal bandwidth;
a \gatherqmm{} over 1 to 16 experts takes the same 0.17~ms (latency-bound; 250~GB/s only at 64 experts), and in-graph
elementwise ops cost 5--7~$\mu$s each against a 0.16~ms synchronisation floor. The step is launch-bound, so the most a kernel
fusion can save is a few dispatches per layer. We tested the cheapest one, a single \gatherqmm{} over a stacked up+gate buffer
(bit-identical logits, \code{tests/test\_fused\_upgate.py}): $-0.2\,\%$ with all experts resident and $+2.6\,\%$ on the cold tier,
both within the run-to-run range (Table~\ref{tab:fused}). \textbf{H4 is not supported} at batch size one on this model; the profile
says that batching tokens, not kernels, is what would change the regime.

\input{tables/fused}

\subsection{Energy per token (H5, single node)}
\label{sec:results-energy}
Table~\ref{tab:energy} reports package power sampled at 1~s by \code{powermetrics} while four configurations decoded 256 tokens on
the laptop (\code{benchmarks/raw/gateG/energy\_laptop/}). All-resident decoding costs 0.101~J per token at 47~W whether mlx-lm or
\titan{} runs it; bounded residency on the cold tier costs five times more per token in gross terms (0.50~J at 16~W) because the
machine waits on the disk most of the time; the probe prefetcher brings this to 0.44~J at 18~W. Net of the 13.2~W idle power the
ordering reverses between the two cold configurations (0.086 vs 0.119~J), which is the usual ambiguity of attributing idle power.
The same run also gives the first \emph{positive} H1 evidence: over 256-token generations the probe reaches a hit rate of 0.84 and
42.0~\tps{} against 31.9 for demand loading ($+10.1$ [$+5.2$, $+16.9$]~\tps{}, Table~\ref{tab:audit}), whereas the 64-token runs
of Section~\ref{sec:results-prefetch} showed a tie: the per-token install overhead is constant while the benefit of a warm, well-predicted
table grows with sequence length. H1 is therefore \emph{length-dependent} on this hardware, and the cluster-level Pareto study remains
pending.

\input{tables/energy}