\section{Background}
\label{sec:background}

\subsection{Sparse MoE inference}
Let a decoder have layers $l \in \Lset = \{1,\dots,L\}$. In an MoE layer the token representation $h \in \R^{d}$ is routed by
$g = \operatorname{softmax}(W_r h) \in \R^{E}$; the top-$K$ entries of $g$ select experts $S(h) \subset \E=\{1,\dots,E\}$,
$|S(h)| = K$, and the layer output is $\sum_{e \in S(h)} \tilde g_e\, f_e(h)$ where $f_e$ is a gated feed-forward network
(SwiGLU in the models we use) and $\tilde g$ equals $g$ restricted to $S(h)$, renormalised in some architectures. OLMoE
\citep{muennighoff2024olmoe} and Qwen3-MoE \citep{yang2025qwen3} follow this template without a shared expert; DeepSeek-style models add shared experts and
grouped routing, which our registry records but this version does not execute.

During prefill the $T$ prompt tokens are processed together and a layer typically needs most of its experts (with $T=20$ and
$K=8$ the union of selected experts already approaches $E=64$ in OLMoE), so prefill is compute-bound and resembles dense
inference. During decode one token at a time selects $K$ experts per layer; the $L K$ expert weight matrices must be read from
wherever they reside. If every expert is resident in unified memory, the decode step is bound by memory bandwidth and kernel
launch latency \citep{javat2026siliconshowdown}; if some are not, the step is bound by the latency of fetching the missing
ones. The cache hit rate therefore translates almost directly into throughput, which is why expert caching and prediction
dominate the literature of Section~\ref{sec:related}.

\subsection{Apple Silicon as an inference platform}
An Apple Silicon system-on-chip shares one physical memory between CPU, GPU and Neural Engine. For offloading this removes
the PCIe copy but not the fetch: a weight on disk must still be read into memory, and a weight on another machine must cross a
network. The MLX framework exposes lazy arrays evaluated as a graph; inserting an expert into a resident buffer is a slice
assignment that MLX performs in place when the buffer is not otherwise referenced (we measure 0.3~ms for one OLMoE expert in
Section~\ref{sec:implementation}). The \gatherqmm{} primitive multiplies a batch of rows by a batch of quantized matrices selected
by index, which is exactly the operation an MoE layer needs; mlx-lm uses it for all its 50-odd MoE architectures.

Three facts about our cluster matter for every later number. First, all nodes negotiate 1000BASE-T on their Ethernet port
(the switch is gigabit even though the Mac Studios have 10~GbE ports); iperf3 measures 0.945~Gbit/s between any two Studios and
ping 0.5--1.0~ms round trip. Second, no Thunderbolt link is active between any two nodes, so RDMA over Thunderbolt~5 and
MLX's JACCL backend are unavailable until cables are installed. Third, the development laptop is not on the cluster LAN; all
distributed experiments run between LAN nodes with a Mac Studio as coordinator.

\subsection{Baselines}
mlx-lm \citep{hannun2023mlx} is the exact baseline: same weights, same quantization, same kernels. llama.cpp
\citep{llamacpp2026} runs the same base checkpoints through its Metal backend but with its own K-quant formats, so its
speed is reported with a precision flag and a perplexity check rather than as a like-for-like competitor. colibri
\citep{colibri2026} streams experts from disk in C; on macOS its OLMoE engine is CPU-only (its Metal backend covers other
engines), so it serves as a reference for the design space rather than a same-hardware competitor; its cluster mode, which
assigns experts to worker Macs by a deterministic weighted hash, is the closest existing analogue of our static placement.