1\section{Background}2\label{sec:background}34\subsection{Sparse MoE inference}5Let a decoder have layers $l \in \Lset = \{1,\dots,L\}$. In an MoE layer the token representation $h \in \R^{d}$ is routed by6$g = \operatorname{softmax}(W_r h) \in \R^{E}$; the top-$K$ entries of $g$ select experts $S(h) \subset \E=\{1,\dots,E\}$,7$|S(h)| = K$, and the layer output is $\sum_{e \in S(h)} \tilde g_e\, f_e(h)$ where $f_e$ is a gated feed-forward network8(SwiGLU in the models we use) and $\tilde g$ equals $g$ restricted to $S(h)$, renormalised in some architectures. OLMoE9\citep{muennighoff2024olmoe} and Qwen3-MoE \citep{yang2025qwen3} follow this template without a shared expert; DeepSeek-style models add shared experts and10grouped routing, which our registry records but this version does not execute.1112During prefill the $T$ prompt tokens are processed together and a layer typically needs most of its experts (with $T=20$ and13$K=8$ the union of selected experts already approaches $E=64$ in OLMoE), so prefill is compute-bound and resembles dense14inference. During decode one token at a time selects $K$ experts per layer; the $L K$ expert weight matrices must be read from15wherever they reside. If every expert is resident in unified memory, the decode step is bound by memory bandwidth and kernel16launch latency \citep{javat2026siliconshowdown}; if some are not, the step is bound by the latency of fetching the missing17ones. The cache hit rate therefore translates almost directly into throughput, which is why expert caching and prediction18dominate the literature of Section~\ref{sec:related}.1920\subsection{Apple Silicon as an inference platform}21An Apple Silicon system-on-chip shares one physical memory between CPU, GPU and Neural Engine. For offloading this removes22the PCIe copy but not the fetch: a weight on disk must still be read into memory, and a weight on another machine must cross a23network. The MLX framework exposes lazy arrays evaluated as a graph; inserting an expert into a resident buffer is a slice24assignment that MLX performs in place when the buffer is not otherwise referenced (we measure 0.3~ms for one OLMoE expert in25Section~\ref{sec:implementation}). The \gatherqmm{} primitive multiplies a batch of rows by a batch of quantized matrices selected26by index, which is exactly the operation an MoE layer needs; mlx-lm uses it for all its 50-odd MoE architectures.2728Three facts about our cluster matter for every later number. First, all nodes negotiate 1000BASE-T on their Ethernet port29(the switch is gigabit even though the Mac Studios have 10~GbE ports); iperf3 measures 0.945~Gbit/s between any two Studios and30ping 0.5--1.0~ms round trip. Second, no Thunderbolt link is active between any two nodes, so RDMA over Thunderbolt~5 and31MLX's JACCL backend are unavailable until cables are installed. Third, the development laptop is not on the cluster LAN; all32distributed experiments run between LAN nodes with a Mac Studio as coordinator.3334\subsection{Baselines}35mlx-lm \citep{hannun2023mlx} is the exact baseline: same weights, same quantization, same kernels. llama.cpp36\citep{llamacpp2026} runs the same base checkpoints through its Metal backend but with its own K-quant formats, so its37speed is reported with a precision flag and a perplexity check rather than as a like-for-like competitor. colibri38\citep{colibri2026} streams experts from disk in C; on macOS its OLMoE engine is CPU-only (its Metal backend covers other39engines), so it serves as a reference for the design space rather than a same-hardware competitor; its cluster mode, which40assigns experts to worker Macs by a deterministic weighted hash, is the closest existing analogue of our static placement.41