SPB Git

spb/wp33_uqo

Public MIT
4.2 KB · 67 lines latex
1\section{Results}2\label{sec:results}34Every number below has a raw log in \code{benchmarks/raw/}; the path is given with each result. Results marked5\textsc{pending} have a protocol but no log yet in this version.67\subsection{Correctness}8\label{sec:results-correctness}9On four prompts (general, code, reasoning, French) \titan{} reproduces mlx-lm's logits with $\max|\Delta| = 0.0$ (float16) at10capacities 64, 32, 16 and 8 slots per layer, with identical argmax at every position and identical greedy continuations of1132 tokens at 16 slots (\code{benchmarks/raw/tests/numerical\_*.json}, \code{greedy\_*.json}, \code{pytest\_gateB\_*.txt},127 tests passed). The registry's byte ranges read with \code{pread} are byte-equal to \code{mx.load} for sampled experts of both the13per-expert layout (OLMoE) and the stacked layout (Qwen3-30B-A3B, on M3U96a). Qwen3-30B-A3B logit parity:14\textsc{pending} (test launched on M3U96a; result to be inserted from its log).1516\subsection{All-resident control and demand loading}17\label{sec:results-capacity}18Table~\ref{tab:capacity} summarises the laptop capacity sweep (64-token greedy generations, general and code prompts,19page-cached checkpoint, \code{benchmarks/raw/gateB/capacity\_sweep\_laptop/sweep.jsonl} and20\code{benchmarks/raw/gateC/prefetch\_online\_laptop/smoke\_lazyinstall.jsonl}).2122\begin{table}[t]23\centering\small24\caption{OLMoE-1B-7B 4-bit on the M5 Max laptop: decode throughput vs residency (2 repeats after warm-up; ranges over prompts).}25\label{tab:capacity}26\begin{tabular}{llrrr}27\toprule28Backend & Residency & decode \toks{} & hit rate & stall / 64 tokens \\29\midrule30mlx-lm & all & 491--496 & -- & -- \\31\titan{} & 64 slots, preloaded & 489 & 1.00 & 0 \\32\titan{} & 64 slots, not preloaded (sync path) & 201--203 & 1.00 & 0 \\33\titan{} & 32 slots, LRU, demand (v1 installs) & 56--80 & 0.73--0.81 & 0.7--1.0 s \\34\titan{} & 16 slots, LRU, demand (v1 installs) & 36--45 & 0.48--0.60 & 1.7--2.0 s \\35\titan{} & 16 slots, LRU, demand (lazy installs) & 51--73 & 0.48--0.67 & 0.6--0.9 s \\36\titan{} & 16 slots, LRU + transition prefetch ($d{=}1$, $m{=}8$) & 35--56 & 0.63--0.73 & 0.6--1.1 s \\37\bottomrule38\end{tabular}39\end{table}4041Two observations. The all-resident control matches mlx-lm within one percent only when every layer's table is complete;42otherwise each layer pays a host synchronisation, which alone costs 2.4$\times$ in throughput on this model and is the floor43of any demand-loading design, including colibri's. And the demand-loading rows are the H1 \emph{control}: at 16 slots the44model does 53 misses per token (Prop.~\ref{prop:capacity}), i.e.\ about 11~ms of fetch per 2~ms of compute.4546\subsection{Routing locality}47\label{sec:results-locality}48From the v1 traces (18 prompts, 3\,686 tokens), per-layer LRU hit rates are 0.57 at 16 slots, 0.80 at 32 and 0.93 at 48 (LFU49within 0.01), nearly identical on fit and held-out prompts (\code{benchmarks/processed/gateC/sim\_olmoe\_laptop\_ssd\_v1.json});50a one-layer-ahead transition predictor has recall@8 of 0.50 on held-out prompts; a linear probe on the previous layer's51log-probabilities only 0.16 with these few traces. The v2 traces (65 prompts, 6 families) and the depth-$d$ predictors:52\textsc{pending} (simulation running at the time of writing; Table~\ref{tab:sim} is generated from its JSON).5354\subsection{Prefetching (H1)}55\label{sec:results-prefetch}56With the one-layer horizon the online prefetcher raised the hit rate from 0.48 to 0.63 (general) and from 0.67 to 0.7357(code) but reduced throughput from 51 to 35 and from 73 to 56~\toks{} (Table~\ref{tab:capacity}): the lead time58$d\,\tau_\ell = 0.13$~ms is shorter than the 0.21~ms fetch, so predicted experts arrive late and are waited for, and the5950\,\% of wrong predictions add traffic. This is exactly the regime excluded by condition~\eqref{eq:condition}.60Depth-$d$ and token-horizon prefetching, and the cold-NVMe tier: \textsc{pending} (next experiments of Gate C).6162\subsection{Medium model (Qwen3-30B-A3B)}63mlx-lm and \titan{} all-resident baselines on M3U96a (96~GB): \textsc{pending} (running).6465\subsection{Distributed execution (H2, H3), Metal kernels (H4), energy frontier (H5)}66\textsc{pending}: Gates D--F have protocols (Section~\ref{sec:algorithms}, \code{research/protocol.md}) and no logs in this version.67
Files · 258ba81d3935d575aeabc9cdae5eb3cdb039be8d
Menu

Theme