SPB Git

spb/wp33_uqo

Public MIT
1.9 KB · 22 lines latex
1\section{Limitations}2\label{sec:limitations}3\begin{itemize}4  \item \textbf{Scope of evidence.} Version 0.1 reports single-node results on a small model whose experts fit in memory, so the5    regimes where H1--H3 can matter (expert set larger than the budget of a node) are only approached by artificially bounding the6    slot tables. The medium model on the 36--64~GB nodes is the first realistic regime.7  \item \textbf{Topology.} The cluster is connected by gigabit Ethernet with no Thunderbolt link; RDMA/JACCL results would require8    cabling the Studios (a physical change outside the agent's authority). A LAN fetch of one expert costs about 28~ms, which rules9    out moving weights during decode and confines the remote tier to remote \emph{execution}.10  \item \textbf{Synchronisation floor.} Any demand-loading design must read the router's choice on the host once per layer; on this11    model that costs 2.4$\times$ in throughput. Hiding it requires either complete tables (no benefit from caching) or speculative12    execution on predicted experts with verification, which changes nothing semantically only if the verification is exact --13    a Gate D question.14  \item \textbf{Predictors.} Only simple predictors are tested; the probe is fitted on 48 prompts and evaluated on 17, all synthetic.15  \item \textbf{Install path.} The identified bottleneck of H1 (nine lazy slice writes per installed expert) is an implementation16    property of this version, not of the approach; a single-copy install may reverse the verdict and is the first task of v0.2.17  \item \textbf{Measurements.} Stall times in simulation use point estimates of tier latencies; online runs use the page-cache tier18    unless stated; energy is not yet measured.19  \item \textbf{Baselines.} llama.cpp and colibri comparisons are pending and will carry precision flags because their20    quantization formats differ from MLX's.21\end{itemize}22
Files · ffe7decb318f23ecdb239c8be6f9cc7d2310b66c
Menu

Theme