SPB Git

spb/wp33_uqo

Public MIT
1.3 KB · 13 lines latex
1\section{Conclusion}2\label{sec:conclusion}3We have built an MLX-native MoE runtime whose bounded expert residency is semantically invisible (bit-identical logits to mlx-lm4on two architectures, on one node and across two) and whose all-resident path costs nothing (489 vs 494~\tps{}; 121.6 vs 121.7 on5the medium model), characterised the cluster as it actually is (gigabit Ethernet, no Thunderbolt), and tested the five hypotheses.6Three findings carry over to any similar system: the storage layout of experts decides the cold tier (8 to 36~\tps{}); the number of7host-side operations per installed expert, not prediction accuracy, decides whether prefetching pays (a probe within 0.16 of the8oracle hit rate only wins on long generations); and on gigabit Ethernet a remote expert costs more than a local cold miss, so9placement across nodes buys capacity and, when cost-aware, a better loss than static sharding, but not speed. Kernel fusion was10immaterial because decoding at batch size one is launch-bound. The next version follows from the attribution: a single-copy11install, the probe on the medium model, nodes whose budget the model exceeds, energy on the cluster, and, if cabled, the Thunderbolt12tier. Every number here has a log; the losses are reported with the same care as the wins.13
Files · main
Menu

Theme