Paper: abstract/intro/limitations reflect measured verdicts (H1, H4, E)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4 changed files +16 −8
modified
paper/main.pdf
+0 −0
Binary file not shown.
modified
paper/sections/abstract.tex
+9 −4
@@ -8,8 +8,13 @@ gigabit Ethernet. Every design choice is validated against mlx-lm on identical w | |||||
| 8 | bit-identical in logits and within one percent of mlx-lm's decode throughput; with bounded residency the router's choices and | 8 | bit-identical in logits and within one percent of mlx-lm's decode throughput; with bounded residency the router's choices and | ||
| 9 | the logits are unchanged by construction. We state five hypotheses (causal prefetch, topology-aware placement, hierarchical | 9 | the logits are unchanged by construction. We state five hypotheses (causal prefetch, topology-aware placement, hierarchical | ||
| 10 | caching, fused Metal kernels, and adaptive orchestration on the latency--throughput--energy frontier) and test each one with a | 10 | caching, fused Metal kernels, and adaptive orchestration on the latency--throughput--energy frontier) and test each one with a | ||
| 11 | −pre-registered protocol on a live inventory of a 21-node Mac cluster. This version (0.1) is a \emph{protocol and preliminary} | 11 | +pre-registered protocol on a live inventory of a 21-node Mac cluster. This version (0.1) is a \emph{preliminary} working paper whose | ||
| 12 | −working paper: it contains the verified hardware inventory, the prior-art audit, the single-node runtime and its correctness | 12 | +results are mostly negative and fully attributed: on the small model, the best causal predictor (a per-layer logistic probe, | ||
| 13 | −evidence, and the first controlled single-node experiments; distributed and energy experiments are reported only where their | 13 | +recall@8 of 0.76, within 0.16 of the oracle hit rate) only ties with demand loading because installing a prefetched expert costs | ||
| 14 | −raw logs exist, and pending experiments are explicitly labelled as such.% | 14 | +as much graph time as the read it saves (H1 not supported); kernel fusion is immaterial because decode at batch size one is | ||
| 15 | +launch-bound at 28\,\% of memory bandwidth (H4 not supported); and executing half the experts on a second Mac Studio over the | ||||
| 16 | +gigabit Ethernet that actually connects the cluster costs 2.5~ms per layer, more than a cold-NVMe miss, so the remote tier is a | ||||
| 17 | +capacity mechanism rather than a latency one (H2/H3 in this regime). The storage layout, by contrast, matters: an | ||||
| 18 | +expert-contiguous record file raises cold-NVMe decoding from 8 to 36~tokens/s. Energy measurements and the adaptive scheduler | ||||
| 19 | +are pending and labelled as such.% | ||||
| 15 | } | 20 | } | ||
modified
paper/sections/introduction.tex
+4 −3
@@ -63,6 +63,7 @@ per-layer slot tables with LRU/LFU eviction, an exact execution block over \gath | |||||
| 63 | with four causal predictors; (iii) the correctness evidence (bit-identical logits, token-identical greedy decoding); | 63 | with four causal predictors; (iii) the correctness evidence (bit-identical logits, token-identical greedy decoding); | ||
| 64 | (iv) a trace-driven simulator calibrated on measured constants and the first controlled single-node experiments on | 64 | (iv) a trace-driven simulator calibrated on measured constants and the first controlled single-node experiments on | ||
| 65 | OLMoE-1B-7B; (v) a formal statement of the placement--prefetch problem (Section~\ref{sec:formulation}) and the deterministic | 65 | OLMoE-1B-7B; (v) a formal statement of the placement--prefetch problem (Section~\ref{sec:formulation}) and the deterministic | ||
| 66 | −cost-aware heuristic that the distributed gates will use. Distributed execution (Gates E--F of the protocol), Metal kernel work | 66 | +cost-aware heuristic that the distributed gates will use. (vi) the profiling-driven verdict on Metal kernels (Gate~D), (vii) an authenticated TCP transport | ||
| 67 | −(Gate~D) and the energy Pareto study are implemented or planned as described in Section~\ref{sec:limitations} and reported only | 67 | +with a bit-exact remote expert worker and the two-node measurements on the cluster's gigabit Ethernet (Gate~E), and (viii) the | ||
| 68 | −to the extent that raw logs exist. | 68 | +frequency-based placement experiment on the medium model (Gate~F, reported to the extent that its logs exist). The energy | ||
| 69 | +Pareto study (H5) and the adaptive scheduler are pending and labelled as such. | ||||
modified
paper/sections/limitations.tex
+3 −1
@@ -11,7 +11,9 @@ | |||||
| 11 | model that costs 2.4$\times$ in throughput. Hiding it requires either complete tables (no benefit from caching) or speculative | 11 | model that costs 2.4$\times$ in throughput. Hiding it requires either complete tables (no benefit from caching) or speculative | ||
| 12 | execution on predicted experts with verification, which changes nothing semantically only if the verification is exact -- | 12 | execution on predicted experts with verification, which changes nothing semantically only if the verification is exact -- | ||
| 13 | a Gate D question. | 13 | a Gate D question. | ||
| 14 | − \item \textbf{Predictors.} Only simple predictors are tested; the linear probe is under-trained with the small trace corpus. | 14 | + \item \textbf{Predictors.} Only simple predictors are tested; the probe is fitted on 48 prompts and evaluated on 17, all synthetic. | ||
| 15 | + \item \textbf{Install path.} The identified bottleneck of H1 (nine lazy slice writes per installed expert) is an implementation | ||||
| 16 | + property of this version, not of the approach; a single-copy install may reverse the verdict and is the first task of v0.2. | ||||
| 15 | \item \textbf{Measurements.} Stall times in simulation use point estimates of tier latencies; online runs use the page-cache tier | 17 | \item \textbf{Measurements.} Stall times in simulation use point estimates of tier latencies; online runs use the page-cache tier | ||
| 16 | unless stated; energy is not yet measured. | 18 | unless stated; energy is not yet measured. | ||
| 17 | \item \textbf{Baselines.} llama.cpp and colibri comparisons are pending and will carry precision flags because their | 19 | \item \textbf{Baselines.} llama.cpp and colibri comparisons are pending and will carry precision flags because their | ||
| 18 | 20 | ||||