\section{Architecture}
\label{sec:architecture}

Figure~\ref{fig:arch} shows the components of \titan{}. The host model (embeddings, attention, norms, routers, KV cache) is
mlx-lm's own module tree, loaded lazily so that the stacked expert tensors are never materialised; each layer's expert block
is replaced by a \titan{} block that owns a \emph{slot table}. Everything else is orchestration around that table.

\begin{figure}[t]
\centering
\begin{tikzpicture}[node distance=4mm and 6mm, every node/.style={font=\small},
  box/.style={draw, rounded corners=2pt, minimum height=7mm, minimum width=24mm, align=center, fill=gray!8},
  tier/.style={draw, dashed, rounded corners=2pt, minimum height=7mm, minimum width=24mm, align=center},
  arr/.style={-{Latex[length=2mm]}, thick}]
  \node[box] (reg) {Expert registry\\\scriptsize safetensors byte ranges};
  \node[box, right=of reg] (store) {Expert store\\\scriptsize \code{pread}, F\_NOCACHE};
  \node[box, right=of store] (slots) {Slot tables ($L$)\\\scriptsize capacity $C$, LRU/LFU};
  \node[box, right=of slots] (exec) {Execution block\\\scriptsize \gatherqmm{} on slots};
  \node[box, below=of store] (pred) {Predictors\\\scriptsize prev-rank, transition$_d$, probe};
  \node[box, right=of pred] (pf) {Prefetcher\\\scriptsize reader pool, drain/install};
  \node[box, right=of pf] (host) {Host model (mlx-lm)\\\scriptsize attention, router, KV};
  \node[tier, below=of pf] (net) {Transport (Gate E)\\\scriptsize TCP, authenticated};
  \node[tier, right=of net] (sched) {Scheduler (Gate F)\\\scriptsize cost model, placement};
  \draw[arr] (reg) -- (store);
  \draw[arr] (store) -- (slots);
  \draw[arr] (slots) -- (exec);
  \draw[arr] (host) -- node[above,font=\scriptsize]{routes} (exec);
  \draw[arr] (exec) -- node[left,font=\scriptsize]{routes, probs} (pf);
  \draw[arr] (pf) -- (pred);
  \draw[arr] (pf) -- node[left,font=\scriptsize]{install} (slots);
  \draw[arr] (pred) -- node[above,font=\scriptsize]{$\hat S$} (pf);
  \draw[arr, dashed] (net) -- (slots);
  \draw[arr, dashed] (sched) -- (net);
\end{tikzpicture}
\caption{Components of \titan{} on one node. Solid boxes are implemented and tested in version 0.1; dashed boxes are the
distributed components whose status is given in Section~\ref{sec:limitations}.}
\label{fig:arch}
\end{figure}

\paragraph{Expert registry.} The registry parses \code{config.json} and the safetensors headers of a checkpoint directory
and records, for every (layer, expert) and every matrix (gate, up, down) and component (packed weight, scales, biases), the
absolute byte offset and length in the file, together with the routing semantics (softmax-then-top-$k$, renormalisation flag,
shared experts) and the quantization (bits, group size). It accepts only verified formats (OLMoE and Qwen3-MoE in this
version) and refuses the others rather than guessing.

\paragraph{Expert store (SSD tier).} An expert is read with one \code{pread} per tensor at the registry's offsets, wrapped
into MLX arrays without further copies on the Python side; bytes, read count and read time are accounted. The file descriptor
can be opened with \code{F\_NOCACHE} so that reads bypass the unified buffer cache, which is how the cold-NVMe tier is
measured (after \code{purge}); without it the same code measures the page-cache tier.

\paragraph{Slot tables.} Each layer owns $C$ slots in stacked MLX buffers of the exact shapes mlx-lm uses for its stacked
experts, so that the same \gatherqmm{} kernel can run on them. A dictionary maps resident expert ids to slots; eviction is
LRU by default (LFU available). Installing an expert is a slice assignment into the buffers, which MLX performs in place.
Misses are read as a batch per layer and installed lazily so that the layer's kernel consumes the new buffers in the same
graph: there is no per-expert synchronisation. The table records hits, misses, evictions, loads and stall time.

\paragraph{Execution block.} Given the router's indices, the block materialises them on the host (the one synchronisation a
demand-loading design cannot avoid), ensures residency, remaps expert ids to slot ids and runs \gatherqmm{} for the up, gate
and down projections with mlx-lm's compiled SwiGLU. When a layer's table holds every expert (the all-resident control) a
device-side slot map is used and no synchronisation occurs, which recovers mlx-lm's throughput. Prefill rows whose union of
experts exceeds $C$ are chunked greedily by rows, which is exact because rows are independent.

\paragraph{Predictors and prefetcher.} A predictor proposes up to $m$ experts for a layer from information available at the
decision point: the previous token's routing and router probabilities (token horizon), or the routing of layer $l-d$ at the
current token (layer horizon, depth $d$). The prefetcher submits reads to a small thread pool, installs completed fetches when
the target layer is reached, waits for fetches that are needed but still in flight (counted as \emph{late}, with the residual
wait charged as stall), and never changes which experts run.

\paragraph{Hierarchical memory and distributed execution.} The remote tier does not move weights during decode (Section
\ref{sec:prefetch-condition}): a remote node that holds an expert executes it on the coordinator's activations and returns the
$d$-dimensional result, exactly as colibri's cluster mode and as expert parallelism do. The transport (Gate E) is a persistent
authenticated TCP connection per worker with a per-layer batched request, timeouts and local fallback; placement (Gate F)
follows the cost model of Section~\ref{sec:formulation} with measured $c_n$, $M_n$, $\rho_{0n}$ and $\lambda$ per node.