SPB Git

spb/wp31_uqo

Public
1.6 KB · 17 lines latex
1\paragraph{Commands.} Every experiment is a YAML under \texttt{configs/experiments/} executed by one of2\texttt{training.pretrain}, \texttt{training.train\_editor}, \texttt{training.collect\_interventions},3\texttt{training.train\_controller}, \texttt{evaluation.routing}, \texttt{evaluation.latency} or4\texttt{evaluation.run}; launch scripts under \texttt{scripts/} record which node ran which seed. Adapter5hyper-parameters shared by all systems: AdamW ($\beta=(0.9,0.95)$, weight decay 0.1, no decay on vectors), cosine6schedule with 100 warm-up steps to 10\,\% of the peak, learning rate $10^{-3}$ (adapters) / $6\times10^{-4}$7(backbones), gradient clipping 1.0, batch 32 $\times$ 320 characters (tiny) or 16 $\times$ 512 tokens (103M),8forced budgets $\{0,1,2,4,8\}$ with equal probability per position, singles and pairs balanced, $\eta_{\max}=0.25$,9$c=1$, $U\sim\mathcal N(0,0.2^2)$, $s=\rms(H)$ measured on the first batch. Controller: 2,000 steps at10$10^{-3}$, Huber $\delta=0.1$, $(\alpha,\beta,\gamma)=(1,0.5,0.1)$, early stopping on the calibration regret,11$(\lambda,\kappa)\in\{0,0.01,0.03,0.1,0.3\}\times\{0,0.5\}$ selected by loss reduction per unit of extra12operations on calibration rollouts. Label collection: prefix depth $r\sim U\{0,\dots,3\}$, total budget13$\in\{1,2,4,8\}$, four candidates per state, exhaustive audit every fourth batch.14\paragraph{Run index.} The following index is generated from the run manifests (laptop and nodes) by15\texttt{scripts/run\_index.py}; checkpoints remain on the host that produced them.16{\footnotesize\input{appendix/runs_index.tex}}17
Files · 42249a4cac2f515fc1b20d9b1580b0dc6b24635c
Menu

Theme