SPB Git

spb/wp30_uqo

Public
6.6 KB · 93 lines latex
1\section{Problem formulation and taxonomy}2\label{sec:formulation}34\subsection{Protocol notation}5Let $d$ be a source document, $q$ a downstream query, $y^\star$ its reference answer, $S$ the sender model and $R$ the receiver model.6A communication method $c$ under a resource setting $b$ and a question-access policy $a$ produces a message7\begin{equation}8  m = E_{c,b}(d; S, a), \qquad \hat y = G_R\big(q, D_c(m), \mathcal{A}\big),9  \label{eq:protocol}10\end{equation}11where $D_c$ denotes decoding, loading or mapping at the receiver and $\mathcal{A}$ is the explicitly allowed artifact or retrieval12interface (empty for all methods except A6, the reference-plus-retrieval method, and the repairable chains of Section~\ref{sec:handoffs}).13Equation~\eqref{eq:protocol} is notation for a protocol, not a claim that methods share an implementation: for text methods $m$ is a14string the receiver prefills; for cache methods $m$ is a set of tensors installed as the receiver's attention state, and only the fresh15query tokens are prefilled.1617\paragraph{Question access.} Under the \emph{question-blind} regime the sender observes $d$ and a generic instruction but not $q$; the18message is reusable across the questions of a document. Under the \emph{question-aware} regime both sides may use $q$. The two regimes19are reported separately throughout; a selection method that observes the question (SnapKV-style) is never compared as if it were blind.2021\paragraph{Terminology.} We follow a fixed vocabulary (Table~\ref{tab:terms}). In particular, \emph{text compression} is a shorter text22consumed by a receiver; \emph{KV compression} is quantization, selection or transformation of actual attention tensors; \emph{cache23reuse} reuses a compatible, previously computed prefix state; \emph{cache transfer} moves or maps a state for a receiver; and24\emph{knowledge transfer} means improved access to task-relevant information through a defined message---never permanent learning.2526\begin{table}[t]\centering\footnotesize27\caption{Terminology used in this paper.}\label{tab:terms}28\begin{tabular}{@{}p{3.2cm}p{6.2cm}p{5.6cm}@{}}\toprule29Term & Operational meaning & Error avoided \\\midrule30Text compression & Shorter text consumed by a receiver & Calling it KV-cache compression \\31KV compression & Quantization/selection/transformation of attention tensors & Measuring only a shorter prompt \\32Cache reuse & Reuse of a compatible previously computed prefix state & Assuming one family $\Rightarrow$ compatible \\33Cache transfer & Moving/mapping cache state for a receiver & Sending a summary and calling it latent transfer \\34Knowledge transfer & Better access to task information via a message & Claiming a frozen receiver learned a skill \\35Latent communication & An internal representation crosses executions & Calling retrieval embeddings a KV state \\36Shared memory & Addressable external records under a protocol & Treating an identifier as free content access \\37Prefill-free handoff & Receiver does not prefill the transferred document & Claiming the fresh query needs no processing \\38Evidence traceability & Output linked to a resolvable source span & Treating self-reported confidence as proof \\\bottomrule39\end{tabular}40\end{table}4142\subsection{Accounting}43For a dense cache with uniform layers the stored tensor size is44\begin{equation}45  M_{KV} = 2\,B\,L\,T\,H_{KV}\,d_h\,s,46  \label{eq:mkv}47\end{equation}48with batch $B$, layers $L$, stored positions $T$, KV heads $H_{KV}$, head dimension $d_h$ and $s$ bytes per scalar. We use49\eqref{eq:mkv} only as a consistency check: the harness measures stored tensor bytes from the arrays themselves (which, for the50packed quantized caches, include the uint32 words, the per-group scales and biases) and measures the serialized payload from the file51written to disk, including the compatibility manifest. Two ratios are reported with explicit denominators,52\begin{align}53  CR_{\text{payload}} &= \frac{\text{reference serialized payload bytes}}{\text{method serialized payload bytes}},\\54  CR_{KV} &= \frac{\text{native cache tensor bytes}}{\text{stored cache tensor and metadata bytes}},55\end{align}56and a text-token ratio is never treated as a cache-position ratio.5758\paragraph{Timing boundaries.} Each trial records separate wall-clock spans with device synchronization before and after59(\texttt{mx.synchronize}): sender prefill and message generation (\emph{encode}); document prefill into the receiver cache60(\emph{doc prefill}); compression or selection; serialization and transport codec; deserialization and cache installation61(\emph{load}); fresh query prefill up to the first content token (TTFT); decoding; and, for metered methods, retrieval. For a serial62workflow,63\begin{equation}64  T_{\text{total}} = T_{\text{queue}} + T_{\text{read}} + T_{\text{sender}} + T_{\text{encode}} + T_{\text{serialize}}65  + T_{\text{transport}} + T_{\text{load}} + T_{\text{receiver}} + T_{\text{tools}} .66\end{equation}67Three scenarios are assembled from the spans: \emph{cold} (no reusable state exists: build + compress + load + receiver),68\emph{warm per-query} (the state exists: load + receiver), and \emph{amortized over $N$ queries},69\begin{equation}70  \bar C(N) = \frac{C_{\text{training}} + C_{\text{setup}} + C_{\text{document}}}{N} + C_{\text{query}},71  \label{eq:amortized}72\end{equation}73where $C_{\text{training}}$ is zero for all training-free methods and is reported separately for the learned mapper. Components that74a hosted provider does not expose are recorded as missing, not as zero.7576\subsection{Taxonomy}77Table~\ref{tab:taxonomy} lists every method with its representation, question-access regime, training requirement, compatibility78requirement, composability across sources, and its measured status in this study. Methods that were audited but not executed appear79with the reason.8081\begin{table}[p]\centering\scriptsize82\caption{Method taxonomy and coverage. ``Measured'' = executed in WP30 with real model inference; the last rows are audited methods that83were not executed, with the reason.}\label{tab:taxonomy}84\input{tables/table1_taxonomy}85\end{table}8687\subsection{Research questions and registered hypotheses}88Seven research questions (RQ1--RQ7) and ten falsifiable hypotheses were registered before any held-out outcome was examined89(\texttt{protocol/hypotheses.md}; reproduced in Appendix~\ref{app:hypotheses}). In brief: RQ1 compression frontier; RQ2 transfer90compatibility; RQ3 whether rankings change across resource denominators; RQ4 repeated handoffs; RQ5 structured evidence versus prose;91RQ6 locality versus moving the cache; RQ7 adaptive routing. A primary confirmatory family of six paired comparisons is Holm-corrected;92all other comparisons are exploratory.93
Files · f4474e1b03aebffdfcd19ab989b35165571329e6
Menu

Theme