# Literature audit — Group 1 (primary sources)

WP30 — "Cache Compression and Knowledge Transfer in Multi-Agent Systems"
Access date for every URL below: **2026-10-29**. All facts come from pages fetched on that date (arXiv abstract pages via arXiv export API + arXiv HTML full text, GitHub REST API + raw files, Hugging Face API). Nothing is from memory. Fields that could not be confirmed from a fetched page are marked **unverified** or **not stated**.

Verification legend: **V** = read directly from the fetched page; **V-partial** = page fetched but field absent/truncated; **U** = unverified.

---

## 1. SwaT — Swarm Telepathy (source_id `swat2026`)

**URLs fetched (2026-10-29)**
- https://arxiv.org/abs/2610.15095 (abstract page) — V
- https://export.arxiv.org/api/query?id_list=2610.15095 (metadata/abstract) — V
- https://arxiv.org/html/2610.15095 (full text; characters 0–117 603, read in two passes) — V
- https://github.com/Ar-Kareem/SwaT (repo page) — V
- https://api.github.com/repos/Ar-Kareem/SwaT and `/commits?per_page=1` — V
- https://raw.githubusercontent.com/Ar-Kareem/SwaT/master/README.md, `/requirements.txt` — V
- https://raw.githubusercontent.com/Ar-Kareem/SwaT/master/LICENSE → **HTTP 404** (no LICENSE file) — V

**Bibliographic**
- Exact title: *Swarm Telepathy: Prefill-Free KV-Cache Transfer Across Families*
- Authors (as listed): Abdulrahman Alabdulkareem, Irene Tenison, Yang Zhang, Yada Zhu, Boris Katz, Dan Gutfreund
- Venue/status: arXiv preprint, cs.LG. No "Comments" field, no venue stated on the abstract page or in the HTML.
- First submission: v1, 10 Oct 2026 (18:09:19 UTC). Latest revision: **v1** (only version as of 2026-10-29).
- Year: 2026.

**Repository**
- URL: https://github.com/Ar-Kareem/SwaT — exists, public, default branch `master`.
- Created 2026-10-10; last push 2026-10-14. Single commit: `ae2de2285caf82f4d500526bf6e6992747b98eea` (2026-10-14, "Initial commit"). 11 stars, 4 forks.
- **License: none.** GitHub API reports `license: null`; no LICENSE file (404); README has no license statement. Legally "all rights reserved" by default — flag for reproduction.
- Top-level: `.env.example`, `.gitignore`, `README.md`, `assets/`, `env3.10.freeze.txt`, `multiagent/` (incl. `basic_swat_walkthrough.ipynb`, `model/head_budget_optimization/head_budgets/`), `pyrightconfig.json`, `requirements.txt`, `scripts/` (`exp1/`, `exp2/`, `download/`).

**Checkpoints**
- **No pretrained mapper released.** README gives no Hugging Face or download link; the trained mapper is produced locally at `<run>/stage2/exit-checkpoint-24000/renderer.pt`. Default recipe pair: `ibm-granite/granite-3.3-8b-instruct` (sender) → `HuggingFaceTB/SmolLM3-3B` (receiver). Checkpoint availability = **none released (train-it-yourself)**.

**Backends / hardware**
- `requirements.txt`: `torch==2.6.0`, `transformers==5.2.0`, `datasets==4.5.0`, `math-verify`, `latex2sympy2_extended`, etc. Python 3.10 venv.
- Training uses two GPUs (sender on one, receiver on the other; `--gpus 1,2`). GPU model not named anywhere in README or HTML (paper reports "<1 GPU-day", "under 6 GPU-hours" in Fig. 2, ms-level GPU times).
- No mention of vLLM, MLX, Apple Silicon or MPS in README, requirements, or paper text. PyTorch + HF Transformers only. CUDA-only in practice (multi-GPU device placement); MPS support **unverified / not stated**.

**Core algorithm (from paper HTML + README)**
1. Input: the **sender's prefill KV-cache** C^S_d of a text d (all sender layers), plus the **receiver's token IDs** T_R of the same text d (embedding-table lookup only, "0 FLOPs"). Tokenizer alignment is done through **character spans** of the shared text: each sender token's midpoint maps to a fractional receiver RoPE position (Eq. 12); causal mask uses span ends.
2. The mapper M_{i→j} (~300 M params, B∈{7,8,9} blocks, D=1024, H=16, F=4096) runs causal self-attention → causal cross-attention over the sender's de-RoPE'd cache → FFN; output projection + RMS calibration + receiver RoPE on keys. Each block maps a group of sender layers to a group of receiver layers.
3. Output: predicted **keys and values for every receiver KV layer, one entry per receiver token** — i.e., a synthetic receiver prefill cache.
4. Training (only the mapper; both LLMs frozen): Stage 1 = 5 000 steps normalized-MSE reconstruction against the receiver's real prefill cache; Stage 2 = 24 000 steps receiver next-token NLL on a continuation y given task prompt q, gradients flowing through the frozen receiver into SwaT. ~44 M unique English tokens (PG-19, WikiText-103, QASPER/QuALITY/HotpotQA/MuSiQue train splits; then ARC-C, HellaSwag, WinoGrande, MMLU aux, GSM8K traces generated by the receiver).
5. What the receiver still needs: **no prefill of d**. It does need (a) the translated cache and (b) the **receiver tokenization of d** (token IDs → positions/embeddings) — which implies **the source text must be available** to tokenize (the ethics section says the method "requires the sender's text to be available"). The receiver then processes only its own question q (or nothing).

**"Translated cache handoff" vs "compaction" — separate?**
- **Yes, two separate operations, applied in this order: translate first, then compact.** Eq. 8: C̃_U = AM_U(M(C_{x|y})) — the sender's cache is translated by the SwaT mapper into receiver space, then **Attention Matching (AM, Zweiger et al. 2026)** compacts the translated cache to a token budget U matched to the text-summary baseline. SwaT alone (Exp 1) does no compaction; SwaT+AM (Exp 2) is the agent-communication setting compared with text-to-text (T2T).
- Because AM needs **queries in the receiver's space**, Exp 2 first trains a small **query-transfer MLP** (README steps `tq_features`, `tq_pairing`, `train_tq`) mapping sender queries → receiver queries, with a per-receiver head-budget file; receivers without an optimized budget file (e.g., SmolLM3) fall back to `"budget": "uniform"` (weaker). The paper's ablation "Headwise MLP" (−14.71 pts) is the only in-text reference to this component.
- **Receiver prefill of the source is avoided** in both SwaT and SwaT+AM: README walkthrough = "the sender prefills the text, SwaT translates the sender's cache, and the receiver answers from the translated cache without seeing the text." Timing metric = receiver TTFT 4.7 ms vs 20.1 ms for T2T (Table 17).
- Appendix F: for documents beyond the receiver window, the sender prefills the full document, then the cache is split into 8k chunks, each translated+compacted and merged.

**Evaluation**
- 10 cross-family pairs, 6 families (Qwen 3, Llama 3, Gemma 2/3, Ministral 3, Granite 3, SmolLM3; 3B–32B): Qwen3-4B→Llama3-8B; Llama3-8B→Qwen3-14B; Ministral3-8B→Llama3.2-11B; Gemma3-4B→Granite3.3-8B; Qwen3-4B→Gemma2-9B; Gemma2-9B→Ministral3-14B; Gemma3-4B→Llama3-8B; Gemma3-12B→Qwen3-4B; Qwen3-32B→Llama3-8B; Granite3.3-8B→SmolLM3-3B.
- 17 benchmarks: in-domain ARC-C, HellaSwag, WinoGrande, MMLU, GSM8K, QuALITY, QASPER, HotpotQA, MuSiQue; out-of-domain MedQA, OpenBookQA, MATH-500, BoolQ, RACE, LongHealth, SQuAD, NarrativeQA. Plus 5 VLM benchmarks (VQAv2, TallyQA, OK-VQA, GQA, VizWiz) and CMMLU.
- Headline: >90 % retention of receiver's own text score; SwaT+AM vs T2T on 7 long-doc datasets × 3 pairs × 5 budgets: geometric-mean score 55.5 vs 37.3, sender GPU time 2 447 ms vs 10 234 ms (~4.2×), TTFT 4.7 vs 20.1 ms.
- Prior-work positioning (Table 1): C2C and XKV are cross-family but need receiver prefill; Closed-Form (Heo et al.), CacheBridge, KV-Alignment (Dery et al.), Dense Latent (Chen et al.) are same-family; LatentMAS works "within copies of itself"; DroidSpeak reuses caches within same-architecture variants. Ablation with C2C-style token alignment: −3.20 pts.

**Stated limitations**
- One mapper per ordered pair → n(n−1) mappers for n families ("quickly becomes infeasible"); shared latent space would need 2n.
- Only single message handoffs studied; KV-cache "degrades and isn't as composable as text" for repeated transfers/aggregation.
- Ethics: text remains auditable because the sender text exists, but the cache may carry information beyond the text.
- Practical (README): no license; no released checkpoints; uniform AM budgets for most receivers.

---

## 2. Fast KV Compaction via Attention Matching (source_id `compaction2026`)

**URLs fetched (2026-10-29)**
- https://arxiv.org/abs/2602.16284 — V ; export API — V ; https://arxiv.org/html/2602.16284 (0–144 341 chars, two passes) — V
- https://github.com/adamzweiger/compaction — V ; GitHub API repo + latest commit — V ; raw `requirements.txt` — V

**Bibliographic**
- Exact title: *Fast KV Compaction via Attention Matching*
- Authors: Adam Zweiger, Xinghong Fu, Han Guo, Yoon Kim (MIT)
- Venue/status: arXiv preprint, cs.LG. No Comments field; no venue stated in HTML.
- v1: 18 Feb 2026; **latest v2: 26 May 2026**. Year 2026.

**Repository**
- https://github.com/adamzweiger/compaction — exists, public, `main`, created 2026-01-28, last push 2026-04-22. Latest commit `2020f6d37940fe013e7e7f1ebf409e5d107c5670` (2026-03-31, "Merge pull request #2 … ruler-kvmerger-longbench"). 277 stars, 30 forks. 5 commits.
- **License: MIT** (LICENSE file present; GitHub API `spdx_id: MIT`).
- Layout: `compaction/` (methods, algorithms, chunking, query generation), `evaluation/` (`run_qa_evaluation.py`, `run_reasoning_evaluation.py`), `scripts/`, `head_budget_optimization/` (per-head budgets JSON), `models/` (Qwen3, Llama, Gemma3 utilities), `examples/qa_demo.py`, `data/`, `TODO`.

**Checkpoints**
- **Not applicable / none**: the method trains no weights. Only precomputed per-model head-budget JSON files are shipped (`head_budget_optimization/head_budgets`). Demo model `Qwen/Qwen3-4B`.

**Backends / hardware**
- `requirements.txt`: Python 3.12, `torch==2.8.0`, `transformers==4.57.1`, **`vllm==0.11.0`**, `accelerate==1.12.0`, `datasets==4.4.1`. vLLM is used to generate "self-study" conversations (query generation); query vectors extracted via forward hooks; lstsq in FP32 then cast to BF16. Paper mentions FlashAttention/FlexAttention/PyTorch SDPA for serving considerations.
- Hardware in paper: Fig. 1 timing on a single **H100**; stage breakdown on a single **H200** (Gemma-3-12B, 60k-token LongHealth context). Cartridges baseline 5–15 H100-hours per context.
- No mention of MLX / Apple Silicon / MPS. vLLM dependency effectively makes the stock pipeline **CUDA/Linux**; core AM math (lstsq, NNLS, OMP) is plain PyTorch and portable in principle — **MPS support unverified**.

**Core algorithm**
1. Input (per KV head, per layer): original keys/values (K, V) for T tokens **of one model**, plus a set of **reference queries Q_ref** from the same model (repeat-prefill: model asked to repeat the context; self-study: model answers a few fixed prompts about the context; random vectors as a weak option; capped at 50 000 queries/head; collected layer-by-layer "on-policy" with earlier layers already compacted).
2. Output: a compact cache of t < T entries — compacted keys C_k (a subset of original keys chosen by highest-attention heuristic or OMP), compacted values C_v (ordinary least squares) and a per-entry scalar bias β (non-negative least squares) so that attention outputs and attention mass are reproduced. Logical length T is kept so appended tokens get the same positions.
3. Nothing is trained by gradient descent at compaction time ("avoids gradient descent entirely"); the only per-model precomputation is a nonuniform head-budget schedule from sensitivity curves.
4. Receiver: **the same model** that produced the cache decodes from the compact cache; no source text or prefill is needed afterwards. Cross-model use is **not discussed or claimed** in the paper (SwaT is what applies AM to a translated cache).

**Evaluation**
- Models: Qwen3-4B (Qwen3-4B-Instruct-2507 for LongHealth), Llama-3.1-8B-Instruct, Gemma-3-12B-it (only global-attention layers compacted).
- Datasets: QuALITY (50 articles / 894 Q), LongHealth (4 × ~60k-token contexts, 100 Q each), QASPER, LongBench v2 (47-context subset), RULER (appendix, via kvpress; 6 500 tasks).
- Ratios 1 %–100 % of cache; headline 50× "in seconds"; baselines: Cartridges (gradient-based), H2O, SnapKV, PyramidKV, KVzip, DuoAttention, KVMerger, text summarization.
- Runtime (H200, 60k ctx, Gemma-3-12B): self-study ≈139 s, repeat-prefill ≈8 s, OMP ≈565 s, OMP-fast ≈104 s, NNLS ≈2.2 s, LS ≈1.8 s.

**Stated limitations**
- Incompatible with multi-head latent attention (MLA); hybrid sliding-window models get more conservative ratios.
- C_k restricted to a subset of original keys; Cartridges still wins at 100× on LongHealth.
- Head-budget allocation assumes approximately separable heads; unoptimized OMP+self-study takes minutes.
- Base models degrade beyond ~100k tokens; LongBench v2 subset small (47); no quantization experiments; online compaction is a proof of concept.

---

## 3. Cache-to-Cache (C2C) (source_id `c2c2025`)

**URLs fetched (2026-10-29)**
- https://arxiv.org/abs/2510.03215 — V ; export API — V ; https://arxiv.org/html/2510.03215 (0–122 518 chars, two passes) — V
- https://github.com/thu-nics/C2C — V ; GitHub API repo + latest commit — V ; raw `README.md`, `pyproject.toml`, `environment.yml` — V
- https://huggingface.co/api/models/nics-efc/C2C_Fuser — V (490 files, license tag apache-2.0, lastModified 2026-08-14) ; https://huggingface.co/api/collections?owner=nics-efc — V (collection `nics-efc/c2c-68e66ef54b977bd7e58d2d74`, 3 items)
- https://huggingface.co/collections/nics-efc/C2C_Fuser → **404** (wrong slug; real slug above)

**Bibliographic**
- Exact title: *Cache-to-Cache: Direct Semantic Communication Between Large Language Models*
- Authors: Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang
- Venue/status: arXiv Comments field = **"Published in ICLR'26"**; GitHub description "[ICLR'26] The official code implementation…". cs.CL, cs.LG.
- v1: 3 Oct 2025; **latest v2: 2 Mar 2026**. Year 2025 (preprint) / 2026 (ICLR).

**Repository**
- https://github.com/thu-nics/C2C — exists, public, `main`, 35 commits, created 2025-10-03, last push **2026-10-28** (commit `5eac72b90faf7b3dc2638e874fe7cba1e74c8bcf`, "docs: announce AKV open-source release"). 761 stars, 96 forks.
- **License: Apache-2.0** (LICENSE file, GitHub API). Note discrepancy: `pyproject.toml` classifier says "MIT" with placeholder URLs (`your-org/unified_memory`); the LICENSE file (Apache-2.0) governs. HF model repo tagged apache-2.0.
- Package name `rosetta` (`rosetta/model`, `train`, `baseline`, `utils`); `script/`, `recipe/` (`C2C_0.6+0.5.json`, `unified_eval.yaml`), `bash/`, `environment.yml`, `pyproject.toml`.

**Checkpoints — released**
- Hugging Face repo **https://huggingface.co/nics-efc/C2C_Fuser** (tree with per-pair folders, each `final/projector_{i}.pt` + `.json` + `aggregator_config.json`). Pairs (receiver ← sharer):
  - Qwen3-0.6B ← Qwen2.5-0.5B-Instruct (`qwen3_0.6b+qwen2.5_0.5b_Fuser`)
  - Qwen3-0.6B ← Llama-3.2-1B-Instruct (`qwen3_0.6b+llam3.2_1b_Fuser`)
  - Qwen3-0.6B ← Qwen2.5-Math-1.5B (`qwen3_0.6b+qwen2.5_1.5b_math_Fuser`)
  - Qwen3-0.6B ← Qwen3-4B (`qwen3_0.6b+qwen3_4b_Fuser`)
  - Qwen3-0.6B ← Qwen3-4B-Base (`qwen3_0.6b+qwen3_4b_base_Fuser`)
  - Qwen3-1.7B ← Qwen2.5-1.5B-Instruct (`qwen3_1.7b+qwen2.5_1.5b_Fuser`)
  - Qwen3-8B ← Qwen2.5-7B-Instruct (`qwen3_8b+qwen2.5_7b_Fuser`)
- Live demo: https://huggingface.co/spaces/nics-efc/C2C_demo.

**Backends / hardware**
- `pyproject.toml`: `requires-python>=3.10`, `torch==2.6.0`, `transformers==4.52.4`; extras `training` (datasets, accelerate, wandb, peft) and `evaluation`. `environment.yml` is a linux-64 conda env (Tsinghua mirrors). README examples use `device=torch.device("cuda")`, `CUDA_VISIBLE_DEVICES=0..7`.
- Paper: inference timing on a single **NVIDIA A100**, batch 1; training cost ≈6.95–8.44 GPU-hours / 300 steps (Table 12); training GPU type not stated.
- No vLLM, MLX, Apple Silicon or MPS mentioned. HF Transformers `past_key_values` interface → plain PyTorch; MPS portability **unverified**.

**Core algorithm**
1. Input: the **sharer's KV-cache of the prompt text X** *and* the **receiver's own KV-cache of the same X** — the receiver **still prefills X itself** (its cache C_n(X) enters the fuser through a residual path, Eq. 3; Table 3 lists a 27 ms receiver prefill). Tokens are aligned by decoding each receiver token to a string and re-encoding with the sharer's tokenizer (maximal-coverage choice for one-to-many; `<pad>` for chat-template tokens). Layers are aligned "terminally" (last layers paired first).
2. Trained component: the **C2C Fuser**, one per receiver layer — projection module (concat → projection → feature fusion), input-aware dynamic head weighting, and a per-layer learnable **gate** (Gumbel-sigmoid, temperature-annealed) selecting which receiver layers accept fused cache. Both LLMs frozen; loss = next-token prediction on receiver responses (SFT-like). Training data: 500 k OpenHermes-2.5 samples (1 epoch, max len 2 048, bs 256, 1 929 steps); LongBench-E for length scaling; MMLU auxiliary_train for size/combination studies.
3. Output: a **fused receiver cache replacing the receiver's prefill cache**; decoding proceeds normally.
4. What the receiver still needs: **the source text (to prefill it) and its own prefill**. The saving is skipping the sharer's intermediate *text generation*, not the receiver prefill. (SwaT's Table 1 classifies C2C as "requires receiver prefill".)

**Evaluation**
- Families Qwen2.5, Qwen3, Llama 3.2, Gemma 3 (0.6B–14B). Main receiver Qwen3-0.6B with sharers Qwen2.5-0.5B-Instruct, Llama3.2-1B, Qwen3-4B-Base; others: Gemma3-1B, Qwen2.5-Math-1.5B, Qwen2.5-Coder-0.5B, swapped roles, Qwen3-4B→Qwen3-0.6B.
- Benchmarks: OpenBookQA, MMLU-Redux, ARC-Challenge, C-Eval; LongBench (v1/E) for length scaling; appendix: multi-sharer MMLU, multi-receiver/multi-sharer (M projectors + N fusers), agentic GSM8K case study (Cognify workflow: single 41.17, T2T 61.18, C2C 62.55, T-C2C 78.01).
- Baselines: receiver-only, sharer-only, text-to-text (analyze-then-respond), query-level routing. Headline: +6.4–14.2 % vs individual models, +3.1–5.4 % vs T2T, 2.5× latency speedup.

**Stated limitations**
- A much weaker sharer can inject noise/mislead a stronger receiver (failure case A.4.6); gains shrink for larger receivers.
- O(N) scaling to many LLMs open; pairwise focus is "an initial foundation"; multi-sharer support in repo is "preliminary".
- Receiver prefill of the shared context is still required (observed from Eq. 3 / Table 3 rather than stated as a limitation).
- Future work: agentic systems, VLM/VLA cross-modal, speculative decoding/token routing, privacy-aware collaboration.

---

## 4. LatentMAS — Latent Collaboration in Multi-Agent Systems (source_id `latentmas2025`)

**URLs fetched (2026-10-29)**
- https://arxiv.org/abs/2511.20639 — V ; export API — V ; https://arxiv.org/html/2511.20639 (0–155 921 chars, two passes) — V
- https://github.com/Gen-Verse/LatentMAS — V ; GitHub API repo + latest commit — V ; raw `requirements.txt` — V

**Bibliographic**
- Exact title: *Latent Collaboration in Multi-Agent Systems*
- Authors (13): Jiaru Zou, Ruizhong Qiu, Gaotang Li, Xiyuan Yang, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang
- Venue/status: arXiv Comments = **"ICML2026 Spotlight, Project: [GitHub]"**; repo description "[ICML 2026 Spotlight]"; README announces spotlight acceptance 2026-05-01. cs.CL, cs.AI, cs.LG.
- v1: 25 Nov 2025; v2: 8 Dec 2025; v3: 1 Jun 2026; **latest v4: 3 Aug 2026**. Year 2025 (preprint) / 2026 (ICML).

**Repository**
- https://github.com/Gen-Verse/LatentMAS — exists, public, `main`, 39 commits, created 2025-11-25, last push 2026-06-18 (commit `9a9e4d331eb11430bd9e64754c6b252b06d73031`, "Add slides to README"). 1 167 stars, 183 forks.
- **License: Apache-2.0** (LICENSE file, GitHub API).
- Files: `run.py`, `models.py` (HF/vLLM wrapper with latent realignment), `methods/{baseline,text_mas,latent_mas}.py`, `prompts.py`, `data.py`, `utils.py`, `example_logs/`, `requirements.txt`.

**Checkpoints**
- **None (training-free; nothing to release).** Uses stock HF models (examples Qwen3-4B/8B/14B). Links: arXiv, HF paper page, slides, Google Drive of experiment logs. No HF model links.

**Backends / hardware**
- `requirements.txt` (unpinned): `transformers`, `torch`, `numpy`, `tqdm`, `accelerate`, `datasets`; optional `pip install vllm`.
- Paper Appendix C.2: Python + PyTorch + HF Transformers (`past_key_values`), optional **vLLM** backend (prefix caching, tensor parallel). Hardware: **8× NVIDIA A100-80G**. README vLLM caveat: vLLM "does not officially support modifying KV-cache or prompting via latent embeddings"; authors patch internals; hybrid setup (vLLM for final text, HF model for latent rollout) recommends 2 GPUs; HF backend recommended for reproducing numbers.
- No MLX / Apple Silicon / MPS mention. Pure HF-Transformers path is plain PyTorch — MPS portability **unverified**.

**Core algorithm**
1. Within an agent: **auto-regressive latent thoughts** — the last-layer hidden state h_t is fed back as the next input embedding for m latent steps (m ∈ {0,10,20,40,80}, tuned per task), after a **closed-form linear realignment** W_a ≈ W_out^† W_in (ridge regression, Eq. 35, computed once per run) that maps hidden-state space to input-embedding space.
2. Between agents: a **shared latent working memory** = the **KV-caches of all L layers** of the sender agent (context + latent thoughts) are **prepended to the receiver's KV-cache**. No text, no re-encoding; only the final agent decodes text.
3. Nothing is trained ("completely training-free").
4. Receiver requirement: **no re-processing of the predecessor's text**, but the paper **assumes all agents "share the same shape of transformer layers"** (Appendix G) — in practice the same base model; heterogeneous agents would need a trainable adapter (future work). Theorem 3.3 "lossless transfer" depends on identical keys/values.

**Evaluation**
- Models: Qwen3-4B, 8B, 14B; Llama-3 3B and 8B (same model for all agents in a run).
- 9 benchmarks: GSM8K, AIME24, AIME25, GPQA-Diamond, MedQA; ARC-Easy, ARC-Challenge; MBPP-Plus, HumanEval-Plus.
- Baselines: single LLM; Sequential TextMAS (planner/critic/refiner/solver); Hierarchical TextMAS (experts + summarizer). Metrics: accuracy, total output tokens, end-to-end time.
- Headline: up to +14.6 % accuracy vs single agent, +2.8/+4.6 % vs TextMAS, 70.8–83.7 % fewer output tokens, 4×–4.3× faster (2.6×–7× vs vLLM-accelerated TextMAS).
- Theory: Thm 3.1 expressiveness (Ω(d_h·m/log|V|) tokens needed for lossless text equivalent), Thm 3.3 lossless KV transfer, Thm 3.4 complexity.

**Stated limitations**
- Homogeneous agents only (same transformer layer shapes); heterogeneous collaboration needs trainable adapters (Appendix G).
- Excessive latent steps plateau or hurt; latent steps and realignment are task/model-dependent hyperparameters.
- vLLM path is a patched, unofficial use; minor numeric differences.
- No dedicated "Limitations" section located in fetched HTML (V-partial).

---

## 5. Additional / competing work (2024–2026) — from 3 web searches + arXiv export-API verification

Verification: **arXiv-API** = title/authors/dates/abstract fetched from export.arxiv.org on 2026-10-29 (not full text); **snippet** = search snippet only.

| # | source_id | arXiv | Title | Date (v1 → latest) | Venue (Comments) | Relevance (one line) | Verif. |
|---|---|---|---|---|---|---|---|
| 1 | heo2026 | 2608.03893 | Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse (Heo, Shafipour, Zhao, Golub, Kamani, Borkar, Chandran, Zardoshti, Darvish Rouhani) | 2026-08-04 (v1) | none | Training-free per-head ridge mapper for same-family size swaps (Qwen3 14B→32B); requires matched KV head count/dim; cited by SwaT as "Closed Form". | arXiv-API |
| 2 | qu2026 | 2609.00891 | CacheBridge: Efficient Cross-Model KV Cache Transfer (Qu, Lu, Chen, Wang, Lin) | 2026-09-01 (v1) | none | Improves Heo et al. full-head mapping via architecture-indexed per-head support + attention-aligned calibration; same-family; cited by SwaT. | arXiv-API |
| 3 | yun2026 | 2609.32259 | Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs — "HeteroFold" (Yun, Lim, Yoo, Yoo, Annavaram, Karimireddy) | 2026-09-26 (v1) → 2026-09-29 (v2) | none | Direct competitor to SwaT: prefill-free cross-family transfer, frozen models, handles tokenizer/depth differences (Llama-3.1-8B→Ministral-3-14B). | arXiv-API |
| 4 | li2026 | 2608.30963 | A Universal Context-Reuse Layer for Cross-Model KV Sharing (Li, Jiang, Zhao, Li) | 2026-08-31 (v1) | none | Serving-system view of translating KV state across models differing in scale, architecture, tokenizer, family. | arXiv-API |
| 5 | shi2025 | 2510.03346 | KVComm: Enabling Efficient LLM Communication through Selective KV Sharing (Shi, Chiesa, Maguire, Kostic) | 2025-10-02 (v1) → 2026-02-22 (v3) | ICLR 2026 | Selective layer-wise KV sharing by attention importance as the inter-LLM channel (vs hidden states); same-period alternative to C2C. | arXiv-API |
| 6 | ye2025 | 2510.12872 | KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (Ye et al.) | 2025-10-14 (v1) → 2025-11-01 (v2) | NeurIPS 2025; code github.com/FastMAS/KVCOMM | Same-model multi-agent KV reuse with context-aware cache offsetting (~6.7× prefill speedup); efficiency baseline, not cross-model. | arXiv-API |
| 7 | peng2026 | 2608.13317 | StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems (Peng, Zhang, Wang, Aletras) | 2026-08-13 (v1) | COLM 2026 (18 pp.) | Training-free orthogonal alignment of sender final-layer hidden states → receiver input space (continuous prefix); cross-model latent channel without projectors. | arXiv-API |
| 8 | du2025 | 2511.09149 | Enabling Agents to Communicate Entirely in Latent Space — "Interlat" (Du et al.) | 2025-11-12 (v1) → 2026-07-13 (v5) | ACL 2026 | Last-hidden-state latent communication with learned compression; early latent-MAS baseline. | arXiv-API |
| 9 | liu2024 | 2411.02820 | DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving (Liu et al.) | 2024-11-05 (v1) → 2025-07-14 (v4) | none on arXiv (snippet: NSDI 2026) | Same-architecture cross-LLM KV reuse (recompute a few layers); cited by SwaT as same-family precedent. | arXiv-API |
| 10 | liu2026 | 2606.05711 | Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems (Yingzhuo Liu) | 2026-06-04 (v1) → 2026-07-15 (v3) | none | Survey of 18 latent-communication methods 2024–2026; useful taxonomy for related-work section. | arXiv-API |
| 11 | villani2026 | 2609.25053 | LatentPort: Beyond KV Cache — Cross-Model Transfer of Recurrent Memory in Hybrid Language Models (Villani) | 2026-09-07 (v1) | 14 pp. | Transfers GDN recurrent state + KV between Qwen3.5 4B→9B hybrid siblings without prefix replay; shows transfer beyond pure KV. | arXiv-API |
| — | (snippet only) | 2607.26773 | Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM Communication | — | — | Causal-audit methodology (ablate/replace latent message); relevant to WP30 evaluation design. | snippet |
| — | (snippet only) | 2605.28214 | Out of Sight, Not Out of Mind: Unveiling Latent Attack in Latent-based Multi-Agent Systems | — | — | Security of latent/KV handoffs. | snippet |
| — | (snippet only) | 2607.28979 / 2605.22863 | Mixture-of-Translators / Latent Cache Flow | — | — | Named in reference lists only; not fetched. | snippet |

Search URLs used: WebSearch queries "cross-model KV cache transfer translation between different LLMs arXiv 2026", "KV cache sharing between heterogeneous LLMs multi-agent communication arXiv 2025 2026 KVComm DroidSpeak", "latent communication between LLM agents hidden states instead of text arXiv 2026" (2026-10-29); IDs then verified via https://export.arxiv.org/api/query?id_list=….

---

## 6. Cross-source synthesis relevant to WP30

- **Receiver-side requirements differ sharply**: LatentMAS (same model, prepend KV, no text) < SwaT (cross-family, needs receiver token IDs of the source text but no prefill) < C2C (cross-family, needs full receiver prefill of the source text). Only Attention Matching is a pure compaction operator (same model, no text afterwards).
- **"Compaction" is orthogonal to "transfer"** in every source: SwaT is the only one that composes them (translate → AM-compact, Eq. 8) and it needs an extra query-transfer MLP to do so.
- **Hardware**: all four stacks are PyTorch + HF Transformers, evaluated on NVIDIA (A100/H100/H200; SwaT unstated). Two (compaction, LatentMAS) use vLLM optionally/for data generation. **None mention Apple Silicon, MPS or MLX** — any MacLustr reproduction is a port.
- **Licensing**: compaction MIT; C2C Apache-2.0 (+ HF checkpoints apache-2.0); LatentMAS Apache-2.0; **SwaT has no license** (reuse of its code requires author permission).
- **Checkpoints**: only C2C ships trained artifacts (7 fuser pairs, small receivers ≤8B). SwaT must be trained (~300 M params, <1 GPU-day, 2 GPUs).