1\begin{footnotesize}\begin{verbatim}2# WP30 -- Research questions and falsifiable hypotheses (registered before any held-out3 outcome was examined)45Registered: 2026-10-29 (Phase 1). Amendments only as dated entries at the bottom.6Test-split outcomes had **not** been examined at registration (no test split existed yet;7 only the E0 smoke run on a8hand-written 3-sentence memo had been executed, see `runs/logs/`).910## Research questions1112- **RQ1 (compression frontier)** How do text shortening (A2-A4, A8), packed KV quantization13 (B2), token eviction (B3-B6) and14 learned compaction (B7, if run) trade task quality against *measured* serialized bytes and15 resident cache bytes?16- **RQ2 (transfer compatibility)** How much performance is preserved by same-checkpoint17 reuse (B1, cross-process, cross-node)18 versus learned mappings between distinct checkpoints (Track C), relative to the receiver's19 own full-context answer?20- **RQ3 (resource accounting)** Do rankings of methods under a token budget change when the21 budget is bytes, end-to-end latency,22 peak memory or amortized cost over N reuses?23- **RQ4 (repeated handoffs)** How do correct-fact retention, unsupported additions and task24 accuracy evolve after 1, 2, 4, 8 handoffs,25 for prose summaries vs structured evidence, in closed vs repairable chains?26- **RQ5 (evidence)** Under an equal byte budget, does structured evidence with source spans27 (A5) retain exact facts (identifiers,28 numbers, dates, negations) better than an abstractive summary (A4)? Does metered retrieval29 (A6) repair losses, and at what cost?30- **RQ6 (locality)** When is running the next query where the cache resides cheaper than31 moving the cache or recomputing from text,32 as a function of context length, payload bytes, measured bandwidth and queue delay?33- **RQ7 (adaptive policy)** Can a router fitted on development data improve the held-out34 quality/cost frontier over the best35 development-selected static method, once routing overhead and fallbacks are counted?3637## Hypotheses (each may be rejected)3839- **H1** Lossy compression degrades exact identifiers, decimal numbers, negation and40 temporal order more than paraphrasable content:41 on the synthetic private-fact family, the accuracy drop from A1 to A4 at a 512-token42 budget is larger (>= 10 percentage points)43 for `exact_copy`, `negation`, and `revision` question types than for `single_fact`44 questions.45- **H2** Packed KV quantization is model-dependent: at 4 bits/group 64, Llama-3.2-3B retains46 >= 95 % of its B0 accuracy while47 Qwen2.5-3B loses >= 20 percentage points (motivated by E0 smoke: Qwen2.5-1.5B produced48 garbage at 4-bit in both the native49 mlx_lm path and ours; Llama-3.2-3B did not). 8-bit retains >= 97 % for both.50- **H3** Sink+recent eviction (B3) at <= 1/4 retention is worse than a *question-aware*51 attention-informed selection (B5-aware)52 at equal retained positions, but **not** reliably better than random selection (B4) on53 questions whose answer lies in the54 middle of the document.55- **H4** Serialized cache bytes exceed raw UTF-8 document bytes by >= 2 orders of magnitude56 for 3B models; hence B1 transfer only57 wins on total latency when (a) the receiver's prefill cost exceeds transport time, which58 on the measured LAN requires contexts59 above a break-even length we will estimate (prediction: several thousand tokens), or (b)60 the cache is reused >= N* times.61- **H5** Repeated prose handoffs lose facts monotonically; after 4 handoffs the62 retained-correct-fact rate of A4 chains falls below63 70 % while A5 (structured, explicit unknowns) stays above 85 %; unsupported additions64 appear mainly in prose chains.65- **H6** Metered retrieval (A6, repairable chains) restores most of the lost exact facts but66 adds fetched bytes comparable to67 the message itself.68- **H7** Question-aware compression dominates question-blind compression at equal budget for69 *all* lossy methods; the gap is70 largest for attention-based selection (which is query-dependent by construction).71- **H8** A lossless transport codec (A7, zstd) on text yields >= 2x byte savings with < 5 ms72 overhead but **no** change in receiver73 latency or quality; on bf16 KV tensors the codec yields < 1.3x (high-entropy payload).74- **H9** The adaptive router (E8) will not reliably beat the best development-selected75 static method on held-out data once76 overhead is included (we expect a null or small, uncertain gain).77- **H10** Cross-family cache transfer without a trained mapper is not possible; with a78 mapper trained under our budget (if feasible),79 quality is below the receiver's own A4 summary baseline (we expect learned transfer to80 lose to text at our scale).8182## Primary endpoints (frozen at Phase 7 together with the matrix)8384Per task family: exact-match / F1 (LongBench normalization) or numeric correctness with85 tolerance; operational success rate;86serialized payload bytes; end-to-end wall time (defined boundaries, ?6.1 of the brief); peak87 MLX memory.8889## Primary confirmatory comparison family (Holm-corrected; everything else exploratory)90911. A5 vs A4 at matched byte budget (synthetic, question-blind) -- accuracy.922. B2-8bit vs B0 -- accuracy (both models).933. B2-4bit vs B0 -- accuracy (Qwen vs Llama interaction stated as H2).944. B5-aware vs B3 at 1/4 retention -- accuracy.955. A4 chain vs A5 chain at 4 handoffs -- fact retention.966. Router vs best static -- held-out accuracy at equal or lower cost.9798## Amendments99100(none yet)101102- **A-001 (2026-10-30, before freeze, after pilot on dev only).** Core matrix bounded for103 run time: 40 test documents per family (synthetic and104 exec = all; public = first 40 in manifest order), 5 questions per synthetic document,105 question-aware A4/A5 limited to the first 3 questions106 of synthetic documents only; one sender budget (512 tokens) and one retention ratio (1/4)107 in the registered comparisons; B2 at 8 and 4 bits108 (2-bit dropped from the core: pilot showed total failure on Qwen and no additional109 information). Campaign sharded by model pair across two110 machines (D-015). Track C uses the ridge mapper v2 (D-016). None of these changes used111 test outcomes.112- **A-002 (2026-10-30).** Pilot observation recorded without changing hypotheses: on dev, A5113 (structured) was *below* A4 (summary) at the same114 budget; H1/T1 are kept as registered and may be rejected on test.115- **A-003 (2026-10-30 06:40 UTC, after test outcomes of the Qwen shard were examined).**116 Non-scientific code changes after the freeze, listed117 for transparency: (i) `runner.py`/`chains.py`/`fanin.py`: `hardware_id` derived from the118 hostname instead of a laptop constant (records119 produced on cluster nodes before the fix are relabelled by120 `scripts/collect_remote_results.sh`); (ii) `reproduce.py`: trial keys of the121 reproduction use the frozen protocol hash; (iii) `router.py`/`router_eval.py`: the static122 baseline for the question-aware regime is the123 dev-best *aware-eligible* action (`A3|512|aware` on dev) instead of the blind dev-best124 (`A1`), and an explicit `B5-aware` static baseline was125 added -- the router's own decision rule, feature set, cost column, tau and dev-fitted126 table are unchanged. None of these changes alters any127 trial key, prompt, budget, scorer, dataset, or hypothesis; `protocol/frozen.json` keeps128 the v1 hash used in all trial keys, and the129 post-freeze file hashes are recorded in `protocol/frozen_v1.1.json` (the campaign configs130 differ from v1 only by the injected `protocol_hash` field, written immediately after the131 freeze).132- **A-004 (2026-10-30 08:40 UTC, EXPLORATORY, after core outcomes were examined).** The133 5-question core limit (A-001) kept only the first five134 generator questions per synthetic document (2x single fact, 2x exact copy, 1x multi-hop),135 so H1's negation/revision/aggregation/unanswerable136 types were not covered by the confirmatory run. An exploratory extension `E7_qtypes` runs137 questions 6-15 of the 40 synthetic test documents138 for A1, A3-blind, A4-blind, A5-blind, B2-8bit, B3 and B5-aware on both pairs. It is139 labelled exploratory everywhere; no confirmatory test140 is attached to it. Also recorded: A8 (LLMLingua-2) failed with `implementation_error` on141 M3U96a for the Llama pair (no torch in that142 environment); those 240 trials are retried on the laptop as new attempts143 (`core_llama_a8_retry.yaml`), both attempts kept.144\end{verbatim}\end{footnotesize}145