# DECISIONS.md — WP30 dated implementation and scope decisions

All times America/Toronto unless noted.

## 2026-10-29

- **D-001 Project location.** `Recherche-UQO/` had only `immobilier/` and `finance/` categories and no WP29/WP30 directory. WP30 is an AI-systems paper, so a new category `ia/` was created: `Recherche-UQO/ia/cache-compression-knowledge-transfer/` (git repo `wp30_uqo`, to be pushed to spbgit `gitsrv:srv/git/wp30_uqo.git`). WP28 (`finance/distributed-mle-synchronization`) is untouched; no WP29 exists in the tree (the Russell 3000 database paper is numbered FIRC 0001).
- **D-002 Local backend.** Laptop = Apple M5 Max, 18 cores, 48 GiB unified memory, macOS 27.0, Python 3.12.13 (uv available), mlx 0.32.2 + mlx_lm 0.31.3, torch 2.12 (MPS), transformers 5.8.1. mlx_lm exposes real KV tensors (`KVCache.state` = post-RoPE keys/values `[B, H_kv, T, d_h]`), packed `QuantizedKVCache` (uint32 packing + scales + biases, bits ∈ {2,3,4,6,8}, used by `quantized_scaled_dot_product_attention` → genuine quantized execution), `save_prompt_cache`/`load_prompt_cache` (safetensors). This is the Track B/C white-box backend. The `cache.make_mask` hook permits sparse-retention caches with preserved logical offsets.
- **D-003 Models (pending probe).** Candidate checkpoints, all ungated on HF `mlx-community`, bf16 weights so weight quantization is not a confound: `Qwen2.5-3B-Instruct-bf16` (Qwen family), `Llama-3.2-3B-Instruct-bf16` (Llama family), `Qwen2.5-1.5B-Instruct-bf16` (same-family size ladder), `Qwen2.5-7B-Instruct-bf16` (larger receiver, if memory allows). Final choice after `models probe`.
- **D-004 Hosted track budget.** No API key is in the shell environment; `.env` files of WP26/WP27 (same research folder, same author) hold Anthropic/OpenAI keys used for prior UQO papers. Track D will run only after a live probe succeeds, under a hard cap written in `configs/resources.yaml` (initially 25 USD total), and only on public/synthetic data. Local experiments never depend on it.
- **D-005 Cluster.** MacLustr nodes are used only through the existing SSH configuration (`~/.ssh/config` aliases) for genuine two-node cache transfer (E2/E6). No network scanning.
- **D-006 Numerical tolerance for cache equivalence.** Measured on Qwen2.5-1.5B cast to float32 (all 338 parameter tensors fp32): max |Δlogit| between a one-chunk prefill and a token-by-token prefill is ≈ 0.10 (logit scale ≈ 30), max |Δprob| ≈ 7e-7, KL ≈ 3e-8. MLX kernels are therefore not bitwise deterministic across chunkings even in fp32. Registered criterion for "equivalent": KL(ref‖test) < 1e-5 and max |Δprob| < 1e-4 and identical greedy top-1 in fp32; KL < 1e-3, max |Δprob| < 1e-2, identical top-1 in bf16. Identity round trip and 100 % retention must be exactly 0 (they are).
- **D-007 Qwen2.5 4-bit KV finding (smoke, not a result).** Both the native mlx_lm path (`generate(..., kv_bits=4)`) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a *transport-only* variant (B2b) and labelled "inspired".
- **D-008 Task families.** (i) Synthetic private-fact ledgers (generator `wp30.data.synthetic`, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench `data.zip` because the HF loading script is no longer supported by `datasets` 5.x; (iii) executable tasks (`wp30.data.exec_tasks`, hidden tests in a `python -I` subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
- **D-009 Remote node.** M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: `~/wp30-worker/.venv` with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.
- **D-010 Exec family difficulty (dev-only decision, 2026-10-30).** With the 7-rule specification the receiver (Qwen2.5-3B) failed even with the full ticket (A1 pass rate 0 on the smoke item: it duplicated the loop instead of composing the discount rule), so the family could not discriminate between communication methods. The specification was reduced to 6 rules with two missing in the buggy code (region exclusion, tax). Hidden tests unchanged in form (5 subsets, exact float match). Datasets regenerated before any test-split evaluation.
- **D-011 Network path for E2/E6 (2026-10-30).** The laptop cannot open TCP connections to the nodes' LAN addresses (192.168.2.x; its SSH goes through Tailscale, whose ACL blocks non-SSH ports). Genuine two-node measurements therefore run **between two cluster nodes on the wired LAN**: sender/client on M3U96a (192.168.2.80), receiver worker on M3U96b (192.168.2.85). Measured link: iperf3 4 streams ≈ 940 Mbit/s (1 GbE, not 10 GbE as older notes suggested), RTT ≈ 1.2 ms. Both nodes concurrently host a CPU-light workload from another study (WP29 `claude --bare` executors); load average ≈ 1.8 on 32 cores at setup time; recorded as a limitation.
- **D-012 Learned mapper scope (Track C).** Level 2 (same family, different size): Qwen2.5-1.5B → Qwen2.5-3B share the tokenizer (token-aligned), both have 2 KV heads × 128 dims but 28 vs 36 layers → per-receiver-layer linear maps from an aligned sender layer, trained on train-split caches only (reconstruction loss; optional distillation). Level 3 (cross-family Llama-3.2-3B → Qwen2.5-3B) requires tokenizer alignment by character spans; attempted only if level 2 completes; labelled "adapted (SwaT-inspired), not a SwaT reproduction" (SwaT has no public checkpoint or licence).