# DECISIONS.md — WP30 dated implementation and scope decisions

All times America/Toronto unless noted.

## 2026-10-29

- **D-001 Project location.** `Recherche-UQO/` had only `immobilier/` and `finance/` categories and no WP29/WP30 directory. WP30 is an AI-systems paper, so a new category `ia/` was created: `Recherche-UQO/ia/cache-compression-knowledge-transfer/` (git repo `wp30_uqo`, to be pushed to spbgit `gitsrv:srv/git/wp30_uqo.git`). WP28 (`finance/distributed-mle-synchronization`) is untouched; no WP29 exists in the tree (the Russell 3000 database paper is numbered FIRC 0001).
- **D-002 Local backend.** Laptop = Apple M5 Max, 18 cores, 48 GiB unified memory, macOS 27.0, Python 3.12.13 (uv available), mlx 0.32.2 + mlx_lm 0.31.3, torch 2.12 (MPS), transformers 5.8.1. mlx_lm exposes real KV tensors (`KVCache.state` = post-RoPE keys/values `[B, H_kv, T, d_h]`), packed `QuantizedKVCache` (uint32 packing + scales + biases, bits ∈ {2,3,4,6,8}, used by `quantized_scaled_dot_product_attention` → genuine quantized execution), `save_prompt_cache`/`load_prompt_cache` (safetensors). This is the Track B/C white-box backend. The `cache.make_mask` hook permits sparse-retention caches with preserved logical offsets.
- **D-003 Models (pending probe).** Candidate checkpoints, all ungated on HF `mlx-community`, bf16 weights so weight quantization is not a confound: `Qwen2.5-3B-Instruct-bf16` (Qwen family), `Llama-3.2-3B-Instruct-bf16` (Llama family), `Qwen2.5-1.5B-Instruct-bf16` (same-family size ladder), `Qwen2.5-7B-Instruct-bf16` (larger receiver, if memory allows). Final choice after `models probe`.
- **D-004 Hosted track budget.** No API key is in the shell environment; `.env` files of WP26/WP27 (same research folder, same author) hold Anthropic/OpenAI keys used for prior UQO papers. Track D will run only after a live probe succeeds, under a hard cap written in `configs/resources.yaml` (initially 25 USD total), and only on public/synthetic data. Local experiments never depend on it.
- **D-005 Cluster.** MacLustr nodes are used only through the existing SSH configuration (`~/.ssh/config` aliases) for genuine two-node cache transfer (E2/E6). No network scanning.
- **D-006 Numerical tolerance for cache equivalence.** Measured on Qwen2.5-1.5B cast to float32 (all 338 parameter tensors fp32): max |Δlogit| between a one-chunk prefill and a token-by-token prefill is ≈ 0.10 (logit scale ≈ 30), max |Δprob| ≈ 7e-7, KL ≈ 3e-8. MLX kernels are therefore not bitwise deterministic across chunkings even in fp32. Registered criterion for "equivalent": KL(ref‖test) < 1e-5 and max |Δprob| < 1e-4 and identical greedy top-1 in fp32; KL < 1e-3, max |Δprob| < 1e-2, identical top-1 in bf16. Identity round trip and 100 % retention must be exactly 0 (they are).
- **D-007 Qwen2.5 4-bit KV finding (smoke, not a result).** Both the native mlx_lm path (`generate(..., kv_bits=4)`) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a *transport-only* variant (B2b) and labelled "inspired".
- **D-008 Task families.** (i) Synthetic private-fact ledgers (generator `wp30.data.synthetic`, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench `data.zip` because the HF loading script is no longer supported by `datasets` 5.x; (iii) executable tasks (`wp30.data.exec_tasks`, hidden tests in a `python -I` subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
- **D-009 Remote node.** M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: `~/wp30-worker/.venv` with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.