# DECISIONS.md — WP30 dated implementation and scope decisions

All times America/Toronto unless noted.

# 2026-10-29

• D-001 Project location. Recherche-UQO/ had only immobilier/ and finance/ categories and no WP29/WP30 directory. WP30 is an AI-systems paper, so a new category ia/ was created: Recherche-UQO/ia/cache-compression-knowledge-transfer/ (git repo wp30_uqo, to be pushed to spbgit gitsrv:srv/git/wp30_uqo.git). WP28 (finance/distributed-mle-synchronization) is untouched; no WP29 exists in the tree (the Russell 3000 database paper is numbered FIRC 0001).
• D-002 Local backend. Laptop = Apple M5 Max, 18 cores, 48 GiB unified memory, macOS 27.0, Python 3.12.13 (uv available), mlx 0.32.2 + mlx_lm 0.31.3, torch 2.12 (MPS), transformers 5.8.1. mlx_lm exposes real KV tensors (KVCache.state = post-RoPE keys/values [B, H_kv, T, d_h]), packed QuantizedKVCache (uint32 packing + scales + biases, bits ∈ {2,3,4,6,8}, used by quantized_scaled_dot_product_attention → genuine quantized execution), save_prompt_cache/load_prompt_cache (safetensors). This is the Track B/C white-box backend. The cache.make_mask hook permits sparse-retention caches with preserved logical offsets.
• D-003 Models (pending probe). Candidate checkpoints, all ungated on HF mlx-community, bf16 weights so weight quantization is not a confound: Qwen2.5-3B-Instruct-bf16 (Qwen family), Llama-3.2-3B-Instruct-bf16 (Llama family), Qwen2.5-1.5B-Instruct-bf16 (same-family size ladder), Qwen2.5-7B-Instruct-bf16 (larger receiver, if memory allows). Final choice after models probe.
• D-004 Hosted track budget. No API key is in the shell environment; .env files of WP26/WP27 (same research folder, same author) hold Anthropic/OpenAI keys used for prior UQO papers. Track D will run only after a live probe succeeds, under a hard cap written in configs/resources.yaml (initially 25 USD total), and only on public/synthetic data. Local experiments never depend on it.
• D-005 Cluster. MacLustr nodes are used only through the existing SSH configuration (~/.ssh/config aliases) for genuine two-node cache transfer (E2/E6). No network scanning.
• D-006 Numerical tolerance for cache equivalence. Measured on Qwen2.5-1.5B cast to float32 (all 338 parameter tensors fp32): max |Δlogit| between a one-chunk prefill and a token-by-token prefill is ≈ 0.10 (logit scale ≈ 30), max |Δprob| ≈ 7e-7, KL ≈ 3e-8. MLX kernels are therefore not bitwise deterministic across chunkings even in fp32. Registered criterion for "equivalent": KL(ref‖test) < 1e-5 and max |Δprob| < 1e-4 and identical greedy top-1 in fp32; KL < 1e-3, max |Δprob| < 1e-2, identical top-1 in bf16. Identity round trip and 100 % retention must be exactly 0 (they are).
• D-007 Qwen2.5 4-bit KV finding (smoke, not a result). Both the native mlx_lm path (generate(..., kv_bits=4)) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a transport-only variant (B2b) and labelled "inspired".
• D-008 Task families. (i) Synthetic private-fact ledgers (generator wp30.data.synthetic, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench data.zip because the HF loading script is no longer supported by datasets 5.x; (iii) executable tasks (wp30.data.exec_tasks, hidden tests in a python -I subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
• D-009 Remote node. M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: ~/wp30-worker/.venv with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.