# DECISIONS.md — WP30 dated implementation and scope decisions

All times America/Toronto unless noted.

# 2026-10-29

• D-001 Project location. Recherche-UQO/ had only immobilier/ and finance/ categories and no WP29/WP30 directory. WP30 is an AI-systems paper, so a new category ia/ was created: Recherche-UQO/ia/cache-compression-knowledge-transfer/ (git repo wp30_uqo, to be pushed to spbgit gitsrv:srv/git/wp30_uqo.git). WP28 (finance/distributed-mle-synchronization) is untouched; no WP29 exists in the tree (the Russell 3000 database paper is numbered FIRC 0001).
• D-002 Local backend. Laptop = Apple M5 Max, 18 cores, 48 GiB unified memory, macOS 27.0, Python 3.12.13 (uv available), mlx 0.32.2 + mlx_lm 0.31.3, torch 2.12 (MPS), transformers 5.8.1. mlx_lm exposes real KV tensors (KVCache.state = post-RoPE keys/values [B, H_kv, T, d_h]), packed QuantizedKVCache (uint32 packing + scales + biases, bits ∈ {2,3,4,6,8}, used by quantized_scaled_dot_product_attention → genuine quantized execution), save_prompt_cache/load_prompt_cache (safetensors). This is the Track B/C white-box backend. The cache.make_mask hook permits sparse-retention caches with preserved logical offsets.
• D-003 Models (pending probe). Candidate checkpoints, all ungated on HF mlx-community, bf16 weights so weight quantization is not a confound: Qwen2.5-3B-Instruct-bf16 (Qwen family), Llama-3.2-3B-Instruct-bf16 (Llama family), Qwen2.5-1.5B-Instruct-bf16 (same-family size ladder), Qwen2.5-7B-Instruct-bf16 (larger receiver, if memory allows). Final choice after models probe.
• D-004 Hosted track budget. No API key is in the shell environment; .env files of WP26/WP27 (same research folder, same author) hold Anthropic/OpenAI keys used for prior UQO papers. Track D will run only after a live probe succeeds, under a hard cap written in configs/resources.yaml (initially 25 USD total), and only on public/synthetic data. Local experiments never depend on it.
• D-005 Cluster. MacLustr nodes are used only through the existing SSH configuration (~/.ssh/config aliases) for genuine two-node cache transfer (E2/E6). No network scanning.
• D-006 Numerical tolerance for cache equivalence. Measured on Qwen2.5-1.5B cast to float32 (all 338 parameter tensors fp32): max |Δlogit| between a one-chunk prefill and a token-by-token prefill is ≈ 0.10 (logit scale ≈ 30), max |Δprob| ≈ 7e-7, KL ≈ 3e-8. MLX kernels are therefore not bitwise deterministic across chunkings even in fp32. Registered criterion for "equivalent": KL(ref‖test) < 1e-5 and max |Δprob| < 1e-4 and identical greedy top-1 in fp32; KL < 1e-3, max |Δprob| < 1e-2, identical top-1 in bf16. Identity round trip and 100 % retention must be exactly 0 (they are).
• D-007 Qwen2.5 4-bit KV finding (smoke, not a result). Both the native mlx_lm path (generate(..., kv_bits=4)) and the WP30 adapter produce garbage at 4-bit/group-64 for Qwen2.5-1.5B while Llama-3.2-3B survives even 2-bit/group-32 on the smoke memo. The adapter is therefore not the cause; this is the per-token affine quantization of outlier-heavy Qwen keys (the KIVI motivation). Registered as hypothesis H2; a KIVI-inspired per-channel key quantization is scheduled as a transport-only variant (B2b) and labelled "inspired".
• D-008 Task families. (i) Synthetic private-fact ledgers (generator wp30.data.synthetic, 15 question types per document incl. unanswerable; evaluator-only keys); (ii) public LongBench v1 subsets qasper and 2wikimqa (all 200 items each; hotpotqa reserved as a 16K-extension subset), raw jsonl from THUDM/LongBench data.zip because the HF loading script is no longer supported by datasets 5.x; (iii) executable tasks (wp30.data.exec_tasks, hidden tests in a python -I subprocess). Family (ii) is likely in pretraining corpora; family (i) is not.
• D-009 Remote node. M3U96b (Mac Studio M3 Ultra, 96 GB) is the second node for genuine two-node transfer: ~/wp30-worker/.venv with the same mlx/mlx-lm pins; both 3B checkpoints downloaded there. iperf3 available on both ends for link measurement.
• D-010 Exec family difficulty (dev-only decision, 2026-10-30). With the 7-rule specification the receiver (Qwen2.5-3B) failed even with the full ticket (A1 pass rate 0 on the smoke item: it duplicated the loop instead of composing the discount rule), so the family could not discriminate between communication methods. The specification was reduced to 6 rules with two missing in the buggy code (region exclusion, tax). Hidden tests unchanged in form (5 subsets, exact float match). Datasets regenerated before any test-split evaluation.
• D-011 Network path for E2/E6 (2026-10-30). The laptop cannot open TCP connections to the nodes' LAN addresses (192.168.2.x; its SSH goes through Tailscale, whose ACL blocks non-SSH ports). Genuine two-node measurements therefore run between two cluster nodes on the wired LAN: sender/client on M3U96a (192.168.2.80), receiver worker on M3U96b (192.168.2.85). Measured link: iperf3 4 streams ≈ 940 Mbit/s (1 GbE, not 10 GbE as older notes suggested), RTT ≈ 1.2 ms. Both nodes concurrently host a CPU-light workload from another study (WP29 claude --bare executors); load average ≈ 1.8 on 32 cores at setup time; recorded as a limitation.
• D-012 Learned mapper scope (Track C). Level 2 (same family, different size): Qwen2.5-1.5B → Qwen2.5-3B share the tokenizer (token-aligned), both have 2 KV heads × 128 dims but 28 vs 36 layers → per-receiver-layer linear maps from an aligned sender layer, trained on train-split caches only (reconstruction loss; optional distillation). Level 3 (cross-family Llama-3.2-3B → Qwen2.5-3B) requires tokenizer alignment by character spans; attempted only if level 2 completes; labelled "adapted (SwaT-inspired), not a SwaT reproduction" (SwaT has no public checkpoint or licence).
• D-013 Pilot scope reduced (2026-10-30). After 120 trials the pilot ran at ≈5 trials/min (sender generations for question-aware A4/A5 dominate, GPU shared with mapper training), projecting > 12 h for 4 092 trials. Pilot reduced to 4 documents per family and 5 questions per document (1 984 trials) via --resume (trial keys unchanged). Pilot timings are not used for any claim; the core campaign runs alone on the GPU.