Data card — WP31
Synthetic compositional suite v1 (data/splits/synthetic_v1/)
Generated deterministically by data/generators/synthetic.py (SHA-256 per split in manifest.json). Six
families with programmatic answers (arith, bind, fsm, excl, sched; compose held out), five phrasing templates,
depths 1–6. Splits fixed before sampling by (family, template, depth, seed): train (seed 1, templates 0–2,
depths 1–4; 23,005 items after duplicate removal), calib (seed 3; 1,690), ctrl_train (seed 4, controller labels
only; 11,030), dev_{iid,template,depth,compose} (seed 2), test_{iid,template,depth,compose} (seed 9, LOCKED —
not read by any script until protocol/locked_test_manifest.json is unsealed). Cross-split exact duplicates are
dropped from the later split and counted. Character-level tokenizer (97 printable ASCII + 2 specials).
FineWeb-Edu subset (data/processed/fineweb_edu_300m/)
Source: HuggingFaceFW/fineweb-edu, config sample-10BT, streamed in canonical shard order on 2026-10-29
(291,534 documents streamed, truncated to 40,000 characters each); licence ODC-By 1.0 (attribution: Penedo et
al., 2024). Exact-hash deduplication; train/val/test by content hash (98/1/1 %): 289,959,006 / 2,913,824 /
2,883,436 tokens (uint16, EOT-separated). Tokenizer: byte-level BPE, 32,768 symbols, digits split individually,
trained on 60,000 train documents (tokenizer.json, SHA-256 in manifest). Test split locked. The raw text
(data/raw/) and token files are not redistributed; prepare_corpus.py regenerates them.
Counterfactual label sets
experiments/runs/labels_*/labels_{train,calib}.pt: per batch, tokens, targets, prefix actions, four logged
candidate actions with gains and log-propensities, exhaustive 24-action gains on audit batches, and the SHA-256
of the editor checkpoint that produced them. Train partition = ctrl_train stream (editor-unseen; the first
tiny set used train, see D11), calibration partition = calib.