SPB Git

spb/wp31_uqo

Public
1.9 KB · 25 lines markdown
1# Data card — WP3123## Synthetic compositional suite v1 (`data/splits/synthetic_v1/`)4Generated deterministically by `data/generators/synthetic.py` (SHA-256 per split in `manifest.json`). Six5families with programmatic answers (arith, bind, fsm, excl, sched; compose held out), five phrasing templates,6depths 1–6. Splits fixed before sampling by (family, template, depth, seed): train (seed 1, templates 0–2,7depths 1–4; 23,005 items after duplicate removal), calib (seed 3; 1,690), ctrl_train (seed 4, controller labels8only; 11,030), dev_{iid,template,depth,compose} (seed 2), test_{iid,template,depth,compose} (seed 9, LOCKED —9not read by any script until `protocol/locked_test_manifest.json` is unsealed). Cross-split exact duplicates are10dropped from the later split and counted. Character-level tokenizer (97 printable ASCII + 2 specials).1112## FineWeb-Edu subset (`data/processed/fineweb_edu_300m/`)13Source: `HuggingFaceFW/fineweb-edu`, config `sample-10BT`, streamed in canonical shard order on 2026-10-2914(291,534 documents streamed, truncated to 40,000 characters each); licence ODC-By 1.0 (attribution: Penedo et15al., 2024). Exact-hash deduplication; train/val/test by content hash (98/1/1 %): 289,959,006 / 2,913,824 /162,883,436 tokens (uint16, EOT-separated). Tokenizer: byte-level BPE, 32,768 symbols, digits split individually,17trained on 60,000 train documents (`tokenizer.json`, SHA-256 in manifest). Test split locked. The raw text18(`data/raw/`) and token files are not redistributed; `prepare_corpus.py` regenerates them.1920## Counterfactual label sets21`experiments/runs/labels_*/labels_{train,calib}.pt`: per batch, tokens, targets, prefix actions, four logged22candidate actions with gains and log-propensities, exhaustive 24-action gains on audit batches, and the SHA-25623of the editor checkpoint that produced them. Train partition = `ctrl_train` stream (editor-unseen; the first24tiny set used `train`, see D11), calibration partition = `calib`.25
Files · main
Menu

Theme