SPB Git

spb/wp31_uqo

Public
945 B · 65 lines yaml
1seed: 12model:3  name: tiny_p4v24  vocab_size: 1285  d_model: 2566  n_layers: 67  n_heads: 88  n_kv_heads: 49  ffn_width: 76810  max_seq_len: 32011  w: 6412  ffn_w: 19213  n_slots: 1614  n_editor_heads: 215  r_max: 816  readout_zero_init: false17  readout_init_std: 0.218init:19  backbone_checkpoint: experiments/runs/p4_tiny_backbone_v2_seed01_20261029233624/checkpoint20  freeze_backbone: true21  auto_readout_scale: true22data:23  kind: synthetic24  tokenizer: char25  splits_dir: data/splits/synthetic_v126  loss_span: answer27  eval_splits:28  - dev_iid29  - dev_template30  - dev_depth31  - dev_compose32budgets:33- 034- 135- 236- 437- 838budget_probs:39- 0.240- 0.241- 0.242- 0.243- 0.244budget_per: position45train:46  batch_size: 3247  seq_len: 32048  steps: 300049  lr: 0.00150  warmup: 10051  weight_decay: 0.152  grad_clip: 1.053  log_every: 5054  eval_every: 50055  eval_batches: 1256  ckpt_every: 300057em_budgets:58- 059- 160- 261- 862em_max_items: 100063name: p4v2_dense64mode: dense65
Files · d6746e03e0d6828cde427e384b9ef1d2b92c62d6
Menu

Theme