SPB Git

spb/wp31_uqo

Public
968 B · 66 lines yaml
1seed: 12model:3  name: tiny_p4v24  vocab_size: 1285  d_model: 2566  n_layers: 67  n_heads: 88  n_kv_heads: 49  ffn_width: 76810  max_seq_len: 32011  w: 6412  ffn_w: 19213  n_slots: 1614  n_editor_heads: 215  r_max: 816  readout_zero_init: false17  readout_init_std: 0.218  n_editor_blocks: 119init:20  backbone_checkpoint: experiments/runs/p4_tiny_backbone_v2_seed01_20261029233624/checkpoint21  freeze_backbone: true22  auto_readout_scale: true23data:24  kind: synthetic25  tokenizer: char26  splits_dir: data/splits/synthetic_v127  loss_span: answer28  eval_splits:29  - dev_iid30  - dev_template31  - dev_depth32  - dev_compose33budgets:34- 035- 136- 237- 438- 839budget_probs:40- 0.241- 0.242- 0.243- 0.244- 0.245budget_per: position46train:47  batch_size: 3248  seq_len: 32049  steps: 300050  lr: 0.00151  warmup: 10052  weight_decay: 0.153  grad_clip: 1.054  log_every: 5055  eval_every: 50056  eval_batches: 1257  ckpt_every: 300058em_budgets:59- 060- 161- 262- 863em_max_items: 100064name: p4v2_abl_L165mode: prisme66
Files · main
Menu

Theme