SPB Git

spb/wp31_uqo

Public
963 B · 65 lines yaml
1name: p4_joint_mixed_w642seed: 13mode: prisme4model:5  name: tiny_p46  vocab_size: 1287  d_model: 1928  n_layers: 49  n_heads: 610  n_kv_heads: 211  ffn_width: 51212  max_seq_len: 32013  w: 6414  ffn_w: 19215  n_slots: 1616  n_editor_heads: 217  r_max: 818init:19  backbone_checkpoint: experiments/runs/p4_tiny_backbone_seed01_20261029232626/checkpoint20  freeze_backbone: false21  auto_readout_scale: true22data:23  kind: synthetic24  tokenizer: char25  splits_dir: data/splits/synthetic_v126  loss_span: answer27  eval_splits:28  - dev_iid29  - dev_template30  - dev_depth31  - dev_compose32budgets:33- 034- 135- 236- 437- 838budget_probs:39- 0.240- 0.241- 0.242- 0.243- 0.244budget_per: position45train:46  batch_size: 3247  seq_len: 32048  steps: 300049  lr: 0.000350  warmup: 10051  weight_decay: 0.152  grad_clip: 1.053  log_every: 5054  eval_every: 50055  eval_batches: 1256  ckpt_every: 150057em_budgets:58- 059- 260- 861em_max_items: 40062model_overrides:63  readout_zero_init: false64  readout_init_std: 0.265
Files · d6746e03e0d6828cde427e384b9ef1d2b92c62d6
Menu

Theme