SPB Git

spb/wp31_uqo

Public
758 B · 44 lines yaml
1seed: 12model: configs/prisme_103m.yaml3model_overrides:4  readout_zero_init: false5  readout_init_std: 0.26  ctrl_input: h7  ctrl_use_entropy: true8init:9  backbone_checkpoint: experiments/runs/pretrain103m_seed01_20261029232706/checkpoint10  freeze_backbone: true11  auto_readout_scale: true12data:13  kind: tokens14  train_bin: data/processed/fineweb_edu_300m/train.bin15  val_bin: data/processed/fineweb_edu_300m/val.bin16budgets:17- 018- 119- 220- 421- 822budget_probs:23- 0.224- 0.225- 0.226- 0.227- 0.228budget_per: position29train:30  batch_size: 831  seq_len: 51232  steps: 30033  lr: 0.00134  warmup: 10035  weight_decay: 0.136  grad_clip: 1.037  log_every: 1038  eval_every: 15039  eval_batches: 1040  ckpt_every: 30041final_exact_match: false42name: dense_probe_103m43mode: dense44
Files · main
Menu

Theme