SPB Git

spb/wp31_uqo

Public
745 B · 44 lines yaml
1seed: 12model: configs/prisme_103m.yaml3model_overrides:4  readout_zero_init: false5  readout_init_std: 0.26  ctrl_input: h7  ctrl_use_entropy: true8init:9  backbone_checkpoint: experiments/runs/PRETRAIN103M_RUN/checkpoint10  freeze_backbone: true11  auto_readout_scale: true12data:13  kind: tokens14  train_bin: data/processed/fineweb_edu_300m/train.bin15  val_bin: data/processed/fineweb_edu_300m/val.bin16budgets:17- 018- 119- 220- 421- 822budget_probs:23- 0.224- 0.225- 0.226- 0.227- 0.228budget_per: position29train:30  batch_size: 1631  seq_len: 51232  steps: 300033  lr: 0.00134  warmup: 10035  weight_decay: 0.136  grad_clip: 1.037  log_every: 2038  eval_every: 50039  eval_batches: 1040  ckpt_every: 100041final_exact_match: false42name: editor_pilot_103m43mode: prisme44
Files · d6746e03e0d6828cde427e384b9ef1d2b92c62d6
Menu

Theme