SPB Git

spb/wp31_uqo

Public
448 B · 19 lines yaml
1name: pretrain103m2seed: 13model: configs/prisme_103m.yaml4data:5  train_bin: data/processed/fineweb_edu_300m/train.bin6  val_bin: data/processed/fineweb_edu_300m/val.bin7train:8  batch_size: 329  seq_len: 51210  steps: 12000         # 32*512*12000 = 196.6M tokens (pilot budget; see protocol/preregistration.md)11  lr: 6.0e-412  warmup: 30013  weight_decay: 0.114  grad_clip: 1.015  log_every: 2016  eval_every: 50017  eval_batches: 2018  ckpt_every: 100019
Files · 20e88d79efe28fb13610284e7be1ef0ef40c265d
Menu

Theme