SPB Git

spb/wp31_uqo

Public
439 B · 19 lines yaml
1name: pretrain103m2seed: 13model: configs/prisme_103m.yaml4data:5  train_bin: data/processed/fineweb_edu_300m/train.bin6  val_bin: data/processed/fineweb_edu_300m/val.bin7train:8  batch_size: 329  seq_len: 51210  steps: 6000          # 32*512*6000 = 98.3M tokens (actual pilot budget, see DECISIONS D13)11  lr: 6.0e-412  warmup: 30013  weight_decay: 0.114  grad_clip: 1.015  log_every: 2016  eval_every: 50017  eval_batches: 2018  ckpt_every: 100019
Files · main
Menu

Theme