SPB Git

spb/wp31_uqo

Public
1.5 KB

# Model card — PRISME reference models (WP31)

Field Value
Models PRISME-tiny (2.1M / 5.6M char-level synthetic backbones + 0.17M adapters), PRISME-103M (99.5M backbone + 2.63M workspace/controller). 459M and 1.18B: configuration files and meta-device instantiation only (not trained).
Architecture Pre-norm causal decoder (RMSNorm, RoPE, GQA, SwiGLU, tied embeddings) + read-only source bank + 16-slot ephemeral workspace + shared 2-block sparse editor (24 actions + STOP, cap 8) + value controller + differential readout (docs/architecture_spec.md).
Training data tiny: synthetic suite v1 train split (23,005 items); 103M: FineWeb-Edu sample-10BT subset, 196.6M tokens (data/processed/fineweb_edu_300m/manifest.json).
Intended use Research artefact for reproducing the WP31 experiments. Not a general-purpose language model; the 103M backbone saw two tokens per parameter.
Out of scope Any deployment; generation of text for end users; evaluation on benchmarks it was not pre-registered for.
Known limitations Frozen-backbone adapters; learned routing is worse than no routing on held-out synthetic data (negative result); bank memory grows linearly with context.
Checkpoints experiments/runs/<run_id>/checkpoint/{model.safetensors, checkpoint.json} on the node that trained them (run index in reproducibility/run_index.md); laptop copies for the tiny models. Weights SHA-256 in each checkpoint.json.
Licence Code MIT; weights CC BY 4.0; corpus-derived artefacts inherit ODC-By 1.0 attribution.
Files · main
Menu

Theme