Model card — MRPT (Multi-Resolution Patch Transformer), WP41 CHRONOS-X
Task
At decision time t (open of the next minute, all bars up to t complete), forecast the distribution of the next 60 one-minute log returns of one US equity or ETF during regular hours, and derived quantities at H ∈ {5, 15, 30, 60}: cumulative return, realised variance, high/low excursions, tail-event probabilities. Research model, not a product.
Inputs (strictly causal; docs/DATA_CARD.md)
- Branch A: last 390 one-minute bars (15 features each, crossing session boundaries with gap/boundary flags), patch 5 → 78 tokens.
- Branch B: last 390 completed 5-minute buckets (≈ 5 trading days), patch 3 → 130 tokens.
- Branch C: last 260 completed 30-minute buckets (20 trading days), patch 2 → 130 tokens.
- Global token: 9 history-only statistics (log σ_t, overnight gap, calendar, RV ratios, relative volume).
- Future calendar positions of the 60 target minutes (known at origin).
- No ticker identity in the core model (ablation only); no fundamentals, news, quotes.
Architecture (src/chronos_x/models)
Gated feature projection per patch (GLU) + resolution embedding + sinusoidal position (distance to origin) + temporal-gap embedding + missing-fraction scalar; pre-norm Transformer encoder (shared backbone over the concatenated tokens with key-padding masks, or per-scale encoders + cross-attention fusion); regime embedding (MLP on the global statistics, learned codebook, or none) added to the decoder queries; 2-layer Transformer decoder with 60 learned future queries + future-calendar projection, cross-attending to the encoder memory; Student-t head per future minute (μ = σ_t·a, s = σ_t·softplus(b), ν = 2.05 + softplus(c)) or Gaussian or quantile head.
| Preset | d_model | layers | heads | FFN | decoder | trainable parameters (measured) |
|---|---|---|---|---|---|---|
| tiny (tests) | 64 | 2 | 4 | 128 | 1 | 1.0e5–5e5 |
| small | 192 | 4 | 6 | 768 | 2 | see results/metrics/*/mrpt_small/train_summary.json |
| spec (brief) | 256 | 6 | 8 | 1024 | 2 | id. |
| base | 384 | 8 | 8 | 1536 | 2 | id. (target 15–30 M) |
Stage B (joint_path.py): Gaussian copula with low-rank (k = 4) + diagonal correlation over the 60 future minutes,
Student-t marginals frozen from Stage A, trained on the copula log-likelihood of the observed PITs (missing minutes
marginalised exactly), sampled without Cholesky; path scores (energy, variogram) against independent sampling of the
same marginals (H4).
Training (train/trainer.py)
Masked Student-t NLL (FP32 likelihood), AdamW (β = 0.9/0.98), linear warm-up + cosine decay to 10 %, gradient clipping
1.0, early stopping on validation NLL, fixed seeds, crash-safe last.pt/best.pt with resume, JSONL tracking of loss,
grad norm, throughput, data-loader fraction and MPS memory. Mixed precision not enabled in v1.0 (stability first;
measured FP32 throughput in results/metrics/benchmark_*.json).
Baselines (configs/baselines.yaml)
Random walk with trailing RMS scale, seasonal volatility, EWMA-t, HAR-RV (minute scale), GARCH(1,1)-t on 5-minute returns, ridge / elastic net / LightGBM on 19 origin features, MLP and TCN on the 1-minute branch with the same Student-t head, and the 1m-only MRPT (equal parameters) as the patch-transformer baseline.
Baselines not run (honest list)
- Chronos / TimesFM: pretrained weights not available offline on the compute node; their tokenisers expect a single real-valued series without masks or multi-resolution inputs; license/terms not reviewed within this project.
- iTransformer: not implemented (PatchTST-style 1m-only MRPT serves as the transformer control).
- EGARCH: multi-step forecasts need simulation; too slow for the number of evaluation origins.
Evaluation
CRPS (closed form for parametric baselines, sample-based for all), WIS, quantile loss, PIT/KS, 50/80/95 % coverage, sharpness, QLIKE on realised variance, tail-event Brier, directional accuracy, energy/variogram scores for paths; trading-day cluster bootstrap, Diebold–Mariano with HAC lag = overlap, Holm/BH; subgroups by asset class, ADV, volatility, sector, time of day, year, stress days; held-out instruments and untouched 2025 period.
Intended use and limitations
Research on intraday representation learning. Not investment advice; no claim of profitability. Trained on a survivor-heavy universe of liquid US instruments (coverage ≥ 80 %); regular hours only; no quotes, so costs are assumptions.