SPB Git

spb/wp27_uqo

Public
5.8 KB

# Model cards — WP 27

One card per model or provider used or planned. Prices are USD per 1M tokens (input / cached input / output) as entered in LLMCfg.price_assumptions (experiments/config.py); source: provider pricing pages fetched 2026-10-27 (OpenAI) and Anthropic's published price table. The budget tracker (llm/budget.py) prices every call from this table; unknown model ids are priced at the most expensive known output price. Every run manifest stores the table it used.

# 1. gpt-5.4-nano (OpenAI) — primary model

Field Value
Model id / snapshot gpt-5.4-nano (alias; the dated snapshot returned in response.model is recorded per decision via request_id and the manifest)
Price 0.20 / 0.02 / 1.25
Measured usage (2026-10-27, prompt v1) ≈ 760 input, ≈ 230 output tokens per decision, of which ≈ 150 reasoning tokens at reasoning.effort = low → ≈ 0.00044 USD per decision (planning figure 0.0005)
API / format Responses API; text.format = {type: json_schema, name: trading_decision, strict: true}; instructions = frozen system prompt; store = False; prompt_cache_key = ma-v1-<prompt_id>
Reasoning / sampling reasoning.effort = low; temperature not sent (rejected by the gpt-5 family); no seed parameter
Known non-determinism Sampling at the provider; identical prompts may yield different decisions across calls; replay is exact only from the SQLite cache
Data retention store=False: responses are not kept in the account's response store; OpenAI's standard abuse-monitoring retention applies (no zero-data-retention agreement)
Role in design All dose–response, mechanism and safeguard cells; single-prompt baseline of the diversity arm; source of the frozen library
Status Live (key valid 2026-10-27)

# 2. gpt-5.4-mini (OpenAI) — second model for the mix arm

Field Value
Model id gpt-5.4-mini
Price 0.75 / 0.075 / 4.50
Usage assumption Same token profile as nano (760 / 230) → ≈ 0.0016 USD per decision; TBD from a one-seed check before T2 (reasoning tokens may differ)
API / format / reasoning As gpt-5.4-nano
Role mini_25 (model-quality control), mix_25 (two-model mix), mini_25_equalusd (fairness)
Status Live-capable; not yet called; note the engineering item on mixing two OpenAI models in one run (protocol.md §5.2 note ii)

# 3. gpt-5-nano (OpenAI) — cheap alternative / robustness

Field Value
Model id gpt-5-nano
Price 0.05 / 0.005 / 0.40
Usage assumption 760 / 230 → ≈ 0.00013 USD per decision (TBD)
API / format / reasoning As gpt-5.4-nano
Role Optional "previous-generation" replication of share_25 if budget remains after T6; not in the 40 USD plan
Status Live-capable; not planned in the confirmatory tiers

# 4. claude-haiku-5-5 (Anthropic) — planned cross-provider model

Field Value
Model id claude-haiku-5-5
Price (Anthropic published table) 0.10 / 0.01 / 0.50
Usage assumption 760 / 230 → ≈ 0.00019 USD per decision (TBD; no reasoning-token accounting in the adapter)
API / format Messages API; output_config.format = {type: json_schema, schema}; output_config.effort = low; system prompt with cache_control: ephemeral; no thinking parameter sent; stop_reason = refusal mapped to an invalid decision
Known non-determinism Provider sampling; no seed
Data retention Anthropic standard API retention (no zero-data-retention agreement)
Role Cross-provider replication of share_25 and a nano + haiku mix cell
Status Blocked — the Anthropic key available on 2026-10-27 is rejected (HTTP 401 invalid x-api-key). The adapter is implemented and the cells are dry-run with the mock provider mapped to this model id; they enter the design only with a valid key and approved budget

# 5. claude-sonnet-5-5 (Anthropic) — planned frontier-model cell

Field Value
Model id claude-sonnet-5-5
Price (Anthropic published table) 2.00 / 0.20 / 10.00
Usage assumption 760 / 230 → ≈ 0.0038 USD per decision (TBD); one share_25 run ≈ 2.6 USD
API / format / retention As claude-haiku-5-5
Role At most a 2-seed "frontier model" comparison of share_25; outside the 40 USD plan
Status Blocked (same key issue); mock dry-run only

# 6. mock-v1 (local) — deterministic stand-in

Field Value
Implementation llm/providers/mock.py: parses the canonical user message (mid, public PV, private signal, holdings, free cash) and applies a seeded value-plus-noise rule; 60 % limit / 40 % market orders; invalid_rate = 5 % (half non-JSON, half schema-invalid); herd_bias knob adds a common tilt to every mock agent
Price 0 / 0 / 0 (tokens approximated by word counts)
Determinism Fully seeded (PCG64(seed)); reproducible
Role Dry-run of every cell; CRN and validation tests; σ floors; herd_bias sweeps as a sanity check of the coordination metrics
Status Live (local). Not evidence about LLM behaviour

# 7. library (local) — frozen nearest-neighbour replay

Field Value
Implementation llm/library.py: 7 standardized features of the observation (mispricing vs public PV, momentum, signal, inventory and cash ratios, spread, open-order flag); returns the recorded decision of one of the 5 nearest neighbours (seeded draw), limit price rescaled to the current mid
Source Valid decisions recorded in the T1 gpt-5.4-nano runs (results/library/gpt-5.4-nano_single.npz)
Price 0
Role Scale arm at N = 1,000 and the "cached" side of the cached-vs-online comparison; runs are labelled provider library
Caveat Not an adaptive agent; out-of-support at large N must be reported (distance diagnostics, protocol.md §5.4)
Status Implemented; unused until T1 exists
Files · b6c1d09b402a43e723c06ff0c9a6073a51ea3fa7
Menu

Theme