SPB Git

spb/wp27_uqo

Public
5.8 KB · 85 lines markdown
1# Model cards — WP 2723One card per model or provider used or planned. Prices are USD per 1M tokens (input / cached input / output) as entered in `LLMCfg.price_assumptions` (`experiments/config.py`); source: provider pricing pages fetched 2026-10-27 (OpenAI) and Anthropic's published price table. The budget tracker (`llm/budget.py`) prices every call from this table; unknown model ids are priced at the most expensive known output price. Every run manifest stores the table it used.45## 1. gpt-5.4-nano (OpenAI) — primary model67| Field | Value |8|---|---|9| Model id / snapshot | `gpt-5.4-nano` (alias; the dated snapshot returned in `response.model` is recorded per decision via `request_id` and the manifest) |10| Price | 0.20 / 0.02 / 1.25 |11| Measured usage (2026-10-27, prompt v1) | ≈ 760 input, ≈ 230 output tokens per decision, of which ≈ 150 reasoning tokens at `reasoning.effort = low` → ≈ 0.00044 USD per decision (planning figure 0.0005) |12| API / format | Responses API; `text.format = {type: json_schema, name: trading_decision, strict: true}`; `instructions` = frozen system prompt; `store = False`; `prompt_cache_key = ma-v1-<prompt_id>` |13| Reasoning / sampling | `reasoning.effort = low`; `temperature` not sent (rejected by the gpt-5 family); no seed parameter |14| Known non-determinism | Sampling at the provider; identical prompts may yield different decisions across calls; replay is exact only from the SQLite cache |15| Data retention | `store=False`: responses are not kept in the account's response store; OpenAI's standard abuse-monitoring retention applies (no zero-data-retention agreement) |16| Role in design | All dose–response, mechanism and safeguard cells; single-prompt baseline of the diversity arm; source of the frozen library |17| Status | **Live** (key valid 2026-10-27) |1819## 2. gpt-5.4-mini (OpenAI) — second model for the mix arm2021| Field | Value |22|---|---|23| Model id | `gpt-5.4-mini` |24| Price | 0.75 / 0.075 / 4.50 |25| Usage assumption | Same token profile as nano (760 / 230) → ≈ 0.0016 USD per decision; **TBD** from a one-seed check before T2 (reasoning tokens may differ) |26| API / format / reasoning | As gpt-5.4-nano |27| Role | `mini_25` (model-quality control), `mix_25` (two-model mix), `mini_25_equalusd` (fairness) |28| Status | Live-capable; not yet called; note the engineering item on mixing two OpenAI models in one run (`protocol.md` §5.2 note ii) |2930## 3. gpt-5-nano (OpenAI) — cheap alternative / robustness3132| Field | Value |33|---|---|34| Model id | `gpt-5-nano` |35| Price | 0.05 / 0.005 / 0.40 |36| Usage assumption | 760 / 230 → ≈ 0.00013 USD per decision (TBD) |37| API / format / reasoning | As gpt-5.4-nano |38| Role | Optional "previous-generation" replication of `share_25` if budget remains after T6; not in the 40 USD plan |39| Status | Live-capable; not planned in the confirmatory tiers |4041## 4. claude-haiku-5-5 (Anthropic) — planned cross-provider model4243| Field | Value |44|---|---|45| Model id | `claude-haiku-5-5` |46| Price (Anthropic published table) | 0.10 / 0.01 / 0.50 |47| Usage assumption | 760 / 230 → ≈ 0.00019 USD per decision (TBD; no reasoning-token accounting in the adapter) |48| API / format | Messages API; `output_config.format = {type: json_schema, schema}`; `output_config.effort = low`; system prompt with `cache_control: ephemeral`; no `thinking` parameter sent; `stop_reason = refusal` mapped to an invalid decision |49| Known non-determinism | Provider sampling; no seed |50| Data retention | Anthropic standard API retention (no zero-data-retention agreement) |51| Role | Cross-provider replication of `share_25` and a nano + haiku mix cell |52| Status | **Blocked** — the Anthropic key available on 2026-10-27 is rejected (HTTP 401 invalid x-api-key). The adapter is implemented and the cells are dry-run with the mock provider mapped to this model id; they enter the design only with a valid key and approved budget |5354## 5. claude-sonnet-5-5 (Anthropic) — planned frontier-model cell5556| Field | Value |57|---|---|58| Model id | `claude-sonnet-5-5` |59| Price (Anthropic published table) | 2.00 / 0.20 / 10.00 |60| Usage assumption | 760 / 230 → ≈ 0.0038 USD per decision (TBD); one `share_25` run ≈ 2.6 USD |61| API / format / retention | As claude-haiku-5-5 |62| Role | At most a 2-seed "frontier model" comparison of `share_25`; outside the 40 USD plan |63| Status | **Blocked** (same key issue); mock dry-run only |6465## 6. mock-v1 (local) — deterministic stand-in6667| Field | Value |68|---|---|69| Implementation | `llm/providers/mock.py`: parses the canonical user message (mid, public PV, private signal, holdings, free cash) and applies a seeded value-plus-noise rule; 60 % limit / 40 % market orders; `invalid_rate` = 5 % (half non-JSON, half schema-invalid); `herd_bias` knob adds a common tilt to every mock agent |70| Price | 0 / 0 / 0 (tokens approximated by word counts) |71| Determinism | Fully seeded (`PCG64(seed)`); reproducible |72| Role | Dry-run of every cell; CRN and validation tests; σ floors; `herd_bias` sweeps as a sanity check of the coordination metrics |73| Status | Live (local). **Not evidence about LLM behaviour** |7475## 7. library (local) — frozen nearest-neighbour replay7677| Field | Value |78|---|---|79| Implementation | `llm/library.py`: 7 standardized features of the observation (mispricing vs public PV, momentum, signal, inventory and cash ratios, spread, open-order flag); returns the recorded decision of one of the 5 nearest neighbours (seeded draw), limit price rescaled to the current mid |80| Source | Valid decisions recorded in the T1 gpt-5.4-nano runs (`results/library/gpt-5.4-nano_single.npz`) |81| Price | 0 |82| Role | Scale arm at N = 1,000 and the "cached" side of the cached-vs-online comparison; runs are labelled provider `library` |83| Caveat | Not an adaptive agent; out-of-support at large N must be reported (distance diagnostics, `protocol.md` §5.4) |84| Status | Implemented; unused until T1 exists |85
Files · b6c1d09b402a43e723c06ff0c9a6073a51ea3fa7
Menu

Theme