1# Run protocol — WP 2723Version 1.0, 2026-10-27. Operational companion to `identification.md`. Every number marked TBD is filled by the pilot and recorded in the freeze file.45## 1. Environment67- Project root: `~/Desktop/Recherche-UQO/finance/invisible-hand-of-machines`; Python ≥ 3.12 in `.venv` (uv lock `uv.lock`); package `market_agents` with entry point `ma` (`.venv/bin/ma`, equivalently `.venv/bin/python -m market_agents.cli`).8- Keys in `.env` only (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`), loaded by `ma` via `python-dotenv`; never in configs or manifests. Status 2026-10-27: OpenAI key live; **Anthropic key rejected (HTTP 401 invalid x-api-key)** — Claude cells run with the mock provider only until a valid key is installed.9- OpenAI calls use the Responses API with `store=False`, `text.format = json_schema (strict)`, `reasoning.effort = low`, `prompt_cache_key` per prompt id. Anthropic calls (when enabled) use the Messages API with `output_config.format = json_schema`, `effort = low`, system prompt marked for prompt caching.10- Hard limits: project-wide live spend ≤ 40 USD; pilot ≤ 5 USD; per-run `population.llm.usd_cap` set to ≥ 2× the expected run cost so that no confirmatory run halts for budget.11- Results: `results/runs/<design>/<cell>/seed<NNNN>/` (config.yaml, manifest.json, ticks/trades/agents/llm_decisions parquet, outcomes.json) and `results/runs/<design>/outcomes.parquet`. Caches: `results/cache/<design>.sqlite` (one per design; the cache is also the recorded decision library).1213## 2. Commands1415```bash16.venv/bin/python -m pytest tests # 24 tests (unit, property, integration) must pass17.venv/bin/ma run configs/baseline.yaml --seed 1 --out results/runs/baseline/seed0001 # one classical run (~0.5 s)18.venv/bin/ma budget configs/<design>.yaml --tokens-in 760 --tokens-out 230 # calls and USD before any live design19.venv/bin/ma design configs/<design>.yaml --workers 8 --cache results/cache/<design>.sqlite20.venv/bin/ma design configs/<design>.yaml --cells share_25 --max-seeds 1 --cache ... # single cell / single seed21```2223`ma design` runs every cell in a process pool (`--workers`; the Makefile uses 3 for the pilot and 5 for the confirmatory design) with one shared SQLite decision cache (`results/cache/llm_cache.sqlite`). An existing run directory is reused only if its manifest `config_hash` equals the requested configuration's hash (otherwise it is deleted and re-run), so an interrupted design resumes without re-calling the API and a changed configuration can never pick up stale runs. Before relaunching after a code change, delete the design directory and make sure no pool worker is still alive (`pkill -f multiprocessing.spawn`).2425## 3. Phase 5a — mock dry-run (no cost)26271. Apply the gate-4 fixes listed in `research_plan.md` §3 (information parity, activation stream, confirmatory fundamentals, shock size, first commit). Re-run the test suite.282. `ma design configs/exploratory.yaml --workers 8` — all 12 cells × seeds 1–3 with the mock provider (`usd_cap: 0`). Check: every cell completes; `llm_share_traders` equals the slot share (minus market makers at 100 %); `llm_valid_rate` ≈ 0.95 (the mock injects 5 % invalid answers); `n_llm_decisions` = 27–28 per LLM agent; `n_halts` > 0 only in `breaker_25`; `reject_ratio` rises in `poslimit_25`; `ret_corr` and `spillover_dy` defined in `k2_25`.293. Dry-run the Anthropic cells with the mock provider mapped to the Claude model ids (`provider_mix: {mock: 1.0}`, `models: {mock: claude-haiku-5-5}`) to exercise the price table and manifests; label `treatment.provider: anthropic-mock`.304. Free σ estimates: run `share_0` for seeds 1–30 (classical, ≈ 15 s) to obtain the control SD of every outcome.3132## 4. Phase 5b — live pilot (OpenAI, ≤ 5 USD)3334| Cell | `llm_share` | Model / prompt | Seeds | Decisions per run | Planned USD (0.0005/decision) |35|---|---|---|---|---|---|36| share_0 | 0 | — (control, free) | 1–3 | 0 | 0 |37| share_5 | 0.05 | gpt-5.4-nano, single | 1–3 | 135 | 0.20 |38| share_25 | 0.25 | gpt-5.4-nano, single | 1–3 | 675 | 1.01 |39| share_50 | 0.50 | gpt-5.4-nano, single | 1–3 | 1,350 | 2.03 |40| **Total** | | | | 6,480 | **3.24** (measured rate 0.00044 → 2.85) |4142Design file: `configs/pilot_openai.yaml` (per-run `usd_cap` 1.5; add the free `share_0` cell so the parquet is self-contained). Steps: `ma budget` → confirm ≤ 5 USD → `ma design configs/pilot_openai.yaml --cache results/cache/pilot_openai.sqlite` → record in `results/manifests/pilot.json`: total USD, tokens in/out/reasoning per decision, latency, validity rate, failure rate, cache-hit share, realized order and volume shares, and the SD of paired differences (vs `share_0`) for every primary outcome.4344Pilot pass criteria: validity ≥ 80 %, failures ≤ 10 %, cost per decision within ±30 % of 0.0005 USD, no budget halts. Failure → one prompt revision (v2), mock re-run, one-seed re-pilot (≤ 0.7 USD). Pilot seeds 1–3 are excluded from confirmatory estimates.4546## 5. Phase 6–7 — freeze and confirmatory design4748### 5.1 Freeze (before the first confirmatory call)4950`results/manifests/freeze.json` records: git revision; SHA of `prompting.py`, `schema.py`, `outcomes.py`; `config_hash` of the confirmatory base; cell list; seed lists (confirmatory seeds 101–132, used in order); thresholds (bubble 5 %/10 ticks, crash 10 %/20 ticks; sensitivity grid 3/5/8 % and 7/10/15 %); the σ̂_d table from the pilot; the tier budget below; model ids and prices. Confirmatory base = `configs/baseline.yaml` with design-level overrides `fundamentals.jump_prob: 1.0`, `fundamentals.jump_sigma: 4.0` (provisional — chosen so that SD(V^pub/V − 1) ≈ 0.5 %; confirm at freeze), `shocks: [{time: 300, kind: fundamental_jump, size: 10.0}, {time: 450, kind: placebo}]` (+2.1 % on V).5152### 5.2 Cells and budget by tier (gpt-5.4-nano unless stated; 27 decisions per LLM agent per run; planning cost 0.0005 USD per nano decision, 0.0016 per gpt-5.4-mini decision — the mini token profile is TBD and assumed equal to nano's)5354| Tier | Cell | Overrides (on the confirmatory base) | Seeds | Live decisions | USD |55|---|---|---|---|---|---|56| T1 primary | share_0 | `population.llm_share: 0.0` | 101–130 | 0 | 0 |57| | share_25 | `llm_share: 0.25` | 101–110 | 6,750 | 3.38 |58| | share_50 | `llm_share: 0.50` | 101–110 | 13,500 | 6.75 |59| T2 diversity | diverse_25 | `llm_share: 0.25, llm.prompt_set: diverse` | 101–106 | 4,050 | 2.03 |60| | mix_25 | `llm_share: 0.25, llm.provider_mix: {openai: 1.0}` with a second OpenAI model id for half the slots (see note) | 101–106 | 2,028 nano + 2,022 mini | 4.25 |61| | mini_25 | `llm_share: 0.25, llm.models.openai: gpt-5.4-mini` | 101–104 | 2,700 mini | 4.32 |62| T3 dose shape | share_1, share_5, share_10 | `llm_share: 0.01 / 0.05 / 0.10` | 101–104 | 1,728 | 0.86 |63| | share_75, share_100 | `llm_share: 0.75 / 1.00` (95 eligible slots at 100 %) | 101–102 | 9,180 | 4.59 |64| T4 mechanisms (φ = 0.25) | memory_25 | `llm.memory: 5` | 101–103 | 2,025 | 1.01 |65| | channel_25 | `llm.channel: true` (cohorts of 5) | 101–103 | 2,025 | 1.01 |66| | leverage_25 | `population.mix.leveraged: 0.19, trend: 0.06` (+ free `leverage_0` control at φ = 0 for the E5 interaction) | 101–103 | 2,025 | 1.01 |67| | breaker_25 | `exchange.halt_threshold: 0.05` (10-tick window, 20-tick halt) | 101–103 | 2,025 | 1.01 |68| | poslimit_25 | `exchange.position_limit: 300` | 101–102 | 1,350 | 0.68 |69| T5 fairness | mini_25_equalusd | mini_25 with `llm.usd_cap` = mean nano spend of share_25 (TBD ≈ 0.34) → decisions stop at equal USD | 101–103 | ≤ 630 mini | ≤ 1.01 |70| T6 extensions | k2_25 | two assets, `factor_loading: [0.7, 0.7]`, `statarb: 0.05` | 101–102 | 1,350 | 0.68 |71| | library_1000 | `population.n_traders: 1000`, `llm.frozen_library: results/library/gpt-5.4-nano_single.npz`, provider `library` | 101–110 | 0 | 0 |72| | auction_25, interval_10 | call auction every 5 ticks; `decision_interval: 10` — **mock only** | 1–10 | 0 | 0 |73| **Confirmatory total** | | | | ≈ 66,000 | **≈ 33.3** |74| Pilot (§4) | | | | 6,480 | 3.24 |75| Contingency (retries, longer outputs) | | | | | ≈ 3.5 (≈ 9 %) |76| **Project cap** | | | | | **40.00** |7778Notes. (i) Equal-decisions fairness is the default (`decision_cap` unset: every LLM agent gets the same 27 opportunities); equal-USD uses the per-run `usd_cap` and therefore applies to single-provider cells only — the code has no per-provider cap. (ii) The two-model mix: `provider_mix` keys are providers, not models, and `models` maps one model per provider; mixing two OpenAI models in one run requires either a second provider alias (e.g. `openai_mini`) or a per-slot model list — a gate-4 engineering item; until then `mix_25` is dry-run with `{mock: 0.5, openai: 0.5}`. (iii) Claude cells (haiku_25 at ≈ 0.19 USD per 1,000 decisions; sonnet at ≈ 3.8 USD per 1,000) are planned only if a valid key *and* additional budget are approved before the freeze; they are not in the 40 USD plan. (iv) `ma budget` reproduces this table with `--tokens-in 760 --tokens-out 230`.7980### 5.3 Execution order and order of sacrifice8182Tiers run T1 → T6; spend is reconciled after each tier (`sum(llm_usd)` over `outcomes.parquet` plus the provider dashboard). If the cumulative spend exceeds the plan by > 10 %, seeds are cut in this fixed order: T6 → T5 → T3 (share_75/100 first) → T4 poslimit → T2 mini_25 (E3(b) then unavailable) → T4 others. T1 is never cut below 8 seeds. Nothing is added after seeing confirmatory results.8384### 5.4 Frozen-library replay (T6)8586Build the library from the T1 nano runs: `DecisionLibrary.from_runs([...T1 run dirs...]).save("results/library/gpt-5.4-nano_single.npz")`. Replay at N = 1,000 with `activity` scaled by 100/N (as in `scripts/11_scale_benchmark.py`) so order flow per tick is constant. Report the nearest-neighbour distance distribution; if the median standardized distance at N = 1,000 exceeds the 90th percentile observed at N = 100, the arm is reported as out-of-support, not as evidence.8788## 6. Replication checklist89901. `git log -1` matches `freeze.json`; `pytest` passes (24 tests).912. `ma design` of each tier reproduces `outcomes.parquet` from the cache (`--cache` path, `llm_cached_hits` = 1.0, zero USD).923. Every run directory has `manifest.json` with `config_hash`, `git_revision`, `prompt_version`, `price_assumptions`, `llm_usage`, `shocks`, `final_conservation_ok`.934. Analysis scripts read only `outcomes.parquet`, `ticks.parquet`, `trades.parquet`, `llm_decisions.parquet`; tables and figures are regenerated by one command (`paper/Makefile`).945. The sensitivity grid (bubble 3/5/8 %, crash 7/10/15 %) is produced by re-running `run_outcomes` with the alternative thresholds on saved ticks, not by re-simulating.956. Decision logs are published without raw prompts if they contain nothing private (they do not: the prompt is a canonical market snapshot); model ids, dates and prices are listed in `model_cards.md`.967. Deviations from this protocol are listed in the paper's appendix with the date and reason.97