SPB Git

spb/wp27_uqo

Public
4.9 KB · 64 lines latex
1\section{Limitations}2\label{sec:limitations}34The results of this paper are causal statements about a simulator. Every limitation below bounds5what they can say about actual markets, and several bound what they can say about the simulator itself.67\paragraph{Internal validity.}8The confirmatory design uses seven matched seeds per live cell. With paired inference this gives980\,\% power only for standardized effects of roughly $d \approx 1.2$ (Section~\ref{sec:identification});10smaller effects are reported with their confidence intervals and must be read as inconclusive rather11than absent. The seed count was fixed by the monetary cap on API calls (about 40~USD for the whole12project), not by a pilot variance estimate, and the pilot ran concurrently with the confirmatory13design on the same frozen code revision; the pilot therefore informs cost, validity and realized-dose14reporting but did not feed back into the design. The thresholds that define bubbles and crashes were15pre-registered, and the control economy exhibits a floor effect at the crash threshold (no crashes in1630 control seeds), so crash frequencies can only increase under treatment; the sensitivity grid in17Section~\ref{sec:robustness} reports how the conclusions move with the thresholds.1819\paragraph{Provider coverage.}20All live cells use one provider family (OpenAI, \code{gpt-5.4-nano} as the primary model and21\code{gpt-5.4-mini} in the two-model and equal-budget arms). The Anthropic adapter is implemented and22exercised with the mock provider, but the key available when the experiments were run was rejected by23the API, so no Claude-populated market is reported. The cross-provider comparison that the research24questions call for is therefore reduced to a within-provider comparison of two model sizes, and the25``shared-model'' mechanism is tested through prompt diversity and the model mix rather than through26genuinely different foundation-model families. The two-model arm is compared with the single-model27diverse-prompt arm only; the mini-only cell needed to decompose a model-mix effect into a composition28effect and a diversity effect was not run.2930\paragraph{What the LLM agents are.}31An LLM agent here is a small reasoning model answering a short, fixed prompt every twenty ticks with a32strictly validated JSON action. It has no tools, no retrieval, no fine-tuning and, outside the memory33and channel arms, no state. The rationale field is logged but is never treated as evidence about the34model's internal reasoning. Decision frequency is deliberately separated from event frequency: an LLM35agent makes 27~decisions in a 600-tick run while classical agents act on roughly one tick in five, so36the realized share of orders and volume is far below the assigned share of trader slots; both are37reported and the dose--response is read on the assigned share only. Provider outputs are not38deterministic across repeated calls; recorded decisions are cached and replayed exactly, and cache hits39are counted separately from online calls. The frozen nearest-neighbour library used to scale the40population to 1,000 traders is a non-adaptive approximation of recorded behaviour and is reported as a41separate arm, never pooled with online runs.4243\paragraph{The economy.}44The exchange is a partial-equilibrium sandbox: dividends, discount rate, endowments and noise trading45are exogenous; there is no entry, no learning across runs, no short selling by default, and the46market makers follow a bounded inventory rule rather than an optimizing strategy. The stylized model of47Section~\ref{sec:theory} shares these restrictions: linear order submission, a competitive risk-neutral48market maker, Gaussian primitives and a fixed pricing schedule in the feedback extension. Its49propositions are statements about that model; the simulator tests their comparative statics, not their50closed forms. The classical population was calibrated to reproduce a short list of stylized facts51(near-zero return autocorrelation, fat tails, volatility clustering, spreads of 10--30 basis points) at52$N = 100$, and the same activity rates produce a somewhat different microstructure at $N = 1{,}000$ and53$N = 10{,}000$ unless the activity rate is scaled; all penetration experiments are run at $N = 100$.5455\paragraph{External validity.}56Simulated LLM traders are not institutions, and the market here has no regulation, no intermediaries57beyond five quoting agents, no overnight, and no cross-market linkages beyond the two-asset arm. Nothing58in the paper should be read as evidence about how deployed trading systems behave. The claim is narrower:59within a transparent, replayable economy, raising the share of traders driven by one language model60changes price discovery and coordination in the direction, and through the channel, that a61correlated-error model predicts, and those changes can be undone or amplified by interventions that62real markets can also implement. Whether the same mechanism operates at scale in live markets is an63empirical question this design cannot answer.64
Files · d09ab0c71293588caf435dcaebf6e4bc03a129ff
Menu

Theme