1% paper/sections/identification.tex — UQO WP 27, "The Invisible Hand of Machines"2% Section: Experimental identification. Mirrors docs/identification.md, docs/protocol.md and the3% deviations recorded in docs/research_log.md. Hypotheses H1–H6 are those of Table~\ref{tab:predictions}.4\section{Experimental identification}5\label{sec:identification}67The simulator is an experimental apparatus and we describe it as one. Causal language refers to effects8\emph{inside the simulator}: the object identified is the effect of replacing a share of rule-based9trader slots by LLM-controlled slots in this market mechanism, with these model snapshots and these10prompts.1112\subsection{Unit of observation and randomization}13\label{sec:identification:unit}1415The unit of observation is one run: $N=100$ trader slots simulated for 600 ticks, with outcomes16computed on $t\ge100$. A run is indexed by a seed $s$ and a cell $z$, a vector of design settings---the17LLM share $\phi$ (\code{llm\_share}), the provider and model mix, the prompt set, memory, the18communication channel, the leverage composition, the safeguards and the fairness regime---and $Y_s(z)$19is the potential outcome of seed $s$ under cell $z$. All exogenous randomness derives from a master20seed through independent named streams: \code{fundamentals} (dividend path), \code{news}, \code{slots}21(wealth multipliers, latencies, informed draws), \code{assign} (the classical policy of each slot),22\code{llm\_assign} (which eligible slots become LLM-controlled, their provider and decision offsets),23\code{activation}, \code{tiebreak}, \code{shuffle} (within-tick processing order), and one generator24per agent for policy randomness and private-signal noise. Slot assignment is stratified: slots are25ranked by wealth multiplier and the policy types are dealt down the ranking, so each wealth tercile26receives the same policy composition. The treatment replaces a fraction $\phi$ of eligible slots with27proportional quotas within each policy type; market makers are never replaced, so liquidity supply is28held fixed and $\phi=1$ means 95 of 100 slots. A replaced slot keeps its cash, shares, latency29and---after the change described in Section~\ref{sec:identification:changes}---its information set: an30LLM agent is informed, at the same precision, if and only if the slot it replaces was. The treatment31changes the decision policy of a stratified subset of slots and nothing else.3233In a matched pair $\{(s,0),(s,z)\}$ the dividend path and latent value, the news, the shock at $t=300$34and the placebo at $t=450$, every endowment, latency and informed status, the policy of every35non-replaced slot, the activation and tie-break draws, and the random numbers of every non-replaced36agent are identical by construction. What is \emph{not} held fixed is what the treatment is meant to37change: the decisions of the LLM slots, including the provider's sampling noise, which is not seedable38and is treated as part of the treatment, and the market's reaction to them---the book, the fills of39every other agent, and hence the subsequent decisions of the rule-based agents, which condition on the40book. Common random numbers remove the variance of the environment from the paired difference, not41the equilibrium response of the market.4243\subsection{Causal structure}44\label{sec:identification:dag}4546The cell label sets the treatment $Z$; with the \code{llm\_assign} stream and the provider noise, $Z$47determines the LLM slot set and its policies. Those policies generate, inside the run, a set of48mediators $M$---pairwise order-flow correlation among LLM agents, synchrony, aggressiveness, validity49rate and realized dose---and outcomes $Y$ are produced jointly by $M$, by the exogenous streams, and by50a direct path from $Z$ that bypasses LLM behaviour (fewer heuristic agents, halts, position limits). The51effect of $Z$ on $Y$ is identified because $Z$ is set by the design file and the only random input52that depends on it, \code{llm\_assign}, is independent of every other stream by construction; the53effect of $Z$ on each mediator is identified for the same reason, which is how H2 and H5 are tested.54Mediation is \emph{not} identified: $M$ and $Y$ are generated by the same path of draws and provider55noise, so sequential ignorability---independence of $M$ and the potential outcomes given $Z$---fails;56a volatile fundamental path raises both the correlation of LLM orders and the pricing error. We report57mediation only as a descriptive decomposition and treat each mediator as an outcome. $Z$ also bundles58model, prompt, decision frequency and the rule that invalid decisions become holds; the estimand is59intention-to-treat, ``deploy this agent as configured''.6061\subsection{Estimands}62\label{sec:identification:estimands}6364Let $\tau_z=\E_s[Y_s(z)-Y_s(0)]$ be the paired average treatment effect of cell $z$ against the control65of the same seed. In order of priority: (E1) $\tau_z$ at each share against $\phi=0$, the primary66confirmatory contrast being $\phi=0.25$ with \code{gpt-5.4-nano} and the single prompt, $\phi=0.50$ the67secondary; (E2) the dose--response $\tau(\phi)$ over $\phi\in\{0.05,0.10,0.25,0.50,1.00\}$, summarized68by a linear slope and an ordered trend test; (E3) diversity at equal share,69$\E[Y(\text{diverse},0.25)-Y(\text{single},0.25)]$ for prompt diversity at a fixed model and70$\E[Y(\text{mix},0.25)-Y(\text{diverse},0.25)]$ for a two-model mix against the single-model71diverse-prompt cell; (E4) cognition contrasts at $\phi=0.25$---memory of five decisions, a rationale72channel in cohorts of five, adaptive profit-and-loss feedback---each against the single-prompt cell at73the same share; (E5) the interaction with leverage,74$[Y(0.25,\text{lev}_{\text{hi}})-Y(0,\text{lev}_{\text{hi}})]-[Y(0.25,\text{lev}_{\text{base}})-Y(0,\text{lev}_{\text{base}})]$,75and the safeguard contrasts (circuit breaker, position limit) at $\phi=0.25$; (E6) the fairness76contrasts, a more capable model at the share that equalizes the dollar budget with the primary cell,77and a matched-latency cell in which LLM orders arrive with the minimum latency. The leveraged control78at $\phi=0$ is a classical run, free of API cost and deterministic given the seed, so it is produced on79the matched seeds and the second difference of E5 is also within-seed. Two further objects are80descriptive: a local projection of the log price--value gap around the latent shock, and $\tau_z$81divided by the mean realized order share, a rescaling and not a causal effect per order.8283\subsection{Pre-registered outcomes, hypotheses and thresholds}84\label{sec:identification:outcomes}8586Table~\ref{tab:outcomes} lists the outcome families, their lead variables and the sign predicted by the87hypotheses of Table~\ref{tab:predictions}. The coordination statistics exist only when $\phi>0$, so H288is tested as a dose--response among treated cells plus a comparison with the same statistic computed on89the aggressor-signed trades of classical agents of the replaced types. Thresholds were fixed before any90live call: a bubble is $|P/V-1|>5\,\%$ for at least ten ticks, a crash a drop of at least 10\,\% from91the trailing 20-tick maximum within 20 ticks. A sensitivity grid---bubbles at 3, 5 and 8\,\%, crashes92at 7, 10 and 15\,\%---is reported for every instability result by re-scoring saved price paths; no93threshold is re-selected after the fact. The classical control produces no crash at 10\,\%, so a floor94effect is anticipated and the 7\,\% variant is the pre-specified fallback for H3. \code{msd\_log} is95analysed on the $\log_{10}$ scale, \code{time\_to\_absorb} with rank-based inference, and counts by96mean differences with randomization inference.9798\begin{table}[htbp]99\centering100\caption{Pre-registered outcome families, lead variables and predicted signs.}101\label{tab:outcomes}102\footnotesize103\begin{tabular}{@{}p{1.7cm}p{3.2cm}p{5.0cm}p{3.4cm}p{1.3cm}@{}}104\toprule105Family & Variable & Definition & Predicted sign & H \\106\midrule107Efficiency & \code{msd\_log} \newline (lead) & mean $(\log P-\log V)^2$ & $\uparrow$ in $\phi$; $\downarrow$ with diversity & H1, H4 \\108 & \code{mad\_pct} & mean $100\,|\log P-\log V|$ & $\uparrow$ in $\phi$ & H1 \\109 & \code{time\_to\_absorb} & first $h$ after $t=300$ with $|P/V-1|<1\,\%$ for 5 ticks (censored at 100) & $\uparrow$ in $\phi$; $\uparrow$ with safeguards & H1, H6 \\110 & \code{response\_ratio} & $(P_{t+20}-P_{t-1})/(V_{t+20}-V_{t-1})$ at the shock & $|RR-1|$ $\uparrow$ (two-sided) & H1 \\111 & \code{vr5} & variance ratio of log mid at lag 5 & $|VR-1|$ $\uparrow$ (two-sided) & H1 \\112Liquidity & \code{spread\_bp} \newline (lead) & mean quoted spread / mid, bp & $\uparrow$ in $\phi$ & H1b \\113 & \code{eff\_spread\_bp} & $2\,|\text{trade}-\text{mid}|/\text{mid}$, bp & $\uparrow$ in $\phi$ & H1b \\114 & \code{depth5} & mean lots within 5 levels, both sides & $\downarrow$ in $\phi$ & H1b \\115 & \code{lambda\_bp\_100} & slope of $\Delta\log$ mid (bp) on signed volume / 100 & two-sided ($\lambda$ falls as flow gets noisier) & H1b \\116 & \code{resiliency} & half-life of log-spread deviations, ticks & $\uparrow$ in $\phi$ & H1b \\117Instability & \code{n\_crashes} \newline (lead H3) & drops $\ge10\,\%$ from the trailing 20-tick max & $\uparrow$ in $\phi\times$leverage & H3 \\118 & \code{max\_dd} \newline (lead H6) & maximum drawdown of mid & $\uparrow$ in $\phi$; $\downarrow$ with safeguards & H3, H6 \\119 & \code{n\_bubbles}, \newline \code{bubble\_amp} & episodes $|P/V-1|>5\,\%$ for $\ge10$ ticks; max amplitude & $\uparrow$ in $\phi\times$leverage & H3 \\120 & \code{acf\_absr\_1\_10} & $\sum_{k=1}^{10}\mathrm{ACF}_k(|r|)$ & $\uparrow$ in $\phi$ & H3 \\121 & \code{kurtosis} & excess kurtosis of tick returns & $\uparrow$ in $\phi$ & H3 \\122Coordination & \code{llm\_pairwise\_corr} \newline (lead H2) & mean pairwise corr.\ of LLM net order signs, 20-tick windows & $\uparrow$ in $\phi$; $\downarrow$ with diversity & H2, H4 \\123 & \code{llm\_synchrony} \newline (lead H5) & mean $|\text{mean sign}|$ per decision round & $\uparrow$ with channel & H2, H5 \\124\bottomrule125\end{tabular}126\end{table}127128\subsection{Estimators}129\label{sec:identification:estimators}130131For each cell and seed we form $\Delta_s=Y_s(z)-Y_s(0)$ and report its mean, the paired-$t$ 95\,\%132confidence interval and Cohen's $d_z=\bar\Delta/\mathrm{SD}(\Delta)$, with Glass's $\Delta$ against the133control standard deviation from the 30 free classical seeds for comparability across cells. The primary134$p$-value is from randomization inference under the sharp null: within a pair the sign of $\Delta_s$ is135exchangeable, and the $p$-value is the share of the $2^n$ sign assignments whose absolute mean is at136least the observed one; the Wilcoxon signed-rank test is a secondary distribution-free check.137Multi-cell arms and the dose--response use a seed fixed-effects regression,138$Y_{s,z}=\alpha_s+\sum_k\tau_k\mathbf 1[z=k]+\varepsilon_{s,z}$, with standard errors clustered by139seed; because there are at most seven live clusters, inference rests on the wild cluster bootstrap with140Rademacher weights and, for the ordered share, on Page's $L$ trend test for randomized blocks. Within141each family the Holm step-down procedure controls the family-wise error rate at 5\,\%, the lead outcome142is reported adjusted and unadjusted, all tests are two-sided and each sign is compared with its143prediction. For the shock we estimate local projections of the gap144$g_s(z,h)=\log P_{300+h}-\log V_{300+h}$: the coefficient at horizon $h\in[-20,100]$ is the seed mean145of $g_s(z,h)-g_s(0,h)$, with pointwise paired-$t$ bands and a bootstrap sup-$t$ band; the same146projection around the placebo tick must be flat. Descriptive mediation regresses $Y$ on each mediator147with seed and cell fixed effects and reports the product-of-coefficients share of $\tau_z$ associated148with it, as a decomposition and not a causal path.149150\subsection{Power and the number of seeds}151\label{sec:identification:power}152153For a paired design with standardized effect $d=\bar\Delta/\mathrm{SD}(\Delta)$, two-sided154$\alpha=0.05$ and power $0.80$, the normal approximation gives155\begin{equation}156 n\;\approx\;\frac{(z_{0.975}+z_{0.80})^2}{d^2}\;=\;\frac{(1.96+0.84)^2}{d^2}\;=\;\frac{7.85}{d^2},157 \label{eq:power}158\end{equation}159so $d=0.5$ requires 32 pairs, $d=0.8$ requires 13 and $d=1$ requires 8. The live design has seven160matched seeds. Inverting \eqref{eq:power} gives a minimum detectable paired effect of $d\approx1.06$,161and the exact noncentral-$t$ calculation with six degrees of freedom gives $d\approx1.27$; we state162plainly that only paired effects of roughly $1.2$ standard deviations or more are detectable at 80\,\%163power in the live cells. The seed count was fixed by the dollar cap on live calls, not by the pilot164variance, and the pilot's three seeds are never pooled with the confirmatory seeds. Effects below the165minimum detectable effect are not thereby absent: a lead outcome is reported as \emph{refuted} if the166confidence interval of $d$ excludes $d\ge0.5$ in the predicted direction, as showing \emph{no large167effect} if the interval contains zero and excludes the minimum detectable effect, and as168\emph{inconclusive} if it contains both. The classical design, which costs nothing, uses 30 matched169seeds and detects $d\approx0.5$; it carries the stylized-fact validation, the mock shared-model knob,170the leverage, safeguard and information comparative statics and the placebo controls.171172\subsection{Design matrix}173\label{sec:identification:design}174175Table~\ref{tab:design} lists the cells of the two confirmatory designs. The live design runs every cell176on seeds 201--207 with \code{gpt-5.4-nano} and a single generalist prompt unless stated, a per-run177dollar cap well above the expected spend so that no run halts for budget, and cached responses so that178every run replays exactly. The classical design runs on seeds 101--130 with the mock provider, a seeded179rule that is not evidence about LLM behaviour. Both share the baseline economy: 600 ticks, one asset, a180latent $+10$-cent shock at $t=300$ and a placebo at $t=450$.181182\begin{table}[htbp]183\centering184\caption{Design matrix of the confirmatory runs.}185\label{tab:design}186\footnotesize187\begin{threeparttable}188\begin{tabular}{@{}lp{4.4cm}p{4.6cm}lr@{}}189\toprule190Arm & Cell & Treatment & Seeds & Runs \\191\midrule192\multicolumn{5}{@{}l}{\emph{Live design}: \code{confirmatory\_openai.yaml} (gpt-5.4-nano, single prompt unless stated)} \\193Dose & \code{dose\_000} & $\phi=0$ (control; no API cost) & 201--207 & 7 \\194Dose & \code{dose\_005} & $\phi=0.05$ & 201--207 & 7 \\195Dose & \code{dose\_010} & $\phi=0.10$ & 201--207 & 7 \\196Dose & \code{dose\_025} & $\phi=0.25$ (primary contrast) & 201--207 & 7 \\197Dose & \code{dose\_050} & $\phi=0.50$ (secondary contrast) & 201--207 & 7 \\198Dose & \code{dose\_100} & $\phi=1.00$ (95 eligible slots) & 201--207 & 7 \\199Diversity & \code{div\_prompts\_025} & $\phi=0.25$, eight diverse persona prompts & 201--207 & 7 \\200Diversity & \code{div\_models\_025} & $\phi=0.25$, diverse prompts, nano/mini 50/50 & 201--207 & 7 \\201Cognition & \code{memory\_025} & $\phi=0.25$, memory of five past decisions & 201--207 & 7 \\202Cognition & \code{channel\_025} & $\phi=0.25$, rationale channel in cohorts of five & 201--207 & 7 \\203Cognition & \code{adaptive\_025} & $\phi=0.25$, adaptive P\&L feedback in the prompt & 201--207 & 7 \\204Leverage & \code{leverage\_025} & $\phi=0.25$, leveraged slots 19\,\% (trend 6\,\%) & 201--207 & 7 \\205Safeguard & \code{breaker\_025} & $\phi=0.25$, halt when the 10-tick move exceeds 5\,\% & 201--207 & 7 \\206Safeguard & \code{poslimit\_025} & $\phi=0.25$, position limit 300 lots & 201--207 & 7 \\207Fairness & \code{fair\_mini\_005} & $\phi=0.05$, gpt-5.4-mini (equal USD to \code{dose\_025}) & 201--207 & 7 \\208Fairness & \code{fair\_latency\_025} & $\phi=0.25$, LLM latency fixed at the minimum & 201--207 & 7 \\209\midrule210\multicolumn{5}{@{}l}{\emph{Classical design}: \code{classical\_confirmatory.yaml} (mock provider, no API cost)} \\211Control & \code{control} & baseline economy & 101--130 & 30 \\212Placebo & \code{placebo\_shock} & placebo events at $t=300$ and $t=450$ & 101--130 & 30 \\213Feedback & \code{trend\_heavy} & trend followers 16\,\% & 101--130 & 30 \\214Leverage & \code{leverage\_heavy} & leveraged slots 19\,\% & 101--130 & 30 \\215Leverage & \code{leverage\_heavy\_} \newline \code{funding\_shock} & leveraged 19\,\%; $-10$-cent shock, then funding shock & 101--130 & 30 \\216Liquidity & \code{liquidity\_shock} & market makers withdrawn for 60 ticks after the shock & 101--130 & 30 \\217Information & \code{info\_low}, \code{info\_high} & private-signal precision 0.5, 8 & 101--130 & 60 \\218Information & \code{misleading\_news} & half of the news items misleading & 101--130 & 30 \\219Safeguard & \code{breaker\_5}, \code{poslimit\_300} & halt at 5\,\%; position limit 300 lots & 101--130 & 60 \\220Mechanism & \code{auction\_5} & call auction every five ticks & 101--130 & 30 \\221Mock herd & \code{mock\_llm\_25\_indep}, \newline \code{mock\_llm\_50\_indep} & mock LLM slots 25\,\%, 50\,\%, independent errors & 101--130 & 60 \\222Contagion & \code{k2\_control}, \code{k2\_leverage} & two assets, factor loading 0.7; with leverage and funding shock & 101--130 & 60 \\223\midrule224\multicolumn{4}{@{}l}{Total} & 592 \\225\bottomrule226\end{tabular}227\begin{tablenotes}228\item Every live cell is paired with \code{dose\_000} and every classical cell with \code{control} on229the same seed. The leveraged $\phi=0$ control for the E5 interaction is the \code{leverage\_heavy}230configuration re-run on seeds 201--207 (deterministic, no API cost). A pilot on seeds 1--3 at231$\phi\in\{0.05,0.25,0.50\}$ served cost, validity and variance reporting only.232\end{tablenotes}233\end{threeparttable}234\end{table}235236\subsection{What was frozen and what changed}237\label{sec:identification:changes}238239The plan fixed the prompt text and schema, the baseline configuration and its hash, the cell and seed240lists, the outcome definitions, thresholds and sensitivity grid, the estimators, the budget tiers, the241model identifiers and price table, and the code revision recorded in every manifest. Four changes were242made after the plan and before any confirmatory run was analysed; each is logged with its date and243reason. First, the timing of fundamental information was restructured: in the244plan's economy the announced dividend revealed the cash-flow state exactly, so the public value245coincided with the latent value and private signals and news were redundant, leaving H1 unidentified;246in the frozen economy the deviation of the upcoming dividend is latent during the period, the public247value is a filtered estimate updated by news, and private signals concern the latent deviation. Because248no confirmatory run had been analysed, this is the pre-registered economy rather than a deviation, and249the noisy dividends originally proposed for the same purpose were not used. Second, shocks are applied250after the dividend step and the latent jump was reduced from 40 to 10 cents, about $+2.2\,\%$ of value,251so that a lagged adjustment is not counted as a 5\,\% bubble; thresholds were unchanged. Third, two252leaks of common random numbers were closed: LLM slots now inherit the information set of the slot they253replace instead of drawing their own informed status, which had given the treated arm more private254information than the control, and the within-tick processing order was moved to its own stream so that255activation draws no longer diverge across arms. Fourth, the first commit marks the freeze; the pilot256ran concurrently with the confirmatory design on the same revision, its role confined to cost, validity257and variance reporting, and the seed count was set by the budget rather than by the pilot. Two258limitations follow from budget and environment rather than choice: the mini-only cell that would let the259two-model mix be decomposed net of composition was not run, so the mix is compared with the260single-model diverse-prompt cell only; and the Anthropic cells could not be run because the available261key was rejected, so the adapter is dry-run tested with the mock provider and no Claude result is262reported.263264\subsection{Assigned versus realized dose}265\label{sec:identification:dose}266267The treatment is defined on slots, but LLM agents decide once every 20 ticks at staggered offsets,268whereas rule-based agents are activated with probability $0.2$ to $0.5$ per tick. The realized share of269order flow originated by LLM agents is therefore much smaller than the slot share: in the pilot, a27025\,\% slot share produced about 4\,\% of orders and 12 to 16\,\% of traded volume. Every manifest271records both, and estimates are reported against the assigned dose with the realized dose alongside:272the assigned dose is what a deployer chooses, while the realized dose is itself an outcome, since an273agent that holds or returns an invalid decision originates no order. This asymmetry is why the fairness274cells exist: equal decision opportunities is the default, equal dollars is implemented by a per-run275budget cap, and matched latency removes the one speed disadvantage the LLM agents would otherwise276carry.277278\subsection{Internal and external validity}279\label{sec:identification:validity}280281Internal validity concerns inference inside the simulator. Its threats are provider non-determinism,282treated as part of the treatment variance, with exact replay only from the cache; cache replay across283cells, which turns identical early prompts into shared decisions across arms and whose share we report;284the one-sided invalid-decision rule, whose rate is itself an outcome; and model drift, bounded by285calling within a short window and recording the model snapshot. External validity is a different286matter and we do not claim it. The rule-based traders are documented heuristics and the LLM agents are287prompted policies without risk management, regulation or learning across runs; there is one asset in288the prompt, model time has no calendar mapping, and fees and latencies are stylized. The findings are289statements about this market mechanism and these model snapshots: signs and dose--response shapes may290transfer to markets in which many participants obtain their signals from a common model; magnitudes291will not.292