% paper/sections/identification.tex — UQO WP 27, "The Invisible Hand of Machines"
% Section: Experimental identification. Mirrors docs/identification.md, docs/protocol.md and the
% deviations recorded in docs/research_log.md. Hypotheses H1–H6 are those of Table~\ref{tab:predictions}.
\section{Experimental identification}
\label{sec:identification}

The simulator is an experimental apparatus and we describe it as one. Causal language refers to effects
\emph{inside the simulator}: the object identified is the effect of replacing a share of rule-based
trader slots by LLM-controlled slots in this market mechanism, with these model snapshots and these
prompts.

\subsection{Unit of observation and randomization}
\label{sec:identification:unit}

The unit of observation is one run: $N=100$ trader slots simulated for 600 ticks, with outcomes
computed on $t\ge100$. A run is indexed by a seed $s$ and a cell $z$, a vector of design settings---the
LLM share $\phi$ (\code{llm\_share}), the provider and model mix, the prompt set, memory, the
communication channel, the leverage composition, the safeguards and the fairness regime---and $Y_s(z)$
is the potential outcome of seed $s$ under cell $z$. All exogenous randomness derives from a master
seed through independent named streams: \code{fundamentals} (dividend path), \code{news}, \code{slots}
(wealth multipliers, latencies, informed draws), \code{assign} (the classical policy of each slot),
\code{llm\_assign} (which eligible slots become LLM-controlled, their provider and decision offsets),
\code{activation}, \code{tiebreak}, \code{shuffle} (within-tick processing order), and one generator
per agent for policy randomness and private-signal noise. Slot assignment is stratified: slots are
ranked by wealth multiplier and the policy types are dealt down the ranking, so each wealth tercile
receives the same policy composition. The treatment replaces a fraction $\phi$ of eligible slots with
proportional quotas within each policy type; market makers are never replaced, so liquidity supply is
held fixed and $\phi=1$ means 95 of 100 slots. A replaced slot keeps its cash, shares, latency
and---after the change described in Section~\ref{sec:identification:changes}---its information set: an
LLM agent is informed, at the same precision, if and only if the slot it replaces was. The treatment
changes the decision policy of a stratified subset of slots and nothing else.

In a matched pair $\{(s,0),(s,z)\}$ the dividend path and latent value, the news, the shock at $t=300$
and the placebo at $t=450$, every endowment, latency and informed status, the policy of every
non-replaced slot, the activation and tie-break draws, and the random numbers of every non-replaced
agent are identical by construction. What is \emph{not} held fixed is what the treatment is meant to
change: the decisions of the LLM slots, including the provider's sampling noise, which is not seedable
and is treated as part of the treatment, and the market's reaction to them---the book, the fills of
every other agent, and hence the subsequent decisions of the rule-based agents, which condition on the
book. Common random numbers remove the variance of the environment from the paired difference, not
the equilibrium response of the market.

\subsection{Causal structure}
\label{sec:identification:dag}

The cell label sets the treatment $Z$; with the \code{llm\_assign} stream and the provider noise, $Z$
determines the LLM slot set and its policies. Those policies generate, inside the run, a set of
mediators $M$---pairwise order-flow correlation among LLM agents, synchrony, aggressiveness, validity
rate and realized dose---and outcomes $Y$ are produced jointly by $M$, by the exogenous streams, and by
a direct path from $Z$ that bypasses LLM behaviour (fewer heuristic agents, halts, position limits). The
effect of $Z$ on $Y$ is identified because $Z$ is set by the design file and the only random input
that depends on it, \code{llm\_assign}, is independent of every other stream by construction; the
effect of $Z$ on each mediator is identified for the same reason, which is how H2 and H5 are tested.
Mediation is \emph{not} identified: $M$ and $Y$ are generated by the same path of draws and provider
noise, so sequential ignorability---independence of $M$ and the potential outcomes given $Z$---fails;
a volatile fundamental path raises both the correlation of LLM orders and the pricing error. We report
mediation only as a descriptive decomposition and treat each mediator as an outcome. $Z$ also bundles
model, prompt, decision frequency and the rule that invalid decisions become holds; the estimand is
intention-to-treat, ``deploy this agent as configured''.

\subsection{Estimands}
\label{sec:identification:estimands}

Let $\tau_z=\E_s[Y_s(z)-Y_s(0)]$ be the paired average treatment effect of cell $z$ against the control
of the same seed. In order of priority: (E1) $\tau_z$ at each share against $\phi=0$, the primary
confirmatory contrast being $\phi=0.25$ with \code{gpt-5.4-nano} and the single prompt, $\phi=0.50$ the
secondary; (E2) the dose--response $\tau(\phi)$ over $\phi\in\{0.05,0.10,0.25,0.50,1.00\}$, summarized
by a linear slope and an ordered trend test; (E3) diversity at equal share,
$\E[Y(\text{diverse},0.25)-Y(\text{single},0.25)]$ for prompt diversity at a fixed model and
$\E[Y(\text{mix},0.25)-Y(\text{diverse},0.25)]$ for a two-model mix against the single-model
diverse-prompt cell; (E4) cognition contrasts at $\phi=0.25$---memory of five decisions, a rationale
channel in cohorts of five, adaptive profit-and-loss feedback---each against the single-prompt cell at
the same share; (E5) the interaction with leverage,
$[Y(0.25,\text{lev}_{\text{hi}})-Y(0,\text{lev}_{\text{hi}})]-[Y(0.25,\text{lev}_{\text{base}})-Y(0,\text{lev}_{\text{base}})]$,
and the safeguard contrasts (circuit breaker, position limit) at $\phi=0.25$; (E6) the fairness
contrasts, a more capable model at the share that equalizes the dollar budget with the primary cell,
and a matched-latency cell in which LLM orders arrive with the minimum latency. The leveraged control
at $\phi=0$ is a classical run, free of API cost and deterministic given the seed, so it is produced on
the matched seeds and the second difference of E5 is also within-seed. Two further objects are
descriptive: a local projection of the log price--value gap around the latent shock, and $\tau_z$
divided by the mean realized order share, a rescaling and not a causal effect per order.

\subsection{Pre-registered outcomes, hypotheses and thresholds}
\label{sec:identification:outcomes}

Table~\ref{tab:outcomes} lists the outcome families, their lead variables and the sign predicted by the
hypotheses of Table~\ref{tab:predictions}. The coordination statistics exist only when $\phi>0$, so H2
is tested as a dose--response among treated cells plus a comparison with the same statistic computed on
the aggressor-signed trades of classical agents of the replaced types. Thresholds were fixed before any
live call: a bubble is $|P/V-1|>5\,\%$ for at least ten ticks, a crash a drop of at least 10\,\% from
the trailing 20-tick maximum within 20 ticks. A sensitivity grid---bubbles at 3, 5 and 8\,\%, crashes
at 7, 10 and 15\,\%---is reported for every instability result by re-scoring saved price paths; no
threshold is re-selected after the fact. The classical control produces no crash at 10\,\%, so a floor
effect is anticipated and the 7\,\% variant is the pre-specified fallback for H3. \code{msd\_log} is
analysed on the $\log_{10}$ scale, \code{time\_to\_absorb} with rank-based inference, and counts by
mean differences with randomization inference.

\begin{table}[htbp]
\centering
\caption{Pre-registered outcome families, lead variables and predicted signs.}
\label{tab:outcomes}
\footnotesize
\begin{tabular}{@{}p{1.7cm}p{3.2cm}p{5.0cm}p{3.4cm}p{1.3cm}@{}}
\toprule
Family & Variable & Definition & Predicted sign & H \\
\midrule
Efficiency & \code{msd\_log} \newline (lead) & mean $(\log P-\log V)^2$ & $\uparrow$ in $\phi$; $\downarrow$ with diversity & H1, H4 \\
 & \code{mad\_pct} & mean $100\,|\log P-\log V|$ & $\uparrow$ in $\phi$ & H1 \\
 & \code{time\_to\_absorb} & first $h$ after $t=300$ with $|P/V-1|<1\,\%$ for 5 ticks (censored at 100) & $\uparrow$ in $\phi$; $\uparrow$ with safeguards & H1, H6 \\
 & \code{response\_ratio} & $(P_{t+20}-P_{t-1})/(V_{t+20}-V_{t-1})$ at the shock & $|RR-1|$ $\uparrow$ (two-sided) & H1 \\
 & \code{vr5} & variance ratio of log mid at lag 5 & $|VR-1|$ $\uparrow$ (two-sided) & H1 \\
Liquidity & \code{spread\_bp} \newline (lead) & mean quoted spread / mid, bp & $\uparrow$ in $\phi$ & H1b \\
 & \code{eff\_spread\_bp} & $2\,|\text{trade}-\text{mid}|/\text{mid}$, bp & $\uparrow$ in $\phi$ & H1b \\
 & \code{depth5} & mean lots within 5 levels, both sides & $\downarrow$ in $\phi$ & H1b \\
 & \code{lambda\_bp\_100} & slope of $\Delta\log$ mid (bp) on signed volume / 100 & two-sided ($\lambda$ falls as flow gets noisier) & H1b \\
 & \code{resiliency} & half-life of log-spread deviations, ticks & $\uparrow$ in $\phi$ & H1b \\
Instability & \code{n\_crashes} \newline (lead H3) & drops $\ge10\,\%$ from the trailing 20-tick max & $\uparrow$ in $\phi\times$leverage & H3 \\
 & \code{max\_dd} \newline (lead H6) & maximum drawdown of mid & $\uparrow$ in $\phi$; $\downarrow$ with safeguards & H3, H6 \\
 & \code{n\_bubbles}, \newline \code{bubble\_amp} & episodes $|P/V-1|>5\,\%$ for $\ge10$ ticks; max amplitude & $\uparrow$ in $\phi\times$leverage & H3 \\
 & \code{acf\_absr\_1\_10} & $\sum_{k=1}^{10}\mathrm{ACF}_k(|r|)$ & $\uparrow$ in $\phi$ & H3 \\
 & \code{kurtosis} & excess kurtosis of tick returns & $\uparrow$ in $\phi$ & H3 \\
Coordination & \code{llm\_pairwise\_corr} \newline (lead H2) & mean pairwise corr.\ of LLM net order signs, 20-tick windows & $\uparrow$ in $\phi$; $\downarrow$ with diversity & H2, H4 \\
 & \code{llm\_synchrony} \newline (lead H5) & mean $|\text{mean sign}|$ per decision round & $\uparrow$ with channel & H2, H5 \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Estimators}
\label{sec:identification:estimators}

For each cell and seed we form $\Delta_s=Y_s(z)-Y_s(0)$ and report its mean, the paired-$t$ 95\,\%
confidence interval and Cohen's $d_z=\bar\Delta/\mathrm{SD}(\Delta)$, with Glass's $\Delta$ against the
control standard deviation from the 30 free classical seeds for comparability across cells. The primary
$p$-value is from randomization inference under the sharp null: within a pair the sign of $\Delta_s$ is
exchangeable, and the $p$-value is the share of the $2^n$ sign assignments whose absolute mean is at
least the observed one; the Wilcoxon signed-rank test is a secondary distribution-free check.
Multi-cell arms and the dose--response use a seed fixed-effects regression,
$Y_{s,z}=\alpha_s+\sum_k\tau_k\mathbf 1[z=k]+\varepsilon_{s,z}$, with standard errors clustered by
seed; because there are at most seven live clusters, inference rests on the wild cluster bootstrap with
Rademacher weights and, for the ordered share, on Page's $L$ trend test for randomized blocks. Within
each family the Holm step-down procedure controls the family-wise error rate at 5\,\%, the lead outcome
is reported adjusted and unadjusted, all tests are two-sided and each sign is compared with its
prediction. For the shock we estimate local projections of the gap
$g_s(z,h)=\log P_{300+h}-\log V_{300+h}$: the coefficient at horizon $h\in[-20,100]$ is the seed mean
of $g_s(z,h)-g_s(0,h)$, with pointwise paired-$t$ bands and a bootstrap sup-$t$ band; the same
projection around the placebo tick must be flat. Descriptive mediation regresses $Y$ on each mediator
with seed and cell fixed effects and reports the product-of-coefficients share of $\tau_z$ associated
with it, as a decomposition and not a causal path.

\subsection{Power and the number of seeds}
\label{sec:identification:power}

For a paired design with standardized effect $d=\bar\Delta/\mathrm{SD}(\Delta)$, two-sided
$\alpha=0.05$ and power $0.80$, the normal approximation gives
\begin{equation}
  n\;\approx\;\frac{(z_{0.975}+z_{0.80})^2}{d^2}\;=\;\frac{(1.96+0.84)^2}{d^2}\;=\;\frac{7.85}{d^2},
  \label{eq:power}
\end{equation}
so $d=0.5$ requires 32 pairs, $d=0.8$ requires 13 and $d=1$ requires 8. The live design has seven
matched seeds. Inverting \eqref{eq:power} gives a minimum detectable paired effect of $d\approx1.06$,
and the exact noncentral-$t$ calculation with six degrees of freedom gives $d\approx1.27$; we state
plainly that only paired effects of roughly $1.2$ standard deviations or more are detectable at 80\,\%
power in the live cells. The seed count was fixed by the dollar cap on live calls, not by the pilot
variance, and the pilot's three seeds are never pooled with the confirmatory seeds. Effects below the
minimum detectable effect are not thereby absent: a lead outcome is reported as \emph{refuted} if the
confidence interval of $d$ excludes $d\ge0.5$ in the predicted direction, as showing \emph{no large
effect} if the interval contains zero and excludes the minimum detectable effect, and as
\emph{inconclusive} if it contains both. The classical design, which costs nothing, uses 30 matched
seeds and detects $d\approx0.5$; it carries the stylized-fact validation, the mock shared-model knob,
the leverage, safeguard and information comparative statics and the placebo controls.

\subsection{Design matrix}
\label{sec:identification:design}

Table~\ref{tab:design} lists the cells of the two confirmatory designs. The live design runs every cell
on seeds 201--207 with \code{gpt-5.4-nano} and a single generalist prompt unless stated, a per-run
dollar cap well above the expected spend so that no run halts for budget, and cached responses so that
every run replays exactly. The classical design runs on seeds 101--130 with the mock provider, a seeded
rule that is not evidence about LLM behaviour. Both share the baseline economy: 600 ticks, one asset, a
latent $+10$-cent shock at $t=300$ and a placebo at $t=450$.

\begin{table}[htbp]
\centering
\caption{Design matrix of the confirmatory runs.}
\label{tab:design}
\footnotesize
\begin{threeparttable}
\begin{tabular}{@{}lp{4.4cm}p{4.6cm}lr@{}}
\toprule
Arm & Cell & Treatment & Seeds & Runs \\
\midrule
\multicolumn{5}{@{}l}{\emph{Live design}: \code{confirmatory\_openai.yaml} (gpt-5.4-nano, single prompt unless stated)} \\
Dose & \code{dose\_000} & $\phi=0$ (control; no API cost) & 201--207 & 7 \\
Dose & \code{dose\_005} & $\phi=0.05$ & 201--207 & 7 \\
Dose & \code{dose\_010} & $\phi=0.10$ & 201--207 & 7 \\
Dose & \code{dose\_025} & $\phi=0.25$ (primary contrast) & 201--207 & 7 \\
Dose & \code{dose\_050} & $\phi=0.50$ (secondary contrast) & 201--207 & 7 \\
Dose & \code{dose\_100} & $\phi=1.00$ (95 eligible slots) & 201--207 & 7 \\
Diversity & \code{div\_prompts\_025} & $\phi=0.25$, eight diverse persona prompts & 201--207 & 7 \\
Diversity & \code{div\_models\_025} & $\phi=0.25$, diverse prompts, nano/mini 50/50 & 201--207 & 7 \\
Cognition & \code{memory\_025} & $\phi=0.25$, memory of five past decisions & 201--207 & 7 \\
Cognition & \code{channel\_025} & $\phi=0.25$, rationale channel in cohorts of five & 201--207 & 7 \\
Cognition & \code{adaptive\_025} & $\phi=0.25$, adaptive P\&L feedback in the prompt & 201--207 & 7 \\
Leverage & \code{leverage\_025} & $\phi=0.25$, leveraged slots 19\,\% (trend 6\,\%) & 201--207 & 7 \\
Safeguard & \code{breaker\_025} & $\phi=0.25$, halt when the 10-tick move exceeds 5\,\% & 201--207 & 7 \\
Safeguard & \code{poslimit\_025} & $\phi=0.25$, position limit 300 lots & 201--207 & 7 \\
Fairness & \code{fair\_mini\_005} & $\phi=0.05$, gpt-5.4-mini (equal USD to \code{dose\_025}) & 201--207 & 7 \\
Fairness & \code{fair\_latency\_025} & $\phi=0.25$, LLM latency fixed at the minimum & 201--207 & 7 \\
\midrule
\multicolumn{5}{@{}l}{\emph{Classical design}: \code{classical\_confirmatory.yaml} (mock provider, no API cost)} \\
Control & \code{control} & baseline economy & 101--130 & 30 \\
Placebo & \code{placebo\_shock} & placebo events at $t=300$ and $t=450$ & 101--130 & 30 \\
Feedback & \code{trend\_heavy} & trend followers 16\,\% & 101--130 & 30 \\
Leverage & \code{leverage\_heavy} & leveraged slots 19\,\% & 101--130 & 30 \\
Leverage & \code{leverage\_heavy\_} \newline \code{funding\_shock} & leveraged 19\,\%; $-10$-cent shock, then funding shock & 101--130 & 30 \\
Liquidity & \code{liquidity\_shock} & market makers withdrawn for 60 ticks after the shock & 101--130 & 30 \\
Information & \code{info\_low}, \code{info\_high} & private-signal precision 0.5, 8 & 101--130 & 60 \\
Information & \code{misleading\_news} & half of the news items misleading & 101--130 & 30 \\
Safeguard & \code{breaker\_5}, \code{poslimit\_300} & halt at 5\,\%; position limit 300 lots & 101--130 & 60 \\
Mechanism & \code{auction\_5} & call auction every five ticks & 101--130 & 30 \\
Mock herd & \code{mock\_llm\_25\_indep}, \newline \code{mock\_llm\_50\_indep} & mock LLM slots 25\,\%, 50\,\%, independent errors & 101--130 & 60 \\
Contagion & \code{k2\_control}, \code{k2\_leverage} & two assets, factor loading 0.7; with leverage and funding shock & 101--130 & 60 \\
\midrule
\multicolumn{4}{@{}l}{Total} & 592 \\
\bottomrule
\end{tabular}
\begin{tablenotes}
\item Every live cell is paired with \code{dose\_000} and every classical cell with \code{control} on
the same seed. The leveraged $\phi=0$ control for the E5 interaction is the \code{leverage\_heavy}
configuration re-run on seeds 201--207 (deterministic, no API cost). A pilot on seeds 1--3 at
$\phi\in\{0.05,0.25,0.50\}$ served cost, validity and variance reporting only.
\end{tablenotes}
\end{threeparttable}
\end{table}

\subsection{What was frozen and what changed}
\label{sec:identification:changes}

The plan fixed the prompt text and schema, the baseline configuration and its hash, the cell and seed
lists, the outcome definitions, thresholds and sensitivity grid, the estimators, the budget tiers, the
model identifiers and price table, and the code revision recorded in every manifest. Four changes were
made after the plan and before any confirmatory run was analysed; each is logged with its date and
reason. First, the timing of fundamental information was restructured: in the
plan's economy the announced dividend revealed the cash-flow state exactly, so the public value
coincided with the latent value and private signals and news were redundant, leaving H1 unidentified;
in the frozen economy the deviation of the upcoming dividend is latent during the period, the public
value is a filtered estimate updated by news, and private signals concern the latent deviation. Because
no confirmatory run had been analysed, this is the pre-registered economy rather than a deviation, and
the noisy dividends originally proposed for the same purpose were not used. Second, shocks are applied
after the dividend step and the latent jump was reduced from 40 to 10 cents, about $+2.2\,\%$ of value,
so that a lagged adjustment is not counted as a 5\,\% bubble; thresholds were unchanged. Third, two
leaks of common random numbers were closed: LLM slots now inherit the information set of the slot they
replace instead of drawing their own informed status, which had given the treated arm more private
information than the control, and the within-tick processing order was moved to its own stream so that
activation draws no longer diverge across arms. Fourth, the first commit marks the freeze; the pilot
ran concurrently with the confirmatory design on the same revision, its role confined to cost, validity
and variance reporting, and the seed count was set by the budget rather than by the pilot. Two
limitations follow from budget and environment rather than choice: the mini-only cell that would let the
two-model mix be decomposed net of composition was not run, so the mix is compared with the
single-model diverse-prompt cell only; and the Anthropic cells could not be run because the available
key was rejected, so the adapter is dry-run tested with the mock provider and no Claude result is
reported.

\subsection{Assigned versus realized dose}
\label{sec:identification:dose}

The treatment is defined on slots, but LLM agents decide once every 20 ticks at staggered offsets,
whereas rule-based agents are activated with probability $0.2$ to $0.5$ per tick. The realized share of
order flow originated by LLM agents is therefore much smaller than the slot share: in the pilot, a
25\,\% slot share produced about 4\,\% of orders and 12 to 16\,\% of traded volume. Every manifest
records both, and estimates are reported against the assigned dose with the realized dose alongside:
the assigned dose is what a deployer chooses, while the realized dose is itself an outcome, since an
agent that holds or returns an invalid decision originates no order. This asymmetry is why the fairness
cells exist: equal decision opportunities is the default, equal dollars is implemented by a per-run
budget cap, and matched latency removes the one speed disadvantage the LLM agents would otherwise
carry.

\subsection{Internal and external validity}
\label{sec:identification:validity}

Internal validity concerns inference inside the simulator. Its threats are provider non-determinism,
treated as part of the treatment variance, with exact replay only from the cache; cache replay across
cells, which turns identical early prompts into shared decisions across arms and whose share we report;
the one-sided invalid-decision rule, whose rate is itself an outcome; and model drift, bounded by
calling within a short window and recording the model snapshot. External validity is a different
matter and we do not claim it. The rule-based traders are documented heuristics and the LLM agents are
prompted policies without risk management, regulation or learning across runs; there is one asset in
the prompt, model time has no calendar mapping, and fees and latencies are stylized. The findings are
statements about this market mechanism and these model snapshots: signs and dose--response shapes may
transfer to markets in which many participants obtain their signals from a common model; magnitudes
will not.