SPB Git

spb/wp27_uqo

Public
16.8 KB · 264 lines latex
1\subsection{Dose--response: raising the share of LLM-controlled traders}2\label{sec:results_dose}34Table~\ref{tab:dose_conf} reports the dose--response arm of the confirmatory design: six cells in which50, 5, 10, 25, 50 and 100\,\% of the trader slots (the last excluding the five market makers, hence 956agents) are controlled by \code{gpt-5.4-nano} with the single generalist prompt, seven matched seeds per7cell, everything else held fixed. The assigned share overstates the LLM presence in the order flow: an8LLM agent decides every twenty ticks, so a 25\,\% share of slots is 5\,\% of orders and 25\,\% of traded9volume, and the 95\,\% cell is 36\,\% of orders and 84\,\% of volume (Figure~\ref{fig:realized_share}).10We report effects on the assigned share, which is the randomized quantity; the realized shares are11descriptive. Across the 35,652 live decisions of this arm, 96--98\,\% were schema-valid and feasible12(Table~\ref{tab:llm_reliability}); the remainder were infeasible orders (typically a buy exceeding free13cash) and a handful of truncated JSON responses, all converted to \emph{hold} and logged.1415\paragraph{Liquidity rises monotonically with the share.}16The quoted spread falls from 18.6~bp in the control to 15.1, 15.8, 11.0, 7.4 and 6.1~bp as the share17rises ($d_z$ from $-0.9$ to $-2.2$; Holm-adjusted $p < 0.05$ from 10\,\% upward), the effective spread18from 27.5 to 7.5~bp, depth within five levels from 141 to 956 lots, and Kyle's price impact from 9.3 to193.6~bp per 100 lots. Traded volume falls by 19\,\% at 25\,\% and 70\,\% at 95\,\%, because the LLM agents20mostly rest limit orders (only 4\,\% of their orders are marketable) and hold them: the market becomes21deeper, tighter and quieter. Resiliency, the half-life of spread deviations, rises from 0.7 to 2.022ticks at 95\,\%, which is the one liquidity statistic that deteriorates---spreads are tighter but, when23they widen, they take longer to close because fewer agents re-quote.2425\paragraph{Pricing error does not rise.}26The mean absolute deviation of the mid from the latent value is 0.73\,\% in the control and between270.50 and 0.56\,\% in every LLM cell; the paired differences ($-0.17$ to $-0.23$ points) are not28distinguishable from zero with seven seeds (95\,\% CIs span $[-0.7, +0.3]$), and the same holds for the29squared log error, the maximum drawdown and the number of bubble episodes, which is zero in all 42 runs30of the arm at every threshold of the sensitivity grid (Section~\ref{sec:robustness}). Hypothesis H1 in31its directional form---that a shared model degrades price discovery as its share rises---is not32supported on the level of prices: within the detectable range ($d \gtrsim 1.2$) the tracking error does33not increase, and the point estimates go the other way.3435\paragraph{But information is absorbed more slowly and returns become predictable.}36The two statistics that respond to the share in the direction H1 predicts are dynamic. The time to37absorb the latent shock of tick~300 falls from 36 ticks in the control to 20--25 ticks at 5--10\,\%38(when a few LLM agents, some of them inheriting a private signal, add value-anchored demand) and then39rises back to 36 ticks at 50\,\% and 40 ticks at 95\,\%; none of these paired differences is significant40(the control itself varies between 5 and 100 ticks across seeds), but the pattern is monotone from 10\,\%41upward. The variance ratio at five ticks rises from 1.14 to 1.55 (25\,\%), 1.85 (50\,\%) and 2.1242(95\,\%; $d_z = 3.5$, Holm $p < 0.001$), and the first-order autocorrelation of tick returns from 0.02 to430.15, 0.24 and 0.32 ($d_z = 3.5$ at 95\,\%). Volatility clustering rises with the share as well (the sum of44the first ten absolute-return autocorrelations from 2.2 to 3.3, $p = 0.03$ unadjusted at 95\,\%), while45tick volatility falls by half. The joint picture is a market whose price moves less, in smaller and more46persistent steps: LLM agents anchor on the public present value shown in their snapshot, lean against47incoming order flow with resting limit orders, and thereby slow the incorporation of information that48is not yet public. At 95\,\% the price still reaches the latent value---the tracking error is49unchanged---but it gets there in a smooth drift that a trend follower could exploit, and indeed the50trend followers' P\&L rises from 0.6 to 3--4\,\% of initial wealth in the LLM cells.5152\paragraph{Instability does not appear.}53No bubble (5\,\% for ten ticks), no crash (10\,\% in twenty ticks) and no halt occurs in any run of the54dose arm, and the maximum drawdown is 2.2--2.9\,\% throughout. Hypothesis H3 (bubbles and crashes rise55with penetration) is rejected in this economy for the shares and the shock sizes tested; the leverage arm56below asks whether LLM agents amplify instability generated elsewhere.5758\subsection{Coordination: are the LLM agents trading together?}59\label{sec:results_coord}6061Table~\ref{tab:coordination} measures coordination among the LLM agents directly from their logged62decisions and from the trades. Their decisions are highly synchronized at every share: in a decision63round, the absolute net sign of the placed orders averages 0.89--0.95, i.e.\ nine out of ten agents64deciding at the same tick are on the same side. The pairwise correlation of agents' net order signs over6520-tick windows is 0.48 at 5\,\% and \emph{declines} to 0.34 at 95\,\% ($-0.12$ per unit share, cluster-robust66$p < 0.01$), and the correlation of their realized net traded volume declines from 0.41 to zero. Two67things are going on. First, LLM agents are coordinated by the snapshot they share: with one model, one68prompt and one public state, the direction of their orders is nearly common, which is the mechanism69Propositions~\ref{prop:corr} and \ref{prop:diversity} describe. Second, as their number grows they70increasingly trade \emph{with each other}: an agent whose limit order rests on the bid is filled by71another LLM agent's sell, so realized flows decorrelate even though decisions do not. The classical72analogue puts the numbers in perspective: in the control economy the fundamentalists' flows are73correlated at 0.53 and the trend followers' at 0.76---rule-based classical agents of one type are more74correlated among themselves than the LLM agents, because they respond to the same statistic with the same75rule. H2 is therefore supported for decision synchrony and rejected for the cross-sectional prediction76that order-flow correlation rises with the share.7778The within-share association between coordination and outcomes is null (Table~\ref{tab:mechanism_association}):79holding the share and the seed fixed, runs in which the LLM agents were more synchronized do not have a80larger pricing error, a slower absorption or a wider spread. Coordination in this market is pervasive but81not harmful, because the common direction is toward the public value.8283\subsection{Diversity, memory and communication at a fixed share}84\label{sec:results_arms}8586Tables~\ref{tab:arms_conf} and \ref{tab:arms_effects} compare, at the 25\,\% share and on the same seven87seeds, the single-model single-prompt cell with the arms that vary what the LLM agents are given.8889\paragraph{Diversity lowers coordination and liquidity provision (H4, in part).}90Giving the 25 agents eight different persona prompts cuts the pairwise order-flow correlation from910.40 to 0.32 and the synchrony from 0.95 to 0.84; mixing two models (nano and mini, with the diverse92prompts) cuts them to 0.30 and 0.83. Both arms absorb the latent shock four ticks faster93($p \approx 0.03$) and the two-model mix has the highest 20-tick response ratio of all cells (0.5594against 0.34). The cost is liquidity: the two-model cell has a spread of 15.6~bp against 11.0~bp95($d_z = 2.1$, Holm $p < 0.01$) and a drawdown higher by 0.6 points, because the mini model places96market orders five times more often (17\,\% of its orders) and holds more; the diverse-prompt cell has a97tracking error higher by 0.04 points (Holm $p = 0.045$) but an unchanged spread. The direction of98Proposition~\ref{prop:diversity}---less correlated errors, faster information incorporation---is99observed; the magnitude on price levels is small, and the liquidity side effect is not in the stylized100model.101102\paragraph{Memory and a communication channel raise coordination slightly and pricing error slightly.}103Bounded memory of the agent's five previous decisions raises the pairwise correlation to 0.44 and the104tracking error by 0.04 points ($d_z = 1.6$, Holm $p = 0.015$) with a larger drawdown (0.4 points,105$d_z = 2.1$); the communication channel, in which cohorts of five agents read one another's rationales,106raises the pairwise correlation to 0.48, the highest of all 25\,\% cells, and the tracking error by 0.05107points ($d_z = 2.3$, Holm $p = 0.003$), while leaving spreads unchanged. H5 is supported in sign: shared108rationales make decisions more alike and prices slightly less accurate. Adaptive P\&L feedback changes109nothing. These effects are an order of magnitude smaller than the leverage effects below and would not110survive in a design powered for $d \approx 0.5$ unless they replicate.111112\paragraph{Equal budgets and matched latency.}113Five agents of the larger \code{gpt-5.4-mini} model (the equal-USD counterpart of 25 nano agents, at1140.30 against 0.33 USD per run) absorb the shock nine ticks faster than 25 nano agents but leave the115spread at 22~bp instead of 11 and the tracking error 0.14 points higher: fewer, more active agents116provide less liquidity. Against five nano agents (the equal-decision counterpart) the mini agents are117more synchronized (0.98 against 0.89, $p = 0.03$), earn more (5.7 against 3.8\,\% of initial wealth,118$p = 0.03$) and absorb the shock five ticks faster. Giving the 25 nano agents the fastest latency in the119market (0.05 tick instead of a draw from $[0.05, 1]$) changes no primary outcome beyond a 0.4-point120larger drawdown: in this economy, where an LLM decision is taken every twenty ticks, the latency of its121execution is second order.122123\subsection{Leverage and safeguards}124\label{sec:results_leverage}125126The leverage arm raises the leveraged share from 4 to 19\,\% with 25\,\% LLM agents, and the matched127comparator runs the same leverage with no LLM agents on the same seeds. Leverage harms every outcome in128both cells. Relative to the classical comparator, however, the LLM agents \emph{dampen} the damage: the129tracking error is 0.96 instead of 2.16\,\% ($d_z = -1.1$, $p = 0.03$), the latent shock is absorbed in 16130instead of 53 ticks ($p = 0.055$), bubble episodes fall from 0.71 per run to none ($p = 0.09$) and the131maximum drawdown from 7.1 to 5.5\,\% (not significant). The spread is the same (38--39~bp) and the132leveraged traders lose slightly more ($-8.0$ against $-5.9$\,\%). The mechanism is the one documented in the133dose arm: LLM agents rest value-anchored bids under the price and absorb the forced sales of the134leveraged trend followers, which in the classical economy are absorbed only by the cash-constrained135fundamentalists. This is the opposite of Proposition~\ref{prop:spiral}'s corollary that same-signed LLM136positions enlarge the exposed cohort: here the LLM agents are not the leveraged cohort, and they lean137against it.138139The 5\,\% circuit breaker never fires at 25\,\% share, so that cell is identical to the 25\,\% cell by140construction (every decision is a cache hit of the same prompt), which we report rather than hide. The141position limit of 300 lots doubles the spread (22 against 11~bp, $d_z = 1.7$) and lowers the variance ratio142below one, i.e.\ it removes the LLM agents' ability to accumulate inventory and with it both their143liquidity provision and the return persistence; the tracking error and drawdown are unchanged. Among the144safeguards tested, only the one that binds (the position limit) has an effect, and it trades the145liquidity gain for nothing in this economy.146147\subsection{Scaling with a frozen decision library}148\label{sec:results_library}149150The 34,418 valid decisions recorded in the dose arm form a frozen library replayed by nearest-neighbour151lookup on a seven-dimensional feature vector (mispricing relative to the public value, momentum, signal,152inventory and cash ratios, spread, open orders). Table~\ref{tab:library} replays it with 25\,\% library153agents at $N = 100$ and $N = 1{,}000$ (activity scaled so that order flow per tick is constant). At154$N = 100$ the library reproduces the online pattern qualitatively---tighter spread (11.9 against 17.3~bp),155deeper book, higher variance ratio (1.55 against 1.33) and autocorrelation (0.16 against 0.04)---with a156lower feasibility rate (72\,\% of replayed decisions were feasible in the new state against 98\,\% online),157which is the expected out-of-support cost of replaying decisions taken elsewhere. At $N = 1{,}000$, 250158library agents take the spread from 13.0 to 5.5~bp, the depth from 217 to 1,223 lots and the variance159ratio from 1.14 to 1.89, with a tracking error of 0.61 against 0.73\,\% and no bubble or crash. The160library is not an adaptive agent and its runs are not pooled with the online results; they show that the161liquidity-and-persistence signature of the online cells survives a tenfold larger population at zero API162cost.163164\begin{table}[htbp]165\centering166\caption{Dose--response in the assigned share of LLM-controlled trader slots (\code{gpt-5.4-nano}, single167prompt, seven matched seeds per cell). Stars: paired $t$-test against the 0\,\% cell, Holm-adjusted across168the five shares ($^{*}$ 10\,\%, $^{**}$ 5\,\%, $^{***}$ 1\,\%).}169\label{tab:dose_conf}170\small171\resizebox{\textwidth}{!}{\input{tables/tab_dose_conf}}172\end{table}173174\begin{table}[htbp]175\centering176\caption{Coordination among LLM agents by share, and the classical analogue in the 0\,\% cell. Trend:177cluster-robust OLS on the share with seed fixed effects.}178\label{tab:coordination}179\small180\resizebox{\textwidth}{!}{\input{tables/tab_coordination}}181\end{table}182183\begin{table}[htbp]184\centering185\caption{Within-share association between coordination and outcomes (seed and share fixed effects,186cluster-robust by seed; descriptive, not a mediation estimate).}187\label{tab:mechanism_association}188\small189\input{tables/tab_mechanism_association}190\end{table}191192\begin{table}[htbp]193\centering194\caption{Arms at the 25\,\% share: cell means (seven matched seeds). Stars: paired test against the19525\,\% single-model single-prompt cell, Holm-adjusted within the cell.}196\label{tab:arms_conf}197\scriptsize198\resizebox{\textwidth}{!}{\input{tables/tab_arms_conf}}199\end{table}200201\begin{table}[htbp]202\centering203\caption{Arms at the 25\,\% share: paired effects against the 25\,\% single-model single-prompt cell.}204\label{tab:arms_effects}205\scriptsize206\resizebox{\textwidth}{!}{\input{tables/tab_arms_effects}}207\end{table}208209\begin{table}[htbp]210\centering211\caption{LLM decision reliability and cost by cell (all decisions of the confirmatory design).}212\label{tab:llm_reliability}213\scriptsize214\resizebox{\textwidth}{!}{\input{tables/tab_llm_reliability_conf}}215\end{table}216217\begin{table}[htbp]218\centering219\caption{Frozen decision library replayed at $N=100$ and $N=1{,}000$ (five seeds per cell, no API calls).}220\label{tab:library}221\scriptsize222\resizebox{\textwidth}{!}{\input{tables/tab_library}}223\end{table}224225\begin{figure}[htbp]226\centering227\includegraphics[width=\textwidth]{figures/fig_dose_conf}228\caption{Paired effects of the assigned LLM share on nine outcomes (difference from the 0\,\% cell with22995\,\% confidence intervals over seven matched seeds).}230\label{fig:dose}231\end{figure}232233\begin{figure}[htbp]234\centering235\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_realized_share_conf}\caption{Assigned vs realized volume share}\label{fig:realized_share}\end{subfigure}236\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_sync_conf}\caption{Pairwise order-flow correlation of LLM agents}\end{subfigure}237\caption{Realized dose and coordination in the dose--response arm (one point per run).}238\label{fig:realized}239\end{figure}240241\begin{figure}[htbp]242\centering243\includegraphics[width=\textwidth]{figures/fig_paths_conf}244\caption{Mid price, latent and public value in seed 201 at four LLM shares.}245\label{fig:paths_conf}246\end{figure}247248\begin{figure}[htbp]249\centering250\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_arms_conf_spread_bp}\caption{Quoted spread}\end{subfigure}251\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_arms_conf_mad_pct}\caption{Tracking error}\end{subfigure}252\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_arms_conf_llm_pairwise_corr}\caption{Pairwise order-flow correlation (LLM)}\end{subfigure}253\begin{subfigure}{0.49\textwidth}\includegraphics[width=\textwidth]{figures/fig_arms_conf_max_dd}\caption{Maximum drawdown}\end{subfigure}254\caption{Arms of the confirmatory design (cell means with 95\,\% confidence intervals; the 25\,\% single-model cell in grey).}255\label{fig:arms}256\end{figure}257258\begin{figure}[htbp]259\centering260\includegraphics[width=0.6\textwidth]{figures/fig_cost_quality_conf}261\caption{Cost--quality frontier: API cost per run against the tracking error, by cell.}262\label{fig:cost_quality}263\end{figure}264
Files · d09ab0c71293588caf435dcaebf6e4bc03a129ff
Menu

Theme