spb/wp13_uqo

1% ============================================================================2\section{Related Work}3\label{sec:literature}4% ============================================================================56VQuant draws on four strands of research: the design-science tradition in information systems,7which supplies the paper's genre; the literature on tool-augmented language models and LLM8agents, which supplies the mechanisms; work on the evaluation of such agents, which supplies the9standard the paper does not yet meet; and the rapidly growing literature on LLMs in finance and10economics, which supplies the application and its risks.1112\subsection{Design science and system descriptions}13\label{subsec:lit_ds}1415Information-systems research distinguishes behavioural studies, which test theories about the use16of artifacts, from design-science studies, which build and evaluate artifacts that solve17identified problems \citep{hevner2004design}. \citet{peffers2007design} give the canonical18process---problem identification, objectives, design and development, demonstration, evaluation,19communication---and \citet{gregor2013positioning} classify contributions by their maturity, from20situated implementations to nascent design theories. In those terms the present paper reports a21\emph{situated implementation} together with a \emph{demonstration} (a walk-through of one22request); the \emph{evaluation} step is specified but not performed. We adopt this vocabulary23deliberately: it lets us describe a working system in detail while making clear which of the24process's steps remain open.2526\subsection{Tool-augmented language models and agents}27\label{subsec:lit_agents}2829A model confined to its parameters cannot retrieve a current price or perform exact arithmetic.30Chain-of-thought prompting \citep{wei2022chain} improved multi-step reasoning but left31computation inside the model; program-aided language models \citep{gao2023pal} and32program-of-thoughts prompting \citep{chen2023pot} moved it out, having the model write code that a33deterministic interpreter executes, and showed large gains on numerical benchmarks including34financial question answering. Toolformer \citep{schick2023toolformer} taught models to insert API35calls into their own text, and ReAct \citep{yao2023react} interleaved reasoning traces with36actions in an observe--think--act loop that has become the template for agents. The survey of37\citet{mialon2023augmented} names this family \emph{augmented language models} and organises it38along the two axes---reasoning and tool use---that VQuant combines; \citet{wang2024agents} survey39the resulting agent architectures. Retrieval-augmented generation \citep{lewis2020retrieval} is the40special case in which the tool is a document retriever; VQuant's ``retrieval'' surface is41structured (market-data endpoints), textual (web search and extraction), and computational42(code), and the auditability it seeks is the financial analogue of citation grounding.4344Two practical problems arise when the tool surface grows. The first is selection: Gorilla45\citep{patil2024gorilla} and ToolLLM \citep{qin2024toolllm} fine-tune models to choose among46thousands of APIs and introduce retrievers over tool documentation, whereas VQuant transmits its47entire 245-tool catalogue on every call and relies on schema descriptions and a routing prompt.48The second is context: tool results are verbose, and \citet{liu2024lost} show that models use49information in the middle of long contexts poorly even when it fits. Provider-level50function-calling protocols \citep{anthropic2026tooluse} and the Model Context Protocol51\citep{mcp2025spec} standardise how tools are declared and invoked but leave both problems to the52application. VQuant's prompt-level routing (Section~\ref{subsec:routing}) and three-stage53context management (Section~\ref{subsec:context}) are its answers; neither is claimed to be54optimal, and the evaluation protocol of Section~\ref{subsec:protocol} is designed to test them.5556\subsection{Evaluating LLM agents}57\label{subsec:lit_eval}5859The agent literature has converged on benchmark suites with executable environments and60automatic scoring: AgentBench \citep{liu2024agentbench} spans eight environments, Reflexion61\citep{shinn2023reflexion} evaluates self-correction over repeated episodes, and WebGPT62\citep{nakano2021webgpt} pioneered human-preference evaluation of browser-assisted answers. For63open-ended outputs such as reports, model-based grading has become standard, with64\citet{zheng2023judge} documenting both its agreement with human raters and its biases.65\citet{chang2024survey} survey the resulting evaluation landscape. In finance specifically,66FinanceBench \citep{islam2023financebench} supplies questions over public filings with67human-verified answers and reports that unaided retrieval-augmented models answer a majority of68them incorrectly, and FinBen \citep{xie2024finben} assembles a multi-task financial benchmark that69includes agent and quantitative-reasoning tasks. These resources are the natural starting point70for an evaluation of VQuant; the protocol in Section~\ref{subsec:protocol} builds on them.7172\subsection{Large language models in finance and economics}73\label{subsec:lit_finance}7475Text has long been a source of financial signals: \citet{loughran2016textual} survey76dictionary-based textual analysis, \citet{gentzkow2019text} review text-as-data methods for77economists, and \citet{li2021culture} show that embedding-based methods recover latent constructs78such as corporate culture from earnings calls. LLMs extend this line. \citet{lopezlira2026chatgpt}79find that ChatGPT-derived headline sentiment predicts subsequent returns; \citet{chen2023expected}80use LLM representations of news to forecast returns across sixteen markets; and81\citet{hansen2023fedspeak} show that GPT models classify the stance of FOMC communications more82accurately than dictionary and smaller supervised methods.\footnote{A widely circulated study of83LLM-based financial statement analysis \citep{kim2024financial} was withdrawn from arXiv by its84authors in February 2025 pending a review of data inconsistencies; we therefore do not rely on it.}85Domain-adapted models such as BloombergGPT \citep{wu2023bloomberggpt} and FinGPT86\citep{yang2023fingpt} demonstrate the value of finance-specific training data, and87\citet{li2023survey} survey the field. On the applied side, \citet{fatouros2025marketsenseai}88build an LLM-driven stock-selection framework from news, fundamentals, and prices and evaluate it89on realised performance, while \citet{korinek2023generative} and \citet{dell2025deep} assess90what generative and deep-learning methods offer to economic research.9192Most of this work treats the LLM as a \emph{forecasting or classification} engine operating on93text. VQuant takes a different stance: the model is an \emph{orchestrator} that assembles live94data, delegates numerical work to deterministic code, and composes the results into a document.95The classical methods it automates---option pricing \citep{black1973pricing,merton1973theory},96conditional volatility \citep{engle1982autoregressive,bollerslev1986generalized}, Monte Carlo97valuation \citep{glasserman2004monte}, and Value-at-Risk \citep{jorion2007value}---are98established; the broader concern with reproducibility and provenance in financial machine99learning is articulated by \citet{lopezdeprado2018advances}. The point of contact with the100forecasting literature is the risk, documented there, that LLM outputs about numbers are101confidently wrong; VQuant's architecture is an attempt to remove numbers from the model's remit102altogether.103104\subsection{Positioning}105\label{subsec:lit_positioning}106107Relative to this work, the artifact described here is distinctive in three respects, none of108which is an empirical claim. First, it exposes a broad and heterogeneous catalogue of tools under109one protocol and documents the practical mechanisms---routing, batching, truncation,110summarisation---needed to operate it. Second, it enforces a strict separation between reasoning111(performed by the model) and computation (performed by deterministic code), a separation that112\citet{gao2023pal} and \citet{chen2023pot} showed to be beneficial on benchmarks and that VQuant113carries into a production setting with live data. Third, it treats the deliverable---an114exportable, shareable document---as a primary artifact. What the paper does not establish is115whether these choices improve accuracy, latency, or cost relative to simpler designs; that is the116question Section~\ref{subsec:protocol} is written to answer.117