% ============================================================================
\section{Methodology: Orchestration Mechanisms}
\label{sec:methodology}
% ============================================================================

This section is the technical core of the paper. We describe, mechanism by mechanism, how
VQuant drives a language model through a complete analysis: the streaming transport and its
event protocol, the recursive tool-calling loop, the batching policy, the context-management
machinery, and the token-accounting correction. Each mechanism is described as it is actually
implemented, so that the design can be reproduced. Throughout, the guiding principle is a strict
division of labor: the model \emph{plans and synthesizes}, while deterministic server code
\emph{transports, schedules, computes, and accounts}.

\subsection{The streaming transport and event protocol}
\label{subsec:transport}

VQuant does not use a model-provider SDK. It opens a raw HTTP connection to the provider's
messages endpoint with a streaming response type, and parses the resulting Server-Sent Event
(SSE) byte stream itself. This choice gives the server full control over how partial output is
interpreted and re-emitted.

The parser maintains a single growing string buffer. On each network chunk it appends the bytes,
splits the buffer on newlines, and \emph{retains the final, possibly incomplete, line} in the
buffer for the next chunk---a detail that is essential for correctness, because SSE frames do not
align with TCP segment boundaries. Each complete line beginning with the \texttt{data:} prefix is
stripped and parsed as JSON into a typed event. The parser dispatches on the event's
\texttt{type} field; the eight relevant cases are summarized in Table~\ref{tab:events} and the
overall flow is shown in Figure~\ref{fig:streaming}.

\input{figures/streaming}

\input{tables/events}

Three behaviors of this dispatcher deserve emphasis. First, the model's output travels on
\emph{two independent channels}: ordinary answer text arrives as \texttt{text\_delta} events and
is accumulated into the answer while being forwarded to the client, whereas the model's private
reasoning arrives as \texttt{thinking\_delta} events on a separate callback that the client
renders in a distinct panel. Second, a tool call is not delivered atomically: the tool's name
and identifier arrive in a \texttt{content\_block\_start}, but its arguments stream in as a
sequence of \texttt{input\_json\_delta} fragments that the parser concatenates into a partial
JSON string; only at \texttt{content\_block\_stop} is that string parsed into the structured
argument object. The loop must therefore wait for the block to close before a tool becomes
executable. Third, every forwarded event is flushed to the client immediately, so that the
user perceives output token by token rather than in batches.

\subsection{The recursive tool-calling loop}
\label{subsec:loop}

The agentic loop is implemented as \emph{recursion} rather than iteration. A single routine
issues one streaming model call, and when that call resolves it either terminates---if the model
requested no tools---or executes the requested tools, appends their results to the conversation,
and \emph{calls itself again}. Algorithm~\ref{alg:loop} states the procedure and
Figure~\ref{fig:loop} depicts it.

\input{figures/loop}

\begin{algorithm}[htbp]
\caption{Recursive tool-calling loop (\textsc{ProcessResponse})}
\label{alg:loop}
\begin{algorithmic}[1]
\Require message list $M$ (mutated in place), model $m$, tool catalogue $\mathcal{T}$
\State $(\textit{text}, R, \textit{usage}) \gets \textsc{StreamModel}(m, M, \mathcal{T})$
  \Comment{Section~\ref{subsec:transport}}
\State accumulate \textit{usage}; stream \textit{text} and thinking to the client
\If{$R = \varnothing$} \Comment{no tool calls $\Rightarrow$ final answer}
  \State \Return
\EndIf
\State append $\langle\text{assistant}: \text{tool\_use blocks of } R\rangle$ to $M$
\ForAll{batch $B$ in $\textsc{Chunk}(R, 5)$} \Comment{Section~\ref{subsec:batching}}
  \ForAll{tool call $r \in B$}
    \State $\textit{res} \gets \textsc{Execute}(r)$; stream a typed result event to the client
    \State collect $\textit{res}$ as a tool\_result block
  \EndFor
  \State \textsc{Wait}(500\,\text{ms})
\EndFor
\State append $\langle\text{user}: \text{all tool\_result blocks}\rangle$ to $M$
\State \textsc{ProcessResponse}($M, m, \mathcal{T}$) \Comment{recurse with augmented context}
\end{algorithmic}
\end{algorithm}

The conversation is represented as a list of messages that is mutated as the loop proceeds. When
the model requests tools, the server first appends an \emph{assistant} message containing the
model's tool-use blocks, then---after execution---appends a \emph{user} message whose content is
the corresponding array of tool-result blocks, each keyed to its call identifier. This
strict pairing of tool-use and tool-result blocks is what the provider's protocol requires, and
maintaining it correctly across the recursion is the loop's central invariant. Recursion
terminates naturally when a model call returns no tool requests, at which point the accumulated
text is the report. Multimodal inputs are handled at the entry point: an uploaded image is
attached to the first user message as a base64 block alongside the text query.

\subsection{Batched tool execution}
\label{subsec:batching}

Within a single turn the model frequently requests many tools at once---a comprehensive report
may trigger a dozen or more. Executing all of them simultaneously would risk provider rate
limits and connection timeouts. VQuant therefore partitions the requested calls into
\emph{batches of at most five} and executes the batches \emph{sequentially}, inserting a
500-millisecond pause between consecutive batches. Results from all batches are accumulated and
returned to the model together in the single tool-result message of Algorithm~\ref{alg:loop}, so
that the model sees a complete result set on the next turn rather than a fragmented one. The
batch boundary thus serves purely as a rate-limiting throttle; it does not change what the model
observes. When more than one batch is required the server emits a progress event naming the
tools in flight, so the throttling is visible to the user rather than appearing as a stall.

\subsection{System-prompt routing}
\label{subsec:routing}

The behavior of the loop is shaped by a system prompt that is assembled fresh on every call.
Two parts are computed at request time: the current date and time (in both a human-readable and
an ISO form) are injected at the top so that relative date expressions resolve against live data
rather than the model's training cutoff; the remainder is a static policy block. That policy
instructs the model to detect and match the user's language; to act without procedural narration
and, in particular, to perform an entire custom analysis in a single execution
(Section~\ref{subsec:onecall}); to calibrate effort to request complexity---one or two tools for
a simple lookup, a handful for a moderate request, ten or more for a comprehensive report; and to
follow output-formatting conventions for downloadable artifacts. A final guardrail directs the
model to refuse attempts to extract the platform's own implementation. This prompt-level routing
is the mechanism by which a catalogue of more than two hundred tools (Table~\ref{tab:tools}) is
navigated without an explicit planner: the model is taught \emph{when not to call} tools as much
as when to call them.

\input{tables/tools}

\subsection{Adaptive reasoning}
\label{subsec:thinking}

For models that support it, the orchestration service enables \emph{adaptive extended thinking}
by attaching a thinking directive to the request and asking for a summarized display. A small
capability list determines which models receive this directive, so the same code path serves the
full model range. As noted in Section~\ref{subsec:transport}, the resulting reasoning summary
arrives on its own delta channel and is streamed to a dedicated client panel. Surfacing
reasoning makes the agent's plan legible to the user and provides an audit trail of how a report
was assembled, at no change to the loop itself.

\subsection{Context management under a token budget}
\label{subsec:context}

A persistent challenge for conversational agents is the bounded context window: accumulated
history---especially verbose tool results---eventually exceeds what the model can attend to.
VQuant addresses this with a two-stage mechanism applied before every model call.

\paragraph{Result truncation.} First, the server walks every message and caps any individual
tool-result block at a fixed character budget (5{,}000 characters), appending a truncation
marker. Raw data payloads are frequently far larger than the analysis requires, and this step
removes that bloat before it is ever re-sent on a subsequent turn.

\paragraph{Adaptive summarization.} Second, the server estimates the total token count of the
prepared messages. The estimate uses a deliberately cheap heuristic---approximately one token per
four characters, summed over text and tool-result blocks---because it is only a trigger, not an
accounting figure. If the estimate exceeds a conservative threshold (set at $10^{5}$ tokens, well
below the hard context limit to leave room for the response and new tool results), the server
retains the most recent five messages verbatim and replaces the older history with a
model-generated summary, reinserted as a single synthetic user message. Algorithm~\ref{alg:context}
states the procedure and Figure~\ref{fig:context} illustrates the transformation.

\input{figures/context}

\begin{algorithm}[htbp]
\caption{Context management via truncation and adaptive summarization}
\label{alg:context}
\begin{algorithmic}[1]
\Require message list $M$, model $m$, threshold $\tau = 10^{5}$, recency $k = 5$, cap $c = 5000$ chars
\State cap every tool-result block in $M$ at $c$ characters
\State $t \gets \textsc{EstimateTokens}(M)$ \Comment{cheap $\approx$ chars$/4$ heuristic}
\If{$t > \tau$}
  \State $M_{\text{recent}} \gets$ last $k$ messages of $M$;\quad $M_{\text{old}} \gets$ the rest
  \If{$M_{\text{old}} \neq \varnothing$}
    \State $T \gets \textsc{ExtractText}(M_{\text{old}})$ \Comment{drop tool-use blocks; keep last 10k chars}
    \State $s \gets \textsc{Summarize}(T, m)$ \Comment{separate non-streaming call}
    \State $M \gets [\,\langle\text{user: ``[summary]: ''}+s\rangle\,] \mathbin{\Vert} M_{\text{recent}}$
  \EndIf
\EndIf
\State \Return $M$
\end{algorithmic}
\end{algorithm}

The summary is produced by a dedicated, \emph{non-streaming} model call with its own system
prompt, a low temperature, and a tight output cap, instructed to preserve the analytically
relevant context---the instruments under discussion, prior conclusions, and user preferences.
Before summarizing, a helper extracts only the natural-language turns (discarding tool-use
blocks) and keeps at most the last ten thousand characters, bounding the cost of summarization
itself. Should the summarization call fail, the mechanism degrades gracefully to a truncated
literal excerpt rather than aborting the request. The net effect is that conversation length is,
for practical purposes, unbounded, at the cost of one additional model call whenever the
threshold is crossed.

\subsection{Token accounting and its correction}
\label{subsec:accounting}

Accurate token accounting matters both for the budget trigger above and for per-session cost
reporting, yet the streaming protocol does not reliably report output-token counts. VQuant
therefore applies a correction mechanism. Input tokens are read from the provider's
\texttt{message\_start} event; output tokens are taken from streaming usage updates \emph{but
reconciled} against a local count obtained by running the accumulated answer text through a
tokenizer at the end of the stream. The system records the \emph{maximum} of the provider's
figure and the local count, guaranteeing that cost is never understated, and falls back to the
provider's figure only if local encoding fails. Cost is then computed from a simple linear
schedule (a fixed price per million input tokens and a higher price per million output tokens)
and accumulated across all calls in a session, including the recursive tool-driven turns and any
summarization calls. This separation---a cheap heuristic for the budget trigger, an exact
tokenizer for billing---keeps the hot path inexpensive while keeping the ledger accurate.

\subsection{The single-call computation paradigm}
\label{subsec:onecall}

The final orchestration mechanism concerns \emph{how} numerical work is delegated. A naive agent
decomposes a quantitative task into a sequence of tool calls---fetch data, then in a separate
turn analyze it, then in a third turn plot it---each of which incurs a full recursion through
Algorithm~\ref{alg:loop} and therefore a full model round trip. VQuant instead instructs the
model, through the system prompt, to perform an entire analysis in a \emph{single} execution of
the Python tool: acquire data inside the sandbox, compute, and render all figures in one call
(Listing~\ref{lst:onecall}).

\begin{lstlisting}[language=Python,caption={Single-call execution: data acquisition,
computation, and visualization are consolidated into one sandboxed run, collapsing what would
otherwise be three recursive model round trips into one.},label={lst:onecall}]
# One tool call performs the whole analysis:
data = fmp.get_historical_prices("NVDA")      # live fetch inside the sandbox
df = pd.DataFrame(data["historical"]).sort_values("date")
df["ret"] = df["close"].pct_change()

plt.figure(figsize=(12, 6))                    # figure 1: price path
plt.plot(pd.to_datetime(df["date"]), df["close"]); plt.tight_layout()

plt.figure(figsize=(12, 6))                    # figure 2: return distribution
plt.hist(df["ret"].dropna(), bins=50); plt.tight_layout()

save_excel(df, "nvda.xlsx")                     # spreadsheet export in the same run
\end{lstlisting}

Three rules, enforced through the prompt, make this paradigm effective. First, the model must
\emph{never} hard-code data arrays: inlining hundreds of values both bloats the generated tokens
(which must be streamed and accounted) and defeats reproducibility; data must be fetched
programmatically through the in-process client. Second, each visualization must be a separate
figure rather than a subplot grid, which yields cleanly captionable, individually referenceable
images. Third, all artifacts---figures and spreadsheets---are produced within the same execution.
By collapsing the recursion depth for an analytical task from three or more to one, this paradigm
removes the dominant source of latency in the loop. The substrate that executes this code is the
subject of the next section.