% ============================================================================
\section{Performance, Reliability, and Security}
\label{sec:evaluation}
% ============================================================================

This section discusses the engineering properties that make VQuant usable in production:
its performance characteristics, cost discipline, reliability and security measures, and the
limitations that bound the contribution. We frame these as the practical counterparts to the
methodology of Section~\ref{sec:methodology}.

\subsection{Performance and the streaming experience}
\label{subsec:performance}

The dominant cost in an agentic system is not raw computation but \emph{round trips}: every
model invocation and every external request adds latency. VQuant's design attacks this on three
fronts (Table~\ref{tab:robustness}). The single-call paradigm of Section~\ref{subsec:onecall}
collapses what would be three or more model round trips---fetch, analyze, plot---into one,
which is the platform's largest latency saving for analytically intensive queries. Execution
batching overlaps independent tool calls within a turn while bounding instantaneous provider
load. And end-to-end streaming decouples \emph{perceived} latency from \emph{total} latency: the
user sees reasoning, text, tool progress, and figures appear incrementally, so a report that
takes tens of seconds to complete begins responding within a second. This last property is why
the streaming architecture of Section~\ref{subsec:frontend} is a performance feature and not
merely a cosmetic one.

\input{tables/robustness}

\subsection{Cost discipline}
\label{subsec:cost}

Because model usage is metered by token, cost control is a first-order concern. VQuant estimates
token counts locally with a tokenizer before each call, which both drives the
context-management trigger of Algorithm~\ref{alg:context} and yields accurate per-session
accounting. The adaptive-summarization mechanism directly limits cost growth in long sessions by
preventing the unbounded accumulation of history, and truncating oversized tool results removes
payload bloat that would otherwise be re-sent on every turn. The user-facing model selector
(Table~\ref{tab:models}) lets users match model capability to query difficulty, routing simple
lookups to a fast, inexpensive tier and reserving the flagship model for the hardest analyses.

\subsection{Authentication, limits, and bring-your-own-keys}
\label{subsec:auth}

VQuant supports both local accounts and third-party single sign-on, with sessions persisted in
the same store used in either deployment environment. Usage is governed by daily limits that
differ for anonymous and authenticated users, enforced through per-user counters in the data
layer. A notable feature is \emph{bring-your-own-keys}: a user may supply personal API
credentials for the model provider, the market-data provider, and the web-research services;
when present, these are used in place of the platform defaults and exempt the user from the
daily cap. This design cleanly separates the platform's orchestration value from the cost of the
underlying APIs, and it lets power users operate at their own expense and rate limits.

\subsection{Reliability and security}
\label{subsec:security}

Several measures harden the system. All request bodies are validated against schemas at the
boundary, so malformed or untyped input is rejected uniformly before reaching business logic.
Database access is mediated by an object-relational mapper that parameterizes queries,
mitigating injection. The front end escapes rendered content and parses Markdown through a safe
pipeline, limiting cross-site scripting. Secrets are injected at runtime rather than committed,
and the bring-your-own-keys path keeps user credentials out of shared configuration. At the
model layer, the system prompt includes a guardrail that responds to attempts to extract the
platform's own implementation with a refusal rather than disclosure, providing a measure of
protection for the orchestration logic itself. Finally, the platform records analytics
events---active users, query rates, tool-usage frequency, model selection, and
document-generation breakdowns---giving operators the observability needed to detect anomalies
and capacity-plan.

\subsection{Limitations and threats to validity}
\label{subsec:limitations}

We close with an honest accounting of what this paper does and does not establish. First, the
contribution is \emph{architectural and methodological}; we do not provide a controlled
empirical evaluation of report quality, latency, or cost against baseline agents, and the
operational figures in Table~\ref{tab:robustness} are indicative rather than benchmarked.
Second, the platform inherits the limitations of its dependencies: the accuracy of any report is
bounded by the correctness and timeliness of the upstream market-data provider, and the model's
synthesis can still contain interpretive errors even when every number is computed
deterministically. The strict separation of reasoning from computation contains numerical
hallucination but does not eliminate the risk of a flawed \emph{narrative} around correct
numbers. Third, the quantitative methods rest on classical modeling assumptions---log-normal
prices, constant parameters over the estimation window, normally distributed returns in the
parametric VaR---that may be violated in practice; the platform exposes these methods as tools
and does not, by itself, certify their appropriateness for a given asset. Fourth, the
single-call paradigm trades some robustness for speed: a single failed execution forfeits an
entire analysis, whereas a decomposed agent can recover at a finer granularity. We regard the
design as the right default for interactive use but note the trade-off. These limitations
delineate a clear agenda for evaluation, which we sketch in the conclusion.