SPB Git

spb/wp39_uqo

Public
8.0 KB · 137 lines latex
1\section{Results}2\label{sec:results}34\subsection{The baseline ladder}56Table~\ref{tab:headline} reports the out-of-sample performance of every model on the untouched test period7(cutoffs from January 2021 to October 2026, \nlogBFivevsBTwo{} firm-quarters at $h=20$, base rate8\baseRatePrimary\%). Two facts stand out before any frequency comparison. First, point-in-time accounting information9carries most of the predictable variation: the logistic B1 model lowers the log loss from \llBZero{} (base rate) to10\llBOne{} and reaches an AUC of \aucBOne; the boosted LB1 reaches \llLBOne{} and \aucLBOne. Second, daily market data11add a clear and statistically unambiguous increment: B2$-$B1 is \dlogBTwovsBOne{} (logistic) and \dligLBTwovsLBOne{}12(LightGBM, 95\% CI [\loligLBTwovsLBOne, \hiligLBTwovsLBOne]), with AUCs of \aucBTwo{} and \aucLBTwo. This is the13``prices lead earnings'' result of \citet{beaver1980} and \citet{kothari1992} reproduced on a strictly point-in-time14panel with a 20-day horizon.1516\begin{table}[H]17\centering\footnotesize18\caption{Headline out-of-sample forecasting performance, primary target, $h=20$, test period 2021--2026}19\label{tab:headline}20\input{tables/tab05_headline}21\begin{minipage}{\textwidth}\vspace{0.3em}\footnotesize22\emph{Notes.} B0: training base rate; B0b: shrunk firm-history rate; B1--B5: $\ell_2$ logistic regressions; LB1--LB5:23LightGBM with an identical 24-trial budget per information set; ``all'': B2 plus the three intraday resolutions.24Calibration slope and intercept from a logistic recalibration of the predicted logit. Chronological split, purged with a25one-quarter embargo; all transforms fitted on training rows only.26\end{minipage}27\end{table}2829\subsection{H1 and H2: the marginal value of sampling frequency}3031Table~\ref{tab:contrasts} and Figure~\ref{fig:gains} contain the paper's central evidence. Adding the 1-minute families32to the daily-and-accounting benchmark changes the test log loss by \dlogBFivevsBTwo{} in the logistic family (95\%33block-bootstrap CI [\lologBFivevsBTwo, \hilogBFivevsBTwo], Giacomini--White $p=\plogBFivevsBTwo$) and by34\dligLBFivevsLBTwo{} in the boosted family (CI [\loligLBFivevsLBTwo, \hiligLBFivevsLBTwo], $p=\pligLBFivevsLBTwo$).35Positive differences are \emph{deteriorations}. The 30-minute and 5-minute sets behave the same way36(\dligLBThreevsLBTwo{} and \dligLBFourvsLBTwo{} for LightGBM), and concatenating all three resolutions is worse still37(\dligLBallvsLBTwo). H1 is therefore rejected, and with it H2: there is no gain to order across frequencies. The38confidence intervals are informative rather than merely non-significant. The pre-registered minimum detectable39difference at 80\% power, computed from the validation-period variance of the quarterly loss differential and the40\valQuarters{} validation quarters, is \mde{} log-loss units; the realised intervals around the 1-minute increment are narrower than this41for the boosted family (which excludes any gain at all) and of comparable width for the logistic family (which excludes42gains larger than about one fifth of the increment that daily data deliver over accounting data alone). In words: whatever one-minute bars know about the forthcoming quarter, the daily close already43knows.4445\begin{table}[H]46\centering\footnotesize47\caption{Paired out-of-sample contrasts between information sets, primary target, $h=20$}48\label{tab:contrasts}49\adjustbox{max width=\textwidth}{\input{tables/tab05b_contrasts}}50\begin{minipage}{\textwidth}\vspace{0.3em}\footnotesize51\emph{Notes.} $\Delta$ log loss $=\mathcal L(A)-\mathcal L(B)$ averaged over the test observations (negative favours52A). 95\% CI from 1,000 draws of a block bootstrap over calendar quarters of the reveal date; GW $p$ from the53Giacomini--White test on quarterly mean differentials with Newey--West variance; Holm: adjusted within the54``frequency vs daily'' family (B3, B4, B5 vs B2) and the ``fine vs coarse'' family (B5 vs B4, B5 vs B3).55$^{*}$, $^{**}$, $^{***}$: 10, 5, 1\%.56\end{minipage}57\end{table}5859\begin{figure}[H]60\centering61\includegraphics[width=\textwidth]{figures/fig04_frequency_gains.pdf}62\caption{Change in test log loss relative to the daily-and-accounting benchmark B2 when 30-minute, 5-minute, 1-minute or63all intraday families are added (left: logistic; right: LightGBM). Points are means, bars are 95\% block-bootstrap64intervals over calendar quarters; negative values would indicate a gain.}65\label{fig:gains}66\end{figure}6768The continuous target tells the same story: ridge forecasts of the winsorised surprise reach an out-of-sample $R^2$ of69\rTwoBTwo{} with daily data and \rTwoBFive{} with 1-minute data (Appendix Table~\ref{app:tables}), and the paired70difference in squared error is indistinguishable from zero. Calibration (Figure~\ref{fig:reliability}) is close to the71diagonal for the boosted models, with slopes near one, so the comparison is not an artefact of miscalibrated72probabilities.7374\begin{figure}[H]75\centering76\includegraphics[width=0.5\textwidth]{figures/fig05_reliability.pdf}77\caption{Reliability by decile of predicted probability, test period.}78\label{fig:reliability}79\end{figure}8081\subsection{H4: timing}8283If one-minute activity revealed information that accumulates as the release approaches, the increment should grow from84$h=40$ to $h=1$. Table~\ref{tab:horizons} and Figure~\ref{fig:evolution} show no such gradient: the AUCs of B2 and B5 are85within a few thousandths of each other at every horizon, the LightGBM increment changes sign across horizons, and the86rank correlation between the gain and proximity to the release is \hFourrholightgbm{} ($p=\hFourplightgbm$) for the87boosted family and \hFourrhologistic{} ($p=\hFourplogistic$) for the logistic family. The fixed-calendar design88(month-end cutoffs, labelled $h=0$ in the table), which does not condition on the release schedule, gives the same89answer. H4 is rejected.9091\begin{table}[H]92\centering\footnotesize93\caption{The 1-minute increment by horizon (trading days before the release); $h=0$ denotes the fixed-calendar design}94\label{tab:horizons}95\input{tables/tab07_horizons}96\end{table}9798\begin{figure}[H]99\centering100\includegraphics[width=\textwidth]{figures/fig06_signal_evolution.pdf}101\caption{Test AUC of B2 and B5 (left) and the paired log-loss difference B5$-$B2 with 95\% intervals (right) as the102cutoff approaches the earnings release. All firm-quarters, no selection.}103\label{fig:evolution}104\end{figure}105106\subsection{Alternative accounting targets and transfer to unseen firms}107108Table~\ref{tab:targets} repeats the B5-versus-B2 comparison for the secondary outcomes. For net income, ROA decline and109liquidity deterioration the 1-minute increment is within $\pm0.002$ log-loss units of zero in both families; for the110operating-cash-flow innovation it is a deterioration; and for the symmetric placebo target $|S_q|\ge0.5\%$, which a pure111volatility channel would predict, the increment is also zero, so the null is not the result of a signed target masking an112absolute-magnitude signal. Held-out firms (20\% never used in training) behave like seen firms113(Appendix Table~\ref{app:tables}): the 1-minute increment is \dligfirmholdheldout{} on unseen firms versus114\dligfirmholdseen{} on seen firms, both indistinguishable from zero.115116\begin{table}[H]117\centering\footnotesize118\caption{The 1-minute increment for alternative accounting targets, $h=20$}119\label{tab:targets}120\adjustbox{max width=\textwidth}{\input{tables/tab10_targets}}121\end{table}122123\subsection{Rolling performance}124125Figure~\ref{fig:rolling} reports annual out-of-fold AUCs from expanding-window refits (2016--2026) and the yearly126B5$-$B2 differential. The level of predictability is stable (AUC around 0.80 for the boosted models in every year,127including 2020 and 2022), and the 1-minute differential oscillates around zero with no trend, which is also what the128COVID and post-COVID subsamples of Section~\ref{sec:robustness} show.129130\begin{figure}[H]131\centering132\includegraphics[width=\textwidth]{figures/fig08_rolling.pdf}133\caption{Expanding-window annual refits: out-of-fold AUC by year (left) and yearly B5$-$B2 log-loss difference with13495\% intervals (right).}135\label{fig:rolling}136\end{figure}137
Files · main
Menu

Theme