SPB Git

spb/wp39_uqo

Public
3.2 KB · 47 lines latex
1\section{Robustness and negative controls}2\label{sec:robustness}34\paragraph{Shuffled labels.} Permuting the adverse-outcome label within calendar quarter of the cutoff (training and5test rows, three draws) removes all predictability: the test AUC of the 1-minute model falls to \placeboAUC{} and the6B5$-$B2 differential is \placeboMeanD{} on average (Table~\ref{tab:placebo}). The pipeline does not manufacture7differences between information sets from the features alone, and the residual differential on shuffled labels gives8the noise floor against which the primary estimates should be read: the primary B5$-$B2 deteriorations9(Table~\ref{tab:contrasts}) are of the same order of magnitude as this floor, which is one more reason to describe the101-minute increment as zero rather than as a harm.1112\begin{table}[H]13\centering\footnotesize14\caption{Negative control: labels shuffled within calendar quarter}15\label{tab:placebo}16\input{tables/tab08_placebo}17\end{table}1819\paragraph{Symmetric target.} A volatility channel would make 1-minute features informative about the \emph{magnitude}20of the surprise even if not about its sign. The symmetric target $|S_q|\ge0.5\%$ in Table~\ref{tab:targets} shows the21same null, so the result is not an artefact of a signed target.2223\paragraph{Window contamination and sparse bars.} Excluding the 2,377 test observations whose 20-session window contains24the previous release, or restricting to firm-days with at least 98\% of expected bars, leaves the differential unchanged25(Table~\ref{tab:subgroups}). Sessions with fewer than 100 regular-session bars are already excluded from the window,26and the share of expected bars present is itself a feature, so illiquid names do not generate mechanical signals.2728\paragraph{Release timing and sector.} The differential is the same for after-close and before-open releases, and for29financial and non-financial firms (the latter forecast much better, AUC 0.86, because their outcomes are smoother).3031\paragraph{Model family and tuning.} The null holds for penalised logistic regression and for gradient-boosted trees with32identical tuning budgets; the sign of the LightGBM differential flips between the tuned configuration (\dligLBFivevsLBTwo)33and the fixed configuration used in the horizon ladder (Table~\ref{tab:horizons}, $h=20$), which is what one expects34from a true zero.3536\paragraph{Firm hold-out and time.} The increment is zero on firms never seen in training37(\dligfirmholdheldout, CI [\loligfirmholdheldout, \hiligfirmholdheldout]) and in every refit year38(Figure~\ref{fig:rolling}); the only year with a visibly positive differential (2023 for the boosted family) is a39deterioration, not a gain.4041\paragraph{What we could not test.} Delisted firms are absent from the accounting panel, so the stress test that would42matter most, forecasting the deterioration of firms that subsequently failed, cannot be run on this vendor's data; the43delisted-fundamentals backfill is documented as the first item of future work. Original-as-reported accounting values44are not available, so the stringent as-filed specification is deferred to a SEC-filing-level reconstruction. Both45limitations affect the level of predictability, not the comparison across sampling frequencies, which is the object of46the paper.47
Files · main
Menu

Theme