spb/wp23_uqo

1\section{Probabilistic forecasting and calibration}\label{sec:prob}23Tables~\ref{tab:prob_a} and~\ref{tab:prob_b} report the CRPS, the coverage and the width of the 90\% intervals; Figure~\ref{fig:cal_a} plots the share of outcomes below each nominal quantile. For Task~A the LightGBM quantile models are the sharpest and are slightly under-covered (\aCovlightgbmhOneThree\% at one to three months for a nominal 90\%), the linear models are over-covered because their residual quantiles are homoskedastic, the hierarchical network is close to nominal at short horizons and under-covers at seven to twelve months (\aCovhshnpartialhSevenOneTwo\%) because its intervals do not widen with the horizon while the bias grows, and the unembedded MLP produces intervals two and a half times too wide. For Task~B the ranking by CRPS follows the point ranking (LightGBM \bCRPSlightgbmpanelhThree{} at three months, QREFA \bCRPSqrefahThree, QANSS \bCRPSqansshThree, no change \bCRPSnochangehThree), but calibration separates the models sharply. The neural quantile heads (LSTM, GRU, TCN, TFT-lite, DSGFN) under-cover at every horizon (\bCovdsgfnhThree\% to \bCovtftlitehThree\% at three months for a nominal 90\%, falling below 50\% at twelve months): trained with the pinball loss on 34 to 45 origins they fit the central quantiles and collapse the tails. The Gaussian posterior predictive of BHPF and the residual quantiles of the ridge are close to nominal at one and three months (\bCovbhpfhThree\%, \bCovridgepanelhThree\%) and under-cover at six and twelve months as the bias grows. QANSS is over-covered at every horizon (\bCovqansshThree\% at three months, \bCovqansshOneTwo\% at twelve) because its predictive variance adds the published filter variance of the origin level to the learned drift variance, and it is the only model whose twelve-month intervals contain the realised changes; the price is intervals twice as wide as LightGBM's. The decomposition of uncertainty the protocol asks for is only partly identified: the estimation component is the posterior variance of BHPF (small, because the panel is pooled), the transaction-noise component is the WP22 filter variance that QANSS carries, and the market-dynamics component is the residual drift variance; macroeconomic uncertainty is not modelled and is the reason every model fails at twelve months.45\input{tables/tab_prob_a}6\subsection{Probabilities of decline}78The protocol asks for the probability of a price decline. Table~\ref{tab:decline_b} scores the probability of a decline of the quality-adjusted index over the horizon, read off the quantile forecasts, by the Brier score and its skill relative to the climatological frequency of declines (\ShareDeclineshThree\% of cell-months at three months, \ShareDeclineshOneTwo\% at twelve); Figure~\ref{fig:decline_reliability} plots reliability at three months. The integrated QREFA and LightGBM have the best skill at three months (\BrierSkillqrefahThree{} and \BrierSkilllightgbmpanelhThree), DSGFN, BHPF and the ridge follow, QANSS's over-wide distribution leaves it little skill (\BrierSkillqansshThree), and the no-change, TFT-lite and mean-growth probabilities are worse than the climatology. Reliability is good for QREFA, DSGFN, LightGBM and BHPF over the whole range of forecast probabilities, and QANSS's probabilities overstate the frequency of declines throughout (a forecast probability of 0.85 is followed by a decline 64\% of the time), the mirror image of its over-wide intervals, which put too much mass below zero. At twelve months the skill of every model is negative, QANSS's severely so (\BrierSkillqansshOneTwo): with fewer than 10\% of twelve-month windows ending below their start, the climatology is hard to beat and the probabilistic forecasts inherit the bias of the point forecasts.910\input{tables/tab_decline_b}1112\begin{figure}[htbp]\centering13\includegraphics[width=0.55\textwidth]{figures/fig_decline_reliability.pdf}14\caption{Task C: reliability of the forecast probability of an index decline over three months (bins of forecast probability).}\label{fig:decline_reliability}15\end{figure}1617\input{tables/tab_prob_b}1819\begin{figure}[htbp]\centering20\includegraphics[width=0.48\textwidth]{figures/fig_calibration_a.pdf}\hfill\includegraphics[width=0.48\textwidth]{figures/fig_calibration_b.pdf}21\caption{Calibration: share of outcomes below each nominal quantile, Task A (left, horizon band 4--6 months) and Task B (right, horizon 3).}\label{fig:cal_a}22\end{figure}23