SPB Git

spb/wp44_uqo

Public
46.0 KB · 803 lines latex
1% ============================================================================2% Appendix C — Mathematical derivations3% Written by Agent L (Appendices B/C instance), 2026-11-03. Derivations and4% proofs referenced by Part IV (sections/part04_methodology.tex); notation5% follows Part IV. Each derivation is checked against the implementation in6% econometrics/structural_change.py, bibliometrics/03_citation_impact.py,7% bibliometrics/04_diversity_cagr.py, knowledge_graph/02_category_flows.py,8% taxonomy/classify.py, taxonomy/validate_sample.py and9% experiments/emerging_detection.py; where the code and the usual textbook10% convention differ, the code's convention is stated and the difference noted.11% ============================================================================1213\section{Mathematical derivations}14\label{sec:appC}1516Part~IV defines the statistics of this study and states their properties;17this appendix supplies the derivations behind those statements, in the order18in which Part~IV uses them. Each subsection fixes the object as implemented,19derives the property that the main text relies on, and records any point at20which the implementation departs from the convention of the cited source.21The derivations are elementary and self-contained; their purpose is not22novelty but the assurance that the quantities tabulated in Part~IX are what23the text says they are. Throughout, $n$ is the length of a monthly series,24$y_t = \log(N_t + 1)$ its logarithm, and $\mathbf{1}[\cdot]$ the indicator25function.2627\subsection{The interrupted time series as a linear regression}28\label{sec:appC-its}2930\paragraph{Design matrix.}31Equation~\eqref{eq:its} of Part~IV is the linear model $y = X\beta +32\varepsilon$ with $y \in \R^n$ and $X \in \R^{n \times 15}$ whose columns33are, for $t = 0, \dots, n-1$ and event index $t_e$,34\begin{equation}35  x_t \;=\; \bigl(1,\; t,\; \mathbf{1}[t \ge t_e],\; (t - t_e)\,\mathbf{1}[t \ge t_e],\;36  \mathbf{1}[\mathrm{month}(t) = 2],\; \dots,\; \mathbf{1}[\mathrm{month}(t) = 12]\bigr)^{\top},37  \label{eq:appC-design}38\end{equation}39exactly the data frame built by \texttt{\_design()} in40\texttt{econometrics/structural\_change.py} (columns \texttt{const},41\texttt{t}, \texttt{post}, \texttt{t\_post}, \texttt{m2}, \dots,42\texttt{m12}). The post-event trend is centred at $t_e$ rather than at zero43so that $\beta_2$ is the level change \emph{at} the event month and not the44extrapolated intercept difference at $t = 0$; with an uncentred post-trend45the two parametrisations are related by $\beta_2^{\mathrm{unc}} = \beta_2 -46\beta_3 t_e$, and only the centred one has the interpretation used in the47text. The eleven month dummies are linearly independent of the constant48because January is omitted; the full-column-rank condition requires at least49one observation in every month of the year, satisfied by any series of50twelve months or more, and at least two post-event months with $t > t_e$51(with a single one the centred post-trend column is identically zero and52$\beta_2$ and $\beta_3$ are not separately identified), guaranteed by the53driver's rule that twelve months follow every event.54The ordinary least-squares estimator is $\hat\beta = (X^{\top}X)^{-1}55X^{\top}y$ and the residuals $\hat\varepsilon = y - X\hat\beta$.5657\paragraph{The HAC covariance estimator.}58Under the assumption that $\varepsilon_t$ is a stationary, serially59correlated and possibly heteroskedastic sequence with60$\E[x_t \varepsilon_t] = 0$, the asymptotic covariance of $\sqrt{n}(\hat\beta61- \beta)$ is $Q^{-1}\Omega Q^{-1}$ with $Q = \lim n^{-1}X^{\top}X$ and62$\Omega = \sum_{\ell=-\infty}^{\infty} \Gamma_\ell$, where63$\Gamma_\ell = \E[x_t \varepsilon_t \varepsilon_{t-\ell} x_{t-\ell}^{\top}]$64is the autocovariance of the score at lag $\ell$. The estimator of65\citet{newey1987hac} replaces the infinite sum by a kernel-weighted finite66one,67\begin{equation}68  \hat\Omega_L \;=\; \hat\Gamma_0 + \sum_{\ell=1}^{L} w_\ell\,\bigl(\hat\Gamma_\ell + \hat\Gamma_\ell^{\top}\bigr),69  \qquad70  \hat\Gamma_\ell = \frac{1}{n}\sum_{t=\ell}^{n-1} \hat\varepsilon_t \hat\varepsilon_{t-\ell}\, x_t x_{t-\ell}^{\top},71  \qquad72  w_\ell = 1 - \frac{\ell}{L+1},73  \label{eq:appC-nw}74\end{equation}75and the variance estimate is $\hat V = n\,(X^{\top}X)^{-1} \hat\Omega_L76(X^{\top}X)^{-1}$, which is the expression of Part~IV once the factors of77$n$ are absorbed into the un-normalised $\hat\Gamma_\ell$. The Bartlett78weights $w_\ell$ are what make $\hat\Omega_L$ positive semi-definite for every79sample: $\hat\Omega_L$ can be written as $\frac{1}{n(L+1)}\sum_{t}80\bigl(\sum_{s=t-L}^{t} \hat u_s\bigr)\bigl(\sum_{s=t-L}^{t} \hat u_s\bigr)^{\top}$81up to end effects, with $\hat u_s = x_s \hat\varepsilon_s$, a sum of outer82products, whereas the truncated sum with $w_\ell \equiv 1$ is not guaranteed83to be positive semi-definite. The implementation calls84\texttt{statsmodels} with \texttt{cov\_type="HAC"} and \texttt{maxlags=6}, so85$L = 6$ and $w_\ell = 1 - \ell/7$ as stated in Part~IV, without the86small-sample degrees-of-freedom correction and with $p$-values from the87normal approximation to the $t$-ratio, which is the library's default for88robust covariances. With $n$ of the order of one hundred and fifteen89columns in $X$, the constant included, the difference between the normal and the $t_{n-15}$ reference90distribution is immaterial at the magnitudes of the $t$-statistics reported;91what matters is the placebo rank of the next paragraph, which does not92depend on any reference distribution.9394\paragraph{The placebo $p$-value as a rank statistic.}95Let $\mathcal{T}' = \{t' : 24 \le t' \le n - 13\}$ be the candidate event96indices (the implementation's \texttt{min\_pre = 24} and97\texttt{min\_post = 12}) and, for each $t' \in \mathcal{T}'$, let98$t^{(2)}_{t'} = \hat\beta_2(t') / \widehat{\mathrm{se}}(\hat\beta_2(t'))$ be99the HAC $t$-ratio of the level change when the model100\eqref{eq:appC-design} is fitted with $t_e = t'$. The placebo value of the101true event $t_e \in \mathcal{T}'$ is102\begin{equation}103  p^{\mathrm{plac}} \;=\; \frac{1}{|\mathcal{T}'|}\sum_{t' \in \mathcal{T}'}104  \mathbf{1}\bigl[\,|t^{(2)}_{t'}| \ge |t^{(2)}_{t_e}|\,\bigr]105  \;=\; \frac{\mathrm{rank}\bigl(|t^{(2)}_{t_e}|\bigr)}{|\mathcal{T}'|},106\end{equation}107where $\mathrm{rank}$ is the descending rank of the true event's absolute108$t$-ratio among the $|\mathcal{T}'|$ values, ties counted as not better109(\texttt{>=}). Three properties follow directly. Because $t_e \in110\mathcal{T}'$, the sum contains the term $t' = t_e$, so $p^{\mathrm{plac}}111\ge 1/|\mathcal{T}'|$, which for the default window ($n = 101$, hence112$|\mathcal{T}'| = 65$) is about $0.015$: a placebo rank cannot be smaller,113however large the true statistic, and a reported value of $1/|\mathcal{T}'|$114means ``largest of all candidate dates'', not ``significant at that level''.115Because the statistic is a rank, it is invariant to any monotone116transformation of the $|t|$-ratios and in particular does not depend on the117HAC bandwidth through anything but the ordering it induces. And if the118candidate dates were exchangeable under a null hypothesis of no break119anywhere, $p^{\mathrm{plac}}$ would be a valid permutation-type $p$-value120with $P(p^{\mathrm{plac}} \le \alpha) \le \alpha$; they are not exchangeable,121since all $|\mathcal{T}'|$ regressions share the same $n$ observations and122adjacent candidates give nearly identical fits, so the value is reported as123a rank and read as such. The same construction gives the placebo rank of124the slope change from $t^{(3)}_{t'}$. The driver additionally computes the125rank restricted to the baseline candidates $\{t' < t_e\}$, which answers the126protocol's original question (is the reference date special relative to the127pre-period?) and is reported next to the full-range rank.128129\subsection{Bai--Perron multiple breaks by dynamic programming}130\label{sec:appC-bp}131132\paragraph{Segment costs and additivity.}133For a segment $[i, j)$ of the series with $j - i \ge h$, define134\begin{equation}135  S(i, j) \;=\; \min_{a, b \in \R}\ \sum_{t=i}^{j-1} (y_t - a - b\,t)^2,136  \label{eq:appC-segcost}137\end{equation}138the residual sum of squares of a linear trend fitted to the segment alone139(the implementation solves the two-parameter least-squares problem for each140segment with \texttt{numpy.linalg.lstsq}; with \texttt{trend=False} the141regressor $t$ is dropped and $S(i,j)$ is the centred sum of squares). The142model with $m$ breaks $0 < \tau_1 < \dots < \tau_m < n$ assigns its own143$(a, b)$ to each of the $m+1$ segments $[\tau_{k-1}, \tau_k)$ with144$\tau_0 = 0$ and $\tau_{m+1} = n$. Because the parameters of different145segments are distinct and each observation enters exactly one segment, the146residual sum of squares of the whole model separates,147\begin{equation}148  \mathrm{SSR}(\tau_1, \dots, \tau_m) \;=\; \sum_{k=1}^{m+1} S(\tau_{k-1}, \tau_k),149  \label{eq:appC-additive}150\end{equation}151and the global minimisation over $\tau$ decomposes: the optimal partition of152the first $j$ observations into $m+1$ segments contains, as its first $m$153segments, an optimal partition of the first $\tau_m$ observations into $m$154segments. This is the principle of optimality, and it is the only property of155the cost that the dynamic programme uses; it holds for any cost that is a156sum over segments of a function of the segment alone, which is why the same157recursion serves the piecewise-constant model of Section~\ref{sec:appC-pelt}.158It fails for a model with parameters shared across segments (a common slope,159or the seasonal dummies of the ITS), which is why the segmented regression160carries no seasonal terms and the series is seasonally adjusted beforehand161by subtracting the baseline-period month means162(\texttt{run\_structural\_change.py}).163164\paragraph{The recursion.}165Let $C_m(j)$ be the minimum of \eqref{eq:appC-additive} over partitions of166observations $0, \dots, j-1$ into $m+1$ admissible segments. Then167\begin{equation}168  C_0(j) = S(0, j), \qquad169  C_m(j) = \min_{\,mh \le i \le j-h} \bigl\{ C_{m-1}(i) + S(i, j) \bigr\},170  \qquad j \ge (m+1)h,171  \label{eq:appC-dp}172\end{equation}173the lower limit of $i$ ensuring that $m$ segments of length at least $h$174fit before the last break and the upper limit that the last segment has175length at least $h$. The argument minimiser $i^{\ast}_m(j)$ is stored, and176the breaks of the $m$-break solution for the full series are recovered177backward: $\tau_m = i^{\ast}_m(n)$, $\tau_{m-1} = i^{\ast}_{m-1}(\tau_m)$,178and so on, which is what the dictionary \texttt{B[m][j]} of the179implementation accumulates forward instead. This is the dynamic programme180that \citet{bai2003computation} propose for the global minimiser of181\citet{bai1998breaks}; the earlier literature used it for the182piecewise-constant case, and its correctness requires nothing beyond183\eqref{eq:appC-additive} and the induction on $m$: if $C_{m-1}(i)$ is the184true minimum for every $i$, then any partition of $[0, j)$ into $m+1$185segments has a last segment $[i, j)$ and a prefix partition of cost at least186$C_{m-1}(i)$, so its cost is at least the right-hand side of187\eqref{eq:appC-dp}, which is attained.188189\paragraph{Complexity.}190The table of segment costs has one entry per admissible pair $(i, j)$, that191is $\binom{n-h+1}{2} + O(n)$ entries, of order $n^2/2$. The recursion192\eqref{eq:appC-dp} evaluates, for each $m \le M$ and each $j$, a minimum over193at most $n$ candidates, so its cost is $O(n^2 M)$ operations; the194back-tracking costs $O(M)$. The implementation fills the cost table by195solving each segment's least-squares problem independently, at a cost196proportional to the segment length, so the table costs $O(n^3)$ rather than197the $O(n^2)$ obtainable by updating running sums as in198\citet{bai2003computation}; for $n$ of the order of one hundred, as here,199the difference is a fraction of a second and the simpler code was preferred200for its transparency. Trimming enters only through the admissibility201condition $j - i \ge h$; with $h = \max\{8, \lfloor 0.15\,n \rfloor\}$ and202$M = 3$, the default window of $n = 101$ months gives $h = 15$ and a203feasible region $(m+1)h \le n$ for every $m \le 3$, so all four candidate204models exist.205206\paragraph{Information criteria as implemented.}207For $m = 0, \dots, M$ let $\mathrm{SSR}_m = C_m(n)$ and let $p_m = k(m+1) +208m$ be the parameter count, with $k = 2$ for the trend model ($k = 1$ for the209mean model): one intercept and one slope per segment, plus the $m$ break210positions, counted as parameters by convention. The two criteria computed by211the code are212\begin{align}213  \mathrm{BIC}(m) &= n \log\frac{\mathrm{SSR}_m}{n} + p_m \log n, \label{eq:appC-bic}\\214  \mathrm{LWZ}(m) &= n \log\frac{\mathrm{SSR}_m}{n - p_m} + p_m\, c_0\, (\log n)^{2.1}, \qquad c_0 = 0.299, \label{eq:appC-lwz}215\end{align}216and the selected number of breaks is $\hat m = \arg\min_m$ of each. The BIC217form \eqref{eq:appC-bic} is the Gaussian log-likelihood, maximised over the218error variance, in which $-2\log\hat L = n\log(\mathrm{SSR}/n) + \text{const}$,219plus the Schwarz penalty $p \log n$ \citep{schwarz1978bic}; the two220differences from a textbook BIC are that the break positions are counted221among the parameters and that the constant and the segment variances are222treated as common, both as in \citet{bai2003computation}. The LWZ form223\eqref{eq:appC-lwz} is the modified Schwarz criterion of224\citet{liu1997segmented}, whose penalty grows as $(\log n)^{2+\delta_0}$ with225$\delta_0 = 0.1$ and whose variance estimate divides by $n - p_m$; the226constant $c_0 = 0.299$ is the value they recommend and the one used in the227simulations of \citet{bai2003computation}, which find that LWZ selects fewer228spurious breaks than BIC under serial correlation at the cost of some229power. The comparison across $m$ is legitimate because every $\mathrm{SSR}_m$230is a global minimum over the same observations and the same admissible231region; a sequential procedure that fixed earlier breaks before adding the232next would not have this property and would need the sup-$F$ machinery that233is not implemented. Neither criterion yields a confidence set for the break234dates; the text reports the agreement of the two selections and the235sensitivity to the truncation month instead.236237\subsection{PELT: cost, penalty and pruning}238\label{sec:appC-pelt}239240\paragraph{Objective.}241With the $\ell_2$ cost $\mathcal{C}(y_{i:j}) = \sum_{t=i}^{j-1} (y_t -242\bar y_{i:j})^2$ for a segment $[i,j)$ and a penalty $\lambda > 0$ per243changepoint, the problem of equation~\eqref{eq:pelt} of Part~IV is244\begin{equation}245  F(n) \;=\; \min_{m,\ \tau_1 < \dots < \tau_m}\ \Bigl\{ \sum_{k=1}^{m+1} \mathcal{C}(y_{\tau_{k-1}:\tau_k}) + \lambda m \Bigr\},246  \label{eq:appC-peltobj}247\end{equation}248and, by the same additivity argument as in Section~\ref{sec:appC-bp}, it249satisfies the optimal-partitioning recursion of \citet{jackson2005optimal},250\begin{equation}251  F(0) = -\lambda, \qquad F(j) \;=\; \min_{0 \le i < j}\ \bigl\{ F(i) + \mathcal{C}(y_{i:j}) + \lambda \bigr\},252  \label{eq:appC-op}253\end{equation}254where the number of breaks is not fixed in advance but chosen by the255penalty; the convention $F(0) = -\lambda$ cancels the penalty charged to the256first segment. Solving \eqref{eq:appC-op} for all $j$ costs $O(n^2)$ cost257evaluations.258259\paragraph{The pruning condition.}260\citet{killick2012pelt} observe that if there exists a constant $K$ such that261for all $i < j < l$262\begin{equation}263  \mathcal{C}(y_{i:j}) + \mathcal{C}(y_{j:l}) + K \;\le\; \mathcal{C}(y_{i:l}),264  \label{eq:appC-Kcond}265\end{equation}266then any candidate $i$ for which267\begin{equation}268  F(i) + \mathcal{C}(y_{i:j}) + K \;\ge\; F(j)269  \label{eq:appC-prune}270\end{equation}271can never be the last changepoint before any later $l > j$ and may be272discarded from the candidate set. The proof is two lines. For $l > j$, by273\eqref{eq:appC-Kcond} and \eqref{eq:appC-prune},274$F(i) + \mathcal{C}(y_{i:l}) + \lambda \ge F(i) + \mathcal{C}(y_{i:j}) +275\mathcal{C}(y_{j:l}) + K + \lambda \ge F(j) + \mathcal{C}(y_{j:l}) +276\lambda$, so the partition that places a changepoint at $j$ after the277optimal prefix of $[0,j)$ is at least as good as any that ends its last278segment at $i$; hence $i$ never needs to be examined again. For the279$\ell_2$ cost, \eqref{eq:appC-Kcond} holds with $K = 0$, because adding a280split point to a segment can only reduce its residual sum of squares (the281split model nests the unsplit one). The recursion with pruning examines, at282each $j$, only the candidates not yet discarded; under the regularity283conditions of \citeauthor{killick2012pelt} (changepoints whose number grows284linearly with $n$) the expected number of surviving candidates is bounded and285the total cost is linear in $n$, whence the name. For a short series with286few changes, as here, the pruning saves little and the method is simply an287exact solution of \eqref{eq:appC-peltobj}; the point of using it is288exactness at a given penalty, not speed. The implementation289(\texttt{ruptures.Pelt}, \citealp{truong2020ruptures}) is called with290\texttt{model="l2"}, \texttt{min\_size=6} and \texttt{jump=1}, the last two291restricting segments to at least six months and allowing a changepoint at292every index; the library returns the segment end points including $n$, which293the wrapper drops.294295\paragraph{The penalty $2\sigma^2 \log n$.}296Suppose the observations are independent Gaussian with piecewise-constant297mean and a common variance $\sigma^2$. Twice the negative log-likelihood of298a partition, up to a constant, is $\sigma^{-2}\sum_k \mathcal{C}(y_{\tau_{k-1}:\tau_k})$.299Each additional changepoint adds two parameters, a segment mean and a300position, so the Schwarz criterion in this scale adds $2\log n$ per301changepoint. Multiplying through by $\sigma^2$ to express the criterion in302the units of the cost gives303\begin{equation}304  \sum_k \mathcal{C}(y_{\tau_{k-1}:\tau_k}) + 2\sigma^2 \log n \cdot m,305\end{equation}306which is \eqref{eq:appC-peltobj} with $\lambda = 2\sigma^2\log n$. The307implementation replaces $\sigma^2$ by the sample variance of the whole308series, $\widehat{\mathrm{Var}}(y) = n^{-1}\sum_t (y_t - \bar y)^2$309(\texttt{numpy.var}, population form). This is the conservative choice310discussed in Part~IV: in a series with a trend or with genuine level311shifts, the overall variance exceeds the within-segment variance, so the312penalty is larger than the within-segment Schwarz penalty and fewer313changepoints are declared. \citet{yao1988schwarz} proves that the Schwarz314criterion, with a known or consistently estimated $\sigma^2$, selects the315true number of changes with probability tending to one in the316piecewise-constant Gaussian model; the result does not extend to a series317whose mean is a trend, for which a piecewise-constant fit must spend318changepoints to track the slope, and the PELT dates of this study are read319with that in mind.320321\subsection{Bayesian online changepoint detection}322\label{sec:appC-bocpd}323324\paragraph{Conjugate updates.}325Within a run the observations are modelled as326$y \mid \mu, \sigma^2 \sim \mathcal{N}(\mu, \sigma^2)$ with the327Normal--Inverse-Gamma prior328$\mu \mid \sigma^2 \sim \mathcal{N}(\mu_0, \sigma^2/\kappa_0)$,329$\sigma^2 \sim \mathrm{IG}(\alpha_0, \beta_0)$, whose joint density is330proportional to331\begin{equation}332  (\sigma^2)^{-\alpha_0 - 3/2} \exp\Bigl\{ -\frac{\beta_0}{\sigma^2} - \frac{\kappa_0 (\mu - \mu_0)^2}{2\sigma^2} \Bigr\}.333\end{equation}334Multiplying by the likelihood of one observation $y$, proportional to335$(\sigma^2)^{-1/2}\exp\{-(y-\mu)^2/(2\sigma^2)\}$, and completing the square336in $\mu$,337\begin{equation}338  \kappa_0 (\mu - \mu_0)^2 + (y - \mu)^2339  \;=\; (\kappa_0 + 1)\Bigl(\mu - \frac{\kappa_0\mu_0 + y}{\kappa_0 + 1}\Bigr)^2 + \frac{\kappa_0 (y - \mu_0)^2}{\kappa_0 + 1},340\end{equation}341shows that the posterior is again Normal--Inverse-Gamma with342\begin{equation}343  \mu_1 = \frac{\kappa_0 \mu_0 + y}{\kappa_0 + 1}, \quad344  \kappa_1 = \kappa_0 + 1, \quad345  \alpha_1 = \alpha_0 + \tfrac12, \quad346  \beta_1 = \beta_0 + \frac{\kappa_0 (y - \mu_0)^2}{2(\kappa_0 + 1)},347  \label{eq:appC-nigupdate}348\end{equation}349which are the four one-step updates of the implementation350(\texttt{mu\_new}, \texttt{kappa\_new}, \texttt{alpha\_new},351\texttt{beta\_new}). Applying \eqref{eq:appC-nigupdate} $r$ times and352collecting terms gives the batch form quoted in Part~IV, with353$\kappa_r = \kappa_0 + r$, $\mu_r = (\kappa_0\mu_0 + r\bar y)/\kappa_r$,354$\alpha_r = \alpha_0 + r/2$ and355$\beta_r = \beta_0 + \tfrac12\sum_{i \le r}(y_i - \bar y)^2 + \kappa_0 r (\bar y - \mu_0)^2 / (2\kappa_r)$;356the equality of the two forms is a matter of algebra (induction on $r$) and357is the reason the online recursion can be trusted to reproduce the batch358posterior.359360\paragraph{Student-$t$ predictive.}361The predictive density of the next observation integrates the Gaussian362likelihood against the current Normal--Inverse-Gamma posterior. Integrating363$\mu$ first gives $y \mid \sigma^2 \sim \mathcal{N}(\mu_r, \sigma^2 (1 +3641/\kappa_r))$; integrating $\sigma^2 \sim \mathrm{IG}(\alpha_r, \beta_r)$365against this scale mixture of normals gives366\begin{equation}367  p(y \mid r) \;=\; \int \mathcal{N}\Bigl(y;\ \mu_r,\ \sigma^2\frac{\kappa_r + 1}{\kappa_r}\Bigr)\,\mathrm{IG}(\sigma^2; \alpha_r, \beta_r)\, d\sigma^2368  \;=\; t_{2\alpha_r}\Bigl( y;\ \mu_r,\ \frac{\beta_r(\kappa_r + 1)}{\alpha_r \kappa_r} \Bigr),369  \label{eq:appC-student}370\end{equation}371a Student-$t$ density with $2\alpha_r$ degrees of freedom, location $\mu_r$372and squared scale $\beta_r(\kappa_r+1)/(\alpha_r\kappa_r)$: the standard373result that a normal with inverse-gamma mixing on the variance is a374$t$ with degrees of freedom twice the shape and scale equal to the375rate over the shape times the normal's variance factor. The code evaluates376\eqref{eq:appC-student} through \texttt{scipy.stats.t.pdf} with377\texttt{df = 2*alpha} and \texttt{scale = sqrt(beta*(kappa+1)/(alpha*kappa))},378vectorised over the run lengths.379380\paragraph{Run-length recursion and the mass on $r = 0$.}381Let $R_t(r) = P(r_t = r \mid y_{1:t})$ be the filtered run-length posterior382after $t$ observations, with $R_0(0) = 1$, and let $\pi_t(r) = p(y_{t+1} \mid383r_t = r, y_{1:t})$ be the predictive \eqref{eq:appC-student} evaluated with384the hyperparameters of the run of length $r$ ending at $t$. With a constant385hazard $H$ (a geometric prior on run lengths), the joint recursion of386\citet{adams2007bocpd} is, for the growth and the change moves,387\begin{equation}388  \tilde R_{t+1}(r+1) = R_t(r)\,\pi_t(r)\,(1 - H), \qquad389  \tilde R_{t+1}(0) = \sum_{r=0}^{t} R_t(r)\,\pi_t(r)\, H,390  \label{eq:appC-rl}391\end{equation}392followed by normalisation, $R_{t+1}(r) = \tilde R_{t+1}(r) / \sum_{r'}393\tilde R_{t+1}(r')$. Write $Z_t = \sum_{r} R_t(r)\pi_t(r)$ for the marginal394predictive likelihood of $y_{t+1}$. Then $\sum_{r \ge 1}\tilde R_{t+1}(r) =395(1-H) Z_t$ and $\tilde R_{t+1}(0) = H Z_t$, so the normaliser is $Z_t$ and396\begin{equation}397  R_{t+1}(0) \;=\; \frac{H Z_t}{H Z_t + (1-H) Z_t} \;=\; H \qquad \text{for every } t.398  \label{eq:appC-r0}399\end{equation}400The filtered probability that a changepoint occurs \emph{at} the current401step therefore equals the prior hazard identically, whatever the data: the402run-length posterior carries information about where the last change was403only through the relative masses at $r \ge 1$, which are reweighted by the404predictive likelihoods, and not through the mass at $r = 0$, which the405constant hazard fixes. This is why the detection statistics of Part~IV are406not $R_t(0)$. The first, $P(r_t \le 3 \mid y_{1:t}) = \sum_{r=0}^{3} R_t(r)$,407is $H$ plus the mass on runs of length one to three, that is, on changes408that occurred one to three months earlier and have since accumulated409evidence; its value exceeds $H$ only when the recent observations are410better explained by a fresh run than by the continuation of the old one. The411second is the maximum-a-posteriori run length $\hat r_t = \arg\max_r R_t(r)$,412whose resets from a value above twelve to a value at most two are the413online detector's changepoints. The implementation keeps the full414$(n+1)\times(n+1)$ matrix $R$, adds $10^{-300}$ to the normaliser against415underflow, and initialises $\mu_0$ and $\beta_0$ from the first six416observations with $\kappa_0 = \alpha_0 = 1$; the prior mean run length417$1/H = 60$ months is the only tuning constant. The recursion is the filtered418one; the smoothed posterior of \citet{fearnhead2006exact} would require a419backward pass and answers the retrospective question that the offline420methods of Sections~\ref{sec:appC-bp} and~\ref{sec:appC-pelt} already421address.422423\subsection{The Gini coefficient: rank form and Lorenz form}424\label{sec:appC-gini}425426Let $c_{(1)} \le c_{(2)} \le \dots \le c_{(n)}$ be the citation counts of427the $n$ works of a publication year sorted in ascending order, $S =428\sum_i c_{(i)} > 0$ their total, and $L_i = \sum_{j \le i} c_{(j)} / S$ the429cumulative share held by the $i$ least-cited works, with $L_0 = 0$ and430$L_n = 1$. The Lorenz curve \citep{lorenz1905concentration} is the polygon431through the points $(i/n, L_i)$, and the Gini coefficient is defined as one432minus twice the area $A$ under it \citep{ceriani2012gini}. By the433trapezoidal rule, which is exact for a polygon,434\begin{equation}435  A \;=\; \sum_{i=1}^{n} \frac{1}{n}\cdot\frac{L_{i-1} + L_i}{2}436    \;=\; \frac{1}{2n}\Bigl( 2\sum_{i=1}^{n} L_i - L_n \Bigr)437    \;=\; \frac{1}{n}\sum_{i=1}^{n} L_i - \frac{1}{2n}.438\end{equation}439Exchanging the order of summation in $\sum_i L_i$,440\begin{equation}441  \sum_{i=1}^{n} L_i \;=\; \frac{1}{S}\sum_{i=1}^{n}\sum_{j=1}^{i} c_{(j)}442    \;=\; \frac{1}{S}\sum_{j=1}^{n} (n - j + 1)\, c_{(j)}443    \;=\; (n+1) - \frac{1}{S}\sum_{j=1}^{n} j\, c_{(j)},444\end{equation}445so that446\begin{equation}447  G \;=\; 1 - 2A448    \;=\; 1 - \frac{2(n+1)}{n} + \frac{2}{nS}\sum_{j=1}^{n} j\, c_{(j)} + \frac{1}{n}449    \;=\; \frac{2\sum_{j=1}^{n} j\, c_{(j)}}{n \sum_{j} c_{(j)}} - \frac{n+1}{n},450  \label{eq:appC-gini}451\end{equation}452which is equation~\eqref{eq:gini} of Part~IV and the expression that453\texttt{bibliometrics/03\_citation\_impact.py} evaluates in SQL, the rank454$j$ being the window function \texttt{row\_number()} partitioned by year and455ordered by the citation count and then by the work identifier.456The secondary ordering by work identifier is needed only to make the rank a457permutation when counts tie; since tied values receive consecutive ranks and458contribute $c \cdot (j + j')$ to the numerator whichever of the two tied459works gets the smaller rank, the value of $G$ does not depend on how ties are460broken. The bounds follow from \eqref{eq:appC-gini}: $G = 0$ when all461$c_{(j)}$ are equal (then $\sum_j j c_{(j)} = c\,n(n+1)/2$ and the two terms462cancel), and $G = 1 - 1/n$ when a single work holds all citations (then463$\sum_j j c_{(j)} = nS$), so the maximum attainable value is $1 - 1/n$ rather464than one and the coefficient is not normalised by that factor. Uncited works465enter with $c_{(j)} = 0$ at the lowest ranks; adding $u$ uncited works to a466year leaves $\sum_j j c_{(j)}$ shifted by $uS$ and $n$ increased by $u$, and467one checks from \eqref{eq:appC-gini} that $G$ increases, which is the468direction stated in Part~IV and the reason the uncited share accompanies the469coefficient in the tables.470471\subsection{Rao--Stirling and Leinster--Cobbold at order two}472\label{sec:appC-diversity}473474Let $p \in \R^K$ be a probability vector over categories, $\delta_{ij} \in475[0,1]$ a dissimilarity with $\delta_{ii} = 0$, and $Z = (z_{ij})$ the476similarity matrix $z_{ij} = 1 - \delta_{ij}$, so that $z_{ii} = 1$. The477Rao--Stirling index \citep{rao1982diversity, stirling2007diversity} is478$\Delta_{\mathrm{RS}}(p) = \sum_{i \ne j} p_i p_j \delta_{ij} = p^{\top}\Delta p$479with $\Delta = (\delta_{ij})$, the diagonal contributing nothing. The480Leinster--Cobbold diversity of order $q$ \citep{leinster2012diversity} is481${}^{q}D^{Z}(p) = \bigl(\sum_{i : p_i > 0} p_i (Zp)_i^{q-1}\bigr)^{1/(1-q)}$;482at $q = 2$ the exponent $1/(1-q) = -1$ and the inner power is one, so483\begin{equation}484  {}^{2}D^{Z}(p) \;=\; \Bigl(\sum_i p_i (Zp)_i\Bigr)^{-1} \;=\; \frac{1}{p^{\top} Z p}.485\end{equation}486Since $Z = \mathbf{1}\mathbf{1}^{\top} - \Delta$ entrywise (using $z_{ii} = 1487= 1 - \delta_{ii}$ on the diagonal as well), and $p^{\top}\mathbf{1}\mathbf{1}^{\top}p488= (\sum_i p_i)^2 = 1$,489\begin{equation}490  p^{\top} Z p \;=\; 1 - p^{\top}\Delta p \;=\; 1 - \Delta_{\mathrm{RS}}(p),491  \qquad\text{hence}\qquad492  {}^{2}D^{Z}(p) \;=\; \frac{1}{1 - \Delta_{\mathrm{RS}}(p)},493  \label{eq:appC-lc2}494\end{equation}495the identity quoted in Part~IV\@. It shows that at order two the two496families carry the same information: ${}^{2}D^{Z}$ is a strictly increasing497function of $\Delta_{\mathrm{RS}}$ on $[0, 1)$, and the ordering of years or498categories by either index is the same. They differ in scale and in499behaviour near saturation. $\Delta_{\mathrm{RS}}$ is bounded by the mean500dissimilarity and compresses differences among very even distributions;501${}^{2}D^{Z}$ is an effective number of categories, equal to $K$ for the502uniform distribution over $K$ mutually dissimilar categories503($\Delta_{\mathrm{RS}} = 1 - 1/K$) and to one for a distribution concentrated504on a single category ($\Delta_{\mathrm{RS}} = 0$), and it expands the505compressed end. The identity requires only $z_{ii} = 1$ and $z_{ij} = 1 -506\delta_{ij}$; it does not require $\delta$ to be a metric, and the507dissimilarity used, one minus a cosine similarity of category centroids,508is not one. The script \texttt{bibliometrics/04\_diversity\_cagr.py}509computes both quantities from the yearly category mix exactly as510\eqref{eq:appC-lc2} prescribes, $\Delta_{\mathrm{RS}} = p^{\top}(1 - Z)p$511with the diagonal of $Z$ set to one and ${}^{2}D^{Z} = 1/(p^{\top}Zp)$, and512attaches a percentile bootstrap interval to $\Delta_{\mathrm{RS}}$ by513multinomial resampling of the works over categories ($B = 500$, seed 44),514which is the sampling model under which the category counts of a year are515a multinomial draw of fixed size; the interval of ${}^{2}D^{Z}$ follows by516the monotone transformation.517518\subsection{Modularity of a directed weighted graph at a fixed partition}519\label{sec:appC-modularity}520521Let $W \in \R_{\ge 0}^{K \times K}$ be the flow matrix of a citing year,522$W_{ij}$ the number of citations from category $i$ to category $j$, with523$m = \sum_{ij} W_{ij}$, out-strength $k^{\mathrm{out}}_i = \sum_j W_{ij}$ and524in-strength $k^{\mathrm{in}}_j = \sum_i W_{ij}$, and let $g : \{1, \dots, K\}525\to \mathcal{G}$ be a given assignment of categories to communities. The526statistic implemented by \texttt{modularity()} in527\texttt{knowledge\_graph/02\_category\_flows.py} is528\begin{equation}529  Q(W, g) \;=\; \frac{1}{m}\sum_{i,j}\Bigl( W_{ij} - \frac{k^{\mathrm{out}}_i k^{\mathrm{in}}_j}{m} \Bigr)\mathbf{1}[g_i = g_j],530  \label{eq:appC-Q}531\end{equation}532the directed form of \citet{leicht2008directed}, which reduces to the533Newman--Girvan modularity \citep{newman2004community} when $W$ is symmetric534(then $k^{\mathrm{out}} = k^{\mathrm{in}} = k$ and $m = 2|E|$ for an535unweighted graph). Three facts are used in Part~IV\@. First, the null term536is the expected weight of the edge $i \to j$ under random rewiring that537preserves every node's out- and in-strength in expectation: a configuration538model in which each of the $m$ units of weight leaves node $i$ with539probability $k^{\mathrm{out}}_i/m$ and enters node $j$ with probability540$k^{\mathrm{in}}_j/m$ independently gives expected weight541$m \cdot (k^{\mathrm{out}}_i/m)(k^{\mathrm{in}}_j/m) = k^{\mathrm{out}}_i542k^{\mathrm{in}}_j/m$, and summing over $j$ recovers $k^{\mathrm{out}}_i$, so543the null model has the observed strengths. Second, $Q$ vanishes when all544nodes are in one community: then the indicator is identically one and545$\sum_{ij} W_{ij} = m = \sum_{ij} k^{\mathrm{out}}_i k^{\mathrm{in}}_j/m$.546Third, writing $e_{gg} = \sum_{i,j : g_i = g_j = g} W_{ij}/m$ for the fraction547of weight inside community $g$, $a^{\mathrm{out}}_g = \sum_{i : g_i = g}548k^{\mathrm{out}}_i/m$ and $a^{\mathrm{in}}_g$ likewise,549\begin{equation}550  Q \;=\; \sum_{g \in \mathcal{G}} \bigl( e_{gg} - a^{\mathrm{out}}_g\, a^{\mathrm{in}}_g \bigr),551  \label{eq:appC-Qcomm}552\end{equation}553so that $Q \le \sum_g e_{gg} \le 1$ and $Q \ge -\sum_g a^{\mathrm{out}}_g554a^{\mathrm{in}}_g \ge -1$; the value is positive when more weight stays555inside communities than the strength-preserving null predicts. The556self-share reported alongside, $\operatorname{tr}(W)/m = \sum_i e_{ii}$ at557the category partition, is the first term of \eqref{eq:appC-Qcomm} alone and558differs from $Q_{\mathrm{cat}}$ by the correction $\sum_i a^{\mathrm{out}}_i559a^{\mathrm{in}}_i$, which grows with the concentration of flows on a few560large categories. For the area partition, \eqref{eq:appC-Qcomm} is summed561over the eight areas with $e_{\alpha\alpha}$ the within-area share, which is562one minus the cross-area share $X_{\alpha,t}$ of Part~IV weighted by the563area's out-strength; $Q_{\mathrm{area}}$ is therefore a strength-weighted,564null-corrected average of the within-area shares, and a fall in565$Q_{\mathrm{area}}$ can arise from a rise in boundary crossing or from a566shift of out-strength toward areas whose $a^{\mathrm{out}}_\alpha567a^{\mathrm{in}}_\alpha$ is large. Because the partition is fixed rather than568optimised, none of the resolution-limit results for maximised modularity569\citep{fortunato2007resolution} applies; $Q$ is here a descriptive index and570its comparison across years is meaningful only because the partition and571the category definitions are the same in every year.572573\subsection{Krippendorff's \texorpdfstring{$\alpha$}{alpha} with the MASI distance}574\label{sec:appC-alpha}575576\paragraph{General form.}577Krippendorff's $\alpha$ \citep{hayes2007krippendorff, artstein2008agreement}578is defined for any difference function $d(v, v') \ge 0$ on the values that579coders may assign, with $d(v, v) = 0$, as580\begin{equation}581  \alpha \;=\; 1 - \frac{D_o}{D_e},582  \label{eq:appC-alpha}583\end{equation}584where $D_o$ is the mean difference between the values assigned to the same585unit by different coders and $D_e$ the mean difference between all pairs of586values in the pooled data, irrespective of unit. With two coders $A$ and $B$587rating each of $n$ units, the pooled set contains $N = 2n$ values and588\begin{equation}589  D_o \;=\; \frac{1}{n}\sum_{u=1}^{n} d\bigl(a_u, b_u\bigr), \qquad590  D_e \;=\; \frac{1}{N(N-1)}\sum_{c \ne c'} d\bigl(v_c, v_{c'}\bigr)591        \;=\; \frac{1}{\binom{N}{2}}\sum_{c < c'} d\bigl(v_c, v_{c'}\bigr),592  \label{eq:appC-DoDe}593\end{equation}594the second equality using the symmetry of $d$. The coefficient is one when595the coders agree on every unit ($D_o = 0$), zero when the within-unit596differences are what chance pairing of the pooled values would produce597($D_o = D_e$), and negative when disagreement is systematic. Unlike Cohen's598$\kappa$, which compares observed agreement with the product of the two599coders' marginals, $\alpha$ pools the two coders' values into one600distribution for its chance term, so it does not reward coders for having601different marginal distributions; this and its accommodation of arbitrary602difference functions are the reasons it is the recommended coefficient for603set-valued annotation \citep{artstein2008agreement}.604605\paragraph{The MASI distance.}606For set-valued labels $x, y \subseteq \mathcal{K}$, \citet{passonneau2006masi}607defines the Measuring Agreement on Set-valued Items distance as608\begin{gather}609  d_{\mathrm{MASI}}(x, y) \;=\; 1 - J(x, y)\, M(x, y), \qquad610  J(x, y) = \frac{|x \cap y|}{|x \cup y|}, \nonumber\\611  M(x, y) =612  \begin{cases}613    1    & x = y,\\614    2/3  & x \subsetneq y \text{ or } y \subsetneq x,\\615    1/3  & x \cap y \ne \emptyset \text{ and neither contains the other},\\616    0    & x \cap y = \emptyset ,617  \end{cases}618  \label{eq:appC-masi}619\end{gather}620the Jaccard similarity attenuated by a monotonicity factor that621distinguishes nesting from mere overlap. The distance is zero only for622identical non-empty sets, equals one for disjoint sets, and treats the pair623(empty, empty) as identical ($d = 0$) by the convention of the624implementation, which returns zero before computing $J$ for two empty sets;625the pair (empty, non-empty) has $J = 0$ and $d = 1$.626627\paragraph{What the code computes.}628\texttt{taxonomy/validate\_sample.py} forms, for the two raters, the629observed term as the mean of $d_{\mathrm{MASI}}^2$ over the $n$ rated works630and the expected term as the mean of $d_{\mathrm{MASI}}^2$ over all631unordered pairs of the $2n$ pooled label sets (exhaustively when632$2n \le 1200$, otherwise over 200{,}000 random ordered pairs with $i \ne j$,633seed 44), and reports $1 - D_o/D_e$. Two remarks make this precise. The code634uses the constants $0.67$ and $0.33$ for the monotonicity factor rather than635$2/3$ and $1/3$, a rounding of the published definition that changes636distances by at most $0.0034$. More consequentially, it \emph{squares} the637MASI distance before averaging, that is, it applies638\eqref{eq:appC-alpha}--\eqref{eq:appC-DoDe} with $d = d_{\mathrm{MASI}}^2$.639Krippendorff's interval metric is the squared difference, and the code's640comment describes the computation as the ``interval form on the MASI641distance''; \citeauthor{passonneau2006masi}, in contrast, plugs642$d_{\mathrm{MASI}}$ itself into the $\alpha$ formula, and the agreement643packages that implement MASI follow her. The two conventions yield different644numbers from the same ratings---squaring compresses small disagreements and645so, as a rule, raises $\alpha$ when most disagreements are partial---and the646value reported in Part~IX is the squared-distance version, labelled as such.647Both are legitimate instances of \eqref{eq:appC-alpha}; what is not648legitimate is to compare the reported value with published MASI-$\alpha$649figures without this caveat. The mean Jaccard index reported next to it is650$n^{-1}\sum_u J(a_u, b_u)$ with $0/0 = 0$ and is a raw agreement rate with651no chance correction. Both statistics are computed on the full rated sample652and not on the 200-work subsample reserved for the principal investigator,653whose verdicts will add a third coder to \eqref{eq:appC-DoDe} once654checkpoint H-2 is signed.655656\subsection{The multi-label score rule and the tie-break as a total order}657\label{sec:appC-score}658659\paragraph{Score.}660For a work $w$ and a category $k$ the implementation661(\texttt{taxonomy/classify.py}, \texttt{score\_row}) computes five662components, one per signal type $\sigma \in \{\mathrm{T}, \mathrm{F},663\mathrm{X}, \mathrm{D}, \mathrm{kw}\}$,664\begin{align}665  s^{\sigma}_k(w) &\;=\; \min\Bigl\{ 1,\ \sum_{x \in \sigma(w)} \omega^{\sigma}_k(x) \Bigr\}666  \qquad (\sigma \ne \mathrm{kw}), \\667  s^{\mathrm{kw}}_k(w) &\;=\; \min\Bigl\{ 1,\ \sum_{(r, \omega, \phi) \in R_k} \omega\, \mathbf{1}[r \text{ matches } \phi(w)] \Bigr\},668\end{align}669where $\sigma(w)$ is the \emph{set} of the work's topic, subfield, arXiv670category or DBLP stream identifiers (Python \texttt{set()}, so a repeated671identifier contributes once), and the keyword loop stops at the first rule672that brings the running sum to one, which is equivalent to the capped sum.673The score is $s_k(w) = \sum_\sigma s^{\sigma}_k(w) \in [0, 5]$; a category674with $s_k(w) \ge \theta = 1$ is then vetoed if any exclusion pattern of675$Z_k$ matches the concatenation of title and abstract, in which case its676score is set to zero and it is not assigned. The set of assigned categories677is $C(w) = \{k : s_k(w) \ge \theta,\ \text{not vetoed}\}$, and the stored678score of an assigned category is $s_k(w)$ rounded to three decimals. Two679elementary properties justify the reading given in Part~IV\@. The cap makes680each component a bounded, monotone, subadditive function of the matched681weights: adding a signal never lowers a component, and no number of signals682of one type can contribute more than one. And because the threshold equals683the cap, a single weight-one signal of any type suffices for assignment684while signals of weight below one require corroboration from a different685type; formally, $s_k(w) \ge 1$ holds if and only if either some component686reaches one or the sum of the (uncapped) components across at least two687types reaches one.688689\paragraph{Primary category as the minimum of a total order.}690Let $\mathcal{P}(w) = \bigcup_{k \in C(w)} \mathrm{parents}(k)$ be the set691of categories declared parents of at least one assigned category (the692\texttt{child\_of} relation of the ontology). The implementation sorts the693assigned categories by the key694\begin{equation}695  \phi(k) \;=\; \bigl( -s_k(w),\ \mathbf{1}[k \in \mathcal{P}(w)],\ \mathrm{number}(k) \bigr) \in \R \times \{0,1\} \times \mathbb{N}696  \label{eq:appC-key}697\end{equation}698in lexicographic order and takes the first element; this is699$\kappa^{\ast}(w) = \arg\min_{k \in C(w)} \phi(k)$ and coincides with the700lexicographic arg-max of Part~IV (the sign of the first component reversed).701The lexicographic order on $\R \times \{0,1\} \times \mathbb{N}$ is a total702order, and $\phi$ is injective on $\mathcal{K}$ because the ontology numbers703are distinct, so the restriction of $\phi$ to $C(w)$ has a unique minimum704whenever $C(w) \ne \emptyset$: the primary category is well defined and does705not depend on the order in which categories are scanned. The second706component implements ``child before parent'': if $k$ and $k'$ are both707assigned with equal scores and $k'$ is a declared parent of $k$, then708$k' \in \mathcal{P}(w)$ while $k$ may not be, and $k$ precedes $k'$. The rule709is not transitive in the sense of a hierarchy, however. Membership in710$\mathcal{P}(w)$ depends only on whether $k$ is the parent of \emph{some}711assigned category, not of the specific competitor; a category that is the712parent of a low-scoring assigned child is demoted in the tie against an713unrelated category of equal score. This is a property of the key as714implemented, stated here so that the primary counts of Part~IX are read with715it; it affects only exact ties in a score rounded to three decimals, which716are frequent precisely among the categories whose signals are weight-one717keywords. The third component, the ontology number, is arbitrary and makes718the order total; no claim is attached to it.719720\subsection{The emerging-topic label and the absence of look-ahead}721\label{sec:appC-lookahead}722723\paragraph{Filtrations.}724Index months by integers $t = 0, 1, \dots, M$ (January 2018 to the last725complete month) and let the panel of \texttt{experiments/emerging\_detection.py}726be the pair of arrays $n_{k,t}$ (eligible month-precision works of topic $k$727with first month $t$) and $c_{k,t}$ (internal citations received by works of728topic $k$ from works with first month $t$). Define the filtration729\begin{equation}730  \mathcal{F}_T \;=\; \sigma\bigl( n_{k,t},\ c_{k,t} :\ k \in \mathcal{K}_{\mathrm{top}},\ t \le T \bigr),731  \label{eq:appC-filtration}732\end{equation}733the information contained in the panel columns up to and including month734$T$. A statistic is \emph{available at} $T$ if it is735$\mathcal{F}_T$-measurable. Each of the six features of Part~IV is a736function of sums of panel entries over windows $(a, b]$ with $b \le T$:737$g^{12}_{k,T}$ uses $(T-24, T]$, $\mathrm{acc}_{k,T}$ uses $(T-36, T]$,738$z_{k,T}$ uses $[T-27, T]$, $\mathrm{cit}_{k,T}$ uses $(T-24, T]$ of $c$,739$\mathrm{share}_{k,T}$ uses $(T-12, T]$ of every topic, and740$\mathrm{age}_{k,T}$ uses the cumulative sum up to $T$. All are741$\mathcal{F}_T$-measurable by construction. The implementation enforces this742by index arithmetic (\texttt{win(a, lo, hi)} slices columns \texttt{lo+1}743to \texttt{hi} inclusive with \texttt{hi = T\_idx}), not by convention.744745\paragraph{Label.}746The label $\ell_{k,T,h}$ of equation~\eqref{eq:label} is a function of747$V_k(T-h, T]$, $V_k(T, T+h]$, $V_k(T-12, T]$ and of the within-cutoff decile748of $g^{\mathrm{fut}}$ over eligible topics; it is therefore749$\mathcal{F}_{T+h}$-measurable and, because it depends on columns strictly750after $T$, not $\mathcal{F}_T$-measurable. The feature--label pair at cutoff751$T$ thus has the structure of a prediction problem: predict an752$\mathcal{F}_{T+h}$-measurable outcome from an $\mathcal{F}_T$-measurable753input. The eligibility condition $V_k(T-12, T] \ge 20$ is754$\mathcal{F}_T$-measurable, so conditioning on it does not leak the outcome.755756\paragraph{Separation of training and test.}757Let $\mathcal{T}_{\mathrm{tr}}$ and $\mathcal{T}_{\mathrm{te}}$ be the758training and test cutoffs. Every quantity used to fit a model---features,759labels and the standardisation constants of the scaler, which is fitted on760training rows only---is $\mathcal{F}_{T^{\mathrm{tr}}_{\max} + h}$-measurable761with $T^{\mathrm{tr}}_{\max} = \max \mathcal{T}_{\mathrm{tr}}$. The762implementation takes as candidate cutoffs every third month of the panel,763which begins in January 2018 (January, April, July and October of each year),764and admits as training cutoffs those from December 2019 up to December 2021765for $h = 12$ and up to December 2020 for $h = 24$; the realised training766cutoffs therefore run from January 2020 to October 2021 ($h = 12$) or to767October 2020 ($h = 24$), so that $T^{\mathrm{tr}}_{\max} + h$ is October7682022 in both cases, and the first test cutoff is January 2023 (the last is769October 2024 at most, subject to the censoring condition below). The realised770lists are written to \texttt{results/emerging/metrics\_h12.json} and771\texttt{metrics\_h24.json} under \texttt{train\_cutoffs} and772\texttt{test\_cutoffs}.773Hence774\begin{equation}775  T^{\mathrm{tr}}_{\max} + h \;<\; \min \mathcal{T}_{\mathrm{te}},776  \label{eq:appC-gap}777\end{equation}778and the fitted predictor is measurable with respect to a $\sigma$-field779that precedes every test feature, let alone every test label, which closes780at $\max \mathcal{T}_{\mathrm{te}} + h \le M$. Condition \eqref{eq:appC-gap}781is the formal content of ``out of time'' in the sense of782\citet{bergmeir2012crossval}; it is stricter than the plan's wording, which783allowed training cutoffs through 2022; the code's rule is the one that784guarantees \eqref{eq:appC-gap}, and Part~IV describes the same rule. The test cutoffs785are further restricted to those with $T + h \le M$, so that no test label is786right-censored.787788\paragraph{What measurability does not cover.}789The filtration \eqref{eq:appC-filtration} is generated by the panel, and the790panel is built once, in 2026, from a corpus whose topic assignments,791eligibility flags, linkage and month-precision filter were all fixed with792knowledge of the whole period. Formally, the random objects $n_{k,t}$793themselves are not $\mathcal{F}_t$-measurable functionals of the literature794as it stood at $t$: they are functionals of the literature as OpenAlex795indexed and classified it by October 2026. The statement proved above is796therefore conditional: \emph{given} the 2026 panel, no feature at $T$ uses a797column after $T$ and no training quantity uses a column at or after the798first test cutoff. It is not the statement that an observer in 2021 could799have computed the training set, which would require a topic vocabulary and a800corpus frozen in 2021. Part~IV draws the distinction and names the801cluster-based extension that would narrow it; the derivation here makes802precise which half of the claim is proved and which is assumed.803
Files · ba2b7cb2d796858e41df4e8a2376a6db42e5a5df
Menu

Theme