| 392 |
392 |
predictive likelihood of $y_{t+1}$. Then $\sum_{r \ge 1}\tilde R_{t+1}(r) = |
| 393 |
393 |
(1-H) Z_t$ and $\tilde R_{t+1}(0) = H Z_t$, so the normaliser is $Z_t$ and |
| 394 |
394 |
\begin{equation} |
| 395 |
|
− R_{t+1}(0) \;=\; \frac{H Z_t}{H Z_t + (1-H) Z_t} \;=\; H \qquad \text{for every } t . |
|
395 |
+ R_{t+1}(0) \;=\; \frac{H Z_t}{H Z_t + (1-H) Z_t} \;=\; H \qquad \text{for every } t. |
| 396 |
396 |
\label{eq:appC-r0} |
| 397 |
397 |
\end{equation} |
| 398 |
398 |
The filtered probability that a changepoint occurs \emph{at} the current |
| 417 |
417 |
backward pass and answers the retrospective question that the offline |
| 418 |
418 |
methods of Sections~\ref{sec:appC-bp} and~\ref{sec:appC-pelt} already |
| 419 |
419 |
address. |
|
420 |
+ |
|
421 |
+\subsection{The Gini coefficient: rank form and Lorenz form} |
|
422 |
+\label{sec:appC-gini} |
|
423 |
+ |
|
424 |
+Let $c_{(1)} \le c_{(2)} \le \dots \le c_{(n)}$ be the citation counts of |
|
425 |
+the $n$ works of a publication year sorted in ascending order, $S = |
|
426 |
+\sum_i c_{(i)} > 0$ their total, and $L_i = \sum_{j \le i} c_{(j)} / S$ the |
|
427 |
+cumulative share held by the $i$ least-cited works, with $L_0 = 0$ and |
|
428 |
+$L_n = 1$. The Lorenz curve \citep{lorenz1905concentration} is the polygon |
|
429 |
+through the points $(i/n, L_i)$, and the Gini coefficient is defined as one |
|
430 |
+minus twice the area $A$ under it \citep{ceriani2012gini}. By the |
|
431 |
+trapezoidal rule, which is exact for a polygon, |
|
432 |
+\begin{equation} |
|
433 |
+ A \;=\; \sum_{i=1}^{n} \frac{1}{n}\cdot\frac{L_{i-1} + L_i}{2} |
|
434 |
+ \;=\; \frac{1}{2n}\Bigl( 2\sum_{i=1}^{n} L_i - L_n \Bigr) |
|
435 |
+ \;=\; \frac{1}{n}\sum_{i=1}^{n} L_i - \frac{1}{2n}. |
|
436 |
+\end{equation} |
|
437 |
+Exchanging the order of summation in $\sum_i L_i$, |
|
438 |
+\begin{equation} |
|
439 |
+ \sum_{i=1}^{n} L_i \;=\; \frac{1}{S}\sum_{i=1}^{n}\sum_{j=1}^{i} c_{(j)} |
|
440 |
+ \;=\; \frac{1}{S}\sum_{j=1}^{n} (n - j + 1)\, c_{(j)} |
|
441 |
+ \;=\; (n+1) - \frac{1}{S}\sum_{j=1}^{n} j\, c_{(j)}, |
|
442 |
+\end{equation} |
|
443 |
+so that |
|
444 |
+\begin{equation} |
|
445 |
+ G \;=\; 1 - 2A |
|
446 |
+ \;=\; 1 - \frac{2(n+1)}{n} + \frac{2}{nS}\sum_{j=1}^{n} j\, c_{(j)} + \frac{1}{n} |
|
447 |
+ \;=\; \frac{2\sum_{j=1}^{n} j\, c_{(j)}}{n \sum_{j} c_{(j)}} - \frac{n+1}{n}, |
|
448 |
+ \label{eq:appC-gini} |
|
449 |
+\end{equation} |
|
450 |
+which is equation~\eqref{eq:gini} of Part~IV and the expression that |
|
451 |
+\texttt{bibliometrics/03\_citation\_impact.py} evaluates in SQL, the rank |
|
452 |
+$j$ being the window function \texttt{row\_number()} partitioned by year and |
|
453 |
+ordered by the citation count and then by the work identifier. |
|
454 |
+The secondary ordering by work identifier is needed only to make the rank a |
|
455 |
+permutation when counts tie; since tied values receive consecutive ranks and |
|
456 |
+contribute $c \cdot (j + j')$ to the numerator whichever of the two tied |
|
457 |
+works gets the smaller rank, the value of $G$ does not depend on how ties are |
|
458 |
+broken. The bounds follow from \eqref{eq:appC-gini}: $G = 0$ when all |
|
459 |
+$c_{(j)}$ are equal (then $\sum_j j c_{(j)} = c\,n(n+1)/2$ and the two terms |
|
460 |
+cancel), and $G = 1 - 1/n$ when a single work holds all citations (then |
|
461 |
+$\sum_j j c_{(j)} = nS$), so the maximum attainable value is $1 - 1/n$ rather |
|
462 |
+than one and the coefficient is not normalised by that factor. Uncited works |
|
463 |
+enter with $c_{(j)} = 0$ at the lowest ranks; adding $u$ uncited works to a |
|
464 |
+year leaves $\sum_j j c_{(j)}$ shifted by $uS$ and $n$ increased by $u$, and |
|
465 |
+one checks from \eqref{eq:appC-gini} that $G$ increases, which is the |
|
466 |
+direction stated in Part~IV and the reason the uncited share accompanies the |
|
467 |
+coefficient in the tables. |
|
468 |
+ |
|
469 |
+\subsection{Rao--Stirling and Leinster--Cobbold at order two} |
|
470 |
+\label{sec:appC-diversity} |
|
471 |
+ |
|
472 |
+Let $p \in \R^K$ be a probability vector over categories, $\delta_{ij} \in |
|
473 |
+[0,1]$ a dissimilarity with $\delta_{ii} = 0$, and $Z = (z_{ij})$ the |
|
474 |
+similarity matrix $z_{ij} = 1 - \delta_{ij}$, so that $z_{ii} = 1$. The |
|
475 |
+Rao--Stirling index \citep{rao1982diversity, stirling2007diversity} is |
|
476 |
+$\Delta_{\mathrm{RS}}(p) = \sum_{i \ne j} p_i p_j \delta_{ij} = p^{\top}\Delta p$ |
|
477 |
+with $\Delta = (\delta_{ij})$, the diagonal contributing nothing. The |
|
478 |
+Leinster--Cobbold diversity of order $q$ \citep{leinster2012diversity} is |
|
479 |
+${}^{q}D^{Z}(p) = \bigl(\sum_{i : p_i > 0} p_i (Zp)_i^{q-1}\bigr)^{1/(1-q)}$; |
|
480 |
+at $q = 2$ the exponent $1/(1-q) = -1$ and the inner power is one, so |
|
481 |
+\begin{equation} |
|
482 |
+ {}^{2}D^{Z}(p) \;=\; \Bigl(\sum_i p_i (Zp)_i\Bigr)^{-1} \;=\; \frac{1}{p^{\top} Z p}. |
|
483 |
+\end{equation} |
|
484 |
+Since $Z = \mathbf{1}\mathbf{1}^{\top} - \Delta$ entrywise (using $z_{ii} = 1 |
|
485 |
+= 1 - \delta_{ii}$ on the diagonal as well), and $p^{\top}\mathbf{1}\mathbf{1}^{\top}p |
|
486 |
+= (\sum_i p_i)^2 = 1$, |
|
487 |
+\begin{equation} |
|
488 |
+ p^{\top} Z p \;=\; 1 - p^{\top}\Delta p \;=\; 1 - \Delta_{\mathrm{RS}}(p), |
|
489 |
+ \qquad\text{hence}\qquad |
|
490 |
+ {}^{2}D^{Z}(p) \;=\; \frac{1}{1 - \Delta_{\mathrm{RS}}(p)}, |
|
491 |
+ \label{eq:appC-lc2} |
|
492 |
+\end{equation} |
|
493 |
+the identity quoted in Part~IV\@. It shows that at order two the two |
|
494 |
+families carry the same information: ${}^{2}D^{Z}$ is a strictly increasing |
|
495 |
+function of $\Delta_{\mathrm{RS}}$ on $[0, 1)$, and the ordering of years or |
|
496 |
+categories by either index is the same. They differ in scale and in |
|
497 |
+behaviour near saturation. $\Delta_{\mathrm{RS}}$ is bounded by the mean |
|
498 |
+dissimilarity and compresses differences among very even distributions; |
|
499 |
+${}^{2}D^{Z}$ is an effective number of categories, equal to $K$ for the |
|
500 |
+uniform distribution over $K$ mutually dissimilar categories |
|
501 |
+($\Delta_{\mathrm{RS}} = 1 - 1/K$) and to one for a distribution concentrated |
|
502 |
+on a single category ($\Delta_{\mathrm{RS}} = 0$), and it expands the |
|
503 |
+compressed end. The identity requires only $z_{ii} = 1$ and $z_{ij} = 1 - |
|
504 |
+\delta_{ij}$; it does not require $\delta$ to be a metric, and the |
|
505 |
+dissimilarity used, one minus a cosine similarity of category centroids, |
|
506 |
+is not one. The script \texttt{bibliometrics/04\_diversity\_cagr.py} |
|
507 |
+computes both quantities from the yearly category mix exactly as |
|
508 |
+\eqref{eq:appC-lc2} prescribes, $\Delta_{\mathrm{RS}} = p^{\top}(1 - Z)p$ |
|
509 |
+with the diagonal of $Z$ set to one and ${}^{2}D^{Z} = 1/(p^{\top}Zp)$, and |
|
510 |
+attaches a percentile bootstrap interval to $\Delta_{\mathrm{RS}}$ by |
|
511 |
+multinomial resampling of the works over categories ($B = 500$, seed 44), |
|
512 |
+which is the sampling model under which the category counts of a year are |
|
513 |
+a multinomial draw of fixed size; the interval of ${}^{2}D^{Z}$ follows by |
|
514 |
+the monotone transformation. |
|
515 |
+ |
|
516 |
+\subsection{Modularity of a directed weighted graph at a fixed partition} |
|
517 |
+\label{sec:appC-modularity} |
|
518 |
+ |
|
519 |
+Let $W \in \R_{\ge 0}^{K \times K}$ be the flow matrix of a citing year, |
|
520 |
+$W_{ij}$ the number of citations from category $i$ to category $j$, with |
|
521 |
+$m = \sum_{ij} W_{ij}$, out-strength $k^{\mathrm{out}}_i = \sum_j W_{ij}$ and |
|
522 |
+in-strength $k^{\mathrm{in}}_j = \sum_i W_{ij}$, and let $g : \{1, \dots, K\} |
|
523 |
+\to \mathcal{G}$ be a given assignment of categories to communities. The |
|
524 |
+statistic implemented by \texttt{modularity()} in |
|
525 |
+\texttt{knowledge\_graph/02\_category\_flows.py} is |
|
526 |
+\begin{equation} |
|
527 |
+ Q(W, g) \;=\; \frac{1}{m}\sum_{i,j}\Bigl( W_{ij} - \frac{k^{\mathrm{out}}_i k^{\mathrm{in}}_j}{m} \Bigr)\mathbf{1}[g_i = g_j], |
|
528 |
+ \label{eq:appC-Q} |
|
529 |
+\end{equation} |
|
530 |
+the directed form of \citet{leicht2008directed}, which reduces to the |
|
531 |
+Newman--Girvan modularity \citep{newman2004community} when $W$ is symmetric |
|
532 |
+(then $k^{\mathrm{out}} = k^{\mathrm{in}} = k$ and $m = 2|E|$ for an |
|
533 |
+unweighted graph). Three facts are used in Part~IV\@. First, the null term |
|
534 |
+is the expected weight of the edge $i \to j$ under random rewiring that |
|
535 |
+preserves every node's out- and in-strength in expectation: a configuration |
|
536 |
+model in which each of the $m$ units of weight leaves node $i$ with |
|
537 |
+probability $k^{\mathrm{out}}_i/m$ and enters node $j$ with probability |
|
538 |
+$k^{\mathrm{in}}_j/m$ independently gives expected weight |
|
539 |
+$m \cdot (k^{\mathrm{out}}_i/m)(k^{\mathrm{in}}_j/m) = k^{\mathrm{out}}_i |
|
540 |
+k^{\mathrm{in}}_j/m$, and summing over $j$ recovers $k^{\mathrm{out}}_i$, so |
|
541 |
+the null model has the observed strengths. Second, $Q$ vanishes when all |
|
542 |
+nodes are in one community: then the indicator is identically one and |
|
543 |
+$\sum_{ij} W_{ij} = m = \sum_{ij} k^{\mathrm{out}}_i k^{\mathrm{in}}_j/m$. |
|
544 |
+Third, writing $e_{gg} = \sum_{i,j : g_i = g_j = g} W_{ij}/m$ for the fraction |
|
545 |
+of weight inside community $g$, $a^{\mathrm{out}}_g = \sum_{i : g_i = g} |
|
546 |
+k^{\mathrm{out}}_i/m$ and $a^{\mathrm{in}}_g$ likewise, |
|
547 |
+\begin{equation} |
|
548 |
+ Q \;=\; \sum_{g \in \mathcal{G}} \bigl( e_{gg} - a^{\mathrm{out}}_g\, a^{\mathrm{in}}_g \bigr), |
|
549 |
+ \label{eq:appC-Qcomm} |
|
550 |
+\end{equation} |
|
551 |
+so that $Q \le \sum_g e_{gg} \le 1$ and $Q \ge -\sum_g a^{\mathrm{out}}_g |
|
552 |
+a^{\mathrm{in}}_g \ge -1$; the value is positive when more weight stays |
|
553 |
+inside communities than the strength-preserving null predicts. The |
|
554 |
+self-share reported alongside, $\operatorname{tr}(W)/m = \sum_i e_{ii}$ at |
|
555 |
+the category partition, is the first term of \eqref{eq:appC-Qcomm} alone and |
|
556 |
+differs from $Q_{\mathrm{cat}}$ by the correction $\sum_i a^{\mathrm{out}}_i |
|
557 |
+a^{\mathrm{in}}_i$, which grows with the concentration of flows on a few |
|
558 |
+large categories. For the area partition, \eqref{eq:appC-Qcomm} is summed |
|
559 |
+over the eight areas with $e_{\alpha\alpha}$ the within-area share, which is |
|
560 |
+one minus the cross-area share $X_{\alpha,t}$ of Part~IV weighted by the |
|
561 |
+area's out-strength; $Q_{\mathrm{area}}$ is therefore a strength-weighted, |
|
562 |
+null-corrected average of the within-area shares, and a fall in |
|
563 |
+$Q_{\mathrm{area}}$ can arise from a rise in boundary crossing or from a |
|
564 |
+shift of out-strength toward areas whose $a^{\mathrm{out}}_\alpha |
|
565 |
+a^{\mathrm{in}}_\alpha$ is large. Because the partition is fixed rather than |
|
566 |
+optimised, none of the resolution-limit results for maximised modularity |
|
567 |
+\citep{fortunato2007resolution} applies; $Q$ is here a descriptive index and |
|
568 |
+its comparison across years is meaningful only because the partition and |
|
569 |
+the category definitions are the same in every year. |
|
570 |
+ |
|
571 |
+\subsection{Krippendorff's \texorpdfstring{$\alpha$}{alpha} with the MASI distance} |
|
572 |
+\label{sec:appC-alpha} |
|
573 |
+ |
|
574 |
+\paragraph{General form.} |
|
575 |
+Krippendorff's $\alpha$ \citep{hayes2007krippendorff, artstein2008agreement} |
|
576 |
+is defined for any difference function $d(v, v') \ge 0$ on the values that |
|
577 |
+coders may assign, with $d(v, v) = 0$, as |
|
578 |
+\begin{equation} |
|
579 |
+ \alpha \;=\; 1 - \frac{D_o}{D_e}, |
|
580 |
+ \label{eq:appC-alpha} |
|
581 |
+\end{equation} |
|
582 |
+where $D_o$ is the mean difference between the values assigned to the same |
|
583 |
+unit by different coders and $D_e$ the mean difference between all pairs of |
|
584 |
+values in the pooled data, irrespective of unit. With two coders $A$ and $B$ |
|
585 |
+rating each of $n$ units, the pooled set contains $N = 2n$ values and |
|
586 |
+\begin{equation} |
|
587 |
+ D_o \;=\; \frac{1}{n}\sum_{u=1}^{n} d\bigl(a_u, b_u\bigr), \qquad |
|
588 |
+ D_e \;=\; \frac{1}{N(N-1)}\sum_{c \ne c'} d\bigl(v_c, v_{c'}\bigr) |
|
589 |
+ \;=\; \frac{1}{\binom{N}{2}}\sum_{c < c'} d\bigl(v_c, v_{c'}\bigr), |
|
590 |
+ \label{eq:appC-DoDe} |
|
591 |
+\end{equation} |
|
592 |
+the second equality using the symmetry of $d$. The coefficient is one when |
|
593 |
+the coders agree on every unit ($D_o = 0$), zero when the within-unit |
|
594 |
+differences are what chance pairing of the pooled values would produce |
|
595 |
+($D_o = D_e$), and negative when disagreement is systematic. Unlike Cohen's |
|
596 |
+$\kappa$, which compares observed agreement with the product of the two |
|
597 |
+coders' marginals, $\alpha$ pools the two coders' values into one |
|
598 |
+distribution for its chance term, so it does not reward coders for having |
|
599 |
+different marginal distributions; this and its accommodation of arbitrary |
|
600 |
+difference functions are the reasons it is the recommended coefficient for |
|
601 |
+set-valued annotation \citep{artstein2008agreement}. |
|
602 |
+ |
|
603 |
+\paragraph{The MASI distance.} |
|
604 |
+For set-valued labels $x, y \subseteq \mathcal{K}$, \citet{passonneau2006masi} |
|
605 |
+defines the Measuring Agreement on Set-valued Items distance as |
|
606 |
+\begin{gather} |
|
607 |
+ d_{\mathrm{MASI}}(x, y) \;=\; 1 - J(x, y)\, M(x, y), \qquad |
|
608 |
+ J(x, y) = \frac{|x \cap y|}{|x \cup y|}, \nonumber\\ |
|
609 |
+ M(x, y) = |
|
610 |
+ \begin{cases} |
|
611 |
+ 1 & x = y,\\ |
|
612 |
+ 2/3 & x \subsetneq y \text{ or } y \subsetneq x,\\ |
|
613 |
+ 1/3 & x \cap y \ne \emptyset \text{ and neither contains the other},\\ |
|
614 |
+ 0 & x \cap y = \emptyset , |
|
615 |
+ \end{cases} |
|
616 |
+ \label{eq:appC-masi} |
|
617 |
+\end{gather} |
|
618 |
+the Jaccard similarity attenuated by a monotonicity factor that |
|
619 |
+distinguishes nesting from mere overlap. The distance is zero only for |
|
620 |
+identical non-empty sets, equals one for disjoint sets, and treats the pair |
|
621 |
+(empty, empty) as identical ($d = 0$) by the convention of the |
|
622 |
+implementation, which returns zero before computing $J$ for two empty sets; |
|
623 |
+the pair (empty, non-empty) has $J = 0$ and $d = 1$. |
|
624 |
+ |
|
625 |
+\paragraph{What the code computes.} |
|
626 |
+\texttt{taxonomy/validate\_sample.py} forms, for the two raters, the |
|
627 |
+observed term as the mean of $d_{\mathrm{MASI}}^2$ over the $n$ rated works |
|
628 |
+and the expected term as the mean of $d_{\mathrm{MASI}}^2$ over all |
|
629 |
+unordered pairs of the $2n$ pooled label sets (exhaustively when |
|
630 |
+$2n \le 1200$, otherwise over 200{,}000 random ordered pairs with $i \ne j$, |
|
631 |
+seed 44), and reports $1 - D_o/D_e$. Two remarks make this precise. The code |
|
632 |
+uses the constants $0.67$ and $0.33$ for the monotonicity factor rather than |
|
633 |
+$2/3$ and $1/3$, a rounding of the published definition that changes |
|
634 |
+distances by at most $0.0034$. More consequentially, it \emph{squares} the |
|
635 |
+MASI distance before averaging, that is, it applies |
|
636 |
+\eqref{eq:appC-alpha}--\eqref{eq:appC-DoDe} with $d = d_{\mathrm{MASI}}^2$. |
|
637 |
+Krippendorff's interval metric is the squared difference, and the code's |
|
638 |
+comment describes the computation as the ``interval form on the MASI |
|
639 |
+distance''; \citeauthor{passonneau2006masi}, in contrast, plugs |
|
640 |
+$d_{\mathrm{MASI}}$ itself into the $\alpha$ formula, and the agreement |
|
641 |
+packages that implement MASI follow her. The two conventions yield different |
|
642 |
+numbers from the same ratings---squaring compresses small disagreements and |
|
643 |
+so, as a rule, raises $\alpha$ when most disagreements are partial---and the |
|
644 |
+value reported in Part~IX is the squared-distance version, labelled as such. |
|
645 |
+Both are legitimate instances of \eqref{eq:appC-alpha}; what is not |
|
646 |
+legitimate is to compare the reported value with published MASI-$\alpha$ |
|
647 |
+figures without this caveat. The mean Jaccard index reported next to it is |
|
648 |
+$n^{-1}\sum_u J(a_u, b_u)$ with $0/0 = 0$ and is a raw agreement rate with |
|
649 |
+no chance correction. Both statistics are computed on the full rated sample |
|
650 |
+and not on the 200-work subsample reserved for the principal investigator, |
|
651 |
+whose verdicts will add a third coder to \eqref{eq:appC-DoDe} once |
|
652 |
+checkpoint H-2 is signed. |
|
653 |
+ |
|
654 |
+\subsection{The multi-label score rule and the tie-break as a total order} |
|
655 |
+\label{sec:appC-score} |
|
656 |
+ |
|
657 |
+\paragraph{Score.} |
|
658 |
+For a work $w$ and a category $k$ the implementation |
|
659 |
+(\texttt{taxonomy/classify.py}, \texttt{score\_row}) computes five |
|
660 |
+components, one per signal type $\sigma \in \{\mathrm{T}, \mathrm{F}, |
|
661 |
+\mathrm{X}, \mathrm{D}, \mathrm{kw}\}$, |
|
662 |
+\begin{align} |
|
663 |
+ s^{\sigma}_k(w) &\;=\; \min\Bigl\{ 1,\ \sum_{x \in \sigma(w)} \omega^{\sigma}_k(x) \Bigr\} |
|
664 |
+ \qquad (\sigma \ne \mathrm{kw}), \\ |
|
665 |
+ s^{\mathrm{kw}}_k(w) &\;=\; \min\Bigl\{ 1,\ \sum_{(r, \omega, \phi) \in R_k} \omega\, \mathbf{1}[r \text{ matches } \phi(w)] \Bigr\}, |
|
666 |
+\end{align} |
|
667 |
+where $\sigma(w)$ is the \emph{set} of the work's topic, subfield, arXiv |
|
668 |
+category or DBLP stream identifiers (Python \texttt{set()}, so a repeated |
|
669 |
+identifier contributes once), and the keyword loop stops at the first rule |
|
670 |
+that brings the running sum to one, which is equivalent to the capped sum. |
|
671 |
+The score is $s_k(w) = \sum_\sigma s^{\sigma}_k(w) \in [0, 5]$; a category |
|
672 |
+with $s_k(w) \ge \theta = 1$ is then vetoed if any exclusion pattern of |
|
673 |
+$Z_k$ matches the concatenation of title and abstract, in which case its |
|
674 |
+score is set to zero and it is not assigned. The set of assigned categories |
|
675 |
+is $C(w) = \{k : s_k(w) \ge \theta,\ \text{not vetoed}\}$, and the stored |
|
676 |
+score of an assigned category is $s_k(w)$ rounded to three decimals. Two |
|
677 |
+elementary properties justify the reading given in Part~IV\@. The cap makes |
|
678 |
+each component a bounded, monotone, subadditive function of the matched |
|
679 |
+weights: adding a signal never lowers a component, and no number of signals |
|
680 |
+of one type can contribute more than one. And because the threshold equals |
|
681 |
+the cap, a single weight-one signal of any type suffices for assignment |
|
682 |
+while signals of weight below one require corroboration from a different |
|
683 |
+type; formally, $s_k(w) \ge 1$ holds if and only if either some component |
|
684 |
+reaches one or the sum of the (uncapped) components across at least two |
|
685 |
+types reaches one. |
|
686 |
+ |
|
687 |
+\paragraph{Primary category as the minimum of a total order.} |
|
688 |
+Let $\mathcal{P}(w) = \bigcup_{k \in C(w)} \mathrm{parents}(k)$ be the set |
|
689 |
+of categories declared parents of at least one assigned category (the |
|
690 |
+\texttt{child\_of} relation of the ontology). The implementation sorts the |
|
691 |
+assigned categories by the key |
|
692 |
+\begin{equation} |
|
693 |
+ \phi(k) \;=\; \bigl( -s_k(w),\ \mathbf{1}[k \in \mathcal{P}(w)],\ \mathrm{number}(k) \bigr) \in \R \times \{0,1\} \times \mathbb{N} |
|
694 |
+ \label{eq:appC-key} |
|
695 |
+\end{equation} |
|
696 |
+in lexicographic order and takes the first element; this is |
|
697 |
+$\kappa^{\ast}(w) = \arg\min_{k \in C(w)} \phi(k)$ and coincides with the |
|
698 |
+lexicographic arg-max of Part~IV (the sign of the first component reversed). |
|
699 |
+The lexicographic order on $\R \times \{0,1\} \times \mathbb{N}$ is a total |
|
700 |
+order, and $\phi$ is injective on $\mathcal{K}$ because the ontology numbers |
|
701 |
+are distinct, so the restriction of $\phi$ to $C(w)$ has a unique minimum |
|
702 |
+whenever $C(w) \ne \emptyset$: the primary category is well defined and does |
|
703 |
+not depend on the order in which categories are scanned. The second |
|
704 |
+component implements ``child before parent'': if $k$ and $k'$ are both |
|
705 |
+assigned with equal scores and $k'$ is a declared parent of $k$, then |
|
706 |
+$k' \in \mathcal{P}(w)$ while $k$ may not be, and $k$ precedes $k'$. The rule |
|
707 |
+is not transitive in the sense of a hierarchy, however. Membership in |
|
708 |
+$\mathcal{P}(w)$ depends only on whether $k$ is the parent of \emph{some} |
|
709 |
+assigned category, not of the specific competitor; a category that is the |
|
710 |
+parent of a low-scoring assigned child is demoted in the tie against an |
|
711 |
+unrelated category of equal score. This is a property of the key as |
|
712 |
+implemented, stated here so that the primary counts of Part~IX are read with |
|
713 |
+it; it affects only exact ties in a score rounded to three decimals, which |
|
714 |
+are frequent precisely among the categories whose signals are weight-one |
|
715 |
+keywords. The third component, the ontology number, is arbitrary and makes |
|
716 |
+the order total; no claim is attached to it. |
|
717 |
+ |
|
718 |
+\subsection{The emerging-topic label and the absence of look-ahead} |
|
719 |
+\label{sec:appC-lookahead} |
|
720 |
+ |
|
721 |
+\paragraph{Filtrations.} |
|
722 |
+Index months by integers $t = 0, 1, \dots, M$ (January 2018 to the last |
|
723 |
+complete month) and let the panel of \texttt{experiments/emerging\_detection.py} |
|
724 |
+be the pair of arrays $n_{k,t}$ (eligible month-precision works of topic $k$ |
|
725 |
+with first month $t$) and $c_{k,t}$ (internal citations received by works of |
|
726 |
+topic $k$ from works with first month $t$). Define the filtration |
|
727 |
+\begin{equation} |
|
728 |
+ \mathcal{F}_T \;=\; \sigma\bigl( n_{k,t},\ c_{k,t} :\ k \in \mathcal{K}_{\mathrm{top}},\ t \le T \bigr), |
|
729 |
+ \label{eq:appC-filtration} |
|
730 |
+\end{equation} |
|
731 |
+the information contained in the panel columns up to and including month |
|
732 |
+$T$. A statistic is \emph{available at} $T$ if it is |
|
733 |
+$\mathcal{F}_T$-measurable. Each of the six features of Part~IV is a |
|
734 |
+function of sums of panel entries over windows $(a, b]$ with $b \le T$: |
|
735 |
+$g^{12}_{k,T}$ uses $(T-24, T]$, $\mathrm{acc}_{k,T}$ uses $(T-36, T]$, |
|
736 |
+$z_{k,T}$ uses $[T-27, T]$, $\mathrm{cit}_{k,T}$ uses $(T-24, T]$ of $c$, |
|
737 |
+$\mathrm{share}_{k,T}$ uses $(T-12, T]$ of every topic, and |
|
738 |
+$\mathrm{age}_{k,T}$ uses the cumulative sum up to $T$. All are |
|
739 |
+$\mathcal{F}_T$-measurable by construction. The implementation enforces this |
|
740 |
+by index arithmetic (\texttt{win(a, lo, hi)} slices columns \texttt{lo+1} |
|
741 |
+to \texttt{hi} inclusive with \texttt{hi = T\_idx}), not by convention. |
|
742 |
+ |
|
743 |
+\paragraph{Label.} |
|
744 |
+The label $\ell_{k,T,h}$ of equation~\eqref{eq:label} is a function of |
|
745 |
+$V_k(T-h, T]$, $V_k(T, T+h]$, $V_k(T-12, T]$ and of the within-cutoff decile |
|
746 |
+of $g^{\mathrm{fut}}$ over eligible topics; it is therefore |
|
747 |
+$\mathcal{F}_{T+h}$-measurable and, because it depends on columns strictly |
|
748 |
+after $T$, not $\mathcal{F}_T$-measurable. The feature--label pair at cutoff |
|
749 |
+$T$ thus has the structure of a prediction problem: predict an |
|
750 |
+$\mathcal{F}_{T+h}$-measurable outcome from an $\mathcal{F}_T$-measurable |
|
751 |
+input. The eligibility condition $V_k(T-12, T] \ge 20$ is |
|
752 |
+$\mathcal{F}_T$-measurable, so conditioning on it does not leak the outcome. |
|
753 |
+ |
|
754 |
+\paragraph{Separation of training and test.} |
|
755 |
+Let $\mathcal{T}_{\mathrm{tr}}$ and $\mathcal{T}_{\mathrm{te}}$ be the |
|
756 |
+training and test cutoffs. Every quantity used to fit a model---features, |
|
757 |
+labels and the standardisation constants of the scaler, which is fitted on |
|
758 |
+training rows only---is $\mathcal{F}_{T^{\mathrm{tr}}_{\max} + h}$-measurable |
|
759 |
+with $T^{\mathrm{tr}}_{\max} = \max \mathcal{T}_{\mathrm{tr}}$. The |
|
760 |
+implementation sets $T^{\mathrm{tr}}_{\max}$ to December 2021 for $h = 12$ |
|
761 |
+and to December 2020 for $h = 24$, so that $T^{\mathrm{tr}}_{\max} + h$ is |
|
762 |
+December 2022 in both cases, and the first test cutoff is January 2023. |
|
763 |
+Hence |
|
764 |
+\begin{equation} |
|
765 |
+ T^{\mathrm{tr}}_{\max} + h \;<\; \min \mathcal{T}_{\mathrm{te}}, |
|
766 |
+ \label{eq:appC-gap} |
|
767 |
+\end{equation} |
|
768 |
+and the fitted predictor is measurable with respect to a $\sigma$-field |
|
769 |
+that precedes every test feature, let alone every test label, which closes |
|
770 |
+at $\max \mathcal{T}_{\mathrm{te}} + h \le M$. Condition \eqref{eq:appC-gap} |
|
771 |
+is the formal content of ``out of time'' in the sense of |
|
772 |
+\citet{bergmeir2012crossval}; it is stricter than the plan's wording, which |
|
773 |
+allowed training cutoffs through 2022, and stricter for $h = 24$ than the |
|
774 |
+description of Part~IV, which gives December 2021 as the last training |
|
775 |
+cutoff for both horizons---the code's rule is the one that guarantees |
|
776 |
+\eqref{eq:appC-gap}, and the manuscript follows the code. The test cutoffs |
|
777 |
+are further restricted to those with $T + h \le M$, so that no test label is |
|
778 |
+right-censored. |
|
779 |
+ |
|
780 |
+\paragraph{What measurability does not cover.} |
|
781 |
+The filtration \eqref{eq:appC-filtration} is generated by the panel, and the |
|
782 |
+panel is built once, in 2026, from a corpus whose topic assignments, |
|
783 |
+eligibility flags, linkage and month-precision filter were all fixed with |
|
784 |
+knowledge of the whole period. Formally, the random objects $n_{k,t}$ |
|
785 |
+themselves are not $\mathcal{F}_t$-measurable functionals of the literature |
|
786 |
+as it stood at $t$: they are functionals of the literature as OpenAlex |
|
787 |
+indexed and classified it by October 2026. The statement proved above is |
|
788 |
+therefore conditional: \emph{given} the 2026 panel, no feature at $T$ uses a |
|
789 |
+column after $T$ and no training quantity uses a column at or after the |
|
790 |
+first test cutoff. It is not the statement that an observer in 2021 could |
|
791 |
+have computed the training set, which would require a topic vocabulary and a |
|
792 |
+corpus frozen in 2021. Part~IV draws the distinction and names the |
|
793 |
+cluster-based extension that would narrow it; the derivation here makes |
|
794 |
+precise which half of the claim is proved and which is assumed. |