% part09_results.tex — Part IX, Empirical Results. Written by Agent L (Part IX instance, with Agent I/J roles), 2026-11-03.
% Corpus v3 result files (results/, data/metadata/, reports/classification_report.json); every number is a macro of
% tables/numbers.tex or a cell of a table generated by tables/make_tables.py (tables_part09). Items marked \pending{} await
% a result file (taxonomy validation metrics, semantic atlas) or the PI checkpoint H-4 (interpretation of structural change).

\section{Scope, sources and reading rules}
\label{sec:r-scope}

This part collects the quantitative findings of the study and reads them against the ten research questions and
the eight pre-specified hypotheses of the protocol. It is deliberately a reporting part: the methods are those of
Part~IV, the interpretation of the literature is left to Parts~V--VII, and the discussion of what the findings
mean for computer science is left to Part~XI. Three kinds of material appear below. The first is the set of
descriptive series on volume, composition, openness and influence, which answer RQ1, RQ3 and RQ9 and provide
the evidence for H2 and H7. The second is the structural-change analysis of the monthly series, in total and by
category, which answers RQ2 and tests H1 and H3. The third is the set of network and detection results, which
answer RQ4--RQ6 and RQ10 and test H4--H6 and H8. The actor networks of RQ6 (authors, countries, institutions,
venues) are analysed at length in Part~VIII; this part does not repeat that analysis but summarises its verdicts
where a hypothesis requires them and cross-references its tables by label.

The evidence base is the eligible corpus of Part~III in its third build, with \PrismaCsCoreEligible{} works in the
computer-science core, of which \PrismaEligibleByPeriodBaseline{} have a first date in the baseline window and
\PrismaEligibleByPeriodPost{} in the post window (Table~\ref{tab:p9-periods}). Every monthly statistic below is
computed on the month-precision subset, \PrismaEligibleDatePrecisionMonth{} works, because the
\PrismaEligibleDatePrecisionYear{} works whose only date is a year would otherwise be stacked on January
(Section~\ref{sec:data-dates}; Figure~\ref{fig:volume-monthly-raw}); yearly statistics use all eligible works.
Table~\ref{tab:p9-periods} shows that the two periods are not alike in completeness: the post period has a
higher share of month-precision dates, a higher arXiv-linked share and a much lower share of works with at
least one resolved reference, the last being an artefact of reference indexing, which lags publication by more
than a year for a substantial fraction of records (Table~\ref{tab:indexing_lag}, Section~\ref{sec:net-boundaries}).
Any comparison of the two periods inherits these differences, and the text names them wherever they bear on a
reading.

\input{tables/tab_p9_periods}

The category-level results depend on the classification of Part~IV. Table~\ref{tab:p9-classification} reports its
coverage: the share of eligible works that received a primary category rises from under four fifths in 2018 to
over four fifths in the last years, and the number of labels per classified work rises monotonically through
the window. Both movements matter for what follows. The rise in coverage means that category counts grow
slightly faster than eligible counts for reasons unrelated to research activity, so growth rates of categories
should be read against the coverage column rather than against the eligible totals of
Table~\ref{tab:eligible_year}; the rise in labels per work means that multi-label counts, which are the input of
the monthly category series, grow faster than primary counts, and that co-classification statistics
(Section~\ref{sec:r-rq5}) are more generous in the post period. The classification run itself processed
\ClassificationWorks{} records, a superset of the eligible corpus that includes the penumbra set and records
later removed by the eligibility filter, of which \ClassificationClassified{} received at least one category
and \ClassificationAssignments{} category assignments were made in total. The validation of the assignment
against two independent raters on a stratified sample is part of the protocol (checkpoint H-2) and its
metrics file was not available when this part was written; Section~\ref{sec:r-validation} is reserved for it.

\input{tables/tab_p9_classification}

Four reading rules apply throughout. First, the series end at the last complete month of the panel,
\ItsLastComplete{}, and the yearly tables carry 2026 as an incomplete row; right-truncation affects every
statistic that depends on citations or references more than it affects counts of works, and the fixed-window
citation statistics are complete only for the publication years flagged in Table~\ref{tab:citation_windows}.
Second, the reference date of 30~November 2022 is a historical marker. The interrupted-time-series estimates
are changes in level and slope at that date net of trend and season; they are not treatment effects, and
the alternative-date and placebo analyses exist precisely to show how specific to that month the estimated
changes are \citep{bernal2017its, hausman2018rdit}. Third, the forty category series are analysed with the
same battery of tests, so forty $p$-values are produced for every statistic; at the conventional five per
cent level two of them are expected to fall below the threshold under a global null, and the text reads
counts of significant categories with that expectation in mind rather than applying a formal correction
\citep{benjamini1995fdr}, because the hypotheses were stated for named categories and are tested one by one.
Fourth, the verdict on each hypothesis is one of \emph{supported}, \emph{not supported}, \emph{opposite}
(the data move against the stated direction) or \emph{inconclusive} (the test cannot discriminate), and the
hypotheses are restated in full before each verdict. Interpretive statements about the structural-change
results that go beyond the estimates are marked as pending the principal investigator's review (checkpoint
H-4), as the protocol requires.

\section{RQ1: the volume and composition of computer-science research}
\label{sec:r-rq1}

\subsection{Volume}

The yearly totals of Table~\ref{tab:eligible_year} and Figure~\ref{fig:volume-yearly} describe a corpus that
was almost flat through the baseline and grew afterwards. The eligible count moves from
\VolumeYearlyTwoZeroOneEightN{} works in 2018 to \VolumeYearlyTwoZeroTwoTwoN{} in 2022, a span in which the
2020 and 2021 cohorts are the largest and 2022 is slightly below them, and then to
\VolumeYearlyTwoZeroTwoThreeN{} in 2023, \VolumeYearlyTwoZeroTwoFourN{} in 2024 and
\VolumeYearlyTwoZeroTwoFiveN{} in 2025; the 2026 cohort, \VolumeYearlyTwoZeroTwoSixN{} works at the snapshot,
covers ten months of first dates and an indexing lag, and is not comparable. Three features of the
composition columns of Table~\ref{tab:volume_yearly} qualify the growth. The DBLP-indexed count
(\VolumeYearlyTwoZeroOneEightNDblp{} in 2018, \VolumeYearlyTwoZeroTwoFiveNDblp{} in 2025) grows much less than
the total, so that the DBLP share of the corpus falls in the post period; the OpenAlex-primary count
(\VolumeYearlyTwoZeroOneEightNCsPrimary{} to \VolumeYearlyTwoZeroTwoFiveNCsPrimary{}) and especially the
arXiv-linked count (\VolumeYearlyTwoZeroOneEightNArxiv{} to \VolumeYearlyTwoZeroTwoFiveNArxiv{}) grow faster
than the total. The post-period growth of the corpus is therefore concentrated in the parts of it that DBLP's
venue-based curation does not cover, which is consistent both with a real shift of output toward preprints and
toward venues outside DBLP's lists, and with a change in OpenAlex's own coverage over the period
(Section~\ref{sec:net-boundaries} examines the DBLP-only records; Table~\ref{tab:dblp_only}). The two cannot be
separated with the data at hand, and every growth statement of this part should be read as a statement about
the indexed record of computer science rather than about its production.

The mean number of authors per work rises from \VolumeYearlyTwoZeroOneEightMeanAuthors{} in 2018 to
\VolumeYearlyTwoZeroTwoFiveMeanAuthors{} in 2025, continuing a trend that Part~VIII documents in detail
(Section~\ref{sec:net-teams}); the mean number of declared references per work rises from
\VolumeYearlyTwoZeroOneEightMeanRefs{} to a maximum of \VolumeYearlyTwoZeroTwoOneMeanRefs{} in 2021 and then
falls to \VolumeYearlyTwoZeroTwoFiveMeanRefs{} in 2025 and \VolumeYearlyTwoZeroTwoSixMeanRefs{} in 2026, a
decline that is entirely attributable to reference indexing lag and is the clearest single illustration of the
right-truncation problem in the corpus.

The monthly series (Figure~\ref{fig:its-total}) carries the same information at the resolution at which the
structural-change tests operate. Its pre-period slope, \ItsSeriesTotalItsChatgptPreSlope{} in log points per
month, corresponds to a growth of about a third of a per cent per month, which compounds to the yearly
totals above; the acceleration visible to the eye after 2022 is quantified in Section~\ref{sec:r-rq2}.

\subsection{Composition by area and category}

Figure~\ref{fig:area-shares} and Table~\ref{tab:p9-area-shares} give the composition of the classified corpus by
the eight top-level areas of the ontology, using the primary category of each work. The dominant area in every
year is Perception, Language, Robotics and Interaction, which holds a little over a third of primary
assignments and whose share is stable within two percentage points over the window; it contains Computer
Vision, the largest single category (Table~\ref{tab:p9-category-periods}). The movement is elsewhere. The share
of the AI and Machine Learning area rises in every year of the window, from under a tenth in 2018 to over a
tenth in 2022 and to close to a fifth in 2025; its rise is the mirror image of the fall of Data, Information and
Distributed Systems, whose share declines in every year, and of the smaller declines of Theory of Computation
and of Software, Programming Languages and Formal Methods. Security, Privacy and Trust gains about one and a
half points over the window; Hardware and Computational Science are approximately stable. The period rows of
the table, which split at the reference date rather than at the calendar year, show the same ordering.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_area_shares.pdf}
\caption{Share of eligible works by top-level area (primary category) per year. Source:
\texttt{results/graph/category\_counts\_yearly.csv}; 2026 omitted as incomplete.}
\label{fig:area-shares}
\end{figure}

\input{tables/tab_p9_area_shares}

Two cautions apply to this picture. The classifier assigns the primary category by the largest score, and a work
on, say, a vision--language model is counted once, under whichever of Computer Vision, Natural Language
Processing, Multimodal AI or Large Language Models scores highest; the area shares of
Table~\ref{tab:p9-area-shares} are therefore sensitive to the tie-breaking of the score, and the multi-label
counts of Table~\ref{tab:p9-category-periods} give a less brittle view of the same movement. Second, the
coverage of the classifier rises over the window (Table~\ref{tab:p9-classification}), and the works that enter
the classified set late are not a random sample of the eligible corpus: they are predominantly works with
abstracts, which the AI categories, defined largely by keyword patterns, pick up more readily than the
venue-defined categories of theory or systems. A part of the rise of the AI area is therefore a rise in the
classifiability of the literature, and the text does not attempt to separate it.

Table~\ref{tab:p9-category-periods} descends to the forty categories. Sorted by their post-period share, the
table shows a top tier that is unchanged across periods (Computer Vision, Computer Networks, Human--Computer
Interaction, Cybersecurity, Natural Language Processing), a single category whose rank changes by an order
of magnitude (Large Language Models, which moves from under half a per cent of primary assignments in the
baseline to over four per cent in the post period, with a multi-label count more than three times its
baseline value), and a cluster of AI categories with the same profile at smaller scale: Generative AI,
AI Safety and Alignment, Multimodal AI, Foundation Models and Federated Learning all have post-period
multi-label counts above three times their baseline counts. The categories whose multi-label counts are
below their baseline value in the post period are Computer Networks, Computer Architecture, Formal
Verification, Programming Languages and Operating Systems, and the compound annual growth rates of the last
two columns confirm that the fall is a fall of shares more than of volumes: among these, only Computer
Architecture and Cloud Computing have a negative post-period CAGR of primary counts, and the others grow in
absolute terms while losing share. The post-period CAGRs are computed over the two complete years 2023 to
2025 (equation~\eqref{eq:cagr}) and are more volatile than the four-year baseline rates; they also carry the
coverage drift noted above.

\input{tables/tab_p9_category_periods}

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_category_growth.pdf}
\caption{Yearly primary-category output indexed to 2018 for the eight fastest- and eight slowest-growing
categories (2018--2025). Source: \texttt{results/graph/category\_counts\_yearly.csv}.}
\label{fig:category-growth}
\end{figure}

Figure~\ref{fig:category-growth} shows the growth indices of the extremes. The fastest-growing categories are,
without exception, AI categories that were small in 2018 (Foundation Models, Large Language Models, Generative
AI, Multimodal AI, Federated Learning, AI Safety and Alignment, Autonomous Scientific Discovery) together with
Quantum Computing; their indices are large partly because their 2018 bases are small, and the table's shares
are the better measure of their weight. The slowest-growing categories are the venue-defined systems and
theory categories (Operating Systems, Compilers, Computer Architecture, Programming Languages, Formal
Verification, Cloud Computing), whose indices stay near or below their 2018 value until 2023 and rise modestly
afterwards; these are also the categories whose DBLP coverage is highest, so their flatness is the flatness of
the DBLP-indexed count noted above.

\subsection{Hypothesis H2: composition}
\label{sec:r-h2}

H2 states that the share of the AI-related categories (categories 1--8 and 38 of the mandate, which form the AI
and Machine Learning area of the ontology) in computer-science output increases in the post period, and that
the share of at least one non-AI area (theory, systems, networks) declines in relative terms while growing in
absolute terms. Table~\ref{tab:p9-ai-shares} tests the first clause with the bootstrap intervals of
Part~IV, obtained by resampling the works of each year with replacement \citep{efron1979bootstrap}. The area's
share rises from 2018 to 2022 and from 2022 to 2025, but the rise is not uniform across its nine categories.
Seven of them (Large Language Models, Foundation Models, Multimodal AI, Generative AI, AI Agents and
Autonomous Systems, Reinforcement Learning, AI Safety and Alignment) have a 2025 interval that lies entirely
above their 2022 interval; Machine Learning Theory has overlapping intervals in the two years; and Deep
Learning Architectures has a 2025 interval entirely \emph{below} its 2022 interval, its share having peaked
in the baseline. AI Agents and Autonomous Systems is the one category whose 2025 share remains below its 2018
share, a consequence of the category's definition (it includes the pre-LLM agent and multi-agent literature,
whose share was declining through the baseline) rather than evidence against the hypothesis. The Large
Language Models category accounts for the greater part of the area's post-period gain, and the hypothesis is
better described as supported for the generative and language-model categories than for machine learning as
a whole. The intervals are narrow because the
resampling treats the classifier as fixed; the uncertainty that matters is the classifier's, not the
sampling's, and the intervals should be read as a lower bound on it.

\input{tables/tab_p9_ai_shares}

The second clause is tested by Table~\ref{tab:p9-area-shares} together with the CAGR columns of
Table~\ref{tab:p9-category-periods}. The Data, Information and Distributed Systems area (which contains
Computer Networks) and the Theory of Computation area both lose share in every year of the window, and their
primary counts in 2025 exceed their 2022 counts, so both decline relatively while growing absolutely; the
Software, Programming Languages and Formal Methods area shows the same pattern. The second clause is therefore
met by three areas rather than by one. The verdict on H2 is \emph{supported} on both clauses, with the
qualification that the relative decline of the non-AI areas begins before the reference date (it is visible in
every year from 2018) and continues through it without a visible change of pace in the yearly data; the
monthly analysis of Section~\ref{sec:r-rq2} examines whether a change of pace exists at monthly resolution.
Table~\ref{tab:diversity} adds the diversity indices of the category mix: the Rao--Stirling index and the
Leinster--Cobbold index of order two are both lower in 2023--2025 than in 2018--2021, so the concentration of
primary assignments on fewer, mutually similar categories increased slightly, which is what the rise of a
cluster of related AI categories implies. The similarity matrix behind these indices is the co-classification
Jaccard similarity rather than the semantic one, because the semantic category similarities were not yet
available (\pending{semantic atlas: results/semantic/category\_similarity\_baseline.csv and \_post.csv; the
diversity indices should be re-estimated with the semantic distance when they exist}).

\section{RQ2: acceleration and structural change}
\label{sec:r-rq2}

\subsection{The total series at the reference date}
\label{sec:r-its-total}

Figure~\ref{fig:its-total} shows the monthly series of month-precision works from \ItsFirst{} to
\ItsLastComplete{} with the fitted values of the interrupted time series of equation~\eqref{eq:its}, and
Table~\ref{tab:p9-its-total-events} reports the estimates at the reference date and at the four alternative
dates. At December 2022 the estimated change in log level is \ItsSeriesTotalItsChatgptLevelChange{}, a value
that is small, negative and indistinguishable from zero (HAC $p$ = \ItsSeriesTotalItsChatgptLevelP{}); the
estimated change in monthly log slope is \ItsSeriesTotalItsChatgptSlopeChange{}, which, added to the
pre-period slope of \ItsSeriesTotalItsChatgptPreSlope{}, implies a post-period growth rate of roughly one and
four tenths of a per cent per month against roughly three tenths before, a more than fourfold increase in the
rate of growth. The HAC $p$-value of the slope change rounds to \ItsSeriesTotalItsChatgptSlopeP{}, and the fit
explains \ItsSeriesTotalItsChatgptRTwo{} of the variance of the log series over
\ItsSeriesTotalNMonths{} months. The estimates describe a kink, not a jump: the series did not move to a higher
level in December 2022, it began, at some point, to grow faster.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_its_total.pdf}
\caption{Monthly output (month-precision works) with the interrupted-time-series fit at December 2022; the
annotation gives the level and slope changes with their HAC and placebo $p$-values. Sources:
\texttt{results/econometrics/fit\_total.csv}, \texttt{its\_results.json}.}
\label{fig:its-total}
\end{figure}

\input{tables/tab_p9_its_total_events}

Whether ``some point'' is the reference date is the question the remaining rows of
Table~\ref{tab:p9-its-total-events} and the diagnostics of Table~\ref{tab:p9-total-robustness} address. Moving
the event to May 2022 (DALL$\cdot$E~2) or April 2023 (GPT-4) changes the slope estimate by less than a tenth of
its value; moving it to June 2020 (GPT-3) halves it but leaves it clearly positive, and moving it to October
2024 (o1) gives a comparable slope change on top of a steeper pre-period. The level estimates vary in sign
across the five dates and two of them (DALL$\cdot$E~2, o1) have HAC $p$-values below one per cent, with
opposite signs. The honest summary of the five columns is that the series exhibits a sustained acceleration
over 2022--2024 that any event date placed within that span will pick up as a slope change, and that no
single month within it shows a level shift that the others do not.

\input{tables/tab_p9_total_robustness}

The placebo distribution (Figure~\ref{fig:placebo-total}) makes this precise. Over the
\ItsSeriesTotalPlaceboChatgptNPlacebo{} candidate event months, the level $t$-statistic of December 2022 ranks
\ItsSeriesTotalPlaceboChatgptLevelTRank{}th in absolute value (placebo $p$ =
\ItsSeriesTotalPlaceboChatgptLevelTPlaceboP{}), that is, almost every other month of the series produces a
larger apparent level shift than the reference date does; the slope $t$-statistic ranks first (placebo $p$ =
\ItsSeriesTotalPlaceboChatgptSlopeTPlaceboP{}, the smallest attainable value being one over the number of
candidates). Restricted to the baseline months, the placebo $p$ of the slope is
\ItsSeriesTotalPlaceboChatgptBaselineOnlySlopeTPlaceboP{}: no month before the reference date yields a slope
change as large as the one estimated there, which is the pre-trend diagnostic the protocol asked for and is
the same statement as the first rank, read from the other side. The donut specification, which drops the
event month and three months on each side, moves the level estimate to
\ItsSeriesTotalItsChatgptDonutLevelChange{} (HAC $p$ = \ItsSeriesTotalItsChatgptDonutLevelP{}) and leaves the
slope change essentially unchanged; the donut level estimate is larger than the full-sample one because the
months it removes, the winter of 2022--2023, lie below the post-period trend line, so that the trend fitted
without them extrapolates to a higher December 2022 value. The sensitivity rows show that ending the series in
March or August 2026 instead of June changes the slope estimate in the third decimal and leaves the level
estimate within a hundredth of zero, of unstable sign and insignificant.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_placebo_total.pdf}
\caption{Placebo distribution of the ITS $t$-statistics (level and slope) over every candidate event month of
the total series; the true event month is highlighted. Source: \texttt{results/econometrics/placebo\_total.csv}.}
\label{fig:placebo-total}
\end{figure}

The data-driven methods date the kink earlier than the reference date. The Bai--Perron procedure selects a
single break under both criteria ($m$ = \ItsSeriesTotalBaiPerronMBic{} by BIC, and the same number by LWZ; the
BIC falls from \ItsSeriesTotalBaiPerronBicByMZero{} with no break to \ItsSeriesTotalBaiPerronBicByMOne{} with
one and rises again for two and three), and places it in April 2022, eight months before the reference date;
with the series ended in August 2026 the date moves to February 2022, so the location is not sharp. The
trimming parameter $h$ = \ItsSeriesTotalBaiPerronH{} months forbids a second break after April 2025 and within
fifteen months of the first, so a second kink in late 2024 or 2025 would not be found by this procedure. PELT,
which fits a piecewise-constant mean rather than a trend and therefore flags the points where the level
moves most, places changepoints in May 2023 and December 2024; the first is the point at which the series,
growing at the post-period rate, has risen far enough above its baseline mean for a step to be the cheapest
description, and it should not be read as an independent dating of the acceleration. The Bayesian online
detector assigns a maximum posterior probability of
\ItsSeriesTotalBocpdCpProbTwoZeroTwoTwoOneTwoToTwoZeroTwoThreeZeroThree{} to a changepoint within the four
months preceding any month of December 2022 to March 2023, and its maximum-a-posteriori run length never
resets over the whole series: an observer following the monthly totals with this detector would not have
concluded, at any point, that the process had changed. The reconciliation of the four methods is that the
acceleration is gradual at monthly resolution, which a trend model with one kink summarises well and a
step model or an online detector with a constant-mean likelihood summarises poorly.

\subsection{Hypothesis H1: volume}
\label{sec:r-h1}

H1 states that the monthly growth rate of computer-science works is higher in the post period than in the
baseline, but that the level shift at November 2022 is not distinguishable from placebo breaks in the baseline
once seasonality and indexing lag are modelled; the protocol adds that growth differences are expected to be
concentrated in a few categories rather than uniform. On the total series the first clause is
\emph{supported}: the slope change is positive, large relative to the pre-period slope, robust to the donut
specification and to the truncation month, and ranks first among all candidate dates and above every
baseline date in the placebo distribution. The second clause is also \emph{supported}: the level change at
the reference date is small, of uncertain sign across specifications, and ranks near the bottom of the
placebo distribution. The clause about concentration is examined in Section~\ref{sec:r-its-categories} and is
\emph{not supported} in its stated form, because the slope change is positive and conventionally significant
in nearly every category, although its magnitude varies by an order of magnitude across them.

Two qualifications are part of the verdict. The first is that the acceleration predates the reference date in
the unknown-break estimates (spring 2022), so H1's first clause is supported in the weaker sense that the
post period grows faster, not in the sense that the growth rate changed at the boundary; a reader who takes
the Bai--Perron date at face value would describe a change that began during the baseline and was already
under way when ChatGPT was released. \pending{H-4: whether to describe the aggregate acceleration as beginning
in spring 2022 (Bai--Perron) or as a 2022--2024 phenomenon without a sharp onset (placebo and alternative
dates)}. The second is that the indexing-lag model the protocol envisaged was replaced by truncation
(Section~\ref{sec:m-truncation}); the hypothesis is therefore tested conditional on the series ending at the
last complete month, and the sensitivity rows of Table~\ref{tab:p9-total-robustness} are the only evidence on
what a different truncation would do. With about a hundred observations and a break near the middle of the
sample, the design has ample power for a slope change of this size and little for a level shift of a few per
cent \citep{zhang2011itspower}, so the null result on the level should be read as ``not detectable'' rather
than ``absent''.

\subsection{The category series}
\label{sec:r-its-categories}

The same battery is applied to the forty multi-label category series. Figure~\ref{fig:breaks-by-category}
orders the categories by their estimated slope change at December 2022; Tables~\ref{tab:p9-its-slopes}
and~\ref{tab:p9-its-levels} give the slope and level estimates with their HAC and placebo $p$-values, and
Table~\ref{tab:p9-unknown-breaks} the data-driven dates. Table~\ref{tab:breaks} in Appendix~F carries the
same estimates in the layout of the result file.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_breaks_by_category.pdf}
\caption{ITS slope change at December 2022 by category, with the level change annotated; bars in colour have a
placebo $p$ below 0.05 for the slope. Source: \texttt{results/econometrics/breaks.csv}.}
\label{fig:breaks-by-category}
\end{figure}

\input{tables/tab_p9_its_slopes}

The slope changes are positive in thirty-seven of the forty categories and negative in three (Blockchain
Systems, Deep Learning Architectures, Federated Learning), and the HAC $p$-value is below five per cent in
all but the first two of these; by the conventional criterion almost every category accelerated. The
placebo column tells a different story. Only one category, Applied Cryptography, has a placebo $p$ for the
slope below five per cent, and with forty series two such values are expected under a global null; the
total series is the only series whose December 2022 slope change ranks first among candidate dates. The
reason is visible in the pre-slope column: the category series are noisier than the total, their placebo
distributions are wider, and a slope change that is large in absolute terms, as in Large Language Models
(\ItsSeriesCatLlmItsChatgptSlopeChange{} per month on a pre-period slope of
\ItsSeriesCatLlmItsChatgptPreSlope{}, HAC $p$ rounding to \ItsSeriesCatLlmItsChatgptSlopeP{}), ranks only
\ItsSeriesCatLlmPlaceboChatgptSlopeTRank{}th of \ItsSeriesCatLlmPlaceboChatgptNPlacebo{} candidate dates
(placebo $p$ = \ItsSeriesCatLlmPlaceboChatgptSlopeTPlaceboP{}) because the series has accelerated at many
dates. The placebo rank over all candidate dates is the appropriate measure of whether December 2022 is special
for a given category, and by that measure it is not special for any of them. The baseline-only rank, which
asks the weaker question of whether any month \emph{before} the reference date produced a larger slope
change, answers no for a substantial minority of categories (the zero entries of the last placebo column,
which include AI Agents, Algorithms, Computational Biology and Autonomous Scientific Discovery): their
post-period acceleration exceeds anything in their baseline, even though it does not exceed what other
post-period dates would show.

The ordering of the slope changes is nonetheless informative about where the acceleration is largest. The
five largest changes belong to AI Agents and Autonomous Systems, Large Language Models, Multimodal AI,
Information Retrieval and Distributed Computing; Generative AI, Cloud Computing, Natural Language Processing,
Quantum Computing and Compilers follow. The presence of Information Retrieval, Distributed Computing, Cloud
Computing and Compilers in the upper part of the ranking is a finding that the hypotheses did not anticipate:
these are categories that the LLM literature draws on (retrieval, serving infrastructure, compilation of
models to accelerators) and whose multi-label counts grow because works on language models carry their labels
(Section~\ref{sec:r-rq5}). At the other end, the categories with the smallest positive changes are Edge
Computing, Robotics, Computational Biology, Foundation Models and Computer Vision, all five with pre-period
slopes above the median of the forty, so that a continuation of their growth registers as a small change;
Foundation Models in particular has one of the three steepest pre-period slopes of all categories and a
post-period slope that is only slightly steeper. The three negative changes are of the same kind: Federated
Learning had the steepest baseline slope of all categories and Deep Learning Architectures one of the
steeper ones, and both decelerate to rates that remain positive; Blockchain Systems decelerates from a
2018--2020 boom.

\input{tables/tab_p9_its_levels}

The level changes (Table~\ref{tab:p9-its-levels}) are mostly negative, which is the signature of a trend model
fitted through a kink: when growth accelerates gradually from mid-2022, a regression with a kink fixed at
December 2022 absorbs the months between the true onset and the fixed date into a negative intercept shift.
Fifteen categories have a level change with a HAC $p$ below five per cent, of which only Human--Computer
Interaction, Computer Networks and Sustainable Computing have a positive one; none has a placebo $p$ for the
level below five per cent, and the baseline-only placebo $p$ of the level exceeds a quarter in every
category. The donut estimates are in most cases positive where the full-sample ones are negative, for the
reason given for the total series, and are largest in Large Language Models
(\ItsSeriesCatLlmItsChatgptDonutLevelChange{} with HAC $p$ rounding to
\ItsSeriesCatLlmItsChatgptDonutLevelP{}), Multimodal AI and Generative AI, the three series in which the
winter of 2022--2023 lies furthest below the subsequent trend. The BOCPD column confirms the absence of an
online-detectable change: the maximum four-month changepoint probability over December 2022 to March 2023
exceeds one quarter only in Cloud Computing and Operating Systems, two small and noisy series, and the
detector resets its run length in that window for no category.

\input{tables/tab_p9_unknown_breaks}

The data-driven dates of Table~\ref{tab:p9-unknown-breaks} spread over the window. The Bai--Perron dates
cluster in 2022 (February to August) for a group that includes Large Language Models,
Natural Language Processing, Multimodal AI, Computer Vision, Cybersecurity, Applied Cryptography, Software
Engineering, Sustainable Computing and Privacy-Preserving Computation, and in 2023 (April to November) for a
group that includes Algorithms, Computational Complexity, Computer Architecture, Compilers, Database Systems,
Operating Systems and Speech and Audio Computing; several categories have a second break in mid-2024 and a
few (Blockchain, Federated Learning, Computational Biology, Autonomous Scientific Discovery) have their first
break in 2020. The pre-specified H3 window of October 2022 to June 2023 contains a BIC break date for seven
categories, listed in the table's penultimate column, and a PELT changepoint for twelve. The temporal
pattern that emerges is of two waves: an earlier kink, in 2022, in the categories closest to the generative
model literature and in the large applied categories, and a later kink, in 2023, in the venue-defined
theory and systems categories, whose flat baseline makes a later acceleration easier to date. This reading is
descriptive and is marked for review (\pending{H-4: two-wave reading of the Bai--Perron dates}).

\input{tables/tab_p9_altdates}

Table~\ref{tab:p9-altdates} completes the picture with the level change at each of the five pre-specified
dates. For the AI categories the o1 column (October 2024) carries the largest positive level changes of any
date, which is the second, 2024 kink seen by the Bai--Perron procedure in a different form; the GPT-3 column
(June 2020) carries negative level changes for most categories, which is the pandemic-year irregularity of
the series rather than anything specific to the event. The five columns are the same series with the break
moved, and the table is read across a row as a profile of when a category's level departs from its fitted
trend, not as five tests.

\subsection{Hypothesis H3: structural breaks}
\label{sec:r-h3}

H3 states that Bai--Perron tests on category-level monthly series locate at least one break in the window
from October 2022 to June 2023 for Large Language Models, Generative AI, AI Agents and AI Safety, and that
for Formal Verification, Computational Complexity, Operating Systems and Computer Architecture no break is
detected in that window at the five per cent level after placebo correction. Table~\ref{tab:p9-h3} assembles
the evidence for the eight named categories.

\input{tables/tab_p9_h3}

The first clause is \emph{not supported}. None of the four AI categories has a Bai--Perron break in the
window: Large Language Models breaks in July 2022, Generative AI in November 2021, AI Agents in July 2022
(with a second break in July 2024) and AI Safety in June 2021 and September 2022. All four break
\emph{before} the window, which is the opposite of the error the hypothesis guarded against (a break too late
to be attributed to the boundary): their series accelerated during 2022, in the months of the diffusion-model
and instruction-tuning literature, and the reference date falls inside an acceleration that had already
begun. The PELT dates of these categories fall in late 2023, after the window, for the step-model reason
given above. The second clause is \emph{opposite} for two of its four categories: Computational Complexity
and Computer Architecture have Bai--Perron breaks in April and May 2023, inside the window, and Formal
Verification and Operating Systems break outside it (February 2022 and July 2024; September 2023). The
``five per cent level after placebo correction'' part of the clause cannot be applied as written, because
the Bai--Perron dates carry no $p$-value in the implementation (Section~\ref{sec:m-bp}); the placebo
statistic that exists is the ITS slope placebo, which exceeds five per cent for all eight categories, so that
by that statistic no category, including the four AI categories, has a December 2022 change that stands out
from its own history. The hypothesis, in short, predicted that the boundary would separate categories with
and without a break, and the data show breaks on both sides of the boundary in both groups, earlier in the
AI categories and later in the theory and systems categories. The verdict is \emph{not supported} for the
first clause and \emph{opposite} for the second, with the uncertainty that the Bai--Perron dates have no
sampling distribution attached and move by several months when the truncation month changes
(Table~\ref{tab:p9-total-robustness} for the total series). \pending{H-4: interpretation of the pre-window
breaks of the AI categories}.

\section{RQ3: influential contributions and the distribution of attention}
\label{sec:r-rq3}

RQ3 asks which publications introduced the most influential methodological or theoretical contributions. The
quantitative part of the answer is the citation analysis of Part~VIII and the landmark set of Part~IV; the
qualitative part, the identification of what each landmark contributed, belongs to the specialist reviews of
Parts~V--VII and to checkpoint H-3, and this section reports only what the counts show.

\subsection{Fixed-window citation statistics}

Table~\ref{tab:citation_windows} and Figure~\ref{fig:citation_inequality} give the fixed-window statistics by
publication year. For the cohorts with a complete three-year window (2018 to 2023), the mean number of
citations received within the window rises from \CitationWindowsYearlyTwoZeroOneEightMeanCThreeSix{} for the
2018 cohort to \CitationWindowsYearlyTwoZeroTwoTwoMeanCThreeSix{} for 2022 and
\CitationWindowsYearlyTwoZeroTwoThreeMeanCThreeSix{} for 2023, while the share of works uncited after three
years falls from \CitationWindowsYearlyTwoZeroOneEightShareUncitedThreeSix{} to
\CitationWindowsYearlyTwoZeroTwoTwoShareUncitedThreeSix{} and rises again to
\CitationWindowsYearlyTwoZeroTwoThreeShareUncitedThreeSix{} for the 2023 cohort, whose window closes at the
end of 2025 and is therefore the first to feel the reference-indexing lag. The Gini coefficient of three-year
citations, the inequality statistic of hypothesis H5, falls monotonically through the baseline cohorts, from
\CitationWindowsYearlyTwoZeroOneEightGiniCThreeSix{} (2018) to
\CitationWindowsYearlyTwoZeroTwoTwoGiniCThreeSix{} (2022), and rises to
\CitationWindowsYearlyTwoZeroTwoThreeGiniCThreeSix{} for the 2023 cohort. The baseline decline is driven by the
falling share of uncited works, which is the largest contributor to a Gini of this kind
\citep{nielsen2021inequality}; the 2023 reversal coincides with the return of the uncited share and is, on the
evidence available, a truncation effect rather than a change in the concentration of attention. The 99th
percentile of three-year citations is almost flat across the complete cohorts (Table~\ref{tab:citation_windows}),
so the top of the distribution did not pull away from the body within fixed windows.

The field-weighted citation impact supplied by OpenAlex, whose normalisation is the database's own and not
the study's (Section~\ref{sec:m-citations}), has a mean near \CitationWindowsYearlyTwoZeroOneEightMeanFwci{}
in 2018 and \CitationWindowsYearlyTwoZeroTwoThreeMeanFwci{} in 2023 and a median that rises from
\CitationWindowsYearlyTwoZeroOneEightMedFwci{} to \CitationWindowsYearlyTwoZeroTwoTwoMedFwci{} over the
baseline; a mean well above one for a field whose works are normalised by field means that the
computer-science corpus is cited above the average of the OpenAlex fields its works are assigned to, and
the gap between mean and median is the skewness that the Gini measures.

\subsection{The most cited works and the landmark set}

Table~\ref{tab:top_cited} lists the ten most cited works of each publication year by three-year window
citations, and Section~\ref{sec:net-citations} of Part~VIII discusses it. The list contains the works that a
reader of the period would expect at the top of computer science (contextualised word representations,
BERT, the Vision Transformer, the Swin Transformer, latent diffusion, masked autoencoders, the scientific
Python stack) alongside records that are duplicates of canonical works misdated to a later year (the
transformer paper under 2025, the batch-normalisation paper under 2024, the original GAN paper under 2023,
the Bitcoin white paper under 2019 and 2022) and records whose three-year count exceeds their lifetime
count or whose lifetime count is zero, which are database errors. The ranking of influential works by raw
citations is therefore conditional on an audit that Part~VIII specifies and that has not been performed,
and this part does not derive a list of influential contributions from it.

The landmark set is the study's instrument for that purpose. Selected by field-normalised impact within
category and year with per-area quotas (Section~\ref{sec:m-kg}), it contains \SelectionLandmarks{} works
drawn from a high-priority set of \SelectionHighPriority{}; Table~\ref{tab:p9-landmarks} gives its
composition by area and year. The quotas make the set broader than a citation ranking would be: the AI and
Machine Learning area holds the largest quota but less than a quarter of the set, and the theory area,
whose works are cited least, holds the smallest. By year the set is smallest for the 2021 cohort and largest
for 2023, 2024 and 2026; the size of the 2026 cohort reflects the selection's reliance on the database's
field-weighted impact for the most recent works, where citation counts are smallest and the normalisation
least stable, and the 2026 landmarks should be treated as provisional. \SelectionShareWithAbstractLandmarks{} of the landmarks have an
abstract and \SelectionShareOaLandmarks{} have an open-access location, which bounds what the knowledge-graph
extraction and the specialist reviews can work from. The identification of the methodological or
theoretical contribution of each landmark is the object of the specialist reviews and of checkpoint H-3
and is not reported here (\pending{H-3: landmark list and specialist reviews}).

\input{tables/tab_p9_landmarks}

\subsection{Hypothesis H5: concentration}
\label{sec:r-h5}

H5 states that citation inequality (the Gini of 24-month citations within year) is higher in the post period
than in the baseline for AI categories, and that venue concentration (HHI by source) is stable or lower
because of preprint growth. The citation component is stated for the AI categories, and the per-category
citation-window statistics are not among the result files; the verdict on that component is
\emph{inconclusive} (\pending{macro: per-category Gini of fixed-window citations, results/bibliometrics;
not produced by the current pipeline}). The aggregate counterpart, read from Table~\ref{tab:citation_windows}
with the 36-month Gini that the pipeline computes in place of the 24-month one, falls through the baseline
and rises for the single complete post-period cohort, a movement that Section~\ref{sec:net-citations}
attributes to the uncited share and to right truncation; it is not evidence of higher concentration. The
venue component is examined in Section~\ref{sec:net-venues}: the Herfindahl index with arXiv counted as a
source rises from \VenueHhiYearlyTwoZeroOneEightHhi{} in 2018 to \VenueHhiYearlyTwoZeroTwoFiveHhi{} in 2025
and the top-source share from \VenueHhiYearlyTwoZeroOneEightTopOneShare{} to
\VenueHhiYearlyTwoZeroTwoFiveTopOneShare{}, because arXiv is the top source and its share grows; the
protocol anticipated this mechanism and predicted stability or a fall once preprints are treated separately,
and the index without the preprint type is not among the result files. The venue component is therefore
\emph{opposite} on the index as computed and \emph{inconclusive} on the index the hypothesis intended
(\pending{macro: venue HHI excluding repository sources}). The number of distinct sources grows in every
year (Table~\ref{tab:venue_hhi}), so the non-arXiv venue landscape widens rather than concentrates.

\section{RQ4: citation networks and knowledge flows}
\label{sec:r-rq4}

\subsection{Flows between fields}

The part of RQ4 that concerns the flow of knowledge between computer science and other fields is answered in
Section~\ref{sec:net-flows} of Part~VIII from the resolved references of the eligible works
(Table~\ref{tab:reference_coverage}, Table~\ref{tab:field_flows}, Figure~\ref{fig:field_inflows}). The share
of references that stay inside the corpus rises through the window as the corpus itself grows, and the share
pointing to the largest other OpenAlex fields (medicine, engineering, mathematics, physics, the social
sciences) falls, with no change of pace at the boundary; the age distribution of cited works shifts toward
younger work in every year (Table~\ref{tab:cited_age}, Figure~\ref{fig:cited_age}), again without a visible
inflection. Those results are not repeated here. This section reports the flows \emph{within} computer
science, between the areas and categories of the ontology, which depend on the classification and were
not available to Part~VIII.

\subsection{Flows between areas}

Figure~\ref{fig:area-flow-matrix} shows the $8 \times 8$ area flow matrices of equation~\eqref{eq:flow},
row-normalised, for the baseline and post spans, and Table~\ref{tab:p9-area-flow-change} gives their
difference in percentage points. The diagonal dominates both matrices: in every area a majority of internal
references stay within the area, from about three fifths in Security, Privacy and Trust to well over four
fifths in Perception, Language, Robotics and Interaction, which cites itself more than any other area does
and whose self-share does not fall in the post period. The off-diagonal structure is a hub-and-spoke pattern
centred on two areas: every area sends a larger share of its references to AI and Machine Learning in the
post span than in the baseline (by between one and two and a half points for the systems, security,
software, theory and computational-science areas), and the AI area itself sends a larger share to
Perception, Language, Robotics and Interaction and a smaller share to itself. The largest movements in the
table are the falls of the self-shares of the AI, Data, Software and Security areas, each of about three
points, matched by rises of their references to the AI area and, for the AI area itself, to the vision and
language categories. The Hardware area is the exception that moves least: its row changes by less
than a point in every cell.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_area_flow_matrix.pdf}
\caption{Citation-flow matrices between areas (row shares of internal references, citing area in rows) for
the baseline and post spans. Source: \texttt{results/graph/area\_flows\_yearly.csv}.}
\label{fig:area-flow-matrix}
\end{figure}

\input{tables/tab_p9_area_flow_change}

The matrices are built from internal references only, that is, references whose cited work is in the corpus
and has a primary category, and the number of such references grows by an order of magnitude over the window
(Table~\ref{tab:p9-modularity}, second column) because works of 2018 could cite few corpus works, all of them
from 2018, whereas works of 2025 can cite seven years of the corpus. The 2018 matrix is therefore computed on a
small and peculiar set, and the comparison of spans is best read from 2019 onward.

\subsection{Cross-area shares and modularity}

Table~\ref{tab:p9-cross-area} and Figure~\ref{fig:cross-area} report the statistic behind H4, the cross-area
share of equation~\eqref{eq:cross-area}, by citing area and year. The areas differ by a factor of three in
their propensity to cite outside themselves: the AI area sends about half of its internal references to other
areas (most of them to vision and language), the perception area about a sixth, and the remaining six
between a quarter and two fifths. Over time the AI area's cross-area share rises through the baseline, from
2018 to 2022, and is flat from 2022 onward; the Data, Security, Software and Theory areas rise slowly and
steadily through the whole window; the Hardware area is flat; and the Perception area falls from 2018 to
2019 and is flat thereafter. The aggregate share, in the penultimate rows of the table, rises from 2022 to
2025 after being flat through the baseline. Part of that rise is composition: references made by AI-area
works, which are the most boundary-crossing, form a growing share of all references. The composition-adjusted
aggregate, which holds each area's weight at its baseline share of internal references, rises by less than
the raw one over 2022--2025 but still rises, so the boundary-crossing propensity of the areas themselves
increased slightly in the post period, by an amount that is about half of the raw increase.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_cross_area.pdf}
\caption{Cross-area reference shares by citing area and year, and modularity of the category citation-flow
network at the category and area partitions. Sources: \texttt{results/graph/cross\_area\_share\_yearly.csv},
\texttt{modularity\_yearly.csv}.}
\label{fig:cross-area}
\end{figure}

\input{tables/tab_p9_cross_area}

The modularity of the yearly flow network (Table~\ref{tab:p9-modularity}) summarises the same information at
the two partitions. $Q_{\mathrm{area}}$ falls in every year from 2018 to 2024 and is flat in 2025, with the
largest single-year falls in 2020--2021 and in 2023; $Q_{\mathrm{cat}}$ is flat through 2022 and falls
afterwards; and the uncorrected self-share of citations within the citing category is flat through the
baseline and falls from 2023. The category partition and the area partition therefore tell slightly
different stories: the assortativity of citations by \emph{area} declined throughout the window, whereas
the assortativity by \emph{category} held until the reference year and declined afterwards, which is the
signature of categories within the same area (Large Language Models and Natural Language Processing, Computer
Vision and Multimodal AI) citing each other more in the post period. Because the partition is fixed by the
classifier rather than optimised, these values are descriptive indices of assortativity and not community
structure, and their decline is consistent with both a real blurring of category boundaries and a drift of
the classifier's own boundaries as the vocabulary of the literature changed (Section~\ref{sec:m-networks}).

\input{tables/tab_p9_modularity}

\subsection{Hypothesis H4: convergence}
\label{sec:r-h4}

H4 states that the share of citations crossing area boundaries (systems to AI, security to AI, HCI to AI,
and so on) rises in the post period relative to the baseline, and that the modularity of the citation-flow
network by area decreases. Both clauses are \emph{supported} in direction. The cross-area shares of the
systems, security, software and theory areas rise in the post period, and the specific flows the hypothesis
names, from each of these areas toward AI and Machine Learning, are the cells of
Table~\ref{tab:p9-area-flow-change} with the largest positive changes; the area modularity is lower in every
post-period year than in any baseline year. The support is qualified in three ways. The rises are small,
of the order of two to four percentage points over the window for the areas concerned, and they continue
trends that are visible within the baseline; the aggregate increase is about half composition and half
behaviour; and the Perception area, which is the largest and contains the HCI category named in the
hypothesis, does not increase its cross-area share at all, so that ``HCI to AI'' is not supported at the area
level at which the statistic is computed (the category-level flows of Table~\ref{tab:p9-llm-partners} and of
the co-classification analysis in Section~\ref{sec:r-rq5} show HCI among the categories most often paired with
Large Language Models, which is a different measurement). The modularity decline is monotone across the
boundary rather than concentrated at it, so the hypothesis is supported as a statement about the post period
relative to the baseline and not as a statement about a change at the reference date. The classifier
caveat applies with full force: a change in how the classifier partitions the literature would produce the
same movements, and the semantic-level check of H4 is pending (Section~\ref{sec:r-rq5}).

\section{RQ5: foundation models and the boundaries between disciplines}
\label{sec:r-rq5}

RQ5 asks whether the emergence of foundation models changed the relationship between previously distinct
computer-science disciplines. Three measurements bear on it: the citation flows of Section~\ref{sec:r-rq4},
the co-classification of works across categories, and the semantic distances between category centroids in
the embedding space. The first was reported above; the second is reported here; the third depends on the
semantic atlas, which was not available when this part was written, and its subsections are reserved.

\subsection{Co-classification}

Table~\ref{tab:p9-cooccurrence} lists the fifteen most frequent pairs of categories carried jointly by a
work in its label set, for the baseline and post spans. The baseline list is dominated by pairs that
describe the standing structure of the field: Computer Vision with Deep Learning Architectures, Computer
Networks with Cybersecurity and with Edge Computing, Applied Cryptography with Cybersecurity, Computer
Architecture with Distributed Computing, Cloud with Distributed Computing. Large Language Models with
Natural Language Processing is the third pair of the baseline list, which is a reminder that the category
is defined by a vocabulary (language models, pre-training, transformers) that predates ChatGPT by several
years. The post-period list keeps the vision and networking pairs at the top but is otherwise reorganised
around the generative-model categories: Large Language Models with Natural Language Processing moves to
second place with more than twice its baseline count; Computer Vision pairs with Foundation Models, with
Multimodal AI and with Generative AI; Deep Learning Architectures pairs with Foundation Models and with
Natural Language Processing; and Foundation Models with Large Language Models enters the list. Six of the
fifteen post-period pairs involve a category that did not appear in any baseline pair.

\input{tables/tab_p9_cooccurrence}

Table~\ref{tab:p9-llm-partners} looks at the same phenomenon from the side of the Large Language Models
category. Its partners in the post span are, after Natural Language Processing, Foundation Models, Deep
Learning Architectures, Computer Vision, Multimodal AI, Information Retrieval, Software Engineering, AI
Safety and Alignment, Machine Learning Theory, AI Agents, Generative AI, Cybersecurity and Human--Computer
Interaction; the ratio column shows that the pairings with Information Retrieval, Software Engineering,
Cybersecurity and Human--Computer Interaction, which are categories outside the AI area, grew by factors
comparable to or larger than the pairing with Natural Language Processing. These are the category-level
counterparts of the area flows of Section~\ref{sec:r-rq4}: works that carry the LLM label increasingly also
carry the label of an application or systems category, which is one mechanism by which the multi-label
series of those categories accelerated (Section~\ref{sec:r-its-categories}). The measurement is conditional
on the classifier in the strongest possible way, since co-classification is a property of the label set, and
on abstract availability, since a work without an abstract rarely carries more than one label; both rise in
the post period (Table~\ref{tab:p9-classification}), and part of the growth of every pair is attributable to
them.

\input{tables/tab_p9_llm_partners}

\subsection{Semantic distances between categories}
\label{sec:r-semantic}

The protocol's semantic test of convergence compares the cosine similarity between category centroids in
the embedding space, estimated separately on baseline and post works, and reports the drift of each
category's centroid between the two periods (Section~\ref{sec:m-semantic}). The result files
\texttt{results/semantic/}\allowbreak\texttt{category\_similarity\_baseline.csv},
\texttt{category\_similarity\_post.csv} and \texttt{category\_drift.csv} did not exist when this part was written. \pending{semantic atlas: insert the
category similarity comparison (baseline versus post, mean off-diagonal similarity, the pairs whose
similarity changed most) and the drift table when results/semantic/ is produced; the diversity indices of
Table~\ref{tab:diversity} are to be re-estimated with the semantic distance at the same time}.

\subsection{Semantic clusters and their stability}

The clustering of the embedded corpus into topics and the bootstrap stability of that clustering
(\texttt{results/semantic/clusters.csv}, \texttt{stability.json}) are likewise pending. \pending{semantic
atlas: number of clusters, share of works in noise, adjusted Rand index of the bootstrap replicates, and the
clusters that are new in the post period, from results/semantic/clusters.csv and stability.json}.

\subsection{Reading of RQ5 on the available evidence}

With two of the three measurements available, the answer to RQ5 is that the generative-model categories
have become the most connected part of the classification, both in what their works cite and in what other
labels their works carry, and that the areas of computer science that were most distinct from AI in the
baseline (systems, security, software, theory) now send a somewhat larger share of their references to it.
Whether this amounts to a change in the relationship between disciplines, or to the growth of one large
cross-cutting literature alongside disciplines that remain distinct, cannot be settled with citation and
label data: the self-shares of Table~\ref{tab:p9-area-flow-change} remain above three fifths in every area,
and the Perception area, the largest, is as self-contained in the post period as it was before. The
semantic measurement, when available, is the one that can discriminate between the two readings, because it
does not depend on the classifier's boundaries.

\section{RQ6: institutions, countries and collaboration}
\label{sec:r-rq6}

Part~VIII analyses the actor networks in full, and this section records the aggregate indicators that the
hypotheses require and the verdict on H6. The international co-authorship share, among works with at least
one resolved country, moves from \InternationalCollabYearlyTwoZeroOneEightShareIntl{} in 2018 to a maximum of
\InternationalCollabYearlyTwoZeroTwoZeroShareIntl{} in 2020 and declines in every subsequent year to
\InternationalCollabYearlyTwoZeroTwoFiveShareIntl{} in 2025 (Table~\ref{tab:collaboration},
Figure~\ref{fig:collaboration}). The share of works with a company affiliation, among works with any typed
affiliation, rises from \IndustryShareYearlyTwoZeroOneEightShareCompany{} in 2018 to a maximum of
\IndustryShareYearlyTwoZeroTwoOneShareCompany{} in 2021 and falls to
\IndustryShareYearlyTwoZeroTwoFiveShareCompany{} in 2025 (Table~\ref{tab:industry}). The country shares with
fractional counting (Table~\ref{tab:country_shares}, Figure~\ref{fig:country_shares}) show the share of
China rising through the window and the share of the United States falling, under both counting
conventions, as the continuation of a baseline trend; the joint output of the two countries falls in
absolute terms in the post period while both grow (Table~\ref{tab:country_network},
Section~\ref{sec:net-international}). The institutional hierarchy is stable: the institutions with the
largest post-period output are, with few exceptions, those with the largest baseline output
(Table~\ref{tab:top_institutions}), and the concentration of output across institutions does not increase
(Table~\ref{tab:institution_concentration}).

\subsection{Hypothesis H6: geography and collaboration}
\label{sec:r-h6}

H6 states that the share of works with industry affiliations and with China-affiliated authors rises in the
post period in AI categories, and that the international co-authorship share does not rise. The third clause
is \emph{supported}: the international share does not rise and in fact declines, with the composition
explanation that Part~VIII develops (the fastest-growing producers are the least internationalised). The
China clause is \emph{supported} in the aggregate, as a continuation of a trend that predates the boundary,
and \emph{inconclusive} for AI categories, because the category-specific country shares are not among the
result files (\pending{macro: China-affiliated share by category and period, results/bibliometrics}). The
industry clause is \emph{opposite} in the aggregate, where the company share peaks in 2021 and declines in
every post-period year, and \emph{inconclusive} for AI categories (\pending{macro: company-affiliated share by
category and period}). Two remarks qualify the opposite verdict. The company share is measured on works with
a typed affiliation, and the affiliation coverage of the corpus changes over the window
(Table~\ref{tab:collaboration}, last column), so a falling share can coexist with a rising count of
industry works; the count does rise (Table~\ref{tab:industry}). And the literature from which the
hypothesis was drawn measures industry participation at the frontier of machine learning research
\citep{ahmed2023industry}, whereas the corpus measures it over all of computer science, where the growth of
the post period is concentrated in categories and venues with low industry participation.

\section{RQ7 and RQ8: breakthroughs, incremental gains and unresolved problems}
\label{sec:r-rq7}

RQ7 asks which research areas demonstrate genuine breakthroughs versus incremental benchmark improvements,
and RQ8 which scientific problems remain unresolved despite rapid publication growth. Neither question has a
bibliometric answer, and the study does not pretend otherwise: the evidence for them is the knowledge-graph
extraction over the high-priority set (method, benchmark, dataset and claim nodes with provenance and
confidence, Section~\ref{sec:m-kg}) and the specialist reviews of Parts~V--VII, which are synthesised in
Part~X. At the time of writing the extraction had not produced result files that this part could tabulate,
and the specialist reviews were subject to checkpoint H-3. What the quantitative results of this part
contribute to the two questions is a set of constraints. The categories with the fastest post-period growth
(Section~\ref{sec:r-rq1}) and the strongest acceleration (Section~\ref{sec:r-its-categories}) are the ones in
which benchmark-driven incremental work is most plausible, because volume growth of this speed is not
compatible with a proportional growth in the number of distinct problems addressed; the categories whose
volume is flat but whose cross-area references to AI rise (Section~\ref{sec:r-rq4}) are the ones in which an
imported method is being applied to standing problems, which is where a breakthrough, if any, would register
as a change of method rather than of volume. The emerging-topic detector of Section~\ref{sec:r-rq10} provides
a third constraint: the topics it ranks highly are candidates for RQ7, and the topics with sustained volume
and declining citation acceleration are candidates for RQ8. These are hypotheses for Part~X, not findings
(\pending{H-3: knowledge-graph extraction counts by relation type, benchmark proliferation and method-adoption
series for the high-priority set, when produced}).

\section{RQ9: openness, preprints and the evaluation of evidence}
\label{sec:r-rq9}

Table~\ref{tab:p9-openness} assembles the openness indicators by publication year; Figure~\ref{fig:types-openness}
in Part~III shows the same series graphically. Three of them rise through the whole window and faster in the
post period. The share of eligible works linked to an arXiv e-print rises in every year, from
\VolumeYearlyTwoZeroOneEightNArxiv{} of \VolumeYearlyTwoZeroOneEightN{} works in 2018 to
\VolumeYearlyTwoZeroTwoFiveNArxiv{} of \VolumeYearlyTwoZeroTwoFiveN{} in 2025, roughly a doubling of the share
and a quadrupling of the count; the share of works whose surviving record is of preprint type rises more
steeply, because recent preprints have not yet been matched to a published version, and the 2026 value is
dominated by that effect. The open-access share rises from \VolumeYearlyTwoZeroOneEightNOa{} to
\VolumeYearlyTwoZeroTwoFiveNOa{} works, a rise in share of about twenty points over the window of which the
larger part occurs in the baseline; the abstract-availability share rises through 2024 and falls in 2025
and 2026 for records not yet enriched. The DBLP-indexed share falls in the post period, as
Section~\ref{sec:r-rq1} noted, and the conference share of the corpus is flat or slightly declining, which
means that the growth of preprints is not matched by a growth of conference publication in the indexed
record. The number of references per work (last column) is a metadata-completeness indicator rather than
an openness one and is discussed in Section~\ref{sec:r-rq1}.

\input{tables/tab_p9_openness}

\subsection{Hypothesis H7: preprints and openness}
\label{sec:r-h7}

H7 states that the share of works with an arXiv version and the share with linked code rise in the post
period. The arXiv clause is \emph{supported}: the share rises in every post-period year and the post-period
share is above the baseline share by a wide margin (Table~\ref{tab:p9-periods}), with the caveat that the
post-period arXiv share is inflated by preprints that will later be merged with a published version and by
the growth of OpenAlex's coverage of arXiv. The code clause is \emph{inconclusive}: the pipeline does not
extract code links, and no result file measures the share of works with a linked repository
(\pending{macro: share of works with a code link by year; not produced by the current pipeline, scoped in
Phase 3 of the plan}). The evaluation-standards part of RQ9, which asks how benchmarks and reporting
practices evolved, is likewise outside what the bibliographic record measures and is addressed from the
knowledge-graph extraction in Part~X when available.

\section{RQ10: early detection of emerging topics}
\label{sec:r-rq10}

\subsection{Out-of-time performance of the detectors}

Table~\ref{tab:p9-emerging-setup} summarises the evaluation design implemented in
Section~\ref{sec:m-emerging}: OpenAlex topics as units, features computed at a cutoff from data at or before
it, a label defined by top-decile growth with doubling over the following twelve or twenty-four months,
training cutoffs that end before the reference date and test cutoffs in 2023--2024. At the twelve-month
horizon the pooled test set has \EmergNTest{} (topic, cutoff) rows with a positive rate of
\EmergPosRateTest{}, lower than the training rate of \EmergPosRateTrain{}; at twenty-four months the rates
are \EmergtwofourPosRateTest{} and \EmergtwofourPosRateTrain{}. The positive rate is below the ten per cent
that a decile label would give because of the doubling condition, and it is lower in the test period than in
training because the corpus as a whole grew faster in 2023--2026, so that doubling against the trailing
window became harder for an individual topic.

\input{tables/tab_p9_emerging_setup}

Tables~\ref{tab:emerging_h12} and~\ref{tab:emerging_h24} in Appendix~F give the pooled metrics, and
Figure~\ref{fig:emerging-metrics} displays them. At the twelve-month horizon the gradient boosting model
attains an AUC of \EmergGbmAuc{}, the $z$-score baseline \EmergZscoreBaselineAuc{} and the logistic model
\EmergLogisticAuc{}; precision at fifty is \EmergGbmPrecisionAtFiveZero{} for the boosted model and
\EmergZscoreBaselinePrecisionAtFiveZero{} for the baseline, against a chance level of about the positive
rate. At the twenty-four-month horizon the ordering reverses: the $z$-score baseline attains an AUC of
\EmergtwofourZscoreBaselineAuc{} and a precision at fifty of \EmergtwofourZscoreBaselinePrecisionAtFiveZero{},
and both trained models fall below it (\EmergtwofourLogisticAuc{} and \EmergtwofourGbmAuc{}). The per-cutoff
values of Table~\ref{tab:p9-emerging-cutoffs} show that the pooled figures hide large variation: the
twelve-month AUC of the boosted model ranges over the eight test cutoffs from a little above six tenths to
near one, the near-perfect value belonging to the January 2024 cutoff, at which the baseline is also near
perfect and the logistic model near zero, a configuration that indicates a cutoff with very few positives
whose ranking is determined by one or two topics rather than a cutoff at which the models excel. Excluding
that cutoff, the boosted model exceeds the baseline at five of the seven remaining twelve-month cutoffs,
and the baseline exceeds both trained models at every twenty-four-month cutoff.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_emerging_metrics.pdf}
\caption{Out-of-time detection performance of the three detectors by horizon (AUC and precision at 20 and
50). Sources: \texttt{results/emerging/metrics\_h12.json}, \texttt{metrics\_h24.json}.}
\label{fig:emerging-metrics}
\end{figure}

\input{tables/tab_p9_emerging_cutoffs}

Three features of the design explain the pattern. The training set is small, with eight cutoffs at twelve
months and four at twenty-four, and the positives within it number in the dozens; a boosted model with three
hundred trees fitted to that many positives learns the idiosyncrasies of the 2020--2021 cutoffs, which the
reversal at twenty-four months, where training is smallest, bears out. The $z$-score is itself one of the
six features, so the trained models can at best refine a ranking that the baseline already provides, and at
the longer horizon the refinement overfits. And the label's doubling condition ties positives tightly to
recent abnormal growth, which is what the $z$-score measures, so that a longer horizon, over which abnormal
growth has more time to translate into doubling, favours the simplest signal. The logistic model's
performance below the baseline at both horizons, and below one half at some cutoffs, indicates that the
linear combination of standardised features is dominated by the size and age controls, which are
negatively associated with the label.

\subsection{Hypothesis H8: detection}
\label{sec:r-h8}

H8 states that the detector trained on cutoffs up to 2022 achieves an AUC above 0.70 and a precision at
fifty above 0.40 on test cutoffs in 2023--2024 against the baseline growth $z$-score rule, and that the
preregistered known-emergence set is detected with a median lead time of at least six months before each
topic's peak growth month. The first clause is \emph{not supported}. The best trained model's twelve-month
AUC, \EmergGbmAuc{}, is at the threshold to three decimals and marginally below it before rounding; its
precision at fifty is an order of magnitude below the threshold; and at twenty-four months the trained
models do not beat the baseline at all, so the comparison ``against the $z$-score rule'' is lost rather than
won. The AUC values say that the signals rank future emergers above non-emergers with a probability of about
seven in ten at twelve months and, through the baseline, of about eight in ten at twenty-four months, which
is informative; the precision values say that a list of fifty flagged topics at a given cutoff contains very
few of the topics that will double, which is the operationally relevant statement and the one the threshold
was set for.

The second clause concerns the known-emergence set, and Section~\ref{sec:m-emerging} explains why it could
not be tested as written: the twelve developments do not map to OpenAlex topics, so the check actually run
applies the $z$-score rule to keyword series defined by regular expressions, and measures the lead of the
rule's first firing over the ex-post peak growth month. Figure~\ref{fig:known-set} shows the twelve series
with the first alarm, and Table~\ref{tab:known_set} the detection months, peak months and leads. The rule
fires for all \KnownSetNDetected{} concepts, with a positive lead for \KnownSetNPositiveLead{} of them and a
median lead of \KnownSetMedianLeadMonths{} months. The leads are longest for large language models
(\KnownLargeLanguageModelsLeadMonths{} months, with a detection in
\KnownLargeLanguageModelsDetectionMonth{} and a peak in \KnownLargeLanguageModelsPeakGrowthMonth{}) and for
reasoning models (\KnownReasoningModelsLeadMonths{} months, the regular expression for which includes
chain-of-thought prompting and therefore fires in \KnownReasoningModelsDetectionMonth{}, long before the
test-time-compute literature of 2024--2025 that gives the series its peak), and shortest for neural radiance
fields (\KnownNeuralRadianceFieldsLeadMonths{} months) and for mechanistic interpretability, whose lead is
negative (\KnownMechanisticInterpretabilityLeadMonths{} months): that series' fastest growth, in
\KnownMechanisticInterpretabilityPeakGrowthMonth{}, came from a base of a few works and the rule's volume
condition was not met until \KnownMechanisticInterpretabilityDetectionMonth{}. On the check as run, the
second clause is \emph{supported}: the median lead exceeds six months. The support is weak for three
reasons that the method section states and that bear repeating: the units are hand-written keyword series
rather than the detector's topics, the detector is the baseline rule rather than the trained model, and the
peak month is an ex-post quantity, so that a lead measured against it is not a lead that an observer could
have known at the time. The verdict on H8 as a whole is \emph{not supported} for the trained detector and
\emph{supported with reservations} for the lead-time clause on its substitute test.

\begin{figure}[htbp]\centering
\includegraphics[width=0.95\textwidth]{fig_known_set.pdf}
\caption{Monthly keyword series for the twelve preregistered concepts with the $z$-score rule's first alarm.
Source: \texttt{results/emerging/known\_set\_monthly.csv}.}
\label{fig:known-set}
\end{figure}

The scientific content of the exercise lies in what it rules out. A retrospective evaluation with a fixed
2026 topic vocabulary, a growth-only label and a handful of training cutoffs is the most favourable setting a
detector of this kind can be given short of look-ahead, and in it the trained models do not improve on a
one-line rule. Two of the mandate's seven candidate signals (semantic novelty, interdisciplinary connectivity)
and three others (methodological adoption, benchmark proliferation, open-source activity) are not in the
feature set (Section~\ref{sec:m-emerging}); the result is therefore a lower bound on what the full framework
could achieve, and an upper bound on what growth and citation signals alone achieve. \pending{H-4: whether
to describe the detector as a negative result for RQ10 or as a baseline for the semantic-cluster extension}.

\section{Validation of the classification}
\label{sec:r-validation}

Every category-level result of this part is conditional on the rule-based classifier of Part~IV, and the
protocol requires its validation on a stratified sample of two thousand works rated independently by two
raters, with Cohen's $\kappa$ between the raters, and the precision, recall and F1 of the assignment against
each rater and against their consensus, by category (Section~\ref{sec:m-validation}; checkpoint H-2). The
ratings of the two raters exist in \texttt{taxonomy/validation/}, but the metrics file
\texttt{taxonomy/validation/metrics.json} and the disagreement list had not been produced when this part was
written, so the validation cannot be reported here. \pending{taxonomy validation: insert the inter-rater
$\kappa$, the overall precision, recall and F1 of the assignment against the consensus, the number of
disagreements, and the categories with the lowest F1, from taxonomy/validation/metrics.json and
disagreements.csv; the generated Table~\ref{tab:validation} will exist once the file does}. Until then, the
reader should weight the category results by the known failure modes of the classifier (Section~\ref{sec:m-classification}):
works without an abstract are classified from venue and arXiv category alone, which favours the venue-defined
categories; the keyword-defined categories (the AI area, Sustainable Computing, Autonomous Scientific
Discovery) are sensitive to vocabulary drift; and the primary category of a multi-label work is decided by a
score whose ties are not random. The record-linkage precision estimated on the manually reviewed sample
(\LinkagesampleDoiAutoPrecision{} for DOI links, \LinkagesampleTitleYearAutoPrecision{} for title-and-year
links; Section~\ref{sec:m-linkprecision}) is the one validation statistic of the corpus that is available,
and it bounds the error introduced by linkage rather than by classification.

\section{Summary of the hypotheses and of what remains uncertain}
\label{sec:r-summary}

Table~\ref{tab:p9-hypotheses} collects the verdicts. The table is a reading aid and not a result file; each
verdict is argued in the section cited and carries the qualifications stated there.

\begin{table}[htbp]\centering
\caption{Verdicts on the pre-specified hypotheses on the evidence of this part.}
\label{tab:p9-hypotheses}
\small
\begin{tabular}{p{0.07\textwidth}p{0.36\textwidth}p{0.17\textwidth}p{0.3\textwidth}}
\toprule
\textbf{H} & \textbf{Clause} & \textbf{Verdict} & \textbf{Where and why} \\
\midrule
H1 & Post-period monthly growth rate higher than baseline & supported & Sec.~\ref{sec:r-h1}; slope change ranks first among all candidate dates \\
   & Level shift at the boundary not distinguishable from placebo breaks & supported & placebo rank of the level near the bottom; acceleration dated to spring 2022 by Bai--Perron \\
   & Growth differences concentrated in a few categories & not supported & Sec.~\ref{sec:r-its-categories}; positive slope change in 37 of 40 categories \\
H2 & AI-category share rises in the post period & supported & Sec.~\ref{sec:r-h2}; driven by the language and generative categories; Deep Learning Architectures falls \\
   & A non-AI area declines relatively while growing absolutely & supported & three areas (data and distributed, theory, software) \\
H3 & Bai--Perron break in Oct.\ 2022--June 2023 for LLM, generative AI, agents, AI safety & not supported & Sec.~\ref{sec:r-h3}; all four break before the window (2021--2022) \\
   & No break in the window for formal verification, complexity, OS, architecture & opposite (two of four) & complexity and architecture break inside the window; no $p$-value attached to the dates \\
H4 & Cross-area citation shares rise & supported, small & Sec.~\ref{sec:r-h4}; about half composition; Perception area does not rise \\
   & Area modularity decreases & supported & monotone through the window, not concentrated at the boundary \\
H5 & Citation Gini higher in post period for AI categories & inconclusive & Sec.~\ref{sec:r-h5}; per-category windows not produced; aggregate movement is a truncation effect \\
   & Venue HHI stable or lower & opposite as computed, inconclusive as intended & index without repositories not produced \\
H6 & Industry share rises (AI categories) & opposite (aggregate), inconclusive (AI) & Sec.~\ref{sec:r-h6}; company share peaks in 2021 \\
   & China share rises (AI categories) & supported (aggregate), inconclusive (AI) & continuation of a baseline trend \\
   & International co-authorship share does not rise & supported & declines from 2020 \\
H7 & arXiv-version share rises & supported & Sec.~\ref{sec:r-h7}; inflated by unmatched recent preprints \\
   & Linked-code share rises & inconclusive & not measured \\
H8 & Trained detector AUC above 0.70 and P@50 above 0.40 versus the baseline & not supported & Sec.~\ref{sec:r-h8}; AUC at the threshold, P@50 an order of magnitude below; baseline wins at 24 months \\
   & Known-emergence set detected with median lead at least 6 months & supported with reservations & substitute keyword test, baseline rule, ex-post peak \\
\bottomrule
\end{tabular}
\end{table}

Four sources of uncertainty run through every verdict and are stated here once. The first is right
truncation: the post period ends in an incomplete year, the reference-indexing lag removes a growing share
of references from the most recent cohorts, and every citation-based statistic of 2024 and later is
provisional; the structural-change analysis handles this by truncating at \ItsLastComplete{} and varying the
truncation month, and the citation analysis by reporting only complete windows, but neither device recovers
what the database has not yet indexed. The second is the boundary of the database: the growth of the corpus
in the post period is concentrated in records outside DBLP's curation, and the study cannot separate growth
in computer-science output from growth in OpenAlex's coverage of it \citep{culbert2025openalexcoverage,
alperin2024openalex}; the DBLP-only records of Table~\ref{tab:dblp_only} and the strict core of
\PrismaStrictCoreDblpAndCsPrimary{} works are the instruments for a sensitivity analysis that the current
result files do not yet contain (\pending{macro: structural-change estimates on the strict core, as the
protocol's sensitivity analysis}). The third is the classifier, whose validation is pending and whose
boundaries are fixed in 2026 vocabulary and applied to 2018 works; every category-level and every flow
statistic inherits it, and the semantic measurements that do not depend on it are the ones still to come.
The fourth is the inferential status of the time-series results: the reference date is a marker, the
estimates at it are descriptions of a kink in a series that was already bending, and the placebo and
alternative-date analyses show that the kink is not tied to the month. What the evidence supports is a
multi-year acceleration of the indexed output of computer science, concentrated in share terms in the
language and generative-model categories but present in nearly all of them, accompanied by a modest increase
in the citation traffic from the other areas toward AI and by a reorganisation of the co-classification
structure around the foundation-model categories, and a continuation, without inflection, of the baseline
trends in collaboration, geography and openness. What it does not support is a discrete change at the
reference date in any series examined, a concentration of the acceleration in a few categories, or the
ability of simple growth and citation signals to pick the emerging topics of 2023--2024 with useful
precision. \pending{H-4: the summary reading of this paragraph}.