spb/wp44_uqo

1% part09_results.tex — Part IX, Empirical Results. Written by Agent L (Part IX instance, with Agent I/J roles), 2026-11-03.2% Corpus v3 result files (results/, data/metadata/, reports/classification_report.json); every number is a macro of3% tables/numbers.tex or a cell of a table generated by tables/make_tables.py (tables_part09). Items marked \pending{} await4% a result file (taxonomy validation metrics, semantic atlas) or the PI checkpoint H-4 (interpretation of structural change).56\section{Scope, sources and reading rules}7\label{sec:r-scope}89This part collects the quantitative findings of the study and reads them against the ten research questions and10the eight pre-specified hypotheses of the protocol. It is deliberately a reporting part: the methods are those of11Part~IV, the interpretation of the literature is left to Parts~V--VII, and the discussion of what the findings12mean for computer science is left to Part~XI. Three kinds of material appear below. The first is the set of13descriptive series on volume, composition, openness and influence, which answer RQ1, RQ3 and RQ9 and provide14the evidence for H2 and H7. The second is the structural-change analysis of the monthly series, in total and by15category, which answers RQ2 and tests H1 and H3. The third is the set of network and detection results, which16answer RQ4--RQ6 and RQ10 and test H4--H6 and H8. The actor networks of RQ6 (authors, countries, institutions,17venues) are analysed at length in Part~VIII; this part does not repeat that analysis but summarises its verdicts18where a hypothesis requires them and cross-references its tables by label.1920The evidence base is the eligible corpus of Part~III in its third build, with \PrismaCsCoreEligible{} works in the21computer-science core, of which \PrismaEligibleByPeriodBaseline{} have a first date in the baseline window and22\PrismaEligibleByPeriodPost{} in the post window (Table~\ref{tab:p9-periods}). Every monthly statistic below is23computed on the month-precision subset, \PrismaEligibleDatePrecisionMonth{} works, because the24\PrismaEligibleDatePrecisionYear{} works whose only date is a year would otherwise be stacked on January25(Section~\ref{sec:data-dates}; Figure~\ref{fig:volume-monthly-raw}); yearly statistics use all eligible works.26Table~\ref{tab:p9-periods} shows that the two periods are not alike in completeness: the post period has a27higher share of month-precision dates, a higher arXiv-linked share and a much lower share of works with at28least one resolved reference, the last being an artefact of reference indexing, which lags publication by more29than a year for a substantial fraction of records (Table~\ref{tab:indexing_lag}, Section~\ref{sec:net-boundaries}).30Any comparison of the two periods inherits these differences, and the text names them wherever they bear on a31reading.3233\input{tables/tab_p9_periods}3435The category-level results depend on the classification of Part~IV. Table~\ref{tab:p9-classification} reports its36coverage: the share of eligible works that received a primary category rises from under four fifths in 2018 to37over four fifths in the last years, and the number of labels per classified work rises monotonically through38the window. Both movements matter for what follows. The rise in coverage means that category counts grow39slightly faster than eligible counts for reasons unrelated to research activity, so growth rates of categories40should be read against the coverage column rather than against the eligible totals of41Table~\ref{tab:eligible_year}; the rise in labels per work means that multi-label counts, which are the input of42the monthly category series, grow faster than primary counts, and that co-classification statistics43(Section~\ref{sec:r-rq5}) are more generous in the post period. The classification run itself processed44\ClassificationWorks{} records, a superset of the eligible corpus that includes the penumbra set and records45later removed by the eligibility filter, of which \ClassificationClassified{} received at least one category46and \ClassificationAssignments{} category assignments were made in total. The validation of the assignment47against two independent raters on a stratified sample is part of the protocol (checkpoint H-2) and its48metrics file was not available when this part was written; Section~\ref{sec:r-validation} is reserved for it.4950\input{tables/tab_p9_classification}5152Four reading rules apply throughout. First, the series end at the last complete month of the panel,53\ItsLastComplete{}, and the yearly tables carry 2026 as an incomplete row; right-truncation affects every54statistic that depends on citations or references more than it affects counts of works, and the fixed-window55citation statistics are complete only for the publication years flagged in Table~\ref{tab:citation_windows}.56Second, the reference date of 30~November 2022 is a historical marker. The interrupted-time-series estimates57are changes in level and slope at that date net of trend and season; they are not treatment effects, and58the alternative-date and placebo analyses exist precisely to show how specific to that month the estimated59changes are \citep{bernal2017its, hausman2018rdit}. Third, the forty category series are analysed with the60same battery of tests, so forty $p$-values are produced for every statistic; at the conventional five per61cent level two of them are expected to fall below the threshold under a global null, and the text reads62counts of significant categories with that expectation in mind rather than applying a formal correction63\citep{benjamini1995fdr}, because the hypotheses were stated for named categories and are tested one by one.64Fourth, the verdict on each hypothesis is one of \emph{supported}, \emph{not supported}, \emph{opposite}65(the data move against the stated direction) or \emph{inconclusive} (the test cannot discriminate), and the66hypotheses are restated in full before each verdict. Interpretive statements about the structural-change67results that go beyond the estimates are marked as pending the principal investigator's review (checkpoint68H-4), as the protocol requires.6970\section{RQ1: the volume and composition of computer-science research}71\label{sec:r-rq1}7273\subsection{Volume}7475The yearly totals of Table~\ref{tab:eligible_year} and Figure~\ref{fig:volume-yearly} describe a corpus that76was almost flat through the baseline and grew afterwards. The eligible count moves from77\VolumeYearlyTwoZeroOneEightN{} works in 2018 to \VolumeYearlyTwoZeroTwoTwoN{} in 2022, a span in which the782020 and 2021 cohorts are the largest and 2022 is slightly below them, and then to79\VolumeYearlyTwoZeroTwoThreeN{} in 2023, \VolumeYearlyTwoZeroTwoFourN{} in 2024 and80\VolumeYearlyTwoZeroTwoFiveN{} in 2025; the 2026 cohort, \VolumeYearlyTwoZeroTwoSixN{} works at the snapshot,81covers ten months of first dates and an indexing lag, and is not comparable. Three features of the82composition columns of Table~\ref{tab:volume_yearly} qualify the growth. The DBLP-indexed count83(\VolumeYearlyTwoZeroOneEightNDblp{} in 2018, \VolumeYearlyTwoZeroTwoFiveNDblp{} in 2025) grows much less than84the total, so that the DBLP share of the corpus falls in the post period; the OpenAlex-primary count85(\VolumeYearlyTwoZeroOneEightNCsPrimary{} to \VolumeYearlyTwoZeroTwoFiveNCsPrimary{}) and especially the86arXiv-linked count (\VolumeYearlyTwoZeroOneEightNArxiv{} to \VolumeYearlyTwoZeroTwoFiveNArxiv{}) grow faster87than the total. The post-period growth of the corpus is therefore concentrated in the parts of it that DBLP's88venue-based curation does not cover, which is consistent both with a real shift of output toward preprints and89toward venues outside DBLP's lists, and with a change in OpenAlex's own coverage over the period90(Section~\ref{sec:net-boundaries} examines the DBLP-only records; Table~\ref{tab:dblp_only}). The two cannot be91separated with the data at hand, and every growth statement of this part should be read as a statement about92the indexed record of computer science rather than about its production.9394The mean number of authors per work rises from \VolumeYearlyTwoZeroOneEightMeanAuthors{} in 2018 to95\VolumeYearlyTwoZeroTwoFiveMeanAuthors{} in 2025, continuing a trend that Part~VIII documents in detail96(Section~\ref{sec:net-teams}); the mean number of declared references per work rises from97\VolumeYearlyTwoZeroOneEightMeanRefs{} to a maximum of \VolumeYearlyTwoZeroTwoOneMeanRefs{} in 2021 and then98falls to \VolumeYearlyTwoZeroTwoFiveMeanRefs{} in 2025 and \VolumeYearlyTwoZeroTwoSixMeanRefs{} in 2026, a99decline that is entirely attributable to reference indexing lag and is the clearest single illustration of the100right-truncation problem in the corpus.101102The monthly series (Figure~\ref{fig:its-total}) carries the same information at the resolution at which the103structural-change tests operate. Its pre-period slope, \ItsSeriesTotalItsChatgptPreSlope{} in log points per104month, corresponds to a growth of about a third of a per cent per month, which compounds to the yearly105totals above; the acceleration visible to the eye after 2022 is quantified in Section~\ref{sec:r-rq2}.106107\subsection{Composition by area and category}108109Figure~\ref{fig:area-shares} and Table~\ref{tab:p9-area-shares} give the composition of the classified corpus by110the eight top-level areas of the ontology, using the primary category of each work. The dominant area in every111year is Perception, Language, Robotics and Interaction, which holds a little over a third of primary112assignments and whose share is stable within two percentage points over the window; it contains Computer113Vision, the largest single category (Table~\ref{tab:p9-category-periods}). The movement is elsewhere. The share114of the AI and Machine Learning area rises in every year of the window, from under a tenth in 2018 to over a115tenth in 2022 and to close to a fifth in 2025; its rise is the mirror image of the fall of Data, Information and116Distributed Systems, whose share declines in every year, and of the smaller declines of Theory of Computation117and of Software, Programming Languages and Formal Methods. Security, Privacy and Trust gains about one and a118half points over the window; Hardware and Computational Science are approximately stable. The period rows of119the table, which split at the reference date rather than at the calendar year, show the same ordering.120121\begin{figure}[htbp]\centering122\includegraphics[width=0.95\textwidth]{fig_area_shares.pdf}123\caption{Share of eligible works by top-level area (primary category) per year. Source:124\texttt{results/graph/category\_counts\_yearly.csv}; 2026 omitted as incomplete.}125\label{fig:area-shares}126\end{figure}127128\input{tables/tab_p9_area_shares}129130Two cautions apply to this picture. The classifier assigns the primary category by the largest score, and a work131on, say, a vision--language model is counted once, under whichever of Computer Vision, Natural Language132Processing, Multimodal AI or Large Language Models scores highest; the area shares of133Table~\ref{tab:p9-area-shares} are therefore sensitive to the tie-breaking of the score, and the multi-label134counts of Table~\ref{tab:p9-category-periods} give a less brittle view of the same movement. Second, the135coverage of the classifier rises over the window (Table~\ref{tab:p9-classification}), and the works that enter136the classified set late are not a random sample of the eligible corpus: they are predominantly works with137abstracts, which the AI categories, defined largely by keyword patterns, pick up more readily than the138venue-defined categories of theory or systems. A part of the rise of the AI area is therefore a rise in the139classifiability of the literature, and the text does not attempt to separate it.140141Table~\ref{tab:p9-category-periods} descends to the forty categories. Sorted by their post-period share, the142table shows a top tier that is unchanged across periods (Computer Vision, Computer Networks, Human--Computer143Interaction, Cybersecurity, Natural Language Processing), a single category whose rank changes by an order144of magnitude (Large Language Models, which moves from under half a per cent of primary assignments in the145baseline to over four per cent in the post period, with a multi-label count more than three times its146baseline value), and a cluster of AI categories with the same profile at smaller scale: Generative AI,147AI Safety and Alignment, Multimodal AI, Foundation Models and Federated Learning all have post-period148multi-label counts above three times their baseline counts. The categories whose multi-label counts are149below their baseline value in the post period are Computer Networks, Computer Architecture, Formal150Verification, Programming Languages and Operating Systems, and the compound annual growth rates of the last151two columns confirm that the fall is a fall of shares more than of volumes: among these, only Computer152Architecture and Cloud Computing have a negative post-period CAGR of primary counts, and the others grow in153absolute terms while losing share. The post-period CAGRs are computed over the two complete years 2023 to1542025 (equation~\eqref{eq:cagr}) and are more volatile than the four-year baseline rates; they also carry the155coverage drift noted above.156157\input{tables/tab_p9_category_periods}158159\begin{figure}[htbp]\centering160\includegraphics[width=0.95\textwidth]{fig_category_growth.pdf}161\caption{Yearly primary-category output indexed to 2018 for the eight fastest- and eight slowest-growing162categories (2018--2025). Source: \texttt{results/graph/category\_counts\_yearly.csv}.}163\label{fig:category-growth}164\end{figure}165166Figure~\ref{fig:category-growth} shows the growth indices of the extremes. The fastest-growing categories are,167without exception, AI categories that were small in 2018 (Foundation Models, Large Language Models, Generative168AI, Multimodal AI, Federated Learning, AI Safety and Alignment, Autonomous Scientific Discovery) together with169Quantum Computing; their indices are large partly because their 2018 bases are small, and the table's shares170are the better measure of their weight. The slowest-growing categories are the venue-defined systems and171theory categories (Operating Systems, Compilers, Computer Architecture, Programming Languages, Formal172Verification, Cloud Computing), whose indices stay near or below their 2018 value until 2023 and rise modestly173afterwards; these are also the categories whose DBLP coverage is highest, so their flatness is the flatness of174the DBLP-indexed count noted above.175176\subsection{Hypothesis H2: composition}177\label{sec:r-h2}178179H2 states that the share of the AI-related categories (categories 1--8 and 38 of the mandate, which form the AI180and Machine Learning area of the ontology) in computer-science output increases in the post period, and that181the share of at least one non-AI area (theory, systems, networks) declines in relative terms while growing in182absolute terms. Table~\ref{tab:p9-ai-shares} tests the first clause with the bootstrap intervals of183Part~IV, obtained by resampling the works of each year with replacement \citep{efron1979bootstrap}. The area's184share rises from 2018 to 2022 and from 2022 to 2025, but the rise is not uniform across its nine categories.185Seven of them (Large Language Models, Foundation Models, Multimodal AI, Generative AI, AI Agents and186Autonomous Systems, Reinforcement Learning, AI Safety and Alignment) have a 2025 interval that lies entirely187above their 2022 interval; Machine Learning Theory has overlapping intervals in the two years; and Deep188Learning Architectures has a 2025 interval entirely \emph{below} its 2022 interval, its share having peaked189in the baseline. AI Agents and Autonomous Systems is the one category whose 2025 share remains below its 2018190share, a consequence of the category's definition (it includes the pre-LLM agent and multi-agent literature,191whose share was declining through the baseline) rather than evidence against the hypothesis. The Large192Language Models category accounts for the greater part of the area's post-period gain, and the hypothesis is193better described as supported for the generative and language-model categories than for machine learning as194a whole. The intervals are narrow because the195resampling treats the classifier as fixed; the uncertainty that matters is the classifier's, not the196sampling's, and the intervals should be read as a lower bound on it.197198\input{tables/tab_p9_ai_shares}199200The second clause is tested by Table~\ref{tab:p9-area-shares} together with the CAGR columns of201Table~\ref{tab:p9-category-periods}. The Data, Information and Distributed Systems area (which contains202Computer Networks) and the Theory of Computation area both lose share in every year of the window, and their203primary counts in 2025 exceed their 2022 counts, so both decline relatively while growing absolutely; the204Software, Programming Languages and Formal Methods area shows the same pattern. The second clause is therefore205met by three areas rather than by one. The verdict on H2 is \emph{supported} on both clauses, with the206qualification that the relative decline of the non-AI areas begins before the reference date (it is visible in207every year from 2018) and continues through it without a visible change of pace in the yearly data; the208monthly analysis of Section~\ref{sec:r-rq2} examines whether a change of pace exists at monthly resolution.209Table~\ref{tab:diversity} adds the diversity indices of the category mix: the Rao--Stirling index and the210Leinster--Cobbold index of order two are both lower in 2023--2025 than in 2018--2021, so the concentration of211primary assignments on fewer, mutually similar categories increased slightly, which is what the rise of a212cluster of related AI categories implies. The similarity matrix behind these indices is the co-classification213Jaccard similarity rather than the semantic one, because the semantic category similarities were not yet214available (\pending{semantic atlas: results/semantic/category\_similarity\_baseline.csv and \_post.csv; the215diversity indices should be re-estimated with the semantic distance when they exist}).216217\section{RQ2: acceleration and structural change}218\label{sec:r-rq2}219220\subsection{The total series at the reference date}221\label{sec:r-its-total}222223Figure~\ref{fig:its-total} shows the monthly series of month-precision works from \ItsFirst{} to224\ItsLastComplete{} with the fitted values of the interrupted time series of equation~\eqref{eq:its}, and225Table~\ref{tab:p9-its-total-events} reports the estimates at the reference date and at the four alternative226dates. At December 2022 the estimated change in log level is \ItsSeriesTotalItsChatgptLevelChange{}, a value227that is small, negative and indistinguishable from zero (HAC $p$ = \ItsSeriesTotalItsChatgptLevelP{}); the228estimated change in monthly log slope is \ItsSeriesTotalItsChatgptSlopeChange{}, which, added to the229pre-period slope of \ItsSeriesTotalItsChatgptPreSlope{}, implies a post-period growth rate of roughly one and230four tenths of a per cent per month against roughly three tenths before, a more than fourfold increase in the231rate of growth. The HAC $p$-value of the slope change rounds to \ItsSeriesTotalItsChatgptSlopeP{}, and the fit232explains \ItsSeriesTotalItsChatgptRTwo{} of the variance of the log series over233\ItsSeriesTotalNMonths{} months. The estimates describe a kink, not a jump: the series did not move to a higher234level in December 2022, it began, at some point, to grow faster.235236\begin{figure}[htbp]\centering237\includegraphics[width=0.95\textwidth]{fig_its_total.pdf}238\caption{Monthly output (month-precision works) with the interrupted-time-series fit at December 2022; the239annotation gives the level and slope changes with their HAC and placebo $p$-values. Sources:240\texttt{results/econometrics/fit\_total.csv}, \texttt{its\_results.json}.}241\label{fig:its-total}242\end{figure}243244\input{tables/tab_p9_its_total_events}245246Whether ``some point'' is the reference date is the question the remaining rows of247Table~\ref{tab:p9-its-total-events} and the diagnostics of Table~\ref{tab:p9-total-robustness} address. Moving248the event to May 2022 (DALL$\cdot$E~2) or April 2023 (GPT-4) changes the slope estimate by less than a tenth of249its value; moving it to June 2020 (GPT-3) halves it but leaves it clearly positive, and moving it to October2502024 (o1) gives a comparable slope change on top of a steeper pre-period. The level estimates vary in sign251across the five dates and two of them (DALL$\cdot$E~2, o1) have HAC $p$-values below one per cent, with252opposite signs. The honest summary of the five columns is that the series exhibits a sustained acceleration253over 2022--2024 that any event date placed within that span will pick up as a slope change, and that no254single month within it shows a level shift that the others do not.255256\input{tables/tab_p9_total_robustness}257258The placebo distribution (Figure~\ref{fig:placebo-total}) makes this precise. Over the259\ItsSeriesTotalPlaceboChatgptNPlacebo{} candidate event months, the level $t$-statistic of December 2022 ranks260\ItsSeriesTotalPlaceboChatgptLevelTRank{}th in absolute value (placebo $p$ =261\ItsSeriesTotalPlaceboChatgptLevelTPlaceboP{}), that is, almost every other month of the series produces a262larger apparent level shift than the reference date does; the slope $t$-statistic ranks first (placebo $p$ =263\ItsSeriesTotalPlaceboChatgptSlopeTPlaceboP{}, the smallest attainable value being one over the number of264candidates). Restricted to the baseline months, the placebo $p$ of the slope is265\ItsSeriesTotalPlaceboChatgptBaselineOnlySlopeTPlaceboP{}: no month before the reference date yields a slope266change as large as the one estimated there, which is the pre-trend diagnostic the protocol asked for and is267the same statement as the first rank, read from the other side. The donut specification, which drops the268event month and three months on each side, moves the level estimate to269\ItsSeriesTotalItsChatgptDonutLevelChange{} (HAC $p$ = \ItsSeriesTotalItsChatgptDonutLevelP{}) and leaves the270slope change essentially unchanged; the donut level estimate is larger than the full-sample one because the271months it removes, the winter of 2022--2023, lie below the post-period trend line, so that the trend fitted272without them extrapolates to a higher December 2022 value. The sensitivity rows show that ending the series in273March or August 2026 instead of June changes the slope estimate in the third decimal and leaves the level274estimate within a hundredth of zero, of unstable sign and insignificant.275276\begin{figure}[htbp]\centering277\includegraphics[width=0.95\textwidth]{fig_placebo_total.pdf}278\caption{Placebo distribution of the ITS $t$-statistics (level and slope) over every candidate event month of279the total series; the true event month is highlighted. Source: \texttt{results/econometrics/placebo\_total.csv}.}280\label{fig:placebo-total}281\end{figure}282283The data-driven methods date the kink earlier than the reference date. The Bai--Perron procedure selects a284single break under both criteria ($m$ = \ItsSeriesTotalBaiPerronMBic{} by BIC, and the same number by LWZ; the285BIC falls from \ItsSeriesTotalBaiPerronBicByMZero{} with no break to \ItsSeriesTotalBaiPerronBicByMOne{} with286one and rises again for two and three), and places it in April 2022, eight months before the reference date;287with the series ended in August 2026 the date moves to February 2022, so the location is not sharp. The288trimming parameter $h$ = \ItsSeriesTotalBaiPerronH{} months forbids a second break after April 2025 and within289fifteen months of the first, so a second kink in late 2024 or 2025 would not be found by this procedure. PELT,290which fits a piecewise-constant mean rather than a trend and therefore flags the points where the level291moves most, places changepoints in May 2023 and December 2024; the first is the point at which the series,292growing at the post-period rate, has risen far enough above its baseline mean for a step to be the cheapest293description, and it should not be read as an independent dating of the acceleration. The Bayesian online294detector assigns a maximum posterior probability of295\ItsSeriesTotalBocpdCpProbTwoZeroTwoTwoOneTwoToTwoZeroTwoThreeZeroThree{} to a changepoint within the four296months preceding any month of December 2022 to March 2023, and its maximum-a-posteriori run length never297resets over the whole series: an observer following the monthly totals with this detector would not have298concluded, at any point, that the process had changed. The reconciliation of the four methods is that the299acceleration is gradual at monthly resolution, which a trend model with one kink summarises well and a300step model or an online detector with a constant-mean likelihood summarises poorly.301302\subsection{Hypothesis H1: volume}303\label{sec:r-h1}304305H1 states that the monthly growth rate of computer-science works is higher in the post period than in the306baseline, but that the level shift at November 2022 is not distinguishable from placebo breaks in the baseline307once seasonality and indexing lag are modelled; the protocol adds that growth differences are expected to be308concentrated in a few categories rather than uniform. On the total series the first clause is309\emph{supported}: the slope change is positive, large relative to the pre-period slope, robust to the donut310specification and to the truncation month, and ranks first among all candidate dates and above every311baseline date in the placebo distribution. The second clause is also \emph{supported}: the level change at312the reference date is small, of uncertain sign across specifications, and ranks near the bottom of the313placebo distribution. The clause about concentration is examined in Section~\ref{sec:r-its-categories} and is314\emph{not supported} in its stated form, because the slope change is positive and conventionally significant315in nearly every category, although its magnitude varies by an order of magnitude across them.316317Two qualifications are part of the verdict. The first is that the acceleration predates the reference date in318the unknown-break estimates (spring 2022), so H1's first clause is supported in the weaker sense that the319post period grows faster, not in the sense that the growth rate changed at the boundary; a reader who takes320the Bai--Perron date at face value would describe a change that began during the baseline and was already321under way when ChatGPT was released. \pending{H-4: whether to describe the aggregate acceleration as beginning322in spring 2022 (Bai--Perron) or as a 2022--2024 phenomenon without a sharp onset (placebo and alternative323dates)}. The second is that the indexing-lag model the protocol envisaged was replaced by truncation324(Section~\ref{sec:m-truncation}); the hypothesis is therefore tested conditional on the series ending at the325last complete month, and the sensitivity rows of Table~\ref{tab:p9-total-robustness} are the only evidence on326what a different truncation would do. With about a hundred observations and a break near the middle of the327sample, the design has ample power for a slope change of this size and little for a level shift of a few per328cent \citep{zhang2011itspower}, so the null result on the level should be read as ``not detectable'' rather329than ``absent''.330331\subsection{The category series}332\label{sec:r-its-categories}333334The same battery is applied to the forty multi-label category series. Figure~\ref{fig:breaks-by-category}335orders the categories by their estimated slope change at December 2022; Tables~\ref{tab:p9-its-slopes}336and~\ref{tab:p9-its-levels} give the slope and level estimates with their HAC and placebo $p$-values, and337Table~\ref{tab:p9-unknown-breaks} the data-driven dates. Table~\ref{tab:breaks} in Appendix~F carries the338same estimates in the layout of the result file.339340\begin{figure}[htbp]\centering341\includegraphics[width=0.95\textwidth]{fig_breaks_by_category.pdf}342\caption{ITS slope change at December 2022 by category, with the level change annotated; bars in colour have a343placebo $p$ below 0.05 for the slope. Source: \texttt{results/econometrics/breaks.csv}.}344\label{fig:breaks-by-category}345\end{figure}346347\input{tables/tab_p9_its_slopes}348349The slope changes are positive in thirty-seven of the forty categories and negative in three (Blockchain350Systems, Deep Learning Architectures, Federated Learning), and the HAC $p$-value is below five per cent in351all but the first two of these; by the conventional criterion almost every category accelerated. The352placebo column tells a different story. Only one category, Applied Cryptography, has a placebo $p$ for the353slope below five per cent, and with forty series two such values are expected under a global null; the354total series is the only series whose December 2022 slope change ranks first among candidate dates. The355reason is visible in the pre-slope column: the category series are noisier than the total, their placebo356distributions are wider, and a slope change that is large in absolute terms, as in Large Language Models357(\ItsSeriesCatLlmItsChatgptSlopeChange{} per month on a pre-period slope of358\ItsSeriesCatLlmItsChatgptPreSlope{}, HAC $p$ rounding to \ItsSeriesCatLlmItsChatgptSlopeP{}), ranks only359\ItsSeriesCatLlmPlaceboChatgptSlopeTRank{}th of \ItsSeriesCatLlmPlaceboChatgptNPlacebo{} candidate dates360(placebo $p$ = \ItsSeriesCatLlmPlaceboChatgptSlopeTPlaceboP{}) because the series has accelerated at many361dates. The placebo rank over all candidate dates is the appropriate measure of whether December 2022 is special362for a given category, and by that measure it is not special for any of them. The baseline-only rank, which363asks the weaker question of whether any month \emph{before} the reference date produced a larger slope364change, answers no for a substantial minority of categories (the zero entries of the last placebo column,365which include AI Agents, Algorithms, Computational Biology and Autonomous Scientific Discovery): their366post-period acceleration exceeds anything in their baseline, even though it does not exceed what other367post-period dates would show.368369The ordering of the slope changes is nonetheless informative about where the acceleration is largest. The370five largest changes belong to AI Agents and Autonomous Systems, Large Language Models, Multimodal AI,371Information Retrieval and Distributed Computing; Generative AI, Cloud Computing, Natural Language Processing,372Quantum Computing and Compilers follow. The presence of Information Retrieval, Distributed Computing, Cloud373Computing and Compilers in the upper part of the ranking is a finding that the hypotheses did not anticipate:374these are categories that the LLM literature draws on (retrieval, serving infrastructure, compilation of375models to accelerators) and whose multi-label counts grow because works on language models carry their labels376(Section~\ref{sec:r-rq5}). At the other end, the categories with the smallest positive changes are Edge377Computing, Robotics, Computational Biology, Foundation Models and Computer Vision, all five with pre-period378slopes above the median of the forty, so that a continuation of their growth registers as a small change;379Foundation Models in particular has one of the three steepest pre-period slopes of all categories and a380post-period slope that is only slightly steeper. The three negative changes are of the same kind: Federated381Learning had the steepest baseline slope of all categories and Deep Learning Architectures one of the382steeper ones, and both decelerate to rates that remain positive; Blockchain Systems decelerates from a3832018--2020 boom.384385\input{tables/tab_p9_its_levels}386387The level changes (Table~\ref{tab:p9-its-levels}) are mostly negative, which is the signature of a trend model388fitted through a kink: when growth accelerates gradually from mid-2022, a regression with a kink fixed at389December 2022 absorbs the months between the true onset and the fixed date into a negative intercept shift.390Fifteen categories have a level change with a HAC $p$ below five per cent, of which only Human--Computer391Interaction, Computer Networks and Sustainable Computing have a positive one; none has a placebo $p$ for the392level below five per cent, and the baseline-only placebo $p$ of the level exceeds a quarter in every393category. The donut estimates are in most cases positive where the full-sample ones are negative, for the394reason given for the total series, and are largest in Large Language Models395(\ItsSeriesCatLlmItsChatgptDonutLevelChange{} with HAC $p$ rounding to396\ItsSeriesCatLlmItsChatgptDonutLevelP{}), Multimodal AI and Generative AI, the three series in which the397winter of 2022--2023 lies furthest below the subsequent trend. The BOCPD column confirms the absence of an398online-detectable change: the maximum four-month changepoint probability over December 2022 to March 2023399exceeds one quarter only in Cloud Computing and Operating Systems, two small and noisy series, and the400detector resets its run length in that window for no category.401402\input{tables/tab_p9_unknown_breaks}403404The data-driven dates of Table~\ref{tab:p9-unknown-breaks} spread over the window. The Bai--Perron dates405cluster in 2022 (February to August) for a group that includes Large Language Models,406Natural Language Processing, Multimodal AI, Computer Vision, Cybersecurity, Applied Cryptography, Software407Engineering, Sustainable Computing and Privacy-Preserving Computation, and in 2023 (April to November) for a408group that includes Algorithms, Computational Complexity, Computer Architecture, Compilers, Database Systems,409Operating Systems and Speech and Audio Computing; several categories have a second break in mid-2024 and a410few (Blockchain, Federated Learning, Computational Biology, Autonomous Scientific Discovery) have their first411break in 2020. The pre-specified H3 window of October 2022 to June 2023 contains a BIC break date for seven412categories, listed in the table's penultimate column, and a PELT changepoint for twelve. The temporal413pattern that emerges is of two waves: an earlier kink, in 2022, in the categories closest to the generative414model literature and in the large applied categories, and a later kink, in 2023, in the venue-defined415theory and systems categories, whose flat baseline makes a later acceleration easier to date. This reading is416descriptive and is marked for review (\pending{H-4: two-wave reading of the Bai--Perron dates}).417418\input{tables/tab_p9_altdates}419420Table~\ref{tab:p9-altdates} completes the picture with the level change at each of the five pre-specified421dates. For the AI categories the o1 column (October 2024) carries the largest positive level changes of any422date, which is the second, 2024 kink seen by the Bai--Perron procedure in a different form; the GPT-3 column423(June 2020) carries negative level changes for most categories, which is the pandemic-year irregularity of424the series rather than anything specific to the event. The five columns are the same series with the break425moved, and the table is read across a row as a profile of when a category's level departs from its fitted426trend, not as five tests.427428\subsection{Hypothesis H3: structural breaks}429\label{sec:r-h3}430431H3 states that Bai--Perron tests on category-level monthly series locate at least one break in the window432from October 2022 to June 2023 for Large Language Models, Generative AI, AI Agents and AI Safety, and that433for Formal Verification, Computational Complexity, Operating Systems and Computer Architecture no break is434detected in that window at the five per cent level after placebo correction. Table~\ref{tab:p9-h3} assembles435the evidence for the eight named categories.436437\input{tables/tab_p9_h3}438439The first clause is \emph{not supported}. None of the four AI categories has a Bai--Perron break in the440window: Large Language Models breaks in July 2022, Generative AI in November 2021, AI Agents in July 2022441(with a second break in July 2024) and AI Safety in June 2021 and September 2022. All four break442\emph{before} the window, which is the opposite of the error the hypothesis guarded against (a break too late443to be attributed to the boundary): their series accelerated during 2022, in the months of the diffusion-model444and instruction-tuning literature, and the reference date falls inside an acceleration that had already445begun. The PELT dates of these categories fall in late 2023, after the window, for the step-model reason446given above. The second clause is \emph{opposite} for two of its four categories: Computational Complexity447and Computer Architecture have Bai--Perron breaks in April and May 2023, inside the window, and Formal448Verification and Operating Systems break outside it (February 2022 and July 2024; September 2023). The449``five per cent level after placebo correction'' part of the clause cannot be applied as written, because450the Bai--Perron dates carry no $p$-value in the implementation (Section~\ref{sec:m-bp}); the placebo451statistic that exists is the ITS slope placebo, which exceeds five per cent for all eight categories, so that452by that statistic no category, including the four AI categories, has a December 2022 change that stands out453from its own history. The hypothesis, in short, predicted that the boundary would separate categories with454and without a break, and the data show breaks on both sides of the boundary in both groups, earlier in the455AI categories and later in the theory and systems categories. The verdict is \emph{not supported} for the456first clause and \emph{opposite} for the second, with the uncertainty that the Bai--Perron dates have no457sampling distribution attached and move by several months when the truncation month changes458(Table~\ref{tab:p9-total-robustness} for the total series). \pending{H-4: interpretation of the pre-window459breaks of the AI categories}.460461\section{RQ3: influential contributions and the distribution of attention}462\label{sec:r-rq3}463464RQ3 asks which publications introduced the most influential methodological or theoretical contributions. The465quantitative part of the answer is the citation analysis of Part~VIII and the landmark set of Part~IV; the466qualitative part, the identification of what each landmark contributed, belongs to the specialist reviews of467Parts~V--VII and to checkpoint H-3, and this section reports only what the counts show.468469\subsection{Fixed-window citation statistics}470471Table~\ref{tab:citation_windows} and Figure~\ref{fig:citation_inequality} give the fixed-window statistics by472publication year. For the cohorts with a complete three-year window (2018 to 2023), the mean number of473citations received within the window rises from \CitationWindowsYearlyTwoZeroOneEightMeanCThreeSix{} for the4742018 cohort to \CitationWindowsYearlyTwoZeroTwoTwoMeanCThreeSix{} for 2022 and475\CitationWindowsYearlyTwoZeroTwoThreeMeanCThreeSix{} for 2023, while the share of works uncited after three476years falls from \CitationWindowsYearlyTwoZeroOneEightShareUncitedThreeSix{} to477\CitationWindowsYearlyTwoZeroTwoTwoShareUncitedThreeSix{} and rises again to478\CitationWindowsYearlyTwoZeroTwoThreeShareUncitedThreeSix{} for the 2023 cohort, whose window closes at the479end of 2025 and is therefore the first to feel the reference-indexing lag. The Gini coefficient of three-year480citations, the inequality statistic of hypothesis H5, falls monotonically through the baseline cohorts, from481\CitationWindowsYearlyTwoZeroOneEightGiniCThreeSix{} (2018) to482\CitationWindowsYearlyTwoZeroTwoTwoGiniCThreeSix{} (2022), and rises to483\CitationWindowsYearlyTwoZeroTwoThreeGiniCThreeSix{} for the 2023 cohort. The baseline decline is driven by the484falling share of uncited works, which is the largest contributor to a Gini of this kind485\citep{nielsen2021inequality}; the 2023 reversal coincides with the return of the uncited share and is, on the486evidence available, a truncation effect rather than a change in the concentration of attention. The 99th487percentile of three-year citations is almost flat across the complete cohorts (Table~\ref{tab:citation_windows}),488so the top of the distribution did not pull away from the body within fixed windows.489490The field-weighted citation impact supplied by OpenAlex, whose normalisation is the database's own and not491the study's (Section~\ref{sec:m-citations}), has a mean near \CitationWindowsYearlyTwoZeroOneEightMeanFwci{}492in 2018 and \CitationWindowsYearlyTwoZeroTwoThreeMeanFwci{} in 2023 and a median that rises from493\CitationWindowsYearlyTwoZeroOneEightMedFwci{} to \CitationWindowsYearlyTwoZeroTwoTwoMedFwci{} over the494baseline; a mean well above one for a field whose works are normalised by field means that the495computer-science corpus is cited above the average of the OpenAlex fields its works are assigned to, and496the gap between mean and median is the skewness that the Gini measures.497498\subsection{The most cited works and the landmark set}499500Table~\ref{tab:top_cited} lists the ten most cited works of each publication year by three-year window501citations, and Section~\ref{sec:net-citations} of Part~VIII discusses it. The list contains the works that a502reader of the period would expect at the top of computer science (contextualised word representations,503BERT, the Vision Transformer, the Swin Transformer, latent diffusion, masked autoencoders, the scientific504Python stack) alongside records that are duplicates of canonical works misdated to a later year (the505transformer paper under 2025, the batch-normalisation paper under 2024, the original GAN paper under 2023,506the Bitcoin white paper under 2019 and 2022) and records whose three-year count exceeds their lifetime507count or whose lifetime count is zero, which are database errors. The ranking of influential works by raw508citations is therefore conditional on an audit that Part~VIII specifies and that has not been performed,509and this part does not derive a list of influential contributions from it.510511The landmark set is the study's instrument for that purpose. Selected by field-normalised impact within512category and year with per-area quotas (Section~\ref{sec:m-kg}), it contains \SelectionLandmarks{} works513drawn from a high-priority set of \SelectionHighPriority{}; Table~\ref{tab:p9-landmarks} gives its514composition by area and year. The quotas make the set broader than a citation ranking would be: the AI and515Machine Learning area holds the largest quota but less than a quarter of the set, and the theory area,516whose works are cited least, holds the smallest. By year the set is smallest for the 2021 cohort and largest517for 2023, 2024 and 2026; the size of the 2026 cohort reflects the selection's reliance on the database's518field-weighted impact for the most recent works, where citation counts are smallest and the normalisation519least stable, and the 2026 landmarks should be treated as provisional. \SelectionShareWithAbstractLandmarks{} of the landmarks have an520abstract and \SelectionShareOaLandmarks{} have an open-access location, which bounds what the knowledge-graph521extraction and the specialist reviews can work from. The identification of the methodological or522theoretical contribution of each landmark is the object of the specialist reviews and of checkpoint H-3523and is not reported here (\pending{H-3: landmark list and specialist reviews}).524525\input{tables/tab_p9_landmarks}526527\subsection{Hypothesis H5: concentration}528\label{sec:r-h5}529530H5 states that citation inequality (the Gini of 24-month citations within year) is higher in the post period531than in the baseline for AI categories, and that venue concentration (HHI by source) is stable or lower532because of preprint growth. The citation component is stated for the AI categories, and the per-category533citation-window statistics are not among the result files; the verdict on that component is534\emph{inconclusive} (\pending{macro: per-category Gini of fixed-window citations, results/bibliometrics;535not produced by the current pipeline}). The aggregate counterpart, read from Table~\ref{tab:citation_windows}536with the 36-month Gini that the pipeline computes in place of the 24-month one, falls through the baseline537and rises for the single complete post-period cohort, a movement that Section~\ref{sec:net-citations}538attributes to the uncited share and to right truncation; it is not evidence of higher concentration. The539venue component is examined in Section~\ref{sec:net-venues}: the Herfindahl index with arXiv counted as a540source rises from \VenueHhiYearlyTwoZeroOneEightHhi{} in 2018 to \VenueHhiYearlyTwoZeroTwoFiveHhi{} in 2025541and the top-source share from \VenueHhiYearlyTwoZeroOneEightTopOneShare{} to542\VenueHhiYearlyTwoZeroTwoFiveTopOneShare{}, because arXiv is the top source and its share grows; the543protocol anticipated this mechanism and predicted stability or a fall once preprints are treated separately,544and the index without the preprint type is not among the result files. The venue component is therefore545\emph{opposite} on the index as computed and \emph{inconclusive} on the index the hypothesis intended546(\pending{macro: venue HHI excluding repository sources}). The number of distinct sources grows in every547year (Table~\ref{tab:venue_hhi}), so the non-arXiv venue landscape widens rather than concentrates.548549\section{RQ4: citation networks and knowledge flows}550\label{sec:r-rq4}551552\subsection{Flows between fields}553554The part of RQ4 that concerns the flow of knowledge between computer science and other fields is answered in555Section~\ref{sec:net-flows} of Part~VIII from the resolved references of the eligible works556(Table~\ref{tab:reference_coverage}, Table~\ref{tab:field_flows}, Figure~\ref{fig:field_inflows}). The share557of references that stay inside the corpus rises through the window as the corpus itself grows, and the share558pointing to the largest other OpenAlex fields (medicine, engineering, mathematics, physics, the social559sciences) falls, with no change of pace at the boundary; the age distribution of cited works shifts toward560younger work in every year (Table~\ref{tab:cited_age}, Figure~\ref{fig:cited_age}), again without a visible561inflection. Those results are not repeated here. This section reports the flows \emph{within} computer562science, between the areas and categories of the ontology, which depend on the classification and were563not available to Part~VIII.564565\subsection{Flows between areas}566567Figure~\ref{fig:area-flow-matrix} shows the $8 \times 8$ area flow matrices of equation~\eqref{eq:flow},568row-normalised, for the baseline and post spans, and Table~\ref{tab:p9-area-flow-change} gives their569difference in percentage points. The diagonal dominates both matrices: in every area a majority of internal570references stay within the area, from about three fifths in Security, Privacy and Trust to well over four571fifths in Perception, Language, Robotics and Interaction, which cites itself more than any other area does572and whose self-share does not fall in the post period. The off-diagonal structure is a hub-and-spoke pattern573centred on two areas: every area sends a larger share of its references to AI and Machine Learning in the574post span than in the baseline (by between one and two and a half points for the systems, security,575software, theory and computational-science areas), and the AI area itself sends a larger share to576Perception, Language, Robotics and Interaction and a smaller share to itself. The largest movements in the577table are the falls of the self-shares of the AI, Data, Software and Security areas, each of about three578points, matched by rises of their references to the AI area and, for the AI area itself, to the vision and579language categories. The Hardware area is the exception that moves least: its row changes by less580than a point in every cell.581582\begin{figure}[htbp]\centering583\includegraphics[width=0.95\textwidth]{fig_area_flow_matrix.pdf}584\caption{Citation-flow matrices between areas (row shares of internal references, citing area in rows) for585the baseline and post spans. Source: \texttt{results/graph/area\_flows\_yearly.csv}.}586\label{fig:area-flow-matrix}587\end{figure}588589\input{tables/tab_p9_area_flow_change}590591The matrices are built from internal references only, that is, references whose cited work is in the corpus592and has a primary category, and the number of such references grows by an order of magnitude over the window593(Table~\ref{tab:p9-modularity}, second column) because works of 2018 could cite few corpus works, all of them594from 2018, whereas works of 2025 can cite seven years of the corpus. The 2018 matrix is therefore computed on a595small and peculiar set, and the comparison of spans is best read from 2019 onward.596597\subsection{Cross-area shares and modularity}598599Table~\ref{tab:p9-cross-area} and Figure~\ref{fig:cross-area} report the statistic behind H4, the cross-area600share of equation~\eqref{eq:cross-area}, by citing area and year. The areas differ by a factor of three in601their propensity to cite outside themselves: the AI area sends about half of its internal references to other602areas (most of them to vision and language), the perception area about a sixth, and the remaining six603between a quarter and two fifths. Over time the AI area's cross-area share rises through the baseline, from6042018 to 2022, and is flat from 2022 onward; the Data, Security, Software and Theory areas rise slowly and605steadily through the whole window; the Hardware area is flat; and the Perception area falls from 2018 to6062019 and is flat thereafter. The aggregate share, in the penultimate rows of the table, rises from 2022 to6072025 after being flat through the baseline. Part of that rise is composition: references made by AI-area608works, which are the most boundary-crossing, form a growing share of all references. The composition-adjusted609aggregate, which holds each area's weight at its baseline share of internal references, rises by less than610the raw one over 2022--2025 but still rises, so the boundary-crossing propensity of the areas themselves611increased slightly in the post period, by an amount that is about half of the raw increase.612613\begin{figure}[htbp]\centering614\includegraphics[width=0.95\textwidth]{fig_cross_area.pdf}615\caption{Cross-area reference shares by citing area and year, and modularity of the category citation-flow616network at the category and area partitions. Sources: \texttt{results/graph/cross\_area\_share\_yearly.csv},617\texttt{modularity\_yearly.csv}.}618\label{fig:cross-area}619\end{figure}620621\input{tables/tab_p9_cross_area}622623The modularity of the yearly flow network (Table~\ref{tab:p9-modularity}) summarises the same information at624the two partitions. $Q_{\mathrm{area}}$ falls in every year from 2018 to 2024 and is flat in 2025, with the625largest single-year falls in 2020--2021 and in 2023; $Q_{\mathrm{cat}}$ is flat through 2022 and falls626afterwards; and the uncorrected self-share of citations within the citing category is flat through the627baseline and falls from 2023. The category partition and the area partition therefore tell slightly628different stories: the assortativity of citations by \emph{area} declined throughout the window, whereas629the assortativity by \emph{category} held until the reference year and declined afterwards, which is the630signature of categories within the same area (Large Language Models and Natural Language Processing, Computer631Vision and Multimodal AI) citing each other more in the post period. Because the partition is fixed by the632classifier rather than optimised, these values are descriptive indices of assortativity and not community633structure, and their decline is consistent with both a real blurring of category boundaries and a drift of634the classifier's own boundaries as the vocabulary of the literature changed (Section~\ref{sec:m-networks}).635636\input{tables/tab_p9_modularity}637638\subsection{Hypothesis H4: convergence}639\label{sec:r-h4}640641H4 states that the share of citations crossing area boundaries (systems to AI, security to AI, HCI to AI,642and so on) rises in the post period relative to the baseline, and that the modularity of the citation-flow643network by area decreases. Both clauses are \emph{supported} in direction. The cross-area shares of the644systems, security, software and theory areas rise in the post period, and the specific flows the hypothesis645names, from each of these areas toward AI and Machine Learning, are the cells of646Table~\ref{tab:p9-area-flow-change} with the largest positive changes; the area modularity is lower in every647post-period year than in any baseline year. The support is qualified in three ways. The rises are small,648of the order of two to four percentage points over the window for the areas concerned, and they continue649trends that are visible within the baseline; the aggregate increase is about half composition and half650behaviour; and the Perception area, which is the largest and contains the HCI category named in the651hypothesis, does not increase its cross-area share at all, so that ``HCI to AI'' is not supported at the area652level at which the statistic is computed (the category-level flows of Table~\ref{tab:p9-llm-partners} and of653the co-classification analysis in Section~\ref{sec:r-rq5} show HCI among the categories most often paired with654Large Language Models, which is a different measurement). The modularity decline is monotone across the655boundary rather than concentrated at it, so the hypothesis is supported as a statement about the post period656relative to the baseline and not as a statement about a change at the reference date. The classifier657caveat applies with full force: a change in how the classifier partitions the literature would produce the658same movements, and the semantic-level check of H4 is pending (Section~\ref{sec:r-rq5}).659660\section{RQ5: foundation models and the boundaries between disciplines}661\label{sec:r-rq5}662663RQ5 asks whether the emergence of foundation models changed the relationship between previously distinct664computer-science disciplines. Three measurements bear on it: the citation flows of Section~\ref{sec:r-rq4},665the co-classification of works across categories, and the semantic distances between category centroids in666the embedding space. The first was reported above; the second is reported here; the third depends on the667semantic atlas, which was not available when this part was written, and its subsections are reserved.668669\subsection{Co-classification}670671Table~\ref{tab:p9-cooccurrence} lists the fifteen most frequent pairs of categories carried jointly by a672work in its label set, for the baseline and post spans. The baseline list is dominated by pairs that673describe the standing structure of the field: Computer Vision with Deep Learning Architectures, Computer674Networks with Cybersecurity and with Edge Computing, Applied Cryptography with Cybersecurity, Computer675Architecture with Distributed Computing, Cloud with Distributed Computing. Large Language Models with676Natural Language Processing is the third pair of the baseline list, which is a reminder that the category677is defined by a vocabulary (language models, pre-training, transformers) that predates ChatGPT by several678years. The post-period list keeps the vision and networking pairs at the top but is otherwise reorganised679around the generative-model categories: Large Language Models with Natural Language Processing moves to680second place with more than twice its baseline count; Computer Vision pairs with Foundation Models, with681Multimodal AI and with Generative AI; Deep Learning Architectures pairs with Foundation Models and with682Natural Language Processing; and Foundation Models with Large Language Models enters the list. Six of the683fifteen post-period pairs involve a category that did not appear in any baseline pair.684685\input{tables/tab_p9_cooccurrence}686687Table~\ref{tab:p9-llm-partners} looks at the same phenomenon from the side of the Large Language Models688category. Its partners in the post span are, after Natural Language Processing, Foundation Models, Deep689Learning Architectures, Computer Vision, Multimodal AI, Information Retrieval, Software Engineering, AI690Safety and Alignment, Machine Learning Theory, AI Agents, Generative AI, Cybersecurity and Human--Computer691Interaction; the ratio column shows that the pairings with Information Retrieval, Software Engineering,692Cybersecurity and Human--Computer Interaction, which are categories outside the AI area, grew by factors693comparable to or larger than the pairing with Natural Language Processing. These are the category-level694counterparts of the area flows of Section~\ref{sec:r-rq4}: works that carry the LLM label increasingly also695carry the label of an application or systems category, which is one mechanism by which the multi-label696series of those categories accelerated (Section~\ref{sec:r-its-categories}). The measurement is conditional697on the classifier in the strongest possible way, since co-classification is a property of the label set, and698on abstract availability, since a work without an abstract rarely carries more than one label; both rise in699the post period (Table~\ref{tab:p9-classification}), and part of the growth of every pair is attributable to700them.701702\input{tables/tab_p9_llm_partners}703704\subsection{Semantic distances between categories}705\label{sec:r-semantic}706707The protocol's semantic test of convergence compares the cosine similarity between category centroids in708the embedding space, estimated separately on baseline and post works, and reports the drift of each709category's centroid between the two periods (Section~\ref{sec:m-semantic}). The result files710\texttt{results/semantic/}\allowbreak\texttt{category\_similarity\_baseline.csv},711\texttt{category\_similarity\_post.csv} and \texttt{category\_drift.csv} did not exist when this part was written. \pending{semantic atlas: insert the712category similarity comparison (baseline versus post, mean off-diagonal similarity, the pairs whose713similarity changed most) and the drift table when results/semantic/ is produced; the diversity indices of714Table~\ref{tab:diversity} are to be re-estimated with the semantic distance at the same time}.715716\subsection{Semantic clusters and their stability}717718The clustering of the embedded corpus into topics and the bootstrap stability of that clustering719(\texttt{results/semantic/clusters.csv}, \texttt{stability.json}) are likewise pending. \pending{semantic720atlas: number of clusters, share of works in noise, adjusted Rand index of the bootstrap replicates, and the721clusters that are new in the post period, from results/semantic/clusters.csv and stability.json}.722723\subsection{Reading of RQ5 on the available evidence}724725With two of the three measurements available, the answer to RQ5 is that the generative-model categories726have become the most connected part of the classification, both in what their works cite and in what other727labels their works carry, and that the areas of computer science that were most distinct from AI in the728baseline (systems, security, software, theory) now send a somewhat larger share of their references to it.729Whether this amounts to a change in the relationship between disciplines, or to the growth of one large730cross-cutting literature alongside disciplines that remain distinct, cannot be settled with citation and731label data: the self-shares of Table~\ref{tab:p9-area-flow-change} remain above three fifths in every area,732and the Perception area, the largest, is as self-contained in the post period as it was before. The733semantic measurement, when available, is the one that can discriminate between the two readings, because it734does not depend on the classifier's boundaries.735736\section{RQ6: institutions, countries and collaboration}737\label{sec:r-rq6}738739Part~VIII analyses the actor networks in full, and this section records the aggregate indicators that the740hypotheses require and the verdict on H6. The international co-authorship share, among works with at least741one resolved country, moves from \InternationalCollabYearlyTwoZeroOneEightShareIntl{} in 2018 to a maximum of742\InternationalCollabYearlyTwoZeroTwoZeroShareIntl{} in 2020 and declines in every subsequent year to743\InternationalCollabYearlyTwoZeroTwoFiveShareIntl{} in 2025 (Table~\ref{tab:collaboration},744Figure~\ref{fig:collaboration}). The share of works with a company affiliation, among works with any typed745affiliation, rises from \IndustryShareYearlyTwoZeroOneEightShareCompany{} in 2018 to a maximum of746\IndustryShareYearlyTwoZeroTwoOneShareCompany{} in 2021 and falls to747\IndustryShareYearlyTwoZeroTwoFiveShareCompany{} in 2025 (Table~\ref{tab:industry}). The country shares with748fractional counting (Table~\ref{tab:country_shares}, Figure~\ref{fig:country_shares}) show the share of749China rising through the window and the share of the United States falling, under both counting750conventions, as the continuation of a baseline trend; the joint output of the two countries falls in751absolute terms in the post period while both grow (Table~\ref{tab:country_network},752Section~\ref{sec:net-international}). The institutional hierarchy is stable: the institutions with the753largest post-period output are, with few exceptions, those with the largest baseline output754(Table~\ref{tab:top_institutions}), and the concentration of output across institutions does not increase755(Table~\ref{tab:institution_concentration}).756757\subsection{Hypothesis H6: geography and collaboration}758\label{sec:r-h6}759760H6 states that the share of works with industry affiliations and with China-affiliated authors rises in the761post period in AI categories, and that the international co-authorship share does not rise. The third clause762is \emph{supported}: the international share does not rise and in fact declines, with the composition763explanation that Part~VIII develops (the fastest-growing producers are the least internationalised). The764China clause is \emph{supported} in the aggregate, as a continuation of a trend that predates the boundary,765and \emph{inconclusive} for AI categories, because the category-specific country shares are not among the766result files (\pending{macro: China-affiliated share by category and period, results/bibliometrics}). The767industry clause is \emph{opposite} in the aggregate, where the company share peaks in 2021 and declines in768every post-period year, and \emph{inconclusive} for AI categories (\pending{macro: company-affiliated share by769category and period}). Two remarks qualify the opposite verdict. The company share is measured on works with770a typed affiliation, and the affiliation coverage of the corpus changes over the window771(Table~\ref{tab:collaboration}, last column), so a falling share can coexist with a rising count of772industry works; the count does rise (Table~\ref{tab:industry}). And the literature from which the773hypothesis was drawn measures industry participation at the frontier of machine learning research774\citep{ahmed2023industry}, whereas the corpus measures it over all of computer science, where the growth of775the post period is concentrated in categories and venues with low industry participation.776777\section{RQ7 and RQ8: breakthroughs, incremental gains and unresolved problems}778\label{sec:r-rq7}779780RQ7 asks which research areas demonstrate genuine breakthroughs versus incremental benchmark improvements,781and RQ8 which scientific problems remain unresolved despite rapid publication growth. Neither question has a782bibliometric answer, and the study does not pretend otherwise: the evidence for them is the knowledge-graph783extraction over the high-priority set (method, benchmark, dataset and claim nodes with provenance and784confidence, Section~\ref{sec:m-kg}) and the specialist reviews of Parts~V--VII, which are synthesised in785Part~X. At the time of writing the extraction had not produced result files that this part could tabulate,786and the specialist reviews were subject to checkpoint H-3. What the quantitative results of this part787contribute to the two questions is a set of constraints. The categories with the fastest post-period growth788(Section~\ref{sec:r-rq1}) and the strongest acceleration (Section~\ref{sec:r-its-categories}) are the ones in789which benchmark-driven incremental work is most plausible, because volume growth of this speed is not790compatible with a proportional growth in the number of distinct problems addressed; the categories whose791volume is flat but whose cross-area references to AI rise (Section~\ref{sec:r-rq4}) are the ones in which an792imported method is being applied to standing problems, which is where a breakthrough, if any, would register793as a change of method rather than of volume. The emerging-topic detector of Section~\ref{sec:r-rq10} provides794a third constraint: the topics it ranks highly are candidates for RQ7, and the topics with sustained volume795and declining citation acceleration are candidates for RQ8. These are hypotheses for Part~X, not findings796(\pending{H-3: knowledge-graph extraction counts by relation type, benchmark proliferation and method-adoption797series for the high-priority set, when produced}).798799\section{RQ9: openness, preprints and the evaluation of evidence}800\label{sec:r-rq9}801802Table~\ref{tab:p9-openness} assembles the openness indicators by publication year; Figure~\ref{fig:types-openness}803in Part~III shows the same series graphically. Three of them rise through the whole window and faster in the804post period. The share of eligible works linked to an arXiv e-print rises in every year, from805\VolumeYearlyTwoZeroOneEightNArxiv{} of \VolumeYearlyTwoZeroOneEightN{} works in 2018 to806\VolumeYearlyTwoZeroTwoFiveNArxiv{} of \VolumeYearlyTwoZeroTwoFiveN{} in 2025, roughly a doubling of the share807and a quadrupling of the count; the share of works whose surviving record is of preprint type rises more808steeply, because recent preprints have not yet been matched to a published version, and the 2026 value is809dominated by that effect. The open-access share rises from \VolumeYearlyTwoZeroOneEightNOa{} to810\VolumeYearlyTwoZeroTwoFiveNOa{} works, a rise in share of about twenty points over the window of which the811larger part occurs in the baseline; the abstract-availability share rises through 2024 and falls in 2025812and 2026 for records not yet enriched. The DBLP-indexed share falls in the post period, as813Section~\ref{sec:r-rq1} noted, and the conference share of the corpus is flat or slightly declining, which814means that the growth of preprints is not matched by a growth of conference publication in the indexed815record. The number of references per work (last column) is a metadata-completeness indicator rather than816an openness one and is discussed in Section~\ref{sec:r-rq1}.817818\input{tables/tab_p9_openness}819820\subsection{Hypothesis H7: preprints and openness}821\label{sec:r-h7}822823H7 states that the share of works with an arXiv version and the share with linked code rise in the post824period. The arXiv clause is \emph{supported}: the share rises in every post-period year and the post-period825share is above the baseline share by a wide margin (Table~\ref{tab:p9-periods}), with the caveat that the826post-period arXiv share is inflated by preprints that will later be merged with a published version and by827the growth of OpenAlex's coverage of arXiv. The code clause is \emph{inconclusive}: the pipeline does not828extract code links, and no result file measures the share of works with a linked repository829(\pending{macro: share of works with a code link by year; not produced by the current pipeline, scoped in830Phase 3 of the plan}). The evaluation-standards part of RQ9, which asks how benchmarks and reporting831practices evolved, is likewise outside what the bibliographic record measures and is addressed from the832knowledge-graph extraction in Part~X when available.833834\section{RQ10: early detection of emerging topics}835\label{sec:r-rq10}836837\subsection{Out-of-time performance of the detectors}838839Table~\ref{tab:p9-emerging-setup} summarises the evaluation design implemented in840Section~\ref{sec:m-emerging}: OpenAlex topics as units, features computed at a cutoff from data at or before841it, a label defined by top-decile growth with doubling over the following twelve or twenty-four months,842training cutoffs that end before the reference date and test cutoffs in 2023--2024. At the twelve-month843horizon the pooled test set has \EmergNTest{} (topic, cutoff) rows with a positive rate of844\EmergPosRateTest{}, lower than the training rate of \EmergPosRateTrain{}; at twenty-four months the rates845are \EmergtwofourPosRateTest{} and \EmergtwofourPosRateTrain{}. The positive rate is below the ten per cent846that a decile label would give because of the doubling condition, and it is lower in the test period than in847training because the corpus as a whole grew faster in 2023--2026, so that doubling against the trailing848window became harder for an individual topic.849850\input{tables/tab_p9_emerging_setup}851852Tables~\ref{tab:emerging_h12} and~\ref{tab:emerging_h24} in Appendix~F give the pooled metrics, and853Figure~\ref{fig:emerging-metrics} displays them. At the twelve-month horizon the gradient boosting model854attains an AUC of \EmergGbmAuc{}, the $z$-score baseline \EmergZscoreBaselineAuc{} and the logistic model855\EmergLogisticAuc{}; precision at fifty is \EmergGbmPrecisionAtFiveZero{} for the boosted model and856\EmergZscoreBaselinePrecisionAtFiveZero{} for the baseline, against a chance level of about the positive857rate. At the twenty-four-month horizon the ordering reverses: the $z$-score baseline attains an AUC of858\EmergtwofourZscoreBaselineAuc{} and a precision at fifty of \EmergtwofourZscoreBaselinePrecisionAtFiveZero{},859and both trained models fall below it (\EmergtwofourLogisticAuc{} and \EmergtwofourGbmAuc{}). The per-cutoff860values of Table~\ref{tab:p9-emerging-cutoffs} show that the pooled figures hide large variation: the861twelve-month AUC of the boosted model ranges over the eight test cutoffs from a little above six tenths to862near one, the near-perfect value belonging to the January 2024 cutoff, at which the baseline is also near863perfect and the logistic model near zero, a configuration that indicates a cutoff with very few positives864whose ranking is determined by one or two topics rather than a cutoff at which the models excel. Excluding865that cutoff, the boosted model exceeds the baseline at five of the seven remaining twelve-month cutoffs,866and the baseline exceeds both trained models at every twenty-four-month cutoff.867868\begin{figure}[htbp]\centering869\includegraphics[width=0.95\textwidth]{fig_emerging_metrics.pdf}870\caption{Out-of-time detection performance of the three detectors by horizon (AUC and precision at 20 and87150). Sources: \texttt{results/emerging/metrics\_h12.json}, \texttt{metrics\_h24.json}.}872\label{fig:emerging-metrics}873\end{figure}874875\input{tables/tab_p9_emerging_cutoffs}876877Three features of the design explain the pattern. The training set is small, with eight cutoffs at twelve878months and four at twenty-four, and the positives within it number in the dozens; a boosted model with three879hundred trees fitted to that many positives learns the idiosyncrasies of the 2020--2021 cutoffs, which the880reversal at twenty-four months, where training is smallest, bears out. The $z$-score is itself one of the881six features, so the trained models can at best refine a ranking that the baseline already provides, and at882the longer horizon the refinement overfits. And the label's doubling condition ties positives tightly to883recent abnormal growth, which is what the $z$-score measures, so that a longer horizon, over which abnormal884growth has more time to translate into doubling, favours the simplest signal. The logistic model's885performance below the baseline at both horizons, and below one half at some cutoffs, indicates that the886linear combination of standardised features is dominated by the size and age controls, which are887negatively associated with the label.888889\subsection{Hypothesis H8: detection}890\label{sec:r-h8}891892H8 states that the detector trained on cutoffs up to 2022 achieves an AUC above 0.70 and a precision at893fifty above 0.40 on test cutoffs in 2023--2024 against the baseline growth $z$-score rule, and that the894preregistered known-emergence set is detected with a median lead time of at least six months before each895topic's peak growth month. The first clause is \emph{not supported}. The best trained model's twelve-month896AUC, \EmergGbmAuc{}, is at the threshold to three decimals and marginally below it before rounding; its897precision at fifty is an order of magnitude below the threshold; and at twenty-four months the trained898models do not beat the baseline at all, so the comparison ``against the $z$-score rule'' is lost rather than899won. The AUC values say that the signals rank future emergers above non-emergers with a probability of about900seven in ten at twelve months and, through the baseline, of about eight in ten at twenty-four months, which901is informative; the precision values say that a list of fifty flagged topics at a given cutoff contains very902few of the topics that will double, which is the operationally relevant statement and the one the threshold903was set for.904905The second clause concerns the known-emergence set, and Section~\ref{sec:m-emerging} explains why it could906not be tested as written: the twelve developments do not map to OpenAlex topics, so the check actually run907applies the $z$-score rule to keyword series defined by regular expressions, and measures the lead of the908rule's first firing over the ex-post peak growth month. Figure~\ref{fig:known-set} shows the twelve series909with the first alarm, and Table~\ref{tab:known_set} the detection months, peak months and leads. The rule910fires for all \KnownSetNDetected{} concepts, with a positive lead for \KnownSetNPositiveLead{} of them and a911median lead of \KnownSetMedianLeadMonths{} months. The leads are longest for large language models912(\KnownLargeLanguageModelsLeadMonths{} months, with a detection in913\KnownLargeLanguageModelsDetectionMonth{} and a peak in \KnownLargeLanguageModelsPeakGrowthMonth{}) and for914reasoning models (\KnownReasoningModelsLeadMonths{} months, the regular expression for which includes915chain-of-thought prompting and therefore fires in \KnownReasoningModelsDetectionMonth{}, long before the916test-time-compute literature of 2024--2025 that gives the series its peak), and shortest for neural radiance917fields (\KnownNeuralRadianceFieldsLeadMonths{} months) and for mechanistic interpretability, whose lead is918negative (\KnownMechanisticInterpretabilityLeadMonths{} months): that series' fastest growth, in919\KnownMechanisticInterpretabilityPeakGrowthMonth{}, came from a base of a few works and the rule's volume920condition was not met until \KnownMechanisticInterpretabilityDetectionMonth{}. On the check as run, the921second clause is \emph{supported}: the median lead exceeds six months. The support is weak for three922reasons that the method section states and that bear repeating: the units are hand-written keyword series923rather than the detector's topics, the detector is the baseline rule rather than the trained model, and the924peak month is an ex-post quantity, so that a lead measured against it is not a lead that an observer could925have known at the time. The verdict on H8 as a whole is \emph{not supported} for the trained detector and926\emph{supported with reservations} for the lead-time clause on its substitute test.927928\begin{figure}[htbp]\centering929\includegraphics[width=0.95\textwidth]{fig_known_set.pdf}930\caption{Monthly keyword series for the twelve preregistered concepts with the $z$-score rule's first alarm.931Source: \texttt{results/emerging/known\_set\_monthly.csv}.}932\label{fig:known-set}933\end{figure}934935The scientific content of the exercise lies in what it rules out. A retrospective evaluation with a fixed9362026 topic vocabulary, a growth-only label and a handful of training cutoffs is the most favourable setting a937detector of this kind can be given short of look-ahead, and in it the trained models do not improve on a938one-line rule. Two of the mandate's seven candidate signals (semantic novelty, interdisciplinary connectivity)939and three others (methodological adoption, benchmark proliferation, open-source activity) are not in the940feature set (Section~\ref{sec:m-emerging}); the result is therefore a lower bound on what the full framework941could achieve, and an upper bound on what growth and citation signals alone achieve. \pending{H-4: whether942to describe the detector as a negative result for RQ10 or as a baseline for the semantic-cluster extension}.943944\section{Validation of the classification}945\label{sec:r-validation}946947Every category-level result of this part is conditional on the rule-based classifier of Part~IV, and the948protocol requires its validation on a stratified sample of two thousand works rated independently by two949raters, with Cohen's $\kappa$ between the raters, and the precision, recall and F1 of the assignment against950each rater and against their consensus, by category (Section~\ref{sec:m-validation}; checkpoint H-2). The951ratings of the two raters exist in \texttt{taxonomy/validation/}, but the metrics file952\texttt{taxonomy/validation/metrics.json} and the disagreement list had not been produced when this part was953written, so the validation cannot be reported here. \pending{taxonomy validation: insert the inter-rater954$\kappa$, the overall precision, recall and F1 of the assignment against the consensus, the number of955disagreements, and the categories with the lowest F1, from taxonomy/validation/metrics.json and956disagreements.csv; the generated Table~\ref{tab:validation} will exist once the file does}. Until then, the957reader should weight the category results by the known failure modes of the classifier (Section~\ref{sec:m-classification}):958works without an abstract are classified from venue and arXiv category alone, which favours the venue-defined959categories; the keyword-defined categories (the AI area, Sustainable Computing, Autonomous Scientific960Discovery) are sensitive to vocabulary drift; and the primary category of a multi-label work is decided by a961score whose ties are not random. The record-linkage precision estimated on the manually reviewed sample962(\LinkagesampleDoiAutoPrecision{} for DOI links, \LinkagesampleTitleYearAutoPrecision{} for title-and-year963links; Section~\ref{sec:m-linkprecision}) is the one validation statistic of the corpus that is available,964and it bounds the error introduced by linkage rather than by classification.965966\section{Summary of the hypotheses and of what remains uncertain}967\label{sec:r-summary}968969Table~\ref{tab:p9-hypotheses} collects the verdicts. The table is a reading aid and not a result file; each970verdict is argued in the section cited and carries the qualifications stated there.971972\begin{table}[htbp]\centering973\caption{Verdicts on the pre-specified hypotheses on the evidence of this part.}974\label{tab:p9-hypotheses}975\small976\begin{tabular}{p{0.07\textwidth}p{0.36\textwidth}p{0.17\textwidth}p{0.3\textwidth}}977\toprule978\textbf{H} & \textbf{Clause} & \textbf{Verdict} & \textbf{Where and why} \\979\midrule980H1 & Post-period monthly growth rate higher than baseline & supported & Sec.~\ref{sec:r-h1}; slope change ranks first among all candidate dates \\981   & Level shift at the boundary not distinguishable from placebo breaks & supported & placebo rank of the level near the bottom; acceleration dated to spring 2022 by Bai--Perron \\982   & Growth differences concentrated in a few categories & not supported & Sec.~\ref{sec:r-its-categories}; positive slope change in 37 of 40 categories \\983H2 & AI-category share rises in the post period & supported & Sec.~\ref{sec:r-h2}; driven by the language and generative categories; Deep Learning Architectures falls \\984   & A non-AI area declines relatively while growing absolutely & supported & three areas (data and distributed, theory, software) \\985H3 & Bai--Perron break in Oct.\ 2022--June 2023 for LLM, generative AI, agents, AI safety & not supported & Sec.~\ref{sec:r-h3}; all four break before the window (2021--2022) \\986   & No break in the window for formal verification, complexity, OS, architecture & opposite (two of four) & complexity and architecture break inside the window; no $p$-value attached to the dates \\987H4 & Cross-area citation shares rise & supported, small & Sec.~\ref{sec:r-h4}; about half composition; Perception area does not rise \\988   & Area modularity decreases & supported & monotone through the window, not concentrated at the boundary \\989H5 & Citation Gini higher in post period for AI categories & inconclusive & Sec.~\ref{sec:r-h5}; per-category windows not produced; aggregate movement is a truncation effect \\990   & Venue HHI stable or lower & opposite as computed, inconclusive as intended & index without repositories not produced \\991H6 & Industry share rises (AI categories) & opposite (aggregate), inconclusive (AI) & Sec.~\ref{sec:r-h6}; company share peaks in 2021 \\992   & China share rises (AI categories) & supported (aggregate), inconclusive (AI) & continuation of a baseline trend \\993   & International co-authorship share does not rise & supported & declines from 2020 \\994H7 & arXiv-version share rises & supported & Sec.~\ref{sec:r-h7}; inflated by unmatched recent preprints \\995   & Linked-code share rises & inconclusive & not measured \\996H8 & Trained detector AUC above 0.70 and P@50 above 0.40 versus the baseline & not supported & Sec.~\ref{sec:r-h8}; AUC at the threshold, P@50 an order of magnitude below; baseline wins at 24 months \\997   & Known-emergence set detected with median lead at least 6 months & supported with reservations & substitute keyword test, baseline rule, ex-post peak \\998\bottomrule999\end{tabular}1000\end{table}10011002Four sources of uncertainty run through every verdict and are stated here once. The first is right1003truncation: the post period ends in an incomplete year, the reference-indexing lag removes a growing share1004of references from the most recent cohorts, and every citation-based statistic of 2024 and later is1005provisional; the structural-change analysis handles this by truncating at \ItsLastComplete{} and varying the1006truncation month, and the citation analysis by reporting only complete windows, but neither device recovers1007what the database has not yet indexed. The second is the boundary of the database: the growth of the corpus1008in the post period is concentrated in records outside DBLP's curation, and the study cannot separate growth1009in computer-science output from growth in OpenAlex's coverage of it \citep{culbert2025openalexcoverage,1010alperin2024openalex}; the DBLP-only records of Table~\ref{tab:dblp_only} and the strict core of1011\PrismaStrictCoreDblpAndCsPrimary{} works are the instruments for a sensitivity analysis that the current1012result files do not yet contain (\pending{macro: structural-change estimates on the strict core, as the1013protocol's sensitivity analysis}). The third is the classifier, whose validation is pending and whose1014boundaries are fixed in 2026 vocabulary and applied to 2018 works; every category-level and every flow1015statistic inherits it, and the semantic measurements that do not depend on it are the ones still to come.1016The fourth is the inferential status of the time-series results: the reference date is a marker, the1017estimates at it are descriptions of a kink in a series that was already bending, and the placebo and1018alternative-date analyses show that the kink is not tied to the month. What the evidence supports is a1019multi-year acceleration of the indexed output of computer science, concentrated in share terms in the1020language and generative-model categories but present in nearly all of them, accompanied by a modest increase1021in the citation traffic from the other areas toward AI and by a reorganisation of the co-classification1022structure around the foundation-model categories, and a continuation, without inflection, of the baseline1023trends in collaboration, geography and openness. What it does not support is a discrete change at the1024reference date in any series examined, a concentration of the acceleration in a few categories, or the1025ability of simple growth and citation signals to pick the emerging topics of 2023--2024 with useful1026precision. \pending{H-4: the summary reading of this paragraph}.1027