spb/wp44_uqo

1% part09_results.tex — Part IX, Empirical Results. Written by Agent L (Part IX instance, with Agent I/J roles), 2026-11-03.2% Corpus v3 result files (results/, data/metadata/, reports/classification_report.json); every number is a macro of3% tables/numbers.tex or a cell of a table generated by tables/make_tables.py (tables_part09). Items marked \pending{} await4% a result file (taxonomy validation metrics, semantic atlas) or the PI checkpoint H-4 (interpretation of structural change).56\section{Scope, sources and reading rules}7\label{sec:r-scope}89This part collects the quantitative findings of the study and reads them against the ten research questions and10the eight pre-specified hypotheses of the protocol. It is deliberately a reporting part: the methods are those of11Part~IV, the interpretation of the literature is left to Parts~V--VII, and the discussion of what the findings12mean for computer science is left to Part~XI. Three kinds of material appear below. The first is the set of13descriptive series on volume, composition, openness and influence, which answer RQ1, RQ3 and RQ9 and provide14the evidence for H2 and H7. The second is the structural-change analysis of the monthly series, in total and by15category, which answers RQ2 and tests H1 and H3. The third is the set of network and detection results, which16answer RQ4--RQ6 and RQ10 and test H4--H6 and H8. The actor networks of RQ6 (authors, countries, institutions,17venues) are analysed at length in Part~VIII; this part does not repeat that analysis but summarises its verdicts18where a hypothesis requires them and cross-references its tables by label.1920The evidence base is the eligible corpus of Part~III in its third build, with \PrismaCsCoreEligible{} works in the21computer-science core, of which \PrismaEligibleByPeriodBaseline{} have a first date in the baseline window and22\PrismaEligibleByPeriodPost{} in the post window (Table~\ref{tab:p9-periods}). Every monthly statistic below is23computed on the month-precision subset, \PrismaEligibleDatePrecisionMonth{} works, because the24\PrismaEligibleDatePrecisionYear{} works whose only date is a year would otherwise be stacked on January25(Section~\ref{sec:data-dates}; Figure~\ref{fig:volume-monthly-raw}); yearly statistics use all eligible works.26Table~\ref{tab:p9-periods} shows that the two periods are not alike in completeness: the post period has a27higher share of month-precision dates, a higher arXiv-linked share and a much lower share of works with at28least one resolved reference, the last being an artefact of reference indexing, which lags publication by more29than a year for a substantial fraction of records (Table~\ref{tab:indexing_lag}, Section~\ref{sec:net-boundaries}).30Any comparison of the two periods inherits these differences, and the text names them wherever they bear on a31reading.3233\input{tables/tab_p9_periods}3435The category-level results depend on the classification of Part~IV. Table~\ref{tab:p9-classification} reports its36coverage: the share of eligible works that received a primary category rises from under four fifths in 2018 to37over four fifths in the last years, and the number of labels per classified work rises monotonically through38the window. Both movements matter for what follows. The rise in coverage means that category counts grow39slightly faster than eligible counts for reasons unrelated to research activity, so growth rates of categories40should be read against the coverage column rather than against the eligible totals of41Table~\ref{tab:eligible_year}; the rise in labels per work means that multi-label counts, which are the input of42the monthly category series, grow faster than primary counts, and that co-classification statistics43(Section~\ref{sec:r-rq5}) are more generous in the post period. The classification run itself processed44\ClassificationWorks{} records, a superset of the eligible corpus that includes the penumbra set and records45later removed by the eligibility filter, of which \ClassificationClassified{} received at least one category46and \ClassificationAssignments{} category assignments were made in total. The validation of the assignment47against two independent raters on a stratified sample is part of the protocol (checkpoint H-2) and its48metrics file was not available when this part was written; Section~\ref{sec:r-validation} is reserved for it.4950\input{tables/tab_p9_classification}5152Four reading rules apply throughout. First, the series end at the last complete month of the panel,53\ItsLastComplete{}, and the yearly tables carry 2026 as an incomplete row; right-truncation affects every54statistic that depends on citations or references more than it affects counts of works, and the fixed-window55citation statistics are complete only for the publication years flagged in Table~\ref{tab:citation_windows}.56Second, the reference date of 30~November 2022 is a historical marker. The interrupted-time-series estimates57are changes in level and slope at that date net of trend and season; they are not treatment effects, and58the alternative-date and placebo analyses exist precisely to show how specific to that month the estimated59changes are \citep{bernal2017its, hausman2018rdit}. Third, the forty category series are analysed with the60same battery of tests, so forty $p$-values are produced for every statistic; at the conventional five per61cent level two of them are expected to fall below the threshold under a global null, and the text reads62counts of significant categories with that expectation in mind rather than applying a formal correction63\citep{benjamini1995fdr}, because the hypotheses were stated for named categories and are tested one by one.64Fourth, the verdict on each hypothesis is one of \emph{supported}, \emph{not supported}, \emph{opposite}65(the data move against the stated direction) or \emph{inconclusive} (the test cannot discriminate), and the66hypotheses are restated in full before each verdict. Interpretive statements about the structural-change67results that go beyond the estimates are marked as pending the principal investigator's review (checkpoint68H-4), as the protocol requires.6970\section{RQ1: the volume and composition of computer-science research}71\label{sec:r-rq1}7273\subsection{Volume}7475The yearly totals of Table~\ref{tab:eligible_year} and Figure~\ref{fig:volume-yearly} describe a corpus that76was almost flat through the baseline and grew afterwards. The eligible count moves from77\VolumeYearlyTwoZeroOneEightN{} works in 2018 to \VolumeYearlyTwoZeroTwoTwoN{} in 2022, a span in which the782020 and 2021 cohorts are the largest and 2022 is slightly below them, and then to79\VolumeYearlyTwoZeroTwoThreeN{} in 2023, \VolumeYearlyTwoZeroTwoFourN{} in 2024 and80\VolumeYearlyTwoZeroTwoFiveN{} in 2025; the 2026 cohort, \VolumeYearlyTwoZeroTwoSixN{} works at the snapshot,81covers ten months of first dates and an indexing lag, and is not comparable. Three features of the82composition columns of Table~\ref{tab:volume_yearly} qualify the growth. The DBLP-indexed count83(\VolumeYearlyTwoZeroOneEightNDblp{} in 2018, \VolumeYearlyTwoZeroTwoFiveNDblp{} in 2025) grows much less than84the total, so that the DBLP share of the corpus falls in the post period; the OpenAlex-primary count85(\VolumeYearlyTwoZeroOneEightNCsPrimary{} to \VolumeYearlyTwoZeroTwoFiveNCsPrimary{}) and especially the86arXiv-linked count (\VolumeYearlyTwoZeroOneEightNArxiv{} to \VolumeYearlyTwoZeroTwoFiveNArxiv{}) grow faster87than the total. The post-period growth of the corpus is therefore concentrated in the parts of it that DBLP's88venue-based curation does not cover, which is consistent both with a real shift of output toward preprints and89toward venues outside DBLP's lists, and with a change in OpenAlex's own coverage over the period90(Section~\ref{sec:net-boundaries} examines the DBLP-only records; Table~\ref{tab:dblp_only}). The two cannot be91separated with the data at hand, and every growth statement of this part should be read as a statement about92the indexed record of computer science rather than about its production.9394The mean number of authors per work rises from \VolumeYearlyTwoZeroOneEightMeanAuthors{} in 2018 to95\VolumeYearlyTwoZeroTwoFiveMeanAuthors{} in 2025, continuing a trend that Part~VIII documents in detail96(Section~\ref{sec:net-teams}); the mean number of declared references per work rises from97\VolumeYearlyTwoZeroOneEightMeanRefs{} to a maximum of \VolumeYearlyTwoZeroTwoOneMeanRefs{} in 2021 and then98falls to \VolumeYearlyTwoZeroTwoFiveMeanRefs{} in 2025 and \VolumeYearlyTwoZeroTwoSixMeanRefs{} in 2026, a99decline that is entirely attributable to reference indexing lag and is the clearest single illustration of the100right-truncation problem in the corpus.101102The monthly series (Figure~\ref{fig:its-total}) carries the same information at the resolution at which the103structural-change tests operate. Its pre-period slope, \ItsSeriesTotalItsChatgptPreSlope{} in log points per104month, corresponds to a growth of about a third of a per cent per month, which compounds to the yearly105totals above; the acceleration visible to the eye after 2022 is quantified in Section~\ref{sec:r-rq2}.106107\subsection{Composition by area and category}108109Figure~\ref{fig:area-shares} and Table~\ref{tab:p9-area-shares} give the composition of the classified corpus by110the eight top-level areas of the ontology, using the primary category of each work. The dominant area in every111year is Perception, Language, Robotics and Interaction, which holds a little over a third of primary112assignments and whose share is stable within two percentage points over the window; it contains Computer113Vision, the largest single category (Table~\ref{tab:p9-category-periods}). The movement is elsewhere. The share114of the AI and Machine Learning area rises in every year of the window, from under a tenth in 2018 to over a115tenth in 2022 and to close to a fifth in 2025; its rise is the mirror image of the fall of Data, Information and116Distributed Systems, whose share declines in every year, and of the smaller declines of Theory of Computation117and of Software, Programming Languages and Formal Methods. Security, Privacy and Trust gains about one and a118half points over the window; Hardware and Computational Science are approximately stable. The period rows of119the table, which split at the reference date rather than at the calendar year, show the same ordering.120121\begin{figure}[htbp]\centering122\includegraphics[width=0.95\textwidth]{fig_area_shares.pdf}123\caption{Share of eligible works by top-level area (primary category) per year. Source:124\texttt{results/graph/category\_counts\_yearly.csv}; 2026 omitted as incomplete.}125\label{fig:area-shares}126\end{figure}127128\input{tables/tab_p9_area_shares}129130Two cautions apply to this picture. The classifier assigns the primary category by the largest score, and a work131on, say, a vision--language model is counted once, under whichever of Computer Vision, Natural Language132Processing, Multimodal AI or Large Language Models scores highest; the area shares of133Table~\ref{tab:p9-area-shares} are therefore sensitive to the tie-breaking of the score, and the multi-label134counts of Table~\ref{tab:p9-category-periods} give a less brittle view of the same movement. Second, the135coverage of the classifier rises over the window (Table~\ref{tab:p9-classification}), and the works that enter136the classified set late are not a random sample of the eligible corpus: they are predominantly works with137abstracts, which the AI categories, defined largely by keyword patterns, pick up more readily than the138venue-defined categories of theory or systems. A part of the rise of the AI area is therefore a rise in the139classifiability of the literature, and the text does not attempt to separate it.140141Table~\ref{tab:p9-category-periods} descends to the forty categories. Sorted by their post-period share, the142table shows a top tier that is unchanged across periods (Computer Vision, Computer Networks, Human--Computer143Interaction, Cybersecurity, Natural Language Processing), a single category whose rank changes by an order144of magnitude (Large Language Models, which moves from under half a per cent of primary assignments in the145baseline to over four per cent in the post period, with a multi-label count more than three times its146baseline value), and a cluster of AI categories with the same profile at smaller scale: Generative AI,147AI Safety and Alignment, Multimodal AI, Foundation Models and Federated Learning all have post-period148multi-label counts above three times their baseline counts. The categories whose multi-label counts are149below their baseline value in the post period are Computer Networks, Computer Architecture, Formal150Verification, Programming Languages and Operating Systems, and the compound annual growth rates of the last151two columns confirm that the fall is a fall of shares more than of volumes: among these, only Computer152Architecture and Cloud Computing have a negative post-period CAGR of primary counts, and the others grow in153absolute terms while losing share. The post-period CAGRs are computed over the two complete years 2023 to1542025 (equation~\eqref{eq:cagr}) and are more volatile than the four-year baseline rates; they also carry the155coverage drift noted above.156157\input{tables/tab_p9_category_periods}158159\begin{figure}[htbp]\centering160\includegraphics[width=0.95\textwidth]{fig_category_growth.pdf}161\caption{Yearly primary-category output indexed to 2018 for the eight fastest- and eight slowest-growing162categories (2018--2025). Source: \texttt{results/graph/category\_counts\_yearly.csv}.}163\label{fig:category-growth}164\end{figure}165166Figure~\ref{fig:category-growth} shows the growth indices of the extremes. The fastest-growing categories are,167without exception, AI categories that were small in 2018 (Foundation Models, Large Language Models, Generative168AI, Multimodal AI, Federated Learning, AI Safety and Alignment, Autonomous Scientific Discovery) together with169Quantum Computing; their indices are large partly because their 2018 bases are small, and the table's shares170are the better measure of their weight. The slowest-growing categories are the venue-defined systems and171theory categories (Operating Systems, Compilers, Computer Architecture, Programming Languages, Formal172Verification, Cloud Computing), whose indices stay near or below their 2018 value until 2023 and rise modestly173afterwards; these are also the categories whose DBLP coverage is highest, so their flatness is the flatness of174the DBLP-indexed count noted above.175176\subsection{Hypothesis H2: composition}177\label{sec:r-h2}178179H2 states that the share of the AI-related categories (categories 1--8 and 38 of the mandate, which form the AI180and Machine Learning area of the ontology) in computer-science output increases in the post period, and that181the share of at least one non-AI area (theory, systems, networks) declines in relative terms while growing in182absolute terms. Table~\ref{tab:p9-ai-shares} tests the first clause with the bootstrap intervals of183Part~IV, obtained by resampling the works of each year with replacement \citep{efron1979bootstrap}. The area's184share rises from 2018 to 2022 and from 2022 to 2025, but the rise is not uniform across its nine categories.185Seven of them (Large Language Models, Foundation Models, Multimodal AI, Generative AI, AI Agents and186Autonomous Systems, Reinforcement Learning, AI Safety and Alignment) have a 2025 interval that lies entirely187above their 2022 interval; Machine Learning Theory has overlapping intervals in the two years; and Deep188Learning Architectures has a 2025 interval entirely \emph{below} its 2022 interval, its share having peaked189in the baseline. AI Agents and Autonomous Systems is the one category whose 2025 share remains below its 2018190share, a consequence of the category's definition (it includes the pre-LLM agent and multi-agent literature,191whose share was declining through the baseline) rather than evidence against the hypothesis. The Large192Language Models category accounts for the greater part of the area's post-period gain, and the hypothesis is193better described as supported for the generative and language-model categories than for machine learning as194a whole. The intervals are narrow because the195resampling treats the classifier as fixed; the uncertainty that matters is the classifier's, not the196sampling's, and the intervals should be read as a lower bound on it.197198\input{tables/tab_p9_ai_shares}199200The second clause is tested by Table~\ref{tab:p9-area-shares} together with the CAGR columns of201Table~\ref{tab:p9-category-periods}. The Data, Information and Distributed Systems area (which contains202Computer Networks) and the Theory of Computation area both lose share in every year of the window, and their203primary counts in 2025 exceed their 2022 counts, so both decline relatively while growing absolutely; the204Software, Programming Languages and Formal Methods area shows the same pattern. The second clause is therefore205met by three areas rather than by one. The verdict on H2 is \emph{supported} on both clauses, with the206qualification that the relative decline of the non-AI areas begins before the reference date (it is visible in207every year from 2018) and continues through it without a visible change of pace in the yearly data; the208monthly analysis of Section~\ref{sec:r-rq2} examines whether a change of pace exists at monthly resolution.209Table~\ref{tab:diversity} adds the diversity indices of the category mix: the Rao--Stirling index and the210Leinster--Cobbold index of order two are both lower in 2023--2025 than in 2018--2021, so the concentration of211primary assignments on fewer, mutually similar categories increased slightly, which is what the rise of a212cluster of related AI categories implies. The similarity matrix behind these indices is the co-classification213Jaccard similarity rather than the semantic one, because the semantic category similarities were not yet214available (\pending{semantic atlas: results/semantic/category\_similarity\_baseline.csv and \_post.csv; the215diversity indices should be re-estimated with the semantic distance when they exist}).216217\section{RQ2: acceleration and structural change}218\label{sec:r-rq2}219220\subsection{The total series at the reference date}221\label{sec:r-its-total}222223Figure~\ref{fig:its-total} shows the monthly series of month-precision works from \ItsFirst{} to224\ItsLastComplete{} with the fitted values of the interrupted time series of equation~\eqref{eq:its}, and225Table~\ref{tab:p9-its-total-events} reports the estimates at the reference date and at the four alternative226dates. At December 2022 the estimated change in log level is \ItsSeriesTotalItsChatgptLevelChange{}, a value227that is small, negative and indistinguishable from zero (HAC $p$ = \ItsSeriesTotalItsChatgptLevelP{}); the228estimated change in monthly log slope is \ItsSeriesTotalItsChatgptSlopeChange{}, which, added to the229pre-period slope of \ItsSeriesTotalItsChatgptPreSlope{}, implies a post-period growth rate of roughly one and230four tenths of a per cent per month against roughly three tenths before, a more than fourfold increase in the231rate of growth. The HAC $p$-value of the slope change rounds to \ItsSeriesTotalItsChatgptSlopeP{}, and the fit232explains \ItsSeriesTotalItsChatgptRTwo{} of the variance of the log series over233\ItsSeriesTotalNMonths{} months. The estimates describe a kink, not a jump: the series did not move to a higher234level in December 2022, it began, at some point, to grow faster.235236\begin{figure}[htbp]\centering237\includegraphics[width=0.95\textwidth]{fig_its_total.pdf}238\caption{Monthly output (month-precision works) with the interrupted-time-series fit at December 2022; the239annotation gives the level and slope changes with their HAC and placebo $p$-values. Sources:240\texttt{results/econometrics/fit\_total.csv}, \texttt{its\_results.json}.}241\label{fig:its-total}242\end{figure}243244\input{tables/tab_p9_its_total_events}245246Whether ``some point'' is the reference date is the question the remaining rows of247Table~\ref{tab:p9-its-total-events} and the diagnostics of Table~\ref{tab:p9-total-robustness} address. Moving248the event to May 2022 (DALL$\cdot$E~2) or April 2023 (GPT-4) changes the slope estimate by less than a tenth of249its value; moving it to June 2020 (GPT-3) halves it but leaves it clearly positive, and moving it to October2502024 (o1) gives a comparable slope change on top of a steeper pre-period. The level estimates vary in sign251across the five dates and two of them (DALL$\cdot$E~2, o1) have HAC $p$-values below one per cent, with252opposite signs. The honest summary of the five columns is that the series exhibits a sustained acceleration253over 2022--2024 that any event date placed within that span will pick up as a slope change, and that no254single month within it shows a level shift that the others do not.255256\input{tables/tab_p9_total_robustness}257258The placebo distribution (Figure~\ref{fig:placebo-total}) makes this precise. Over the259\ItsSeriesTotalPlaceboChatgptNPlacebo{} candidate event months, the level $t$-statistic of December 2022 ranks260\ItsSeriesTotalPlaceboChatgptLevelTRank{}th in absolute value (placebo $p$ =261\ItsSeriesTotalPlaceboChatgptLevelTPlaceboP{}), that is, almost every other month of the series produces a262larger apparent level shift than the reference date does; the slope $t$-statistic ranks first (placebo $p$ =263\ItsSeriesTotalPlaceboChatgptSlopeTPlaceboP{}, the smallest attainable value being one over the number of264candidates). Restricted to the baseline months, the placebo $p$ of the slope is265\ItsSeriesTotalPlaceboChatgptBaselineOnlySlopeTPlaceboP{}: no month before the reference date yields a slope266change as large as the one estimated there, which is the pre-trend diagnostic the protocol asked for and is267the same statement as the first rank, read from the other side. The donut specification, which drops the268event month and three months on each side, moves the level estimate to269\ItsSeriesTotalItsChatgptDonutLevelChange{} (HAC $p$ = \ItsSeriesTotalItsChatgptDonutLevelP{}) and leaves the270slope change essentially unchanged; the donut level estimate is larger than the full-sample one because the271months it removes, the winter of 2022--2023, lie below the post-period trend line, so that the trend fitted272without them extrapolates to a higher December 2022 value. The sensitivity rows show that ending the series in273March or August 2026 instead of June changes the slope estimate in the third decimal and leaves the level274estimate within a hundredth of zero, of unstable sign and insignificant.275276\begin{figure}[htbp]\centering277\includegraphics[width=0.95\textwidth]{fig_placebo_total.pdf}278\caption{Placebo distribution of the ITS $t$-statistics (level and slope) over every candidate event month of279the total series; the true event month is highlighted. Source: \texttt{results/econometrics/placebo\_total.csv}.}280\label{fig:placebo-total}281\end{figure}282283The data-driven methods date the kink earlier than the reference date. The Bai--Perron procedure selects a284single break under both criteria ($m$ = \ItsSeriesTotalBaiPerronMBic{} by BIC, and the same number by LWZ; the285BIC falls from \ItsSeriesTotalBaiPerronBicByMZero{} with no break to \ItsSeriesTotalBaiPerronBicByMOne{} with286one and rises again for two and three), and places it in April 2022, eight months before the reference date;287with the series ended in August 2026 the date moves to February 2022, so the location is not sharp. The288trimming parameter $h$ = \ItsSeriesTotalBaiPerronH{} months forbids a second break after April 2025 and within289fifteen months of the first, so a second kink in late 2024 or 2025 would not be found by this procedure. PELT,290which fits a piecewise-constant mean rather than a trend and therefore flags the points where the level291moves most, places changepoints in May 2023 and December 2024; the first is the point at which the series,292growing at the post-period rate, has risen far enough above its baseline mean for a step to be the cheapest293description, and it should not be read as an independent dating of the acceleration. The Bayesian online294detector assigns a maximum posterior probability of295\ItsSeriesTotalBocpdCpProbTwoZeroTwoTwoOneTwoToTwoZeroTwoThreeZeroThree{} to a changepoint within the four296months preceding any month of December 2022 to March 2023, and its maximum-a-posteriori run length never297resets over the whole series: an observer following the monthly totals with this detector would not have298concluded, at any point, that the process had changed. The reconciliation of the four methods is that the299acceleration is gradual at monthly resolution, which a trend model with one kink summarises well and a300step model or an online detector with a constant-mean likelihood summarises poorly.301302\subsection{Hypothesis H1: volume}303\label{sec:r-h1}304305H1 states that the monthly growth rate of computer-science works is higher in the post period than in the306baseline, but that the level shift at November 2022 is not distinguishable from placebo breaks in the baseline307once seasonality and indexing lag are modelled; the protocol adds that growth differences are expected to be308concentrated in a few categories rather than uniform. On the total series the first clause is309\emph{supported}: the slope change is positive, large relative to the pre-period slope, robust to the donut310specification and to the truncation month, and ranks first among all candidate dates and above every311baseline date in the placebo distribution. The second clause is also \emph{supported}: the level change at312the reference date is small, of uncertain sign across specifications, and ranks near the bottom of the313placebo distribution. The clause about concentration is examined in Section~\ref{sec:r-its-categories} and is314\emph{not supported} in its stated form, because the slope change is positive and conventionally significant315in nearly every category, although its magnitude varies by an order of magnitude across them.316317Two qualifications are part of the verdict. The first is that the acceleration predates the reference date in318the unknown-break estimates (spring 2022), so H1's first clause is supported in the weaker sense that the319post period grows faster, not in the sense that the growth rate changed at the boundary; a reader who takes320the Bai--Perron date at face value would describe a change that began during the baseline and was already321under way when ChatGPT was released. \pending{H-4: whether to describe the aggregate acceleration as beginning322in spring 2022 (Bai--Perron) or as a 2022--2024 phenomenon without a sharp onset (placebo and alternative323dates)}. The second is that the indexing-lag model the protocol envisaged was replaced by truncation324(Section~\ref{sec:m-truncation}); the hypothesis is therefore tested conditional on the series ending at the325last complete month, and the sensitivity rows of Table~\ref{tab:p9-total-robustness} are the only evidence on326what a different truncation would do. With about a hundred observations and a break near the middle of the327sample, the design has ample power for a slope change of this size and little for a level shift of a few per328cent \citep{zhang2011itspower}, so the null result on the level should be read as ``not detectable'' rather329than ``absent''.330331\subsection{The category series}332\label{sec:r-its-categories}333334The same battery is applied to the forty multi-label category series. Figure~\ref{fig:breaks-by-category}335orders the categories by their estimated slope change at December 2022; Tables~\ref{tab:p9-its-slopes}336and~\ref{tab:p9-its-levels} give the slope and level estimates with their HAC and placebo $p$-values, and337Table~\ref{tab:p9-unknown-breaks} the data-driven dates. Table~\ref{tab:breaks} in Appendix~F carries the338same estimates in the layout of the result file.339340\begin{figure}[htbp]\centering341\includegraphics[width=0.95\textwidth]{fig_breaks_by_category.pdf}342\caption{ITS slope change at December 2022 by category, with the level change annotated; bars in colour have a343placebo $p$ below 0.05 for the slope. Source: \texttt{results/econometrics/breaks.csv}.}344\label{fig:breaks-by-category}345\end{figure}346347\input{tables/tab_p9_its_slopes}348349The slope changes are positive in thirty-seven of the forty categories and negative in three (Blockchain350Systems, Deep Learning Architectures, Federated Learning), and the HAC $p$-value is below five per cent in351all but the first two of these; by the conventional criterion almost every category accelerated. The352placebo column tells a different story. Only one category, Applied Cryptography, has a placebo $p$ for the353slope below five per cent, and with forty series two such values are expected under a global null; the354total series is the only series whose December 2022 slope change ranks first among candidate dates. The355reason is visible in the pre-slope column: the category series are noisier than the total, their placebo356distributions are wider, and a slope change that is large in absolute terms, as in Large Language Models357(\ItsSeriesCatLlmItsChatgptSlopeChange{} per month on a pre-period slope of358\ItsSeriesCatLlmItsChatgptPreSlope{}, HAC $p$ rounding to \ItsSeriesCatLlmItsChatgptSlopeP{}), ranks only359\ItsSeriesCatLlmPlaceboChatgptSlopeTRank{}th of \ItsSeriesCatLlmPlaceboChatgptNPlacebo{} candidate dates360(placebo $p$ = \ItsSeriesCatLlmPlaceboChatgptSlopeTPlaceboP{}) because the series has accelerated at many361dates. The placebo rank over all candidate dates is the appropriate measure of whether December 2022 is special362for a given category, and by that measure it is not special for any of them. The baseline-only rank, which363asks the weaker question of whether any month \emph{before} the reference date produced a larger slope364change, answers no for a substantial minority of categories (the zero entries of the last placebo column,365which include AI Agents, Algorithms, Computational Biology and Autonomous Scientific Discovery): their366post-period acceleration exceeds anything in their baseline, even though it does not exceed what other367post-period dates would show.368369The ordering of the slope changes is nonetheless informative about where the acceleration is largest. The370five largest changes belong to AI Agents and Autonomous Systems, Large Language Models, Multimodal AI,371Information Retrieval and Distributed Computing; Generative AI, Cloud Computing, Natural Language Processing,372Quantum Computing and Compilers follow. The presence of Information Retrieval, Distributed Computing, Cloud373Computing and Compilers in the upper part of the ranking is a finding that the hypotheses did not anticipate:374these are categories that the LLM literature draws on (retrieval, serving infrastructure, compilation of375models to accelerators) and whose multi-label counts grow because works on language models carry their labels376(Section~\ref{sec:r-rq5}). At the other end, the categories with the smallest positive changes are Edge377Computing, Robotics, Computational Biology, Foundation Models and Computer Vision, all five with pre-period378slopes above the median of the forty, so that a continuation of their growth registers as a small change;379Foundation Models in particular has one of the three steepest pre-period slopes of all categories and a380post-period slope that is only slightly steeper. The three negative changes are of the same kind: Federated381Learning had the steepest baseline slope of all categories and Deep Learning Architectures one of the382steeper ones, and both decelerate to rates that remain positive; Blockchain Systems decelerates from a3832018--2020 boom.384385\input{tables/tab_p9_its_levels}386387The level changes (Table~\ref{tab:p9-its-levels}) are mostly negative, which is the signature of a trend model388fitted through a kink: when growth accelerates gradually from mid-2022, a regression with a kink fixed at389December 2022 absorbs the months between the true onset and the fixed date into a negative intercept shift.390Fifteen categories have a level change with a HAC $p$ below five per cent, of which only Human--Computer391Interaction, Computer Networks and Sustainable Computing have a positive one; none has a placebo $p$ for the392level below five per cent, and the baseline-only placebo $p$ of the level exceeds a quarter in every393category. The donut estimates are in most cases positive where the full-sample ones are negative, for the394reason given for the total series, and are largest in Large Language Models395(\ItsSeriesCatLlmItsChatgptDonutLevelChange{} with HAC $p$ rounding to396\ItsSeriesCatLlmItsChatgptDonutLevelP{}), Multimodal AI and Generative AI, the three series in which the397winter of 2022--2023 lies furthest below the subsequent trend. The BOCPD column confirms the absence of an398online-detectable change: the maximum four-month changepoint probability over December 2022 to March 2023399exceeds one quarter only in Cloud Computing and Operating Systems, two small and noisy series, and the400detector resets its run length in that window for no category.401402\input{tables/tab_p9_unknown_breaks}403404The data-driven dates of Table~\ref{tab:p9-unknown-breaks} spread over the window. The Bai--Perron dates405cluster in 2022 (February to August) for a group that includes Large Language Models,406Natural Language Processing, Multimodal AI, Computer Vision, Cybersecurity, Applied Cryptography, Software407Engineering, Sustainable Computing and Privacy-Preserving Computation, and in 2023 (April to November) for a408group that includes Algorithms, Computational Complexity, Computer Architecture, Compilers, Database Systems,409Operating Systems and Speech and Audio Computing; several categories have a second break in mid-2024 and a410few (Blockchain, Federated Learning, Computational Biology, Autonomous Scientific Discovery) have their first411break in 2020. The pre-specified H3 window of October 2022 to June 2023 contains a BIC break date for seven412categories, listed in the table's penultimate column, and a PELT changepoint for twelve. The temporal413pattern that emerges is of two waves: an earlier kink, in 2022, in the categories closest to the generative414model literature and in the large applied categories, and a later kink, in 2023, in the venue-defined415theory and systems categories, whose flat baseline makes a later acceleration easier to date. This reading is416descriptive and is marked for review (\pending{H-4: two-wave reading of the Bai--Perron dates}).417418\input{tables/tab_p9_altdates}419420Table~\ref{tab:p9-altdates} completes the picture with the level change at each of the five pre-specified421dates. For the AI categories the o1 column (October 2024) carries the largest positive level changes of any422date, which is the second, 2024 kink seen by the Bai--Perron procedure in a different form; the GPT-3 column423(June 2020) carries negative level changes for most categories, which is the pandemic-year irregularity of424the series rather than anything specific to the event. The five columns are the same series with the break425moved, and the table is read across a row as a profile of when a category's level departs from its fitted426trend, not as five tests.427428\subsection{Hypothesis H3: structural breaks}429\label{sec:r-h3}430431H3 states that Bai--Perron tests on category-level monthly series locate at least one break in the window432from October 2022 to June 2023 for Large Language Models, Generative AI, AI Agents and AI Safety, and that433for Formal Verification, Computational Complexity, Operating Systems and Computer Architecture no break is434detected in that window at the five per cent level after placebo correction. Table~\ref{tab:p9-h3} assembles435the evidence for the eight named categories.436437\input{tables/tab_p9_h3}438439The first clause is \emph{not supported}. None of the four AI categories has a Bai--Perron break in the440window: Large Language Models breaks in July 2022, Generative AI in November 2021, AI Agents in July 2022441(with a second break in July 2024) and AI Safety in June 2021 and September 2022. All four break442\emph{before} the window, which is the opposite of the error the hypothesis guarded against (a break too late443to be attributed to the boundary): their series accelerated during 2022, in the months of the diffusion-model444and instruction-tuning literature, and the reference date falls inside an acceleration that had already445begun. The PELT dates of these categories fall in late 2023, after the window, for the step-model reason446given above. The second clause is \emph{opposite} for two of its four categories: Computational Complexity447and Computer Architecture have Bai--Perron breaks in April and May 2023, inside the window, and Formal448Verification and Operating Systems break outside it (February 2022 and July 2024; September 2023). The449``five per cent level after placebo correction'' part of the clause cannot be applied as written, because450the Bai--Perron dates carry no $p$-value in the implementation (Section~\ref{sec:m-bp}); the placebo451statistic that exists is the ITS slope placebo, which exceeds five per cent for all eight categories, so that452by that statistic no category, including the four AI categories, has a December 2022 change that stands out453from its own history. The hypothesis, in short, predicted that the boundary would separate categories with454and without a break, and the data show breaks on both sides of the boundary in both groups, earlier in the455AI categories and later in the theory and systems categories. The verdict is \emph{not supported} for the456first clause and \emph{opposite} for the second, with the uncertainty that the Bai--Perron dates have no457sampling distribution attached and move by several months when the truncation month changes458(Table~\ref{tab:p9-total-robustness} for the total series). \pending{H-4: interpretation of the pre-window459breaks of the AI categories}.460461\section{RQ3: influential contributions and the distribution of attention}462\label{sec:r-rq3}463464RQ3 asks which publications introduced the most influential methodological or theoretical contributions. The465quantitative part of the answer is the citation analysis of Part~VIII and the landmark set of Part~IV; the466qualitative part, the identification of what each landmark contributed, belongs to the specialist reviews of467Parts~V--VII and to checkpoint H-3, and this section reports only what the counts show.468469\subsection{Fixed-window citation statistics}470471Table~\ref{tab:citation_windows} and Figure~\ref{fig:citation_inequality} give the fixed-window statistics by472publication year. For the cohorts with a complete three-year window (2018 to 2023), the mean number of473citations received within the window rises from \CitationWindowsYearlyTwoZeroOneEightMeanCThreeSix{} for the4742018 cohort to \CitationWindowsYearlyTwoZeroTwoTwoMeanCThreeSix{} for 2022 and475\CitationWindowsYearlyTwoZeroTwoThreeMeanCThreeSix{} for 2023, while the share of works uncited after three476years falls from \CitationWindowsYearlyTwoZeroOneEightShareUncitedThreeSix{} to477\CitationWindowsYearlyTwoZeroTwoTwoShareUncitedThreeSix{} and rises again to478\CitationWindowsYearlyTwoZeroTwoThreeShareUncitedThreeSix{} for the 2023 cohort, whose window closes at the479end of 2025 and is therefore the first to feel the reference-indexing lag. The Gini coefficient of three-year480citations, the inequality statistic of hypothesis H5, falls monotonically through the baseline cohorts, from481\CitationWindowsYearlyTwoZeroOneEightGiniCThreeSix{} (2018) to482\CitationWindowsYearlyTwoZeroTwoTwoGiniCThreeSix{} (2022), and rises to483\CitationWindowsYearlyTwoZeroTwoThreeGiniCThreeSix{} for the 2023 cohort. The baseline decline is driven by the484falling share of uncited works, which is the largest contributor to a Gini of this kind485\citep{nielsen2021inequality}; the 2023 reversal coincides with the return of the uncited share and is, on the486evidence available, a truncation effect rather than a change in the concentration of attention. The 99th487percentile of three-year citations is almost flat across cohorts (\CitationWindowsYearlyTwoZeroOneEightPNineNineCThreeSix{}488in 2018, \CitationWindowsYearlyTwoZeroTwoThreePNineNineCThreeSix{} in 2023), so the top of the distribution489did not pull away from the body within fixed windows.490491The field-weighted citation impact supplied by OpenAlex, whose normalisation is the database's own and not492the study's (Section~\ref{sec:m-citations}), has a mean near \CitationWindowsYearlyTwoZeroOneEightMeanFwci{}493in 2018 and \CitationWindowsYearlyTwoZeroTwoThreeMeanFwci{} in 2023 and a median that rises from494\CitationWindowsYearlyTwoZeroOneEightMedFwci{} to \CitationWindowsYearlyTwoZeroTwoTwoMedFwci{} over the495baseline; a mean well above one for a field whose works are normalised by field means that the496computer-science corpus is cited above the average of the OpenAlex fields its works are assigned to, and497the gap between mean and median is the skewness that the Gini measures.498499\subsection{The most cited works and the landmark set}500501Table~\ref{tab:top_cited} lists the ten most cited works of each publication year by three-year window502citations, and Section~\ref{sec:net-citations} of Part~VIII discusses it. The list contains the works that a503reader of the period would expect at the top of computer science (contextualised word representations,504BERT, the Vision Transformer, the Swin Transformer, latent diffusion, masked autoencoders, the scientific505Python stack) alongside records that are duplicates of canonical works misdated to a later year (the506transformer paper under 2025, the batch-normalisation paper under 2024, the original GAN paper under 2023,507the Bitcoin white paper under 2019 and 2022) and records whose three-year count exceeds their lifetime508count or whose lifetime count is zero, which are database errors. The ranking of influential works by raw509citations is therefore conditional on an audit that Part~VIII specifies and that has not been performed,510and this part does not derive a list of influential contributions from it.511512The landmark set is the study's instrument for that purpose. Selected by field-normalised impact within513category and year with per-area quotas (Section~\ref{sec:m-kg}), it contains \SelectionLandmarks{} works514drawn from a high-priority set of \SelectionHighPriority{}; Table~\ref{tab:p9-landmarks} gives its515composition by area and year. The quotas make the set broader than a citation ranking would be: the AI and516Machine Learning area holds the largest quota but less than a quarter of the set, and the theory area,517whose works are cited least, holds the smallest. By year the set is roughly balanced over 2018--2025 and518largest for 2026, which reflects the selection's reliance on the database's field-weighted impact for the519most recent cohort, where citation counts are smallest and the normalisation least stable; the 2026520landmarks should be treated as provisional. \SelectionShareWithAbstractLandmarks{} of the landmarks have an521abstract and \SelectionShareOaLandmarks{} have an open-access location, which bounds what the knowledge-graph522extraction and the specialist reviews can work from. The identification of the methodological or523theoretical contribution of each landmark is the object of the specialist reviews and of checkpoint H-3524and is not reported here (\pending{H-3: landmark list and specialist reviews}).525526\input{tables/tab_p9_landmarks}527528\subsection{Hypothesis H5: concentration}529\label{sec:r-h5}530531H5 states that citation inequality (the Gini of 24-month citations within year) is higher in the post period532than in the baseline for AI categories, and that venue concentration (HHI by source) is stable or lower533because of preprint growth. The citation component is stated for the AI categories, and the per-category534citation-window statistics are not among the result files; the verdict on that component is535\emph{inconclusive} (\pending{macro: per-category Gini of fixed-window citations, results/bibliometrics;536not produced by the current pipeline}). The aggregate counterpart, read from Table~\ref{tab:citation_windows}537with the 36-month Gini that the pipeline computes in place of the 24-month one, falls through the baseline538and rises for the single complete post-period cohort, a movement that Section~\ref{sec:net-citations}539attributes to the uncited share and to right truncation; it is not evidence of higher concentration. The540venue component is examined in Section~\ref{sec:net-venues}: the Herfindahl index with arXiv counted as a541source rises from \VenueHhiYearlyTwoZeroOneEightHhi{} in 2018 to \VenueHhiYearlyTwoZeroTwoFiveHhi{} in 2025542and the top-source share from \VenueHhiYearlyTwoZeroOneEightTopOneShare{} to543\VenueHhiYearlyTwoZeroTwoFiveTopOneShare{}, because arXiv is the top source and its share grows; the544protocol anticipated this mechanism and predicted stability or a fall once preprints are treated separately,545and the index without the preprint type is not among the result files. The venue component is therefore546\emph{opposite} on the index as computed and \emph{inconclusive} on the index the hypothesis intended547(\pending{macro: venue HHI excluding repository sources}). The number of distinct sources grows in every548year (Table~\ref{tab:venue_hhi}), so the non-arXiv venue landscape widens rather than concentrates.549550\section{RQ4: citation networks and knowledge flows}551\label{sec:r-rq4}552553\subsection{Flows between fields}554555The part of RQ4 that concerns the flow of knowledge between computer science and other fields is answered in556Section~\ref{sec:net-flows} of Part~VIII from the resolved references of the eligible works557(Table~\ref{tab:reference_coverage}, Table~\ref{tab:field_flows}, Figure~\ref{fig:field_inflows}). The share558of references that stay inside the corpus rises through the window as the corpus itself grows, and the share559pointing to the largest other OpenAlex fields (medicine, engineering, mathematics, physics, the social560sciences) falls, with no change of pace at the boundary; the age distribution of cited works shifts toward561younger work in every year (Table~\ref{tab:cited_age}, Figure~\ref{fig:cited_age}), again without a visible562inflection. Those results are not repeated here. This section reports the flows \emph{within} computer563science, between the areas and categories of the ontology, which depend on the classification and were564not available to Part~VIII.565566\subsection{Flows between areas}567568Figure~\ref{fig:area-flow-matrix} shows the $8 \times 8$ area flow matrices of equation~\eqref{eq:flow},569row-normalised, for the baseline and post spans, and Table~\ref{tab:p9-area-flow-change} gives their570difference in percentage points. The diagonal dominates both matrices: in every area a majority of internal571references stay within the area, from about three fifths in Security, Privacy and Trust to well over four572fifths in Perception, Language, Robotics and Interaction, which cites itself more than any other area does573and whose self-share does not fall in the post period. The off-diagonal structure is a hub-and-spoke pattern574centred on two areas: every area sends a larger share of its references to AI and Machine Learning in the575post span than in the baseline (by between one and two and a half points for the systems, security,576software, theory and computational-science areas), and the AI area itself sends a larger share to577Perception, Language, Robotics and Interaction and a smaller share to itself. The largest movements in the578table are the falls of the self-shares of the AI, Data, Software and Security areas, each of about three579points, matched by rises of their references to the AI area and, for the AI area itself, to the vision and580language categories. The Hardware area is the exception that moves least: its row changes by less581than a point in every cell.582583\begin{figure}[htbp]\centering584\includegraphics[width=0.95\textwidth]{fig_area_flow_matrix.pdf}585\caption{Citation-flow matrices between areas (row shares of internal references, citing area in rows) for586the baseline and post spans. Source: \texttt{results/graph/area\_flows\_yearly.csv}.}587\label{fig:area-flow-matrix}588\end{figure}589590\input{tables/tab_p9_area_flow_change}591592The matrices are built from internal references only, that is, references whose cited work is in the corpus593and has a primary category, and the number of such references grows by an order of magnitude over the window594(Table~\ref{tab:p9-modularity}, second column) because works of 2018 could cite few corpus works, all of them595from 2018, whereas works of 2025 can cite seven years of the corpus. The 2018 matrix is therefore computed on a596small and peculiar set, and the comparison of spans is best read from 2019 onward.597598\subsection{Cross-area shares and modularity}599600Table~\ref{tab:p9-cross-area} and Figure~\ref{fig:cross-area} report the statistic behind H4, the cross-area601share of equation~\eqref{eq:cross-area}, by citing area and year. The areas differ by a factor of three in602their propensity to cite outside themselves: the AI area sends about half of its internal references to other603areas (most of them to vision and language), the perception area about a sixth, and the remaining six604between a quarter and two fifths. Over time the AI area's cross-area share rises through the baseline, from6052018 to 2022, and is flat from 2022 onward; the Data, Security, Software and Theory areas rise slowly and606steadily through the whole window; the Hardware area is flat; and the Perception area falls from 2018 to6072019 and is flat thereafter. The aggregate share, in the penultimate rows of the table, rises from 2022 to6082025 after being flat through the baseline. Part of that rise is composition: references made by AI-area609works, which are the most boundary-crossing, form a growing share of all references. The composition-adjusted610aggregate, which holds each area's weight at its baseline share of internal references, rises by less than611the raw one over 2022--2025 but still rises, so the boundary-crossing propensity of the areas themselves612increased slightly in the post period, by an amount that is about half of the raw increase.613614\begin{figure}[htbp]\centering615\includegraphics[width=0.95\textwidth]{fig_cross_area.pdf}616\caption{Cross-area reference shares by citing area and year, and modularity of the category citation-flow617network at the category and area partitions. Sources: \texttt{results/graph/cross\_area\_share\_yearly.csv},618\texttt{modularity\_yearly.csv}.}619\label{fig:cross-area}620\end{figure}621622\input{tables/tab_p9_cross_area}623624The modularity of the yearly flow network (Table~\ref{tab:p9-modularity}) summarises the same information at625the two partitions. $Q_{\mathrm{area}}$ falls in every year from 2018 to 2024 and is flat in 2025, with the626largest single-year falls in 2020--2021 and in 2023; $Q_{\mathrm{cat}}$ is flat through 2022 and falls627afterwards; and the uncorrected self-share of citations within the citing category is flat through the628baseline and falls from 2023. The category partition and the area partition therefore tell slightly629different stories: the assortativity of citations by \emph{area} declined throughout the window, whereas630the assortativity by \emph{category} held until the reference year and declined afterwards, which is the631signature of categories within the same area (Large Language Models and Natural Language Processing, Computer632Vision and Multimodal AI) citing each other more in the post period. Because the partition is fixed by the633classifier rather than optimised, these values are descriptive indices of assortativity and not community634structure, and their decline is consistent with both a real blurring of category boundaries and a drift of635the classifier's own boundaries as the vocabulary of the literature changed (Section~\ref{sec:m-networks}).636637\input{tables/tab_p9_modularity}638639\subsection{Hypothesis H4: convergence}640\label{sec:r-h4}641642H4 states that the share of citations crossing area boundaries (systems to AI, security to AI, HCI to AI,643and so on) rises in the post period relative to the baseline, and that the modularity of the citation-flow644network by area decreases. Both clauses are \emph{supported} in direction. The cross-area shares of the645systems, security, software and theory areas rise in the post period, and the specific flows the hypothesis646names, from each of these areas toward AI and Machine Learning, are the cells of647Table~\ref{tab:p9-area-flow-change} with the largest positive changes; the area modularity is lower in every648post-period year than in any baseline year. The support is qualified in three ways. The rises are small,649of the order of two to four percentage points over the window for the areas concerned, and they continue650trends that are visible within the baseline; the aggregate increase is about half composition and half651behaviour; and the Perception area, which is the largest and contains the HCI category named in the652hypothesis, does not increase its cross-area share at all, so that ``HCI to AI'' is not supported at the area653level at which the statistic is computed (the category-level flows of Table~\ref{tab:p9-llm-partners} and of654the co-classification analysis in Section~\ref{sec:r-rq5} show HCI among the categories most often paired with655Large Language Models, which is a different measurement). The modularity decline is monotone across the656boundary rather than concentrated at it, so the hypothesis is supported as a statement about the post period657relative to the baseline and not as a statement about a change at the reference date. The classifier658caveat applies with full force: a change in how the classifier partitions the literature would produce the659same movements, and the semantic-level check of H4 is pending (Section~\ref{sec:r-rq5}).660661\section{RQ5: foundation models and the boundaries between disciplines}662\label{sec:r-rq5}663664RQ5 asks whether the emergence of foundation models changed the relationship between previously distinct665computer-science disciplines. Three measurements bear on it: the citation flows of Section~\ref{sec:r-rq4},666the co-classification of works across categories, and the semantic distances between category centroids in667the embedding space. The first was reported above; the second is reported here; the third depends on the668semantic atlas, which was not available when this part was written, and its subsections are reserved.669670\subsection{Co-classification}671672Table~\ref{tab:p9-cooccurrence} lists the fifteen most frequent pairs of categories carried jointly by a673work in its label set, for the baseline and post spans. The baseline list is dominated by pairs that674describe the standing structure of the field: Computer Vision with Deep Learning Architectures, Computer675Networks with Cybersecurity and with Edge Computing, Applied Cryptography with Cybersecurity, Computer676Architecture with Distributed Computing, Cloud with Distributed Computing. Large Language Models with677Natural Language Processing is the third pair of the baseline list, which is a reminder that the category678is defined by a vocabulary (language models, pre-training, transformers) that predates ChatGPT by several679years. The post-period list keeps the vision and networking pairs at the top but is otherwise reorganised680around the generative-model categories: Large Language Models with Natural Language Processing moves to681second place with more than twice its baseline count; Computer Vision pairs with Foundation Models, with682Multimodal AI and with Generative AI; Deep Learning Architectures pairs with Foundation Models and with683Natural Language Processing; and Foundation Models with Large Language Models enters the list. Six of the684fifteen post-period pairs involve a category that did not appear in any baseline pair.685686\input{tables/tab_p9_cooccurrence}687688Table~\ref{tab:p9-llm-partners} looks at the same phenomenon from the side of the Large Language Models689category. Its partners in the post span are, after Natural Language Processing, Foundation Models, Deep690Learning Architectures, Computer Vision, Multimodal AI, Information Retrieval, Software Engineering, AI691Safety and Alignment, Machine Learning Theory, AI Agents, Generative AI, Cybersecurity and Human--Computer692Interaction; the ratio column shows that the pairings with Information Retrieval, Software Engineering,693Cybersecurity and Human--Computer Interaction, which are categories outside the AI area, grew by factors694comparable to or larger than the pairing with Natural Language Processing. These are the category-level695counterparts of the area flows of Section~\ref{sec:r-rq4}: works that carry the LLM label increasingly also696carry the label of an application or systems category, which is one mechanism by which the multi-label697series of those categories accelerated (Section~\ref{sec:r-its-categories}). The measurement is conditional698on the classifier in the strongest possible way, since co-classification is a property of the label set, and699on abstract availability, since a work without an abstract rarely carries more than one label; both rise in700the post period (Table~\ref{tab:p9-classification}), and part of the growth of every pair is attributable to701them.702703\input{tables/tab_p9_llm_partners}704705\subsection{Semantic distances between categories}706\label{sec:r-semantic}707708The protocol's semantic test of convergence compares the cosine similarity between category centroids in709the embedding space, estimated separately on baseline and post works, and reports the drift of each710category's centroid between the two periods (Section~\ref{sec:m-semantic}). The result files711\texttt{results/semantic/category\_similarity\_baseline.csv}, \texttt{category\_similarity\_post.csv} and712\texttt{category\_drift.csv} did not exist when this part was written. \pending{semantic atlas: insert the713category similarity comparison (baseline versus post, mean off-diagonal similarity, the pairs whose714similarity changed most) and the drift table when results/semantic/ is produced; the diversity indices of715Table~\ref{tab:diversity} are to be re-estimated with the semantic distance at the same time}.716717\subsection{Semantic clusters and their stability}718719The clustering of the embedded corpus into topics and the bootstrap stability of that clustering720(\texttt{results/semantic/clusters.csv}, \texttt{stability.json}) are likewise pending. \pending{semantic721atlas: number of clusters, share of works in noise, adjusted Rand index of the bootstrap replicates, and the722clusters that are new in the post period, from results/semantic/clusters.csv and stability.json}.723724\subsection{Reading of RQ5 on the available evidence}725726With two of the three measurements available, the answer to RQ5 is that the generative-model categories727have become the most connected part of the classification, both in what their works cite and in what other728labels their works carry, and that the areas of computer science that were most distinct from AI in the729baseline (systems, security, software, theory) now send a somewhat larger share of their references to it.730Whether this amounts to a change in the relationship between disciplines, or to the growth of one large731cross-cutting literature alongside disciplines that remain distinct, cannot be settled with citation and732label data: the self-shares of Table~\ref{tab:p9-area-flow-change} remain above three fifths in every area,733and the Perception area, the largest, is as self-contained in the post period as it was before. The734semantic measurement, when available, is the one that can discriminate between the two readings, because it735does not depend on the classifier's boundaries.736737\section{RQ6: institutions, countries and collaboration}738\label{sec:r-rq6}739740Part~VIII analyses the actor networks in full, and this section records the aggregate indicators that the741hypotheses require and the verdict on H6. The international co-authorship share, among works with at least742one resolved country, moves from \InternationalCollabYearlyTwoZeroOneEightShareIntl{} in 2018 to a maximum of743\InternationalCollabYearlyTwoZeroTwoZeroShareIntl{} in 2020 and declines in every subsequent year to744\InternationalCollabYearlyTwoZeroTwoFiveShareIntl{} in 2025 (Table~\ref{tab:collaboration},745Figure~\ref{fig:collaboration}). The share of works with a company affiliation, among works with any typed746affiliation, rises from \IndustryShareYearlyTwoZeroOneEightShareCompany{} in 2018 to a maximum of747\IndustryShareYearlyTwoZeroTwoOneShareCompany{} in 2021 and falls to748\IndustryShareYearlyTwoZeroTwoFiveShareCompany{} in 2025 (Table~\ref{tab:industry}). The country shares with749fractional counting (Table~\ref{tab:country_shares}, Figure~\ref{fig:country_shares}) show the share of750China rising through the window and the share of the United States falling, under both counting751conventions, as the continuation of a baseline trend; the joint output of the two countries falls in752absolute terms in the post period while both grow (Table~\ref{tab:country_network},753Section~\ref{sec:net-international}). The institutional hierarchy is stable: the institutions with the754largest post-period output are, with few exceptions, those with the largest baseline output755(Table~\ref{tab:top_institutions}), and the concentration of output across institutions does not increase756(Table~\ref{tab:institution_concentration}).757758\subsection{Hypothesis H6: geography and collaboration}759\label{sec:r-h6}760761H6 states that the share of works with industry affiliations and with China-affiliated authors rises in the762post period in AI categories, and that the international co-authorship share does not rise. The third clause763is \emph{supported}: the international share does not rise and in fact declines, with the composition764explanation that Part~VIII develops (the fastest-growing producers are the least internationalised). The765China clause is \emph{supported} in the aggregate, as a continuation of a trend that predates the boundary,766and \emph{inconclusive} for AI categories, because the category-specific country shares are not among the767result files (\pending{macro: China-affiliated share by category and period, results/bibliometrics}). The768industry clause is \emph{opposite} in the aggregate, where the company share peaks in 2021 and declines in769every post-period year, and \emph{inconclusive} for AI categories (\pending{macro: company-affiliated share by770category and period}). Two remarks qualify the opposite verdict. The company share is measured on works with771a typed affiliation, and the affiliation coverage of the corpus changes over the window772(Table~\ref{tab:collaboration}, last column), so a falling share can coexist with a rising count of773industry works; the count does rise (Table~\ref{tab:industry}). And the literature from which the774hypothesis was drawn measures industry participation at the frontier of machine learning research775\citep{ahmed2023industry}, whereas the corpus measures it over all of computer science, where the growth of776the post period is concentrated in categories and venues with low industry participation.777778\section{RQ7 and RQ8: breakthroughs, incremental gains and unresolved problems}779\label{sec:r-rq7}780781RQ7 asks which research areas demonstrate genuine breakthroughs versus incremental benchmark improvements,782and RQ8 which scientific problems remain unresolved despite rapid publication growth. Neither question has a783bibliometric answer, and the study does not pretend otherwise: the evidence for them is the knowledge-graph784extraction over the high-priority set (method, benchmark, dataset and claim nodes with provenance and785confidence, Section~\ref{sec:m-kg}) and the specialist reviews of Parts~V--VII, which are synthesised in786Part~X. At the time of writing the extraction had not produced result files that this part could tabulate,787and the specialist reviews were subject to checkpoint H-3. What the quantitative results of this part788contribute to the two questions is a set of constraints. The categories with the fastest post-period growth789(Section~\ref{sec:r-rq1}) and the strongest acceleration (Section~\ref{sec:r-its-categories}) are the ones in790which benchmark-driven incremental work is most plausible, because volume growth of this speed is not791compatible with a proportional growth in the number of distinct problems addressed; the categories whose792volume is flat but whose cross-area references to AI rise (Section~\ref{sec:r-rq4}) are the ones in which an793imported method is being applied to standing problems, which is where a breakthrough, if any, would register794as a change of method rather than of volume. The emerging-topic detector of Section~\ref{sec:r-rq10} provides795a third constraint: the topics it ranks highly are candidates for RQ7, and the topics with sustained volume796and declining citation acceleration are candidates for RQ8. These are hypotheses for Part~X, not findings797(\pending{H-3: knowledge-graph extraction counts by relation type, benchmark proliferation and method-adoption798series for the high-priority set, when produced}).799800\section{RQ9: openness, preprints and the evaluation of evidence}801\label{sec:r-rq9}802803Table~\ref{tab:p9-openness} assembles the openness indicators by publication year; Figure~\ref{fig:types-openness}804in Part~III shows the same series graphically. Three of them rise through the whole window and faster in the805post period. The share of eligible works linked to an arXiv e-print rises in every year, from806\VolumeYearlyTwoZeroOneEightNArxiv{} of \VolumeYearlyTwoZeroOneEightN{} works in 2018 to807\VolumeYearlyTwoZeroTwoFiveNArxiv{} of \VolumeYearlyTwoZeroTwoFiveN{} in 2025, roughly a doubling of the share808and a quadrupling of the count; the share of works whose surviving record is of preprint type rises more809steeply, because recent preprints have not yet been matched to a published version, and the 2026 value is810dominated by that effect. The open-access share rises from \VolumeYearlyTwoZeroOneEightNOa{} to811\VolumeYearlyTwoZeroTwoFiveNOa{} works, a rise in share of about twenty points over the window of which the812larger part occurs in the baseline; the abstract-availability share rises through 2024 and falls in 2025813and 2026 for records not yet enriched. The DBLP-indexed share falls in the post period, as814Section~\ref{sec:r-rq1} noted, and the conference share of the corpus is flat or slightly declining, which815means that the growth of preprints is not matched by a growth of conference publication in the indexed816record. The number of references per work (last column) is a metadata-completeness indicator rather than817an openness one and is discussed in Section~\ref{sec:r-rq1}.818819\input{tables/tab_p9_openness}820821\subsection{Hypothesis H7: preprints and openness}822\label{sec:r-h7}823824H7 states that the share of works with an arXiv version and the share with linked code rise in the post825period. The arXiv clause is \emph{supported}: the share rises in every post-period year and the post-period826share is above the baseline share by a wide margin (Table~\ref{tab:p9-periods}), with the caveat that the827post-period arXiv share is inflated by preprints that will later be merged with a published version and by828the growth of OpenAlex's coverage of arXiv. The code clause is \emph{inconclusive}: the pipeline does not829extract code links, and no result file measures the share of works with a linked repository830(\pending{macro: share of works with a code link by year; not produced by the current pipeline, scoped in831Phase 3 of the plan}). The evaluation-standards part of RQ9, which asks how benchmarks and reporting832practices evolved, is likewise outside what the bibliographic record measures and is addressed from the833knowledge-graph extraction in Part~X when available.834835\section{RQ10: early detection of emerging topics}836\label{sec:r-rq10}837838\subsection{Out-of-time performance of the detectors}839840Table~\ref{tab:p9-emerging-setup} summarises the evaluation design implemented in841Section~\ref{sec:m-emerging}: OpenAlex topics as units, features computed at a cutoff from data at or before842it, a label defined by top-decile growth with doubling over the following twelve or twenty-four months,843training cutoffs that end before the reference date and test cutoffs in 2023--2024. At the twelve-month844horizon the pooled test set has \EmergNTest{} (topic, cutoff) rows with a positive rate of845\EmergPosRateTest{}, lower than the training rate of \EmergPosRateTrain{}; at twenty-four months the rates846are \EmergtwofourPosRateTest{} and \EmergtwofourPosRateTrain{}. The positive rate is below the ten per cent847that a decile label would give because of the doubling condition, and it is lower in the test period than in848training because the corpus as a whole grew faster in 2023--2026, so that doubling against the trailing849window became harder for an individual topic.850851\input{tables/tab_p9_emerging_setup}852853Tables~\ref{tab:emerging_h12} and~\ref{tab:emerging_h24} in Appendix~F give the pooled metrics, and854Figure~\ref{fig:emerging-metrics} displays them. At the twelve-month horizon the gradient boosting model855attains an AUC of \EmergGbmAuc{}, the $z$-score baseline \EmergZscoreBaselineAuc{} and the logistic model856\EmergLogisticAuc{}; precision at fifty is \EmergGbmPrecisionAtFiveZero{} for the boosted model and857\EmergZscoreBaselinePrecisionAtFiveZero{} for the baseline, against a chance level of about the positive858rate. At the twenty-four-month horizon the ordering reverses: the $z$-score baseline attains an AUC of859\EmergtwofourZscoreBaselineAuc{} and a precision at fifty of \EmergtwofourZscoreBaselinePrecisionAtFiveZero{},860and both trained models fall below it (\EmergtwofourLogisticAuc{} and \EmergtwofourGbmAuc{}). The per-cutoff861values of Table~\ref{tab:p9-emerging-cutoffs} show that the pooled figures hide large variation: the862twelve-month AUC of the boosted model ranges over the eight test cutoffs from a little above six tenths to863near one, the near-perfect value belonging to the January 2024 cutoff, at which the baseline is also near864perfect and the logistic model near zero, a configuration that indicates a cutoff with very few positives865whose ranking is determined by one or two topics rather than a cutoff at which the models excel. Excluding866that cutoff, the boosted model exceeds the baseline at five of the seven remaining twelve-month cutoffs,867and the baseline exceeds both trained models at every twenty-four-month cutoff.868869\begin{figure}[htbp]\centering870\includegraphics[width=0.95\textwidth]{fig_emerging_metrics.pdf}871\caption{Out-of-time detection performance of the three detectors by horizon (AUC and precision at 20 and87250). Sources: \texttt{results/emerging/metrics\_h12.json}, \texttt{metrics\_h24.json}.}873\label{fig:emerging-metrics}874\end{figure}875876\input{tables/tab_p9_emerging_cutoffs}877878Three features of the design explain the pattern. The training set is small, with eight cutoffs at twelve879months and four at twenty-four, and the positives within it number in the dozens; a boosted model with three880hundred trees fitted to that many positives learns the idiosyncrasies of the 2020--2021 cutoffs, which the881reversal at twenty-four months, where training is smallest, bears out. The $z$-score is itself one of the882six features, so the trained models can at best refine a ranking that the baseline already provides, and at883the longer horizon the refinement overfits. And the label's doubling condition ties positives tightly to884recent abnormal growth, which is what the $z$-score measures, so that a longer horizon, over which abnormal885growth has more time to translate into doubling, favours the simplest signal. The logistic model's886performance below the baseline at both horizons, and below one half at some cutoffs, indicates that the887linear combination of standardised features is dominated by the size and age controls, which are888negatively associated with the label.889890\subsection{Hypothesis H8: detection}891\label{sec:r-h8}892893H8 states that the detector trained on cutoffs up to 2022 achieves an AUC above 0.70 and a precision at894fifty above 0.40 on test cutoffs in 2023--2024 against the baseline growth $z$-score rule, and that the895preregistered known-emergence set is detected with a median lead time of at least six months before each896topic's peak growth month. The first clause is \emph{not supported}. The best trained model's twelve-month897AUC, \EmergGbmAuc{}, is at the threshold to three decimals and marginally below it before rounding; its898precision at fifty is an order of magnitude below the threshold; and at twenty-four months the trained899models do not beat the baseline at all, so the comparison ``against the $z$-score rule'' is lost rather than900won. The AUC values say that the signals rank future emergers above non-emergers with a probability of about901seven in ten at twelve months and, through the baseline, of about eight in ten at twenty-four months, which902is informative; the precision values say that a list of fifty flagged topics at a given cutoff contains very903few of the topics that will double, which is the operationally relevant statement and the one the threshold904was set for.905906The second clause concerns the known-emergence set, and Section~\ref{sec:m-emerging} explains why it could907not be tested as written: the twelve developments do not map to OpenAlex topics, so the check actually run908applies the $z$-score rule to keyword series defined by regular expressions, and measures the lead of the909rule's first firing over the ex-post peak growth month. Figure~\ref{fig:known-set} shows the twelve series910with the first alarm, and Table~\ref{tab:known_set} the detection months, peak months and leads. The rule911fires for all \KnownSetNDetected{} concepts, with a positive lead for \KnownSetNPositiveLead{} of them and a912median lead of \KnownSetMedianLeadMonths{} months. The leads are longest for large language models913(\KnownLargeLanguageModelsLeadMonths{} months, with a detection in914\KnownLargeLanguageModelsDetectionMonth{} and a peak in \KnownLargeLanguageModelsPeakGrowthMonth{}) and for915reasoning models (\KnownReasoningModelsLeadMonths{} months, the regular expression for which includes916chain-of-thought prompting and therefore fires in \KnownReasoningModelsDetectionMonth{}, long before the917test-time-compute literature of 2024--2025 that gives the series its peak), and shortest for neural radiance918fields (\KnownNeuralRadianceFieldsLeadMonths{} months) and for mechanistic interpretability, whose lead is919negative (\KnownMechanisticInterpretabilityLeadMonths{} months): that series' fastest growth, in920\KnownMechanisticInterpretabilityPeakGrowthMonth{}, came from a base of a few works and the rule's volume921condition was not met until \KnownMechanisticInterpretabilityDetectionMonth{}. On the check as run, the922second clause is \emph{supported}: the median lead exceeds six months. The support is weak for three923reasons that the method section states and that bear repeating: the units are hand-written keyword series924rather than the detector's topics, the detector is the baseline rule rather than the trained model, and the925peak month is an ex-post quantity, so that a lead measured against it is not a lead that an observer could926have known at the time. The verdict on H8 as a whole is \emph{not supported} for the trained detector and927\emph{supported with reservations} for the lead-time clause on its substitute test.928929\begin{figure}[htbp]\centering930\includegraphics[width=0.95\textwidth]{fig_known_set.pdf}931\caption{Monthly keyword series for the twelve preregistered concepts with the $z$-score rule's first alarm.932Source: \texttt{results/emerging/known\_set\_monthly.csv}.}933\label{fig:known-set}934\end{figure}935936The scientific content of the exercise lies in what it rules out. A retrospective evaluation with a fixed9372026 topic vocabulary, a growth-only label and a handful of training cutoffs is the most favourable setting a938detector of this kind can be given short of look-ahead, and in it the trained models do not improve on a939one-line rule. Two of the mandate's seven candidate signals (semantic novelty, interdisciplinary connectivity)940and three others (methodological adoption, benchmark proliferation, open-source activity) are not in the941feature set (Section~\ref{sec:m-emerging}); the result is therefore a lower bound on what the full framework942could achieve, and an upper bound on what growth and citation signals alone achieve. \pending{H-4: whether943to describe the detector as a negative result for RQ10 or as a baseline for the semantic-cluster extension}.944945\section{Validation of the classification}946\label{sec:r-validation}947948Every category-level result of this part is conditional on the rule-based classifier of Part~IV, and the949protocol requires its validation on a stratified sample of two thousand works rated independently by two950raters, with Cohen's $\kappa$ between the raters, and the precision, recall and F1 of the assignment against951each rater and against their consensus, by category (Section~\ref{sec:m-validation}; checkpoint H-2). The952ratings of the two raters exist in \texttt{taxonomy/validation/}, but the metrics file953\texttt{taxonomy/validation/metrics.json} and the disagreement list had not been produced when this part was954written, so the validation cannot be reported here. \pending{taxonomy validation: insert the inter-rater955$\kappa$, the overall precision, recall and F1 of the assignment against the consensus, the number of956disagreements, and the categories with the lowest F1, from taxonomy/validation/metrics.json and957disagreements.csv; the generated Table~\ref{tab:validation} will exist once the file does}. Until then, the958reader should weight the category results by the known failure modes of the classifier (Section~\ref{sec:m-classification}):959works without an abstract are classified from venue and arXiv category alone, which favours the venue-defined960categories; the keyword-defined categories (the AI area, Sustainable Computing, Autonomous Scientific961Discovery) are sensitive to vocabulary drift; and the primary category of a multi-label work is decided by a962score whose ties are not random. The record-linkage precision estimated on the manually reviewed sample963(\LinkagesampleDoiAutoPrecision{} for DOI links, \LinkagesampleTitleYearAutoPrecision{} for title-and-year964links; Section~\ref{sec:m-linkprecision}) is the one validation statistic of the corpus that is available,965and it bounds the error introduced by linkage rather than by classification.966967\section{Summary of the hypotheses and of what remains uncertain}968\label{sec:r-summary}969970Table~\ref{tab:p9-hypotheses} collects the verdicts. The table is a reading aid and not a result file; each971verdict is argued in the section cited and carries the qualifications stated there.972973\begin{table}[htbp]\centering974\caption{Verdicts on the pre-specified hypotheses on the evidence of this part.}975\label{tab:p9-hypotheses}976\small977\begin{tabular}{p{0.07\textwidth}p{0.36\textwidth}p{0.17\textwidth}p{0.3\textwidth}}978\toprule979\textbf{H} & \textbf{Clause} & \textbf{Verdict} & \textbf{Where and why} \\980\midrule981H1 & Post-period monthly growth rate higher than baseline & supported & Sec.~\ref{sec:r-h1}; slope change ranks first among all candidate dates \\982   & Level shift at the boundary not distinguishable from placebo breaks & supported & placebo rank of the level near the bottom; acceleration dated to spring 2022 by Bai--Perron \\983   & Growth differences concentrated in a few categories & not supported & Sec.~\ref{sec:r-its-categories}; positive slope change in 37 of 40 categories \\984H2 & AI-category share rises in the post period & supported & Sec.~\ref{sec:r-h2}; driven by the language and generative categories; Deep Learning Architectures falls \\985   & A non-AI area declines relatively while growing absolutely & supported & three areas (data and distributed, theory, software) \\986H3 & Bai--Perron break in Oct.\ 2022--June 2023 for LLM, generative AI, agents, AI safety & not supported & Sec.~\ref{sec:r-h3}; all four break before the window (2021--2022) \\987   & No break in the window for formal verification, complexity, OS, architecture & opposite (two of four) & complexity and architecture break inside the window; no $p$-value attached to the dates \\988H4 & Cross-area citation shares rise & supported, small & Sec.~\ref{sec:r-h4}; about half composition; Perception area does not rise \\989   & Area modularity decreases & supported & monotone through the window, not concentrated at the boundary \\990H5 & Citation Gini higher in post period for AI categories & inconclusive & Sec.~\ref{sec:r-h5}; per-category windows not produced; aggregate movement is a truncation effect \\991   & Venue HHI stable or lower & opposite as computed, inconclusive as intended & index without repositories not produced \\992H6 & Industry share rises (AI categories) & opposite (aggregate), inconclusive (AI) & Sec.~\ref{sec:r-h6}; company share peaks in 2021 \\993   & China share rises (AI categories) & supported (aggregate), inconclusive (AI) & continuation of a baseline trend \\994   & International co-authorship share does not rise & supported & declines from 2020 \\995H7 & arXiv-version share rises & supported & Sec.~\ref{sec:r-h7}; inflated by unmatched recent preprints \\996   & Linked-code share rises & inconclusive & not measured \\997H8 & Trained detector AUC above 0.70 and P@50 above 0.40 versus the baseline & not supported & Sec.~\ref{sec:r-h8}; AUC at the threshold, P@50 an order of magnitude below; baseline wins at 24 months \\998   & Known-emergence set detected with median lead at least 6 months & supported with reservations & substitute keyword test, baseline rule, ex-post peak \\999\bottomrule1000\end{tabular}1001\end{table}10021003Four sources of uncertainty run through every verdict and are stated here once. The first is right1004truncation: the post period ends in an incomplete year, the reference-indexing lag removes a growing share1005of references from the most recent cohorts, and every citation-based statistic of 2024 and later is1006provisional; the structural-change analysis handles this by truncating at \ItsLastComplete{} and varying the1007truncation month, and the citation analysis by reporting only complete windows, but neither device recovers1008what the database has not yet indexed. The second is the boundary of the database: the growth of the corpus1009in the post period is concentrated in records outside DBLP's curation, and the study cannot separate growth1010in computer-science output from growth in OpenAlex's coverage of it \citep{culbert2025openalexcoverage,1011alperin2024openalex}; the DBLP-only records of Table~\ref{tab:dblp_only} and the strict core of1012\PrismaStrictCoreDblpAndCsPrimary{} works are the instruments for a sensitivity analysis that the current1013result files do not yet contain (\pending{macro: structural-change estimates on the strict core, as the1014protocol's sensitivity analysis}). The third is the classifier, whose validation is pending and whose1015boundaries are fixed in 2026 vocabulary and applied to 2018 works; every category-level and every flow1016statistic inherits it, and the semantic measurements that do not depend on it are the ones still to come.1017The fourth is the inferential status of the time-series results: the reference date is a marker, the1018estimates at it are descriptions of a kink in a series that was already bending, and the placebo and1019alternative-date analyses show that the kink is not tied to the month. What the evidence supports is a1020multi-year acceleration of the indexed output of computer science, concentrated in share terms in the1021language and generative-model categories but present in nearly all of them, accompanied by a modest increase1022in the citation traffic from the other areas toward AI and by a reorganisation of the co-classification1023structure around the foundation-model categories, and a continuation, without inflection, of the baseline1024trends in collaboration, geography and openness. What it does not support is a discrete change at the1025reference date in any series examined, a concentration of the acceleration in a few categories, or the1026ability of simple growth and citation signals to pick the emerging topics of 2023--2024 with useful1027precision. \pending{H-4: the summary reading of this paragraph}.1028