WP44: fixes to Part II and appendices A/B/C/E/H after critique (31 issues resolved)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Simon-Pierre Boucher committed 2 h ago (Nov 3, 2026) parent 280d691 Browse files

12 changed files +664 −433

modified manuscript/appendices/appA_taxonomy.tex +4 −4

		`@@ -392,7 +392,7 @@ Field & Name & Mapped topics \\`
392	392	\subsection{Coverage of the DBLP venue vocabulary}
393	393	\label{app:taxonomy-dblp}
394	394
395		−DBLP streams are the venue signal. The stream table from which the mapping was built lists every DBLP
	395	+DBLP streams are the venue signal. The counts quoted in this subsection and in Table~\ref{tab:appA-dblp-unmapped} are ontology-build statistics copied from the generated ``DBLP coverage'' section of \texttt{taxonomy/ONTOLOGY.md} (computed at build time from \texttt{dblp\_streams\_2018\_2026.csv}); they describe the mapping, are not corpus quantities, and are not regenerated by \texttt{tables/make\_numbers.py}. The stream table from which the mapping was built lists every DBLP
396	396	journal or conference series with at least one hundred records in 2018--2026 (3{,}569 streams); the ontology
397	397	maps 771 of them. Measured on the DBLP records of 2018--2026 and excluding CoRR (the arXiv mirror stream,
398	398	which carries no venue information), the mapped streams cover 2{,}017{,}910 of 3{,}319{,}884 records
		`@@ -672,7 +672,7 @@ are to be added when the corpus reveals them, through an amendment. And keyword`
672	672	mentions as well as on contributions, a failure mode discussed with the others in
673	673	Section~\ref{sec:m-classification}.
674	674
675		−The ontology was validated, after the classification run, on a stratified sample whose design is fixed in
	675	+The ontology is to be validated, after the classification run, on a stratified sample whose design is fixed in
676	676	\texttt{taxonomy/VALIDATION.md}: fifty eligible works with an abstract per rule-assigned primary category
677	677	(forty strata) plus two hundred works to which the rules assigned no category, drawn with seed~44; two
678	678	independent LLM raters that never see the rule-based assignment and receive only the definitions of
		`@@ -680,5 +680,5 @@ Table~\ref{tab:appA-leaves} with the title, abstract, venue and year of each wor`
680	680	as third rater on a 200-work subsample (checkpoint H-2). The pre-specified decision rule is that a category
681	681	with consensus $F_1 < 0.60$ triggers a mapping revision, logged as a protocol amendment, and a re-run of the
682	682	classification. The agreement statistics, the per-category precision and recall, the list of categories that
683		−met or failed the decision rule, and any amendment that resulted are reported in Part~IX; nothing about them
684		−is asserted in this appendix.
	683	+met or failed the decision rule, and any amendment that resulted are reported in Part~IX; the validation had
	684	+not been run when this appendix was written, and nothing about its outcome is asserted here.

modified manuscript/appendices/appB_search_and_sources.tex +36 −21

		`@@ -82,7 +82,7 @@ option resuming an interrupted transfer), up to five outer attempts per file`
82	82	with a linearly increasing pause, and the transfer was accepted only when the
83	83	byte count on disk equalled the content length in the manifest. The measured
84	84	single-stream throughput from the node was about 71~MB/s and the aggregate
85		−over eight streams about 110~MB/s, which fixed the worker count at eight and
	85	+over eight streams about 108~MB/s, which fixed the worker count at eight and
86	86	the wall time at one hour and forty-six minutes for the whole bucket
87	87	(\texttt{ingestion/openalex\_snapshot.py --workers 8}).
88	88
		`@@ -108,7 +108,7 @@ be applied later by the eligibility predicate rather than silently at`
108	108	ingestion. The retained records were re-compressed with Zstandard (level~6)
109	109	into one file per part, \texttt{works/cs/}\allowbreak\texttt{updated\_date=X\_\_part\_NNNN.jsonl.zst},
110	110	about 20~GB in total for
111		−\PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{}
	111	+\PrismaIdentifiedOpenalexPassOneCsTopicFilter{}
112	112	records. No text search of any kind was performed: the scope of the OpenAlex
113	113	layer is defined entirely by the topic classifier's field assignment, which is
114	114	the search strategy appropriate to a field-level mapping and whose
		`@@ -178,9 +178,13 @@ works indexed by DBLP that OpenAlex labels with no computer-science topic.`
178	178	Rather than accept OpenAlex's boundary, the pipeline reads the bucket a
179	179	second time in \emph{identifier-set mode}. The linkage script writes
180	180	the list \texttt{data/metadata/}\allowbreak\texttt{needed\_oa\_ids.csv} of OpenAlex identifiers to
181		−recover (\LinkNeededOaIdsSecondPassDblpDoi{} reached through a DBLP DOI and
182		−\LinkNeededOaIdsSecondPassArxivDoi{} through an arXiv DOI), and
183		−\texttt{openalex\_snapshot.py --id-list} streams every part file again,
	181	+recover, namely the works of the slim table outside the retained subset that
	182	+an unlinked DBLP DOI or an unlinked arXiv DOI reaches. The list is re-derived
	183	+at every linkage run; after the run on the union of both passes it holds only
	184	+the residue that the second pass left outside the subset
	185	+(\LinkNeededOaIdsSecondPassDblpDoi{} identifiers reached through a DBLP DOI
	186	+and \LinkNeededOaIdsSecondPassArxivDoi{} through an arXiv DOI). Given the
	187	+list, \texttt{openalex\_snapshot.py --id-list} streams every part file again,
184	188	keeping exactly the records whose identifier is in the set and writing no slim
185	189	rows. The identifier set is loaded in each worker process (macOS uses the
186	190	\texttt{spawn} start method, so module globals are rebuilt per worker), the
		`@@ -192,12 +196,13 @@ pass costs the same bandwidth as the first---about one hour and fifty minutes`
192	196	of streaming---because an identifier lookup cannot be pushed to the bucket;
193	197	the price was accepted because the alternative, fetching the identifiers one
194	198	by one through the API, would have exceeded the public budget by two orders of
195		−magnitude. The PRISMA counts of Part~III were generated from the first pass
196		−(\PrismaIdentifiedOpenalexTotalRecordsLoaded{} records loaded); the second
197		−pass adds its records to $\mathcal{W}_0$ when it completes and the counts are
198		−regenerated by the same scripts, so the identification total in the final
199		−PRISMA figure is \pending{PrismaIdentifiedOpenalexTotalRecordsLoaded after the second pass}
200		−rather than the first-pass value quoted here.
	199	+magnitude. The PRISMA counts of Part~III are generated from the union of both passes:
	200	+the first pass retained \PrismaIdentifiedOpenalexPassOneCsTopicFilter{}
	201	+records through the topic filter, the second added
	202	+\PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{} records reached through
	203	+DBLP or arXiv identifiers, and the identification total of the PRISMA figure
	204	+is their sum, \PrismaIdentifiedOpenalexTotalRecordsLoaded{} records loaded
	205	+into $\mathcal{W}_0$.
201	206
202	207	\subsection{OpenAlex API increment}
203	208	\label{sec:appB-increment}
		`@@ -236,10 +241,15 @@ from the ledger. Every page is stored verbatim as`
236	241	(\texttt{api\_increment/ledger.json}) records per slice the pages, records,
237	242	reported count, completion and truncation flags, and globally the number of
238	243	calls and the cost. At the time of writing the harvest had used the budget
239		−of several consecutive days and was not complete; its totals
240		−(\pending{IncrementApiCalls}, \pending{IncrementRecordsRetrieved},
241		−\pending{IncrementTruncatedSlices}) will be reported from the ledger when the
242		−last slice closes. The conversion step (\texttt{ingest\_api\_increment.py})
	244	+of several consecutive days and was not complete: \IncrementApiCalls{} calls
	245	+had retrieved \IncrementRecordsRetrieved{} records at a cost of
	246	+USD~\IncrementCostUsd{}, and \IncrementSlicesDone{} of the sixty-six
	247	+day-slices (thirty-three days times two strata) were closed. The number of
	248	+truncated slices is \IncrementTruncatedSlices{}: the \texttt{cs\_secondary}
	249	+stratum of 16~October 2026 reported a count nearly four times the
	250	+10{,}000-record paging limit, so that roughly a quarter of its records were
	251	+retrieved. The remaining slices will be reported from the ledger when they
	252	+close. The conversion step (\texttt{ingest\_api\_increment.py})
243	253	rewrites the pages into the same \texttt{.jsonl.zst} part format as the
244	254	snapshot filter (\texttt{works\_pass3/}), skipping any work whose identifier
245	255	is already in the pass-1 or pass-2 tables and writing an
		`@@ -294,7 +304,7 @@ parsed; counts by type and by year for the entire archive are written to`
294	304	\texttt{dblp\_parse\_report.json}, and the records of 2018--2026
295	305	(\LinkDblpTwoZeroOneEightTwoZeroTwoSix{} after the type restriction of the
296	306	linkage step) are the ones that enter the corpus. The total number of
297		−publication records in the dump is \pending{DblpParsedRecords}. A second
	307	+publication records in the dump is \DblpParsedRecords{}. A second
298	308	script (\texttt{dblp\_streams.py}) lists the streams with at least one hundred
299	309	records of type article or inproceedings in 2018--2026, with their first and
300	310	last years and a sample journal or booktitle name; this list
		`@@ -362,7 +372,7 @@ booktitle, journal, month, year, address, publisher, URL, DOI, pages,`
362	372	abstract, volume, number); braces inside values are removed and whitespace
363	373	normalised. Entries of type \texttt{comment}, \texttt{string} and
364	374	\texttt{preamble} are skipped. The export is released under CC~BY~4.0. The
365		−number of entries parsed is \pending{AclParsedEntries}; the linkage step uses
	375	+number of entries parsed is \AclParsedEntries{}; the linkage step uses
366	376	only the \textit{inproceedings} and \textit{article} entries
367	377	(\LinkAclRecords{}), of which \LinkAclTwoZeroOneEightTwoZeroTwoSix{} fall in
368	378	2018--2026. The Anthology's role is a coverage check for natural-language
		`@@ -445,7 +455,7 @@ No. & Item & This study \\`
445	455	12 & Updates & One update by API increment to 2026-11-02; no further update is planned for this version. \\
446	456	13 & Dates of searches & OpenAlex snapshot released 2026-10-14, read 2026-11-02; DBLP dump 2026-11-02; arXiv harvest 2026-11-02 to 03; ACL export 2026-11-02; API increment from 2026-11-03. \\
447	457	14 & Peer review of the search & No librarian review; the strategy is preregistered in \texttt{protocol/PROTOCOL.md} (P3) and audited by the independent agents of Phase~8. \\
448		−15 & Total records & OpenAlex \PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{} (pass 1); DBLP \PrismaIdentifiedDblpTwoZeroOneEightTwoZeroTwoSix{}; arXiv \PrismaIdentifiedArxivCsTwoZeroOneEightTwoZeroTwoSix{}; ACL \PrismaIdentifiedAclTwoZeroOneEightTwoZeroTwoSix{} (all 2018--2026). \\
	458	+15 & Total records & OpenAlex \PrismaIdentifiedOpenalexTotalRecordsLoaded{} (\PrismaIdentifiedOpenalexPassOneCsTopicFilter{} in pass 1, \PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{} in pass 2); DBLP \PrismaIdentifiedDblpTwoZeroOneEightTwoZeroTwoSix{}; arXiv \PrismaIdentifiedArxivCsTwoZeroOneEightTwoZeroTwoSix{}; ACL \PrismaIdentifiedAclTwoZeroOneEightTwoZeroTwoSix{} (all 2018--2026). \\
449	459	16 & Deduplication & Deterministic record linkage across sources (Section~\ref{sec:appB-linkage}) and preprint--publication merging within OpenAlex (Part~III, Part~IV); OpenAlex works are the counting unit; \PrismaExcludedEFiveDuplicate{} preprint members of merged pairs are flagged E5. \\
450	460	\bottomrule
451	461	\end{tabularx}
		`@@ -498,8 +508,8 @@ E3 & \texttt{e3\_xpac} & \texttt{is\_xpac} and not \textit{in\_dblp} & \PrismaEx`
498	508	E4 & \texttt{e4\_no\_title} & title null or empty after trimming & \PrismaExcludedEFourNoTitle{} \\
499	509	E5 & \texttt{e5\_duplicate} & $w$ is the preprint member of a surviving preprint--published pair (Part~IV, eq.~\eqref{eq:duppair}) & \PrismaExcludedEFiveDuplicate{} \\
500	510	E6 & \texttt{e6\_retracted} & \texttt{is\_retracted} (flag only; not part of eligibility) & \PrismaExcludedESixRetracted{} \\
501		−E7 & \texttt{e7\_generic\_deposit} & primary source name $\in \mathcal{G}$ and not \textit{in\_dblp} and no linked arXiv record & \pending{PrismaExcludedESevenGenericDeposit} \\
502		−core & \texttt{cs\_core} & \textit{in\_dblp} or \texttt{cs\_primary} or (\texttt{cs\_any} and linked arXiv primary category begins with \texttt{cs.}) & \PrismaCsCoreEligible{} eligible \\
	511	+E7 & \texttt{e7\_generic\_deposit} & primary source name $\in \mathcal{G}$ and not \textit{in\_dblp} and no linked arXiv record & \PrismaExcludedESevenGenericDeposit{} \\
	512	+core & \texttt{cs\_core} & \textit{in\_dblp} or \texttt{cs\_primary} or linked arXiv primary category begins with \texttt{cs.} (amendment A-006) & \PrismaCsCoreEligible{} eligible \\
503	513	pen. & \texttt{cs\_penumbra} & \texttt{cs\_any} and not \texttt{cs\_core} & \PrismaCsPenumbraEligibleOtherwise{} otherwise eligible \\
504	514	\midrule
505	515	--- & \texttt{eligible} & $\neg$E1 $\wedge$ $\neg$E2 $\wedge$ $\neg$E3 $\wedge$ $\neg$E4 $\wedge$ $\neg$E5 $\wedge$ $\neg$E7 $\wedge$ core & \PrismaCsCoreEligible{} \\
		`@@ -521,7 +531,12 @@ The PRISMA counts are computed twice, marginally (the column of`
521	531	Table~\ref{tab:appB-codes}) and sequentially in the order E1, E2, E3, E4, E5,
522	532	E7, and the script asserts that the loaded total equals the sum of the
523	533	sequential exclusions and the after-exclusion count
524		−(\PrismaAfterExclusions{}); the macro set of Part~III reports both. The E6 and
	534	+(\PrismaAfterExclusions{}); the macro set of Part~III reports both. Since
	535	+amendment A-006 the script also asserts that the after-exclusion count equals
	536	+the core plus the penumbra plus the records that belong to neither set
	537	+(\PrismaNeitherCoreNorPenumbraAfterExclusions{}, works with no
	538	+computer-science topic reached only through a DOI and not carrying an arXiv
	539	+\texttt{cs.} primary category). The E6 and
525	540	penumbra rows are not exclusions and do not enter the sum. The numbering
526	541	conflict between the protocol's E7 (penumbra) and the code's E7
527	542	(generic deposit) is resolved by amendment A-005 in favour of the code.

modified manuscript/appendices/appC_derivations.tex +19 −11

		`@@ -47,8 +47,10 @@ the two parametrisations are related by $\beta_2^{\mathrm{unc}} = \beta_2 -`
47	47	text. The eleven month dummies are linearly independent of the constant
48	48	because January is omitted; the full-column-rank condition requires at least
49	49	one observation in every month of the year, satisfied by any series of
50		−twelve months or more, and at least one post-event month with $t > t_e$,
51		−guaranteed by the driver's rule that twelve months follow every event.
	50	+twelve months or more, and at least two post-event months with $t > t_e$
	51	+(with a single one the centred post-trend column is identically zero and
	52	+$\beta_2$ and $\beta_3$ are not separately identified), guaranteed by the
	53	+driver's rule that twelve months follow every event.
52	54	The ordinary least-squares estimator is $\hat\beta = (X^{\top}X)^{-1}
53	55	X^{\top}y$ and the residuals $\hat\varepsilon = y - X\hat\beta$.
54	56
		`@@ -83,8 +85,8 @@ to be positive semi-definite. The implementation calls`
83	85	$L = 6$ and $w_\ell = 1 - \ell/7$ as stated in Part~IV, without the
84	86	small-sample degrees-of-freedom correction and with $p$-values from the
85	87	normal approximation to the $t$-ratio, which is the library's default for
86		−robust covariances. With $n$ of the order of one hundred and fourteen
87		−regressors, the difference between the normal and the $t_{n-15}$ reference
	88	+robust covariances. With $n$ of the order of one hundred and fifteen
	89	+columns in $X$, the constant included, the difference between the normal and the $t_{n-15}$ reference
88	90	distribution is immaterial at the magnitudes of the $t$-statistics reported;
89	91	what matters is the placebo rank of the next paragraph, which does not
90	92	depend on any reference distribution.
		`@@ -757,9 +759,17 @@ training and test cutoffs. Every quantity used to fit a model---features,`
757	759	labels and the standardisation constants of the scaler, which is fitted on
758	760	training rows only---is $\mathcal{F}_{T^{\mathrm{tr}}_{\max} + h}$-measurable
759	761	with $T^{\mathrm{tr}}_{\max} = \max \mathcal{T}_{\mathrm{tr}}$. The
760		−implementation sets $T^{\mathrm{tr}}_{\max}$ to December 2021 for $h = 12$
761		−and to December 2020 for $h = 24$, so that $T^{\mathrm{tr}}_{\max} + h$ is
762		−December 2022 in both cases, and the first test cutoff is January 2023.
	762	+implementation takes as candidate cutoffs every third month of the panel,
	763	+which begins in January 2018 (January, April, July and October of each year),
	764	+and admits as training cutoffs those from December 2019 up to December 2021
	765	+for $h = 12$ and up to December 2020 for $h = 24$; the realised training
	766	+cutoffs therefore run from January 2020 to October 2021 ($h = 12$) or to
	767	+October 2020 ($h = 24$), so that $T^{\mathrm{tr}}_{\max} + h$ is October
	768	+2022 in both cases, and the first test cutoff is January 2023 (the last is
	769	+October 2024 at most, subject to the censoring condition below). The realised
	770	+lists are written to \texttt{results/emerging/metrics\_h12.json} and
	771	+\texttt{metrics\_h24.json} under \texttt{train\_cutoffs} and
	772	+\texttt{test\_cutoffs}.
763	773	Hence
764	774	\begin{equation}
765	775	T^{\mathrm{tr}}_{\max} + h \;<\; \min \mathcal{T}_{\mathrm{te}},
		`@@ -770,10 +780,8 @@ that precedes every test feature, let alone every test label, which closes`
770	780	at $\max \mathcal{T}_{\mathrm{te}} + h \le M$. Condition \eqref{eq:appC-gap}
771	781	is the formal content of ``out of time'' in the sense of
772	782	\citet{bergmeir2012crossval}; it is stricter than the plan's wording, which
773		−allowed training cutoffs through 2022, and stricter for $h = 24$ than the
774		−description of Part~IV, which gives December 2021 as the last training
775		−cutoff for both horizons---the code's rule is the one that guarantees
776		−\eqref{eq:appC-gap}, and the manuscript follows the code. The test cutoffs
	783	+allowed training cutoffs through 2022; the code's rule is the one that
	784	+guarantees \eqref{eq:appC-gap}, and Part~IV describes the same rule. The test cutoffs
777	785	are further restricted to those with $T + h \le M$, so that no test label is
778	786	right-censored.
779	787

modified manuscript/appendices/appE_data_dictionary.tex +2 −2

		`@@ -127,7 +127,7 @@ Column & Type & Definition & Source \\`
127	127	\bottomrule
128	128	\endlastfoot
129	129	\texttt{key} & string & DBLP record key, unique & element attribute \texttt{key} \\
130		−\texttt{rec\_\allowbreak type} & string & Record type: article, inproceedings, proceedings, book, incollection, phdthesis, mastersthesis & element name \\
	130	+\texttt{rec\_\allowbreak type} & string & Record type: article, inproceedings, proceedings, book, incollection, phdthesis, mastersthesis, data & element name \\
131	131	\texttt{mdate} & string & Last modification date of the record & attribute \texttt{mdate} \\
132	132	\texttt{publtype} & string & Publication sub-type (e.g.\ informal, survey, withdrawn); null if absent & attribute \texttt{publtype} \\
133	133	\texttt{title} & string & Title, inner markup flattened to text & \texttt{<title>} \\
		`@@ -250,7 +250,7 @@ Column & Type & Definition & Source \\`
250	250	\texttt{title} & string & Title; \texttt{display\_\allowbreak name} when \texttt{title} is absent & \texttt{title}, \texttt{display\_\allowbreak name} \\
251	251	\texttt{publication\_\allowbreak year} & int32 & Publication year & \texttt{publication\_\allowbreak year} \\
252	252	\texttt{publication\_\allowbreak date} & string & Publication date; OpenAlex sets \texttt{YYYY-01-01} when only the year is known (see \texttt{date\_\allowbreak precision}, Table~\ref{tab:appE-corpus}) & \texttt{publication\_\allowbreak date} \\
253		−\texttt{created\_\allowbreak date} & string & Date the record entered OpenAlex (used for indexing-lag diagnostics) & \texttt{created\_\allowbreak date} \\
	253	+\texttt{created\_\allowbreak date} & string & Date the record was created in OpenAlex's current data model; for most works it reflects a re-ingestion in the second half of 2025 rather than first indexing, and it could not be used for the indexing-lag profile (Part~III, Section~\ref{sec:data-dates}) & \texttt{created\_\allowbreak date} \\
254	254	\texttt{updated\_\allowbreak date} & string & Last update, date part only & \texttt{updated\_\allowbreak date[:10]} \\
255	255	\texttt{type} & string & OpenAlex work type & \texttt{type} \\
256	256	\texttt{type\_\allowbreak crossref} & string & Crossref type when the record has one & \texttt{type\_\allowbreak crossref} \\

modified manuscript/appendices/appH_reproducibility.tex +17 −13

		`@@ -22,15 +22,18 @@ this appendix and is updated whenever a step changes; where the two differ, the`
22	22	\label{app:repro-env}
23	23
24	24	Everything described in the runbook was executed on one node of the MacLustr cluster, M3U96b (Mac Studio,
25		−Apple M3~Ultra, 28~cores, 96~GB of unified memory, 103{,}079{,}215{,}104 bytes as recorded by the environment
26		−capture), in the working directory \texttt{\textasciitilde/\allowbreak wp44\_\allowbreak uqo}, under Python 3.12.15 in a virtual
	25	+Apple M3~Ultra, 96~GB of unified memory, 103{,}079{,}215{,}104 bytes as recorded by the environment
	26	+capture; the capture records no core count, the runbook header states 28 cores and the cluster inventory 32,
	27	+and the discrepancy is left on record), in the working directory \texttt{\textasciitilde/\allowbreak wp44\_\allowbreak uqo}, under Python 3.12.15 in a virtual
27	28	environment created with \texttt{uv} from \texttt{requirements.\allowbreak txt} and then frozen to
28	29	\texttt{reproducibility/\allowbreak environment\_\allowbreak M3U96b.\allowbreak txt} (92 lines: 88 pinned packages, the interpreter version, the
29	30	operating-system version string, the processor and the memory size). The operating-system version string
30	31	captured in that file is 26.4, whereas the runbook header states macOS~27.0; the discrepancy is recorded here
31	32	rather than resolved, since the node was updated between captures and neither value affects the computation.
32	33	Two further nodes were used after the incidents of Section~\ref{app:repro-incidents}: M5M36 for the second
33		−snapshot pass and M4M64c and M3U96a for embedding shards; they run the same locked environment.
	34	+snapshot pass and M4M64c and M3U96a for embedding shards; they were set up from the same
	35	+\texttt{requirements.txt}, but no environment capture exists for them under \texttt{reproducibility/}, so the
	36	+identity of their environments with the one frozen on M3U96b is an assumption rather than a recorded fact.
34	37	Table~\ref{tab:appH-packages} lists the packages on which results depend; the complete list is the lock file.
35	38	No step requires credentials except the two LLM raters of the taxonomy validation and the knowledge-graph
36	39	extraction (an Anthropic API key read from a file outside the repository, never stored in it).
		`@@ -102,11 +105,11 @@ and normalised by the same step~6; it is not yet a numbered step of the runbook.`
102	105	\endfoot
103	106	\bottomrule
104	107	\endlastfoot
105		−1 & OpenAlex works snapshot, pass~1 (CS filter + slim table): \texttt{python ingestion/\allowbreak openalex\_\allowbreak snapshot.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex/\allowbreak works --manifest data/\allowbreak raw/\allowbreak openalex/\allowbreak works\_\allowbreak manifest\_\allowbreak 2026-10-14.\allowbreak json --workers 8} & public S3 bucket, release 2026-10-14 & \texttt{works/\allowbreak cs/\allowbreak .\allowbreak jsonl.\allowbreak zst} (15{,}490{,}159 records), \texttt{works/\allowbreak slim/\allowbreak .\allowbreak parquet} (464{,}765{,}485 rows), \texttt{ledger.\allowbreak sqlite} & 1~h 46~min at $\approx$108~MB/s \\
	108	+1 & OpenAlex works snapshot, pass~1 (CS filter + slim table): \texttt{python ingestion/\allowbreak openalex\_\allowbreak snapshot.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex/\allowbreak works --manifest data/\allowbreak raw/\allowbreak openalex/\allowbreak works\_\allowbreak manifest\_\allowbreak 2026-10-14.\allowbreak json --workers 8} & public S3 bucket, release 2026-10-14 & \texttt{works/\allowbreak cs/\allowbreak .\allowbreak jsonl.\allowbreak zst} (\PrismaIdentifiedOpenalexPassOneCsTopicFilter{} records), \texttt{works/\allowbreak slim/\allowbreak .\allowbreak parquet} (\LinkSlimTotal{} rows), \texttt{ledger.\allowbreak sqlite} & 1~h 46~min at $\approx$108~MB/s \\
106	109	2 & OpenAlex entities: \texttt{python ingestion/\allowbreak openalex\_\allowbreak entities.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex} & S3 & \texttt{topics/\allowbreak }, \ldots, \texttt{institutions/\allowbreak }; \texttt{entities\_\allowbreak ledger.\allowbreak json} & 5~min \\
107		−3 & DBLP: \texttt{python ingestion/\allowbreak dblp\_\allowbreak parse.\allowbreak py --xml data/\allowbreak raw/\allowbreak dblp/\allowbreak dblp.\allowbreak xml.\allowbreak gz --out data/\allowbreak raw/\allowbreak dblp} & dump of 2026-11-02, MD5 verified & \texttt{dblp\_\allowbreak publications.\allowbreak parquet} (8{,}840{,}866 records), parse report & 4~min 45~s \\
108		−4 & arXiv OAI-PMH: \texttt{python ingestion/\allowbreak arxiv\_\allowbreak oai.\allowbreak py --out data/\allowbreak raw/\allowbreak arxiv --from 2018-01-01 --until 2026-11-02} & OAI feed, set \texttt{cs} & raw XML pages (758), \texttt{arxiv\_\allowbreak cs.\allowbreak parquet} (915{,}115 records) & $\approx$1~h \\
109		−5 & ACL Anthology: \texttt{python ingestion/\allowbreak acl\_\allowbreak parse.\allowbreak py --bib data/\allowbreak raw/\allowbreak acl/\allowbreak anthology+abstracts.\allowbreak bib.\allowbreak gz --out data/\allowbreak raw/\allowbreak acl} & BibTeX export & \texttt{acl\_\allowbreak anthology.\allowbreak parquet} (138{,}796 entries) & 10~s \\
	110	+3 & DBLP: \texttt{python ingestion/\allowbreak dblp\_\allowbreak parse.\allowbreak py --xml data/\allowbreak raw/\allowbreak dblp/\allowbreak dblp.\allowbreak xml.\allowbreak gz --out data/\allowbreak raw/\allowbreak dblp} & dump of 2026-11-02, MD5 verified & \texttt{dblp\_\allowbreak publications.\allowbreak parquet} (\DblpParsedRecords{} records), parse report & 4~min 45~s \\
	111	+4 & arXiv OAI-PMH: \texttt{python ingestion/\allowbreak arxiv\_\allowbreak oai.\allowbreak py --out data/\allowbreak raw/\allowbreak arxiv --from 2018-01-01 --until 2026-11-02} & OAI feed, set \texttt{cs} & raw XML pages (758), \texttt{arxiv\_\allowbreak cs.\allowbreak parquet} (\LinkArxivCsRecords{} records) & $\approx$1~h \\
	112	+5 & ACL Anthology: \texttt{python ingestion/\allowbreak acl\_\allowbreak parse.\allowbreak py --bib data/\allowbreak raw/\allowbreak acl/\allowbreak anthology+abstracts.\allowbreak bib.\allowbreak gz --out data/\allowbreak raw/\allowbreak acl} & BibTeX export & \texttt{acl\_\allowbreak anthology.\allowbreak parquet} (\AclParsedEntries{} entries) & 10~s \\
110	113	6 & Normalise: \texttt{python ingestion/\allowbreak build\_\allowbreak corpus.\allowbreak py --cs-dir data/\allowbreak raw/\allowbreak openalex/\allowbreak works/\allowbreak cs --out data/\allowbreak processed --workers 12} & step 1 & \texttt{tables/\allowbreak <table>/\allowbreak *.\allowbreak parquet} & 2~min 10~s \\
111	114	7 & Compact: \texttt{ulimit -n 65536; python ingestion/\allowbreak compact\_\allowbreak tables.\allowbreak py \textasciitilde/\allowbreak wp44\_\allowbreak uqo} & step 6 & \texttt{compact/\allowbreak <table>.\allowbreak parquet} & 30~s \\
112	115	8 & Link: \texttt{python ingestion/\allowbreak link\_\allowbreak sources.\allowbreak py --root \textasciitilde/\allowbreak wp44\_\allowbreak uqo} & steps 3--7 & \texttt{wp44.\allowbreak duckdb} with \texttt{link\_\allowbreak *}; \texttt{data/\allowbreak metadata/\allowbreak linkage\_\allowbreak report.\allowbreak json}; \texttt{needed\_\allowbreak oa\_\allowbreak ids.\allowbreak csv} & 10~s \\
		`@@ -266,8 +269,9 @@ Nothing under \texttt{data/\allowbreak raw/\allowbreak } or \texttt{data/\allowb`
266	269	repository contains is the code, the ontology and its audit table, the manifests, ledgers and hashes, the
267	270	PRISMA counts and the linkage report, the aggregated result tables, the figures, the generated number macros
268	271	and the manuscript (protocol \S P8). Record-level derived tables may be released where the source licence
269		−permits: OpenAlex is CC0, DBLP is ODC-BY~1.0, arXiv metadata obtained through OAI-PMH is under the CC0
270		−terms of that feed, and the ACL Anthology is CC~BY~4.0; a derived record table therefore carries the
	272	+permits: OpenAlex is CC0, DBLP is ODC-BY~1.0, arXiv metadata obtained through OAI-PMH is used under the arXiv
	273	+metadata terms of use (Part~III, Table~\ref{tab:data-sources}), which release the metadata itself under CC0,
	274	+and the ACL Anthology is CC~BY~4.0; a derived record table therefore carries the
271	275	attribution required by ODC-BY and CC~BY for the DBLP and ACL fields it contains. Full texts are never
272	276	redistributed and were never obtained through paywalls. The Semantic Scholar Academic Graph was not used:
273	277	bulk access requires an API key that was not available and anonymous calls were refused, and the mandate's
		`@@ -284,12 +288,12 @@ citation-intent signals derived from that graph are consequently absent from the`
284	288	\toprule
285	289	Source & What was obtained, when, size & Integrity & Licence \\
286	290	\midrule
287		−OpenAlex works snapshot, release 2026-10-14 & \texttt{data/\allowbreak jsonl/\allowbreak works/\allowbreak }: 1{,}898 part files, 464{,}765{,}485 works, 675.3~GB gzip; streamed 2026-11-02 21:19--23:05~ET over HTTPS with 8 parallel streams; retained: CS subset 20~GB (15{,}490{,}159 records) and slim table 9.3~GB (all works) & manifest; per-file content length and SHA-256 in \texttt{works/\allowbreak ledger.\allowbreak sqlite} (1{,}898/1{,}898 ok, 0 parse errors, record total = manifest total) & CC0 \\
	291	+OpenAlex works snapshot, release 2026-10-14 & \texttt{data/\allowbreak jsonl/\allowbreak works/\allowbreak }: 1{,}898 part files, \LinkSlimTotal{} works, 675.3~GB gzip; streamed 2026-11-02 21:19--23:05~ET over HTTPS with 8 parallel streams; retained: CS subset 20~GB (\PrismaIdentifiedOpenalexPassOneCsTopicFilter{} records) and slim table 9.3~GB (all works) & manifest; per-file content length and SHA-256 in \texttt{works/\allowbreak ledger.\allowbreak sqlite} (1{,}898/1{,}898 ok, 0 parse errors, record total = manifest total) & CC0 \\
288	292	OpenAlex entity snapshots & topics 4{,}516; subfields 252; fields 26; domains 4; sources 268{,}629; institutions 143{,}455; publishers 10{,}713; funders 41{,}799; concepts 65{,}026; 2026-11-02; 0.6~GB & \texttt{entities\_\allowbreak ledger.\allowbreak json} (SHA-256 per file) & CC0 \\
289	293	OpenAlex API increment & works with any CS topic, publication date 2026-10-01 to 2026-11-02, sliced by day and CS-primary status, basic paging $\le$10{,}000 per slice; from 2026-11-03 (public budget 0.10~USD/day, several days) & raw JSON pages; \texttt{api\_\allowbreak increment/\allowbreak ledger.\allowbreak json} (counts, cost) & CC0 \\
290		−DBLP & \texttt{dblp.\allowbreak xml.\allowbreak gz} (2026-11-02 22:39~UTC), \texttt{dblp.\allowbreak dtd}, \texttt{.md5}; 1.12~GB & MD5 \texttt{ff1c424c\ldots27c0} verified; parse report (8{,}840{,}866 records) & ODC-BY 1.0 \\
291		−arXiv OAI-PMH & set \texttt{cs}, metadata format \texttt{arXiv}, datestamps 2018-01-01 to 2026-11-02 in monthly windows, raw XML pages kept; 2026-11-02/03 & 758 pages, 915{,}115 unique records; \texttt{harvest\_\allowbreak state.\allowbreak json}, parse report & CC0 (metadata) \\
292		−ACL Anthology & \texttt{anthology+abstracts.\allowbreak bib.\allowbreak gz}; 2026-11-02; 46.7~MB, 138{,}796 entries & SHA-256 \texttt{44c3f7e9\ldots58e3} & CC BY 4.0 \\
	294	+DBLP & \texttt{dblp.\allowbreak xml.\allowbreak gz} (2026-11-02 22:39~UTC), \texttt{dblp.\allowbreak dtd}, \texttt{.md5}; 1.12~GB & MD5 \texttt{ff1c424c\ldots27c0} verified; parse report (\DblpParsedRecords{} records) & ODC-BY 1.0 \\
	295	+arXiv OAI-PMH & set \texttt{cs}, metadata format \texttt{arXiv}, datestamps 2018-01-01 to 2026-11-02 in monthly windows, raw XML pages kept; 2026-11-02/03 & 758 pages, \LinkArxivCsRecords{} unique records; \texttt{harvest\_\allowbreak state.\allowbreak json}, parse report & arXiv metadata terms (CC0) \\
	296	+ACL Anthology & \texttt{anthology+abstracts.\allowbreak bib.\allowbreak gz}; 2026-11-02; 46.7~MB, \AclParsedEntries{} entries & SHA-256 \texttt{44c3f7e9\ldots58e3} & CC BY 4.0 \\
293	297	\bottomrule
294	298	\end{tabularx}
295	299	\end{singlespace}

modified manuscript/references_part02.bib +24 −48

		`@@ -10,26 +10,6 @@`
10	10	note = {verified: arxiv API 2026-11-03}
11	11	}
12	12
13		−@misc{kinney2023s2,
14		− title = {The {Semantic} {Scholar} {Open} {Data} {Platform}},
15		− author = {Rodney Kinney and Chloe Anastasiades and Russell Authur and Iz Beltagy and Jonathan Bragg and Alexandra Buraczynski and Isabel Cachola and Stefan Candra and Yoganand Chandrasekhar and Arman Cohan and others},
16		− year = {2023},
17		− eprint = {2301.10140},
18		− archiveprefix = {arXiv},
19		− howpublished = {arXiv preprint arXiv:2301.10140},
20		− note = {verified: arxiv API 2026-11-03}
21		−}
22		−
23		−@misc{culbert2024openalex,
24		− title = {Reference {Coverage} {Analysis} of {OpenAlex} compared to {Web} of {Science} and {Scopus}},
25		− author = {Jack Culbert and Anne Hobert and Najko Jahn and Nick Haupka and Marion Schmidt and Paul Donner and Philipp Mayr},
26		− year = {2024},
27		− eprint = {2401.16359},
28		− archiveprefix = {arXiv},
29		− howpublished = {arXiv preprint arXiv:2401.16359},
30		− note = {verified: arxiv API 2026-11-03}
31		−}
32		−
33	13	@misc{sutskever2014seq2seq,
34	14	title = {Sequence to {Sequence} {Learning} with {Neural} {Networks}},
35	15	author = {Ilya Sutskever and Oriol Vinyals and Quoc V. Le},
		`@@ -800,16 +780,6 @@`
800	780	note = {verified: arxiv API 2026-11-03}
801	781	}
802	782
803		−@misc{ott2022benchmarks,
804		− title = {Mapping global dynamics of benchmark creation and saturation in artificial intelligence},
805		− author = {Simon Ott and Adriano Barbosa-Silva and Kathrin Blagec and Jan Brauner and Matthias Samwald},
806		− year = {2022},
807		− eprint = {2203.04592},
808		− archiveprefix = {arXiv},
809		− howpublished = {arXiv preprint arXiv:2203.04592},
810		− note = {verified: arxiv API 2026-11-03}
811		−}
812		−
813	783	@misc{dehghani2021lottery,
814	784	title = {The {Benchmark} {Lottery}},
815	785	author = {Mostafa Dehghani and Yi Tay and Alexey A. Gritsenko and Zhe Zhao and Neil Houlsby and Fernando Diaz and Donald Metzler and Oriol Vinyals},
		`@@ -2318,17 +2288,6 @@`
2318	2288	note = {verified: crossref 2026-11-03}
2319	2289	}
2320	2290
2321		−@article{fortunato2018sos,
2322		− title = {Science of science},
2323		− author = {Fortunato, Santo and Bergstrom, Carl T. and Börner, Katy and Evans, James A. and Helbing, Dirk and Milojević, Staša and Petersen, Alexander M. and Radicchi, Filippo and Sinatra, Roberta and Uzzi, Brian and others},
2324		− year = {2018},
2325		− journal = {Science},
2326		− volume = {359},
2327		− number = {6379},
2328		− doi = {10.1126/science.aao0185},
2329		− note = {verified: crossref 2026-11-03}
2330		−}
2331		−
2332	2291	@book{wang2021sos,
2333	2292	title = {The {Science} of {Science}},
2334	2293	author = {Wang, Dashun and Barabási, Albert-László},
		`@@ -2424,13 +2383,7 @@`
2424	2383	howpublished = {Blog post, 30 November 2022, \url{https://openai.com/index/chatgpt/}},
2425	2384	note = {Accessed 3 November 2026; original URL openai.com/blog/chatgpt archived by the Internet Archive on 2 December 2022}
2426	2385	}
2427		−@misc{openalex2025topics,
2428		− title = {Topics},
2429		− author = {{OpenAlex}},
2430		− year = {2025},
2431		− howpublished = {OpenAlex documentation, \url{https://help.openalex.org/data/topics/}},
2432		− note = {Accessed 3 November 2026}
2433		−}
	2386	+
2434	2387	@misc{arxiv2026stats,
2435	2388	title = {{arXiv} monthly submissions},
2436	2389	author = {{arXiv}},
		`@@ -2438,3 +2391,26 @@`
2438	2391	howpublished = {\url{https://arxiv.org/stats/monthly_submissions}},
2439	2392	note = {Accessed 3 November 2026}
2440	2393	}
	2394	+
	2395	+@article{culbert2025openalex,
	2396	+ author = {Culbert, Jack H. and Hobert, Anne and Jahn, Najko and Haupka, Nick and Schmidt, Marion and Donner, Paul and Mayr, Philipp},
	2397	+ title = {Reference coverage analysis of {OpenAlex} compared to {Web of Science} and {Scopus}},
	2398	+ journal = {Scientometrics},
	2399	+ year = {2025},
	2400	+ volume = {130},
	2401	+ pages = {2475--2492},
	2402	+ doi = {10.1007/s11192-025-05293-3},
	2403	+ note = {verified: crossref 2026-11-03}
	2404	+}
	2405	+
	2406	+@article{fortunato2018science,
	2407	+ author = {Fortunato, Santo and Bergstrom, Carl T. and B{\"o}rner, Katy and Evans, James A. and Helbing, Dirk and Milojevi{\'c}, Sta{\v{s}}a and Petersen, Alexander M. and Radicchi, Filippo and Sinatra, Roberta and Uzzi, Brian and Vespignani, Alessandro and Waltman, Ludo and Wang, Dashun and Barab{\'a}si, Albert-L{\'a}szl{\'o}},
	2408	+ title = {Science of science},
	2409	+ journal = {Science},
	2410	+ year = {2018},
	2411	+ volume = {359},
	2412	+ number = {6379},
	2413	+ pages = {eaao0185},
	2414	+ doi = {10.1126/science.aao0185},
	2415	+ note = {verified: crossref 2026-11-03}
	2416	+}

modified manuscript/references_part03.bib +0 −40

		`@@ -34,15 +34,6 @@`
34	34	howpublished = {arXiv preprint arXiv:1905.00075},
35	35	note = {verified: arxiv 2026-11-03}
36	36	}
37		−@misc{kinney2023s2,
38		− author = {Kinney, Rodney and Anastasiades, Chloe and Authur, Russell and Beltagy, Iz and Bragg, Jonathan and Buraczynski, Alexandra and Cachola, Isabel and Candra, Stefan and Chandrasekhar, Yoganand and Cohan, Arman and others},
39		− title = {The {Semantic Scholar} Open Data Platform},
40		− year = {2023},
41		− eprint = {2301.10140},
42		− archiveprefix = {arXiv},
43		− howpublished = {arXiv preprint arXiv:2301.10140},
44		− note = {verified: arxiv 2026-11-03}
45		−}
46	37	@article{culbert2025openalex,
47	38	author = {Culbert, Jack H. and Hobert, Anne and Jahn, Najko and Haupka, Nick and Schmidt, Marion and Donner, Paul and Mayr, Philipp},
48	39	title = {Reference coverage analysis of {OpenAlex} compared to {Web of Science} and {Scopus}},
		`@@ -72,16 +63,6 @@`
72	63	doi = {10.1162/qss_a_00286},
73	64	note = {verified: crossref 2026-11-03}
74	65	}
75		−@article{martinmartin2021comparison,
76		− author = {Mart{\'i}n-Mart{\'i}n, Alberto and Thelwall, Mike and Orduna-Malea, Enrique and Delgado L{\'o}pez-C{\'o}zar, Emilio},
77		− title = {{Google Scholar}, {Microsoft Academic}, {Scopus}, {Dimensions}, {Web of Science}, and {OpenCitations}' {COCI}: a multidisciplinary comparison of coverage via citations},
78		− journal = {Scientometrics},
79		− year = {2021},
80		− volume = {126},
81		− pages = {871--906},
82		− doi = {10.1007/s11192-020-03690-4},
83		− note = {verified: crossref 2026-11-03}
84		−}
85	66	@article{mongeon2016coverage,
86	67	author = {Mongeon, Philippe and Paul-Hus, Ad{\`e}le},
87	68	title = {The journal coverage of {Web of Science} and {Scopus}: a comparative analysis},
		`@@ -92,16 +73,6 @@`
92	73	doi = {10.1007/s11192-015-1765-5},
93	74	note = {verified: crossref 2026-11-03}
94	75	}
95		−@article{radev2013acl,
96		− author = {Radev, Dragomir R. and Muthukrishnan, Pradeep and Qazvinian, Vahed and Abu-Jbara, Amjad},
97		− title = {The {ACL} anthology network corpus},
98		− journal = {Language Resources and Evaluation},
99		− year = {2013},
100		− volume = {47},
101		− pages = {919--944},
102		− doi = {10.1007/s10579-012-9211-2},
103		− note = {verified: crossref 2026-11-03}
104		−}
105	76	@article{fellegi1969linkage,
106	77	author = {Fellegi, Ivan P. and Sunter, Alan B.},
107	78	title = {A theory for record linkage},
		`@@ -152,17 +123,6 @@`
152	123	doi = {10.1007/s11192-012-0775-9},
153	124	note = {verified: crossref 2026-11-03 (online 2012, volume 94 = 2013)}
154	125	}
155		−@article{fortunato2018science,
156		− author = {Fortunato, Santo and Bergstrom, Carl T. and B{\"o}rner, Katy and Evans, James A. and Helbing, Dirk and Milojevi{\'c}, Sta{\v{s}}a and Petersen, Alexander M. and Radicchi, Filippo and Sinatra, Roberta and Uzzi, Brian and Vespignani, Alessandro and Waltman, Ludo and Wang, Dashun and Barab{\'a}si, Albert-L{\'a}szl{\'o}},
157		− title = {Science of science},
158		− journal = {Science},
159		− year = {2018},
160		− volume = {359},
161		− number = {6379},
162		− pages = {eaao0185},
163		− doi = {10.1126/science.aao0185},
164		− note = {verified: crossref 2026-11-03}
165		−}
166	126	@article{nosek2018preregistration,
167	127	author = {Nosek, Brian A. and Ebersole, Charles R. and DeHaven, Alexander C. and Mellor, David T.},
168	128	title = {The preregistration revolution},

modified manuscript/references_part04.bib +0 −54

		`@@ -12,27 +12,6 @@`
12	12	doi = {10.1016/j.joi.2016.02.007}
13	13	}
14	14
15		−@incollection{waltman2019fieldnorm,
16		− author = {Waltman, Ludo and van Eck, Nees Jan},
17		− title = {Field Normalization of Scientometric Indicators},
18		− booktitle = {Springer Handbook of Science and Technology Indicators},
19		− publisher = {Springer},
20		− year = {2019},
21		− pages = {281--300},
22		− doi = {10.1007/978-3-030-02511-3_11}
23		−}
24		−
25		−@article{wang2013window,
26		− author = {Wang, Jian},
27		− title = {Citation time window choice for research impact evaluation},
28		− journal = {Scientometrics},
29		− year = {2013},
30		− volume = {94},
31		− number = {3},
32		− pages = {851--872},
33		− doi = {10.1007/s11192-012-0775-9}
34		−}
35		−
36	15	@misc{openalex2026fwci,
37	16	author = {{OpenAlex}},
38	17	title = {Works: attributes ({F}ield-{W}eighted {C}itation {I}mpact, \texttt{fwci})},
		`@@ -237,17 +216,6 @@`
237	216	doi = {10.1007/BF01908075}
238	217	}
239	218
240		−@article{uzzi2013atypical,
241		− author = {Uzzi, Brian and Mukherjee, Satyam and Stringer, Michael and Jones, Ben},
242		− title = {Atypical combinations and scientific impact},
243		− journal = {Science},
244		− year = {2013},
245		− volume = {342},
246		− number = {6157},
247		− pages = {468--472},
248		− doi = {10.1126/science.1240474}
249		−}
250		−
251	219	@article{wang2017novelty,
252	220	author = {Wang, Jian and Veugelers, Reinhilde and Stephan, Paula},
253	221	title = {Bias against novelty in science: A cautionary tale for users of bibliometric indicators},
		`@@ -270,17 +238,6 @@`
270	238	doi = {10.1093/ije/dyw098}
271	239	}
272	240
273		−@article{bernal2018controls,
274		− author = {Lopez Bernal, James and Cummins, Steven and Gasparrini, Antonio},
275		− title = {The use of controls in interrupted time series studies of public health interventions},
276		− journal = {International Journal of Epidemiology},
277		− year = {2018},
278		− volume = {47},
279		− number = {6},
280		− pages = {2082--2093},
281		− doi = {10.1093/ije/dyy135}
282		−}
283		−
284	241	@article{newey1987hac,
285	242	author = {Newey, Whitney K. and West, Kenneth D.},
286	243	title = {A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix},
		`@@ -314,17 +271,6 @@`
314	271	doi = {10.2307/2998540}
315	272	}
316	273
317		−@article{bai2003computation,
318		− author = {Bai, Jushan and Perron, Pierre},
319		− title = {Computation and analysis of multiple structural change models},
320		− journal = {Journal of Applied Econometrics},
321		− year = {2003},
322		− volume = {18},
323		− number = {1},
324		− pages = {1--22},
325		− doi = {10.1002/jae.659}
326		−}
327		−
328	274	@article{liu1997segmented,
329	275	author = {Liu, Jian and Wu, Shiying and Zidek, James V.},
330	276	title = {On segmented multivariate regression},

modified manuscript/sections/part02_evolution.tex +13 −13

		`@@ -21,7 +21,7 @@ The \emph{arXiv subject classes} are the second system. arXiv was created in 199`
21	21
22	22	The \emph{DBLP computer science bibliography} is the third. DBLP does not classify papers at all; it classifies \emph{venues}. Begun in 1993 as a database and logic programming bibliography and extended to the whole of computer science, it records the tables of contents of conferences and journals with curated author identities \citep{ley2002dblp}. Membership in DBLP is itself a judgement that a venue belongs to computer science, and the venue stream to which a paper belongs is a strong signal of community: a paper in the proceedings of OSDI is an operating-systems paper in the sociological sense whatever its subject matter. DBLP therefore encodes the discipline as a set of communities rather than as a set of topics, and the two encodings disagree precisely where this paper expects change, namely where machine-learning methods enter venues that were not built for them.
23	23
24		−The fourth system is the family of \emph{algorithmic, citation-based classifications} that underlie modern open bibliographic indexes. OpenAlex, which succeeded the Microsoft Academic Graph after the latter was discontinued at the end of 2021 \citep{wang2020mag,priem2022openalex}, assigns to every work a ranked list of \emph{topics} from a fixed vocabulary of about 4{,}500 topics grouped into subfields, fields and domains; the topics were obtained by clustering the citation network and labelling the clusters with a language model, and are assigned to new works by a classifier over title, abstract and venue \citep{openalex2025topics}. The approach descends from the publication-level clustering methodology of \citet{waltman2012classification}, refined by the Leiden community-detection algorithm \citep{traag2019leiden}, and it inherits the known properties of that family: clusters are well connected and reproducible, but their granularity is a free parameter with no natural value \citep{sjogarde2018granularity}, and citation-based taxonomies, while more accurate than text-based or journal-based ones on average \citep{klavans2017taxonomy}, lag behind the literature by construction because a cluster must accumulate citations before it can exist. The Semantic Scholar academic graph offers a comparable but differently constructed resource \citep{kinney2023s2}. Older, journal-based classifications of the kind proposed by \citet{glanzel2003classification} remain in use in evaluative scientometrics but are too coarse, and too journal-centric, for a conference-dominated discipline.
	24	+The fourth system is the family of \emph{algorithmic, citation-based classifications} that underlie modern open bibliographic indexes. OpenAlex, which succeeded the Microsoft Academic Graph after the latter was discontinued at the end of 2021 \citep{wang2020mag,priem2022openalex}, assigns to every work a ranked list of \emph{topics} from a fixed vocabulary of about 4{,}500 topics grouped into subfields, fields and domains; the topics were obtained by clustering the citation network and labelling the clusters with a language model, and are assigned to new works by a classifier over title, abstract and venue \citep{openalex2024topics}. The approach descends from the publication-level clustering methodology of \citet{waltman2012classification}, refined by the Leiden community-detection algorithm \citep{traag2019leiden}, and it inherits the known properties of that family: clusters are well connected and reproducible, but their granularity is a free parameter with no natural value \citep{sjogarde2018granularity}, and citation-based taxonomies, while more accurate than text-based or journal-based ones on average \citep{klavans2017taxonomy}, lag behind the literature by construction because a cluster must accumulate citations before it can exist. The Semantic Scholar academic graph offers a comparable but differently constructed resource \citep{kinney2023s2ag}. Older, journal-based classifications of the kind proposed by \citet{glanzel2003classification} remain in use in evaluative scientometrics but are too coarse, and too journal-centric, for a conference-dominated discipline.
25	25
26	26	Each of these systems is a model of the discipline with its own inductive bias. The CCS encodes expert judgement frozen in 2012; arXiv classes encode author self-positioning at submission; DBLP streams encode venue communities; OpenAlex topics encode citation neighbourhoods as of a recent snapshot. None of them is ground truth, and this study treats their disagreements as data rather than as noise.
27	27
		`@@ -46,22 +46,22 @@ The ontology's limitations are stated in Appendix~A and are not repeated here, e`
46	46
47	47	The disagreements among the four systems are systematic, and they fall into three kinds.
48	48
49		−The first is \emph{coverage}. Comparisons of bibliographic databases consistently show that no source indexes the whole of a discipline and that the gaps are not random: conference proceedings, preprints, and non-English venues are the most unevenly covered \citep{visser2021comparison,martinmartin2021coverage}. OpenAlex, the backbone of this corpus, inherits the broad but uneven coverage of Microsoft Academic and Crossref; its reference lists are less complete than those of the commercial indexes for part of the period, which affects citation-based measures more than counts (a preprint finding; \citealp{culbert2024openalex}. The linking phase of this study illustrates the scale of the problem at the level of record identity. The OpenAlex snapshot contains \LinkOaCsRecords{} works classified in computer science between 2018 and 2026; DBLP contributes \LinkDblpTwoZeroOneEightTwoZeroTwoSix{} records over the same years, of which \LinkDblpLinkedDistinct{} could be matched to an OpenAlex work by DOI or by title and year in the first linking pass and \LinkDblpUnlinked{} could not; the arXiv \texttt{cs} harvest contributes \LinkArxivCsRecords{} preprints, of which \LinkArxivLinkedDistinct{} were matched, overwhelmingly through the arXiv identifier that OpenAlex stores. These are linkage statistics, not findings about research, but they establish that a classification applied to one source cannot simply be transferred to another: a non-trivial fraction of the venue-defined (DBLP) universe has no citation-defined (OpenAlex) counterpart, and vice versa. Part~III describes the second linking pass and the treatment of unmatched records.
	49	+The first is \emph{coverage}. Comparisons of bibliographic databases consistently show that no source indexes the whole of a discipline and that the gaps are not random: conference proceedings, preprints, and non-English venues are the most unevenly covered \citep{visser2021comparison,martinmartin2021coverage}. OpenAlex, the backbone of this corpus, inherits the broad but uneven coverage of Microsoft Academic and Crossref; its reference lists are less complete than those of the commercial indexes for part of the period, which affects citation-based measures more than counts \citep{culbert2025openalex}. The linking phase of this study illustrates the scale of the problem at the level of record identity. The first pass over the OpenAlex snapshot retained \PrismaIdentifiedOpenalexPassOneCsTopicFilter{} works that carry a computer-science topic and a publication year between 2018 and 2026; a second pass added \PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{} works that OpenAlex labels with no computer-science topic but that DBLP or arXiv identifiers reach, for \PrismaIdentifiedOpenalexTotalRecordsLoaded{} records loaded in all. DBLP contributes \LinkDblpTwoZeroOneEightTwoZeroTwoSix{} records over the same years, of which \LinkDblpLinkedDistinct{} could be matched to an OpenAlex work by DOI or by title and year once both passes had been linked and \LinkDblpUnlinked{} could not; the arXiv \texttt{cs} harvest contributes \LinkArxivCsRecords{} preprints, of which \LinkArxivLinkedDistinct{} were matched, overwhelmingly through the arXiv identifier that OpenAlex stores. These are linkage statistics, not findings about research, but they establish that a classification applied to one source cannot simply be transferred to another: a non-trivial fraction of the venue-defined (DBLP) universe has no citation-defined (OpenAlex) counterpart, and vice versa. Part~III describes the two passes and the treatment of unmatched records.
50	50
51	51	The second kind of disagreement is \emph{granularity and boundary}. The arXiv class \texttt{cs.LG} is one node among forty; in OpenAlex, machine learning spreads across dozens of topics in several fields, including statistics, engineering and medicine, because the citation network does not respect the boundary of the computer-science field. Conversely, OpenAlex's field~17 (``Computer Science'') contains clusters on educational technology, media studies and physics that no computer scientist would claim, and the ontology of this study deliberately leaves most of them unmapped (Appendix~A). The practical consequence is that the apparent size of ``AI'' depends on which boundary is used: a venue-based definition (DBLP streams of NeurIPS, ICML, ICLR, AAAI and the vision and language conferences) gives a different set from a topic-based definition, which gives a different set again from an arXiv-class definition, and the three sets diverge most in the applied sciences, where machine-learning methods are used without the paper belonging to any machine-learning community. The multi-label design of the ontology, and the use of all three signal types with capped weights in equation~\eqref{eq:p2-score}, is the response to this divergence: a work that is an AI paper by every criterion scores high on all of them, while a work that is an AI paper by vocabulary alone is held to a higher evidentiary standard.
52	52
53	53	The third kind is \emph{temporal}. The CCS was fixed in 2012 and the arXiv classes are older still; OpenAlex topics were generated from a citation network observed in the early 2020s and are reassigned when the snapshot is rebuilt; DBLP streams are stable but new venues appear and old ones are renamed. A classification that does not change cannot register the emergence of a field except as the growth of an existing node; a classification that is regenerated cannot be compared across snapshots without care. Neither property is a defect, but they pull in opposite directions, and a study of structural change must choose. This study freezes all classifications at a single snapshot (Part~III), so that any growth it reports is growth of a fixed category, and it treats the emergence of \emph{new} categories as a separate detection problem (Part~IV, Section~\ref{sec:p2-claims}).
54	54
55		−A final, sociological disagreement should be named explicitly because it affects what counts as a ``paper''. Computer science is unusual among the sciences in treating selective conferences as the primary venue of record, a practice that has been defended on grounds of speed and criticised on grounds of reviewing quality and the fragmentation of the literature \citep{vardi2009conferences,fortnow2009growup}. The convention has two consequences for bibliometrics. Conference papers are indexed less uniformly than journal articles, so coverage disagreements are concentrated in exactly the venues that the discipline values most; and the conference calendar imposes a strong seasonality on publication dates, so that monthly series must be deseasonalised before any break can be read off them. The preprint culture adds a third consequence. Computer scientists came to arXiv late relative to physicists and mathematicians, and a large share of computer-science preprints are never matched to a published version, either because they are not published or because the match fails \citep{lin2020preprints}. A corpus that counts preprints and published versions as separate works will over-count the preprint-heavy subfields; a corpus that links them will under-count whenever linkage fails; and the subfields where linkage fails most often are, again, the fast-moving ones.
	55	+A final, sociological disagreement should be named explicitly because it affects what counts as a ``paper''. Computer science is unusual among the sciences in treating selective conferences as the primary venue of record, a practice that has been defended on grounds of speed and criticised on grounds of reviewing quality and the fragmentation of the literature \citep{vardi2009conferences,fortnow2009growup}. The convention has two consequences for bibliometrics. Conference papers are indexed less uniformly than journal articles, so coverage disagreements are concentrated in exactly the venues that the discipline values most; and the conference calendar imposes a strong seasonality on publication dates, so that monthly series must be deseasonalised before any break can be read off them. The preprint culture adds a third consequence. Computer scientists came to arXiv late relative to physicists and mathematicians, and a non-trivial share of computer-science preprints are never matched to a published version, either because they are not published or because the match fails: according to its abstract, the matching study of \citet{lin2020preprints} found that about a quarter of the arXiv computer-science preprints of 2008--2017 in its sample had no identifiable published counterpart, and that a further eleven per cent had been published under a different title, which a title-based match would miss. A corpus that counts preprints and published versions as separate works will over-count the preprint-heavy subfields; a corpus that links them will under-count whenever linkage fails; and the subfields where linkage fails most often are, again, the fast-moving ones.
56	56
57	57	\subsection{Why the growth of a field is hard to measure}
58	58	\label{sec:p2-growth}
59	59
60	60	The questions of this paper are ultimately about rates: of publication, of citation, of movement between categories. Rates presuppose denominators, and the denominators of science are themselves contested quantities.
61	61
62		−The literature on the growth of science offers two sobering lessons. The first is that measured growth depends heavily on the database. Using cited references as a proxy for the literature that scientists actually use, \citet{bornmann2015growth} estimated the long-run growth of science at roughly three per cent per year with a doubling time of about a quarter of a century, while estimates based on the publication counts of individual databases vary widely with the database's coverage policy and its own expansion. A later analysis with segmented growth curves across several databases found that the growth rate has itself changed over time, with faster growth in recent decades, but that the location of the segments depends on the source \citep{bornmann2021growth}. For the present study the lesson is that a change in the volume of computer-science publications between 2018 and 2026 is the sum of a change in research activity and a change in indexing, and that only the first is of scientific interest. OpenAlex grew substantially during the period as it absorbed new sources and improved its matching; the indexing lag (the delay between a work's publication and its appearance in the snapshot) is longest for the most recent months, so that any series ending in 2026 slopes downward at its end for reasons that have nothing to do with research. These are among the pre-period diagnostics and sensitivity checks required by CLAUDE.md, \S6.5 and specified in Part~IV.
	62	+The literature on the growth of science offers two sobering lessons. The first is that measured growth depends heavily on the database. Using cited references as a proxy for the literature that scientists actually use, \citet{bornmann2015growth} identified three phases of growth, each at roughly three times the rate of the previous one: under one per cent per year up to the middle of the eighteenth century, two to three per cent up to the inter-war period, and eight to nine per cent per year from then to 2010, a doubling time of about nine years for modern science; estimates based on the publication counts of individual databases vary widely with the database's coverage policy and its own expansion. A later analysis with segmented growth curves across several databases found that the growth rate has itself changed over time, with faster growth in recent decades, but that the location of the segments depends on the source \citep{bornmann2021growth}. For the present study the lesson is that a change in the volume of computer-science publications between 2018 and 2026 is the sum of a change in research activity and a change in indexing, and that only the first is of scientific interest. OpenAlex grew substantially during the period as it absorbed new sources and improved its matching; the indexing lag (the delay between a work's publication and its appearance in the snapshot) is longest for the most recent months, so that any series ending in 2026 slopes downward at its end for reasons that have nothing to do with research. These are among the pre-period diagnostics and sensitivity checks required by CLAUDE.md, \S6.5 and specified in Part~IV.
63	63
64		−The second lesson concerns the ``science of science'' as a field in its own right. The programme described by \citet{fortunato2018sos} and developed at book length by \citet{wang2021sos} has established robust regularities: citation distributions are heavy-tailed within every field; team size has grown steadily; novelty in the sense of atypical combinations of prior work is associated with higher impact at the top of the distribution; and the productivity of individual scientists is approximately stable over careers. Those regularities are the background against which any field-specific claim must be read. A rise in the citation inequality of AI papers after 2022, for instance, is interesting only if it exceeds the inequality that heavy-tailed citation distributions produce in every growing field; the hypotheses H1--H8 of the preregistered protocol (Part~III) are formulated as comparisons between the post-2022 period and the baseline for that reason.
	64	+The second lesson concerns the ``science of science'' as a field in its own right. The programme described by \citet{fortunato2018science} and developed at book length by \citet{wang2021sos} has established robust regularities: citation distributions are heavy-tailed within every field; team size has grown steadily; novelty in the sense of atypical combinations of prior work is associated with higher impact at the top of the distribution; and the productivity of individual scientists is approximately stable over careers. Those regularities are the background against which any field-specific claim must be read. A rise in the citation inequality of AI papers after 2022, for instance, is interesting only if it exceeds the inequality that heavy-tailed citation distributions produce in every growing field; the hypotheses H1--H8 of the preregistered protocol (Part~III) are formulated as comparisons between the post-2022 period and the baseline for that reason.
65	65
66	66	A further difficulty is specific to computer science and concerns what the discipline takes itself to be. \citet{denning2005science} argued that computer science is a science of information processes and not merely an engineering of artefacts, and \citet{wing2006computational} that ``computational thinking'' is a general intellectual skill rather than a professional one; both arguments were written at a moment when the discipline was defending its boundaries. The post-2022 period raises the converse question: whether the discipline's methods have diffused so widely that the boundary is no longer where the classifications place it. Papers that apply learned models to protein structure, materials, weather or law are classified as computer science by some systems and not others; a transformation that consists partly of such diffusion is partly invisible to a classification that keeps the boundary fixed. The ontology's eighth area, computational science and discovery, and the ``penumbra'' set defined in Part~III (works with a computer-science topic but without venue or arXiv support) exist to make that diffusion observable without inflating the core.
67	67
		`@@ -139,7 +139,7 @@ Visibility in classifications & Accommodated by existing top-level nodes & New v`
139	139
140	140	Three conclusions follow from the comparison and shape the hypotheses tested later. First, the mechanisms of propagation differ in kind. The web changed the problems of other subfields; deep learning changed their methods; the transformer era changed the \emph{models} they use, which can be adopted without changing either problems or methods. A subfield can therefore register the third transformation at low cost, by using a pretrained model as a component, and the resulting growth in the vocabulary of the subfield need not indicate a reorganisation of its research. Distinguishing component adoption from reorganisation is the purpose of the citation-flow and embedding analyses of Parts~IV and~VIII.
141	141
142		−Second, each wave shortened the publication cycle and shifted the version of record toward the preprint. The consequence for this study is that the arXiv harvest and the OpenAlex index measure different things: the former records when a result was first made public, the latter when it entered the citation system. A break detected in one series and not in the other is informative about the mechanism, not merely about measurement.
	142	+Second, each wave shortened the publication cycle and shifted the version of record toward the preprint. The consequence for this study is that the arXiv harvest and the OpenAlex index measure different things: the former records when a result was first made public, the latter when it entered the citation system. A break detected in one series and not in the other may be informative about the mechanism, but it may equally reflect indexing lag, linkage failure or right truncation (Part~III, Section~\ref{sec:data-dates}); only the date diagnostics of Part~III separate the two readings.
143	143
144	144	Third, the three waves were cumulative in infrastructure and in institutions. The industrial laboratories that produced the web systems trained the generation that produced deep learning, and the frameworks and compute of the deep-learning wave made the transformer era possible. The post-2022 period inherited all of it: the laboratories, the frameworks, the model hubs, the benchmark culture, and the preprint norm. Whatever is new about the period must be identified against that inheritance rather than attributed to a single product release, and the baseline window of 2018--2022 was chosen so that the inheritance is observed directly rather than assumed.
145	145
		`@@ -148,9 +148,9 @@ Third, the three waves were cumulative in infrastructure and in institutions. Th`
148	148
149	149	The three waves left computer science, on the eve of the reference date, with a publication system unlike that of any other science, and the system's properties determine what the corpus can and cannot see. Four of them are load-bearing for the analyses that follow.
150	150
151		−The first is the dominance of the preprint. By 2022 the arXiv was the first and often the only public version of most machine-learning results, and a growing share of systems, theory and security results as well; the matching study of \citet{lin2020preprints} found that a majority of computer-science preprints in their sample could not be matched to a published version, and that the matched ones had been published with a median delay of many months. The consequence is that the arXiv series lead the indexed series, and that a corpus which dates works by their indexed publication date will place a change later than it occurred and will smear it over the publication lag. Part~III dates each work by the earliest of its preprint and publication dates when both are known, and Part~IX reports the breaks on both clocks.
	151	+The first is the dominance of the preprint. By 2022 the arXiv was the first and often the only public version of most machine-learning results, and a growing share of systems, theory and security results as well; the matching study of \citet{lin2020preprints} found, according to its abstract, that about three quarters of the arXiv computer-science preprints of 2008--2017 in its sample had been published in a peer-reviewed venue (66 per cent under an unchanged title and 11 per cent under a different one), so that roughly a quarter had no identifiable published version. The consequence is that the arXiv series lead the indexed series, and that a corpus which dates works by their indexed publication date will place a change later than it occurred and will smear it over the publication lag. Part~III dates each work by the earliest of its preprint and publication dates when both are known, and Part~IX reports the breaks on both clocks.
152	152
153		−The second is the conference calendar. The major venues of every area have fixed annual deadlines, and preprint postings cluster in the weeks before and after them; monthly series of computer-science output are therefore strongly seasonal, with a pattern that differs by area because the calendars differ. Seasonality is not a nuisance to be removed once and forgotten; a change in the \emph{amplitude} of seasonality, for instance because a venue grew or because preprints began to be posted without regard to deadlines, is itself a structural change, and the seasonal component $\mathbf{s}_c(t)$ of equation~\eqref{eq:p2-its} is modelled separately for the baseline and the research period for that reason.
	153	+The second is the conference calendar. The major venues of every area have fixed annual deadlines, and preprint postings cluster in the weeks before and after them; monthly series of computer-science output are therefore strongly seasonal, with a pattern that differs by area because the calendars differ. Seasonality is not a nuisance to be removed once and forgotten; a change in the \emph{amplitude} of seasonality, for instance because a venue grew or because preprints began to be posted without regard to deadlines, is itself a structural change, but the seasonal component $\mathbf{s}_c(t)$ of equation~\eqref{eq:p2-its} is, in the implementation of Part~IV, a single set of eleven month dummies common to the baseline and the research period, so that a change in the amplitude of seasonality is absorbed by the residuals rather than estimated; this is a limitation of the specification, not a feature of it.
154	154
155	155	The third is the state of peer review. The volume of submissions to the leading machine-learning conferences had grown by an order of magnitude in a decade; reviewer pools had not. The debate about whether conferences remained an adequate venue of record \citep{vardi2009conferences,fortnow2009growup} was, by 2022, less about prestige than about capacity, and the reproducibility programmes of \citet{pineau2020reproducibility} were one institutional response. The relevance to this study is twofold. Reviewing capacity constrains the number of works that can appear in venue-defined categories, so that growth under capacity constraints shows up as preprint growth and as the founding of new venues rather than as growth in existing streams; and the reviews themselves became, after 2022, one of the first places where the use of language models was measured \citep{liang2024monitoring}.
156	156
		`@@ -187,9 +187,9 @@ The boundary development is the \emph{theory of deep learning}. The empirical ob`
187	187	\subsection{Programming languages, compilers and formal verification}
188	188	\label{sec:p2-pl}
189	189
190		−The programming-languages literature of the baseline is marked by the success of a language. Rust, whose ownership type system gives memory safety without garbage collection \citep{matsakis2014rust}, became the first systems language to be adopted at scale on the strength of a safety guarantee; the RustBelt project supplied the semantic foundation by proving, in a separation-logic framework mechanised in Coq, that the safe fragment is sound and that the unsafe library code on which it rests respects the invariants \citep{jung2018rustbelt}. The result is characteristic of the period's formal-methods culture: a verification effort that follows an engineering artefact rather than preceding it, and that uses a proof assistant as its medium.
	190	+The programming-languages literature of the baseline is marked by the success of a language. Rust, whose ownership type system gives memory safety without garbage collection \citep{matsakis2014rust}, became one of the first systems languages to be adopted at scale on the strength of a safety guarantee; the RustBelt project supplied the semantic foundation by proving, in a separation-logic framework mechanised in Coq, that the safe fragment is sound and that the unsafe library code on which it rests respects the invariants \citep{jung2018rustbelt}. The result is characteristic of the period's formal-methods culture: a verification effort that follows an engineering artefact rather than preceding it, and that uses a proof assistant as its medium.
191	191
192		−Proof assistants themselves reached a new stage. The verified C compiler CompCert \citep{leroy2009compcert} and the verified microkernel seL4 \citep{klein2009sel4} had shown a decade earlier that production software could be proved correct; the Kepler conjecture was formally proved in 2017 after a twelve-year effort \citep{hales2017kepler}; and the Lean theorem prover \citep{demoura2015lean}, re-implemented as a programming language in its fourth version \citep{demoura2021lean4}, acquired a community-maintained mathematical library whose scale and organisation were new in formalised mathematics \citep{mathlib2020}. The relevance to the post-2022 period is direct: Lean and its library became the environment in which language models were later asked to prove theorems, so that formal verification, one of the categories predicted \emph{not} to break in H3, is also one where the contact with AI is most visible in the recent literature. The hypothesis is therefore a sharp one.
	192	+Proof assistants themselves reached a new stage. The verified C compiler CompCert \citep{leroy2009compcert} and the verified microkernel seL4 \citep{klein2009sel4} had shown a decade earlier that production software could be proved correct; the formal proof of the Kepler conjecture, begun in 2003, was completed in 2014 and published in 2017 \citep{hales2017kepler}; and the Lean theorem prover \citep{demoura2015lean}, re-implemented as a programming language in its fourth version \citep{demoura2021lean4}, acquired a community-maintained mathematical library whose scale and organisation were new in formalised mathematics \citep{mathlib2020}. The relevance to the post-2022 period is direct: Lean and its library became the environment in which language models were later asked to prove theorems, so that formal verification, one of the categories predicted \emph{not} to break in H3, is also one where the contact with AI is most visible in the recent literature. The hypothesis is therefore a sharp one.
193	193
194	194	Software engineering had already begun to absorb learned models before 2022. The survey of \citet{allamanis2018survey} organised a decade of ``big code'' research around the naturalness hypothesis, that source code is as statistically regular as natural language; the CodeXGLUE benchmark collection \citep{lu2021codexglue} standardised evaluation; and the Codex models \citep{chen2021codex} made program synthesis from natural language a product. The software-engineering category is thus expected to show an early and strong AI signal, and its baseline is not an AI-free one.
195	195
		`@@ -248,7 +248,7 @@ Taken together, the baseline descriptions support a conclusion that the empirica`
248	248	\subsection{What ChatGPT was and was not}
249	249	\label{sec:p2-chatgpt}
250	250
251		−On 30 November 2022 OpenAI released ChatGPT as a ``research preview'': a conversational interface to a language model of the GPT-3.5 series, fine-tuned by the reinforcement-learning-from-human-feedback method of InstructGPT, free to use through a web browser \citep{openai2022chatgpt}. No technical report accompanied the release; the announcement described the training procedure by reference to \citet{ouyang2022instructgpt} and listed limitations (plausible but incorrect answers, sensitivity to phrasing, verbosity, incomplete refusal of harmful requests) that the subsequent literature would study at length. It is important for the design of this paper to be precise about what was new. The underlying capabilities, few-shot prompting, instruction following, code generation, had been published between 2020 and 2022 \citep{brown2020gpt3,chen2021codex,ouyang2022instructgpt} and were available through a commercial programming interface. The architecture was five years old. What ChatGPT added was an interface that required no programming, a price of zero, and a conversational format that made the model's abilities and failures legible to anyone; its diffusion among the public was by all accounts faster than that of any previous software product, though the figures usually cited for it come from industry analysts rather than from verifiable sources and are not repeated here.
	251	+On 30 November 2022 OpenAI released ChatGPT as a ``research preview'': a conversational interface to a language model of the GPT-3.5 series, fine-tuned by the reinforcement-learning-from-human-feedback method of InstructGPT, free to use through a web browser \citep{openai2022chatgpt}. No technical report accompanied the release; the announcement described the training procedure by reference to \citet{ouyang2022instructgpt} and listed limitations (plausible but incorrect answers, sensitivity to phrasing, verbosity, incomplete refusal of harmful requests) that the subsequent literature would study at length. It is important for the design of this paper to be precise about what was new. The underlying capabilities, few-shot prompting, instruction following, code generation, had been published between 2020 and 2022 \citep{brown2020gpt3,chen2021codex,ouyang2022instructgpt} and were available through a commercial programming interface. The architecture was five years old. What ChatGPT added was an interface that required no programming, a price of zero, and a conversational format that made the model's abilities and failures legible to anyone; the figures usually cited for the speed of its public adoption come from industry analysts rather than from verifiable sources and are not repeated here.
252	252
253	253	For a study of the \emph{research} literature, this distinction has a consequence that is easily missed. The research community did not learn about large language models on 30 November 2022; it had been building and publishing them for three years. What changed on that date, if anything, was the attention of everyone else: funders, universities, firms, governments, and the researchers in subfields that had not previously had a reason to engage. If the date marks anything in the corpus, it should mark a change in \emph{who} publishes on language models and \emph{where}, more than a change in what the language-model community itself does. That expectation is formalised in H4 and H6 and in the reading of the cross-area citation flows in Part~VIII.
254	254
		`@@ -321,7 +321,7 @@ What this literature does not establish is whether the growth after 2022 differs`
321	321
322	322	The most direct evidence of a change in the literature after 2022 is a change in its language. In a preprint, \citet{liang2024mapping} estimated, with a distributional method that measures the frequency of words whose use the models favour, that the fraction of abstracts in computer science with substantial LLM-modified content rose rapidly from the end of 2022, with computer science leading other fields; the same group found a comparable rise in the reviews of major machine-learning conferences \citep{liang2024monitoring}, and later extended the method to writing across society \citep{liang2025widespread}; both are cited here in their preprint versions. \citet{kobak2025delving} obtained the same conclusion for biomedical abstracts by a different route, measuring the excess frequency of words such as ``delves'' and ``underscores'' relative to a pre-2023 baseline, in a study designed to need no training data from the models themselves; earlier and smaller preprint analyses of the same marker words had reached the same result \citep{geng2024writing,gray2024contamination}. Surveys of researchers confirm the adoption that the text analyses infer: a Nature survey of 1{,}600 scientists found wide use of generative tools for writing and coding, with strong concerns about reliability and misuse \citep{vannoorden2023survey}, and the multidisciplinary collection assembled by \citet{dwivedi2023sowhat} and the integrity analysis of \citet{binnashwan2023chatgpt} document the reactions of the research and education communities in the first months.
323	323
324		−For this study the finding is a measurement problem before it is a result. The classification of Section~\ref{sec:p2-ontology} and the embedding methods of Part~IV both read text, and text that is partly produced by a model trained on the pre-2023 literature is, in a precise sense, less informative about what is new in a paper. Two responses are built into the design. The venue and arXiv signals of equation~\eqref{eq:p2-score} are immune to prose; and the robustness checks of Appendix~D re-estimate the main breaks with classifications that use no keyword signals, so that a break present in the keyword-based series and absent in the venue-based one is flagged as possibly linguistic.
	324	+For this study the finding is a measurement problem before it is a result. The classification of Section~\ref{sec:p2-ontology} and the embedding methods of Part~IV both read text, and text that is partly produced by a model trained on the pre-2023 literature is, in a precise sense, less informative about what is new in a paper. Two responses are built into the design. The venue and arXiv signals of equation~\eqref{eq:p2-score} are immune to prose; and a planned robustness check (Appendix~D) would re-estimate the main breaks with classifications that use no keyword signals, so that a break present in the keyword-based series and absent in the venue-based one could be flagged as possibly linguistic; the classifier does not yet offer that variant, and the check is recorded as planned rather than performed.
325	325
326	326	The measurement of LLM-generated text in \emph{peer reviews} deserves separate mention because it bears on the quality of the venue signal. \citet{liang2024monitoring} estimated that a non-trivial share of the reviews submitted to several 2024 machine-learning conferences had been substantially modified by a language model, with the share higher among reviews submitted close to the deadline and among reviewers who did not respond to author rebuttals. If reviewing is partly delegated to models, then acceptance at a selective venue, which the ontology treats as the strongest community signal, is partly a judgement rendered by the models under study. The corpus cannot measure this directly; it is recorded here as a limitation of the venue signal in the research period that does not apply to the baseline.
327	327
		`@@ -333,7 +333,7 @@ A second body of work concerns not the quantity of research but its quality. \ci`
333	333	\subsection{Benchmark saturation and the reproducibility of evaluation}
334	334	\label{sec:p2-claim-benchmarks}
335	335
336		−The benchmark culture of Section~\ref{sec:p2-dl} produced a literature on its own dynamics. In a preprint, \citet{ott2022benchmarks} mapped the creation and saturation of benchmarks across AI tasks and found that many become saturated within a few years of their creation while others are abandoned before saturation; the critiques of \citet{dehghani2021lottery} and \citet{raji2021benchmark} argued that the choice of benchmark is itself an under-examined research decision and that no benchmark can measure general capability. The post-2022 literature extended the concern to \emph{contamination}, the presence of test data in the training corpora of models trained on the web, which makes a benchmark result uninterpretable without knowledge of the training data that the model reports of Table~\ref{tab:p2-timeline} typically withhold. \citet{kapoor2023leakage} documented data leakage as a general cause of irreproducible results in machine-learning-based science, and the agenda of \citet{hendrycks2021unsolved} placed robustness, monitoring and alignment among the open problems of the field. RQ7, which asks whether the period's advances are breakthroughs or incremental benchmark gains, and RQ9, on reproducibility and evaluation standards, are the corpus-level counterparts of these claims.
	336	+The benchmark culture of Section~\ref{sec:p2-dl} produced a literature on its own dynamics. \citet{ott2022benchmarks} mapped the creation and saturation of benchmarks across AI tasks and found that many become saturated within a few years of their creation while others are abandoned before saturation; the critiques of \citet{dehghani2021lottery} and \citet{raji2021benchmark} argued that the choice of benchmark is itself an under-examined research decision and that no benchmark can measure general capability. The post-2022 literature extended the concern to \emph{contamination}, the presence of test data in the training corpora of models trained on the web, which makes a benchmark result uninterpretable without knowledge of the training data that the model reports of Table~\ref{tab:p2-timeline} typically withhold. \citet{kapoor2023leakage} documented data leakage as a general cause of irreproducible results in machine-learning-based science, and the agenda of \citet{hendrycks2021unsolved} placed robustness, monitoring and alignment among the open problems of the field. RQ7, which asks whether the period's advances are breakthroughs or incremental benchmark gains, and RQ9, on reproducibility and evaluation standards, are the corpus-level counterparts of these claims.
337	337
338	338	\subsection{Concentration, disruption and the pace of science}
339	339	\label{sec:p2-claim-pace}

modified manuscript/sections/part03_data.tex +143 −87

		`@@ -29,7 +29,7 @@ coverage of DBLP, the publisher-centred coverage of Crossref, the preprint-centr`
29	29	aggregating coverage of OpenAlex each miss a distinct part of the literature and type the part they do cover
30	30	in distinct ways. Comparative studies of bibliographic data sources have documented large differences in
31	31	coverage between Web of Science, Scopus, Dimensions, Crossref and Microsoft Academic
32		−\citep{visser2021comparison,martinmartin2021comparison}, and the journal-level coverage of the two commercial
	32	+\citep{visser2021comparison,martinmartin2021coverage}, and the journal-level coverage of the two commercial
33	33	indexes is known to under-represent conference proceedings, which carry a large share of computer-science
34	34	output \citep{mongeon2016coverage}. For the open successor of Microsoft Academic, OpenAlex, the first
35	35	independent assessments report strong coverage of works with DOIs together with weaker and uneven metadata on
		`@@ -58,7 +58,9 @@ before parsing; every one of the 1{,}898 files parsed without error and the sum`
58	58	equals the manifest total of \LinkSlimTotal{} works. Two outputs were written during this single pass. The
59	59	first is the full JSON record of every work carrying \emph{any} topic in OpenAlex field~17 (Computer Science)
60	60	with a publication year between 2018 and 2026, stored as Zstandard-compressed JSON Lines and partitioned by
61		−the snapshot's own date partitions (\LinkOaCsRecords{} records, about 20~GB). The second is a \emph{slim}
	61	+the snapshot's own date partitions (\PrismaIdentifiedOpenalexPassOneCsTopicFilter{} records in this first
	62	+pass, about 20~GB; a second pass by identifier list, described in Section~\ref{sec:data-linkage}, later added
	63	+\PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{} records). The second is a \emph{slim}
62	64	table of every work in the snapshot (identifier, DOI, year, type, primary topic, subfield, field and domain,
63	65	citation count and a few flags), kept in Parquet so that works cited by computer-science papers but published
64	66	in other fields can be resolved for the interdisciplinary analyses of Part~VIII (decision D2 of
		`@@ -119,8 +121,8 @@ the caveats attached to its category labels and versioning, are discussed by \ci`
119	121	\subsection{ACL Anthology}
120	122
121	123	The ACL Anthology is the curated archive of the computational-linguistics community and has itself served as a
122		−bibliometric corpus \citep{radev2013acl}. Its BibTeX export with abstracts (\texttt{anthology+abstracts.bib.gz},
123		−46.7~MB, SHA-256 recorded) was downloaded on 2026-11-02 and parsed into a table of 138{,}796 entries, of which
	124	+bibliometric corpus \citep{radev2013aclanthology}. Its BibTeX export with abstracts (\texttt{anthology+abstracts.bib.gz},
	125	+46.7~MB, SHA-256 recorded) was downloaded on 2026-11-02 and parsed into a table of \AclParsedEntries{} entries, of which
124	126	\LinkAclRecords{} are conference papers or journal articles and \LinkAclTwoZeroOneEightTwoZeroTwoSix{} fall in
125	127	2018--2026. The Anthology plays a diagnostic role: because its venue structure is complete and hand-maintained,
126	128	the share of its entries that the backbone can be matched to is a direct measure of OpenAlex coverage for a
		`@@ -136,7 +138,7 @@ every DOI cited in this manuscript, a cross-check of publisher-assigned document`
136	138	cross-check of citation counts (\texttt{is-referenced-by-count}) on a sample of highly cited works in
137	139	Part~IX.
138	140
139		−The Semantic Scholar Academic Graph \citep{kinney2023s2} was listed in the mandate among the priority sources
	141	+The Semantic Scholar Academic Graph \citep{kinney2023s2ag} was listed in the mandate among the priority sources
140	142	because of two signals it alone provides, the influential-citation classification and citation intent. At the
141	143	time of the feasibility audit, anonymous requests to its API returned HTTP~429 with an instruction to apply for
142	144	a key, and the bulk-dataset endpoint requires a key outright. In keeping with the rule that inaccessible data
		`@@ -189,8 +191,10 @@ and conservative at this stage: a work whose primary topic lies in, say, statist`
189	191	but which carries a secondary computer-science topic is a candidate, and the decision whether it belongs to
190	192	the core or to the penumbra is deferred to the eligibility stage, where it is made explicit and counted
191	193	(Section~\ref{sec:data-core}). The filter yields
192		−\PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{} records in the first pass over the
193		−snapshot. The DBLP filter retains the publication records of type article, inproceedings, incollection, book,
	194	+\PrismaIdentifiedOpenalexPassOneCsTopicFilter{} records in the first pass over the snapshot; a second pass by
	195	+identifier list (Section~\ref{sec:data-linkage}) adds \PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{}
	196	+records that carry no computer-science topic but are reached through DBLP and arXiv identifiers, so that
	197	+\PrismaIdentifiedOpenalexTotalRecordsLoaded{} OpenAlex records are loaded in all. The DBLP filter retains the publication records of type article, inproceedings, incollection, book,
194	198	phdthesis and data with a year in $[2018, 2026]$: \PrismaIdentifiedDblpTwoZeroOneEightTwoZeroTwoSix{} records.
195	199	The arXiv filter retains the records of set \texttt{cs} whose first submission falls in the window:
196	200	\PrismaIdentifiedArxivCsTwoZeroOneEightTwoZeroTwoSix{} records. The ACL filter retains the conference papers
		`@@ -250,21 +254,26 @@ producing spurious links between generic titles.`
250	254
251	255	For DBLP, the order is: (i) DOI equality, (ii) arXiv identifier equality, restricted to records not yet linked
252	256	on either side, (iii) normalised title equality with a year difference of at most one, restricted to records
253		−not yet linked on either side. The DOI step linked \LinkLinkDblpOaDoi{} records and the title step
	257	+not yet linked on either side and, when a first author is recorded in both sources, to pairs whose
	258	+first-author surnames agree. The DOI step linked \LinkLinkDblpOaDoi{} records and the title step
254	259	\LinkLinkDblpOaTitleYear{}; the arXiv step linked none, because the extraction of arXiv URLs from the DBLP
255	260	\texttt{ee} fields returned no candidate (\LinkDblpWithArxiv{} records with an extracted identifier). The
256	261	\texttt{ee} fields of this dump evidently do not carry \texttt{arxiv.org/abs} URLs in the form the extractor
257	262	expects; whether the CoRR entries are reached by the DOI step instead depends on DBLP's current conventions for
258	263	the \texttt{ee} field, which the audit did not characterise, and the question is left to the linkage-precision
259		−review. The step is kept in the pipeline because other dumps may differ. In total \LinkDblpLinkedDistinct{} of the \LinkDblpTwoZeroOneEightTwoZeroTwoSix{} DBLP
260		−records in the window were linked to an OpenAlex work after the first pass, leaving \LinkDblpUnlinked{}
261		−unlinked. The composition of the unlinked set is informative: \LinkDblpUnlinkedByTypeArticle{} journal
262		−articles, \LinkDblpUnlinkedByTypeInproceedings{} conference papers, \LinkDblpUnlinkedByTypePhdthesis{}
263		−doctoral theses, \LinkDblpUnlinkedByTypeData{} data records, \LinkDblpUnlinkedByTypeIncollection{} chapters
264		−and \LinkDblpUnlinkedByTypeBook{} books. Of these, \LinkDblpUnlinkedWithDoiInSlim{} carry a DOI that
265		−\emph{does} exist in the slim table of all OpenAlex works, which means the work is in OpenAlex but was not
266		−caught by the computer-science topic filter; this is the second error of the topic classifier described above
267		−and the reason for the second pass.
	264	+review. The step is kept in the pipeline because other dumps may differ. The linkage is run on the union of
	265	+the two snapshot passes, so that every count in this section is a count after both passes. In total
	266	+\LinkDblpLinkedDistinct{} of the \LinkDblpTwoZeroOneEightTwoZeroTwoSix{} DBLP records in the window were
	267	+linked to an OpenAlex work, leaving \LinkDblpUnlinked{} unlinked. The composition of the unlinked set is
	268	+informative: \LinkDblpUnlinkedByTypeArticle{} journal articles, \LinkDblpUnlinkedByTypeInproceedings{}
	269	+conference papers, \LinkDblpUnlinkedByTypePhdthesis{} doctoral theses, \LinkDblpUnlinkedByTypeData{} data
	270	+records, \LinkDblpUnlinkedByTypeIncollection{} chapters and \LinkDblpUnlinkedByTypeBook{} books. A DBLP
	271	+record whose DOI exists in the slim table of all OpenAlex works but not among the loaded records describes a
	272	+work that is in OpenAlex but was not caught by the computer-science topic filter; this is the second error of
	273	+the topic classifier described above and the reason for the second pass. After that pass,
	274	+\LinkDblpUnlinkedWithDoiInSlim{} unlinked DBLP records are still in this situation; their full records were
	275	+not returned by the identifier scan, for a reason the pipeline has not characterised, and the list is kept
	276	+for the linkage-precision review.
268	277
269	278	For arXiv, the order is: (i) arXiv identifier equality against the identifier OpenAlex records for the work,
270	279	(ii) DOI equality for e-prints that carry a journal DOI, (iii) normalised title equality with a year
		`@@ -274,8 +283,8 @@ version \citep{lin2020preprints}. The steps linked \LinkLinkArxivOaArxivId{}, \L`
274	283	For the ACL Anthology, the order is: (i) DOI equality, (ii) equality between the Anthology identifier and the
275	284	identifier embedded in an \texttt{aclanthology.org} landing-page URL among the work's OpenAlex locations,
276	285	(iii) normalised title equality with a year difference of at most one. The steps linked
277		−\LinkLinkAclOaDoi{} records by DOI and \LinkLinkAclOaTitleYear{} by title, with the URL step contributing the
278		−remainder, for \LinkAclLinkedDistinctTwoZeroOneEightTwoZeroTwoSix{} distinct 2018--2026 entries linked out of
	286	+\LinkLinkAclOaDoi{} records by DOI and \LinkLinkAclOaTitleYear{} by title; the URL step contributed no link
	287	+in this snapshot, and is kept for the same reason as the DBLP arXiv step. The result is \LinkAclLinkedDistinctTwoZeroOneEightTwoZeroTwoSix{} distinct 2018--2026 entries linked out of
279	288	\LinkAclTwoZeroOneEightTwoZeroTwoSix{}; the residual is examined in Section~\ref{sec:data-quality} as a
280	289	measure of OpenAlex coverage of conference-heavy subfields.
281	290
		`@@ -284,15 +293,20 @@ measure of OpenAlex coverage of conference-heavy subfields.`
284	293	A DBLP record whose DOI resolves to an OpenAlex work outside the topic filter is, by the scope rule, a
285	294	computer-science work that the first pass did not load. The same situation arises for arXiv e-prints whose
286	295	journal DOI exists in OpenAlex without a computer-science topic. The linkage script therefore writes the list
287		−of OpenAlex identifiers that are needed but not loaded: \LinkNeededOaIdsSecondPassDblpDoi{} identifiers from
288		−DBLP DOIs and \LinkNeededOaIdsSecondPassArxivDoi{} from arXiv DOIs. A second pass over the works snapshot,
289		−restricted to this identifier list, retrieves the full records of these works; they are normalised into the
290		−same Parquet tables with a \texttt{\_p2} suffix, the DuckDB views are rebuilt over the union of both passes,
291		−and linkage is re-run so that the DBLP-linked set gains the works that the topic classifier had missed. The
292		−second pass is what makes the DBLP membership rule of Section~\ref{sec:data-core} operational: without it,
293		−``DBLP-indexed'' would silently mean ``DBLP-indexed and labelled computer science by OpenAlex'', and the DBLP
294		−correction would be void precisely where it matters. DBLP records with no OpenAlex counterpart after both
295		−passes (\PrismaLinkedDblpOnlyNoOpenalexRecord{} records before the second pass) are kept in a separate table
	296	+of OpenAlex identifiers that are needed but not loaded, from both routes. A second pass over the works
	297	+snapshot, restricted to this identifier list, retrieves the full records of these works
	298	+(\PrismaIdentifiedOpenalexPassTwoDblpArxivLinked{} records); they are normalised into the same Parquet
	299	+tables with a \texttt{\_p2} suffix, the DuckDB views are rebuilt over the union of both passes, and linkage
	300	+is re-run so that the DBLP-linked set gains the works that the topic classifier had missed. Rebuilt on the
	301	+union, the list of identifiers still needed shrinks to \LinkNeededOaIdsSecondPassDblpDoi{} from DBLP DOIs and
	302	+\LinkNeededOaIdsSecondPassArxivDoi{} from arXiv DOIs: every e-print whose journal DOI exists in OpenAlex was
	303	+reached, and the DBLP residual is the one discussed above. The second pass is what makes the DBLP membership
	304	+rule of Section~\ref{sec:data-core} operational: without it, ``DBLP-indexed'' would silently mean
	305	+``DBLP-indexed and labelled computer science by OpenAlex'', and the DBLP correction would be void precisely
	306	+where it matters. Because the second pass loads records by identifier without any restriction on their year,
	307	+it also admits a small number of works dated outside the window, which the eligibility stage removes under
	308	+code E1. DBLP records with no OpenAlex counterpart after both
	309	+passes (\PrismaLinkedDblpOnlyNoOpenalexRecord{} records) are kept in a separate table
296	310	with their year and type and are reported as such; they enter the volume counts of Part~IX only in the
297	311	sensitivity analysis that adds DBLP-only records, because they lack the abstracts, references and affiliations
298	312	on which every other indicator depends. Most of them are journal articles and conference papers with a DOI
		`@@ -307,12 +321,21 @@ e-prints whose OpenAlex record points to a different version. Title-based links`
307	321	titles of distinct works (extended abstracts and full papers, workshop and conference versions). The protocol
308	322	pre-specifies the estimation of linkage precision on a sample of 500 DBLP--OpenAlex pairs stratified by method,
309	323	with automatic agreement checks (normalised-title similarity, year difference, first-author surname) and a
310		−200-pair subsample reserved for the principal investigator (checkpoint H-1). The sampling script draws 250 DOI
311		−links, 200 title links and 50 arXiv links with a fixed seed; since the arXiv step produced no links in this
312		−dump, the realised sample has 450 pairs, and the precision estimate for the arXiv step is undefined rather
313		−than reported as perfect. At the time of writing the sampling script exists but the sample has not been drawn and the
314		−human checkpoint is unsigned; the automatic agreement rates and the human verdicts will be reported in Part~IX
315		−and Appendix~B, and every conclusion that depends on the title-based links is labelled as pending human review
	324	+subsample of two pairs in five reserved for the principal investigator (checkpoint H-1). The sampling script
	325	+draws 250 DOI links, 200 title links and 50 arXiv links with a fixed seed; since the arXiv step produced no
	326	+links in this dump, the realised sample (\texttt{data/metadata/linkage\_sample.csv}) has 450 pairs, of which
	327	+180 are reserved for the principal investigator, and the precision estimate for the arXiv step is undefined
	328	+rather than reported as perfect. The automatic check passes 235 of the 250 DOI pairs (0.94) and 200 of the
	329	+200 title--year pairs (1.00). The fifteen DOI pairs that fail the automatic check are not fifteen wrong links:
	330	+inspection of the sample shows that most fail on a title that OpenAlex truncated at the subtitle, on a
	331	+first-author name written in a different order or without diacritics, or on a one- or two-year difference
	332	+between the DBLP year and the OpenAlex year; a few pairs, however, carry substantively different titles under
	333	+the same DOI, which is the signature of a retitled version or of a DOI that resolves to a different item than
	334	+the one DBLP describes, and these are genuine candidates for error. The DOI stratum is therefore not
	335	+error-free, contrary to what an exact-key method would suggest, and the human verdicts on the reserved pairs
	336	+are what will settle the stratum precision. At the time of writing no verdict has been entered and the
	337	+checkpoint is unsigned; the stratum-weighted estimate of Part~IV will be reported in Part~IX and Appendix~B
	338	+once it is, and every conclusion that depends on the title-based links is labelled as pending human review
316	339	until H-1 is signed.
317	340
318	341	\section{Deduplication of preprint and published versions}
		`@@ -332,15 +355,19 @@ preprint and published versions into one node per work, in the sense required by`
332	355
333	356	The rule is conservative by design, favouring a small number of certain merges over a larger number of
334	357	probable ones. A \emph{preprint-like} record is one whose OpenAlex type is \texttt{preprint} or whose DOI is
335		−an arXiv DataCite DOI (prefix \texttt{10.48550/arxiv}); a \emph{published-like} record is any other record.
336		−Two records are paired when their normalised titles are identical, both titles have at least 25 characters,
337		−and their publication years differ by at most two. When a published record matches several preprints, the
338		−earliest preprint is kept as its preprint; when a preprint matches several published records, the earliest
339		−published record is kept; all other pairs are discarded, so that the pairing is one-to-one. The pairing is
340		−confined to the loaded OpenAlex records; preprints that are not computer-science candidates cannot be paired,
341		−which is immaterial since they would not enter the corpus. No fuzzy title similarity is used at this stage, so
342		−that a preprint whose title changed at publication is not merged; the consequence, a residual double count, is
343		−discussed below.
	358	+an arXiv DataCite DOI (prefix \texttt{10.48550/arxiv}); a \emph{published-like} record is any other record
	359	+that is itself an admissible twin: dated within the window, not an XPAC expansion record, titled, and of a
	360	+research document type or DBLP-linked. The twin conditions are imposed so that a preprint is never merged
	361	+into a record that the eligibility stage would then remove, which would lose the work altogether. A
	362	+preprint-like and a published-like record are paired by either of two routes: (i) identical normalised titles
	363	+of at least 25 characters with publication years differing by at most two, or (ii) a shared arXiv identifier,
	364	+taken from the arXiv linkage where available and from the OpenAlex record otherwise. When a published record
	365	+matches several preprints, the earliest preprint is kept as its preprint; when a preprint matches several
	366	+published records, the earliest published record is kept; all other pairs are discarded, so that the pairing
	367	+is one-to-one. The pairing is confined to the loaded OpenAlex records; preprints that are not computer-science
	368	+candidates cannot be paired, which is immaterial since they would not enter the corpus. No fuzzy title
	369	+similarity is used at this stage, so that a preprint whose title changed at publication is merged only when
	370	+the second route identifies it; the consequence, a residual double count, is discussed below.
344	371
345	372	\subsection{What is kept from each member of a pair}
346	373
		`@@ -348,22 +375,28 @@ The published record is the surviving node. The preprint record receives the fla`
348	375	the identifier of the published work and is excluded with reason code E5 (Section~\ref{sec:data-eligibility}).
349	376	The published node receives \texttt{preprint\_work\_id} and, through the arXiv linkage, the first-submission
350	377	date of the e-print, so that for every work the \emph{first date} is defined as the earliest of its OpenAlex
351		−publication date, the publication date of its preprint record and the arXiv \texttt{created} date. This first
	378	+publication date, the publication date of its preprint record and the arXiv \texttt{created} date, with one
	379	+exception stated in Section~\ref{sec:data-dates}: a publication date of the form \emph{YYYY}-01-01 is treated
	380	+as a year-only date and is overridden by the preprint or e-print date when one exists. This first
352	381	date is the date variable of all monthly series in Part~IX; the publication date of the version of record is
353	382	kept alongside it. The distinction matters for structural-change analysis: dating a paper by its journal issue
354	383	would displace much of the research activity of 2023 into 2024 and 2025 and would blur any monthly series
355	384	around the November 2022 boundary, whereas the first date places the work at the moment it entered the public
356		−record. Citation counts are those of the surviving published node; OpenAlex attributes to the preprint record
357		−the citations that cite the preprint DOI, and these are not added to the published node, a known
358		−under-count for works whose preprint remained the cited version. Part~IV states the resulting measurement
359		−assumption and Part~IX reports, for the high-priority set, the share of citations that accrue to preprint
360		−records.
	385	+record. OpenAlex attributes to the preprint record the citations that cite the preprint DOI; the impact
	386	+analyses (\texttt{bibliometrics/03\_citation\_impact.py}) therefore take the yearly citation counts of the
	387	+surviving node as the union of its own counts and those of its merged preprint record, so that a work whose
	388	+preprint remained the cited version is not under-counted. The merge is exact only for pairs the pairing found;
	389	+for the residual unmatched preprints discussed below, the citations stay divided between two nodes. Part~IV
	390	+states the resulting measurement assumption and Part~IX reports, for the high-priority set, the share of
	391	+citations that accrue to preprint records.
361	392
362	393	\subsection{Outcome and residual duplication}
363	394
364	395	The pairing identified \PrismaExcludedEFiveDuplicate{} preprint records as duplicates of a published work,
365	396	of which \PrismaExcludedSequentialEFiveDuplicate{} remained after the earlier exclusions had been applied.
366		−Set against the \PrismaEligibleByTypePreprint{} preprints that remain in the eligible corpus as works with no
	397	+Within the eligible corpus, \PrismaVersionRelationshipTitleYear{} works carry a preprint merged by the
	398	+title route and \PrismaVersionRelationshipArxivId{} by the shared-identifier route (a pair that satisfies
	399	+both is counted once, under either route). Set against the \PrismaEligibleByTypePreprint{} preprints that remain in the eligible corpus as works with no
367	400	published counterpart, the figure shows that most e-prints in the candidate set were not matched to a
368	401	published version within OpenAlex. Three explanations coexist and cannot be separated without manual review:
369	402	many e-prints are genuinely unpublished, consistent with the publication rates reported for computer-science
		`@@ -388,8 +421,10 @@ records each criterion removed given those before it.`
388	421	\subsection{Reason codes}
389	422
390	423	\textbf{E1, outside window.} The publication date is missing, earlier than 2018-01-01 or later than
391		−2026-11-02. Because the first-pass filter already restricted the publication year, this code catches only
392		−records with a missing date or a date beyond the extraction date. Forward-dated records, that is works whose
	424	+2026-11-02. Because the first-pass filter already restricted the publication year, this code catches, among
	425	+first-pass records, only those with a missing date or a date beyond the extraction date; it also removes the
	426	+second-pass records, loaded by identifier without a year restriction, whose publication date lies outside the
	427	+window. Forward-dated records, that is works whose
393	428	publication date lies in the future relative to the snapshot, are a documented feature of OpenAlex, where
394	429	publishers register articles for forthcoming issues; the window end removes those dated after the extraction
395	430	date, and the remaining 2026 records are discussed in Section~\ref{sec:data-dates}.
		`@@ -439,16 +474,17 @@ from every impact analysis (citation rankings, landmark selection, field-normali`
439	474	\textbf{E7, generic deposit platform.} Introduced by amendment A-002, this code excludes records whose primary
440	475	source is a generic deposit platform (Zenodo, Figshare, the Japanese Institutional Repositories DataBase,
441	476	LA~Referencia, Dialnet, the DOAJ-as-source entries and the OSTI OAI feed) unless the record is DBLP- or
442		−arXiv-linked. The number of records removed under this code is reported by the macro
443		−\pending{PrismaExcludedSequentialESevenGenericDeposit} once the corpus has been rebuilt under A-002.
444		−
445		−One inconsistency of numbering must be recorded. In the frozen protocol, code E7 designates the
446		−penumbra of works with a secondary computer-science topic only; the implementation reuses the number E7 for
447		−the generic-deposit exclusion of amendment A-002 and carries the penumbra as a separate flag,
448		−\texttt{cs\_penumbra}. The penumbra is not an exclusion in the same sense as E1--E7: it is a scope boundary,
449		−reported separately and used in sensitivity analyses, and is treated in Section~\ref{sec:data-core}. The
450		−conflict of numbering is flagged here for correction in the amendments log before the protocol is cited in
451		−its final form.
	477	+arXiv-linked. Independently of order, \PrismaExcludedESevenGenericDeposit{} records carry the flag;
	478	+sequentially, after the five preceding criteria, \PrismaExcludedSequentialESevenGenericDeposit{} records are
	479	+removed under this code.
	480	+
	481	+One change of numbering must be recorded. In the frozen protocol, code E7 designated the penumbra of works
	482	+with a secondary computer-science topic only; the implementation reuses the number E7 for the generic-deposit
	483	+exclusion of amendment A-002 and carries the penumbra as a separate flag, \texttt{cs\_penumbra}. The
	484	+penumbra is not an exclusion in the same sense as E1--E7: it is a scope boundary, reported separately and used
	485	+in sensitivity analyses, and is treated in Section~\ref{sec:data-core}. Amendment A-005 records the
	486	+redefinition, so that the code labels of the protocol, of the implementation and of this manuscript now
	487	+agree.
452	488
453	489	\subsection{Amendments A-001 and A-002 and their evidence}
454	490
		`@@ -500,23 +536,36 @@ least one of the following holds:`
500	536	\begin{enumerate}
501	537	\item it is linked to a DBLP record (venue-based membership);
502	538	\item its OpenAlex primary field is Computer Science (classifier-based membership);
503		−\item it carries a secondary computer-science topic \emph{and} is linked to an arXiv e-print whose primary
504		− category is in the \texttt{cs} archive (author-declared membership).
	539	+\item it is linked to an arXiv e-print whose primary category is in the \texttt{cs} archive
	540	+ (author-declared membership).
505	541	\end{enumerate}
506	542
507		−A record that survives the exclusions but satisfies none of these, which means it carries a secondary
508		−computer-science topic without venue or author support, belongs to the \emph{penumbra}. The three conditions
509		−correspond to three independent judgements about disciplinary membership: the judgement of DBLP's editors
510		−about venues, the judgement of OpenAlex's classifier about content, and the judgement of the authors who chose
511		−a primary arXiv category. Requiring agreement of any one of them with the first-pass topic filter removes the
512		−over-inclusion that a secondary topic alone would produce, while keeping works that any single source would
513		−have missed.
	543	+The third condition is the form given by amendment A-006. As frozen, it required in addition that the record
	544	+carry a computer-science topic in OpenAlex, which made the clause unreachable for exactly the works the second
	545	+pass recovers (works with no computer-science topic that an arXiv e-print declares as computer science); the
	546	+amendment removes the extra requirement and makes the corpus-definition script assert that every record
	547	+surviving the exclusions falls into exactly one of three classes, core, penumbra or the residual class
	548	+counted below. A record that survives the exclusions, carries a
	549	+computer-science topic and satisfies none of the three conditions, which means it has neither venue nor
	550	+author support, belongs to the \emph{penumbra}. The three conditions correspond to three independent
	551	+judgements about disciplinary membership: the judgement of DBLP's editors about venues, the judgement of
	552	+OpenAlex's classifier about content, and the judgement of the authors who chose a primary arXiv category.
	553	+Requiring any one of them removes the over-inclusion that a secondary topic alone would produce, while keeping
	554	+works that any single source would have missed.
514	555
515	556	\subsection{Counts}
516	557
517	558	After the sequential exclusions, \PrismaAfterExclusions{} loaded records remain. Of these,
518		−\PrismaCsCoreEligible{} satisfy the membership rule and constitute the eligible mapping corpus, and
519		−\PrismaCsPenumbraEligibleOtherwise{} form the penumbra. The penumbra is thus a substantial fraction of the
	559	+\PrismaCsCoreEligible{} satisfy the membership rule and constitute the eligible mapping corpus,
	560	+\PrismaCsPenumbraEligibleOtherwise{} form the penumbra, and \PrismaNeitherCoreNorPenumbraAfterExclusions{}
	561	+belong to neither: they carry no computer-science topic, having been loaded by the second pass, and are
	562	+neither DBLP-linked nor linked to an e-print with a \texttt{cs} primary category. By construction of the
	563	+second-pass identifier list, such records can only have been reached through the journal DOI of an e-print
	564	+harvested in the \texttt{cs} set but whose primary category lies in another archive (a cross-listed
	565	+e-print), and they are outside the corpus by the same reasoning that excludes the penumbra; this reading
	566	+follows from the construction and has not been verified record by record, which the linkage review will do.
	567	+The three classes partition the survivors exactly, and the corpus-definition script stops if they do not.
	568	+The penumbra is thus a substantial fraction of the
520	569	records that a naive ``any computer-science topic'' filter would have called computer science; its exclusion
521	570	from the primary analyses is the main difference between the present corpus and what an OpenAlex query alone
522	571	would return, and it is the reason why no count in this manuscript should be compared directly with the counts
		`@@ -536,8 +585,9 @@ PRISMA-style flow, drawn from the same JSON file that generates the macros of th`
536	585	\centering
537	586	\includegraphics[width=0.9\textwidth]{fig_prisma_flow.pdf}
538	587	\caption{PRISMA-style flow of record identification, exclusion and the final mapping corpus. Identification
539		−counts are per source and overlap; exclusions are sequential in the order E1--E7; the core and the penumbra
540		−partition the records that survive the exclusions. Source: \texttt{data/metadata/prisma\_counts.json},
	588	+counts are per source and overlap; exclusions are sequential in the order E1--E7; the core, the penumbra and
	589	+the small residual class of second-pass records with neither venue nor author support (text) partition the
	590	+records that survive the exclusions. Source: \texttt{data/metadata/prisma\_counts.json},
541	591	rendered by \texttt{figures/make\_figures.py}.}
542	592	\label{fig:prisma}
543	593	\end{figure}
		`@@ -579,7 +629,9 @@ section documents three distinct problems, their diagnosis in the corpus, and th`
579	629	Each eligible work carries up to three dates: the OpenAlex publication date of the surviving record, the
580	630	publication date of its merged preprint record where one exists, and the arXiv first-submission date where the
581	631	work is linked to an e-print. The \emph{first date} of a work is the earliest of the three
582		−(Section~\ref{sec:data-dedup}), and the \emph{first month} is its calendar month. The first date measures
	632	+(Section~\ref{sec:data-dedup}), except that a publication date of the form \emph{YYYY}-01-01, which is in
	633	+most cases a year-only date, is set aside whenever a preprint or e-print date exists, so that the first date
	634	+is then the earlier of those two; the \emph{first month} is its calendar month. The first date measures
583	635	the entry of the work into the public record, which is the quantity relevant to questions about the pace and
584	636	composition of research activity; the publication date of the version of record measures the pace of the
585	637	publishing system and is kept for the analyses of venues and of preprint-to-publication lags. The two
		`@@ -594,12 +646,13 @@ eligible corpus by year found that between about a fifth and somewhat over a qua`
594	646	\emph{YYYY}-01-01, with the proportion varying by source and type; dissertations, book chapters and records
595	647	from repositories are the most affected, journal articles with publisher-registered issue dates the least.
596	648	Amendment A-002 introduced the flag \texttt{date\_precision}, equal to \texttt{year} when the publication
597		−date is the first of January and \texttt{month} otherwise. The rule is a heuristic with a known cost: a work
598		−genuinely published on 1 January is classed as year-precision, and since the rule is applied to the
599		−publication date rather than to the first date, a year-precision work with an arXiv first date is correctly
600		−dated by its e-print. The counts of eligible works by precision class will be reported by the macros
601		−\pending{PrismaEligibleDatePrecisionMonth} and \pending{PrismaEligibleDatePrecisionYear} once the corpus has
602		−been rebuilt under A-002.
	649	+date is the first of January and the work has neither a merged preprint date nor an arXiv first-submission
	650	+date, and \texttt{month} otherwise. A work with a 1 January publication date and an e-print is thus
	651	+month-precision, because its first date is taken from the e-print rather than from the publication date
	652	+(Section~\ref{sec:data-dedup}). The rule is a heuristic with a known cost: a work genuinely published on
	653	+1 January and without a preprint is classed as year-precision. In the eligible corpus,
	654	+\PrismaEligibleDatePrecisionMonth{} works are month-precision and \PrismaEligibleDatePrecisionYear{} are
	655	+year-precision.
603	656
604	657	All monthly series in Part~IX use month-precision records only. Year-precision records enter the annual series,
605	658	where their date is correct, and the composition analyses that do not depend on the month. The exclusion of
		@@ -662,7 +715,8 @@ The rule adopted is conservative. The ``complete'' observation window for monthl
662	715	June~2026; the months from July to November 2026 are reported descriptively, separately, and never enter a
663	716	breakpoint test. Within the complete window, the structural-change models include the last months with
664	717	their observed values and the robustness section of Part~IX re-estimates every break with the window ending
665		−in December 2025, so that a result that depends on the 2026 months is identified as such. The API increment
	718	+in March 2026 and in August 2026, so that a result that depends on the last months of the window, in either
	719	+direction, is identified as such. The API increment
666	720	harvested after the snapshot (Section~\ref{sec:data-sources}) adds the works registered between the snapshot
667	721	date and the extraction date, which narrows but does not close the gap. Finally, forward-dated records,
668	722	works carrying a 2026 publication date for an issue not yet released, are retained when their date precedes
		`@@ -786,7 +840,7 @@ None of these biases is corrected by reweighting. The study reports them, bounds`
786	840	permits, and re-estimates its results on alternative corpora (strict core; corpus with penumbra; corpus with
787	841	XPAC research types; corpus with DBLP-only records; corpus with generic deposits) so that the reader can see
788	842	how far each conclusion depends on each scope decision. This is the approach that the science-of-science
789		−literature recommends for databases whose coverage is known to be uneven \citep{fortunato2018science}, and it
	843	+literature recommends for databases whose coverage is known to be uneven \citep{fortunato2018sos}, and it
790	844	is the only one compatible with a mandate that forbids the fabrication of corrections.
791	845
792	846	\section{Research-integrity procedures}
		`@@ -813,11 +867,13 @@ record \citep{nosek2018preregistration}; the repository's version history and th`
813	867	the available substitute, and the release of Phase~9 will include both. Every departure from the frozen
814	868	protocol is an entry in \texttt{protocol/amendments.csv} with an identifier, a date, the phase, the change,
815	869	the reason and the approver, which until the principal investigator signs is ``executing agent (pending
816		−PI)''. At the time of writing the log holds five entries: A-000 records that the mandate's acceptance
	870	+PI)''. At the time of writing the log holds seven entries: A-000 records that the mandate's acceptance
817	871	criteria were completed by the executing agent after the pasted text was truncated; A-001 and A-002 are the
818	872	eligibility amendments of Section~\ref{sec:data-eligibility}; A-003 and A-004 fix the budgets and the models
819		−of the LLM-based taxonomy validation and knowledge-graph extraction. Analyses added after the freeze are
820		−labelled exploratory in Part~IX.
	873	+of the LLM-based taxonomy validation and knowledge-graph extraction; A-005 records the renumbering of code
	874	+E7 and A-006 the revised arXiv clause of the membership rule (Sections~\ref{sec:data-eligibility}
	875	+and~\ref{sec:data-core}), both prompted by the writing and critique passes of the first manuscript draft. Analyses
	876	+added after the freeze are labelled exploratory in Part~IX.
821	877
822	878	\subsection{Immutable raw snapshots and provenance}
823	879

modified manuscript/sections/part04_methodology.tex +333 −140

		`@@ -35,21 +35,41 @@ Computer Science field, plus the second-pass records recovered through DBLP and`
35	35	arXiv identifiers, Section~\ref{sec:m-linkage}). Each record $w \in \mathcal{W}_0$
36	36	carries an OpenAlex identifier, a type $\tau(w)$ drawn from the OpenAlex
37	37	controlled vocabulary, a publication date $d_{\mathrm{pub}}(w)$, a creation date
38		−$d_{\mathrm{cr}}(w)$ (the date at which the record entered OpenAlex), a title, an
	38	+$d_{\mathrm{cr}}(w)$ (nominally the date at which the record entered OpenAlex;
	39	+Part III shows that for most records it reflects a re-ingestion in the second
	40	+half of 2025, so it is not used as an indexing date), a title, an
39	41	optional abstract, authorships, a primary source (venue), OpenAlex topics with
40	42	their subfield and field, a list of referenced works, and yearly citation counts.
41	43	Linked records from DBLP, arXiv and the ACL Anthology attach to $w$ a DBLP key,
42	44	an arXiv identifier with its arXiv creation date $d_{\mathrm{arx}}(w)$ and primary
43	45	category, and an ACL identifier.
44	46
45		−The temporal position of a work is its \emph{first date},
	47	+The temporal position of a work is its \emph{first date}, computed by
	48	+\texttt{systematic\_review/define\_corpus.py} from three candidate dates: the
	49	+publication date $d_{\mathrm{pub}}(w)$, the publication date
	50	+$d_{\mathrm{pub}}(\pi(w))$ of the preprint record $\pi(w)$ merged into $w$ by
	51	+the deduplication rule of Section~\ref{sec:m-dedup} when one exists, and the
	52	+arXiv creation date $d_{\mathrm{arx}}(w)$ of the linked arXiv record when one
	53	+exists. Let $A(w) \subseteq \{ d_{\mathrm{pub}}(\pi(w)),\, d_{\mathrm{arx}}(w) \}$
	54	+be the set of those two dates that are available for $w$. Then
46	55	\begin{equation}
47		− d(w) \;=\; \min\bigl\{\, d_{\mathrm{pub}}(w),\; d_{\mathrm{pub}}(\pi(w)),\; d_{\mathrm{arx}}(w) \,\bigr\},
	56	+ d(w) \;=\;
	57	+ \begin{cases}
	58	+ \min A(w) & \text{if } d_{\mathrm{pub}}(w) \text{ falls on 1 January and } A(w) \neq \emptyset,\\[2pt]
	59	+ \min\bigl( \{ d_{\mathrm{pub}}(w) \} \cup A(w) \bigr) & \text{otherwise.}
	60	+ \end{cases}
48	61	\label{eq:first-date}
49	62	\end{equation}
50		−where $\pi(w)$ is the preprint record merged into $w$ by the deduplication rule of
51		−Section~\ref{sec:m-dedup} when one exists, and each missing term is replaced by
52		−$d_{\mathrm{pub}}(w)$. The first date, not the date of the version of record, is
	63	+The second branch is the minimum of the available dates. The first branch is
	64	+the override that matters: OpenAlex records a year-only publication date as
	65	+the first of January of that year, and when a preprint or arXiv date exists
	66	+the code \emph{drops} the 1~January date from the minimum instead of letting
	67	+a fictitious date win by construction. The first date can therefore be later
	68	+than the publication date, with a consequence for the period assignment
	69	+below: a record whose publication date is 1~January 2022 and whose linked
	70	+arXiv version was created in March 2023 is dated March 2023 and belongs to
	71	+the post period, although its OpenAlex year is 2022. The first date, not the
	72	+date of the version of record, is
53	73	the quantity that places a work on the time axis, because the research question
54	74	concerns when ideas entered the literature rather than when a publisher released
55	75	them. The publication year $y(w)$ used for citation normalisation is, in
		`@@ -64,11 +84,22 @@ period $B = [t_0, t^{\ast})$ and a post period $P = [t^{\ast}, t_1]$:`
64	84	\begin{equation}
65	85	\mathrm{period}(w) \;=\;
66	86	\begin{cases}
67		− B & \text{if } t_0 \le d(w) < t^{\ast},\\
68		− P & \text{if } t^{\ast} \le d(w) \le t_1 .
	87	+ \text{pre} & \text{if } d(w) < t_0,\\
	88	+ B & \text{if } t_0 \le d(w) < t^{\ast},\\
	89	+ P & \text{if } t^{\ast} \le d(w) \le t_1,\\
	90	+ \text{future} & \text{if } d(w) > t_1 .
69	91	\end{cases}
70	92	\end{equation}
71		−Monthly series use the month $m(w)$ of the first date. Two facts about dates
	93	+The eligibility predicate of Section~\ref{sec:m-eligibility} constrains
	94	+$d_{\mathrm{pub}}(w)$, not $d(w)$, to the window, so an eligible work whose
	95	+preprint or arXiv date precedes 2018 receives the label \emph{pre}
	96	+(\PrismaEligibleByPeriodPre{} eligible works); such works are counted in the
	97	+annual tables, which are keyed by publication year, but belong to neither
	98	+period and to no monthly series, since the series start in January 2018. The
	99	+label \emph{future} is defined for completeness and is empty in the current
	100	+corpus: it could arise only if an arXiv date later than the window end
	101	+rescued a year-only record. Monthly series use the month $m(w)$ of the first
	102	+date. Two facts about dates
72	103	constrain everything that follows. First, a substantial share of OpenAlex
73	104	records carry a publication date equal to the first of January of their year,
74	105	which is the database's convention for year-only precision rather than a real
		`@@ -76,16 +107,31 @@ date. The pipeline therefore attaches to every record a flag`
76	107	\begin{equation}
77	108	\mathrm{prec}(w) \;=\;
78	109	\begin{cases}
79		− \text{year} & \text{if } d_{\mathrm{pub}}(w) \text{ falls on 1 January},\\
	110	+ \text{year} & \text{if } d_{\mathrm{pub}}(w) \text{ falls on 1 January and } A(w) = \emptyset,\\
80	111	\text{month} & \text{otherwise},
81	112	\end{cases}
82	113	\end{equation}
83	114	and every monthly series in this paper is computed on the month-precision
84	115	subset $\mathcal{W}^{\mathrm{m}} = \{w : \mathrm{prec}(w) = \text{month}\}$ only
85		−(amendment A-002). A genuine 1~January publication is thereby misclassified as
86		−year precision and dropped from monthly series; the loss is small and, more
87		−importantly, it does not depend on the period, so it cannot manufacture a break.
88		−Annual series use all eligible records. Second, the right end of the window is
	116	+(amendment A-002): \PrismaEligibleDatePrecisionMonth{} eligible works are at
	117	+month precision and \PrismaEligibleDatePrecisionYear{} at year precision. A
	118	+record whose 1~January date is overridden in equation~\eqref{eq:first-date}
	119	+counts as month precision, because the date that places it is a real one. Two
	120	+consequences follow. A genuine 1~January publication without a linked earlier
	121	+version is misclassified as year precision and dropped from the monthly
	122	+series. More importantly, the exclusion is not period-neutral: whether a
	123	+year-only record is rescued depends on whether it has a linked preprint or
	124	+arXiv version, and that linkage rate rises over the window with the growth of
	125	+arXiv deposit, so the fraction of eligible works that enter the monthly series
	126	+differs between the baseline and the post period, and Part III reports
	127	+year-only shares that vary by source and type. A change in level of a monthly
	128	+series could in principle be produced by a change in date precision alone.
	129	+The manuscript treats this as a maintained caveat rather than a solved
	130	+problem: the year-precision share is reported by year, source and type next to
	131	+the monthly series (Parts III and IX), and the annual series, which use all
	132	+eligible records, are the check that does not depend on precision. A
	133	+sensitivity run that redistributes year-only records over the months of their
	134	+year is not implemented. Second, the right end of the window is
89	135	censored: records published in the last months before the snapshot are
90	136	under-represented because OpenAlex indexes them with a lag. The treatment of this
91	137	right-truncation is specified in Section~\ref{sec:m-truncation}.
		`@@ -198,7 +244,10 @@ DBLP records are restricted to the years 2018--2026 and to the record types`
198	244	\item \textbf{title--year}: among records still unmatched on both sides,
199	245	restrict each side to normalised titles that are \emph{unique} within
200	246	that side, join on equal $\eta$, and keep the pair if the years differ
201		− by at most one.
	247	+ by at most one and the first-author surnames agree whenever both are
	248	+ available (the last whitespace-delimited token of the first author's
	249	+ name, lower-cased, taken from the DBLP author list and from the
	250	+ OpenAlex authorship at position zero).
202	251	\end{enumerate}
203	252	The uniqueness restriction in step 3 is the blocking device: it turns the join
204	253	into a one-to-one map and prevents a frequent title from generating a clique of
		`@@ -234,10 +283,18 @@ not label with any Computer Science topic; they motivate the second snapshot`
234	283	pass, in which these identifiers (\LinkNeededOaIdsSecondPassDblpDoi{} from DBLP
235	284	and \LinkNeededOaIdsSecondPassArxivDoi{} from arXiv) are fetched in full and the
236	285	linkage is rerun with the union of both passes. The corpus is therefore not the
237		−set of works that OpenAlex calls computer science; it is the union of that set
238		−with what DBLP and arXiv call computer science, which is the sense in which the
239		−mandate's requirement to ``cover the complete breadth of computer science''
240		−rather than only the OpenAlex topic model is operationalised.
	286	+set of works that OpenAlex calls computer science: by the membership rule of
	287	+Section~\ref{sec:m-eligibility}, as amended by A-006, it is the union of the
	288	+works whose primary OpenAlex topic lies in computer science with the works
	289	+that DBLP indexes and the works whose authors filed them under an arXiv
	290	+\texttt{cs.*} primary category, which is the sense in which the mandate's
	291	+requirement to ``cover the complete breadth of computer science'' rather than
	292	+only the OpenAlex topic model is operationalised. The first version of the
	293	+rule required an OpenAlex computer-science topic \emph{in addition to} the
	294	+arXiv category, which made the arXiv clause unreachable for exactly the
	295	+second-pass works it was meant to admit; amendment A-006 removed that
	296	+requirement after the critic's audit found the records falling between core
	297	+and penumbra.
241	298
242	299	\subsection{Preprint--publication deduplication}
243	300	\label{sec:m-dedup}
		`@@ -246,30 +303,48 @@ A work that exists as an arXiv preprint and as a published version is two`
246	303	OpenAlex records whenever OpenAlex has not merged them. Counting both would
247	304	double-count exactly the kind of work (AI preprints) whose growth is at issue
248	305	(CLAUDE.md §6.2). The script \texttt{systematic\_review/define\_corpus.py}
249		−identifies such pairs within $\mathcal{W}_0$ by a title rule:
	306	+identifies such pairs within $\mathcal{W}_0$ by two routes. Let
	307	+$\mathcal{P} = \{ w : \tau(w) = \text{preprint} \text{ or } \text{DOI}(w)
	308	+\text{ starts with } \text{10.48550/arxiv} \}$ be the preprint-like set and
	309	+$\mathcal{B}$ the set of admissible published twins,
	310	+\begin{equation}
	311	+ \mathcal{B} \;=\; \bigl\{ b \notin \mathcal{P} :\ d_{\mathrm{pub}}(b) \in [t_0, t_1],\;
	312	+ \neg\,\text{is\_xpac}(b),\; \text{title}(b) \neq \emptyset,\;
	313	+ \tau(b) \in T_{\mathrm{keep}} \vee \text{in\_dblp}(b) \bigr\},
	314	+\end{equation}
	315	+that is, records that would themselves pass the window, XPAC, title and type
	316	+exclusions of Section~\ref{sec:m-eligibility} (otherwise the preprint stays
	317	+as the canonical record). With $\|\eta(\cdot)\| \ge 25$ required of both
	318	+normalised titles,
250	319	\begin{equation}
251		− (p, b) \text{ is a candidate pair} \iff
252		− \eta(p) = \eta(b),\;\; \|y(p) - y(b)\| \le 2,\;\;
253		− p \in \mathcal{P},\;\; b \notin \mathcal{P},\;\; \|\text{title}\| \ge 25,
	320	+ (p, b),\ p \in \mathcal{P},\ b \in \mathcal{B},\ \text{is a candidate pair} \iff
	321	+ \bigl[\, \eta(p) = \eta(b) \wedge \|y(p) - y(b)\| \le 2 \,\bigr]
	322	+ \;\vee\; \bigl[\, \mathrm{arx}(p) = \mathrm{arx}(b) \neq \emptyset \,\bigr],
254	323	\label{eq:duppair}
255	324	\end{equation}
256		−where $\mathcal{P} = \{ w : \tau(w) = \text{preprint} \text{ or } \text{DOI}(w)
257		−\text{ starts with } \text{10.48550/arxiv} \}$ is the preprint-like set. A
	325	+where $\mathrm{arx}(\cdot)$ is the arXiv identifier of the record (the linked
	326	+one when the cascade of Section~\ref{sec:m-cascade} supplied it, otherwise the
	327	+identifier OpenAlex itself carries). The first route is labelled
	328	+\textit{title\_year} and the second \textit{arxiv\_id}; among eligible works
	329	+they account for \PrismaVersionRelationshipTitleYear{} and
	330	+\PrismaVersionRelationshipArxivId{} merged pairs respectively (Part III). A
258	331	preprint may match several published records and conversely; the rule keeps,
259	332	for each published record, the earliest-dated preprint and, for each preprint,
260	333	the earliest-dated published record, and discards any pair that is not the
261		−first in both orderings. The surviving pairs define the map $\pi$ of
262		−equation~\eqref{eq:first-date}: the published record $b$ receives
263		−$\pi(b) = p$, $\text{preprint\_date}(b) = d_{\mathrm{pub}}(p)$ and the earlier of
264		−the two dates as its first date; the preprint $p$ receives the flag
265		−\textit{e5\_duplicate} and leaves the eligible corpus. The version relationship
266		−is therefore kept on the surviving node, as the plan requires, but the match is
267		−by exact normalised title only. Two limitations follow. A preprint whose title
268		−changed between versions is not merged, and the pair survives as two works; a
269		−preprint and an unrelated published paper with an identical normalised title
270		−within two years are wrongly merged. The second error is rare for titles of at
271		−least 25 characters; the first is the more consequential one and biases the
272		−preprint share upward, which is the conservative direction for hypothesis H7.
	334	+first in both orderings (a pair found by both routes is counted once, and
	335	+which of the two labels it retains is not fixed by the rule). The surviving
	336	+pairs define the map $\pi$ of equation~\eqref{eq:first-date}: the published
	337	+record $b$ receives $\pi(b) = p$, $\text{preprint\_date}(b) = d_{\mathrm{pub}}(p)$
	338	+and the first date of equation~\eqref{eq:first-date}; the preprint $p$
	339	+receives the flag \textit{e5\_duplicate} and leaves the eligible corpus. The
	340	+version relationship is therefore kept on the surviving node, as the plan
	341	+requires, but a title match is exact. Two limitations follow. A preprint whose
	342	+title changed between versions is merged only if the two records share an
	343	+arXiv identifier, and otherwise survives as a second work; a preprint and an
	344	+unrelated published paper with an identical normalised title within two years
	345	+are wrongly merged. The second error is rare for titles of at least 25
	346	+characters; the first is the more consequential one and biases the preprint
	347	+share upward, which is the conservative direction for hypothesis H7.
273	348	Deduplication across the OpenAlex--DBLP boundary is not needed because DBLP
274	349	records are never counted as works: a DBLP record either links to an OpenAlex
275	350	record, in which case the OpenAlex record carries the flag \textit{in\_dblp}, or
		`@@ -299,14 +374,21 @@ flags are`
299	374	\end{align}
300	375	The membership flags encode the computer-science rule of the protocol:
301	376	\begin{align}
302		− \text{core}(w) &= \text{in\_dblp}(w) \;\vee\; \text{cs\_primary}(w) \;\vee\;
303		− \bigl(\text{cs\_any}(w) \wedge \text{arxiv\_cs\_primary}(w)\bigr),\\
	377	+ \text{core}(w) &= \text{in\_dblp}(w) \;\vee\; \text{cs\_primary}(w) \;\vee\; \text{arxiv\_cs\_primary}(w),\\
304	378	\text{penumbra}(w) &= \text{cs\_any}(w) \wedge \neg\,\text{core}(w),
305	379	\end{align}
306	380	where \text{cs\_primary} means that the primary OpenAlex topic belongs to field
307	381	17, \text{cs\_any} that some topic does, and \text{arxiv\_cs\_primary} that the
308		−linked arXiv record's primary category starts with \texttt{cs.}. The eligibility
309		−predicate is
	382	+linked arXiv record's primary category starts with \texttt{cs.} (amendment
	383	+A-006; the earlier form of the third clause, $\text{cs\_any} \wedge
	384	+\text{arxiv\_cs\_primary}$, is discussed in Section~\ref{sec:m-cascade}). A
	385	+record that survives the exclusions but is neither core nor penumbra---no
	386	+computer-science topic at all, not DBLP-indexed, and no \texttt{cs.} arXiv
	387	+primary category, typically a second-pass work reached through an arXiv DOI
	388	+whose arXiv primary category lies outside computer science---forms a third
	389	+class of \PrismaNeitherCoreNorPenumbraAfterExclusions{} records, reported in
	390	+the PRISMA flow of Part III; the script asserts that the three classes
	391	+partition the surviving records. The eligibility predicate is
310	392	\begin{equation}
311	393	E(w) \;=\; \neg e_1 \wedge \neg e_2 \wedge \neg e_3 \wedge \neg e_4 \wedge \neg e_5 \wedge \neg e_7 \wedge \text{core}(w).
312	394	\label{eq:eligible}
		`@@ -349,10 +431,15 @@ last stratum is empty in the current run, for the reason given in`
349	431	Section~\ref{sec:m-cascade}), selected by ordering each stratum on a hash of the
350	432	DBLP key salted with the seed 44 and taking the first records, which is a
351	433	reproducible simple random sample within stratum. For each pair the script
352		−computes an automatic agreement vector: a string similarity between the two raw
353		−titles (Python's \texttt{difflib} ratio, twice the number of matched characters
354		−in the longest-matching-block alignment divided by the total length), the absolute year difference, and whether the last tokens of the two
355		−first-author names coincide; a pair is \textit{auto\_ok} if the similarity is at
	434	+computes an automatic agreement vector: a string similarity between the two
	435	+titles after a normalisation close to $\eta$ (NFKD decomposition with
	436	+non-ASCII characters dropped, lower-casing, runs of non-alphanumeric
	437	+characters collapsed to one space), namely Python's \texttt{difflib} ratio
	438	+(twice the number of matched characters in the longest-matching-block
	439	+alignment divided by the total length), the absolute year difference, and
	440	+whether the last tokens of the two first-author names coincide (lower-cased,
	441	+diacritics stripped, DBLP's four-digit disambiguation suffix removed); a pair
	442	+is \textit{auto\_ok} if the similarity is at
356	443	least 0.9, the years differ by at most one and the surnames agree whenever both
357	444	are present. Two of every five pairs are reserved for the principal
358	445	investigator (\textit{pi\_review}), whose verdicts constitute the human
		`@@ -365,8 +452,19 @@ weighted estimate`
365	452	\label{eq:linkprec}
366	453	\end{equation}
367	454	where $L_s$ is the number of links of method $s$ in the full table and $L$
368		−their sum, so that the title--year stratum, which is the only one where errors
369		−are plausible, is weighted by its small share of the links. The estimate
	455	+their sum, so that each stratum is weighted by its share of the links; since
	456	+the DOI step produces the great majority of them, $\hat p$ is dominated by the
	457	+DOI stratum. This matters because the DOI stratum is not the error-free one.
	458	+The automatic check (\texttt{data/metadata/linkage\_sample\_summary.json})
	459	+passes every title--year pair of the sample but fails a small number of DOI
	460	+pairs, of two kinds: a DOI that covers an entire edited volume, so that a
	461	+chapter is linked to the record of the book or of another chapter, and a
	462	+version of record whose title differs from the DBLP title under the same DOI.
	463	+Only the first kind is a linkage error; the second is a correct link that
	464	+the title test rejects. An identifier match is thus near-conclusive for the
	465	+existence of a relation between the two records but not for the identity of
	466	+the work, and the principal investigator's verdicts (H-1), not the automatic
	467	+rate, settle the precision of each stratum. The estimate
370	468	\pending{linkage precision from the H-1 sample, data/metadata/linkage\_sample\_summary.json}
371	469	is reported in Part III once the checkpoint is signed; until then the automatic
372	470	agreement rate is the only available figure and is labelled as such. Recall is
		`@@ -564,7 +662,8 @@ definitions follow the review of \citet{waltman2016review} where a choice had`
564	662	to be made; what is specific to this study is the citation-window discipline
565	663	imposed by the eight-year span of the corpus and the explicit treatment of the
566	664	two counting conventions. The scripts are \texttt{bibliometrics/01\_volume\_series.py},
567		−\texttt{02\_collaboration.py} and \texttt{03\_citation\_impact.py}.
	665	+\texttt{02\_collaboration.py}, \texttt{03\_citation\_impact.py} and
	666	+\texttt{04\_diversity\_cagr.py}.
568	667
569	668	\subsection{Volume, composition and growth}
570	669
		`@@ -585,10 +684,17 @@ compound annual growth rate`
585	684	applied to annual totals and to annual category totals; it is a descriptive
586	685	summary of two end points and says nothing about the path between them, which
587	686	is why the structural-change analysis of Section~\ref{sec:m-structural} works
588		−on the full monthly series instead. The CAGR is computed at table-making time
589		−from the annual counts rather than by a dedicated script; the plan lists it
590		−among the Phase-4 indicators and it is reported for complete years only, since
591		−2026 is right-censored (Section~\ref{sec:m-truncation}).
	687	+on the full monthly series instead. The CAGR is computed by
	688	+\texttt{bibliometrics/04\_diversity\_cagr.py} on the annual primary-label
	689	+counts of each category (\texttt{results/graph/category\_counts\_yearly.csv})
	690	+over two spans of complete years, 2018 to 2022 ($\Delta = 4$) within the
	691	+baseline and 2023 to 2025 ($\Delta = 2$) within the post period, together with
	692	+the category's share of the year's classified works in 2018, 2022 and 2025;
	693	+it is undefined when the count of the start year is zero. The plan lists it
	694	+among the Phase-4 indicators; 2026 is excluded because it is right-censored
	695	+(Section~\ref{sec:m-truncation}). The two spans split the window at the
	696	+calendar year, so the whole of 2022 is counted in the baseline span, unlike the
	697	+periods of Section~\ref{sec:m-notation}.
592	698
593	699	\subsection{Fixed citation windows}
594	700	\label{sec:m-citations}
		`@@ -612,6 +718,21 @@ receive windows that differ by eleven months in effective length. This`
612	718	granularity is imposed by the data (OpenAlex does not provide citation dates
613	719	below the year) and is the same for the baseline and the post period, so it
614	720	blurs comparisons within a year but does not bias comparisons between years.
	721	+Two further conventions of \texttt{03\_citation\_impact.py} govern every
	722	+impact statistic. The citations of a canonical node include those of its
	723	+merged preprint: the yearly counts of $\pi(w)$ are appended to those of $w$
	724	+before the windows are summed (the \texttt{CY} union in the script), so a
	725	+work that circulated as a preprint and was then published is credited with
	726	+the citations of both records. This is deliberate, since the two records are
	727	+one work, but it means that $c^{(h)}_w$ can exceed what the OpenAlex record of
	728	+the version of record shows on its own; the union applies only to pairs
	729	+found by the rule of Section~\ref{sec:m-dedup}, a preprint that OpenAlex has
	730	+already merged with its publication being a single record. And the population
	731	+is the set of eligible, \emph{non-retracted} works: retracted works stay in the
	732	+mapping corpus (Section~\ref{sec:m-eligibility}) but are excluded from the
	733	+window sums, from the reference classes below and from the inequality
	734	+statistics, so the yearly $n$ of the citation tables falls short of the
	735	+volume tables by the number of retracted works.
615	736	The window is the device that neutralises citation-age bias: comparing the
616	737	total citation count of a 2019 work with that of a 2025 work compares seven
617	738	years of exposure with one, whereas comparing $c^{(36)}$ compares equal
		`@@ -619,7 +740,7 @@ exposures. The cost is that the last window-length of the corpus cannot be`
619	740	evaluated; $c^{(36)}$ is complete for publication years up to 2023 given a
620	741	snapshot taken late in 2026, and $c^{(12)}$ up to 2024. Tables flag the years
621	742	for which a window is incomplete, and the choice of window length follows the
622		−evidence in \citet{wang2013window} that short windows rank works materially
	743	+evidence in \citet{wang2013citationwindow} that short windows rank works materially
623	744	differently from long ones in slow-moving fields, which is why two lengths are
624	745	reported rather than one.
625	746
		`@@ -628,14 +749,14 @@ reported rather than one.`
628	749	Raw citation counts differ across subfields and document types by factors that
629	750	have nothing to do with influence. The normalised citation score of a work is
630	751	its window count divided by the mean window count of its reference class
631		−\citep{waltman2016review, waltman2019fieldnorm}:
	752	+\citep{waltman2016review, waltman2019fieldnormalization}:
632	753	\begin{equation}
633	754	\mathrm{NCS}^{(h)}_w \;=\; \frac{c^{(h)}_w}{\bar c^{(h)}\bigl(y(w), \tau(w), f(w)\bigr)}, \qquad
634	755	\bar c^{(h)}(y, \tau, f) = \frac{1}{\|\mathcal{W}_{y,\tau,f}\|}\sum_{v \in \mathcal{W}_{y,\tau,f}} c^{(h)}_v ,
635	756	\label{eq:ncs}
636	757	\end{equation}
637	758	where $f(w)$ is the primary OpenAlex subfield of $w$ and $\mathcal{W}_{y,\tau,f}$
638		−the set of eligible works sharing year, type and subfield. The reference class
	759	+the set of eligible, non-retracted works sharing year, type and subfield. The reference class
639	760	is computed on the eligible corpus, not on all of OpenAlex, so the
640	761	normalisation is relative to computer science as delimited here: an NCS of 2
641	762	means twice the mean of comparable computer-science works, not twice the world
		`@@ -660,7 +781,8 @@ reasons FWCI is reported for comparison and never used to select works.`
660	781	\subsection{Inequality and concentration}
661	782
662	783	Citation inequality within a publication year is measured by the Gini
663		−coefficient of $c^{(36)}$ among the $n$ eligible works of that year. With the
	784	+coefficient of $c^{(36)}$ (and, as a secondary figure, of $c^{(12)}$) among
	785	+the $n$ eligible, non-retracted works of that year. With the
664	786	works sorted so that $c_{(1)} \le c_{(2)} \le \dots \le c_{(n)}$, the estimator
665	787	implemented in SQL is the rank form
666	788	\begin{equation}
		`@@ -691,8 +813,13 @@ largest single share are reported with the index. Because arXiv is a single`
691	813	OpenAlex source, growth in preprints mechanically raises $\mathrm{HHI}_y$ even
692	814	when the journal and conference landscape becomes more dispersed; this is the
693	815	reason the protocol's H5 predicts a stable or falling HHI only ``because of
694		−preprint growth'' with the preprint source treated separately, and the tables
695		−report the index with and without the preprint type.
	816	+preprint growth'' with the preprint source treated separately. The
	817	+implemented table (\texttt{venue\_hhi\_yearly.csv}, written by
	818	+\texttt{02\_collaboration.py}) computes the index over all types in a single
	819	+pass; a variant that excludes the preprint type is not computed, so the
	820	+preprint contribution has to be judged from the largest single share reported
	821	+next to the index, which in the years when arXiv is the largest source is
	822	+arXiv's share.
696	823
697	824	\subsection{Collaboration, countries and institutions}
698	825
		`@@ -725,10 +852,14 @@ table notes.`
725	852	\subsection{Diversity indices}
726	853	\label{sec:m-diversity}
727	854
728		−Interdisciplinarity is measured on the distribution of a work's references, or
729		−of a category's citations, over the categories of the ontology. Let
730		−$p = (p_1, \dots, p_K)$ be such a distribution and $\delta_{ij} \in [0,1]$ a
731		−dissimilarity between categories $i$ and $j$. The Rao--Stirling index
	855	+Interdisciplinarity is measured on a distribution over the categories of the
	856	+ontology; the protocol speaks of the distribution of a work's references or of
	857	+a category's citations, whereas the implemented pipeline applies the indices
	858	+to the \emph{category mix} of each year, the distribution of the year's
	859	+classified eligible works over their primary categories (the reference-based
	860	+variant is not computed). Let $p = (p_1, \dots, p_K)$ be such a distribution
	861	+and $\delta_{ij} \in [0,1]$ a dissimilarity between categories $i$ and $j$.
	862	+The Rao--Stirling index
732	863	\citep{rao1982diversity, stirling2007diversity}
733	864	\begin{equation}
734	865	\Delta_{\mathrm{RS}}(p) \;=\; \sum_{i \ne j} p_i\, p_j\, \delta_{ij}
		`@@ -760,21 +891,28 @@ categories (large $q$ counts only the dominant ones). Following`
760	891	closed form ${}^{2}D^{Z}(p) = 1 / \sum_{i,j} p_i z_{ij} p_j = 1/(1 - \Delta_{\mathrm{RS}}(p))$
761	892	when $z_{ii} = 1$, so that the two families are monotone transforms of each
762	893	other at this order and differ only in scale and in their behaviour at the
763		−saturation end. The dissimilarity $\delta_{ij}$ between categories is one
764		−minus the cosine similarity of the category centroids of
765		−Section~\ref{sec:m-semantic} in the baseline period, so that the disparity
766		−component is estimated from the corpus rather than imposed. The current
767		−pipeline computes the category flow matrices from which $p$ is formed
768		−(Section~\ref{sec:m-networks}) and the centroid similarities, but the two
769		−diversity indices are not yet produced by a script; they are defined here
770		−because the protocol commits to them, and their values in Part IX are
771		−The yearly Rao--Stirling and Leinster--Cobbold indices of the category mix,
772		−with bootstrap intervals from multinomial resampling of works over categories
773		−($B = 500$, seed 44), are computed by \texttt{bibliometrics/04\_diversity\_cagr.py}
774		−and reported in Part~IX; the dissimilarity $d_{ij}$ is one minus the cosine
775		−similarity of category centroids when embeddings are available and one minus
776		−the co-classification Jaccard similarity otherwise.
777		−until the computation is added and audited.
	894	+saturation end. Both indices are computed for every year up to 2025 by
	895	+\texttt{bibliometrics/04\_diversity\_cagr.py} and reported in Part~IX\@. The
	896	+dissimilarity is $\delta_{ij} = 1 - z_{ij}$, where $z_{ij}$ is the cosine
	897	+similarity of the category centroids of Section~\ref{sec:m-novelty} when the
	898	+centroid matrix exists (the script takes the post-period matrix first and
	899	+the baseline matrix as a fallback) and, when no centroid matrix is available,
	900	+the co-classification Jaccard similarity $n_{ij} / (n_i + n_j - n_{ij})$
	901	+computed from the yearly co-occurrence counts of the full label sets
	902	+(Section~\ref{sec:m-networks}), with $n_i$ the multi-label total of category
	903	+$i$; the diagonal is set to one in both cases and the similarity source
	904	+actually used is written into the output table, because values computed from
	905	+the two sources are not on the same scale and must not be compared across
	906	+them. The disparity component is thus estimated from the corpus rather than
	907	+imposed. The Rao--Stirling index carries a percentile interval from $B = 500$
	908	+multinomial resamples of the year's works over the categories (seed 44), the
	909	+same resamples that give the yearly category-share intervals; the
	910	+Leinster--Cobbold index of order two is reported without an interval. The
	911	+multinomial bootstrap treats the category assignments of a year as
	912	+independent draws from the observed mix, so its interval reflects only that
	913	+sampling variability and not the classification error of
	914	+Section~\ref{sec:m-validation}, which is the larger source of uncertainty for
	915	+these indices.
778	916
779	917	\section{Citation and collaboration networks}
780	918	\label{sec:m-networks}
		@@ -1017,7 +1155,7 @@ that are new; the interpretation of high novelty as ``new'' rather than
1017	1155	``interdisciplinary'' requires the clusters and the flow statistics as
1018	1156	corroboration, and the literature on novelty indicators warns that
1019	1157	citation-based and text-based novelty measures do not agree well
1020		−\citep{uzzi2013atypical, wang2017novelty}.
	1158	+\citep{uzzi2013novelty, wang2017novelty}.
1021	1159
1022	1160	\section{Structural change in monthly series}
1023	1161	\label{sec:m-structural}
		`@@ -1042,8 +1180,12 @@ uses no causal vocabulary about it (CLAUDE.md §6.5).`
1042	1180	\label{sec:m-truncation}
1043	1181
1044	1182	Each series is a vector of monthly counts $N_t$ on the month-precision
1045		−subset, from February 2018 (the default start of the driver script, which
1046		−drops one month and is immaterial) to the last complete month
	1183	+subset, from February 2018 (the driver's default \texttt{--first}; the
	1184	+script documents no reason for omitting January 2018, the omission shortens
	1185	+every series by one baseline month, to \ItsSeriesTotalNMonths{} months for
	1186	+the default end, and the manuscript makes no claim about its effect beyond
	1187	+noting that the omitted month is the first of the baseline and enters only the
	1188	+pre-trend) to the last complete month
1047	1189	$t_{\mathrm{L}}$, by default June 2026, with sensitivity runs at March and
1048	1190	August 2026. Category series $N_{k,t}$ use the multi-label count restricted to
1049	1191	month-precision works, and a series is analysed only if it has at least 48
		`@@ -1064,10 +1206,20 @@ years assumes that the profile is stationary, and the profile is itself`
1064	1206	affected by the events under study (OpenAlex's own ingestion changed over the
1065	1207	period); a correction would import that assumption into every test. Truncating
1066	1208	to the last month at which the lag profile indicates near-complete indexing
1067		−loses a few months of data but introduces no model. Donut windows (dropping
1068		−the months adjacent to the event) are not implemented; the alternative event
1069		−dates of Section~\ref{sec:m-altdates} serve the related purpose of showing
1070		−whether the estimated change is tied to a particular month.
	1209	+loses a few months of data but introduces no model. A donut specification is
	1210	+also run (\texttt{--donut}, default three): the ITS of Section~\ref{sec:m-its}
	1211	+is re-estimated for the reference date after removing the event month and the
	1212	+three months on each side of it (every month whose first day lies within
	1213	+$3 \times 31$ days of the first day of the event month, seven months in all,
	1214	+leaving \ItsSeriesTotalItsChatgptDonutN{} observations for the total series),
	1215	+so that the level and slope changes are identified from months at least a
	1216	+quarter away from the marker and not from the transition months themselves.
	1217	+For the total series the donut estimate of the level change is
	1218	+\ItsSeriesTotalItsChatgptDonutLevelChange{} in log points with HAC
	1219	+$p$-value \ItsSeriesTotalItsChatgptDonutLevelP{}, to be read against the
	1220	+full-sample estimate of Part IX\@. The alternative event dates of
	1221	+Section~\ref{sec:m-altdates} serve the related purpose of showing whether the
	1222	+estimated change is tied to a particular month.
1071	1223
1072	1224	\subsection{Interrupted time series with seasonality and HAC errors}
1073	1225	\label{sec:m-its}
		`@@ -1119,9 +1271,14 @@ The placebo $p$-value of the true event is its rank among these:`
1119	1271	\label{eq:placebo}
1120	1272	\end{equation}
1121	1273	and likewise for $\beta_3$. The set $\mathcal{T}'$ contains the true event, so
1122		−the smallest attainable value is $1/\|\mathcal{T}'\|$. This statistic is the
	1274	+the smallest attainable value is $1/\|\mathcal{T}'\|$. A second rank,
	1275	+restricted to the candidate dates \emph{before} the true event (the plan's
	1276	+original wording), is computed alongside it; because that restriction removes
	1277	+the true event from the candidate set, the baseline-only rank can be exactly
	1278	+zero when no baseline date produces a larger $t$-statistic, and Part IX states
	1279	+this convention next to the two values. This statistic is the
1123	1280	``placebo in time'' diagnostic recommended for interrupted time series
1124		−\citep{bernal2018controls}: if a level shift of comparable $t$-statistic is
	1281	+\citep{bernal2018itscontrol}: if a level shift of comparable $t$-statistic is
1125	1282	found at a quarter of all candidate dates, a conventional $p$-value from the
1126	1283	HAC standard error of $\beta_2$ overstates the evidence, because the series
1127	1284	has enough curvature and persistence that any date looks like a break. The
		`@@ -1129,8 +1286,8 @@ value is not a $p$-value under a null hypothesis in the usual sense (the`
1129	1286	candidate dates are not exchangeable draws from a null distribution, and the
1130	1287	regressions share the same data), and the manuscript calls it a placebo rank.
1131	1288	Two implementation details depart from the plan's wording. The plan speaks of
1132		−placebo breaks ``at every month of the baseline''; the implementation ranges
1133		−over the baseline \emph{and} the post period, so that a change in 2023 or
	1289	+placebo breaks ``at every month of the baseline''; the primary implementation
	1290	+ranges over the baseline \emph{and} the post period, so that a change in 2023 or
1134	1291	2024 that is larger than the one at the reference date lowers the rank of the
1135	1292	reference date, which is the conservative direction for hypothesis H1 and is
1136	1293	why the wider range was kept. And the minimum of 24 pre-months and 12
		`@@ -1141,10 +1298,10 @@ comparison.`
1141	1298	\subsection{Multiple breaks: dynamic programming with information criteria}
1142	1299	\label{sec:m-bp}
1143	1300
1144		−The Bai--Perron framework \citep{bai1998breaks, bai2003computation} estimates
	1301	+The Bai--Perron framework \citep{bai1998breaks, bai2003breaks} estimates
1145	1302	$m$ unknown break dates $\tau_1 < \dots < \tau_m$ as the global minimiser of
1146	1303	the sum of squared residuals of a segment-wise regression. The implementation
1147		−is the pure dynamic programme of \citet{bai2003computation} on the log series
	1304	+is the pure dynamic programme of \citet{bai2003breaks} on the log series
1148	1305	with a segment-wise linear trend: for a segment $[i, j)$ of at least $h$
1149	1306	observations,
1150	1307	\begin{equation}
		`@@ -1159,8 +1316,9 @@ is obtained recursively,`
1159	1316	with the break positions recovered by back-tracking. The minimum segment
1160	1317	length is $h = \max\{8, \lfloor 0.15\, n \rfloor\}$ months, a trimming of 15 per
1161	1318	cent that is the usual choice in this literature; it forbids breaks in the
1162		−first and last $h$ months, which for the default window of 101 months
1163		−($h = 15$) means that no break can be placed after April 2025, and it also
	1319	+first and last $h$ months, which for the default window of
	1320	+\ItsSeriesTotalNMonths{} months ($h = 15$) means that no break can be placed
	1321	+after April 2025, and it also
1164	1322	forbids two breaks closer than $h$ months. The number of breaks, up to three, is selected by two
1165	1323	criteria computed on the global minima $\mathrm{SSR}_m = C_m(n)$,
1166	1324	\begin{align}
		`@@ -1170,7 +1328,7 @@ criteria computed on the global minima $\mathrm{SSR}_m = C_m(n)$,`
1170	1328	with $p_m = 2(m+1) + m$ parameters (an intercept and a slope per segment plus
1171	1329	the $m$ break dates). The BIC is Schwarz's criterion \citep{schwarz1978bic};
1172	1330	the LWZ criterion of \citet{liu1997segmented}, in the form used by
1173		−\citet{bai2003computation}, penalises additional breaks more heavily and is
	1331	+\citet{bai2003breaks}, penalises additional breaks more heavily and is
1174	1332	known to select fewer spurious breaks when the errors are serially correlated,
1175	1333	which they are in these series (the monthly residual autocorrelation is what
1176	1334	the HAC errors of the ITS correct for). Both selections are reported. The
		`@@ -1178,9 +1336,13 @@ sequential sup-$F$ tests of \citeauthor{bai1998breaks} are not implemented; in`
1178	1336	their absence the break dates carry no $p$-value, and their stability is
1179	1337	assessed instead through the agreement of the two criteria, the sensitivity to
1180	1338	$t_{\mathrm{L}}$, and the agreement with the two methods below. The regression
1181		−in each segment contains no seasonal terms, so a seasonal pattern can only be
1182		−absorbed by the error; with a trimming of eight or more months per segment
1183		−this cannot create a break by itself but it inflates $\mathrm{SSR}_m$ uniformly.
	1339	+in each segment contains no seasonal terms; instead the driver seasonally
	1340	+adjusts the log series before the search, subtracting from each month the
	1341	+mean deviation of its calendar month from the overall mean, both means
	1342	+computed on the baseline months only (before December 2022) so that the
	1343	+adjustment uses no post-period information. The same adjusted series is the
	1344	+input of the PELT and BOCPD methods below; only the ITS models seasonality
	1345	+inside the regression.
1184	1346
1185	1347	\subsection{PELT}
1186	1348
		`@@ -1306,7 +1468,8 @@ retrospective evaluation with a fixed vocabulary and is stated as the first`
1306	1468	limitation of the framework.
1307	1469
1308	1470	The panel consists of two $K \times M$ matrices over topics $k$ and months
1309		−$t$ from January 2018 to the last month (October 2026 by default): the volume
	1471	+$t$ from January 2018 to the last month (June 2026 by default, the driver's
	1472	+\texttt{--last-month}, which must be the last complete month of the panel): the volume
1310	1473	$n_{k,t}$, the number of month-precision eligible works of topic $k$ with
1311	1474	first month $t$, and the citation inflow $c_{k,t}$, the number of internal
1312	1475	citation edges whose cited work belongs to topic $k$ and whose citing work has
		`@@ -1333,9 +1496,10 @@ log growth, the second its change over a year (acceleration), the third a`
1333	1496	$z$-score of the last quarter's monthly volume against the trailing two years
1334	1497	(the ``abnormal growth'' signal, which also serves as the baseline detector),
1335	1498	the fourth citation acceleration, the fifth a size control, the sixth the
1336		−topic's age in months since it reached five works. Every window ends at $T$;
1337		−the one-month overlap between the $z$-score's trailing window and the
1338		−quarter it is compared with is immaterial. The log-ratio form of growth with
	1499	+topic's age in months since it reached five works. Every window ends at $T$,
	1500	+and the quarter that the $z$-score evaluates ($T-2$ to $T$) does not overlap
	1501	+the trailing window ($T-27$ to $T-3$) against which it is standardised. The
	1502	+log-ratio form of growth with
1339	1503	the $+1$ shift keeps the features finite for topics that were empty in the
1340	1504	earlier window. Of the seven candidate signals of the mandate, the
1341	1505	implemented set covers abnormal growth, citation acceleration and (through the
		`@@ -1376,20 +1540,27 @@ positives separately.`
1376	1540
1377	1541	\subsection{Cutoffs, models and metrics}
1378	1542
1379		−Training cutoffs are every third month from December 2019 to December 2021
1380		−for the 12-month horizon and to December 2020 for the 24-month horizon, so
1381		−that every training label is observable before the first test cutoff;
1382		−test cutoffs are every third month from January 2023 to October 2024 for
1383		−which $T + h$ lies within the panel. With $h = 24$ the last training label
1384		−uses data to December 2023, which is after the reference date but more than a
1385		−year before the first test label window closes, and no test cutoff precedes
1386		−2023; the twelve months of 2022 are used by neither set as a cutoff, which
1387		−is the temporal gap that separates training from test in the sense of
1388		−\citet{bergmeir2012crossval}. The plan proposed training cutoffs through 2022;
1389		−the implementation stops at December 2021 so that the training labels do not
1390		−extend into the test period at $h = 24$, a stricter separation than the plan.
1391		−Rows are the eligible (topic, cutoff) pairs; features are the six quantities
1392		−above with non-finite values replaced by zero.
	1543	+Cutoffs are the months whose index from January 2018 is a multiple of three,
	1544	+that is, January, April, July and October of each year. Training cutoffs are
	1545	+those from December 2019 up to a horizon-dependent last month, December 2021
	1546	+for $h = 12$ and December 2020 for $h = 24$, so that the realised training
	1547	+cutoffs run from January 2020 to October 2021 (eight cutoffs) at $h = 12$ and
	1548	+from January 2020 to October 2020 (four cutoffs) at $h = 24$; in both cases
	1549	+the last training label window closes in October 2022, before the reference
	1550	+date and before the first test cutoff. Test cutoffs are those from January
	1551	+2023 to October 2024 whose label window $T + h$ lies within the panel: with
	1552	+the default last month of June 2026, January 2023 to October 2024 at $h = 12$
	1553	+and January 2023 to April 2024 at $h = 24$. No cutoff of either set falls in
	1554	+2022, and the interval between the close of the last training label (October
	1555	+2022) and the first test cutoff (January 2023) is the temporal separation of
	1556	+training from test in the sense of \citet{bergmeir2012crossval}. The plan
	1557	+proposed training cutoffs through 2022; the implementation stops earlier so
	1558	+that no training label is observed after the reference date, a stricter
	1559	+separation than the plan, bought at the price of few training cutoffs at
	1560	+$h = 24$. The realised cutoff lists are written to the metrics file of each
	1561	+horizon (\texttt{train\_cutoffs}, \texttt{test\_cutoffs}). Rows are the
	1562	+eligible (topic, cutoff) pairs; features are the six quantities above with
	1563	+non-finite values replaced by zero.
1393	1564
1394	1565	Three predictors are compared: the $z$-score baseline, which ranks topics by
1395	1566	$z_{k,T}$ alone and involves no fitting; an $\ell_2$-regularised logistic
		`@@ -1419,20 +1590,38 @@ generation, state-space sequence models, vision--language models, mechanistic`
1419	1590	interpretability, speculative decoding, mixture of experts at scale,
1420	1591	test-time compute and reasoning models, AI for code generation) and predicts
1421	1592	that the detector flags each at least six months before its peak growth
1422		−month. The evaluation requires a mapping from each development to one or
1423		−more OpenAlex topics, and this mapping is the weak point of the design: it
1424		−must be made by hand, after the taxonomy work, and a development that has no
1425		−topic of its own (because the OpenAlex topic model folds it into a larger
1426		−one) cannot be detected by a topic-level detector whatever its merits. The
1427		−script reserves a table for this mapping and it is empty in the current
1428		−version; the lead-time statistic is therefore
1429		−\pending{known-emergence lead times, experiments/known\_set.csv not yet populated},
1430		−and the mapping, when made, will be listed in the appendix with the reason
1431		−for each choice. Lead time itself is defined, for a development mapped to
1432		−topic $k$, as the number of months between the first test cutoff at which
1433		−$k$ ranks in the top 50 and the month of maximum $g^{12}_{k,t}$ over the
1434		−window; a development whose peak growth precedes the first test cutoff has
1435		−undefined lead time and is reported as such rather than dropped.
	1593	+month. Under H8 the lead time of a development mapped to a topic $k$ would be
	1594	+the number of months between the first test cutoff at which $k$ ranks in the
	1595	+top 50 and the month of maximum $g^{12}_{k,t}$ over the window. That
	1596	+definition requires a mapping from each development to one or more OpenAlex
	1597	+topics, and the mapping cannot be made: the twelve developments have no
	1598	+topic of their own in the OpenAlex model (there is no topic for LLM agents,
	1599	+vision--language models, speculative decoding or mixture of experts), so a
	1600	+topic-level detector cannot be scored on them whatever its merits. The check
	1601	+actually run (\texttt{experiments/known\_set.py},
	1602	+\texttt{results/emerging/known\_set\_lead\_times.csv}) is therefore a
	1603	+separate lead-time check on the preregistered concepts, and it departs from
	1604	+the H8 definition in three respects that are stated wherever its results
	1605	+appear. First, the units are keyword series: each development is defined by
	1606	+a hand-written regular expression over title and abstract, listed in the
	1607	+script, and its monthly series counts the month-precision eligible works it
	1608	+matches. Second, the detector applied is the $z$-score baseline rule, not the
	1609	+trained models: a development is \emph{detected} in the first month $t$ at
	1610	+which the mean of months $t-2$ to $t$ exceeds the mean of months $t-26$ to
	1611	+$t-3$ by more than three of their standard deviations, the trailing
	1612	+twelve-month volume is at least thirty works, and that volume is at least
	1613	+twice the volume of the twelve months before. Third, the \emph{peak growth
	1614	+month} is the month of maximum twelve-month log growth over the whole
	1615	+series, chosen with full-sample knowledge, so the lead time (peak month minus
	1616	+detection month) is an ex-post quantity: a detection may precede a peak that
	1617	+no contemporaneous observer could have identified, and a negative lead time
	1618	+means that the series' fastest growth preceded the rule's firing. The
	1619	+regular expressions are themselves 2026 knowledge, since they name each
	1620	+development as it came to be called. The check thus measures whether a simple
	1621	+abnormal-growth rule on a named concept fires early relative to that
	1622	+concept's peak growth; it is not the H8 test of the trained detector, and
	1623	+Part IX reports it under that description
	1624	+(\pending{known-set lead times: no macro yet for results/emerging/known\_set\_lead\_times.csv}).
1436	1625
1437	1626	\subsection{Look-ahead safeguards and what they do not cover}
1438	1627
		`@@ -1565,13 +1754,17 @@ criteria and the sensitivity to truncation. The detector metrics carry`
1565	1754	per-cutoff values, whose dispersion across cutoffs is the relevant measure of
1566	1755	stability, since the rows within a cutoff are not independent. And the
1567	1756	cluster count and the category similarity matrices carry the split-half ARI\@.
1568		−Where the manuscript reports a bootstrap interval for a corpus statistic (the
1569		−Gini coefficient by year, the cross-area shares), it is the percentile
1570		−interval of the non-parametric bootstrap \citep{efron1979bootstrap} over
1571		−works, with 1,000 replications and seed 44, computed by the table-making
1572		−script at reporting time; the protocol's commitment to ``bootstrap or HAC
1573		−intervals'' is met by these two devices, and a reported figure without an
1574		−interval is a population count for which none is appropriate. The
	1757	+The only bootstrap in the pipeline is that of
	1758	+\texttt{bibliometrics/04\_diversity\_cagr.py} (Section~\ref{sec:m-diversity}):
	1759	+percentile intervals \citep{efron1979bootstrap} for the yearly Rao--Stirling
	1760	+index and the yearly category shares, from $B = 500$ multinomial resamples of
	1761	+each year's works over the categories with seed 44. No other corpus
	1762	+statistic---the Gini coefficients, the cross-area shares, the modularity
	1763	+values---carries a resampling interval, and the manuscript reports none for
	1764	+them; the protocol's commitment to ``bootstrap or HAC intervals'' is met by
	1765	+the diversity bootstrap and by the HAC errors of the ITS, and a reported
	1766	+figure without an interval is a population count for which none has been
	1767	+computed. The
1575	1768	indicators of Section~\ref{sec:m-bibliometrics} that depend on OpenAlex's
1576	1769	nightly recomputation (FWCI, total citation counts) are snapshot quantities
1577	1770	and are dated.
		`@@ -1581,7 +1774,7 @@ and are dated.`
1581	1774	Every stochastic step is seeded with the constant 44: the validation and
1582	1775	linkage samples (hash ordering salted with the seed), the atlas sample and the
1583	1776	two UMAP fits, the split-half permutations, the gradient boosting classifier,
1584		−and the bootstrap. The regular-expression classifier, the SQL stages and the
	1777	+and the diversity bootstrap. The regular-expression classifier, the SQL stages and the
1585	1778	structural-change toolkit are deterministic. Two steps are deterministic only
1586	1779	up to the platform: embedding on Apple-silicon GPUs with float16 storage
1587	1780	produces vectors that differ at the fourth decimal from a CPU run, and UMAP's
		`@@ -1619,10 +1812,10 @@ their hashes.`
1619	1812
1620	1813	\subsection{Human checkpoints and the status of conclusions}
1621	1814
1622		−Five checkpoints (H-1 to H-5, \texttt{reports/HUMAN\_REVIEW\_CHECKPOINTS.md})
1623		−require the principal investigator's review of the linkage sample, the
1624		−taxonomy validation sample, the landmark list, the structural-change results
1625		−and the conclusions. Until a checkpoint is signed, the manuscript marks the
	1815	+Six checkpoints (H-0 to H-5, \texttt{reports/HUMAN\_REVIEW\_CHECKPOINTS.md})
	1816	+require the principal investigator's review of the plan, protocol and
	1817	+amendments (H-0), the linkage sample, the taxonomy validation sample, the
	1818	+landmark list, the structural-change results and the conclusions. Until a checkpoint is signed, the manuscript marks the
1626	1819	conclusions that depend on it as pending human review, and the numbers that
1627	1820	depend on it are rendered by the \texttt{\textbackslash pending} macro rather
1628	1821	than typed. This is the mechanism by which the mandate's requirement that no

added reports/critic/resolutions.csv +73 −0

Large diff (74 lines) is not loaded by default to keep the page fast.Load diff

spb/wp44_uqo

WP44: fixes to Part II and appendices A/B/C/E/H after critique (31 issues resolved)

Theme