% ============================================================================
% Part III — Data and Systematic Review Protocol
% Written by Agent L (Part III instance), 2026-11-03. Every corpus number is a macro from tables/numbers.tex
% (generated by tables/make_numbers.py from data/metadata/prisma_counts.json and linkage_report.json).
% Sources of truth: RESEARCH_PLAN.md §2–§3, protocol/PROTOCOL.md, protocol/amendments.csv,
% reports/PHASE0_FEASIBILITY.md, data/raw/README.md, ingestion/*.py, systematic_review/define_corpus.py.
% ============================================================================

\section{Information sources and conditions of access}
\label{sec:data-sources}

A field-level mapping study stands or falls with the bibliographic databases on which it is built. The choice
of sources determines which works can be observed at all, how reliably they can be dated and typed, whether
citation links are available, and under what licence derived data may be redistributed. This part documents
these choices for WP44 and reports, stage by stage, how the mapping corpus was assembled from the raw
sources. The exposition follows the order of the pipeline (identification, linkage, deduplication,
eligibility, scope, dating) and closes with the quality indicators and the research-integrity procedures that
govern every later analysis. The reporting structure is modelled on the PRISMA 2020 items that apply to a
mapping study \citep{page2021prisma,page2021prismaee}; items concerned with risk of bias and effect measures
have no counterpart in a bibliometric design and are replaced, as PRISMA-ScR does for scoping reviews
\citep{tricco2018prismascr}, by indicators of metadata quality and coverage (Section~\ref{sec:data-quality}).
The search-reporting recommendations of PRISMA-S \citep{rethlefsen2021prismas} are followed to the extent
that a database-scope strategy without keyword queries permits.

\subsection{Why a multi-source design}

No single database indexes computer science completely, and the gaps are not random. The venue-centred
coverage of DBLP, the publisher-centred coverage of Crossref, the preprint-centred coverage of arXiv and the
aggregating coverage of OpenAlex each miss a distinct part of the literature and type the part they do cover
in distinct ways. Comparative studies of bibliographic data sources have documented large differences in
coverage between Web of Science, Scopus, Dimensions, Crossref and Microsoft Academic
\citep{visser2021comparison,martinmartin2021comparison}, and the journal-level coverage of the two commercial
indexes is known to under-represent conference proceedings, which carry a large share of computer-science
output \citep{mongeon2016coverage}. For the open successor of Microsoft Academic, OpenAlex, the first
independent assessments report strong coverage of works with DOIs together with weaker and uneven metadata on
affiliations, document types and references \citep{alperin2024openalex,culbert2025openalex,
scheidsteger2023openalex,delgadoquiros2024completeness}. These findings motivate the design adopted here: one
source supplies the backbone (OpenAlex), a second supplies the definition of what counts as computer science
(DBLP), a third supplies the preprint layer and its dates (arXiv), and a fourth checks the coverage of a
subfield with a dense, curated venue structure (the ACL Anthology). Crossref is used for verification only.

\subsection{OpenAlex}

OpenAlex \citep{priem2022openalex} is the backbone of the corpus. It supplies, for every work, the
identifiers (OpenAlex~W identifier, DOI, arXiv identifier where present), the publication date and type, the
title and an inverted-index abstract where available, the venue (source) and its type, the authorships with
institutions and countries, the topic assignment at three levels (field, subfield, topic) with a primary
topic, the list of referenced works (the citation graph), the open-access status and licence, and the
field-weighted citation impact together with yearly citation counts. OpenAlex is released under the CC0
public-domain dedication, which permits redistribution of record-level derived data.

Two access routes exist and both were used, for different purposes. The public snapshot is a versioned bulk
release in gzipped JSON Lines, published in a public S3 bucket; the release used here is dated
2026-10-14 and comprises 1{,}898 part files for the \emph{works} entity, 675.3~GB compressed, with a manifest
listing the record count of each file. The snapshot was streamed from the primary compute node, each file's
content length checked against the manifest and a SHA-256 of the compressed bytes recorded in a SQLite ledger
before parsing; every one of the 1{,}898 files parsed without error and the sum of the per-file record counts
equals the manifest total of \LinkSlimTotal{} works. Two outputs were written during this single pass. The
first is the full JSON record of every work carrying \emph{any} topic in OpenAlex field~17 (Computer Science)
with a publication year between 2018 and 2026, stored as Zstandard-compressed JSON Lines and partitioned by
the snapshot's own date partitions (\LinkOaCsRecords{} records, about 20~GB). The second is a \emph{slim}
table of every work in the snapshot (identifier, DOI, year, type, primary topic, subfield, field and domain,
citation count and a few flags), kept in Parquet so that works cited by computer-science papers but published
in other fields can be resolved for the interdisciplinary analyses of Part~VIII (decision D2 of
\texttt{RESEARCH\_PLAN.md}). Records flagged by OpenAlex as XPAC expansion records are retained at this stage
with their flag so that the exclusion decision can be taken, and counted, later (Section~\ref{sec:data-eligibility}).
The smaller entity snapshots (topics, subfields, fields, domains, sources, institutions, publishers, funders,
concepts) were downloaded the same day with a per-file SHA-256 ledger; author entities were not downloaded
because the authorship data embedded in each work record suffice for the analyses planned.

The second route is the REST API. Without an API key the public endpoint operates under a shared daily budget
of USD~0.10, list-and-filter calls are priced at USD~0.0001 each, and cursor paging is unavailable, so that a
single filtered query cannot return more than 10{,}000 results through basic paging. Single-record lookups by
identifier are free. The API was therefore used for two narrowly defined tasks. First, an incremental harvest
of works with any computer-science topic and a publication date between 2026-10-01 and 2026-11-02, intended
to capture records added after the snapshot was cut; the harvest is sliced by publication day and, where a day
exceeds the paging limit, split into records with a computer-science primary field and records without one.
Raw JSON pages are stored verbatim under a dated directory with a ledger of counts and cost, and the job
resumes on the following day when the budget is exhausted. Second, free single-record lookups by DOI serve as
one of the verification routes for the bibliography of this manuscript (Section~\ref{sec:data-integrity}).
A free API key, which would raise the daily budget, requires the principal investigator to act on an e-mail
and was deliberately not obtained without instruction; the mandate forbids assuming credentials that have not
been tested.

\subsection{DBLP}

The DBLP computer science bibliography \citep{ley2002dblp,ley2009dblp} is a curated, venue-centred index that
has for three decades defined, in practice, the set of journals and conference series regarded as computer
science. It is used here in that capacity: a work indexed by DBLP is in computer science by construction, and
DBLP venue streams feed the taxonomy of Part~IV. The complete XML dump (\texttt{dblp.xml.gz}, dated 2026-11-02
22:39~UTC, 1.12~GB) was downloaded together with its DTD and MD5 checksum, the checksum verified, and the file
parsed with a validating XML parser into one row per publication record (articles, conference papers,
proceedings volumes, books, book chapters, doctoral and master's theses, data records; the \texttt{www}
person records are skipped). The whole archive was kept so that per-year counts are available for the
historical baseline and for context. The DBLP dump is released under the Open Data Commons Attribution licence
(ODC-BY~1.0). DBLP contributes, in addition to the scope definition, author person identifiers, venue streams,
and DOI links for most records; the \texttt{ee} fields were also scanned for arXiv URLs, with a result reported
in Section~\ref{sec:data-linkage}.

\subsection{arXiv}

The arXiv e-print repository \citep{ginsparg2011arxiv} is the main preprint channel of computer science and,
for the period under study, the place where a large share of the most cited work first appeared. Its
metadata are harvested through the OAI-PMH interface (\texttt{set=cs}, \texttt{metadataPrefix=arXiv}) in
monthly datestamp windows from January 2018 to 2 November 2026. The harvester waits at least 3.5~s between
requests and honours \texttt{Retry-After} headers; every raw XML page is kept verbatim as the immutable
snapshot and parsed afterwards into a single Parquet table. Two properties of OAI-PMH matter for what follows.
The \texttt{from}/\texttt{until} parameters filter on the record's \emph{datestamp}, the time of its last
update, so that a monthly window contains every record updated in that month, not every record first
submitted then; duplicates across windows are resolved at parse time with the latest datestamp winning. The
\texttt{created} element gives the first submission date, which is the date used throughout this study for
preprints. The harvest returned \LinkArxivCsRecords{} distinct records, of which
\LinkArxivCreatedTwoZeroOneEightTwoZeroTwoSix{} were first submitted in 2018--2026; the remainder are older
e-prints updated during the window. The arXiv metadata are released for reuse under the repository's metadata
terms; e-prints themselves are governed by per-paper licences and are downloaded, for the high-priority set
only, within those licences and at the repository's rate limits. The use of arXiv as a research dataset, and
the caveats attached to its category labels and versioning, are discussed by \citet{clement2019arxiv}.

\subsection{ACL Anthology}

The ACL Anthology is the curated archive of the computational-linguistics community and has itself served as a
bibliometric corpus \citep{radev2013acl}. Its BibTeX export with abstracts (\texttt{anthology+abstracts.bib.gz},
46.7~MB, SHA-256 recorded) was downloaded on 2026-11-02 and parsed into a table of 138{,}796 entries, of which
\LinkAclRecords{} are conference papers or journal articles and \LinkAclTwoZeroOneEightTwoZeroTwoSix{} fall in
2018--2026. The Anthology plays a diagnostic role: because its venue structure is complete and hand-maintained,
the share of its entries that the backbone can be matched to is a direct measure of OpenAlex coverage for a
subfield that publishes overwhelmingly in conference proceedings. It also supplies abstracts for those entries
in which OpenAlex has none. The export is licensed CC~BY~4.0.

\subsection{Crossref, Semantic Scholar and other sources}

Crossref \citep{hendricks2020crossref} is the registration agency behind most DOIs in the corpus and offers
open metadata through a REST API with a polite pool. It is not used to build the corpus, whose backbone
already carries Crossref-derived metadata through OpenAlex, but for three verification purposes: resolution of
every DOI cited in this manuscript, a cross-check of publisher-assigned document types on samples, and a
cross-check of citation counts (\texttt{is-referenced-by-count}) on a sample of highly cited works in
Part~IX.

The Semantic Scholar Academic Graph \citep{kinney2023s2} was listed in the mandate among the priority sources
because of two signals it alone provides, the influential-citation classification and citation intent. At the
time of the feasibility audit, anonymous requests to its API returned HTTP~429 with an instruction to apply for
a key, and the bulk-dataset endpoint requires a key outright. In keeping with the rule that inaccessible data
be recorded rather than circumvented, Semantic Scholar is recorded as unavailable (decision D3) and no analysis
in this manuscript uses influential-citation or citation-intent signals; where Part~VIII discusses the
strength of citation links, it does so with OpenAlex reference lists and the knowledge-graph extraction of
Part~IV, whose limits are stated there. The Microsoft Academic Graph \citep{wang2020mag}, from which OpenAlex
inherited much of its historical coverage, was discontinued at the end of 2021 and is not an independent source
for the period studied. Kaggle credentials for the arXiv snapshot distributed there were absent and were not
needed, OAI-PMH being sufficient. Benchmark repositories and code-hosting platforms enter only in the
reproducibility and benchmark analyses scoped in Part~IV and are not part of the bibliographic corpus.

Table~\ref{tab:data-sources} summarises the sources, the access route, the licence and the role of each.

\begin{table}[htbp]
\centering\small
\caption{Information sources of the WP44 corpus (all accessed 2026-11-02/03).}
\label{tab:data-sources}
\begin{tabularx}{\textwidth}{lXll}
\toprule
Source & Access and snapshot & Licence & Role \\
\midrule
OpenAlex works snapshot & public S3 bucket, release 2026-10-14, 1{,}898 JSONL files, \LinkSlimTotal{} works & CC0 & backbone: identifiers, dates, types, topics, authorships, references, abstracts, OA status \\
OpenAlex REST API & no key; USD~0.10 per day shared budget; increment 2026-10-01 to 2026-11-02; single lookups & CC0 & post-snapshot increment; reference verification \\
DBLP & XML dump 2026-11-02 22:39~UTC, MD5 verified & ODC-BY~1.0 & computer-science scope; venue streams; author identifiers \\
arXiv & OAI-PMH set \texttt{cs}, 2018-01 to 2026-11, raw pages kept & arXiv metadata terms & preprint layer; first-submission dates; categories \\
ACL Anthology & BibTeX export with abstracts, 2026-11-02 & CC~BY~4.0 & coverage check for NLP venues; abstracts \\
Crossref & REST API, polite pool & open metadata & verification of DOIs, types and citation counts \\
Semantic Scholar & API key required; anonymous calls refused & --- & \emph{unavailable, recorded} \\
\bottomrule
\end{tabularx}
\end{table}

\section{Identification: what each source contributes}
\label{sec:data-identification}

There is no keyword search in this study. A mapping of an entire discipline cannot be defined by a query
string without importing the vocabulary of whoever wrote it; it is defined instead by database scope filters
that are themselves documented and reproducible, which is the strategy recommended for mapping studies in
software engineering and adopted here for computer science as a whole
\citep{petersen2008mapping,petersen2015mapping}. The identification stage (S0 in the notation of the research
plan) therefore consists of four scope filters, one per source, applied to immutable snapshots whose dates and
hashes are recorded in \texttt{data/raw/README.md}.

\subsection{Scope filters}

The OpenAlex filter retains every work with at least one topic, primary or secondary, in field~17 and a
publication year in $[2018, 2026]$. The choice of \emph{any} topic rather than the primary topic is deliberate
and conservative at this stage: a work whose primary topic lies in, say, statistics or electrical engineering
but which carries a secondary computer-science topic is a candidate, and the decision whether it belongs to
the core or to the penumbra is deferred to the eligibility stage, where it is made explicit and counted
(Section~\ref{sec:data-core}). The filter yields
\PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{} records in the first pass over the
snapshot. The DBLP filter retains the publication records of type article, inproceedings, incollection, book,
phdthesis and data with a year in $[2018, 2026]$: \PrismaIdentifiedDblpTwoZeroOneEightTwoZeroTwoSix{} records.
The arXiv filter retains the records of set \texttt{cs} whose first submission falls in the window:
\PrismaIdentifiedArxivCsTwoZeroOneEightTwoZeroTwoSix{} records. The ACL filter retains the conference papers
and journal articles of the Anthology with a year in the window:
\PrismaIdentifiedAclTwoZeroOneEightTwoZeroTwoSix{} records. The four counts overlap heavily, since most DBLP
and ACL records and a majority of arXiv records also exist in OpenAlex; the overlap is resolved by record
linkage, not by summation, and no total of identified records is reported as if the sources were disjoint.

\subsection{The OpenAlex topic classifier as a scope instrument}

Because the OpenAlex filter relies on an automated topic classifier, its properties bear on everything
downstream. OpenAlex assigns to each work up to three topics from a fixed list of 4{,}516, organised in 252
subfields, 26 fields and four domains; the assignment is made by a model applied to the title, abstract and
venue, and the field label of a work is inherited from its topics rather than assigned directly. Two
consequences follow. A work can be labelled computer science because one of its topics belongs to field~17
even when its content is only marginally computational, and conversely a work in a computer-science venue can
receive no computer-science topic when its abstract is missing or its vocabulary is domain-specific. The first
error inflates the candidate set with what will be called the penumbra; the second removes genuine
computer-science works from the first-pass candidate set. DBLP is used to correct the second error, which is
why the linkage stage has two passes (Section~\ref{sec:data-linkage}). The literature on OpenAlex has so far
examined its coverage and metadata completeness rather than the accuracy of its topic labels
\citep{alperin2024openalex,delgadoquiros2024completeness}; the present study treats the labels as a scope
instrument whose errors are bounded by the DBLP and arXiv memberships and quantified in
Section~\ref{sec:data-quality}, not as ground truth.

\subsection{Verification of the scale against the mandate}

The mandate set planning targets of more than one million candidate records and more than one hundred
thousand deeply classified records, subject to availability. The feasibility audit established, before any
ingestion, that the OpenAlex API alone reported more than six million works with a computer-science primary
field in 2018--2026 and that the DBLP dump held more than four million records in the same years; the targets
were therefore attainable without recourse to any restricted source. The audit also established what was
\emph{not} attainable: a Semantic Scholar key, an OpenAlex key, Kaggle credentials and the AWS command-line
tool were all absent, and the ingestion was designed around their absence (HTTPS streaming of the public
bucket, basic paging of the public API). The record of this audit is \texttt{reports/PHASE0\_FEASIBILITY.md}.

\section{Record linkage across sources}
\label{sec:data-linkage}

The four candidate sets describe overlapping populations with different identifiers. Record linkage (stage S1)
establishes, for each DBLP, arXiv and ACL record, the OpenAlex work it describes, so that one node per work
can be formed with the union of the metadata. The methods are deterministic and ordered by reliability; the
probabilistic framework of \citet{fellegi1969linkage} is not needed because the available keys (DOI, arXiv
identifier) are close to exact, and the residual title-based step is restricted to blocks where it is
unambiguous. The implementation follows the standard decomposition of data matching into key normalisation,
blocking, comparison and classification \citep{christen2012matching}; Part~IV gives the formal statement of the
keys and of the precision-estimation design, and this section reports what was done and what it produced.

\subsection{Keys and matching order}

All keys are normalised before comparison. DOIs are lower-cased. arXiv identifiers are lower-cased and
stripped of their version suffix (\texttt{v2}, \texttt{v3}, \ldots), so that all versions of an e-print map to
one identifier. Titles are reduced to a normalised string by lower-casing and deleting every character that is
not a letter or digit; the normalised title is used only when it has at least 25 characters, is unique within
its source, and is unique within OpenAlex, which is the blocking rule that keeps the title step from
producing spurious links between generic titles.

For DBLP, the order is: (i) DOI equality, (ii) arXiv identifier equality, restricted to records not yet linked
on either side, (iii) normalised title equality with a year difference of at most one, restricted to records
not yet linked on either side. The DOI step linked \LinkLinkDblpOaDoi{} records and the title step
\LinkLinkDblpOaTitleYear{}; the arXiv step linked none, because the extraction of arXiv URLs from the DBLP
\texttt{ee} fields returned no candidate (\LinkDblpWithArxiv{} records with an extracted identifier). The most
plausible reason is that DBLP now records the arXiv version of its CoRR entries as a DataCite DOI of the form
\texttt{10.48550/arXiv.*}, which the DOI step already handles; the step is kept in the pipeline because other
dumps may differ. In total \LinkDblpLinkedDistinct{} of the \LinkDblpTwoZeroOneEightTwoZeroTwoSix{} DBLP
records in the window were linked to an OpenAlex work after the first pass, leaving \LinkDblpUnlinked{}
unlinked. The composition of the unlinked set is informative: \LinkDblpUnlinkedByTypeArticle{} journal
articles, \LinkDblpUnlinkedByTypeInproceedings{} conference papers, \LinkDblpUnlinkedByTypePhdthesis{}
doctoral theses, \LinkDblpUnlinkedByTypeData{} data records, \LinkDblpUnlinkedByTypeIncollection{} chapters
and \LinkDblpUnlinkedByTypeBook{} books. Of these, \LinkDblpUnlinkedWithDoiInSlim{} carry a DOI that
\emph{does} exist in the slim table of all OpenAlex works, which means the work is in OpenAlex but was not
caught by the computer-science topic filter; this is the second error of the topic classifier described above
and the reason for the second pass.

For arXiv, the order is: (i) arXiv identifier equality against the identifier OpenAlex records for the work,
(ii) DOI equality for e-prints that carry a journal DOI, (iii) normalised title equality with a year
difference of at most two, the wider tolerance reflecting the typical lag between a preprint and its published
version \citep{lin2020preprints}. The steps linked \LinkLinkArxivOaArxivId{}, \LinkLinkArxivOaDoi{} and
\LinkLinkArxivOaTitleYear{} records respectively, for \LinkArxivLinkedDistinct{} distinct e-prints linked.
For the ACL Anthology, the order is: (i) DOI equality, (ii) equality between the Anthology identifier and the
identifier embedded in an \texttt{aclanthology.org} landing-page URL among the work's OpenAlex locations,
(iii) normalised title equality with a year difference of at most one. The steps linked
\LinkLinkAclOaDoi{} records by DOI and \LinkLinkAclOaTitleYear{} by title, with the URL step contributing the
remainder, for \LinkAclLinkedDistinctTwoZeroOneEightTwoZeroTwoSix{} distinct 2018--2026 entries linked out of
\LinkAclTwoZeroOneEightTwoZeroTwoSix{}; the residual is examined in Section~\ref{sec:data-quality} as a
measure of OpenAlex coverage of conference-heavy subfields.

\subsection{The second snapshot pass}

A DBLP record whose DOI resolves to an OpenAlex work outside the topic filter is, by the scope rule, a
computer-science work that the first pass did not load. The same situation arises for arXiv e-prints whose
journal DOI exists in OpenAlex without a computer-science topic. The linkage script therefore writes the list
of OpenAlex identifiers that are needed but not loaded: \LinkNeededOaIdsSecondPassDblpDoi{} identifiers from
DBLP DOIs and \LinkNeededOaIdsSecondPassArxivDoi{} from arXiv DOIs. A second pass over the works snapshot,
restricted to this identifier list, retrieves the full records of these works; they are normalised into the
same Parquet tables with a \texttt{\_p2} suffix, the DuckDB views are rebuilt over the union of both passes,
and linkage is re-run so that the DBLP-linked set gains the works that the topic classifier had missed. The
second pass is what makes the DBLP membership rule of Section~\ref{sec:data-core} operational: without it,
``DBLP-indexed'' would silently mean ``DBLP-indexed and labelled computer science by OpenAlex'', and the DBLP
correction would be void precisely where it matters. DBLP records with no OpenAlex counterpart after both
passes (\PrismaLinkedDblpOnlyNoOpenalexRecord{} records before the second pass) are kept in a separate table
with their year and type and are reported as such; they enter the volume counts of Part~IX only in the
sensitivity analysis that adds DBLP-only records, because they lack the abstracts, references and affiliations
on which every other indicator depends. Most of them are journal articles and conference papers with a DOI
that OpenAlex does not hold at all, together with doctoral theses, which DBLP indexes from university
repositories that OpenAlex covers unevenly.

\subsection{Precision of the linkage}

Exact-key links are correct whenever the key is correct in both sources; the known failure modes are DOIs
reassigned by publishers between versions, DOIs registered for a proceedings volume rather than a paper, and
e-prints whose OpenAlex record points to a different version. Title-based links can fail on near-identical
titles of distinct works (extended abstracts and full papers, workshop and conference versions). The protocol
pre-specifies the estimation of linkage precision on a sample of 500 DBLP--OpenAlex pairs stratified by method,
with automatic agreement checks (normalised-title similarity, year difference, first-author surname) and a
200-pair subsample reserved for the principal investigator (checkpoint H-1). The sampling script draws 250 DOI
links, 200 title links and 50 arXiv links with a fixed seed; since the arXiv step produced no links in this
dump, the realised sample has 450 pairs, and the precision estimate for the arXiv step is undefined rather
than reported as perfect. At the time of writing the sampling script exists but the sample has not been drawn and the
human checkpoint is unsigned; the automatic agreement rates and the human verdicts will be reported in Part~IX
and Appendix~B, and every conclusion that depends on the title-based links is labelled as pending human review
until H-1 is signed.