WP44: Appendices A (taxonomy), E (data dictionary), H (reproducibility) by Agent L
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
11 changed files +2,944 −6
added
agents/briefs/WRITER_APPENDICES_BC.md
+22 −0
@@ -0,0 +1,22 @@ | |||||
| 1 | +# Brief — Appendix B (search strategies and sources) and Appendix C (mathematical derivations) — files manuscript/appendices/appB_search_and_sources.tex, appC_derivations.tex | ||||
| 2 | + | ||||
| 3 | +Follow agents/briefs/WRITER_COMMON.md. Files start with \section{...} (inside \begin{appendices}). | ||||
| 4 | +- Appendix B (≈ 8–12 pages): the exact acquisition procedures as executed — read ingestion/*.py, data/raw/README.md, | ||||
| 5 | + reports/PHASE0_FEASIBILITY.md, RESEARCH_PLAN.md §2–3, manuscript/sections/part03_data.tex (do not repeat its prose; this | ||||
| 6 | + appendix is the operational record). For each source: endpoint/bucket, release/version, filters (OpenAlex field 17 rule, | ||||
| 7 | + year range, id-set second pass, API increment slicing by day × primary-CS/non-CS, arXiv OAI set and windows, ACL export, DBLP | ||||
| 8 | + dump + DTD), rate limits and budgets, licence, integrity checks (manifest totals, SHA-256 per file, MD5), what was NOT | ||||
| 9 | + obtained (Semantic Scholar, OpenAlex key, full texts), and the equivalent of a PRISMA-S search record. Include a table of | ||||
| 10 | + exclusion codes E1–E7 with their SQL predicates paraphrased, the keep/hard-exclude type lists and the generic-deposit list | ||||
| 11 | + (from systematic_review/define_corpus.py), and the linkage cascade with its thresholds. | ||||
| 12 | +- Appendix C (≈ 10–14 pages): derivations and proofs referenced by Part IV (manuscript/sections/part04_methodology.tex — read it | ||||
| 13 | + first and match its notation): (i) the ITS model as a regression with the HAC covariance estimator and the placebo p-value | ||||
| 14 | + as a rank statistic; (ii) Bai–Perron dynamic programme: SSR additivity, the recursion, complexity O(n² m), BIC and LWZ | ||||
| 15 | + criteria as implemented in econometrics/structural_change.py; (iii) PELT cost and pruning condition, penalty 2σ² log n; | ||||
| 16 | + (iv) BOCPD: Normal–Inverse-Gamma conjugate updates, Student-t predictive, run-length recursion, the fact that with a | ||||
| 17 | + constant hazard the mass on r=0 equals H (hence the MAP-run-length detection statistic used in the code); (v) Gini as a | ||||
| 18 | + rank formula and its equivalence with the Lorenz definition; (vi) Rao–Stirling and Leinster–Cobbold (q=2) identity; | ||||
| 19 | + (vii) Newman–Girvan modularity for directed weighted graphs; (viii) Krippendorff's α with MASI distance; (ix) the | ||||
| 20 | + multi-label score rule and the tie-break as a total order; (x) the emerging-topic label and why the feature construction | ||||
| 21 | + is look-ahead-free (formal statement in terms of filtrations). Derivations must be correct and self-contained; cite the | ||||
| 22 | + original sources (verify any new citation; most keys exist in manuscript/references.bib and references_part04.bib — reuse them). | ||||
modified
manuscript/appendices/appA_taxonomy.tex
+684 −1
@@ -1 +1,684 @@ | |||||
| 1 | −% appA_taxonomy — to be written (Phase 7) | 1 | +% ============================================================================ | ||
| 2 | +% Appendix A — The ontology of computer-science research (taxonomy) | ||||
| 3 | +% Written by Agent L (appendices A/E/H instance), 2026-11-03. | ||||
| 4 | +% Sources of truth: taxonomy/ONTOLOGY.md (hand-written §1–§3 and generated tables), taxonomy/ontology.yaml (v1.0, | ||||
| 5 | +% 2026-11-03), taxonomy/topic_map.csv, taxonomy/classify.py, taxonomy/VALIDATION.md. Counts in this appendix are | ||||
| 6 | +% properties of the ontology (numbers of signals, mapped topics and venues), computed from ontology.yaml; they are | ||||
| 7 | +% not corpus results. Validation results are reported in Part IX. | ||||
| 8 | +% ============================================================================ | ||||
| 9 | + | ||||
| 10 | +\section{The Ontology of Computer-Science Research} | ||||
| 11 | +\label{app:taxonomy} | ||||
| 12 | + | ||||
| 13 | +This appendix documents the classification scheme that assigns every eligible work of the corpus to one or | ||||
| 14 | +more of the forty research categories listed in the mandate (Section~\ref{sec:m-classification} describes the | ||||
| 15 | +scoring rule and its validation design). The scheme is an \emph{extensible hierarchical ontology} rather than a | ||||
| 16 | +flat keyword list: forty leaf categories are grouped into eight areas, a small set of parent--child edges records | ||||
| 17 | +where a leaf specialises another leaf, and a denser set of ``related'' edges records thematic adjacency. The | ||||
| 18 | +mapping from bibliographic signals to leaves is data, not prose. It lives in one Python module per area | ||||
| 19 | +(\texttt{taxonomy/src/*.py}), is validated and serialised by \texttt{taxonomy/build\_ontology.py} into the | ||||
| 20 | +machine-readable file \texttt{taxonomy/ontology.yaml} (version~1.0, built on 2026-11-03 from the OpenAlex topic | ||||
| 21 | +export of 2026-10-14 and the DBLP stream table of 2018--2026) and into an audit table of OpenAlex topics | ||||
| 22 | +(\texttt{taxonomy/topic\_map.csv}). Everything stated below is derived from those files; the tables were | ||||
| 23 | +regenerated from the YAML file rather than transcribed, and the counts they contain are properties of the | ||||
| 24 | +ontology itself (how many topics, venues and patterns point to a category), not statistics of the corpus. The | ||||
| 25 | +agreement of the rules with independent raters on the stratified validation sample is reported in Part~IX and is | ||||
| 26 | +deliberately not anticipated here. | ||||
| 27 | + | ||||
| 28 | +\subsection{Structure: areas, leaves and edges} | ||||
| 29 | +\label{app:taxonomy-structure} | ||||
| 30 | + | ||||
| 31 | +The ontology has three levels. The eight areas of Table~\ref{tab:appA-areas} are an organisational device for the | ||||
| 32 | +manuscript (they correspond to the disciplinary reviews of Parts~V--VII and to the area-level flow statistics of | ||||
| 33 | +Part~VIII); classification operates on leaves only, and every leaf belongs to exactly one area. The forty leaves | ||||
| 34 | +are numbered as in the mandate (Table~\ref{tab:appA-leaves}). Seven \texttt{parent\_\allowbreak of}/\texttt{child\_\allowbreak of} | ||||
| 35 | +edges record genuine specialisation: | ||||
| 36 | +\texttt{foundation\_\allowbreak models} $\rightarrow$ \texttt{llm} and \texttt{multimodal\_\allowbreak ai}; | ||||
| 37 | +\texttt{programming\_\allowbreak languages} $\rightarrow$ \texttt{compilers}; | ||||
| 38 | +\texttt{distributed\_\allowbreak computing} $\rightarrow$ \texttt{cloud\_\allowbreak computing} and \texttt{edge\_\allowbreak computing}; | ||||
| 39 | +\texttt{computer\_\allowbreak architecture} $\rightarrow$ \texttt{ai\_\allowbreak accelerators}; | ||||
| 40 | +\texttt{privacy\_\allowbreak preserving\_\allowbreak computation} $\rightarrow$ \texttt{federated\_\allowbreak learning}. | ||||
| 41 | +These edges have one operational consequence, the tie-breaking rule for the primary category (a child wins | ||||
| 42 | +over its parent at equal score), and one interpretive consequence: a work carrying both labels is not | ||||
| 43 | +double-counted as evidence of convergence between distinct fields, since the two labels are nested by | ||||
| 44 | +construction. The \texttt{related} edges (236 directed entries in the YAML file, 144 distinct unordered pairs) | ||||
| 45 | +carry no weight in classification; they document which categories the ontology's author expected to share | ||||
| 46 | +works, and they are used in Part~IX only to separate expected from unexpected co-classification. | ||||
| 47 | + | ||||
| 48 | +\begin{table}[htbp] | ||||
| 49 | +\begin{singlespace} | ||||
| 50 | +\centering | ||||
| 51 | +\caption{The eight areas and their leaves (\texttt{ontology.yaml}, \texttt{areas}).} | ||||
| 52 | +\label{tab:appA-areas} | ||||
| 53 | +\small | ||||
| 54 | +\begin{tabularx}{\textwidth}{p{3.3cm} p{3.9cm} c X} | ||||
| 55 | +\toprule | ||||
| 56 | +Area id & Name & Leaves & Leaf identifiers \\ | ||||
| 57 | +\midrule | ||||
| 58 | +\texttt{ai\_\allowbreak ml} & Artificial Intelligence and Machine Learning & 9 & | ||||
| 59 | + \texttt{llm}, \texttt{foundation\_\allowbreak models}, \texttt{multimodal\_\allowbreak ai}, \texttt{generative\_\allowbreak ai}, \texttt{ai\_\allowbreak agents}, | ||||
| 60 | + \texttt{reinforcement\_\allowbreak learning}, \texttt{ml\_\allowbreak theory}, \texttt{deep\_\allowbreak learning\_\allowbreak architectures}, \texttt{ai\_\allowbreak safety} \\ | ||||
| 61 | +\texttt{perception\_\allowbreak interaction} & Perception, Language, Robotics and Interaction & 6 & | ||||
| 62 | + \texttt{computer\_\allowbreak vision}, \texttt{nlp}, \texttt{speech\_\allowbreak audio}, \texttt{robotics}, \texttt{hci}, \texttt{computer\_\allowbreak graphics} \\ | ||||
| 63 | +\texttt{software\_\allowbreak pl} & Software, Programming Languages and Formal Methods & 4 & | ||||
| 64 | + \texttt{software\_\allowbreak engineering}, \texttt{programming\_\allowbreak languages}, \texttt{compilers}, \texttt{formal\_\allowbreak verification} \\ | ||||
| 65 | +\texttt{theory} & Theory of Computation & 2 & \texttt{algorithms}, \texttt{computational\_\allowbreak complexity} \\ | ||||
| 66 | +\texttt{data\_\allowbreak distributed} & Data, Information and Distributed Systems & 6 & | ||||
| 67 | + \texttt{database\_\allowbreak systems}, \texttt{information\_\allowbreak retrieval}, \texttt{distributed\_\allowbreak computing}, \texttt{cloud\_\allowbreak computing}, | ||||
| 68 | + \texttt{edge\_\allowbreak computing}, \texttt{computer\_\allowbreak networks} \\ | ||||
| 69 | +\texttt{security\_\allowbreak privacy} & Security, Privacy and Trust & 5 & | ||||
| 70 | + \texttt{cybersecurity}, \texttt{applied\_\allowbreak cryptography}, \texttt{blockchain}, \texttt{privacy\_\allowbreak preserving\_\allowbreak computation}, | ||||
| 71 | + \texttt{federated\_\allowbreak learning} \\ | ||||
| 72 | +\texttt{hardware\_\allowbreak emerging} & Hardware, Architecture and Emerging Computing & 5 & | ||||
| 73 | + \texttt{computer\_\allowbreak architecture}, \texttt{ai\_\allowbreak accelerators}, \texttt{operating\_\allowbreak systems}, \texttt{quantum\_\allowbreak computing}, | ||||
| 74 | + \texttt{sustainable\_\allowbreak computing} \\ | ||||
| 75 | +\texttt{computational\_\allowbreak science} & Computational Science and Discovery & 3 & | ||||
| 76 | + \texttt{scientific\_\allowbreak computing}, \texttt{computational\_\allowbreak biology}, \texttt{autonomous\_\allowbreak scientific\_\allowbreak discovery} \\ | ||||
| 77 | +\bottomrule | ||||
| 78 | +\end{tabularx} | ||||
| 79 | +\end{singlespace} | ||||
| 80 | +\end{table} | ||||
| 81 | + | ||||
| 82 | +Two features of the grouping deserve comment because they are choices, not facts. First, \texttt{ai\_\allowbreak safety} | ||||
| 83 | +is placed under the AI area rather than under security and privacy, although adversarial robustness and | ||||
| 84 | +backdoors are scored in both \texttt{ai\_\allowbreak safety} and \texttt{cybersecurity}; the ontology treats safety as a | ||||
| 85 | +property of learning systems, with security as the adjacent field. Second, \texttt{computer\_\allowbreak graphics} sits in | ||||
| 86 | +the perception area next to vision and HCI rather than in a visual-computing area of its own, and | ||||
| 87 | +\texttt{federated\_\allowbreak learning} sits under privacy rather than under distributed computing, which reflects the | ||||
| 88 | +definitional emphasis of the leaf (training without centralising raw data) rather than its algorithmic | ||||
| 89 | +kinship. Readers who would group the leaves differently can do so from the leaf-level tables of Part~IX | ||||
| 90 | +without re-running anything, because areas are never used in scoring. | ||||
| 91 | + | ||||
| 92 | +\subsection{The forty categories} | ||||
| 93 | +\label{app:taxonomy-leaves} | ||||
| 94 | + | ||||
| 95 | +Table~\ref{tab:appA-leaves} reproduces the definition of each leaf exactly as it is given to the classifier's | ||||
| 96 | +author and to the validation raters (the raters receive these definitions and nothing else about the rules). | ||||
| 97 | +Each definition states the object of the category, lists its typical sub-topics and names the neighbouring | ||||
| 98 | +category to which excluded work belongs; the latter clause is what makes the definitions a partition rather than | ||||
| 99 | +a list of keywords, and it is the part most often invoked in the ambiguity decisions of | ||||
| 100 | +Section~\ref{app:taxonomy-ambiguities}. Identifiers are those of the YAML file; numbers are those of the | ||||
| 101 | +mandate. | ||||
| 102 | + | ||||
| 103 | +\begin{singlespace} | ||||
| 104 | +\begin{small} | ||||
| 105 | +\begin{longtable}{@{}r p{3.4cm} p{10.6cm}@{}} | ||||
| 106 | +\caption{The forty leaf categories: identifier, area and definition (\texttt{ontology.yaml}, \texttt{categories[].definition}). Parent/child edges are given under the identifier.} | ||||
| 107 | +\label{tab:appA-leaves}\\ | ||||
| 108 | +\toprule | ||||
| 109 | +No. & Category & Definition \\ | ||||
| 110 | +\midrule | ||||
| 111 | +\endfirsthead | ||||
| 112 | +\caption[]{The forty leaf categories (continued).}\\ | ||||
| 113 | +\toprule | ||||
| 114 | +No. & Category & Definition \\ | ||||
| 115 | +\midrule | ||||
| 116 | +\endhead | ||||
| 117 | +\midrule | ||||
| 118 | +\multicolumn{3}{r}{\footnotesize continued on next page}\\ | ||||
| 119 | +\endfoot | ||||
| 120 | +\bottomrule | ||||
| 121 | +\endlastfoot | ||||
| 122 | +1 & \textbf{Large Language Models}\newline {\footnotesize \texttt{llm}\newline area \texttt{ai\_\allowbreak ml}; child of \texttt{foundation\_\allowbreak models}} & Research whose object is a transformer-based language model trained at scale: pre-training, instruction tuning, prompting and in-context learning, reasoning, retrieval augmentation, efficient inference, evaluation and benchmarks of LLMs. Excludes classical NLP tasks solved with small task-specific models (nlp) and non-language foundation models (foundation\_models). \\[2pt] | ||||
| 123 | +2 & \textbf{Foundation Models}\newline {\footnotesize \texttt{foundation\_\allowbreak models}\newline area \texttt{ai\_\allowbreak ml}; parent of \texttt{llm}, \texttt{multimodal\_\allowbreak ai}} & Large pre-trained models intended for adaptation to many downstream tasks in any modality: scaling laws, pre-training objectives, self-supervised representation learning at scale, transfer, fine-tuning and adapter methods, model releases and their evaluation. Language-specific work is llm (a child); vision-language work is multimodal\_ai (a child). \\[2pt] | ||||
| 124 | +3 & \textbf{Multimodal AI}\newline {\footnotesize \texttt{multimodal\_\allowbreak ai}\newline area \texttt{ai\_\allowbreak ml}; child of \texttt{foundation\_\allowbreak models}} & Models and systems that jointly process two or more modalities (vision-language, audio-visual, video-text, 3D-language, sensors with language): vision-language models and multimodal LLMs, captioning, visual question answering, cross-modal retrieval and grounding. Unimodal vision is computer\_vision; pure speech is speech\_audio. \\[2pt] | ||||
| 125 | +4 & \textbf{Generative AI}\newline {\footnotesize \texttt{generative\_\allowbreak ai}\newline area \texttt{ai\_\allowbreak ml}} & Models that synthesise new content (images, video, audio, 3D, text, code, molecules) and their training, sampling, evaluation, detection and misuse: GANs, VAEs, diffusion and flow models, autoregressive generators, text-to-X synthesis, deepfakes and AI-generated-content detection. LLM text generation as such is llm unless the contribution concerns the generative model or its outputs' authenticity. \\[2pt] | ||||
| 126 | +5 & \textbf{AI Agents and Autonomous Systems}\newline {\footnotesize \texttt{ai\_\allowbreak agents}\newline area \texttt{ai\_\allowbreak ml}} & Software agents that perceive, decide and act with some autonomy: LLM-based agents and tool use, multi-agent systems, planning and automated reasoning for acting, agent architectures, negotiation, agentic workflows and benchmarks, embodied agents in simulated environments. Physical robots are robotics; RL algorithms as such are reinforcement\_learning. \\[2pt] | ||||
| 127 | +6 & \textbf{Reinforcement Learning}\newline {\footnotesize \texttt{reinforcement\_\allowbreak learning}\newline area \texttt{ai\_\allowbreak ml}} & Learning to act from reward signals: value- and policy-based methods, model-based RL, offline and inverse RL, multi-agent RL, bandits, exploration, reward design, RL from human feedback, and RL theory. Applications to robot control are shared with robotics. \\[2pt] | ||||
| 128 | +7 & \textbf{Machine Learning Theory}\newline {\footnotesize \texttt{ml\_\allowbreak theory}\newline area \texttt{ai\_\allowbreak ml}} & Statistical and computational learning theory and principled ML methodology: generalisation and sample complexity, optimisation theory for learning, kernel and Bayesian methods, probabilistic models, causal inference, online learning, PAC learning, interpretability and fairness as formal problems. Architecture engineering is deep\_learning\_architectures. \\[2pt] | ||||
| 129 | +8 & \textbf{Deep Learning Architectures}\newline {\footnotesize \texttt{deep\_\allowbreak learning\_\allowbreak architectures}\newline area \texttt{ai\_\allowbreak ml}} & Design, training and compression of neural network architectures as such: CNNs, RNNs, transformers and attention, graph neural networks, state-space models, normalisation and optimisers in practice, neural architecture search, pruning, quantisation and distillation, neuromorphic and spiking networks as models. Task-specific results go to the application category; hardware goes to ai\_accelerators. \\[2pt] | ||||
| 130 | +9 & \textbf{Computer Vision}\newline {\footnotesize \texttt{computer\_\allowbreak vision}\newline area \texttt{perception\_\allowbreak interaction}} & Extracting information from images, video and 3D data: recognition, detection, segmentation, tracking, pose, depth and 3D reconstruction, image restoration and enhancement, medical and remote-sensing image analysis, visual representation learning. Image synthesis is generative\_ai; vision-language is multimodal\_ai; rendering is computer\_graphics. \\[2pt] | ||||
| 131 | +10 & \textbf{Natural Language Processing}\newline {\footnotesize \texttt{nlp}\newline area \texttt{perception\_\allowbreak interaction}} & Computational processing of human language: machine translation, parsing and tagging, information extraction, sentiment and text classification, summarisation, question answering, dialogue, text mining, computational linguistics, low-resource and multilingual NLP, NLP evaluation and resources. LLM-centred work is llm (also classified here when the task framing dominates). \\[2pt] | ||||
| 132 | +11 & \textbf{Speech and Audio Computing}\newline {\footnotesize \texttt{speech\_\allowbreak audio}\newline area \texttt{perception\_\allowbreak interaction}} & Processing and generation of speech, music and general audio: speech recognition and synthesis, speaker and language recognition, enhancement and separation, spoken language understanding, music information retrieval and generation, audio event detection, neural audio codecs. Text-only dialogue is nlp. \\[2pt] | ||||
| 133 | +12 & \textbf{Robotics}\newline {\footnotesize \texttt{robotics}\newline area \texttt{perception\_\allowbreak interaction}} & Machines that sense and act in the physical world: manipulation, locomotion, motion and path planning, SLAM and localisation, robot learning, human-robot interaction, autonomous vehicles and drones as robotic systems, soft and swarm robotics, teleoperation. RL algorithms are reinforcement\_learning; robotic process automation and surgical-outcome studies are excluded. \\[2pt] | ||||
| 134 | +13 & \textbf{Human-Computer Interaction}\newline {\footnotesize \texttt{hci}\newline area \texttt{perception\_\allowbreak interaction}} & Design, evaluation and understanding of interactive systems and their users: user studies and usability, interaction techniques, VR/AR/XR, accessibility and assistive technology, CSCW and social computing, human-AI interaction, ubiquitous and wearable computing, affective computing. Rendering is computer\_graphics; physical robots are robotics. \\[2pt] | ||||
| 135 | +14 & \textbf{Software Engineering}\newline {\footnotesize \texttt{software\_\allowbreak engineering}\newline area \texttt{software\_\allowbreak pl}} & How software is specified, built, tested, maintained and studied: requirements, architecture, testing and debugging, program repair, mining software repositories, empirical studies of developers, DevOps, model-driven engineering, open-source ecosystems, and AI-assisted software development. Language design is programming\_languages; proofs of correctness are formal\_verification. \\[2pt] | ||||
| 136 | +15 & \textbf{Programming Languages}\newline {\footnotesize \texttt{programming\_\allowbreak languages}\newline area \texttt{software\_\allowbreak pl}; parent of \texttt{compilers}} & Design, semantics, type systems and implementation principles of programming languages: type theory, operational and denotational semantics, functional and logic programming, effects, concurrency models, memory safety, domain-specific languages, program analysis as a language problem. Code generation and optimisation pipelines are compilers (a child); machine-checked correctness is formal\_verification. \\[2pt] | ||||
| 137 | +16 & \textbf{Compilers}\newline {\footnotesize \texttt{compilers}\newline area \texttt{software\_\allowbreak pl}; child of \texttt{programming\_\allowbreak languages}} & Translation and optimisation of programs for execution: compiler construction, intermediate representations, optimisation passes, code generation, JIT compilation, auto-vectorisation and parallelisation, polyhedral methods, high-level synthesis, and compilers for machine-learning workloads and accelerators. A child of programming\_languages. \\[2pt] | ||||
| 138 | +17 & \textbf{Formal Verification}\newline {\footnotesize \texttt{formal\_\allowbreak verification}\newline area \texttt{software\_\allowbreak pl}} & Mathematically rigorous establishment of system properties: model checking, theorem proving and proof assistants, SAT/SMT solving and automated reasoning, deductive and runtime verification, verified software and hardware, formal semantics as a verification tool, verification of neural networks, smart contracts and protocols, automata and temporal logics. Language semantics as design is programming\_languages. \\[2pt] | ||||
| 139 | +18 & \textbf{Algorithms and Data Structures}\newline {\footnotesize \texttt{algorithms}\newline area \texttt{theory}} & Design and analysis of algorithms and data structures: graph, string, geometric, approximation, randomized, online, streaming, parallel and parameterized algorithms, combinatorial optimisation and metaheuristics, algorithmic game theory, algorithm engineering, and the combinatorics that underpins them. Lower bounds and hardness as such are computational\_complexity; numerical algorithms are scientific\_computing. \\[2pt] | ||||
| 140 | +19 & \textbf{Computational Complexity}\newline {\footnotesize \texttt{computational\_\allowbreak complexity}\newline area \texttt{theory}} & The intrinsic difficulty of computational problems: complexity classes and their separations, hardness of approximation, circuit, proof, communication and query complexity, fine-grained and parameterized complexity, pseudorandomness and derandomization, computability and automata theory, algebraic complexity, and quantum complexity. Algorithm design is algorithms; cryptographic hardness assumptions are applied\_cryptography. \\[2pt] | ||||
| 141 | +20 & \textbf{Database Systems}\newline {\footnotesize \texttt{database\_\allowbreak systems}\newline area \texttt{data\_\allowbreak distributed}} & Storage, querying and management of structured and semi-structured data: query processing and optimisation, transactions and concurrency control, indexing and storage engines, data integration, cleaning and provenance, big-data processing frameworks, data mining as a data-management problem, learned database components, vector and graph databases. Ranking of unstructured documents is information\_retrieval. \\[2pt] | ||||
| 142 | +21 & \textbf{Information Retrieval}\newline {\footnotesize \texttt{information\_\allowbreak retrieval}\newline area \texttt{data\_\allowbreak distributed}} & Finding and ranking relevant information for a user need: search engines and ranking models, dense and sparse retrieval, recommender systems, query understanding, evaluation and test collections, web search and mining, scholarly and digital-library retrieval, retrieval components of RAG systems. Structured query processing is database\_systems; text understanding tasks are nlp. \\[2pt] | ||||
| 143 | +22 & \textbf{Distributed Computing}\newline {\footnotesize \texttt{distributed\_\allowbreak computing}\newline area \texttt{data\_\allowbreak distributed}; parent of \texttt{cloud\_\allowbreak computing}, \texttt{edge\_\allowbreak computing}} & Computation across multiple processors or machines: distributed algorithms and consensus, fault tolerance and replication, parallel programming and HPC systems software, cluster and grid resource management, distributed storage and middleware, distributed and parallel ML training systems. Cloud service models are cloud\_computing and edge placement is edge\_computing (both children). \\[2pt] | ||||
| 144 | +23 & \textbf{Cloud Computing}\newline {\footnotesize \texttt{cloud\_\allowbreak computing}\newline area \texttt{data\_\allowbreak distributed}; child of \texttt{distributed\_\allowbreak computing}} & Computing delivered as elastic, multi-tenant, on-demand services: IaaS/PaaS/SaaS and serverless, virtualisation and containers, resource provisioning and auto-scaling, SLAs and pricing, cloud-native architectures, data-centre resource management, cloud security and migration. A child of distributed\_computing; placement at the network edge is edge\_computing. \\[2pt] | ||||
| 145 | +24 & \textbf{Edge Computing}\newline {\footnotesize \texttt{edge\_\allowbreak computing}\newline area \texttt{data\_\allowbreak distributed}; child of \texttt{distributed\_\allowbreak computing}} & Computation placed near data sources and users: edge/fog/MEC architectures, computation offloading, on-device and TinyML inference, edge-cloud collaboration and split computing, IoT platforms as computing substrates, latency-critical edge services, edge caching and orchestration. Radio access and protocols are computer\_networks; model compression as such is deep\_learning\_architectures or ai\_accelerators. \\[2pt] | ||||
| 146 | +25 & \textbf{Computer Networks}\newline {\footnotesize \texttt{computer\_\allowbreak networks}\newline area \texttt{data\_\allowbreak distributed}} & Design, protocols, measurement and performance of communication networks: Internet architecture and transport, routing, wireless and mobile networks, 5G/6G and software-defined networking, network management, sensor and vehicular networks, content delivery and caching, network coding and information theory applied to networking. Physical-layer signal processing is included only when framed as networking. \\[2pt] | ||||
| 147 | +26 & \textbf{Cybersecurity}\newline {\footnotesize \texttt{cybersecurity}\newline area \texttt{security\_\allowbreak privacy}} & Protecting computing systems, networks, software and users from attack: vulnerability discovery and exploitation, malware and intrusion detection, authentication and access control, system and hardware security, web and mobile security, digital forensics, security of AI systems, security operations and human factors of security. Cryptographic primitives and protocols are applied\_cryptography; data-protection techniques are privacy\_preserving\_computation. \\[2pt] | ||||
| 148 | +27 & \textbf{Applied Cryptography}\newline {\footnotesize \texttt{applied\_\allowbreak cryptography}\newline area \texttt{security\_\allowbreak privacy}} & Design, analysis, implementation and deployment of cryptographic primitives and protocols: symmetric and public-key schemes, hash functions, signatures, key exchange, post-quantum cryptography, zero-knowledge proofs, secure multi-party computation and homomorphic encryption as primitives, cryptanalysis, side-channel-resistant implementations, cryptographic hardware, provable security, quantum key distribution. Deployment of these primitives for data protection is privacy\_preserving\_computation; ledgers are blockchain. \\[2pt] | ||||
| 149 | +28 & \textbf{Blockchain Systems}\newline {\footnotesize \texttt{blockchain}\newline area \texttt{security\_\allowbreak privacy}} & Distributed ledgers and the systems built on them: consensus for permissioned and permissionless chains, smart contracts and their security, scalability (sharding, layer-2, rollups), cryptocurrencies and DeFi mechanisms, blockchain applications and governance, measurement of blockchain ecosystems. Generic BFT consensus theory is shared with distributed\_computing; primitives are applied\_cryptography. \\[2pt] | ||||
| 150 | +29 & \textbf{Computer Architecture}\newline {\footnotesize \texttt{computer\_\allowbreak architecture}\newline area \texttt{hardware\_\allowbreak emerging}; parent of \texttt{ai\_\allowbreak accelerators}} & Organisation and design of processors, memory systems and digital hardware: microarchitecture, caches and memory hierarchies, interconnects and networks-on-chip, FPGAs and reconfigurable computing, embedded and real-time systems, hardware/software co-design, electronic design automation, VLSI and low-power design, processing-in-memory. Accelerators specialised for ML are ai\_accelerators (a child); kernels are operating\_systems. \\[2pt] | ||||
| 151 | +30 & \textbf{Accelerators and AI Hardware}\newline {\footnotesize \texttt{ai\_\allowbreak accelerators}\newline area \texttt{hardware\_\allowbreak emerging}; child of \texttt{computer\_\allowbreak architecture}} & Hardware and hardware-aware software for efficient machine-learning computation: GPU/TPU/NPU design and programming, ASIC and FPGA DNN accelerators, neuromorphic and spiking hardware, in-memory and analog/photonic computing for AI, quantisation and sparsity exploited by hardware, efficient LLM inference and serving systems, ML compilers' hardware mapping, benchmarks such as MLPerf. A child of computer\_architecture. \\[2pt] | ||||
| 152 | +31 & \textbf{Operating Systems}\newline {\footnotesize \texttt{operating\_\allowbreak systems}\newline area \texttt{hardware\_\allowbreak emerging}} & System software that manages hardware and provides execution abstractions: kernels and drivers, scheduling, memory management, file and storage systems, virtualisation and containers, OS security and isolation, I/O and networking stacks, real-time and embedded OS, systems support for new hardware (persistent memory, CXL, SmartNICs). Distributed coordination is distributed\_computing; hardware design is computer\_architecture. \\[2pt] | ||||
| 153 | +32 & \textbf{Quantum Computing}\newline {\footnotesize \texttt{quantum\_\allowbreak computing}\newline area \texttt{hardware\_\allowbreak emerging}} & Computation and information processing exploiting quantum mechanics: quantum algorithms and complexity, quantum error correction and fault tolerance, qubit hardware platforms and control, quantum software, compilers and simulators, variational and NISQ algorithms, quantum machine learning, quantum communication and key distribution, quantum networking. Quantum physics without a computational or information-processing framing is out of scope; post-quantum cryptography is applied\_cryptography. \\[2pt] | ||||
| 154 | +33 & \textbf{Scientific Computing}\newline {\footnotesize \texttt{scientific\_\allowbreak computing}\newline area \texttt{computational\_\allowbreak science}} & Numerical and computational methods and software for mathematical models of science and engineering: numerical linear algebra, discretisation of differential equations, solvers and preconditioners, simulation frameworks, uncertainty quantification, reduced-order and surrogate models, scientific machine learning (PINNs, neural operators), high-performance scientific software and performance portability, mathematical software, computational physics/chemistry/engineering tooling. Domain biology is computational\_biology; parallel systems software is distributed\_computing. \\[2pt] | ||||
| 155 | +34 & \textbf{Computational Biology}\newline {\footnotesize \texttt{computational\_\allowbreak biology}\newline area \texttt{computational\_\allowbreak science}} & Computational methods for biological and biomedical data: bioinformatics and genomics, protein structure and design, molecular modelling and computational drug discovery, single-cell and multi-omics analysis, systems and synthetic biology, biomedical text mining, computational neuroscience, biomedical and health informatics methods, DNA/molecular computing. Medical image analysis is shared with computer\_vision; numerical methods as such are scientific\_computing. \\[2pt] | ||||
| 156 | +35 & \textbf{Computer Graphics}\newline {\footnotesize \texttt{computer\_\allowbreak graphics}\newline area \texttt{perception\_\allowbreak interaction}} & Synthesis and manipulation of visual content from models: rendering, geometry processing and modelling, animation and simulation for graphics, neural rendering and 3D representations, visualisation, procedural content, game and VR engines. Image understanding is computer\_vision; pixel-space generative models are generative\_ai; user studies of VR are hci. \\[2pt] | ||||
| 157 | +36 & \textbf{Privacy-Preserving Computation}\newline {\footnotesize \texttt{privacy\_\allowbreak preserving\_\allowbreak computation}\newline area \texttt{security\_\allowbreak privacy}; parent of \texttt{federated\_\allowbreak learning}} & Computing on, releasing or sharing data while limiting what is learned about individuals: differential privacy, homomorphic encryption, secure multi-party computation and TEEs applied to data analysis and ML, anonymisation and synthetic data, private information retrieval, privacy attacks on models and their defences, machine unlearning, privacy engineering and regulation (GDPR). Federated learning is a child; cryptographic constructions as such are applied\_cryptography. \\[2pt] | ||||
| 158 | +37 & \textbf{Federated Learning}\newline {\footnotesize \texttt{federated\_\allowbreak learning}\newline area \texttt{security\_\allowbreak privacy}; child of \texttt{privacy\_\allowbreak preserving\_\allowbreak computation}} & Training models across multiple clients or institutions without centralising their raw data: federated averaging and optimisation under heterogeneity, communication efficiency, client selection, personalisation, secure aggregation and privacy of federated systems, robustness to malicious clients, vertical and cross-silo federated learning, federated fine-tuning of foundation models, decentralised/gossip learning. A child of privacy\_preserving\_computation; also linked to distributed\_computing and edge\_computing. \\[2pt] | ||||
| 159 | +38 & \textbf{AI Safety and Alignment}\newline {\footnotesize \texttt{ai\_\allowbreak safety}\newline area \texttt{ai\_\allowbreak ml}} & Making AI systems safe, aligned, robust, fair, interpretable and governable: alignment and preference learning framed as a safety problem, red-teaming and jailbreaks, adversarial robustness and backdoors, hallucination and truthfulness, bias and fairness, mechanistic interpretability, evaluation of dangerous capabilities, AI governance and regulation. Generic system security is cybersecurity; XAI as a statistical method only is ml\_theory. \\[2pt] | ||||
| 160 | +39 & \textbf{Sustainable Computing}\newline {\footnotesize \texttt{sustainable\_\allowbreak computing}\newline area \texttt{hardware\_\allowbreak emerging}} & The energy, carbon, water and material footprint of computing and how to reduce it: energy-efficient and carbon-aware systems, data-centre energy and cooling, Green AI and the cost of training and inference, life-cycle and embodied carbon of hardware, green software engineering and networking, energy harvesting and intermittent computing, computing for sustainability when the object studied is the computing system. Generic low-power circuit design is computer\_architecture; energy-efficient communication is computer\_networks. \\[2pt] | ||||
| 161 | +40 & \textbf{Autonomous Scientific Discovery}\newline {\footnotesize \texttt{autonomous\_\allowbreak scientific\_\allowbreak discovery}\newline area \texttt{computational\_\allowbreak science}} & AI and automation systems that perform or substantially drive the scientific process: AI scientist and research-agent systems, automated hypothesis generation and literature synthesis, self-driving laboratories and closed-loop experimentation, AI-driven discovery of materials, molecules, equations, algorithms or theorems, autonomous experimental design, benchmarks for research agents, and the epistemology and evaluation of machine-generated science. Using ML inside a scientific pipeline without autonomy is scientific\_computing or computational\_biology; LLM agents in general are ai\_agents. \\[2pt] | ||||
| 162 | +\end{longtable} | ||||
| 163 | +\end{small} | ||||
| 164 | +\end{singlespace} | ||||
| 165 | + | ||||
| 166 | +\subsection{Signal types and the assignment rule} | ||||
| 167 | +\label{app:taxonomy-rule} | ||||
| 168 | + | ||||
| 169 | +Every leaf lists weighted signals of five types, and the five types are the only evidence the classifier ever | ||||
| 170 | +sees. (i)~\emph{OpenAlex topics}: the up-to-three topic identifiers that OpenAlex attaches to a work, drawn | ||||
| 171 | +from any of the 26 OpenAlex fields and not only from field~17; a topic may appear under several leaves with | ||||
| 172 | +different weights. (ii)~\emph{OpenAlex subfields}: a coarse fallback used by 21 leaf--subfield entries over | ||||
| 173 | +eleven subfields (the ten field-17 subfields and subfield~1312, Molecular Biology, for computational biology), | ||||
| 174 | +always with a small weight. (iii)~\emph{arXiv categories}: the primary category and cross-lists of the linked | ||||
| 175 | +arXiv record, 162 entries over 49 distinct categories, namely | ||||
| 176 | +\texttt{cs.AI}, \texttt{cs.AR}, \texttt{cs.CC}, \texttt{cs.CE}, \texttt{cs.CG}, \texttt{cs.CL}, \texttt{cs.CR}, \texttt{cs.CV}, \texttt{cs.CY}, \texttt{cs.DB}, \texttt{cs.DC}, \texttt{cs.DL}, \texttt{cs.DM}, \texttt{cs.DS}, \texttt{cs.ET}, \texttt{cs.FL}, \texttt{cs.GR}, \texttt{cs.GT}, \texttt{cs.HC}, \texttt{cs.IR}, \texttt{cs.IT}, \texttt{cs.LG}, \texttt{cs.LO}, \texttt{cs.MA}, \texttt{cs.MM}, \texttt{cs.MS}, \texttt{cs.NA}, \texttt{cs.NE}, \texttt{cs.NI}, \texttt{cs.OS}, \texttt{cs.PF}, \texttt{cs.PL}, \texttt{cs.RO}, \texttt{cs.SD}, \texttt{cs.SE}, \texttt{cs.SI}, \texttt{cs.SY}, \texttt{eess.AS}, \texttt{eess.IV}, \texttt{eess.SP}, \texttt{eess.SY}, \texttt{math.IT}, \texttt{math.OC}, \texttt{math.ST}, \texttt{physics.comp-ph}, \texttt{q-bio.GN}, \texttt{q-bio.QM}, \texttt{quant-ph}, \texttt{stat.ML}. (iv)~\emph{DBLP streams}: the venue stream (journal or conference series, | ||||
| 177 | +the prefix of the DBLP key) of the linked DBLP record, 1{,}320 entries over 771 distinct streams. | ||||
| 178 | +(v)~\emph{Keyword patterns}: 273 regular expressions matched, in the present version of the ontology without | ||||
| 179 | +exception, on the concatenation of title and abstract (the YAML schema allows title-only and abstract-only | ||||
| 180 | +patterns, but none is used); 24 exclusion patterns veto a leaf when they match the same text. | ||||
| 181 | + | ||||
| 182 | +The rule itself is the one written out in equations~\eqref{eq:score} and~\eqref{eq:assign}: within each | ||||
| 183 | +signal type the weights of the matched signals are summed and capped at~1.0; the capped values of the five | ||||
| 184 | +types are summed; the leaf is assigned when the total reaches the threshold $\theta = 1.0$ and no exclusion | ||||
| 185 | +pattern matches; the primary leaf is the argmax, ties going to the child before the parent and then to the | ||||
| 186 | +lower category number. The weights are therefore to be read on a scale where 1.0 means ``this signal alone | ||||
| 187 | +suffices''. Table~\ref{tab:appA-signals} reports, for each leaf, how many signals of each type it lists and | ||||
| 188 | +how many of them carry weight~1.0; Table~\ref{tab:appA-area-signals} aggregates the same counts by area. | ||||
| 189 | +Three regularities are visible. First, every one of the forty leaves has at least one keyword pattern of | ||||
| 190 | +weight 1.0 (106 of the 273 patterns are at 1.0, the rest spread between 0.2 and 0.9), so that a title or | ||||
| 191 | +abstract naming the object of the category assigns it without metadata support; the build script refuses to | ||||
| 192 | +serialise an ontology in which this is false for the thirteen categories that the design singles out (the | ||||
| 193 | +six AI categories that OpenAlex topics do not separate and seven small categories without a topic or venue of | ||||
| 194 | +their own), and in version~1.0 the property happens to hold for all forty. Second, the number of topic | ||||
| 195 | +signals varies by an order of magnitude across leaves, from 4 for \texttt{compilers}, | ||||
| 196 | +\texttt{reinforcement\_\allowbreak learning} and \texttt{federated\_\allowbreak learning} to 45 for \texttt{computer\_\allowbreak networks} and 44 for \texttt{computer\_\allowbreak vision}; this | ||||
| 197 | +mirrors the granularity of the OpenAlex topic model, which has many clusters for networking and vision and | ||||
| 198 | +none for federated learning, and not the importance of the categories. Third, venue signals are dense (291 of | ||||
| 199 | +the 1{,}320 stream entries are at weight 1.0, almost all for single-purpose venues) but concentrated on the | ||||
| 200 | +systems, networking and theory leaves, where the DBLP venue structure is informative, and sparse for the | ||||
| 201 | +post-2022 AI leaves, where the same few venues host everything. | ||||
| 202 | + | ||||
| 203 | +\begin{singlespace} | ||||
| 204 | +\begin{footnotesize} | ||||
| 205 | +\begin{longtable}{@{}r p{4.3cm} r r r r r r r r r r@{}} | ||||
| 206 | +\caption{Signals per category in \texttt{ontology.yaml}: number of weighted OpenAlex topics (T), of which with weight 1.0 (T$_1$); OpenAlex subfields (F); arXiv categories (X); DBLP streams (D), of which with weight 1.0 (D$_1$); keyword patterns (K), of which with weight 1.0 (K$_1$); exclusion patterns (Z); related edges (R). Computed from the YAML file.} | ||||
| 207 | +\label{tab:appA-signals}\\ | ||||
| 208 | +\toprule | ||||
| 209 | +No. & Category & T & T$_1$ & F & X & D & D$_1$ & K & K$_1$ & Z & R \\ | ||||
| 210 | +\midrule | ||||
| 211 | +\endfirsthead | ||||
| 212 | +\caption[]{Signals per category (continued).}\\ | ||||
| 213 | +\toprule | ||||
| 214 | +No. & Category & T & T$_1$ & F & X & D & D$_1$ & K & K$_1$ & Z & R \\ | ||||
| 215 | +\midrule | ||||
| 216 | +\endhead | ||||
| 217 | +\midrule | ||||
| 218 | +\multicolumn{12}{r}{continued on next page}\\ | ||||
| 219 | +\endfoot | ||||
| 220 | +\bottomrule | ||||
| 221 | +\endlastfoot | ||||
| 222 | +1 & \texttt{llm} & 6 & 1 & 1 & 3 & 12 & 0 & 12 & 3 & 0 & 5 \\ | ||||
| 223 | +2 & \texttt{foundation\_\allowbreak models} & 5 & 0 & 1 & 4 & 8 & 0 & 10 & 2 & 0 & 5 \\ | ||||
| 224 | +3 & \texttt{multimodal\_\allowbreak ai} & 5 & 1 & 1 & 4 & 15 & 0 & 9 & 4 & 1 & 5 \\ | ||||
| 225 | +4 & \texttt{generative\_\allowbreak ai} & 5 & 1 & 0 & 6 & 9 & 0 & 11 & 4 & 2 & 6 \\ | ||||
| 226 | +5 & \texttt{ai\_\allowbreak agents} & 5 & 1 & 1 & 2 & 12 & 3 & 9 & 5 & 1 & 5 \\ | ||||
| 227 | +6 & \texttt{reinforcement\_\allowbreak learning} & 4 & 1 & 0 & 5 & 8 & 0 & 9 & 3 & 1 & 5 \\ | ||||
| 228 | +7 & \texttt{ml\_\allowbreak theory} & 20 & 1 & 1 & 4 & 51 & 2 & 11 & 1 & 0 & 6 \\ | ||||
| 229 | +8 & \texttt{deep\_\allowbreak learning\_\allowbreak architectures} & 11 & 2 & 1 & 4 & 21 & 0 & 12 & 1 & 1 & 6 \\ | ||||
| 230 | +9 & \texttt{computer\_\allowbreak vision} & 44 & 9 & 1 & 2 & 58 & 13 & 10 & 3 & 0 & 6 \\ | ||||
| 231 | +10 & \texttt{nlp} & 22 & 4 & 1 & 2 & 29 & 13 & 8 & 2 & 0 & 5 \\ | ||||
| 232 | +11 & \texttt{speech\_\allowbreak audio} & 10 & 3 & 1 & 4 & 16 & 5 & 5 & 3 & 0 & 4 \\ | ||||
| 233 | +12 & \texttt{robotics} & 23 & 6 & 0 & 3 & 41 & 27 & 4 & 1 & 2 & 5 \\ | ||||
| 234 | +13 & \texttt{hci} & 24 & 4 & 1 & 3 & 44 & 22 & 10 & 1 & 0 & 6 \\ | ||||
| 235 | +14 & \texttt{software\_\allowbreak engineering} & 20 & 7 & 1 & 1 & 39 & 18 & 5 & 3 & 0 & 6 \\ | ||||
| 236 | +15 & \texttt{programming\_\allowbreak languages} & 6 & 1 & 0 & 3 & 22 & 6 & 6 & 3 & 0 & 4 \\ | ||||
| 237 | +16 & \texttt{compilers} & 4 & 0 & 0 & 5 & 25 & 2 & 5 & 3 & 0 & 5 \\ | ||||
| 238 | +17 & \texttt{formal\_\allowbreak verification} & 10 & 1 & 0 & 4 & 43 & 10 & 7 & 5 & 0 & 6 \\ | ||||
| 239 | +18 & \texttt{algorithms} & 35 & 2 & 1 & 7 & 68 & 9 & 10 & 2 & 0 & 6 \\ | ||||
| 240 | +19 & \texttt{computational\_\allowbreak complexity} & 9 & 0 & 1 & 7 & 28 & 2 & 7 & 5 & 1 & 5 \\ | ||||
| 241 | +20 & \texttt{database\_\allowbreak systems} & 21 & 1 & 1 & 1 & 36 & 9 & 5 & 2 & 1 & 5 \\ | ||||
| 242 | +21 & \texttt{information\_\allowbreak retrieval} & 20 & 2 & 1 & 3 & 29 & 5 & 6 & 3 & 0 & 5 \\ | ||||
| 243 | +22 & \texttt{distributed\_\allowbreak computing} & 16 & 2 & 1 & 4 & 51 & 15 & 7 & 3 & 1 & 8 \\ | ||||
| 244 | +23 & \texttt{cloud\_\allowbreak computing} & 11 & 2 & 0 & 4 & 29 & 5 & 5 & 2 & 1 & 6 \\ | ||||
| 245 | +24 & \texttt{edge\_\allowbreak computing} & 13 & 1 & 0 & 4 & 38 & 1 & 5 & 2 & 1 & 7 \\ | ||||
| 246 | +25 & \texttt{computer\_\allowbreak networks} & 45 & 9 & 1 & 4 & 96 & 42 & 5 & 1 & 0 & 5 \\ | ||||
| 247 | +26 & \texttt{cybersecurity} & 34 & 7 & 0 & 1 & 29 & 16 & 6 & 2 & 1 & 8 \\ | ||||
| 248 | +27 & \texttt{applied\_\allowbreak cryptography} & 14 & 3 & 0 & 4 & 29 & 9 & 6 & 3 & 0 & 6 \\ | ||||
| 249 | +28 & \texttt{blockchain} & 6 & 1 & 0 & 3 & 19 & 3 & 4 & 2 & 1 & 5 \\ | ||||
| 250 | +29 & \texttt{computer\_\allowbreak architecture} & 21 & 4 & 1 & 5 & 59 & 6 & 4 & 2 & 1 & 7 \\ | ||||
| 251 | +30 & \texttt{ai\_\allowbreak accelerators} & 11 & 1 & 0 & 5 & 44 & 1 & 5 & 3 & 1 & 7 \\ | ||||
| 252 | +31 & \texttt{operating\_\allowbreak systems} & 12 & 0 & 0 & 5 & 39 & 6 & 4 & 2 & 1 & 6 \\ | ||||
| 253 | +32 & \texttt{quantum\_\allowbreak computing} & 16 & 2 & 0 & 6 & 27 & 4 & 3 & 2 & 1 & 5 \\ | ||||
| 254 | +33 & \texttt{scientific\_\allowbreak computing} & 41 & 4 & 1 & 6 & 57 & 10 & 4 & 2 & 0 & 8 \\ | ||||
| 255 | +34 & \texttt{computational\_\allowbreak biology} & 34 & 3 & 1 & 3 & 60 & 12 & 3 & 2 & 0 & 7 \\ | ||||
| 256 | +35 & \texttt{computer\_\allowbreak graphics} & 15 & 2 & 1 & 4 & 21 & 8 & 9 & 2 & 0 & 5 \\ | ||||
| 257 | +36 & \texttt{privacy\_\allowbreak preserving\_\allowbreak computation} & 13 & 1 & 0 & 4 & 26 & 1 & 5 & 3 & 1 & 7 \\ | ||||
| 258 | +37 & \texttt{federated\_\allowbreak learning} & 4 & 0 & 0 & 4 & 15 & 0 & 5 & 2 & 1 & 6 \\ | ||||
| 259 | +38 & \texttt{ai\_\allowbreak safety} & 8 & 1 & 0 & 5 & 13 & 4 & 14 & 7 & 1 & 7 \\ | ||||
| 260 | +39 & \texttt{sustainable\_\allowbreak computing} & 11 & 2 & 0 & 6 & 28 & 2 & 4 & 2 & 1 & 7 \\ | ||||
| 261 | +40 & \texttt{autonomous\_\allowbreak scientific\_\allowbreak discovery} & 12 & 0 & 0 & 8 & 26 & 0 & 4 & 3 & 1 & 8 \\ | ||||
| 262 | +\midrule | ||||
| 263 | +& Total (entries) & 646 & 93 & 21 & 162 & 1320 & 291 & 273 & 106 & 24 & 236 \\ | ||||
| 264 | +\end{longtable} | ||||
| 265 | +\end{footnotesize} | ||||
| 266 | +\end{singlespace} | ||||
| 267 | + | ||||
| 268 | +\begin{table}[htbp] | ||||
| 269 | +\begin{singlespace} | ||||
| 270 | +\centering | ||||
| 271 | +\caption{Signal entries aggregated by area (sums of the rows of Table~\ref{tab:appA-signals}; a topic or venue mapped to several leaves of the same area is counted once per leaf).} | ||||
| 272 | +\label{tab:appA-area-signals} | ||||
| 273 | +\small | ||||
| 274 | +\begin{tabular}{l r r r r r} | ||||
| 275 | +\toprule | ||||
| 276 | +Area & Leaves & Topic entries & arXiv entries & DBLP entries & Keyword patterns \\ | ||||
| 277 | +\midrule | ||||
| 278 | +\texttt{ai\_\allowbreak ml} & 9 & 69 & 37 & 149 & 97 \\ | ||||
| 279 | +\texttt{perception\_\allowbreak interaction} & 6 & 138 & 18 & 209 & 46 \\ | ||||
| 280 | +\texttt{software\_\allowbreak pl} & 4 & 40 & 13 & 129 & 23 \\ | ||||
| 281 | +\texttt{theory} & 2 & 44 & 14 & 96 & 17 \\ | ||||
| 282 | +\texttt{data\_\allowbreak distributed} & 6 & 126 & 20 & 279 & 33 \\ | ||||
| 283 | +\texttt{security\_\allowbreak privacy} & 5 & 71 & 16 & 118 & 26 \\ | ||||
| 284 | +\texttt{hardware\_\allowbreak emerging} & 5 & 71 & 27 & 197 & 20 \\ | ||||
| 285 | +\texttt{computational\_\allowbreak science} & 3 & 87 & 17 & 143 & 11 \\ | ||||
| 286 | +\bottomrule | ||||
| 287 | +\end{tabular} | ||||
| 288 | +\end{singlespace} | ||||
| 289 | +\end{table} | ||||
| 290 | + | ||||
| 291 | +The weight scale has a second consequence that the reader of Part~IX must keep in mind. A signal with weight | ||||
| 292 | +below 1.0 can only contribute through corroboration by a signal of \emph{another} type, because repetitions | ||||
| 293 | +within a type are capped. The design relies on this in both directions: a general machine-learning venue at | ||||
| 294 | +0.3 and the \texttt{cs.LG} category at 0.2 never assign \texttt{ml\_\allowbreak theory} by themselves, whereas a 0.6 topic | ||||
| 295 | +and a 0.4 arXiv category do reach the threshold. One discrepancy between the documentation and the | ||||
| 296 | +implementation of the within-type aggregation (maximum versus sum over matched topics of the same leaf) is | ||||
| 297 | +recorded in Section~\ref{sec:m-score}; the implemented rule, the sum, is the one described here and the one | ||||
| 298 | +fixed by the unit tests. | ||||
| 299 | + | ||||
| 300 | +\subsection{Coverage of the OpenAlex topic vocabulary} | ||||
| 301 | +\label{app:taxonomy-openalex} | ||||
| 302 | + | ||||
| 303 | +The OpenAlex export of 2026-10-14 contains 4{,}516 topics, 302 of them in field~17 (Computer Science). The | ||||
| 304 | +ontology maps 240 of the 302 (79.5\,\%) to at least one leaf and leaves 62 unmapped with an explicit reason | ||||
| 305 | +each (Section~\ref{app:taxonomy-unmapped}). Table~\ref{tab:appA-subfields} breaks the field-17 coverage down | ||||
| 306 | +by subfield. Coverage is complete for Computer Vision and Pattern Recognition, Hardware and Architecture, | ||||
| 307 | +Human--Computer Interaction, Signal Processing and Software, and nearly complete for Computational Theory and | ||||
| 308 | +Mathematics; it is lowest for Information Systems (45 of 78), the subfield into which OpenAlex places most of | ||||
| 309 | +its educational-technology and media clusters, and for Artificial Intelligence (61 of 77), where the unmapped | ||||
| 310 | +topics are physical-science, generic residual or empty clusters rather than AI research. | ||||
| 311 | + | ||||
| 312 | +\begin{table}[htbp] | ||||
| 313 | +\begin{singlespace} | ||||
| 314 | +\centering | ||||
| 315 | +\caption{Coverage of the OpenAlex field~17 (Computer Science) topics by subfield: number of topics in the 2026-10-14 export, mapped to at least one category, and left unmapped with a stated reason (Table~\ref{tab:appA-unmapped}).} | ||||
| 316 | +\label{tab:appA-subfields} | ||||
| 317 | +\small | ||||
| 318 | +\begin{tabular}{l l r r r} | ||||
| 319 | +\toprule | ||||
| 320 | +Subfield & Name & Topics & Mapped & Unmapped \\ | ||||
| 321 | +\midrule | ||||
| 322 | +1702 & Artificial Intelligence & 77 & 61 & 16 \\ | ||||
| 323 | +1703 & Computational Theory and Mathematics & 24 & 23 & 1 \\ | ||||
| 324 | +1704 & Computer Graphics and Computer-Aided Design & 4 & 3 & 1 \\ | ||||
| 325 | +1705 & Computer Networks and Communications & 43 & 35 & 8 \\ | ||||
| 326 | +1706 & Computer Science Applications & 10 & 7 & 3 \\ | ||||
| 327 | +1707 & Computer Vision and Pattern Recognition & 35 & 35 & 0 \\ | ||||
| 328 | +1708 & Hardware and Architecture & 8 & 8 & 0 \\ | ||||
| 329 | +1709 & Human-Computer Interaction & 8 & 8 & 0 \\ | ||||
| 330 | +1710 & Information Systems & 78 & 45 & 33 \\ | ||||
| 331 | +1711 & Signal Processing & 10 & 10 & 0 \\ | ||||
| 332 | +1712 & Software & 5 & 5 & 0 \\ | ||||
| 333 | +\midrule | ||||
| 334 | +& Field 17 total & 302 & 240 & 62 \\ | ||||
| 335 | +\bottomrule | ||||
| 336 | +\end{tabular} | ||||
| 337 | +\end{singlespace} | ||||
| 338 | +\end{table} | ||||
| 339 | + | ||||
| 340 | +Topics outside field~17 matter because computer-science work is frequently assigned a primary topic from | ||||
| 341 | +Engineering, Mathematics, Physics or the life sciences, and because several leaves (computational biology, | ||||
| 342 | +scientific computing, quantum computing, speech and audio, robotics) have their natural OpenAlex home outside | ||||
| 343 | +field~17. The ontology therefore maps 170 further topics from eighteen other fields | ||||
| 344 | +(Table~\ref{tab:appA-fields}), Engineering (71) and Mathematics (20) being the largest contributors, for a | ||||
| 345 | +total of 410 distinct mapped topics and 646 topic--leaf weights. These adjacent topics were selected by | ||||
| 346 | +reading the OpenAlex keyword lists of about 120 candidate topics named in the area modules; the remaining | ||||
| 347 | +roughly 4{,}000 topics of the export were not inspected, which is the main known gap of the mapping | ||||
| 348 | +(Section~\ref{app:taxonomy-limits}). Of the 410 mapped topics, 146 point to more than one leaf, up to six for | ||||
| 349 | +the broadest clusters (for instance T10054, \emph{Parallel Computing and Optimization Techniques}, scores in | ||||
| 350 | +\texttt{computer\_\allowbreak architecture} at 0.8, \texttt{compilers} and \texttt{distributed\_\allowbreak computing} at 0.6, | ||||
| 351 | +\texttt{scientific\_\allowbreak computing} at 0.4, \texttt{ai\_\allowbreak accelerators} at 0.3 and \texttt{operating\_\allowbreak systems} at | ||||
| 352 | +0.2). The full audit list of multiply mapped topics is the last generated section of \texttt{ONTOLOGY.md}; | ||||
| 353 | +it is the first place to look when a co-classification pattern in Part~IX seems suspicious, because a topic | ||||
| 354 | +mapped to two leaves at weight 1.0 produces that pair mechanically. | ||||
| 355 | + | ||||
| 356 | +\begin{table}[htbp] | ||||
| 357 | +\begin{singlespace} | ||||
| 358 | +\centering | ||||
| 359 | +\caption{Distinct OpenAlex topics mapped to at least one category, by OpenAlex field of the topic (\texttt{topic\_map.csv}).} | ||||
| 360 | +\label{tab:appA-fields} | ||||
| 361 | +\small | ||||
| 362 | +\begin{tabular}{r l r} | ||||
| 363 | +\toprule | ||||
| 364 | +Field & Name & Mapped topics \\ | ||||
| 365 | +\midrule | ||||
| 366 | +17 & Computer Science & 240 \\ | ||||
| 367 | +22 & Engineering & 71 \\ | ||||
| 368 | +26 & Mathematics & 20 \\ | ||||
| 369 | +13 & Biochemistry, Genetics and Molecular Biology & 18 \\ | ||||
| 370 | +31 & Physics and Astronomy & 15 \\ | ||||
| 371 | +33 & Social Sciences & 10 \\ | ||||
| 372 | +18 & Decision Sciences & 8 \\ | ||||
| 373 | +27 & Medicine & 6 \\ | ||||
| 374 | +12 & Arts and Humanities & 5 \\ | ||||
| 375 | +28 & Neuroscience & 4 \\ | ||||
| 376 | +23 & Environmental Science & 3 \\ | ||||
| 377 | +19 & Earth and Planetary Sciences & 2 \\ | ||||
| 378 | +32 & Psychology & 2 \\ | ||||
| 379 | +11 & Agricultural and Biological Sciences & 1 \\ | ||||
| 380 | +14 & Business, Management and Accounting & 1 \\ | ||||
| 381 | +20 & Economics, Econometrics and Finance & 1 \\ | ||||
| 382 | +21 & Energy & 1 \\ | ||||
| 383 | +25 & Materials Science & 1 \\ | ||||
| 384 | +36 & Health Professions & 1 \\ | ||||
| 385 | +\midrule | ||||
| 386 | +& All fields & 410 \\ | ||||
| 387 | +\bottomrule | ||||
| 388 | +\end{tabular} | ||||
| 389 | +\end{singlespace} | ||||
| 390 | +\end{table} | ||||
| 391 | + | ||||
| 392 | +\subsection{Coverage of the DBLP venue vocabulary} | ||||
| 393 | +\label{app:taxonomy-dblp} | ||||
| 394 | + | ||||
| 395 | +DBLP streams are the venue signal. The stream table from which the mapping was built lists every DBLP | ||||
| 396 | +journal or conference series with at least one hundred records in 2018--2026 (3{,}569 streams); the ontology | ||||
| 397 | +maps 771 of them. Measured on the DBLP records of 2018--2026 and excluding CoRR (the arXiv mirror stream, | ||||
| 398 | +which carries no venue information), the mapped streams cover 2{,}017{,}910 of 3{,}319{,}884 records | ||||
| 399 | +(60.8\,\%). Among the 599 largest streams, 496 are mapped (82.8\,\%); the 103 unmapped large streams | ||||
| 400 | +(Table~\ref{tab:appA-dblp-unmapped} lists the 25 largest) are of two kinds. Some are out of scope for the | ||||
| 401 | +forty categories although DBLP indexes them: control theory (\emph{IEEE Transactions on Automatic Control}, | ||||
| 402 | +\emph{Automatica}, \emph{IEEE Control Systems Letters}), instrumentation and industrial electronics, | ||||
| 403 | +geo-information, agriculture, production research, information-systems and engineering-education | ||||
| 404 | +conferences (HICSS, AMCIS, ICIS, FIE). Others are too generic to be a category signal at any weight, the | ||||
| 405 | +largest being \emph{IEEE Access} with 108{,}838 records, followed by \emph{Information}, \emph{Complexity}, | ||||
| 406 | +\emph{Systems} and \emph{PeerJ Computer Science}; works published there are classified from their topics, | ||||
| 407 | +arXiv categories and text alone. A record in an unmapped stream is therefore not excluded from the corpus | ||||
| 408 | +(scope is defined in Part~III, not here); it simply receives no venue evidence. Because weights are assigned | ||||
| 409 | +per stream, workshop volumes that share a stream key with a flagship conference inherit the flagship's | ||||
| 410 | +weight, a known coarseness of the venue signal. | ||||
| 411 | + | ||||
| 412 | +\begin{table}[htbp] | ||||
| 413 | +\begin{singlespace} | ||||
| 414 | +\centering | ||||
| 415 | +\caption{The 25 largest DBLP streams (2018--2026 records, CoRR excluded) deliberately left unmapped, out of 103 unmapped streams among the 599 largest (\texttt{ONTOLOGY.md}, generated section ``DBLP coverage'').} | ||||
| 416 | +\label{tab:appA-dblp-unmapped} | ||||
| 417 | +\small | ||||
| 418 | +\begin{tabular}{l r l} | ||||
| 419 | +\toprule | ||||
| 420 | +Stream & Records & Venue \\ | ||||
| 421 | +\midrule | ||||
| 422 | +\texttt{journals/access} & 108,838 & IEEE Access \\ | ||||
| 423 | +\texttt{journals/tim} & 13,470 & IEEE Trans. Instrum. Meas. \\ | ||||
| 424 | +\texttt{journals/tie} & 10,844 & IEEE Trans. Ind. Electron. \\ | ||||
| 425 | +\texttt{conf/hicss} & 7,816 & HICSS \\ | ||||
| 426 | +\texttt{conf/iecon} & 7,727 & IECON \\ | ||||
| 427 | +\texttt{journals/cea} & 7,291 & Comput. Electron. Agric. \\ | ||||
| 428 | +\texttt{conf/smc} & 6,048 & SMC \\ | ||||
| 429 | +\texttt{journals/complexity} & 5,786 & Complex. \\ | ||||
| 430 | +\texttt{journals/ress} & 5,732 & Reliab. Eng. Syst. Saf. \\ | ||||
| 431 | +\texttt{journals/tac} & 5,687 & IEEE Trans. Autom. Control. \\ | ||||
| 432 | +\texttt{journals/information} & 5,621 & Inf. \\ | ||||
| 433 | +\texttt{journals/taes} & 5,543 & IEEE Trans. Aerosp. Electron. Syst. \\ | ||||
| 434 | +\texttt{journals/jfi} & 5,432 & J. Frankl. Inst. \\ | ||||
| 435 | +\texttt{journals/ijgi} & 5,118 & ISPRS Int. J. Geo Inf. \\ | ||||
| 436 | +\texttt{journals/automatica} & 4,727 & Autom. \\ | ||||
| 437 | +\texttt{conf/eucc} & 4,160 & ECC \\ | ||||
| 438 | +\texttt{conf/iccsa} & 3,817 & ICCSA (4) \\ | ||||
| 439 | +\texttt{conf/fie} & 3,807 & FIE \\ | ||||
| 440 | +\texttt{conf/amcis} & 3,798 & AMCIS \\ | ||||
| 441 | +\texttt{journals/peerj-cs} & 3,768 & PeerJ Comput. Sci. \\ | ||||
| 442 | +\texttt{journals/systems} & 3,667 & Syst. \\ | ||||
| 443 | +\texttt{journals/csysl} & 3,647 & IEEE Control. Syst. Lett. \\ | ||||
| 444 | +\texttt{journals/aei} & 3,628 & Adv. Eng. Informatics \\ | ||||
| 445 | +\texttt{journals/ijpr} & 3,591 & Int. J. Prod. Res. \\ | ||||
| 446 | +\texttt{journals/sj} & 3,558 & IEEE Syst. J. \\ | ||||
| 447 | +\bottomrule | ||||
| 448 | +\end{tabular} | ||||
| 449 | +\end{singlespace} | ||||
| 450 | +\end{table} | ||||
| 451 | + | ||||
| 452 | +\subsection{Unmapped field-17 topics and the reasons} | ||||
| 453 | +\label{app:taxonomy-unmapped} | ||||
| 454 | + | ||||
| 455 | +OpenAlex places in field~17 a number of clusters that are not computer-science research. The ontology leaves | ||||
| 456 | +62 field-17 topics unmapped and records a reason for each; Table~\ref{tab:appA-reasons} groups the reasons | ||||
| 457 | +into eight codes and Table~\ref{tab:appA-unmapped} lists the topics. The dominant case, 29 topics, is | ||||
| 458 | +education and e-learning research (pedagogy, learning outcomes, teacher development, educational | ||||
| 459 | +technology studied as an intervention rather than as software); such work is not one of the forty categories, | ||||
| 460 | +and educational technology \emph{as software} is still captured through software-engineering and HCI venues | ||||
| 461 | +and keywords when the contribution is technical. Nine topics are physical-science or engineering clusters | ||||
| 462 | +without computing content (sensor instrumentation, photovoltaics, geochemistry, seismology, nonlinear | ||||
| 463 | +dynamics, control theory, transportation safety, chemical and environmental engineering). Five are generic | ||||
| 464 | +residual labels (``Computer Science and Engineering'', ``Diverse Interdisciplinary Research Studies'', | ||||
| 465 | +``Mathematics, Computing, and Information Processing'', ``Statistical and Computational Modeling'', | ||||
| 466 | +``Advanced Technologies and Applied Computing'') whose keyword lists carry no specific content; works bearing | ||||
| 467 | +them reach categories through their other topics, venue, arXiv category and text. Four are media, | ||||
| 468 | +communication, literary or philosophical clusters, four are out-of-scope disciplines (psychiatry, economic | ||||
| 469 | +growth, cognitive science, pandemic health care), one is the history of computing (kept out of the | ||||
| 470 | +research-category map but relevant to Part~II), one is the small cognitive-informatics cluster that has no | ||||
| 471 | +counterpart among the forty leaves and was not forced into \texttt{ai\_\allowbreak agents}, and nine are residual topics | ||||
| 472 | +with zero works in the export. The works column of Table~\ref{tab:appA-unmapped} is the OpenAlex | ||||
| 473 | +\texttt{works\_\allowbreak count} of the topic over all years and fields; it is reported to show the scale of what is set | ||||
| 474 | +aside (the education cluster alone exceeds one million works, the media cluster T13745 alone 204{,}789) and is | ||||
| 475 | +not a count of corpus records. The hand-written design note of \texttt{ONTOLOGY.md} speaks of 63 unmapped | ||||
| 476 | +clusters; the generated tables, which are authoritative, list 62. | ||||
| 477 | + | ||||
| 478 | +\begin{table}[htbp] | ||||
| 479 | +\begin{singlespace} | ||||
| 480 | +\centering | ||||
| 481 | +\caption{Reason codes for the unmapped field~17 topics (Table~\ref{tab:appA-unmapped}), with the number of topics and the sum of their OpenAlex \texttt{works\_\allowbreak count}.} | ||||
| 482 | +\label{tab:appA-reasons} | ||||
| 483 | +\small | ||||
| 484 | +\begin{tabularx}{\textwidth}{c X r r} | ||||
| 485 | +\toprule | ||||
| 486 | +Code & Reason recorded in the ontology & Topics & Works (sum) \\ | ||||
| 487 | +\midrule | ||||
| 488 | +E & Education / e-learning research; not a CS research category; technical ed-tech captured elsewhere & 29 & 1{,}027{,}591 \\ | ||||
| 489 | +P & Physical-science or engineering cluster without computing contribution, placed in field~17 by OpenAlex & 9 & 352{,}761 \\ | ||||
| 490 | +Z & Residual OpenAlex topic with zero works in the export; nothing to map & 9 & 0 \\ | ||||
| 491 | +G & Generic, uninformative residual label; works reach categories through other signals & 5 & 30{,}206 \\ | ||||
| 492 | +M & Media, communication, cultural, literary or philosophy-of-science cluster; not CS research & 4 & 275{,}159 \\ | ||||
| 493 | +O & Out-of-scope discipline (psychiatry, economics, cognitive science, pandemic health care) & 4 & 33{,}228 \\ | ||||
| 494 | +H & History of computing technologies (history of science); background for Part~II only & 1 & 72{,}582 \\ | ||||
| 495 | +C & Cognitive informatics / brain informatics; idiosyncratic cluster with no counterpart among the leaves & 1 & 13{,}053 \\ | ||||
| 496 | +\midrule | ||||
| 497 | +& Total & 62 & 1{,}804{,}580 \\ | ||||
| 498 | +\bottomrule | ||||
| 499 | +\end{tabularx} | ||||
| 500 | +\end{singlespace} | ||||
| 501 | +\end{table} | ||||
| 502 | + | ||||
| 503 | +\begin{singlespace} | ||||
| 504 | +\begin{footnotesize} | ||||
| 505 | +\begin{longtable}{@{}l p{7.4cm} l r l@{}} | ||||
| 506 | +\caption{The 62 OpenAlex field~17 topics left unmapped, with the subfield, the number of works of the topic in the 2026-10-14 topic export (all years, all fields) and the reason code of Table~\ref{tab:appA-reasons}.} | ||||
| 507 | +\label{tab:appA-unmapped}\\ | ||||
| 508 | +\toprule | ||||
| 509 | +Topic & Name & Subfield & Works & Reason \\ | ||||
| 510 | +\midrule | ||||
| 511 | +\endfirsthead | ||||
| 512 | +\caption[]{Unmapped field~17 topics (continued).}\\ | ||||
| 513 | +\toprule | ||||
| 514 | +Topic & Name & Subfield & Works & Reason \\ | ||||
| 515 | +\midrule | ||||
| 516 | +\endhead | ||||
| 517 | +\midrule | ||||
| 518 | +\multicolumn{5}{r}{continued on next page}\\ | ||||
| 519 | +\endfoot | ||||
| 520 | +\bottomrule | ||||
| 521 | +\endlastfoot | ||||
| 522 | +T13745 & Media and Digital Communication & 1705 & 204,789 & M \\ | ||||
| 523 | +T14496 & Educational Technology and E-Learning & 1710 & 163,032 & E \\ | ||||
| 524 | +T14516 & English Language Learning and Teaching & 1710 & 107,202 & E \\ | ||||
| 525 | +T14430 & Higher Education and Teaching Methods & 1710 & 103,711 & E \\ | ||||
| 526 | +T12797 & History of Computing Technologies & 1706 & 72,582 & H \\ | ||||
| 527 | +T12564 & Sensor Technology and Measurement Systems & 1705 & 71,005 & P \\ | ||||
| 528 | +T11276 & Solar Radiation and Photovoltaics & 1702 & 70,601 & P \\ | ||||
| 529 | +T14455 & Technology and Education Systems & 1705 & 70,588 & E \\ | ||||
| 530 | +T12157 & Geochemistry and Geologic Mapping & 1702 & 69,914 & P \\ | ||||
| 531 | +T12171 & Open Education and E-Learning & 1706 & 66,332 & E \\ | ||||
| 532 | +T14498 & Educational Technology in Learning & 1710 & 63,096 & E \\ | ||||
| 533 | +T14216 & Multimedia Learning Systems & 1710 & 62,005 & E \\ | ||||
| 534 | +T11187 & Nonlinear Dynamics and Pattern Formation & 1705 & 61,626 & P \\ | ||||
| 535 | +T14103 & Educational Management and Quality & 1710 & 57,729 & E \\ | ||||
| 536 | +T11370 & Mobile Learning in Education & 1710 & 55,961 & E \\ | ||||
| 537 | +T12408 & Educational Innovations and Challenges & 1710 & 53,673 & E \\ | ||||
| 538 | +T12395 & Educational Innovations and Technology & 1710 & 45,138 & E \\ | ||||
| 539 | +T14025 & Educational Technology and Assessment & 1710 & 37,519 & E \\ | ||||
| 540 | +T13602 & Educational Methods and Media Use & 1710 & 35,313 & E \\ | ||||
| 541 | +T13194 & ICT in Developing Communities & 1710 & 32,298 & M \\ | ||||
| 542 | +T13615 & Transportation Systems and Safety & 1704 & 26,785 & P \\ | ||||
| 543 | +T14215 & Chemical and Environmental Engineering Research & 1710 & 25,315 & P \\ | ||||
| 544 | +T13018 & Seismology and Earthquake Studies & 1702 & 23,104 & P \\ | ||||
| 545 | +T13081 & Digital Education and Society & 1710 & 22,995 & E \\ | ||||
| 546 | +T14029 & Thoreau and American Literature & 1710 & 19,567 & M \\ | ||||
| 547 | +T13386 & Educational Methods and Teacher Development & 1710 & 19,565 & E \\ | ||||
| 548 | +T14345 & Scientific Research and Philosophical Inquiry & 1710 & 18,505 & M \\ | ||||
| 549 | +T13016 & Innovative Educational Technologies & 1710 & 16,953 & E \\ | ||||
| 550 | +T13674 & Computer Science and Engineering & 1702 & 16,620 & G \\ | ||||
| 551 | +T13559 & Edcuational Technology Systems & 1702 & 16,507 & E \\ | ||||
| 552 | +T14381 & Psychiatry, Mental Health, Neuroscience & 1702 & 15,800 & O \\ | ||||
| 553 | +T14519 & Innovations in Education and Learning Technologies & 1710 & 14,509 & E \\ | ||||
| 554 | +T13062 & Cognitive Computing and Networks & 1702 & 13,053 & C \\ | ||||
| 555 | +T13867 & Economic Growth and Development & 1710 & 9,780 & O \\ | ||||
| 556 | +T13898 & Diverse Interdisciplinary Research Studies & 1702 & 6,495 & G \\ | ||||
| 557 | +T14246 & E-Learning and Knowledge Management & 1706 & 6,373 & E \\ | ||||
| 558 | +T13523 & Mathematics, Computing, and Information Processing & 1703 & 6,343 & G \\ | ||||
| 559 | +T12805 & Cognitive Science and Mapping & 1702 & 5,430 & O \\ | ||||
| 560 | +T13935 & Mathematical Control Systems and Analysis & 1702 & 4,409 & P \\ | ||||
| 561 | +T14409 & Educational Systems and Policies & 1710 & 3,650 & E \\ | ||||
| 562 | +T13565 & Education and Learning Interventions & 1710 & 2,867 & E \\ | ||||
| 563 | +T14071 & Healthcare during COVID-19 Pandemic & 1710 & 2,218 & O \\ | ||||
| 564 | +T14403 & Educational Challenges and Innovations & 1710 & 1,150 & E \\ | ||||
| 565 | +T13482 & Engineering Education and Technology & 1710 & 765 & E \\ | ||||
| 566 | +T14351 & Statistical and Computational Modeling & 1702 & 731 & G \\ | ||||
| 567 | +T13932 & Education, Technology, and Economics & 1710 & 577 & E \\ | ||||
| 568 | +T13873 & Educational Methods and Technology & 1710 & 329 & E \\ | ||||
| 569 | +T13676 & Educational and Technological Research & 1710 & 52 & E \\ | ||||
| 570 | +T13345 & Advanced Technologies and Applied Computing & 1705 & 17 & G \\ | ||||
| 571 | +T14387 & Environmental Engineering and Cultural Studies & 1710 & 2 & P \\ | ||||
| 572 | +T13514 & Experience-Based Knowledge Management & 1702 & 0 & Z \\ | ||||
| 573 | +T13567 & AI and Multimedia in Education & 1702 & 0 & E \\ | ||||
| 574 | +T13693 & Smart Systems and Machine Learning & 1705 & 0 & Z \\ | ||||
| 575 | +T13748 & Advanced Statistical Modeling Techniques & 1705 & 0 & Z \\ | ||||
| 576 | +T13836 & Educational Research and Pedagogy & 1705 & 0 & E \\ | ||||
| 577 | +T13964 & Educational Technology and Optimization & 1710 & 0 & E \\ | ||||
| 578 | +T14064 & Organizational and Employee Performance & 1702 & 0 & Z \\ | ||||
| 579 | +T14155 & Advanced Technology in Applications & 1710 & 0 & Z \\ | ||||
| 580 | +T14174 & Artificial Intelligence and Decision Support Systems & 1710 & 0 & Z \\ | ||||
| 581 | +T14175 & Intuitionistic Fuzzy Systems Applications & 1702 & 0 & Z \\ | ||||
| 582 | +T14413 & Advanced Technologies in Various Fields & 1702 & 0 & Z \\ | ||||
| 583 | +T14431 & Diverse Research and Applications & 1710 & 0 & Z \\ | ||||
| 584 | +\end{longtable} | ||||
| 585 | +\end{footnotesize} | ||||
| 586 | +\end{singlespace} | ||||
| 587 | + | ||||
| 588 | +\subsection{Known ambiguities and the decisions taken} | ||||
| 589 | +\label{app:taxonomy-ambiguities} | ||||
| 590 | + | ||||
| 591 | +Forty categories drawn up for a mandate are not a partition of the literature, and several pairs or triples | ||||
| 592 | +overlap by construction. The ontology's author resolved each overlap by a stated rule rather than by leaving | ||||
| 593 | +it to the weights; the rules are recorded here verbatim in substance, because they determine how the | ||||
| 594 | +multi-label statistics of Part~IX must be read. | ||||
| 595 | + | ||||
| 596 | +\paragraph{Large language models, foundation models, generative AI and NLP.} | ||||
| 597 | +\texttt{foundation\_\allowbreak models} is the parent concept (any modality, adaptation-oriented); \texttt{llm} is its | ||||
| 598 | +language child and the home of the ChatGPT/GPT/LLaMA-era literature; \texttt{generative\_\allowbreak ai} concerns the | ||||
| 599 | +generative model and its outputs (diffusion, GANs, text-to-X synthesis, deepfakes and detection), not text | ||||
| 600 | +generation by LLMs as such; \texttt{nlp} keeps task-framed language work. Multi-label assignment is expected | ||||
| 601 | +and intended: an instruction-tuning paper scores in \texttt{llm} and \texttt{nlp}, a diffusion language model | ||||
| 602 | +in \texttt{llm} and \texttt{generative\_\allowbreak ai}. The OpenAlex topic T10028, labelled ``Topic Modeling'', is in | ||||
| 603 | +fact the cluster hosting BERT, pre-trained language models and retrieval-augmented generation and is weighted | ||||
| 604 | +1.0 for \texttt{llm}; a reader who consults the OpenAlex label alone would misjudge this choice. | ||||
| 605 | + | ||||
| 606 | +\paragraph{Agents, reinforcement learning and robotics.} | ||||
| 607 | +Software autonomy (LLM agents, multi-agent systems, planning) is \texttt{ai\_\allowbreak agents}; learning from reward | ||||
| 608 | +is \texttt{reinforcement\_\allowbreak learning}; physical embodiment is \texttt{robotics}. The word ``agent'' never | ||||
| 609 | +counts on its own: the patterns require a computing qualifier (``LLM agent'', ``autonomous agent'', | ||||
| 610 | +``multi-agent system'') and an exclusion vetoes chemical, medical, commercial and other non-computing | ||||
| 611 | +senses (reagent, contrast agent, real-estate agent). | ||||
| 612 | + | ||||
| 613 | +\paragraph{AI safety, cybersecurity and the statistical side of interpretability.} | ||||
| 614 | +Adversarial examples, backdoors and prompt injection score in both \texttt{ai\_\allowbreak safety} and | ||||
| 615 | +\texttt{cybersecurity} at 0.8, so that either a second signal or the text decides; fairness and governance | ||||
| 616 | +belong to \texttt{ai\_\allowbreak safety}; explainability as a statistical method is \texttt{ml\_\allowbreak theory} (0.5), whereas | ||||
| 617 | +interpretability of large models as a safety practice is \texttt{ai\_\allowbreak safety} (1.0). | ||||
| 618 | + | ||||
| 619 | +\paragraph{Cloud, distributed and edge computing.} | ||||
| 620 | +\texttt{distributed\_\allowbreak computing} is the parent (algorithms, HPC systems software, consensus); | ||||
| 621 | +\texttt{cloud\_\allowbreak computing} is the service model (virtualisation, serverless, data-centre resource management); | ||||
| 622 | +\texttt{edge\_\allowbreak computing} is placement near users and devices (offloading, multi-access edge computing, | ||||
| 623 | +on-device inference). Internet-of-things topics are shared between \texttt{edge\_\allowbreak computing} and | ||||
| 624 | +\texttt{computer\_\allowbreak networks}. | ||||
| 625 | + | ||||
| 626 | +\paragraph{Cryptography, security and privacy.} | ||||
| 627 | +Primitives and protocols are \texttt{applied\_\allowbreak cryptography}; attacks, defences and systems hardening are | ||||
| 628 | +\texttt{cybersecurity}; using cryptography or statistics to protect data is | ||||
| 629 | +\texttt{privacy\_\allowbreak preserving\_\allowbreak computation}, whose child \texttt{federated\_\allowbreak learning} has no OpenAlex topic | ||||
| 630 | +and no venue of its own and is carried by keywords. The topic T10764 (``Privacy-Preserving Technologies in | ||||
| 631 | +Data'') hosts both the parent (1.0) and the child (0.7). | ||||
| 632 | + | ||||
| 633 | +\paragraph{Architecture, accelerators and operating systems.} | ||||
| 634 | +Accelerators are a child of architecture and include efficient-inference \emph{systems} (serving engines, | ||||
| 635 | +attention kernels, quantisation for hardware); operating systems have no OpenAlex topic and are carried by | ||||
| 636 | +the OSDI, SOSP, EuroSys, USENIX ATC and FAST streams, by \texttt{cs.OS} and by kernel and file-system keywords. | ||||
| 637 | + | ||||
| 638 | +\paragraph{Quantum computing and quantum physics.} | ||||
| 639 | +Only computing and information-processing framings count. Physics topics on qubit platforms are weighted | ||||
| 640 | +0.2--0.3, and an anchored exclusion removes texts about quantum dots, quantum chemistry, quantum field theory | ||||
| 641 | +and similar physical senses whenever no computing term (qubit, quantum algorithm, error correction, QKD, | ||||
| 642 | +NISQ, \ldots) appears anywhere in the title or abstract. | ||||
| 643 | + | ||||
| 644 | +\paragraph{Scientific computing, computational biology and autonomous discovery.} | ||||
| 645 | +A numerical method is \texttt{scientific\_\allowbreak computing}; a biological object is \texttt{computational\_\allowbreak biology}; | ||||
| 646 | +AI that drives the scientific process itself (AI-scientist systems, self-driving laboratories, automated | ||||
| 647 | +hypothesis generation, AI-discovered algorithms or theorems) is \texttt{autonomous\_\allowbreak scientific\_\allowbreak discovery}, | ||||
| 648 | +which is keyword-driven and vetoed when no AI or automation term is present. | ||||
| 649 | + | ||||
| 650 | +\paragraph{Algorithms and complexity.} | ||||
| 651 | +STOC, FOCS and ICALP are shared; ECCC, \emph{ToCT} and \texttt{cs.CC} are complexity-only. Metaheuristics and | ||||
| 652 | +evolutionary computation are counted as \texttt{algorithms} at 0.5--0.6 because the corpus is expected to | ||||
| 653 | +contain many such works and no other leaf claims them. | ||||
| 654 | + | ||||
| 655 | +\paragraph{Education, media and other field-17 residues.} | ||||
| 656 | +As documented in Section~\ref{app:taxonomy-unmapped}, the educational, media, physical-science and generic | ||||
| 657 | +clusters that OpenAlex places in field~17 are left unmapped, with four exceptions mapped at low weight; this | ||||
| 658 | +is a deliberate narrowing of ``computer science'' relative to the OpenAlex field, and its effect on corpus | ||||
| 659 | +scope is examined in Part~III, where scope is defined by DBLP membership and OpenAlex primary field rather | ||||
| 660 | +than by the ontology. | ||||
| 661 | + | ||||
| 662 | +\subsection{Limitations of the mapping and the validation design} | ||||
| 663 | +\label{app:taxonomy-limits} | ||||
| 664 | + | ||||
| 665 | +The weights are expert priors set by reading keyword lists and venue scopes, not parameters fitted to | ||||
| 666 | +labelled data, and the ontology's own documentation says so. Three limitations follow and are repeated here | ||||
| 667 | +because they bound what the classification can show. Venue weights are per DBLP stream, so a workshop sharing | ||||
| 668 | +a stream key with its main conference inherits the main conference's weight. OpenAlex keyword lists were read | ||||
| 669 | +in full for the 302 field-17 topics and for about 120 CS-adjacent topics named in the area modules, but not for | ||||
| 670 | +the remaining roughly 4{,}000 topics; relevant clusters from Medicine or Materials Science may be missing and | ||||
| 671 | +are to be added when the corpus reveals them, through an amendment. And keyword patterns on abstracts fire on | ||||
| 672 | +mentions as well as on contributions, a failure mode discussed with the others in | ||||
| 673 | +Section~\ref{sec:m-classification}. | ||||
| 674 | + | ||||
| 675 | +The ontology was validated, after the classification run, on a stratified sample whose design is fixed in | ||||
| 676 | +\texttt{taxonomy/VALIDATION.md}: fifty eligible works with an abstract per rule-assigned primary category | ||||
| 677 | +(forty strata) plus two hundred works to which the rules assigned no category, drawn with seed~44; two | ||||
| 678 | +independent LLM raters that never see the rule-based assignment and receive only the definitions of | ||||
| 679 | +Table~\ref{tab:appA-leaves} with the title, abstract, venue and year of each work; the principal investigator | ||||
| 680 | +as third rater on a 200-work subsample (checkpoint H-2). The pre-specified decision rule is that a category | ||||
| 681 | +with consensus $F_1 < 0.60$ triggers a mapping revision, logged as a protocol amendment, and a re-run of the | ||||
| 682 | +classification. The agreement statistics, the per-category precision and recall, the list of categories that | ||||
| 683 | +met or failed the decision rule, and any amendment that resulted are reported in Part~IX; nothing about them | ||||
| 684 | +is asserted in this appendix. | ||||
modified
manuscript/appendices/appB_search_and_sources.tex
+613 −1
@@ -1 +1,613 @@ | |||||
| 1 | −% appB_search_and_sources — to be written (Phase 7) | 1 | +% ============================================================================ | ||
| 2 | +% Appendix B — Search strategies, sources and acquisition record | ||||
| 3 | +% Written by Agent L (Appendices B/C instance), 2026-11-03. Operational record | ||||
| 4 | +% of what ingestion/*.py, systematic_review/define_corpus.py and the runbook | ||||
| 5 | +% actually did; Part III gives the rationale and must not be repeated here. | ||||
| 6 | +% Corpus numbers only through macros of tables/numbers.tex; sizes, hashes, | ||||
| 7 | +% dates, file counts and parameter values are provenance identifiers and are | ||||
| 8 | +% typed from data/raw/README.md, reports/PHASE0_FEASIBILITY.md and the ledgers. | ||||
| 9 | +% ============================================================================ | ||||
| 10 | + | ||||
| 11 | +\section{Search strategies, sources and acquisition record} | ||||
| 12 | +\label{sec:appB} | ||||
| 13 | + | ||||
| 14 | +This appendix is the operational counterpart of Part~III\@. Part~III explains | ||||
| 15 | +why four bibliographic sources were combined and what each contributes; the | ||||
| 16 | +present appendix records, source by source, the endpoint or bucket that was | ||||
| 17 | +read, the release or version that was read, the filters that were applied, | ||||
| 18 | +the rate limits and budgets under which the reading took place, the licence | ||||
| 19 | +of the material, the integrity checks performed on arrival, and what was | ||||
| 20 | +sought but not obtained. It then gives the reporting items of the PRISMA-S | ||||
| 21 | +extension for literature searches \citep{rethlefsen2021prismas} in the form | ||||
| 22 | +they take for a scope-filtered mapping study, the exclusion codes of the | ||||
| 23 | +eligibility stage with their predicates, the type and platform lists on which | ||||
| 24 | +those predicates depend, and the record-linkage cascade with its thresholds. | ||||
| 25 | +Every statement below can be checked against the scripts in | ||||
| 26 | +\texttt{ingestion/} and \texttt{systematic\_review/}, the runbook | ||||
| 27 | +(\texttt{reproducibility/RUNBOOK.md}) and the provenance note | ||||
| 28 | +\texttt{data/raw/README.md}; where a count is a property of the corpus it is | ||||
| 29 | +rendered by a macro of \texttt{tables/numbers.tex}, and where no macro exists | ||||
| 30 | +the value is marked pending rather than typed. | ||||
| 31 | + | ||||
| 32 | +\subsection{Acquisition at a glance} | ||||
| 33 | +\label{sec:appB-glance} | ||||
| 34 | + | ||||
| 35 | +Table~\ref{tab:appB-acq} lists the five acquisitions in the order of the | ||||
| 36 | +runbook. All were performed from the compute node \texttt{M3U96b} between the | ||||
| 37 | +evening of 2~November and 3~November~2026 (Eastern Time), with no credential | ||||
| 38 | +of any kind: the OpenAlex key, the Semantic Scholar key and the Kaggle | ||||
| 39 | +credentials were tested as absent before the first download | ||||
| 40 | +(\texttt{reports/PHASE0\_FEASIBILITY.md}, §2). The absence of credentials is | ||||
| 41 | +not incidental. It determined the choice of the snapshot over the API for | ||||
| 42 | +OpenAlex (decision D1 of the plan), it excluded Semantic Scholar (D3), and it | ||||
| 43 | +capped the post-snapshot increment at the public daily budget | ||||
| 44 | +(Section~\ref{sec:appB-increment}). | ||||
| 45 | + | ||||
| 46 | +\begin{table}[htbp] | ||||
| 47 | +\centering\footnotesize | ||||
| 48 | +\caption{Acquisitions performed for the WP44 corpus, in runbook order.} | ||||
| 49 | +\label{tab:appB-acq} | ||||
| 50 | +\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{1.8cm}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{2.2cm}>{\raggedright\arraybackslash}p{4.0cm}} | ||||
| 51 | +\toprule | ||||
| 52 | +Source & Object read & Obtained & Integrity record \\ | ||||
| 53 | +\midrule | ||||
| 54 | +OpenAlex works & public snapshot, release 2026-10-14, \texttt{data/jsonl/works/}, 1{,}898 part files, 675{,}298{,}563{,}638 bytes gzip & 2026-11-02 21:19 to 23:05 ET, HTTPS, 8 parallel streams & manifest copy; content length and SHA-256 per file in \texttt{works/ledger.sqlite}; 1{,}898/1{,}898 done, 0 malformed lines; record total equal to the manifest total \\ | ||||
| 55 | +OpenAlex entities & snapshots of topics, subfields, fields, domains, sources, institutions, publishers, funders, concepts & 2026-11-02 & \texttt{entities\_ledger.json}: content length and SHA-256 per file \\ | ||||
| 56 | +OpenAlex API & works with any CS topic and publication date 2026-10-01 to 2026-11-02, by day and by primary field & 2026-11-03 onward, within the public budget & \texttt{api\_increment/}\allowbreak\texttt{ledger.json}: calls, cost, records and pages per slice; raw pages kept verbatim \\ | ||||
| 57 | +DBLP & \texttt{dblp.xml.gz} dated 2026-11-02 22:39 UTC, 1{,}116{,}270{,}104 bytes, with \texttt{dblp.dtd} and \texttt{dblp.xml.gz.md5} & 2026-11-02 & published MD5 \texttt{ff1c424c3b4cef36}\allowbreak\texttt{582f647c36df27c0} verified on the whole file before parsing; parse report with size, MD5 of the first GiB, counts by type and year \\ | ||||
| 58 | +arXiv & OAI-PMH, set \texttt{cs}, metadata prefix \texttt{arXiv}, datestamps 2018-01-01 to 2026-11-02 & 2026-11-02 to 03 & \texttt{harvest\_state.json} (pages, records, seconds per window); 758 raw XML pages kept; \texttt{arxiv\_parse\_}\allowbreak\texttt{report.json} \\ | ||||
| 59 | +ACL Anthology & \texttt{anthology+abstracts.bib.gz}, 46{,}723{,}382 bytes & 2026-11-02 & SHA-256 \texttt{44c3f7e907f221b740}\allowbreak\texttt{bea09069818a3d7c0d}\allowbreak\texttt{fd88b2da3b750d9f02}\allowbreak\texttt{955efc58b3}; \texttt{acl\_parse\_}\allowbreak\texttt{report.json} \\ | ||||
| 60 | +\bottomrule | ||||
| 61 | +\end{tabularx} | ||||
| 62 | +\end{table} | ||||
| 63 | + | ||||
| 64 | +\subsection{OpenAlex works snapshot} | ||||
| 65 | +\label{sec:appB-openalex} | ||||
| 66 | + | ||||
| 67 | +\paragraph{Bucket, release and manifest.} | ||||
| 68 | +The works entity of the OpenAlex snapshot \citep{priem2022openalex} is | ||||
| 69 | +distributed as gzip-compressed JSON Lines files in the public S3 bucket | ||||
| 70 | +\texttt{s3://openalex/} under the prefix \texttt{data/jsonl/works/}, partitioned by the date at which the | ||||
| 71 | +records were last updated (\texttt{updated\_date=YYYY-MM-DD/}\allowbreak\texttt{part\_NNNN.gz}). | ||||
| 72 | +The release read is the one whose \texttt{manifest.json} carries the date | ||||
| 73 | +2026-10-14; the manifest lists 1{,}898 part files with, for each, its URL, | ||||
| 74 | +its content length in bytes and its record count, together with the totals | ||||
| 75 | +(\LinkSlimTotal{} works, 675{,}298{,}563{,}638 bytes). A copy of the manifest | ||||
| 76 | +is kept as \texttt{data/raw/openalex/works\_manifest\_2026-10-14.json} and is | ||||
| 77 | +the reference against which the ledger is checked. No AWS client was | ||||
| 78 | +installed on the node; the bucket is public and each object was fetched over | ||||
| 79 | +HTTPS from \url{https://openalex.s3.amazonaws.com/} with \texttt{curl} | ||||
| 80 | +(\texttt{-L --retry 5 --retry-delay 3 --retry-all-errors -C -}, the last | ||||
| 81 | +option resuming an interrupted transfer), up to five outer attempts per file | ||||
| 82 | +with a linearly increasing pause, and the transfer was accepted only when the | ||||
| 83 | +byte count on disk equalled the content length in the manifest. The measured | ||||
| 84 | +single-stream throughput from the node was about 71~MB/s and the aggregate | ||||
| 85 | +over eight streams about 110~MB/s, which fixed the worker count at eight and | ||||
| 86 | +the wall time at one hour and forty-six minutes for the whole bucket | ||||
| 87 | +(\texttt{ingestion/openalex\_snapshot.py --workers 8}). | ||||
| 88 | + | ||||
| 89 | +\paragraph{Filter applied on the stream (pass 1).} | ||||
| 90 | +Each part file was decompressed in memory and parsed line by line | ||||
| 91 | +(\texttt{orjson}); a line that failed to parse was counted as malformed and | ||||
| 92 | +skipped, and the count of such lines (zero for every file) is in the ledger. | ||||
| 93 | +For every parsed work $w$ the script evaluated the predicate | ||||
| 94 | +\begin{equation} | ||||
| 95 | + \mathrm{CS}(w) \;=\; \Bigl[\exists\, \theta \in \mathrm{topics}(w) : | ||||
| 96 | + \mathrm{field}(\theta) = \text{\texttt{fields/17}}\Bigr] | ||||
| 97 | + \;\vee\; \Bigl[\mathrm{field}\bigl(\mathrm{primary\_topic}(w)\bigr) = \text{\texttt{fields/17}}\Bigr], | ||||
| 98 | + \label{eq:appB-csfilter} | ||||
| 99 | +\end{equation} | ||||
| 100 | +that is, whether any of the up to three topics attached to the record, or its | ||||
| 101 | +primary topic, belongs to the OpenAlex field with identifier~17 | ||||
| 102 | +(\emph{Computer Science}; \citealp{openalex2024topics}), and kept the | ||||
| 103 | +\emph{complete} JSON record when $\mathrm{CS}(w)$ held and | ||||
| 104 | +\texttt{publication\_year} lay in $[2018, 2026]$. Records flagged | ||||
| 105 | +\texttt{is\_xpac} (OpenAlex's expansion of coverage from non-Crossref | ||||
| 106 | +sources) were kept at this stage with their flag, so that the exclusion could | ||||
| 107 | +be applied later by the eligibility predicate rather than silently at | ||||
| 108 | +ingestion. The retained records were re-compressed with Zstandard (level~6) | ||||
| 109 | +into one file per part, \texttt{works/cs/}\allowbreak\texttt{updated\_date=X\_\_part\_NNNN.jsonl.zst}, | ||||
| 110 | +about 20~GB in total for | ||||
| 111 | +\PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{} | ||||
| 112 | +records. No text search of any kind was performed: the scope of the OpenAlex | ||||
| 113 | +layer is defined entirely by the topic classifier's field assignment, which is | ||||
| 114 | +the search strategy appropriate to a field-level mapping and whose | ||||
| 115 | +consequences are discussed in Part~III\@. | ||||
| 116 | + | ||||
| 117 | +\paragraph{Slim table of every work.} | ||||
| 118 | +In the same pass the script wrote, for \emph{every} work in the snapshot | ||||
| 119 | +regardless of field or year, one row of a slim Parquet table with the | ||||
| 120 | +following columns: OpenAlex identifier (as an integer), DOI (lower-cased, | ||||
| 121 | +prefix removed), publication year and date, type, \texttt{cited\_by\_count}, | ||||
| 122 | +primary topic, subfield, field and domain identifiers, the value of | ||||
| 123 | +$\mathrm{CS}(w)$, the XPAC, open-access and retraction flags, language, | ||||
| 124 | +primary source identifier, number of referenced works, number of authors, | ||||
| 125 | +whether an abstract is present, and FWCI\@. The 1{,}898 per-part files were | ||||
| 126 | +later concatenated into \texttt{compact/slim\_all\_works.parquet} | ||||
| 127 | +(\LinkSlimTotal{} rows). The slim table is what allows a reference from a | ||||
| 128 | +computer-science work to a work in another field to be typed and dated | ||||
| 129 | +(Part~IV), and it is also what made the second pass of | ||||
| 130 | +Section~\ref{sec:appB-pass2} possible without re-reading the bucket for a | ||||
| 131 | +search. | ||||
| 132 | + | ||||
| 133 | +\paragraph{Integrity and provenance.} | ||||
| 134 | +Before parsing, the SHA-256 digest of the gzip bytes of each part file was | ||||
| 135 | +computed and written, with the file's URL, content length, manifest record | ||||
| 136 | +count, parsed record count, retained record count, malformed-line count, | ||||
| 137 | +download and total times, attempt count and timestamps, to a SQLite ledger | ||||
| 138 | +(\texttt{works/ledger.sqlite}, table \texttt{files}); a second table | ||||
| 139 | +(\texttt{runs}) records the manifest date and totals, the worker count, the | ||||
| 140 | +host name and the command line of every invocation. The ledger makes the job | ||||
| 141 | +resumable (a file with status \texttt{done} is never re-read) and is the | ||||
| 142 | +provenance record of the raw layer. The raw \texttt{.gz} files were deleted | ||||
| 143 | +after a successful parse (decision D6 of the plan): the snapshot is a public, | ||||
| 144 | +versioned release, the manifest and the per-file digests permit an | ||||
| 145 | +independent party to re-download and verify any part, and keeping 675~GB of | ||||
| 146 | +compressed JSON on the node would have precluded the rest of the pipeline. | ||||
| 147 | +Three checks were made at the end of the pass and are recorded in | ||||
| 148 | +\texttt{data/raw/README.md}: all 1{,}898 files have status \texttt{done}; the | ||||
| 149 | +sum of parsed records over files equals the manifest total; and the sum of | ||||
| 150 | +malformed lines is zero. The OpenAlex snapshot is released under CC0~1.0 | ||||
| 151 | +(\texttt{LICENSE.txt} in the bucket, copied next to the manifest), which | ||||
| 152 | +permits redistribution of derived record-level data. | ||||
| 153 | + | ||||
| 154 | +\paragraph{Entity snapshots.} | ||||
| 155 | +The smaller entity snapshots were downloaded from the same bucket by | ||||
| 156 | +\texttt{ingestion/}\allowbreak\texttt{openalex\_entities.py}, using each | ||||
| 157 | +entity's own manifest, with the same length check and SHA-256 ledger | ||||
| 158 | +(\texttt{entities\_ledger.json}). The manifests list 4{,}516 topics, | ||||
| 159 | +252 subfields, 26 fields, 4 domains, 268{,}629 sources, 143{,}455 institutions, | ||||
| 160 | +10{,}713 publishers, 41{,}799 funders and 65{,}026 concepts; the topics | ||||
| 161 | +snapshot was flattened to \texttt{topics\_flat.csv} (identifier, name, | ||||
| 162 | +subfield, field, domain, works count, keywords, description) and is the | ||||
| 163 | +vocabulary over which the ontology of Appendix~A maps OpenAlex topics to | ||||
| 164 | +categories. The authors entity was not downloaded, because the authorship | ||||
| 165 | +blocks embedded in each work record already carry the author identifier, | ||||
| 166 | +display name, ORCID, position and institutions; the keywords entity | ||||
| 167 | +(1{,}881{,}723 entries in its manifest) was not downloaded either, the | ||||
| 168 | +per-work keyword lists being embedded as well. | ||||
| 169 | + | ||||
| 170 | +\subsection{Second snapshot pass by identifier set} | ||||
| 171 | +\label{sec:appB-pass2} | ||||
| 172 | + | ||||
| 173 | +The first pass retains what OpenAlex calls computer science. The record | ||||
| 174 | +linkage of Section~\ref{sec:appB-linkage} showed that | ||||
| 175 | +\LinkDblpUnlinkedWithDoiInSlim{} DBLP records of 2018--2026 have a DOI that | ||||
| 176 | +exists in the slim table of all works but not in the retained subset, that is, | ||||
| 177 | +works indexed by DBLP that OpenAlex labels with no computer-science topic. | ||||
| 178 | +Rather than accept OpenAlex's boundary, the pipeline reads the bucket a | ||||
| 179 | +second time in \emph{identifier-set mode}. The linkage script writes | ||||
| 180 | +the list \texttt{data/metadata/needed\_oa\_ids.csv} of OpenAlex identifiers to | ||||
| 181 | +recover (\LinkNeededOaIdsSecondPassDblpDoi{} reached through a DBLP DOI and | ||||
| 182 | +\LinkNeededOaIdsSecondPassArxivDoi{} through an arXiv DOI), and | ||||
| 183 | +\texttt{openalex\_snapshot.py --id-list} streams every part file again, | ||||
| 184 | +keeping exactly the records whose identifier is in the set and writing no slim | ||||
| 185 | +rows. The identifier set is loaded in each worker process (macOS uses the | ||||
| 186 | +\texttt{spawn} start method, so module globals are rebuilt per worker), the | ||||
| 187 | +same download, length and SHA-256 discipline applies, and the output lands in | ||||
| 188 | +\texttt{works\_pass2/cs/} with its own ledger. The recovered records are | ||||
| 189 | +normalised with the suffix \texttt{\_p2} and the linkage is rerun on the | ||||
| 190 | +union of both passes (\texttt{ingestion/run\_pass2\_chain.sh}). The second | ||||
| 191 | +pass costs the same bandwidth as the first---about one hour and fifty minutes | ||||
| 192 | +of streaming---because an identifier lookup cannot be pushed to the bucket; | ||||
| 193 | +the price was accepted because the alternative, fetching the identifiers one | ||||
| 194 | +by one through the API, would have exceeded the public budget by two orders of | ||||
| 195 | +magnitude. The PRISMA counts of Part~III were generated from the first pass | ||||
| 196 | +(\PrismaIdentifiedOpenalexTotalRecordsLoaded{} records loaded); the second | ||||
| 197 | +pass adds its records to $\mathcal{W}_0$ when it completes and the counts are | ||||
| 198 | +regenerated by the same scripts, so the identification total in the final | ||||
| 199 | +PRISMA figure is \pending{PrismaIdentifiedOpenalexTotalRecordsLoaded after the second pass} | ||||
| 200 | +rather than the first-pass value quoted here. | ||||
| 201 | + | ||||
| 202 | +\subsection{OpenAlex API increment} | ||||
| 203 | +\label{sec:appB-increment} | ||||
| 204 | + | ||||
| 205 | +The snapshot ends on 14~October 2026 and the observation window on | ||||
| 206 | +2~November 2026. The gap was covered by the REST API | ||||
| 207 | +(\texttt{https://api.openalex.org/works}) without a key. Two constraints of the | ||||
| 208 | +unkeyed API shaped the harvest and are recorded here because they bound what | ||||
| 209 | +the increment can contain. First, cursor paging requires a key, so the | ||||
| 210 | +harvest used basic page numbering, which the API caps at 10{,}000 results per | ||||
| 211 | +filter expression; the harvest is therefore \emph{sliced}, one request series | ||||
| 212 | +per publication day $d$ and per primary-field class, with the filter | ||||
| 213 | +\begin{quote}\ttfamily | ||||
| 214 | +topics.field.id:17, from\_publication\_date:$d$, to\_publication\_date:$d$, | ||||
| 215 | +primary\_topic.field.id:17 \quad (slice ``cs\_primary'')\\ | ||||
| 216 | +topics.field.id:17, from\_publication\_date:$d$, to\_publication\_date:$d$, | ||||
| 217 | +primary\_topic.field.id:!17 \quad (slice ``cs\_secondary'') | ||||
| 218 | +\end{quote} | ||||
| 219 | +for every day from 2026-10-01 to 2026-11-02 (the start was placed two weeks | ||||
| 220 | +before the snapshot date to absorb records that OpenAlex back-dates), 200 | ||||
| 221 | +results per page, a \texttt{select} list of the fields used by the | ||||
| 222 | +normaliser (identifiers, dates, type, language, locations, open access, | ||||
| 223 | +authorships, topics, keywords, referenced works, citation counts, abstract | ||||
| 224 | +inverted index, flags, FWCI, funders, indexed-in, full-text flag, APC), the | ||||
| 225 | +polite-pool \texttt{mailto} parameter and a descriptive user agent. A slice | ||||
| 226 | +whose reported \texttt{count} exceeds 10{,}000 is marked \texttt{truncated} in | ||||
| 227 | +the ledger; the two-way split by primary field was chosen because a single | ||||
| 228 | +day of computer-science output in 2026 can exceed the cap while each half | ||||
| 229 | +rarely does. Second, the unkeyed API is metered: list-and-filter calls are | ||||
| 230 | +charged at about \$0.0001 each against a shared budget of \$0.10 per day, | ||||
| 231 | +and the script stops at \$0.085 per day (a margin for the free single-record | ||||
| 232 | +lookups used for reference verification), sleeps 0.12~s between calls, and | ||||
| 233 | +halts on HTTP~429, 403 or any 5xx response so that the next day's run resumes | ||||
| 234 | +from the ledger. Every page is stored verbatim as | ||||
| 235 | +\texttt{api\_increment/pages/<day>\_<slice>/page\_NNNN.json}; the ledger | ||||
| 236 | +(\texttt{api\_increment/ledger.json}) records per slice the pages, records, | ||||
| 237 | +reported count, completion and truncation flags, and globally the number of | ||||
| 238 | +calls and the cost. At the time of writing the harvest had used the budget | ||||
| 239 | +of several consecutive days and was not complete; its totals | ||||
| 240 | +(\pending{IncrementApiCalls}, \pending{IncrementRecordsRetrieved}, | ||||
| 241 | +\pending{IncrementTruncatedSlices}) will be reported from the ledger when the | ||||
| 242 | +last slice closes. The conversion step (\texttt{ingest\_api\_increment.py}) | ||||
| 243 | +rewrites the pages into the same \texttt{.jsonl.zst} part format as the | ||||
| 244 | +snapshot filter (\texttt{works\_pass3/}), skipping any work whose identifier | ||||
| 245 | +is already in the pass-1 or pass-2 tables and writing an | ||||
| 246 | +\texttt{increment\_report.json} (pages, new works, works already present); the | ||||
| 247 | +number of works added by the increment is \pending{IncrementNewWorks}. Two | ||||
| 248 | +limitations follow from the design and are not recoverable from the ledger: a | ||||
| 249 | +truncated slice loses the records beyond the 10{,}000th in the API's default | ||||
| 250 | +ordering, and a record created after 2~November 2026 with a publication date | ||||
| 251 | +inside the window is absent whatever its date. Both affect only the last | ||||
| 252 | +weeks of a window that is already treated as right-censored (Part~III), so no | ||||
| 253 | +analysis depends on them. | ||||
| 254 | + | ||||
| 255 | +\subsection{DBLP} | ||||
| 256 | +\label{sec:appB-dblp} | ||||
| 257 | + | ||||
| 258 | +\paragraph{Dump and version.} | ||||
| 259 | +DBLP \citep{ley2002dblp, ley2009dblp} publishes its whole database as a | ||||
| 260 | +single XML file. The file read is \texttt{dblp.xml.gz} as served on | ||||
| 261 | +2~November 2026 (file timestamp 22:39 UTC), 1{,}116{,}270{,}104 bytes, together | ||||
| 262 | +with the document type definition \texttt{dblp.dtd} and the published | ||||
| 263 | +checksum file \texttt{dblp.xml.gz.md5}. The MD5 of the whole file, | ||||
| 264 | +\texttt{ff1c424c3b4cef36582f647c36df27c0}, was computed and compared with the | ||||
| 265 | +published value before parsing; the parse report additionally stores the file | ||||
| 266 | +size and an MD5 of the first gibibyte as a quick identity check for later | ||||
| 267 | +sessions. DBLP is released under the Open Data Commons Attribution licence | ||||
| 268 | +(ODC-BY~1.0), which permits the redistribution of derived data with | ||||
| 269 | +attribution. The download from \texttt{dblp.org} proceeded at about 12~MB/s, | ||||
| 270 | +the slowest link of the whole acquisition. | ||||
| 271 | + | ||||
| 272 | +\paragraph{Parsing rule.} | ||||
| 273 | +The DTD matters: DBLP titles use XML entities for accented characters, and a | ||||
| 274 | +parser that does not resolve them against \texttt{dblp.dtd} corrupts titles | ||||
| 275 | +and therefore the title-based linkage. The parser | ||||
| 276 | +(\texttt{ingestion/dblp\_parse.py}) is \texttt{lxml.etree.iterparse} over the | ||||
| 277 | +gzip stream with \texttt{load\_dtd=True}, \texttt{resolve\_entities=True} and | ||||
| 278 | +\texttt{huge\_tree=True}, run from the directory containing the DTD because | ||||
| 279 | +the dump references it relatively. It emits one row per record of the types | ||||
| 280 | +\textit{article}, \textit{inproceedings}, \textit{proceedings}, \textit{book}, | ||||
| 281 | +\textit{incollection}, \textit{phdthesis}, \textit{mastersthesis} and | ||||
| 282 | +\textit{data}; person records (\texttt{www}) are consumed and discarded. For | ||||
| 283 | +each record it keeps the key, type, modification date, publication-type | ||||
| 284 | +attribute, title (concatenated text of the title element, so that embedded | ||||
| 285 | +markup such as subscripts is flattened), year, author names and DBLP person | ||||
| 286 | +identifiers, editors, journal, booktitle, volume, number, pages, publisher, | ||||
| 287 | +series, the DOI (taken from the first electronic-edition URL beginning with | ||||
| 288 | +\texttt{https://doi.org/}, lower-cased), the full list of electronic-edition | ||||
| 289 | +URLs, the URL, the cross-reference to the proceedings record, and the | ||||
| 290 | +\emph{stream}, the part of the key before its last slash | ||||
| 291 | +(\texttt{journals/tocs}, \texttt{conf/nips}), which is DBLP's venue identifier | ||||
| 292 | +and the unit on which the ontology's venue signals are defined. All years are | ||||
| 293 | +parsed; counts by type and by year for the entire archive are written to | ||||
| 294 | +\texttt{dblp\_parse\_report.json}, and the records of 2018--2026 | ||||
| 295 | +(\LinkDblpTwoZeroOneEightTwoZeroTwoSix{} after the type restriction of the | ||||
| 296 | +linkage step) are the ones that enter the corpus. The total number of | ||||
| 297 | +publication records in the dump is \pending{DblpParsedRecords}. A second | ||||
| 298 | +script (\texttt{dblp\_streams.py}) lists the streams with at least one hundred | ||||
| 299 | +records of type article or inproceedings in 2018--2026, with their first and | ||||
| 300 | +last years and a sample journal or booktitle name; this list | ||||
| 301 | +(\texttt{taxonomy/dblp\_streams\_2018\_2026.csv}) is the input of the | ||||
| 302 | +venue-to-category mapping and is not itself a corpus filter. | ||||
| 303 | + | ||||
| 304 | +\subsection{arXiv} | ||||
| 305 | +\label{sec:appB-arxiv} | ||||
| 306 | + | ||||
| 307 | +\paragraph{Protocol and windows.} | ||||
| 308 | +arXiv \citep{ginsparg2011arxiv} metadata were harvested through its OAI-PMH | ||||
| 309 | +endpoint \texttt{https://oaipmh.arxiv.org/oai} with the verb | ||||
| 310 | +\texttt{ListRecords}, the set \texttt{cs} and the metadata prefix | ||||
| 311 | +\texttt{arXiv}, which returns arXiv's own schema (identifier, dates, title, | ||||
| 312 | +abstract, authors with forenames and keynames, categories, comments, journal | ||||
| 313 | +reference, DOI, licence, report number, MSC and ACM classes). The harvest was | ||||
| 314 | +sliced into calendar-month windows on the OAI \texttt{from}/\texttt{until} | ||||
| 315 | +parameters from 2018-01-01 to 2026-11-02; each window was walked with the | ||||
| 316 | +resumption token until exhausted, and every response page was written | ||||
| 317 | +verbatim to \texttt{raw/<window>/page\_NNNN.xml} before being parsed, so that | ||||
| 318 | +the Parquet table can be rebuilt from the raw pages | ||||
| 319 | +(\texttt{--parse-only}) without touching the network. The harvester waited | ||||
| 320 | +3.5~s between requests, honoured the \texttt{Retry-After} header on HTTP~503 | ||||
| 321 | +(arXiv's flow-control response), waited thirty seconds on other errors, and | ||||
| 322 | +gave up a window after twelve consecutive failures (none occurred). The | ||||
| 323 | +state file \texttt{harvest\_state.json} records, per window, the number of | ||||
| 324 | +pages, of records and of seconds, and the harvest of the 107 windows produced | ||||
| 325 | +758 pages in about one hour. | ||||
| 326 | + | ||||
| 327 | +\paragraph{Datestamp semantics and deduplication.} | ||||
| 328 | +A point about OAI-PMH that affects the counts must be stated. The | ||||
| 329 | +\texttt{from}/\texttt{until} parameters select on the record's | ||||
| 330 | +\emph{datestamp}, which is the date of its last modification, not on its | ||||
| 331 | +submission date. A monthly window therefore contains every record touched in | ||||
| 332 | +that month, including old e-prints whose metadata were edited, and a record | ||||
| 333 | +edited in two months appears in two windows. The parser keeps, for each arXiv | ||||
| 334 | +identifier, the occurrence with the latest datestamp, skips records whose | ||||
| 335 | +header carries \texttt{status="deleted"}, normalises whitespace in titles and | ||||
| 336 | +abstracts, lower-cases the DOI, takes the first listed category as the | ||||
| 337 | +primary category, and records the set specifications. The result is | ||||
| 338 | +\LinkArxivCsRecords{} distinct records, of which | ||||
| 339 | +\LinkArxivCreatedTwoZeroOneEightTwoZeroTwoSix{} have a \texttt{created} date | ||||
| 340 | +(first submission) in 2018--2026; the difference consists of older e-prints | ||||
| 341 | +modified during the window, which are kept in the table for linkage but do | ||||
| 342 | +not belong to the observation window. The \texttt{created} date is the one | ||||
| 343 | +used as $d_{\mathrm{arx}}(w)$ in Part~IV\@. arXiv's metadata are offered for | ||||
| 344 | +reuse under its metadata terms; the e-prints themselves were not downloaded | ||||
| 345 | +in the acquisition described here. | ||||
| 346 | + | ||||
| 347 | +\subsection{ACL Anthology} | ||||
| 348 | +\label{sec:appB-acl} | ||||
| 349 | + | ||||
| 350 | +The ACL Anthology \citep{radev2013aclanthology, acl2024anthology} | ||||
| 351 | +distributes its complete bibliography as a BibTeX export. The variant with | ||||
| 352 | +abstracts (\texttt{anthology+}\allowbreak\texttt{abstracts.bib.gz}) was downloaded on | ||||
| 353 | +2~November 2026 (46{,}723{,}382 bytes; SHA-256 in | ||||
| 354 | +Table~\ref{tab:appB-acq}). BibTeX is not a regular language, so the parser | ||||
| 355 | +(\texttt{ingestion/acl\_parse.py}) walks the text by brace depth: it locates | ||||
| 356 | +each \texttt{@type\{} header, finds the matching closing brace, splits the | ||||
| 357 | +body at its first comma into the citation key (which is the Anthology | ||||
| 358 | +identifier, for instance \texttt{P19-1001} or \texttt{2023.acl-long.1}) and | ||||
| 359 | +the field list, and reads each field as a braced, quoted or bare value with | ||||
| 360 | +nested braces allowed. Fifteen fields are kept (title, author, editor, | ||||
| 361 | +booktitle, journal, month, year, address, publisher, URL, DOI, pages, | ||||
| 362 | +abstract, volume, number); braces inside values are removed and whitespace | ||||
| 363 | +normalised. Entries of type \texttt{comment}, \texttt{string} and | ||||
| 364 | +\texttt{preamble} are skipped. The export is released under CC~BY~4.0. The | ||||
| 365 | +number of entries parsed is \pending{AclParsedEntries}; the linkage step uses | ||||
| 366 | +only the \textit{inproceedings} and \textit{article} entries | ||||
| 367 | +(\LinkAclRecords{}), of which \LinkAclTwoZeroOneEightTwoZeroTwoSix{} fall in | ||||
| 368 | +2018--2026. The Anthology's role is a coverage check for natural-language | ||||
| 369 | +processing venues rather than a corpus source in its own right; an ACL entry | ||||
| 370 | +with no OpenAlex counterpart is counted in the linkage report but is never a | ||||
| 371 | +work of the corpus. | ||||
| 372 | + | ||||
| 373 | +\subsection{What was sought and not obtained} | ||||
| 374 | +\label{sec:appB-notobtained} | ||||
| 375 | + | ||||
| 376 | +The mandate lists sources beyond the four used, and the integrity rules | ||||
| 377 | +require that inaccessible data be recorded rather than worked around. The | ||||
| 378 | +record is as follows. | ||||
| 379 | + | ||||
| 380 | +\begin{description} | ||||
| 381 | + \item[Semantic Scholar Academic Graph.] An anonymous request to the | ||||
| 382 | + endpoint \texttt{graph/v1/paper/} for \texttt{arXiv:2303.08774} returned HTTP~429 with the | ||||
| 383 | + instruction to apply for a key, and the datasets endpoint requires a key | ||||
| 384 | + outright \citep{kinney2023s2ag}. No key was requested because the project | ||||
| 385 | + rule is not to assume credentials and because applying would have made the | ||||
| 386 | + acquisition depend on an approval outside the project's control. The | ||||
| 387 | + consequence is the absence of the influential-citation and citation-intent | ||||
| 388 | + signals from every analysis, stated wherever it matters. | ||||
| 389 | + \item[OpenAlex API key.] A free key raises the daily budget from \$0.10 to | ||||
| 390 | + \$1 and enables cursor paging. Obtaining it requires the principal | ||||
| 391 | + investigator to confirm an e-mail; this was not done without instruction, | ||||
| 392 | + and the increment of Section~\ref{sec:appB-increment} was run under the | ||||
| 393 | + unkeyed budget. The snapshot route does not depend on the key. | ||||
| 394 | + \item[Full texts.] No full text was downloaded during corpus construction. | ||||
| 395 | + Abstracts come from the OpenAlex inverted index (and from arXiv and the | ||||
| 396 | + Anthology for their records). The plan reserves lawful full-text access | ||||
| 397 | + (arXiv e-prints at one request per three seconds, open-access locations) | ||||
| 398 | + for at most 10{,}000 high-priority works in a later phase; the | ||||
| 399 | + knowledge-graph extraction of Part~IV is abstract-based and labelled as | ||||
| 400 | + such. | ||||
| 401 | + \item[Crossref.] The REST API was reached (\texttt{works?filter=}\allowbreak\texttt{from-pub-date:2022-11-30\&rows=0} | ||||
| 402 | + returned a count for all fields) and is used for the verification of every | ||||
| 403 | + DOI cited in the manuscript and for sample cross-checks of types and | ||||
| 404 | + citation counts; it was not used as a corpus source, OpenAlex already | ||||
| 405 | + carrying its metadata. | ||||
| 406 | + \item[Other sources listed in the mandate.] Kaggle arXiv snapshot, | ||||
| 407 | + Microsoft Academic Graph, Papers with Code, institutional repositories: no Kaggle credentials were configured and | ||||
| 408 | + OAI-PMH made the snapshot unnecessary; the Microsoft Academic Graph was | ||||
| 409 | + discontinued in 2021; benchmark and code repositories are scoped to the | ||||
| 410 | + reproducibility analyses of a later phase and are not bibliographic | ||||
| 411 | + sources; institutional repositories enter only through OpenAlex's | ||||
| 412 | + indexing of them, which the generic-deposit rule of | ||||
| 413 | + Section~\ref{sec:appB-exclusions} then restricts. | ||||
| 414 | +\end{description} | ||||
| 415 | + | ||||
| 416 | +\subsection{PRISMA-S search record} | ||||
| 417 | +\label{sec:appB-prismas} | ||||
| 418 | + | ||||
| 419 | +PRISMA-S \citep{rethlefsen2021prismas} is written for keyword searches of | ||||
| 420 | +bibliographic databases. A scope-filtered mapping has no search string, but | ||||
| 421 | +its sixteen items can still be answered, and answering them makes explicit | ||||
| 422 | +which items have no counterpart. Table~\ref{tab:appB-prismas} gives the | ||||
| 423 | +record; the PRISMA~2020 flow itself \citep{page2021prisma, haddaway2022prismaflow} | ||||
| 424 | +is Figure~\ref{fig:prisma} and Table~\ref{tab:prisma} of Part~III\@. | ||||
| 425 | + | ||||
| 426 | +\begin{table}[htbp] | ||||
| 427 | +\centering\footnotesize | ||||
| 428 | +\caption{PRISMA-S items answered for the WP44 corpus.} | ||||
| 429 | +\label{tab:appB-prismas} | ||||
| 430 | +\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{0.5cm}>{\raggedright\arraybackslash}p{3.2cm}>{\raggedright\arraybackslash}X} | ||||
| 431 | +\toprule | ||||
| 432 | +No. & Item & This study \\ | ||||
| 433 | +\midrule | ||||
| 434 | +1 & Database names & OpenAlex (works and entity snapshots), DBLP, arXiv, ACL Anthology; Crossref for verification only. \\ | ||||
| 435 | +2 & Multi-database searching & Each source read in full under its own scope filter; no interface or vendor platform (direct bucket, dump and OAI access). \\ | ||||
| 436 | +3 & Study registries & Not applicable. \\ | ||||
| 437 | +4 & Online resources and browsing & None. \\ | ||||
| 438 | +5 & Citation searching & None for identification; OpenAlex reference lists are analysed but do not add records to the corpus. \\ | ||||
| 439 | +6 & Contacts & None. \\ | ||||
| 440 | +7 & Other methods & None. \\ | ||||
| 441 | +8 & Full search strategies & OpenAlex: predicate~\eqref{eq:appB-csfilter} with publication year 2018--2026, then identifier-set recovery of DBLP- and arXiv-linked works (Section~\ref{sec:appB-pass2}), then the API filter of Section~\ref{sec:appB-increment}. DBLP: whole dump, record types listed in Section~\ref{sec:appB-dblp}, years 2018--2026 at linkage. arXiv: OAI set \texttt{cs}, datestamps 2018-01-01 to 2026-11-02, \texttt{created} in 2018--2026 at linkage. ACL: whole export, types inproceedings and article, years 2018--2026 at linkage. \\ | ||||
| 442 | +9 & Limits and restrictions & Year window only; no language, type or country restriction at search time (types are an eligibility criterion, Section~\ref{sec:appB-exclusions}). \\ | ||||
| 443 | +10 & Search filters & None beyond the scope predicates; the OpenAlex topic classifier is the instrument that defines ``computer science'' on that layer and its properties are discussed in Part~III\@. \\ | ||||
| 444 | +11 & Prior work & No search strategy was reused from a prior review. \\ | ||||
| 445 | +12 & Updates & One update by API increment to 2026-11-02; no further update is planned for this version. \\ | ||||
| 446 | +13 & Dates of searches & OpenAlex snapshot released 2026-10-14, read 2026-11-02; DBLP dump 2026-11-02; arXiv harvest 2026-11-02 to 03; ACL export 2026-11-02; API increment from 2026-11-03. \\ | ||||
| 447 | +14 & Peer review of the search & No librarian review; the strategy is preregistered in \texttt{protocol/PROTOCOL.md} (P3) and audited by the independent agents of Phase~8. \\ | ||||
| 448 | +15 & Total records & OpenAlex \PrismaIdentifiedOpenalexCsTopicTwoZeroOneEightTwoZeroTwoSixPassOne{} (pass 1); DBLP \PrismaIdentifiedDblpTwoZeroOneEightTwoZeroTwoSix{}; arXiv \PrismaIdentifiedArxivCsTwoZeroOneEightTwoZeroTwoSix{}; ACL \PrismaIdentifiedAclTwoZeroOneEightTwoZeroTwoSix{} (all 2018--2026). \\ | ||||
| 449 | +16 & Deduplication & Deterministic record linkage across sources (Section~\ref{sec:appB-linkage}) and preprint--publication merging within OpenAlex (Part~III, Part~IV); OpenAlex works are the counting unit; \PrismaExcludedEFiveDuplicate{} preprint members of merged pairs are flagged E5. \\ | ||||
| 450 | +\bottomrule | ||||
| 451 | +\end{tabularx} | ||||
| 452 | +\end{table} | ||||
| 453 | + | ||||
| 454 | +\subsection{Exclusion codes and their predicates} | ||||
| 455 | +\label{sec:appB-exclusions} | ||||
| 456 | + | ||||
| 457 | +The eligibility stage (\texttt{systematic\_review/define\_corpus.py}) computes | ||||
| 458 | +seven exclusion flags and two membership flags as columns of the table | ||||
| 459 | +\texttt{corpus\_works}, each a SQL expression over the normalised OpenAlex | ||||
| 460 | +record and the linkage tables. Table~\ref{tab:appB-codes} paraphrases the | ||||
| 461 | +predicates. Three lists on which they depend are reproduced verbatim from the | ||||
| 462 | +script, because their content is the substance of the type rule and of | ||||
| 463 | +amendment A-002. | ||||
| 464 | + | ||||
| 465 | +\begin{itemize} | ||||
| 466 | + \item $T_{\mathrm{keep}}$ (types kept without further condition): | ||||
| 467 | + \texttt{article}, \texttt{conference-}\allowbreak\texttt{paper}, \texttt{preprint}, | ||||
| 468 | + \texttt{book-}\allowbreak\texttt{chapter}, \texttt{book}, \texttt{dissertation}, | ||||
| 469 | + \texttt{report}, \texttt{review}, \texttt{software-}\allowbreak\texttt{paper}. | ||||
| 470 | + \item $T_{\mathrm{hard}}$ (types excluded even for DBLP-linked records): | ||||
| 471 | + \texttt{editorial}, \texttt{erratum}, \texttt{retraction}, | ||||
| 472 | + \texttt{peer-}\allowbreak\texttt{review}, \texttt{paratext}, \texttt{conference-}\allowbreak\texttt{abstract}, | ||||
| 473 | + \texttt{reference-}\allowbreak\texttt{entry}, \texttt{dataset}, \texttt{software}, | ||||
| 474 | + \texttt{supplementary-}\allowbreak\texttt{materials}, \texttt{libguides}, \texttt{letter}, | ||||
| 475 | + \texttt{grant}, \texttt{standard}. | ||||
| 476 | + \item $\mathcal{G}$ (generic deposit platforms, matched on the primary | ||||
| 477 | + source's display name): ``Zenodo (CERN European Organization for Nuclear | ||||
| 478 | + Research)'' and ``Zenodo''; ``Figshare''; ``Institutional Repositories | ||||
| 479 | + DataBase (IRDB)''; ``LA Referencia (Red Federada de Repositorios | ||||
| 480 | + Institucionales de Publicaciones Cient\'{\i}ficas)''; ``Dialnet (Universidad de | ||||
| 481 | + la Rioja)''; ``DOAJ (DOAJ: Directory of Open Access Journals)''; ``OSTI OAI | ||||
| 482 | + (U.S. Department of Energy Office of Scientific and Technical | ||||
| 483 | + Information)''. | ||||
| 484 | +\end{itemize} | ||||
| 485 | + | ||||
| 486 | +\begin{table}[htbp] | ||||
| 487 | +\centering\footnotesize | ||||
| 488 | +\caption{Exclusion and membership codes of \texttt{define\_corpus.py}, with their predicates paraphrased from SQL. $w$ is an OpenAlex record; \textit{in\_dblp}, \textit{in\_acl} and \textit{arxiv\_linked} are read from the link tables; $t_0$ = 2018-01-01, $t_1$ = 2026-11-02.} | ||||
| 489 | +\label{tab:appB-codes} | ||||
| 490 | +\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{0.9cm}>{\raggedright\arraybackslash}p{3.4cm}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{1.9cm}} | ||||
| 491 | +\toprule | ||||
| 492 | +Code & Column & Predicate (true = flagged) & Marginal count \\ | ||||
| 493 | +\midrule | ||||
| 494 | +E1 & \texttt{e1\_outside\_window} & \texttt{publication\_date} is null, or $< t_0$, or $> t_1$ & \PrismaExcludedEOneOutsideWindow{} \\ | ||||
| 495 | +E2 & \texttt{e2\_type} & $\tau(w) \notin T_{\mathrm{keep}}$ and not (\textit{in\_dblp} and $\tau(w) \notin T_{\mathrm{hard}}$); a null type counts as outside $T_{\mathrm{keep}}$ & \PrismaExcludedETwoType{} \\ | ||||
| 496 | +E3 & \texttt{e3\_xpac} & \texttt{is\_xpac} and not \textit{in\_dblp} & \PrismaExcludedEThreeXpac{} \\ | ||||
| 497 | +--- & \texttt{xpac\_kept\_dblp} & \texttt{is\_xpac} and \textit{in\_dblp} (kept, flagged) & \PrismaExcludedXpacKeptDblp{} \\ | ||||
| 498 | +E4 & \texttt{e4\_no\_title} & title null or empty after trimming & \PrismaExcludedEFourNoTitle{} \\ | ||||
| 499 | +E5 & \texttt{e5\_duplicate} & $w$ is the preprint member of a surviving preprint--published pair (Part~IV, eq.~\eqref{eq:duppair}) & \PrismaExcludedEFiveDuplicate{} \\ | ||||
| 500 | +E6 & \texttt{e6\_retracted} & \texttt{is\_retracted} (flag only; not part of eligibility) & \PrismaExcludedESixRetracted{} \\ | ||||
| 501 | +E7 & \texttt{e7\_generic\_deposit} & primary source name $\in \mathcal{G}$ and not \textit{in\_dblp} and no linked arXiv record & \pending{PrismaExcludedESevenGenericDeposit} \\ | ||||
| 502 | +core & \texttt{cs\_core} & \textit{in\_dblp} or \texttt{cs\_primary} or (\texttt{cs\_any} and linked arXiv primary category begins with \texttt{cs.}) & \PrismaCsCoreEligible{} eligible \\ | ||||
| 503 | +pen. & \texttt{cs\_penumbra} & \texttt{cs\_any} and not \texttt{cs\_core} & \PrismaCsPenumbraEligibleOtherwise{} otherwise eligible \\ | ||||
| 504 | +\midrule | ||||
| 505 | +--- & \texttt{eligible} & $\neg$E1 $\wedge$ $\neg$E2 $\wedge$ $\neg$E3 $\wedge$ $\neg$E4 $\wedge$ $\neg$E5 $\wedge$ $\neg$E7 $\wedge$ core & \PrismaCsCoreEligible{} \\ | ||||
| 506 | +\bottomrule | ||||
| 507 | +\end{tabularx} | ||||
| 508 | +\end{table} | ||||
| 509 | + | ||||
| 510 | +Three further columns are derived in the same statement and are used | ||||
| 511 | +throughout the manuscript. The first date, \texttt{first\_date}, is the least | ||||
| 512 | +of the publication date, the merged preprint's date and the arXiv creation | ||||
| 513 | +date, except that a publication date falling on 1~January is treated as | ||||
| 514 | +year-only precision and is overridden by either of the other two when | ||||
| 515 | +available; \texttt{date\_precision} is \texttt{year} when the date falls on | ||||
| 516 | +1~January and no finer date exists, \texttt{month} otherwise; and | ||||
| 517 | +\texttt{period} is \texttt{pre}, \texttt{baseline}, \texttt{post} or | ||||
| 518 | +\texttt{future} according to whether the first date precedes $t_0$, lies in | ||||
| 519 | +$[t_0, \text{2022-11-30})$, in $[\text{2022-11-30}, t_1]$, or after $t_1$. | ||||
| 520 | +The PRISMA counts are computed twice, marginally (the column of | ||||
| 521 | +Table~\ref{tab:appB-codes}) and sequentially in the order E1, E2, E3, E4, E5, | ||||
| 522 | +E7, and the script asserts that the loaded total equals the sum of the | ||||
| 523 | +sequential exclusions and the after-exclusion count | ||||
| 524 | +(\PrismaAfterExclusions{}); the macro set of Part~III reports both. The E6 and | ||||
| 525 | +penumbra rows are not exclusions and do not enter the sum. The numbering | ||||
| 526 | +conflict between the protocol's E7 (penumbra) and the code's E7 | ||||
| 527 | +(generic deposit) is resolved by amendment A-005 in favour of the code. | ||||
| 528 | + | ||||
| 529 | +\subsection{Record-linkage cascade and thresholds} | ||||
| 530 | +\label{sec:appB-linkage} | ||||
| 531 | + | ||||
| 532 | +Part~IV states the cascade in prose; Table~\ref{tab:appB-cascade} gives it as | ||||
| 533 | +executed, step by step, with every threshold. The normalised title | ||||
| 534 | +$\eta(\cdot)$ lower-cases and deletes every character outside $[a$--$z0$--$9]$; | ||||
| 535 | +``unique'' means that the normalised title occurs exactly once on that side | ||||
| 536 | +among the records still unlinked at that step (for OpenAlex in the arXiv and | ||||
| 537 | +ACL cascades, unique among all OpenAlex records, the exclusion of already | ||||
| 538 | +linked identifiers being applied to the pair instead); the length condition | ||||
| 539 | +is on the raw title for DBLP and OpenAlex and on the normalised title for | ||||
| 540 | +arXiv and ACL\@. The DBLP title step adds a first-author check that the other | ||||
| 541 | +cascades do not have: the last whitespace-separated token of the first | ||||
| 542 | +author's name on each side must coincide when both are present. Each link | ||||
| 543 | +carries its method label, which is the stratum of the precision sample of | ||||
| 544 | +Part~IV; the counts per method are in \texttt{data/metadata/linkage\_report.json} | ||||
| 545 | +and Table~\ref{tab:linkage}. | ||||
| 546 | + | ||||
| 547 | +\begin{table}[htbp] | ||||
| 548 | +\centering\footnotesize | ||||
| 549 | +\caption{The deterministic linkage cascade of \texttt{ingestion/link\_sources.py}. Steps within a cascade apply only to records unlinked by the previous steps.} | ||||
| 550 | +\label{tab:appB-cascade} | ||||
| 551 | +\begin{tabularx}{\textwidth}{>{\raggedright\arraybackslash}p{1.6cm}>{\raggedright\arraybackslash}p{1.4cm}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{1.8cm}} | ||||
| 552 | +\toprule | ||||
| 553 | +Cascade & Step & Join condition and thresholds & Method label \\ | ||||
| 554 | +\midrule | ||||
| 555 | +\multirow{3}{*}{DBLP--OA} | ||||
| 556 | + & 1 & equal lower-cased DOI; all matching pairs kept (many-to-many allowed) & \texttt{doi} \\ | ||||
| 557 | + & 2 & equal arXiv identifier (DBLP: parsed from \texttt{ee} URLs by \texttt{arxiv.org/abs/([\^{} ]+)}; both sides stripped of \texttt{v[0-9]+\$}); both records unlinked & \texttt{arxiv} \\ | ||||
| 558 | + & 3 & equal $\eta$, raw title $\ge 25$ characters, $\eta$ unique on both sides, $|y_{\mathrm{DBLP}} - y_{\mathrm{OA}}| \le 1$, first-author surnames equal when both present & \texttt{title\_year} \\ | ||||
| 559 | +\midrule | ||||
| 560 | +\multirow{3}{*}{arXiv--OA} | ||||
| 561 | + & 1 & equal arXiv identifier (OpenAlex side: \texttt{ids.arxiv}, else DOI prefix \texttt{10.48550/arxiv.}, else \texttt{arxiv.org/abs|pdf/} in any location URL); all pairs kept & \texttt{arxiv\_id} \\ | ||||
| 562 | + & 2 & equal lower-cased DOI; arXiv record unlinked & \texttt{doi} \\ | ||||
| 563 | + & 3 & equal $\eta$, $|\eta| \ge 25$, unique on both sides, $|y_{\mathrm{created}} - y_{\mathrm{OA}}| \le 2$, OpenAlex record unlinked & \texttt{title\_year} \\ | ||||
| 564 | +\midrule | ||||
| 565 | +\multirow{3}{*}{ACL--OA} | ||||
| 566 | + & 1 & equal lower-cased DOI & \texttt{doi} \\ | ||||
| 567 | + & 2 & Anthology identifier equal to the segment after \texttt{aclanthology.org/} in any OpenAlex landing-page URL (with or without trailing slash); ACL record unlinked & \texttt{url} \\ | ||||
| 568 | + & 3 & equal $\eta$, $|\eta| \ge 25$, unique on both sides, $|y_{\mathrm{ACL}} - y_{\mathrm{OA}}| \le 1$, OpenAlex record unlinked & \texttt{title\_year} \\ | ||||
| 569 | +\bottomrule | ||||
| 570 | +\end{tabularx} | ||||
| 571 | +\end{table} | ||||
| 572 | + | ||||
| 573 | +The record types admitted on the DBLP side are \textit{article}, | ||||
| 574 | +\textit{inproceedings}, \textit{incollection}, \textit{book}, | ||||
| 575 | +\textit{phdthesis} and \textit{data} with year 2018--2026; on the ACL side | ||||
| 576 | +\textit{inproceedings} and \textit{article}; on the arXiv side every record | ||||
| 577 | +of the harvest, the year condition being applied through the \texttt{created} | ||||
| 578 | +year in the title step only. The linkage runs inside one DuckDB database | ||||
| 579 | +with 24 threads and a 60~GB memory limit and completes in about ten seconds | ||||
| 580 | +on the compacted tables, which is why it was possible to rerun it after each | ||||
| 581 | +amendment rather than patch its output. The precision sample of 500 pairs | ||||
| 582 | +(250 DOI, 200 title--year, 50 arXiv, the last stratum empty for the reason | ||||
| 583 | +given in Part~IV) is drawn by \texttt{systematic\_review/linkage\_sample.py} | ||||
| 584 | +with seed 44, and its automatic agreement checks (title similarity at least | ||||
| 585 | +0.9, year difference at most one, first-author surname agreement) are the | ||||
| 586 | +interim figure until the principal investigator's review (checkpoint H-1). | ||||
| 587 | + | ||||
| 588 | +\subsection{Reproducing the acquisition} | ||||
| 589 | +\label{sec:appB-repro} | ||||
| 590 | + | ||||
| 591 | +The acquisition can be repeated from the runbook in six commands, in the | ||||
| 592 | +order of Table~\ref{tab:appB-acq}, on a machine with Python~3.12 and the | ||||
| 593 | +locked environment of \texttt{reproducibility/environment\_M3U96b.txt}: | ||||
| 594 | +\begin{enumerate} | ||||
| 595 | + \item \texttt{openalex\_snapshot.py} with the manifest copy (then once more with \texttt{--id-list}); | ||||
| 596 | + \item \texttt{openalex\_entities.py}; | ||||
| 597 | + \item \texttt{dblp\_parse.py}; | ||||
| 598 | + \item \texttt{arxiv\_oai.py}; | ||||
| 599 | + \item \texttt{acl\_parse.py}; | ||||
| 600 | + \item \texttt{openalex\_api\_increment.py}, followed by \texttt{ingest\_api\_increment.py}. | ||||
| 601 | +\end{enumerate} | ||||
| 602 | +Two of these are not exactly repeatable and the reader should know which. | ||||
| 603 | +The OpenAlex bucket retains only recent releases, so a later reader must | ||||
| 604 | +either locate the 2026-10-14 release or accept that a newer release will | ||||
| 605 | +differ in record counts and in the nightly-recomputed fields (citation | ||||
| 606 | +counts, FWCI); the per-file digests in the ledger allow a reader with the | ||||
| 607 | +same release to prove identity, and a reader with a different release to | ||||
| 608 | +quantify the difference by record count. The arXiv OAI feed is likewise | ||||
| 609 | +live: a re-harvest of the same datestamp windows returns records as they now | ||||
| 610 | +are, and the raw pages archived with the project are the only way to recover | ||||
| 611 | +the state of 2--3~November 2026. The DBLP dump and the ACL export are dated | ||||
| 612 | +files whose hashes are published here, and a copy of each is kept with the | ||||
| 613 | +project's raw layer. | ||||
modified
manuscript/appendices/appC_derivations.tex
+419 −1
@@ -1 +1,419 @@ | |||||
| 1 | −% appC_derivations — to be written (Phase 7) | 1 | +% ============================================================================ | ||
| 2 | +% Appendix C — Mathematical derivations | ||||
| 3 | +% Written by Agent L (Appendices B/C instance), 2026-11-03. Derivations and | ||||
| 4 | +% proofs referenced by Part IV (sections/part04_methodology.tex); notation | ||||
| 5 | +% follows Part IV. Each derivation is checked against the implementation in | ||||
| 6 | +% econometrics/structural_change.py, bibliometrics/03_citation_impact.py, | ||||
| 7 | +% bibliometrics/04_diversity_cagr.py, knowledge_graph/02_category_flows.py, | ||||
| 8 | +% taxonomy/classify.py, taxonomy/validate_sample.py and | ||||
| 9 | +% experiments/emerging_detection.py; where the code and the usual textbook | ||||
| 10 | +% convention differ, the code's convention is stated and the difference noted. | ||||
| 11 | +% ============================================================================ | ||||
| 12 | + | ||||
| 13 | +\section{Mathematical derivations} | ||||
| 14 | +\label{sec:appC} | ||||
| 15 | + | ||||
| 16 | +Part~IV defines the statistics of this study and states their properties; | ||||
| 17 | +this appendix supplies the derivations behind those statements, in the order | ||||
| 18 | +in which Part~IV uses them. Each subsection fixes the object as implemented, | ||||
| 19 | +derives the property that the main text relies on, and records any point at | ||||
| 20 | +which the implementation departs from the convention of the cited source. | ||||
| 21 | +The derivations are elementary and self-contained; their purpose is not | ||||
| 22 | +novelty but the assurance that the quantities tabulated in Part~IX are what | ||||
| 23 | +the text says they are. Throughout, $n$ is the length of a monthly series, | ||||
| 24 | +$y_t = \log(N_t + 1)$ its logarithm, and $\mathbf{1}[\cdot]$ the indicator | ||||
| 25 | +function. | ||||
| 26 | + | ||||
| 27 | +\subsection{The interrupted time series as a linear regression} | ||||
| 28 | +\label{sec:appC-its} | ||||
| 29 | + | ||||
| 30 | +\paragraph{Design matrix.} | ||||
| 31 | +Equation~\eqref{eq:its} of Part~IV is the linear model $y = X\beta + | ||||
| 32 | +\varepsilon$ with $y \in \R^n$ and $X \in \R^{n \times 15}$ whose columns | ||||
| 33 | +are, for $t = 0, \dots, n-1$ and event index $t_e$, | ||||
| 34 | +\begin{equation} | ||||
| 35 | + x_t \;=\; \bigl(1,\; t,\; \mathbf{1}[t \ge t_e],\; (t - t_e)\,\mathbf{1}[t \ge t_e],\; | ||||
| 36 | + \mathbf{1}[\mathrm{month}(t) = 2],\; \dots,\; \mathbf{1}[\mathrm{month}(t) = 12]\bigr)^{\top}, | ||||
| 37 | + \label{eq:appC-design} | ||||
| 38 | +\end{equation} | ||||
| 39 | +exactly the data frame built by \texttt{\_design()} in | ||||
| 40 | +\texttt{econometrics/structural\_change.py} (columns \texttt{const}, | ||||
| 41 | +\texttt{t}, \texttt{post}, \texttt{t\_post}, \texttt{m2}, \dots, | ||||
| 42 | +\texttt{m12}). The post-event trend is centred at $t_e$ rather than at zero | ||||
| 43 | +so that $\beta_2$ is the level change \emph{at} the event month and not the | ||||
| 44 | +extrapolated intercept difference at $t = 0$; with an uncentred post-trend | ||||
| 45 | +the two parametrisations are related by $\beta_2^{\mathrm{unc}} = \beta_2 - | ||||
| 46 | +\beta_3 t_e$, and only the centred one has the interpretation used in the | ||||
| 47 | +text. The eleven month dummies are linearly independent of the constant | ||||
| 48 | +because January is omitted; the full-column-rank condition requires at least | ||||
| 49 | +one observation in every month of the year, satisfied by any series of | ||||
| 50 | +twelve months or more, and at least one post-event month with $t > t_e$, | ||||
| 51 | +guaranteed by the driver's rule that twelve months follow every event. | ||||
| 52 | +The ordinary least-squares estimator is $\hat\beta = (X^{\top}X)^{-1} | ||||
| 53 | +X^{\top}y$ and the residuals $\hat\varepsilon = y - X\hat\beta$. | ||||
| 54 | + | ||||
| 55 | +\paragraph{The HAC covariance estimator.} | ||||
| 56 | +Under the assumption that $\varepsilon_t$ is a stationary, serially | ||||
| 57 | +correlated and possibly heteroskedastic sequence with | ||||
| 58 | +$\E[x_t \varepsilon_t] = 0$, the asymptotic covariance of $\sqrt{n}(\hat\beta | ||||
| 59 | +- \beta)$ is $Q^{-1}\Omega Q^{-1}$ with $Q = \lim n^{-1}X^{\top}X$ and | ||||
| 60 | +$\Omega = \sum_{\ell=-\infty}^{\infty} \Gamma_\ell$, where | ||||
| 61 | +$\Gamma_\ell = \E[x_t \varepsilon_t \varepsilon_{t-\ell} x_{t-\ell}^{\top}]$ | ||||
| 62 | +is the autocovariance of the score at lag $\ell$. The estimator of | ||||
| 63 | +\citet{newey1987hac} replaces the infinite sum by a kernel-weighted finite | ||||
| 64 | +one, | ||||
| 65 | +\begin{equation} | ||||
| 66 | + \hat\Omega_L \;=\; \hat\Gamma_0 + \sum_{\ell=1}^{L} w_\ell\,\bigl(\hat\Gamma_\ell + \hat\Gamma_\ell^{\top}\bigr), | ||||
| 67 | + \qquad | ||||
| 68 | + \hat\Gamma_\ell = \frac{1}{n}\sum_{t=\ell}^{n-1} \hat\varepsilon_t \hat\varepsilon_{t-\ell}\, x_t x_{t-\ell}^{\top}, | ||||
| 69 | + \qquad | ||||
| 70 | + w_\ell = 1 - \frac{\ell}{L+1}, | ||||
| 71 | + \label{eq:appC-nw} | ||||
| 72 | +\end{equation} | ||||
| 73 | +and the variance estimate is $\hat V = n\,(X^{\top}X)^{-1} \hat\Omega_L | ||||
| 74 | +(X^{\top}X)^{-1}$, which is the expression of Part~IV once the factors of | ||||
| 75 | +$n$ are absorbed into the un-normalised $\hat\Gamma_\ell$. The Bartlett | ||||
| 76 | +weights $w_\ell$ are what make $\hat\Omega_L$ positive semi-definite for every | ||||
| 77 | +sample: $\hat\Omega_L$ can be written as $\frac{1}{n(L+1)}\sum_{t} | ||||
| 78 | +\bigl(\sum_{s=t-L}^{t} \hat u_s\bigr)\bigl(\sum_{s=t-L}^{t} \hat u_s\bigr)^{\top}$ | ||||
| 79 | +up to end effects, with $\hat u_s = x_s \hat\varepsilon_s$, a sum of outer | ||||
| 80 | +products, whereas the truncated sum with $w_\ell \equiv 1$ is not guaranteed | ||||
| 81 | +to be positive semi-definite. The implementation calls | ||||
| 82 | +\texttt{statsmodels} with \texttt{cov\_type="HAC"} and \texttt{maxlags=6}, so | ||||
| 83 | +$L = 6$ and $w_\ell = 1 - \ell/7$ as stated in Part~IV, without the | ||||
| 84 | +small-sample degrees-of-freedom correction and with $p$-values from the | ||||
| 85 | +normal approximation to the $t$-ratio, which is the library's default for | ||||
| 86 | +robust covariances. With $n$ of the order of one hundred and fourteen | ||||
| 87 | +regressors, the difference between the normal and the $t_{n-15}$ reference | ||||
| 88 | +distribution is immaterial at the magnitudes of the $t$-statistics reported; | ||||
| 89 | +what matters is the placebo rank of the next paragraph, which does not | ||||
| 90 | +depend on any reference distribution. | ||||
| 91 | + | ||||
| 92 | +\paragraph{The placebo $p$-value as a rank statistic.} | ||||
| 93 | +Let $\mathcal{T}' = \{t' : 24 \le t' \le n - 13\}$ be the candidate event | ||||
| 94 | +indices (the implementation's \texttt{min\_pre = 24} and | ||||
| 95 | +\texttt{min\_post = 12}) and, for each $t' \in \mathcal{T}'$, let | ||||
| 96 | +$t^{(2)}_{t'} = \hat\beta_2(t') / \widehat{\mathrm{se}}(\hat\beta_2(t'))$ be | ||||
| 97 | +the HAC $t$-ratio of the level change when the model | ||||
| 98 | +\eqref{eq:appC-design} is fitted with $t_e = t'$. The placebo value of the | ||||
| 99 | +true event $t_e \in \mathcal{T}'$ is | ||||
| 100 | +\begin{equation} | ||||
| 101 | + p^{\mathrm{plac}} \;=\; \frac{1}{|\mathcal{T}'|}\sum_{t' \in \mathcal{T}'} | ||||
| 102 | + \mathbf{1}\bigl[\,|t^{(2)}_{t'}| \ge |t^{(2)}_{t_e}|\,\bigr] | ||||
| 103 | + \;=\; \frac{\mathrm{rank}\bigl(|t^{(2)}_{t_e}|\bigr)}{|\mathcal{T}'|}, | ||||
| 104 | +\end{equation} | ||||
| 105 | +where $\mathrm{rank}$ is the descending rank of the true event's absolute | ||||
| 106 | +$t$-ratio among the $|\mathcal{T}'|$ values, ties counted as not better | ||||
| 107 | +(\texttt{>=}). Three properties follow directly. Because $t_e \in | ||||
| 108 | +\mathcal{T}'$, the sum contains the term $t' = t_e$, so $p^{\mathrm{plac}} | ||||
| 109 | +\ge 1/|\mathcal{T}'|$, which for the default window ($n = 101$, hence | ||||
| 110 | +$|\mathcal{T}'| = 65$) is about $0.015$: a placebo rank cannot be smaller, | ||||
| 111 | +however large the true statistic, and a reported value of $1/|\mathcal{T}'|$ | ||||
| 112 | +means ``largest of all candidate dates'', not ``significant at that level''. | ||||
| 113 | +Because the statistic is a rank, it is invariant to any monotone | ||||
| 114 | +transformation of the $|t|$-ratios and in particular does not depend on the | ||||
| 115 | +HAC bandwidth through anything but the ordering it induces. And if the | ||||
| 116 | +candidate dates were exchangeable under a null hypothesis of no break | ||||
| 117 | +anywhere, $p^{\mathrm{plac}}$ would be a valid permutation-type $p$-value | ||||
| 118 | +with $P(p^{\mathrm{plac}} \le \alpha) \le \alpha$; they are not exchangeable, | ||||
| 119 | +since all $|\mathcal{T}'|$ regressions share the same $n$ observations and | ||||
| 120 | +adjacent candidates give nearly identical fits, so the value is reported as | ||||
| 121 | +a rank and read as such. The same construction gives the placebo rank of | ||||
| 122 | +the slope change from $t^{(3)}_{t'}$. The driver additionally computes the | ||||
| 123 | +rank restricted to the baseline candidates $\{t' < t_e\}$, which answers the | ||||
| 124 | +protocol's original question (is the reference date special relative to the | ||||
| 125 | +pre-period?) and is reported next to the full-range rank. | ||||
| 126 | + | ||||
| 127 | +\subsection{Bai--Perron multiple breaks by dynamic programming} | ||||
| 128 | +\label{sec:appC-bp} | ||||
| 129 | + | ||||
| 130 | +\paragraph{Segment costs and additivity.} | ||||
| 131 | +For a segment $[i, j)$ of the series with $j - i \ge h$, define | ||||
| 132 | +\begin{equation} | ||||
| 133 | + S(i, j) \;=\; \min_{a, b \in \R}\ \sum_{t=i}^{j-1} (y_t - a - b\,t)^2, | ||||
| 134 | + \label{eq:appC-segcost} | ||||
| 135 | +\end{equation} | ||||
| 136 | +the residual sum of squares of a linear trend fitted to the segment alone | ||||
| 137 | +(the implementation solves the two-parameter least-squares problem for each | ||||
| 138 | +segment with \texttt{numpy.linalg.lstsq}; with \texttt{trend=False} the | ||||
| 139 | +regressor $t$ is dropped and $S(i,j)$ is the centred sum of squares). The | ||||
| 140 | +model with $m$ breaks $0 < \tau_1 < \dots < \tau_m < n$ assigns its own | ||||
| 141 | +$(a, b)$ to each of the $m+1$ segments $[\tau_{k-1}, \tau_k)$ with | ||||
| 142 | +$\tau_0 = 0$ and $\tau_{m+1} = n$. Because the parameters of different | ||||
| 143 | +segments are distinct and each observation enters exactly one segment, the | ||||
| 144 | +residual sum of squares of the whole model separates, | ||||
| 145 | +\begin{equation} | ||||
| 146 | + \mathrm{SSR}(\tau_1, \dots, \tau_m) \;=\; \sum_{k=1}^{m+1} S(\tau_{k-1}, \tau_k), | ||||
| 147 | + \label{eq:appC-additive} | ||||
| 148 | +\end{equation} | ||||
| 149 | +and the global minimisation over $\tau$ decomposes: the optimal partition of | ||||
| 150 | +the first $j$ observations into $m+1$ segments contains, as its first $m$ | ||||
| 151 | +segments, an optimal partition of the first $\tau_m$ observations into $m$ | ||||
| 152 | +segments. This is the principle of optimality, and it is the only property of | ||||
| 153 | +the cost that the dynamic programme uses; it holds for any cost that is a | ||||
| 154 | +sum over segments of a function of the segment alone, which is why the same | ||||
| 155 | +recursion serves the piecewise-constant model of Section~\ref{sec:appC-pelt}. | ||||
| 156 | +It fails for a model with parameters shared across segments (a common slope, | ||||
| 157 | +or the seasonal dummies of the ITS), which is why the segmented regression | ||||
| 158 | +carries no seasonal terms and the series is seasonally adjusted beforehand | ||||
| 159 | +by subtracting the baseline-period month means | ||||
| 160 | +(\texttt{run\_structural\_change.py}). | ||||
| 161 | + | ||||
| 162 | +\paragraph{The recursion.} | ||||
| 163 | +Let $C_m(j)$ be the minimum of \eqref{eq:appC-additive} over partitions of | ||||
| 164 | +observations $0, \dots, j-1$ into $m+1$ admissible segments. Then | ||||
| 165 | +\begin{equation} | ||||
| 166 | + C_0(j) = S(0, j), \qquad | ||||
| 167 | + C_m(j) = \min_{\,mh \le i \le j-h} \bigl\{ C_{m-1}(i) + S(i, j) \bigr\}, | ||||
| 168 | + \qquad j \ge (m+1)h, | ||||
| 169 | + \label{eq:appC-dp} | ||||
| 170 | +\end{equation} | ||||
| 171 | +the lower limit of $i$ ensuring that $m$ segments of length at least $h$ | ||||
| 172 | +fit before the last break and the upper limit that the last segment has | ||||
| 173 | +length at least $h$. The argument minimiser $i^{\ast}_m(j)$ is stored, and | ||||
| 174 | +the breaks of the $m$-break solution for the full series are recovered | ||||
| 175 | +backward: $\tau_m = i^{\ast}_m(n)$, $\tau_{m-1} = i^{\ast}_{m-1}(\tau_m)$, | ||||
| 176 | +and so on, which is what the dictionary \texttt{B[m][j]} of the | ||||
| 177 | +implementation accumulates forward instead. This is the dynamic programme | ||||
| 178 | +that \citet{bai2003computation} propose for the global minimiser of | ||||
| 179 | +\citet{bai1998breaks}; the earlier literature used it for the | ||||
| 180 | +piecewise-constant case, and its correctness requires nothing beyond | ||||
| 181 | +\eqref{eq:appC-additive} and the induction on $m$: if $C_{m-1}(i)$ is the | ||||
| 182 | +true minimum for every $i$, then any partition of $[0, j)$ into $m+1$ | ||||
| 183 | +segments has a last segment $[i, j)$ and a prefix partition of cost at least | ||||
| 184 | +$C_{m-1}(i)$, so its cost is at least the right-hand side of | ||||
| 185 | +\eqref{eq:appC-dp}, which is attained. | ||||
| 186 | + | ||||
| 187 | +\paragraph{Complexity.} | ||||
| 188 | +The table of segment costs has one entry per admissible pair $(i, j)$, that | ||||
| 189 | +is $\binom{n-h+1}{2} + O(n)$ entries, of order $n^2/2$. The recursion | ||||
| 190 | +\eqref{eq:appC-dp} evaluates, for each $m \le M$ and each $j$, a minimum over | ||||
| 191 | +at most $n$ candidates, so its cost is $O(n^2 M)$ operations; the | ||||
| 192 | +back-tracking costs $O(M)$. The implementation fills the cost table by | ||||
| 193 | +solving each segment's least-squares problem independently, at a cost | ||||
| 194 | +proportional to the segment length, so the table costs $O(n^3)$ rather than | ||||
| 195 | +the $O(n^2)$ obtainable by updating running sums as in | ||||
| 196 | +\citet{bai2003computation}; for $n$ of the order of one hundred, as here, | ||||
| 197 | +the difference is a fraction of a second and the simpler code was preferred | ||||
| 198 | +for its transparency. Trimming enters only through the admissibility | ||||
| 199 | +condition $j - i \ge h$; with $h = \max\{8, \lfloor 0.15\,n \rfloor\}$ and | ||||
| 200 | +$M = 3$, the default window of $n = 101$ months gives $h = 15$ and a | ||||
| 201 | +feasible region $(m+1)h \le n$ for every $m \le 3$, so all four candidate | ||||
| 202 | +models exist. | ||||
| 203 | + | ||||
| 204 | +\paragraph{Information criteria as implemented.} | ||||
| 205 | +For $m = 0, \dots, M$ let $\mathrm{SSR}_m = C_m(n)$ and let $p_m = k(m+1) + | ||||
| 206 | +m$ be the parameter count, with $k = 2$ for the trend model ($k = 1$ for the | ||||
| 207 | +mean model): one intercept and one slope per segment, plus the $m$ break | ||||
| 208 | +positions, counted as parameters by convention. The two criteria computed by | ||||
| 209 | +the code are | ||||
| 210 | +\begin{align} | ||||
| 211 | + \mathrm{BIC}(m) &= n \log\frac{\mathrm{SSR}_m}{n} + p_m \log n, \label{eq:appC-bic}\\ | ||||
| 212 | + \mathrm{LWZ}(m) &= n \log\frac{\mathrm{SSR}_m}{n - p_m} + p_m\, c_0\, (\log n)^{2.1}, \qquad c_0 = 0.299, \label{eq:appC-lwz} | ||||
| 213 | +\end{align} | ||||
| 214 | +and the selected number of breaks is $\hat m = \arg\min_m$ of each. The BIC | ||||
| 215 | +form \eqref{eq:appC-bic} is the Gaussian log-likelihood, maximised over the | ||||
| 216 | +error variance, in which $-2\log\hat L = n\log(\mathrm{SSR}/n) + \text{const}$, | ||||
| 217 | +plus the Schwarz penalty $p \log n$ \citep{schwarz1978bic}; the two | ||||
| 218 | +differences from a textbook BIC are that the break positions are counted | ||||
| 219 | +among the parameters and that the constant and the segment variances are | ||||
| 220 | +treated as common, both as in \citet{bai2003computation}. The LWZ form | ||||
| 221 | +\eqref{eq:appC-lwz} is the modified Schwarz criterion of | ||||
| 222 | +\citet{liu1997segmented}, whose penalty grows as $(\log n)^{2+\delta_0}$ with | ||||
| 223 | +$\delta_0 = 0.1$ and whose variance estimate divides by $n - p_m$; the | ||||
| 224 | +constant $c_0 = 0.299$ is the value they recommend and the one used in the | ||||
| 225 | +simulations of \citet{bai2003computation}, which find that LWZ selects fewer | ||||
| 226 | +spurious breaks than BIC under serial correlation at the cost of some | ||||
| 227 | +power. The comparison across $m$ is legitimate because every $\mathrm{SSR}_m$ | ||||
| 228 | +is a global minimum over the same observations and the same admissible | ||||
| 229 | +region; a sequential procedure that fixed earlier breaks before adding the | ||||
| 230 | +next would not have this property and would need the sup-$F$ machinery that | ||||
| 231 | +is not implemented. Neither criterion yields a confidence set for the break | ||||
| 232 | +dates; the text reports the agreement of the two selections and the | ||||
| 233 | +sensitivity to the truncation month instead. | ||||
| 234 | + | ||||
| 235 | +\subsection{PELT: cost, penalty and pruning} | ||||
| 236 | +\label{sec:appC-pelt} | ||||
| 237 | + | ||||
| 238 | +\paragraph{Objective.} | ||||
| 239 | +With the $\ell_2$ cost $\mathcal{C}(y_{i:j}) = \sum_{t=i}^{j-1} (y_t - | ||||
| 240 | +\bar y_{i:j})^2$ for a segment $[i,j)$ and a penalty $\lambda > 0$ per | ||||
| 241 | +changepoint, the problem of equation~\eqref{eq:pelt} of Part~IV is | ||||
| 242 | +\begin{equation} | ||||
| 243 | + F(n) \;=\; \min_{m,\ \tau_1 < \dots < \tau_m}\ \Bigl\{ \sum_{k=1}^{m+1} \mathcal{C}(y_{\tau_{k-1}:\tau_k}) + \lambda m \Bigr\}, | ||||
| 244 | + \label{eq:appC-peltobj} | ||||
| 245 | +\end{equation} | ||||
| 246 | +and, by the same additivity argument as in Section~\ref{sec:appC-bp}, it | ||||
| 247 | +satisfies the optimal-partitioning recursion of \citet{jackson2005optimal}, | ||||
| 248 | +\begin{equation} | ||||
| 249 | + F(0) = -\lambda, \qquad F(j) \;=\; \min_{0 \le i < j}\ \bigl\{ F(i) + \mathcal{C}(y_{i:j}) + \lambda \bigr\}, | ||||
| 250 | + \label{eq:appC-op} | ||||
| 251 | +\end{equation} | ||||
| 252 | +where the number of breaks is not fixed in advance but chosen by the | ||||
| 253 | +penalty; the convention $F(0) = -\lambda$ cancels the penalty charged to the | ||||
| 254 | +first segment. Solving \eqref{eq:appC-op} for all $j$ costs $O(n^2)$ cost | ||||
| 255 | +evaluations. | ||||
| 256 | + | ||||
| 257 | +\paragraph{The pruning condition.} | ||||
| 258 | +\citet{killick2012pelt} observe that if there exists a constant $K$ such that | ||||
| 259 | +for all $i < j < l$ | ||||
| 260 | +\begin{equation} | ||||
| 261 | + \mathcal{C}(y_{i:j}) + \mathcal{C}(y_{j:l}) + K \;\le\; \mathcal{C}(y_{i:l}), | ||||
| 262 | + \label{eq:appC-Kcond} | ||||
| 263 | +\end{equation} | ||||
| 264 | +then any candidate $i$ for which | ||||
| 265 | +\begin{equation} | ||||
| 266 | + F(i) + \mathcal{C}(y_{i:j}) + K \;\ge\; F(j) | ||||
| 267 | + \label{eq:appC-prune} | ||||
| 268 | +\end{equation} | ||||
| 269 | +can never be the last changepoint before any later $l > j$ and may be | ||||
| 270 | +discarded from the candidate set. The proof is two lines. For $l > j$, by | ||||
| 271 | +\eqref{eq:appC-Kcond} and \eqref{eq:appC-prune}, | ||||
| 272 | +$F(i) + \mathcal{C}(y_{i:l}) + \lambda \ge F(i) + \mathcal{C}(y_{i:j}) + | ||||
| 273 | +\mathcal{C}(y_{j:l}) + K + \lambda \ge F(j) + \mathcal{C}(y_{j:l}) + | ||||
| 274 | +\lambda$, so the partition that places a changepoint at $j$ after the | ||||
| 275 | +optimal prefix of $[0,j)$ is at least as good as any that ends its last | ||||
| 276 | +segment at $i$; hence $i$ never needs to be examined again. For the | ||||
| 277 | +$\ell_2$ cost, \eqref{eq:appC-Kcond} holds with $K = 0$, because adding a | ||||
| 278 | +split point to a segment can only reduce its residual sum of squares (the | ||||
| 279 | +split model nests the unsplit one). The recursion with pruning examines, at | ||||
| 280 | +each $j$, only the candidates not yet discarded; under the regularity | ||||
| 281 | +conditions of \citeauthor{killick2012pelt} (changepoints whose number grows | ||||
| 282 | +linearly with $n$) the expected number of surviving candidates is bounded and | ||||
| 283 | +the total cost is linear in $n$, whence the name. For a short series with | ||||
| 284 | +few changes, as here, the pruning saves little and the method is simply an | ||||
| 285 | +exact solution of \eqref{eq:appC-peltobj}; the point of using it is | ||||
| 286 | +exactness at a given penalty, not speed. The implementation | ||||
| 287 | +(\texttt{ruptures.Pelt}, \citealp{truong2020ruptures}) is called with | ||||
| 288 | +\texttt{model="l2"}, \texttt{min\_size=6} and \texttt{jump=1}, the last two | ||||
| 289 | +restricting segments to at least six months and allowing a changepoint at | ||||
| 290 | +every index; the library returns the segment end points including $n$, which | ||||
| 291 | +the wrapper drops. | ||||
| 292 | + | ||||
| 293 | +\paragraph{The penalty $2\sigma^2 \log n$.} | ||||
| 294 | +Suppose the observations are independent Gaussian with piecewise-constant | ||||
| 295 | +mean and a common variance $\sigma^2$. Twice the negative log-likelihood of | ||||
| 296 | +a partition, up to a constant, is $\sigma^{-2}\sum_k \mathcal{C}(y_{\tau_{k-1}:\tau_k})$. | ||||
| 297 | +Each additional changepoint adds two parameters, a segment mean and a | ||||
| 298 | +position, so the Schwarz criterion in this scale adds $2\log n$ per | ||||
| 299 | +changepoint. Multiplying through by $\sigma^2$ to express the criterion in | ||||
| 300 | +the units of the cost gives | ||||
| 301 | +\begin{equation} | ||||
| 302 | + \sum_k \mathcal{C}(y_{\tau_{k-1}:\tau_k}) + 2\sigma^2 \log n \cdot m, | ||||
| 303 | +\end{equation} | ||||
| 304 | +which is \eqref{eq:appC-peltobj} with $\lambda = 2\sigma^2\log n$. The | ||||
| 305 | +implementation replaces $\sigma^2$ by the sample variance of the whole | ||||
| 306 | +series, $\widehat{\mathrm{Var}}(y) = n^{-1}\sum_t (y_t - \bar y)^2$ | ||||
| 307 | +(\texttt{numpy.var}, population form). This is the conservative choice | ||||
| 308 | +discussed in Part~IV: in a series with a trend or with genuine level | ||||
| 309 | +shifts, the overall variance exceeds the within-segment variance, so the | ||||
| 310 | +penalty is larger than the within-segment Schwarz penalty and fewer | ||||
| 311 | +changepoints are declared. \citet{yao1988schwarz} proves that the Schwarz | ||||
| 312 | +criterion, with a known or consistently estimated $\sigma^2$, selects the | ||||
| 313 | +true number of changes with probability tending to one in the | ||||
| 314 | +piecewise-constant Gaussian model; the result does not extend to a series | ||||
| 315 | +whose mean is a trend, for which a piecewise-constant fit must spend | ||||
| 316 | +changepoints to track the slope, and the PELT dates of this study are read | ||||
| 317 | +with that in mind. | ||||
| 318 | + | ||||
| 319 | +\subsection{Bayesian online changepoint detection} | ||||
| 320 | +\label{sec:appC-bocpd} | ||||
| 321 | + | ||||
| 322 | +\paragraph{Conjugate updates.} | ||||
| 323 | +Within a run the observations are modelled as | ||||
| 324 | +$y \mid \mu, \sigma^2 \sim \mathcal{N}(\mu, \sigma^2)$ with the | ||||
| 325 | +Normal--Inverse-Gamma prior | ||||
| 326 | +$\mu \mid \sigma^2 \sim \mathcal{N}(\mu_0, \sigma^2/\kappa_0)$, | ||||
| 327 | +$\sigma^2 \sim \mathrm{IG}(\alpha_0, \beta_0)$, whose joint density is | ||||
| 328 | +proportional to | ||||
| 329 | +\begin{equation} | ||||
| 330 | + (\sigma^2)^{-\alpha_0 - 3/2} \exp\Bigl\{ -\frac{\beta_0}{\sigma^2} - \frac{\kappa_0 (\mu - \mu_0)^2}{2\sigma^2} \Bigr\}. | ||||
| 331 | +\end{equation} | ||||
| 332 | +Multiplying by the likelihood of one observation $y$, proportional to | ||||
| 333 | +$(\sigma^2)^{-1/2}\exp\{-(y-\mu)^2/(2\sigma^2)\}$, and completing the square | ||||
| 334 | +in $\mu$, | ||||
| 335 | +\begin{equation} | ||||
| 336 | + \kappa_0 (\mu - \mu_0)^2 + (y - \mu)^2 | ||||
| 337 | + \;=\; (\kappa_0 + 1)\Bigl(\mu - \frac{\kappa_0\mu_0 + y}{\kappa_0 + 1}\Bigr)^2 + \frac{\kappa_0 (y - \mu_0)^2}{\kappa_0 + 1}, | ||||
| 338 | +\end{equation} | ||||
| 339 | +shows that the posterior is again Normal--Inverse-Gamma with | ||||
| 340 | +\begin{equation} | ||||
| 341 | + \mu_1 = \frac{\kappa_0 \mu_0 + y}{\kappa_0 + 1}, \quad | ||||
| 342 | + \kappa_1 = \kappa_0 + 1, \quad | ||||
| 343 | + \alpha_1 = \alpha_0 + \tfrac12, \quad | ||||
| 344 | + \beta_1 = \beta_0 + \frac{\kappa_0 (y - \mu_0)^2}{2(\kappa_0 + 1)}, | ||||
| 345 | + \label{eq:appC-nigupdate} | ||||
| 346 | +\end{equation} | ||||
| 347 | +which are the four one-step updates of the implementation | ||||
| 348 | +(\texttt{mu\_new}, \texttt{kappa\_new}, \texttt{alpha\_new}, | ||||
| 349 | +\texttt{beta\_new}). Applying \eqref{eq:appC-nigupdate} $r$ times and | ||||
| 350 | +collecting terms gives the batch form quoted in Part~IV, with | ||||
| 351 | +$\kappa_r = \kappa_0 + r$, $\mu_r = (\kappa_0\mu_0 + r\bar y)/\kappa_r$, | ||||
| 352 | +$\alpha_r = \alpha_0 + r/2$ and | ||||
| 353 | +$\beta_r = \beta_0 + \tfrac12\sum_{i \le r}(y_i - \bar y)^2 + \kappa_0 r (\bar y - \mu_0)^2 / (2\kappa_r)$; | ||||
| 354 | +the equality of the two forms is a matter of algebra (induction on $r$) and | ||||
| 355 | +is the reason the online recursion can be trusted to reproduce the batch | ||||
| 356 | +posterior. | ||||
| 357 | + | ||||
| 358 | +\paragraph{Student-$t$ predictive.} | ||||
| 359 | +The predictive density of the next observation integrates the Gaussian | ||||
| 360 | +likelihood against the current Normal--Inverse-Gamma posterior. Integrating | ||||
| 361 | +$\mu$ first gives $y \mid \sigma^2 \sim \mathcal{N}(\mu_r, \sigma^2 (1 + | ||||
| 362 | +1/\kappa_r))$; integrating $\sigma^2 \sim \mathrm{IG}(\alpha_r, \beta_r)$ | ||||
| 363 | +against this scale mixture of normals gives | ||||
| 364 | +\begin{equation} | ||||
| 365 | + p(y \mid r) \;=\; \int \mathcal{N}\Bigl(y;\ \mu_r,\ \sigma^2\frac{\kappa_r + 1}{\kappa_r}\Bigr)\,\mathrm{IG}(\sigma^2; \alpha_r, \beta_r)\, d\sigma^2 | ||||
| 366 | + \;=\; t_{2\alpha_r}\Bigl( y;\ \mu_r,\ \frac{\beta_r(\kappa_r + 1)}{\alpha_r \kappa_r} \Bigr), | ||||
| 367 | + \label{eq:appC-student} | ||||
| 368 | +\end{equation} | ||||
| 369 | +a Student-$t$ density with $2\alpha_r$ degrees of freedom, location $\mu_r$ | ||||
| 370 | +and squared scale $\beta_r(\kappa_r+1)/(\alpha_r\kappa_r)$: the standard | ||||
| 371 | +result that a normal with inverse-gamma mixing on the variance is a | ||||
| 372 | +$t$ with degrees of freedom twice the shape and scale equal to the | ||||
| 373 | +rate over the shape times the normal's variance factor. The code evaluates | ||||
| 374 | +\eqref{eq:appC-student} through \texttt{scipy.stats.t.pdf} with | ||||
| 375 | +\texttt{df = 2*alpha} and \texttt{scale = sqrt(beta*(kappa+1)/(alpha*kappa))}, | ||||
| 376 | +vectorised over the run lengths. | ||||
| 377 | + | ||||
| 378 | +\paragraph{Run-length recursion and the mass on $r = 0$.} | ||||
| 379 | +Let $R_t(r) = P(r_t = r \mid y_{1:t})$ be the filtered run-length posterior | ||||
| 380 | +after $t$ observations, with $R_0(0) = 1$, and let $\pi_t(r) = p(y_{t+1} \mid | ||||
| 381 | +r_t = r, y_{1:t})$ be the predictive \eqref{eq:appC-student} evaluated with | ||||
| 382 | +the hyperparameters of the run of length $r$ ending at $t$. With a constant | ||||
| 383 | +hazard $H$ (a geometric prior on run lengths), the joint recursion of | ||||
| 384 | +\citet{adams2007bocpd} is, for the growth and the change moves, | ||||
| 385 | +\begin{equation} | ||||
| 386 | + \tilde R_{t+1}(r+1) = R_t(r)\,\pi_t(r)\,(1 - H), \qquad | ||||
| 387 | + \tilde R_{t+1}(0) = \sum_{r=0}^{t} R_t(r)\,\pi_t(r)\, H, | ||||
| 388 | + \label{eq:appC-rl} | ||||
| 389 | +\end{equation} | ||||
| 390 | +followed by normalisation, $R_{t+1}(r) = \tilde R_{t+1}(r) / \sum_{r'} | ||||
| 391 | +\tilde R_{t+1}(r')$. Write $Z_t = \sum_{r} R_t(r)\pi_t(r)$ for the marginal | ||||
| 392 | +predictive likelihood of $y_{t+1}$. Then $\sum_{r \ge 1}\tilde R_{t+1}(r) = | ||||
| 393 | +(1-H) Z_t$ and $\tilde R_{t+1}(0) = H Z_t$, so the normaliser is $Z_t$ and | ||||
| 394 | +\begin{equation} | ||||
| 395 | + R_{t+1}(0) \;=\; \frac{H Z_t}{H Z_t + (1-H) Z_t} \;=\; H \qquad \text{for every } t . | ||||
| 396 | + \label{eq:appC-r0} | ||||
| 397 | +\end{equation} | ||||
| 398 | +The filtered probability that a changepoint occurs \emph{at} the current | ||||
| 399 | +step therefore equals the prior hazard identically, whatever the data: the | ||||
| 400 | +run-length posterior carries information about where the last change was | ||||
| 401 | +only through the relative masses at $r \ge 1$, which are reweighted by the | ||||
| 402 | +predictive likelihoods, and not through the mass at $r = 0$, which the | ||||
| 403 | +constant hazard fixes. This is why the detection statistics of Part~IV are | ||||
| 404 | +not $R_t(0)$. The first, $P(r_t \le 3 \mid y_{1:t}) = \sum_{r=0}^{3} R_t(r)$, | ||||
| 405 | +is $H$ plus the mass on runs of length one to three, that is, on changes | ||||
| 406 | +that occurred one to three months earlier and have since accumulated | ||||
| 407 | +evidence; its value exceeds $H$ only when the recent observations are | ||||
| 408 | +better explained by a fresh run than by the continuation of the old one. The | ||||
| 409 | +second is the maximum-a-posteriori run length $\hat r_t = \arg\max_r R_t(r)$, | ||||
| 410 | +whose resets from a value above twelve to a value at most two are the | ||||
| 411 | +online detector's changepoints. The implementation keeps the full | ||||
| 412 | +$(n+1)\times(n+1)$ matrix $R$, adds $10^{-300}$ to the normaliser against | ||||
| 413 | +underflow, and initialises $\mu_0$ and $\beta_0$ from the first six | ||||
| 414 | +observations with $\kappa_0 = \alpha_0 = 1$; the prior mean run length | ||||
| 415 | +$1/H = 60$ months is the only tuning constant. The recursion is the filtered | ||||
| 416 | +one; the smoothed posterior of \citet{fearnhead2006exact} would require a | ||||
| 417 | +backward pass and answers the retrospective question that the offline | ||||
| 418 | +methods of Sections~\ref{sec:appC-bp} and~\ref{sec:appC-pelt} already | ||||
| 419 | +address. | ||||
modified
manuscript/appendices/appE_data_dictionary.tex
+814 −1
@@ -1 +1,814 @@ | |||||
| 1 | −% appE_data_dictionary — to be written (Phase 7) | 1 | +% ============================================================================ | ||
| 2 | +% Appendix E — Data dictionary | ||||
| 3 | +% Written by Agent L (appendices A/E/H instance), 2026-11-03. Derived from the code that creates each table: | ||||
| 4 | +% ingestion/openalex_snapshot.py (slim schema), ingestion/dblp_parse.py, ingestion/arxiv_oai.py, ingestion/acl_parse.py, | ||||
| 5 | +% ingestion/build_corpus.py (schemas S), ingestion/link_sources.py (key and linkage tables), | ||||
| 6 | +% systematic_review/define_corpus.py (corpus_works), bibliometrics/03_citation_impact.py (work_citation_windows), | ||||
| 7 | +% knowledge_graph/01_citation_graph.py (edge tables), taxonomy/classify.py (classification tables), | ||||
| 8 | +% embeddings/embed_corpus.py (embedding shards). Column names and types are those of the Arrow schemas or of the | ||||
| 9 | +% DuckDB statements; "source" gives the OpenAlex JSON path or the derivation. No corpus number appears here. | ||||
| 10 | +% ============================================================================ | ||||
| 11 | + | ||||
| 12 | +\section{Data Dictionary} | ||||
| 13 | +\label{app:datadict} | ||||
| 14 | + | ||||
| 15 | +This appendix defines every column of every table that the analyses of Parts~VIII--IX read, in the order in | ||||
| 16 | +which the pipeline creates them (Appendix~\ref{app:repro} gives the commands). The dictionary was derived | ||||
| 17 | +from the code rather than from the data: the Arrow schemas declared in the ingestion scripts and the | ||||
| 18 | +\texttt{SELECT} lists of the DuckDB statements are the sources, and a column is described as the code computes | ||||
| 19 | +it, including the conventions that the raw sources do not share (integer identifiers, lower-cased DOIs, | ||||
| 20 | +booleans that are never null). A reader who wants to recompute an indicator of Part~IX should be able to do | ||||
| 21 | +so from this appendix and the released aggregate tables without reading the scripts. | ||||
| 22 | + | ||||
| 23 | +\subsection{Conventions} | ||||
| 24 | +\label{app:datadict-conventions} | ||||
| 25 | + | ||||
| 26 | +\paragraph{Identifiers.} OpenAlex identifiers are stored as integers with the entity letter removed: | ||||
| 27 | +\texttt{https:/\allowbreak /\allowbreak openalex.\allowbreak org/\allowbreak W2741809807} becomes the \texttt{int64} value 2741809807, and the same rule | ||||
| 28 | +applies to authors (A), institutions (I), sources (S), topics (T), subfields, fields, domains and funders | ||||
| 29 | +(function \texttt{oid} in \texttt{build\_\allowbreak corpus.\allowbreak py}; a tail that is not numeric yields a null). DBLP records | ||||
| 30 | +are identified by their DBLP key (\texttt{conf/\allowbreak nips/\allowbreak VaswaniSPUJGKP17}), arXiv records by the bare arXiv | ||||
| 31 | +identifier without version suffix and in lower case (\texttt{2303.\allowbreak 08774}, \texttt{cs/\allowbreak 0112017}), ACL Anthology | ||||
| 32 | +entries by their anthology identifier (\texttt{2020.\allowbreak acl-main.\allowbreak 207}). DOIs are stored without the | ||||
| 33 | +\texttt{https:/\allowbreak /\allowbreak doi.\allowbreak org/\allowbreak } prefix and in lower case, on every side of every linkage. | ||||
| 34 | + | ||||
| 35 | +\paragraph{Types.} Types are Apache Arrow types as declared in the schemas (\texttt{int64}, \texttt{int32}, | ||||
| 36 | +\texttt{int16}, \texttt{float32}, \texttt{bool}, \texttt{string}, \texttt{list<string>}, | ||||
| 37 | +\texttt{list<int64>}); DuckDB-created tables use the corresponding DuckDB types (\texttt{BIGINT}, | ||||
| 38 | +\texttt{INTEGER}, \texttt{DOUBLE}, \texttt{BOOLEAN}, \texttt{VARCHAR}). Dates are ISO-8601 strings | ||||
| 39 | +(\texttt{YYYY-MM-DD}); a missing date is null, never a sentinel. Booleans produced by \texttt{build\_\allowbreak corpus.\allowbreak py} | ||||
| 40 | +are coerced with \texttt{bool()} and are therefore never null (a missing flag reads as false); booleans | ||||
| 41 | +produced by DuckDB expressions can be null only where the definition says so. | ||||
| 42 | + | ||||
| 43 | +\paragraph{Files and views.} The normalised OpenAlex tables are written as one Zstandard-compressed Parquet | ||||
| 44 | +file per snapshot part (\texttt{data/\allowbreak processed/\allowbreak tables/\allowbreak <table>/\allowbreak <part>.\allowbreak parquet}), then compacted into one file | ||||
| 45 | +per table (\texttt{data/\allowbreak processed/\allowbreak compact/\allowbreak <table>.\allowbreak parquet}) because DuckDB could not open 1{,}898 files per | ||||
| 46 | +table within the macOS file-descriptor limit (incident I-2, Appendix~\ref{app:repro}). The second snapshot | ||||
| 47 | +pass writes the same tables with the suffix \texttt{\_\allowbreak p2}; \texttt{link\_\allowbreak sources.\allowbreak py} exposes each table as | ||||
| 48 | +a DuckDB view \texttt{oa\_\allowbreak <table>} that unions the two passes by column name. The raw-source tables are | ||||
| 49 | +exposed as \texttt{dblp\_\allowbreak raw}, \texttt{arxiv\_\allowbreak raw}, \texttt{acl\_\allowbreak raw} and \texttt{oa\_\allowbreak slim}. The database file | ||||
| 50 | +is \texttt{data/\allowbreak processed/\allowbreak wp44.\allowbreak duckdb}; it is a derived artefact and is not redistributed, whereas every | ||||
| 51 | +table it contains can be rebuilt from the Parquet files. | ||||
| 52 | + | ||||
| 53 | +\paragraph{Reading the tables.} Each dictionary table lists the column, its type, its definition and its | ||||
| 54 | +source: an OpenAlex JSON path (for instance \texttt{primary\_\allowbreak topic.\allowbreak subfield.\allowbreak id}) for fields copied from the | ||||
| 55 | +snapshot, a DBLP, arXiv-OAI or BibTeX element for the other sources, or the derivation for computed columns. | ||||
| 56 | +Columns are listed in schema order. | ||||
| 57 | + | ||||
| 58 | +\subsection{Raw-source tables} | ||||
| 59 | +\label{app:datadict-raw} | ||||
| 60 | + | ||||
| 61 | +The four raw sources are parsed once each into a Parquet table that keeps the source's own vocabulary; nothing | ||||
| 62 | +is harmonised at this stage except the identifier conventions above. The OpenAlex \emph{slim} table is the | ||||
| 63 | +only table that covers every work of the snapshot, not only computer science; it exists so that references to | ||||
| 64 | +works outside the corpus can be resolved to a publication year, type and field | ||||
| 65 | +(Section~\ref{app:datadict-graph}). | ||||
| 66 | + | ||||
| 67 | +\begin{singlespace} | ||||
| 68 | +\begin{footnotesize} | ||||
| 69 | +\begin{longtable}{@{}p{3.0cm} p{1.7cm} p{6.1cm} p{4.6cm}@{}} | ||||
| 70 | +\caption{\texttt{slim} (\texttt{data/\allowbreak raw/\allowbreak openalex/\allowbreak works/\allowbreak slim/\allowbreak *.\allowbreak parquet}, compacted to \texttt{compact/\allowbreak slim\_\allowbreak all\_\allowbreak works.\allowbreak parquet}; DuckDB view \texttt{oa\_\allowbreak slim}): one row for every work of the OpenAlex snapshot (\texttt{openalex\_\allowbreak snapshot.\allowbreak py}, \texttt{SLIM\_\allowbreak SCHEMA}).} | ||||
| 71 | +\label{tab:appE-slim}\\ | ||||
| 72 | +\toprule | ||||
| 73 | +Column & Type & Definition & Source \\ | ||||
| 74 | +\midrule | ||||
| 75 | +\endfirsthead | ||||
| 76 | +\caption[]{\texttt{slim} (continued).}\\ | ||||
| 77 | +\toprule | ||||
| 78 | +Column & Type & Definition & Source \\ | ||||
| 79 | +\midrule | ||||
| 80 | +\endhead | ||||
| 81 | +\midrule | ||||
| 82 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 83 | +\endfoot | ||||
| 84 | +\bottomrule | ||||
| 85 | +\endlastfoot | ||||
| 86 | +\texttt{id} & int64 & OpenAlex work identifier (letter stripped) & \texttt{id} \\ | ||||
| 87 | +\texttt{doi} & string & DOI, lower case, prefix removed; null if absent & \texttt{doi} \\ | ||||
| 88 | +\texttt{publication\_\allowbreak year} & int32 & Publication year & \texttt{publication\_\allowbreak year} \\ | ||||
| 89 | +\texttt{publication\_\allowbreak date} & string & Publication date, \texttt{YYYY-MM-DD} & \texttt{publication\_\allowbreak date} \\ | ||||
| 90 | +\texttt{type} & string & OpenAlex work type (article, preprint, conference-paper, \ldots) & \texttt{type} \\ | ||||
| 91 | +\texttt{cited\_\allowbreak by\_\allowbreak count} & int32 & Citations received, all years, as of the snapshot & \texttt{cited\_\allowbreak by\_\allowbreak count} \\ | ||||
| 92 | +\texttt{primary\_\allowbreak topic\_\allowbreak id} & int32 & Primary topic & \texttt{primary\_\allowbreak topic.\allowbreak id} \\ | ||||
| 93 | +\texttt{primary\_\allowbreak subfield\_\allowbreak id} & int32 & Subfield of the primary topic & \texttt{primary\_\allowbreak topic.\allowbreak subfield.\allowbreak id} \\ | ||||
| 94 | +\texttt{primary\_\allowbreak field\_\allowbreak id} & int32 & Field of the primary topic (17 = Computer Science) & \texttt{primary\_\allowbreak topic.\allowbreak field.\allowbreak id} \\ | ||||
| 95 | +\texttt{primary\_\allowbreak domain\_\allowbreak id} & int32 & Domain of the primary topic & \texttt{primary\_\allowbreak topic.\allowbreak domain.\allowbreak id} \\ | ||||
| 96 | +\texttt{cs\_\allowbreak any\_\allowbreak topic} & bool & True if any of the work's topics, or its primary topic, is in field~17 & \texttt{topics[].\allowbreak field.\allowbreak id}, \texttt{primary\_\allowbreak topic.\allowbreak field.\allowbreak id} \\ | ||||
| 97 | +\texttt{is\_\allowbreak xpac} & bool & OpenAlex ``extended publication and citation'' flag (records of lower provenance) & \texttt{is\_\allowbreak xpac} \\ | ||||
| 98 | +\texttt{is\_\allowbreak oa} & bool & Open-access flag & \texttt{open\_\allowbreak access.\allowbreak is\_\allowbreak oa} \\ | ||||
| 99 | +\texttt{is\_\allowbreak retracted} & bool & Retraction flag & \texttt{is\_\allowbreak retracted} \\ | ||||
| 100 | +\texttt{language} & string & ISO language code & \texttt{language} \\ | ||||
| 101 | +\texttt{source\_\allowbreak id} & int64 & Source (venue) of the primary location & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak id} \\ | ||||
| 102 | +\texttt{n\_\allowbreak refs} & int32 & Number of referenced works & length of \texttt{referenced\_\allowbreak works} \\ | ||||
| 103 | +\texttt{n\_\allowbreak authors} & int32 & Number of authors & \texttt{authors\_\allowbreak count}, else length of \texttt{authorships} \\ | ||||
| 104 | +\texttt{has\_\allowbreak abstract} & bool & True if an inverted-index abstract is present & \texttt{abstract\_\allowbreak inverted\_\allowbreak index} $\neq$ null \\ | ||||
| 105 | +\texttt{fwci} & float32 & Field-weighted citation impact as computed by OpenAlex & \texttt{fwci} \\ | ||||
| 106 | +\end{longtable} | ||||
| 107 | +\end{footnotesize} | ||||
| 108 | +\end{singlespace} | ||||
| 109 | + | ||||
| 110 | +\begin{singlespace} | ||||
| 111 | +\begin{footnotesize} | ||||
| 112 | +\begin{longtable}{@{}p{3.0cm} p{1.7cm} p{6.1cm} p{4.6cm}@{}} | ||||
| 113 | +\caption{\texttt{dblp\_\allowbreak publications} (\texttt{data/\allowbreak raw/\allowbreak dblp/\allowbreak dblp\_\allowbreak publications.\allowbreak parquet}; view \texttt{dblp\_\allowbreak raw}): one row per DBLP publication record of every year (\texttt{dblp\_\allowbreak parse.\allowbreak py}). Record types \texttt{www} are skipped.} | ||||
| 114 | +\label{tab:appE-dblp}\\ | ||||
| 115 | +\toprule | ||||
| 116 | +Column & Type & Definition & Source \\ | ||||
| 117 | +\midrule | ||||
| 118 | +\endfirsthead | ||||
| 119 | +\caption[]{\texttt{dblp\_\allowbreak publications} (continued).}\\ | ||||
| 120 | +\toprule | ||||
| 121 | +Column & Type & Definition & Source \\ | ||||
| 122 | +\midrule | ||||
| 123 | +\endhead | ||||
| 124 | +\midrule | ||||
| 125 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 126 | +\endfoot | ||||
| 127 | +\bottomrule | ||||
| 128 | +\endlastfoot | ||||
| 129 | +\texttt{key} & string & DBLP record key, unique & element attribute \texttt{key} \\ | ||||
| 130 | +\texttt{rec\_\allowbreak type} & string & Record type: article, inproceedings, proceedings, book, incollection, phdthesis, mastersthesis & element name \\ | ||||
| 131 | +\texttt{mdate} & string & Last modification date of the record & attribute \texttt{mdate} \\ | ||||
| 132 | +\texttt{publtype} & string & Publication sub-type (e.g.\ informal, survey, withdrawn); null if absent & attribute \texttt{publtype} \\ | ||||
| 133 | +\texttt{title} & string & Title, inner markup flattened to text & \texttt{<title>} \\ | ||||
| 134 | +\texttt{year} & int32 & Year & \texttt{<year>} \\ | ||||
| 135 | +\texttt{authors} & list<string> & Author names in order & \texttt{<author>} \\ | ||||
| 136 | +\texttt{author\_\allowbreak pids} & list<string> & DBLP person identifiers of the authors & \texttt{<author pid=.\allowbreak .\allowbreak .\allowbreak >} \\ | ||||
| 137 | +\texttt{editors} & list<string> & Editor names & \texttt{<editor>} \\ | ||||
| 138 | +\texttt{journal} & string & Journal name (articles) & \texttt{<journal>} \\ | ||||
| 139 | +\texttt{booktitle} & string & Proceedings or book title & \texttt{<booktitle>} \\ | ||||
| 140 | +\texttt{volume}, \texttt{number}, \texttt{pages} & string & Bibliographic locators & \texttt{<volume>}, \texttt{<number>}, \texttt{<pages>} \\ | ||||
| 141 | +\texttt{publisher}, \texttt{series} & string & Publisher and series & \texttt{<publisher>}, \texttt{<series>} \\ | ||||
| 142 | +\texttt{doi} & string & DOI when DBLP records one & \texttt{<ee>} with a doi.org URL \\ | ||||
| 143 | +\texttt{ee} & list<string> & Electronic-edition URLs (DOI, arXiv, publisher) & \texttt{<ee>} \\ | ||||
| 144 | +\texttt{url} & string & DBLP-internal URL (table of contents) & \texttt{<url>} \\ | ||||
| 145 | +\texttt{crossref} & string & Key of the enclosing proceedings or book & \texttt{<crossref>} \\ | ||||
| 146 | +\texttt{stream} & string & Venue stream: the key with its last path component removed (\texttt{conf/\allowbreak nips}, \texttt{journals/\allowbreak corr}); null if the key has no slash & derived from \texttt{key} \\ | ||||
| 147 | +\end{longtable} | ||||
| 148 | +\end{footnotesize} | ||||
| 149 | +\end{singlespace} | ||||
| 150 | + | ||||
| 151 | +\begin{singlespace} | ||||
| 152 | +\begin{footnotesize} | ||||
| 153 | +\begin{longtable}{@{}p{3.0cm} p{1.7cm} p{6.1cm} p{4.6cm}@{}} | ||||
| 154 | +\caption{\texttt{arxiv\_\allowbreak cs} (\texttt{data/\allowbreak raw/\allowbreak arxiv/\allowbreak arxiv\_\allowbreak cs.\allowbreak parquet}; view \texttt{arxiv\_\allowbreak raw}): one row per arXiv record of set \texttt{cs} harvested by OAI-PMH with the \texttt{arXiv} metadata format (\texttt{arxiv\_\allowbreak oai.\allowbreak py}); records returned in several monthly windows are kept once, the latest datestamp winning.} | ||||
| 155 | +\label{tab:appE-arxiv}\\ | ||||
| 156 | +\toprule | ||||
| 157 | +Column & Type & Definition & Source \\ | ||||
| 158 | +\midrule | ||||
| 159 | +\endfirsthead | ||||
| 160 | +\caption[]{\texttt{arxiv\_\allowbreak cs} (continued).}\\ | ||||
| 161 | +\toprule | ||||
| 162 | +Column & Type & Definition & Source \\ | ||||
| 163 | +\midrule | ||||
| 164 | +\endhead | ||||
| 165 | +\midrule | ||||
| 166 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 167 | +\endfoot | ||||
| 168 | +\bottomrule | ||||
| 169 | +\endlastfoot | ||||
| 170 | +\texttt{arxiv\_\allowbreak id} & string & arXiv identifier without version & \texttt{<id>} \\ | ||||
| 171 | +\texttt{datestamp} & string & OAI datestamp (last update of the record) & header \texttt{<datestamp>} \\ | ||||
| 172 | +\texttt{created} & string & Date of the first submitted version & \texttt{<created>} \\ | ||||
| 173 | +\texttt{updated} & string & Date of the latest version & \texttt{<updated>} \\ | ||||
| 174 | +\texttt{title}, \texttt{abstract} & string & Title and abstract & \texttt{<title>}, \texttt{<abstract>} \\ | ||||
| 175 | +\texttt{authors} & list<string> & Author names & \texttt{<authors>/\allowbreak <author>} \\ | ||||
| 176 | +\texttt{categories} & list<string> & All categories (primary first, then cross-lists) & \texttt{<categories>} \\ | ||||
| 177 | +\texttt{primary\_\allowbreak category} & string & First category & first token of \texttt{<categories>} \\ | ||||
| 178 | +\texttt{comments}, \texttt{journal\_\allowbreak ref} & string & Author comments and journal reference & \texttt{<comments>}, \texttt{<journal-ref>} \\ | ||||
| 179 | +\texttt{doi} & string & DOI declared by the authors & \texttt{<doi>} \\ | ||||
| 180 | +\texttt{license} & string & Licence URL of the e-print & \texttt{<license>} \\ | ||||
| 181 | +\texttt{report\_\allowbreak no}, \texttt{msc\_\allowbreak class}, \texttt{acm\_\allowbreak class} & string & Report number, MSC and ACM classes & corresponding elements \\ | ||||
| 182 | +\texttt{set\_\allowbreak specs} & list<string> & OAI set memberships & header \texttt{<setSpec>} \\ | ||||
| 183 | +\end{longtable} | ||||
| 184 | +\end{footnotesize} | ||||
| 185 | +\end{singlespace} | ||||
| 186 | + | ||||
| 187 | +\begin{singlespace} | ||||
| 188 | +\begin{footnotesize} | ||||
| 189 | +\begin{longtable}{@{}p{3.0cm} p{1.7cm} p{6.1cm} p{4.6cm}@{}} | ||||
| 190 | +\caption{\texttt{acl\_\allowbreak anthology} (\texttt{data/\allowbreak raw/\allowbreak acl/\allowbreak acl\_\allowbreak anthology.\allowbreak parquet}; view \texttt{acl\_\allowbreak raw}): one row per entry of the ACL Anthology BibTeX export with abstracts (\texttt{acl\_\allowbreak parse.\allowbreak py}). All fields are strings copied from the BibTeX entry; braces are removed.} | ||||
| 191 | +\label{tab:appE-acl}\\ | ||||
| 192 | +\toprule | ||||
| 193 | +Column & Type & Definition & Source \\ | ||||
| 194 | +\midrule | ||||
| 195 | +\endfirsthead | ||||
| 196 | +\caption[]{\texttt{acl\_\allowbreak anthology} (continued).}\\ | ||||
| 197 | +\toprule | ||||
| 198 | +Column & Type & Definition & Source \\ | ||||
| 199 | +\midrule | ||||
| 200 | +\endhead | ||||
| 201 | +\midrule | ||||
| 202 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 203 | +\endfoot | ||||
| 204 | +\bottomrule | ||||
| 205 | +\endlastfoot | ||||
| 206 | +\texttt{acl\_\allowbreak id} & string & Anthology identifier (BibTeX citation key) & entry key \\ | ||||
| 207 | +\texttt{entry\_\allowbreak type} & string & BibTeX entry type (inproceedings, article, proceedings, \ldots), lower case & entry type \\ | ||||
| 208 | +\texttt{title}, \texttt{author}, \texttt{editor} & string & Title, author list, editor list as written in BibTeX & fields \\ | ||||
| 209 | +\texttt{booktitle}, \texttt{journal} & string & Venue & fields \\ | ||||
| 210 | +\texttt{month}, \texttt{year} & string & Month and year (year is cast to integer in \texttt{acl\_\allowbreak keys}) & fields \\ | ||||
| 211 | +\texttt{address}, \texttt{publisher} & string & Place and publisher & fields \\ | ||||
| 212 | +\texttt{url}, \texttt{doi} & string & Anthology URL and DOI & fields \\ | ||||
| 213 | +\texttt{pages}, \texttt{volume}, \texttt{number} & string & Locators & fields \\ | ||||
| 214 | +\texttt{abstract} & string & Abstract when the export carries one & field \\ | ||||
| 215 | +\end{longtable} | ||||
| 216 | +\end{footnotesize} | ||||
| 217 | +\end{singlespace} | ||||
| 218 | + | ||||
| 219 | +\subsection{Normalised OpenAlex tables} | ||||
| 220 | +\label{app:datadict-openalex} | ||||
| 221 | + | ||||
| 222 | +\texttt{build\_\allowbreak corpus.\allowbreak py} reads the full JSON record of every work retained by the snapshot filter (any | ||||
| 223 | +topic in field~17 and publication year 2018--2026, or, in the second pass, membership of the list of | ||||
| 224 | +DBLP- and arXiv-linked identifiers) and writes nine relational tables. The \texttt{works} table has one row | ||||
| 225 | +per work; the eight others are one-to-many and are keyed by \texttt{work\_\allowbreak id}. Nested JSON is flattened | ||||
| 226 | +without loss of the fields used downstream; fields not listed here (for instance \texttt{sustainable\_\allowbreak development\_\allowbreak goals}, | ||||
| 227 | +\texttt{mesh}, \texttt{concepts}) are not extracted. | ||||
| 228 | + | ||||
| 229 | +\begin{singlespace} | ||||
| 230 | +\begin{footnotesize} | ||||
| 231 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 232 | +\caption{\texttt{works} (\texttt{compact/\allowbreak works.\allowbreak parquet}; view \texttt{oa\_\allowbreak works}): one row per OpenAlex work of the CS extraction (\texttt{build\_\allowbreak corpus.\allowbreak py}, schema \texttt{S["works"]}).} | ||||
| 233 | +\label{tab:appE-works}\\ | ||||
| 234 | +\toprule | ||||
| 235 | +Column & Type & Definition & Source \\ | ||||
| 236 | +\midrule | ||||
| 237 | +\endfirsthead | ||||
| 238 | +\caption[]{\texttt{works} (continued).}\\ | ||||
| 239 | +\toprule | ||||
| 240 | +Column & Type & Definition & Source \\ | ||||
| 241 | +\midrule | ||||
| 242 | +\endhead | ||||
| 243 | +\midrule | ||||
| 244 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 245 | +\endfoot | ||||
| 246 | +\bottomrule | ||||
| 247 | +\endlastfoot | ||||
| 248 | +\texttt{id} & int64 & Work identifier & \texttt{id} \\ | ||||
| 249 | +\texttt{doi} & string & DOI, lower case, prefix removed; null if empty & \texttt{doi} \\ | ||||
| 250 | +\texttt{title} & string & Title; \texttt{display\_\allowbreak name} when \texttt{title} is absent & \texttt{title}, \texttt{display\_\allowbreak name} \\ | ||||
| 251 | +\texttt{publication\_\allowbreak year} & int32 & Publication year & \texttt{publication\_\allowbreak year} \\ | ||||
| 252 | +\texttt{publication\_\allowbreak date} & string & Publication date; OpenAlex sets \texttt{YYYY-01-01} when only the year is known (see \texttt{date\_\allowbreak precision}, Table~\ref{tab:appE-corpus}) & \texttt{publication\_\allowbreak date} \\ | ||||
| 253 | +\texttt{created\_\allowbreak date} & string & Date the record entered OpenAlex (used for indexing-lag diagnostics) & \texttt{created\_\allowbreak date} \\ | ||||
| 254 | +\texttt{updated\_\allowbreak date} & string & Last update, date part only & \texttt{updated\_\allowbreak date[:10]} \\ | ||||
| 255 | +\texttt{type} & string & OpenAlex work type & \texttt{type} \\ | ||||
| 256 | +\texttt{type\_\allowbreak crossref} & string & Crossref type when the record has one & \texttt{type\_\allowbreak crossref} \\ | ||||
| 257 | +\texttt{language} & string & ISO language code & \texttt{language} \\ | ||||
| 258 | +\texttt{cited\_\allowbreak by\_\allowbreak count} & int32 & Citations received, all years, at snapshot date & \texttt{cited\_\allowbreak by\_\allowbreak count} \\ | ||||
| 259 | +\texttt{fwci} & float32 & OpenAlex field-weighted citation impact & \texttt{fwci} \\ | ||||
| 260 | +\texttt{citation\_\allowbreak percentile} & float32 & Citation-normalised percentile & \texttt{citation\_\allowbreak normalized\_\allowbreak percentile.\allowbreak value} \\ | ||||
| 261 | +\texttt{is\_\allowbreak top1}, \texttt{is\_\allowbreak top10} & bool & Top 1\,\% / top 10\,\% flags of that percentile & \texttt{...is\_\allowbreak in\_\allowbreak top\_\allowbreak 1\_\allowbreak percent}, \texttt{...is\_\allowbreak in\_\allowbreak top\_\allowbreak 10\_\allowbreak percent} \\ | ||||
| 262 | +\texttt{is\_\allowbreak oa} & bool & Open-access flag & \texttt{open\_\allowbreak access.\allowbreak is\_\allowbreak oa} \\ | ||||
| 263 | +\texttt{oa\_\allowbreak status} & string & gold, green, hybrid, bronze, diamond, closed & \texttt{open\_\allowbreak access.\allowbreak oa\_\allowbreak status} \\ | ||||
| 264 | +\texttt{license} & string & Licence of the best OA location, else of the primary location & \texttt{best\_\allowbreak oa\_\allowbreak location.\allowbreak license}, \texttt{primary\_\allowbreak location.\allowbreak license} \\ | ||||
| 265 | +\texttt{is\_\allowbreak retracted} & bool & Retraction flag & \texttt{is\_\allowbreak retracted} \\ | ||||
| 266 | +\texttt{is\_\allowbreak paratext} & bool & Paratext flag (front matter, tables of contents) & \texttt{is\_\allowbreak paratext} \\ | ||||
| 267 | +\texttt{is\_\allowbreak xpac} & bool & Extended-provenance flag & \texttt{is\_\allowbreak xpac} \\ | ||||
| 268 | +\texttt{n\_\allowbreak refs} & int32 & Number of referenced works in the record & length of \texttt{referenced\_\allowbreak works} \\ | ||||
| 269 | +\texttt{n\_\allowbreak authors} & int32 & Number of authors & \texttt{authors\_\allowbreak count}, else length of \texttt{authorships} \\ | ||||
| 270 | +\texttt{n\_\allowbreak institutions} & int32 & Number of distinct institution identifiers over all authorships & \texttt{authorships[].\allowbreak institutions[].\allowbreak id} \\ | ||||
| 271 | +\texttt{n\_\allowbreak countries} & int32 & Number of distinct country codes over all authorships & \texttt{authorships[].\allowbreak countries} \\ | ||||
| 272 | +\texttt{countries} & list<string> & Sorted distinct country codes & \texttt{authorships[].\allowbreak countries} \\ | ||||
| 273 | +\texttt{institution\_\allowbreak types} & list<string> & Sorted distinct institution types (education, company, government, \ldots) & \texttt{authorships[].\allowbreak institutions[].\allowbreak type} \\ | ||||
| 274 | +\texttt{primary\_\allowbreak topic\_\allowbreak id} & int32 & Primary topic & \texttt{primary\_\allowbreak topic.\allowbreak id} \\ | ||||
| 275 | +\texttt{primary\_\allowbreak subfield\_\allowbreak id} & int32 & Subfield of the primary topic & \texttt{primary\_\allowbreak topic.\allowbreak subfield.\allowbreak id} \\ | ||||
| 276 | +\texttt{primary\_\allowbreak field\_\allowbreak id} & int32 & Field of the primary topic & \texttt{primary\_\allowbreak topic.\allowbreak field.\allowbreak id} \\ | ||||
| 277 | +\texttt{primary\_\allowbreak domain\_\allowbreak id} & int32 & Domain of the primary topic & \texttt{primary\_\allowbreak topic.\allowbreak domain.\allowbreak id} \\ | ||||
| 278 | +\texttt{cs\_\allowbreak primary} & bool & Primary topic in field~17 & \texttt{primary\_\allowbreak topic.\allowbreak field.\allowbreak id} \\ | ||||
| 279 | +\texttt{cs\_\allowbreak any} & bool & Any topic in field~17 & \texttt{topics[].\allowbreak field.\allowbreak id} \\ | ||||
| 280 | +\texttt{source\_\allowbreak id} & int64 & Venue of the primary location & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak id} \\ | ||||
| 281 | +\texttt{source\_\allowbreak name} & string & Venue name & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak display\_\allowbreak name} \\ | ||||
| 282 | +\texttt{source\_\allowbreak type} & string & journal, conference, repository, ebook platform, \ldots & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak type} \\ | ||||
| 283 | +\texttt{host\_\allowbreak organization} & string & Publisher or host organisation of the venue & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak host\_\allowbreak organization\_\allowbreak name} \\ | ||||
| 284 | +\texttt{is\_\allowbreak in\_\allowbreak doaj} & bool & Venue listed in DOAJ & \texttt{primary\_\allowbreak location.\allowbreak source.\allowbreak is\_\allowbreak in\_\allowbreak doaj} \\ | ||||
| 285 | +\texttt{arxiv\_\allowbreak id} & string & arXiv identifier: from \texttt{ids.\allowbreak arxiv}; else from a DOI of the form \texttt{10.\allowbreak 48550/\allowbreak arxiv.\allowbreak <id>}; else from an \texttt{arxiv.\allowbreak org/\allowbreak abs} or \texttt{/\allowbreak pdf} URL among the locations, version suffix removed & \texttt{ids.\allowbreak arxiv}, \texttt{doi}, \texttt{locations[].\allowbreak landing\_\allowbreak page\_\allowbreak url}, \texttt{locations[].\allowbreak pdf\_\allowbreak url} \\ | ||||
| 286 | +\texttt{pmid} & string & PubMed identifier (tail of the URL) & \texttt{ids.\allowbreak pmid} \\ | ||||
| 287 | +\texttt{mag} & string & Microsoft Academic Graph identifier & \texttt{ids.\allowbreak mag} \\ | ||||
| 288 | +\texttt{has\_\allowbreak abstract} & bool & An abstract could be reconstructed & \texttt{abstract\_\allowbreak inverted\_\allowbreak index} \\ | ||||
| 289 | +\texttt{has\_\allowbreak fulltext} & bool & OpenAlex has indexed full text & \texttt{has\_\allowbreak fulltext} or \texttt{has\_\allowbreak content} \\ | ||||
| 290 | +\texttt{n\_\allowbreak locations} & int32 & Number of locations (hosting copies) & length of \texttt{locations} \\ | ||||
| 291 | +\texttt{indexed\_\allowbreak in} & list<string> & Indexes that list the work (crossref, pubmed, doaj, arxiv, \ldots) & \texttt{indexed\_\allowbreak in} \\ | ||||
| 292 | +\texttt{referenced\_\allowbreak works\_\allowbreak count} & int32 & Reference count declared by OpenAlex; falls back to \texttt{n\_\allowbreak refs} & \texttt{referenced\_\allowbreak works\_\allowbreak count} \\ | ||||
| 293 | +\texttt{n\_\allowbreak grants} & int32 & Number of grant entries & length of \texttt{grants} \\ | ||||
| 294 | +\texttt{apc\_\allowbreak paid\_\allowbreak usd} & int32 & Article-processing charge paid, USD, when known & \texttt{apc\_\allowbreak paid.\allowbreak value\_\allowbreak usd} \\ | ||||
| 295 | +\texttt{is\_\allowbreak authors\_\allowbreak truncated} & bool & Author list truncated by OpenAlex & \texttt{is\_\allowbreak authors\_\allowbreak truncated} \\ | ||||
| 296 | +\texttt{part} & string & Snapshot part file of origin (\texttt{updated\_\allowbreak date=...\_\allowbreak \_\allowbreak part\_\allowbreak NNNN}) & file name \\ | ||||
| 297 | +\end{longtable} | ||||
| 298 | +\end{footnotesize} | ||||
| 299 | +\end{singlespace} | ||||
| 300 | + | ||||
| 301 | +\begin{singlespace} | ||||
| 302 | +\begin{footnotesize} | ||||
| 303 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 304 | +\caption{\texttt{authorships} (view \texttt{oa\_\allowbreak authorships}): one row per (work, author position).} | ||||
| 305 | +\label{tab:appE-authorships}\\ | ||||
| 306 | +\toprule | ||||
| 307 | +Column & Type & Definition & Source \\ | ||||
| 308 | +\midrule | ||||
| 309 | +\endfirsthead | ||||
| 310 | +\caption[]{\texttt{authorships} (continued).}\\ | ||||
| 311 | +\toprule | ||||
| 312 | +Column & Type & Definition & Source \\ | ||||
| 313 | +\midrule | ||||
| 314 | +\endhead | ||||
| 315 | +\midrule | ||||
| 316 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 317 | +\endfoot | ||||
| 318 | +\bottomrule | ||||
| 319 | +\endlastfoot | ||||
| 320 | +\texttt{work\_\allowbreak id} & int64 & Work & \texttt{id} of the work \\ | ||||
| 321 | +\texttt{position} & int32 & Zero-based index in the authorship list & list index \\ | ||||
| 322 | +\texttt{author\_\allowbreak position} & string & first, middle or last & \texttt{authorships[].\allowbreak author\_\allowbreak position} \\ | ||||
| 323 | +\texttt{author\_\allowbreak id} & int64 & OpenAlex author & \texttt{authorships[].\allowbreak author.\allowbreak id} \\ | ||||
| 324 | +\texttt{author\_\allowbreak name} & string & Author display name & \texttt{authorships[].\allowbreak author.\allowbreak display\_\allowbreak name} \\ | ||||
| 325 | +\texttt{orcid} & string & ORCID URL when known & \texttt{authorships[].\allowbreak author.\allowbreak orcid} \\ | ||||
| 326 | +\texttt{is\_\allowbreak corresponding} & bool & Corresponding-author flag & \texttt{authorships[].\allowbreak is\_\allowbreak corresponding} \\ | ||||
| 327 | +\texttt{institution\_\allowbreak ids} & list<int64> & Institutions of this authorship, in order & \texttt{authorships[].\allowbreak institutions[].\allowbreak id} \\ | ||||
| 328 | +\texttt{institution\_\allowbreak names} & list<string> & Their display names & \texttt{...institutions[].\allowbreak display\_\allowbreak name} \\ | ||||
| 329 | +\texttt{institution\_\allowbreak types} & list<string> & Their types & \texttt{...institutions[].\allowbreak type} \\ | ||||
| 330 | +\texttt{countries} & list<string> & Country codes of this authorship & \texttt{authorships[].\allowbreak countries} \\ | ||||
| 331 | +\texttt{n\_\allowbreak raw\_\allowbreak affiliations} & int32 & Number of raw affiliation strings & length of \texttt{authorships[].\allowbreak raw\_\allowbreak affiliation\_\allowbreak strings} \\ | ||||
| 332 | +\end{longtable} | ||||
| 333 | +\end{footnotesize} | ||||
| 334 | +\end{singlespace} | ||||
| 335 | + | ||||
| 336 | +\begin{singlespace} | ||||
| 337 | +\begin{footnotesize} | ||||
| 338 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 339 | +\caption{Small one-to-many tables of the CS extraction: \texttt{work\_\allowbreak topics}, \texttt{work\_\allowbreak keywords}, \texttt{references}, \texttt{abstracts}, \texttt{grants}, \texttt{counts\_\allowbreak by\_\allowbreak year} (views \texttt{oa\_\allowbreak <table>}). Every table carries \texttt{work\_\allowbreak id} (int64) as its first column.} | ||||
| 340 | +\label{tab:appE-small}\\ | ||||
| 341 | +\toprule | ||||
| 342 | +Column & Type & Definition & Source \\ | ||||
| 343 | +\midrule | ||||
| 344 | +\endfirsthead | ||||
| 345 | +\caption[]{Small one-to-many tables (continued).}\\ | ||||
| 346 | +\toprule | ||||
| 347 | +Column & Type & Definition & Source \\ | ||||
| 348 | +\midrule | ||||
| 349 | +\endhead | ||||
| 350 | +\midrule | ||||
| 351 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 352 | +\endfoot | ||||
| 353 | +\bottomrule | ||||
| 354 | +\endlastfoot | ||||
| 355 | +\multicolumn{4}{@{}l}{\textit{\texttt{work\_\allowbreak topics}: one row per topic of the work, at most three}} \\ | ||||
| 356 | +\texttt{rank} & int16 & Zero-based position in the topic list (0 = primary) & list index \\ | ||||
| 357 | +\texttt{topic\_\allowbreak id} & int32 & Topic & \texttt{topics[].\allowbreak id} \\ | ||||
| 358 | +\texttt{score} & float32 & OpenAlex topic score & \texttt{topics[].\allowbreak score} \\ | ||||
| 359 | +\texttt{subfield\_\allowbreak id}, \texttt{field\_\allowbreak id}, \texttt{domain\_\allowbreak id} & int32 & Hierarchy of the topic & \texttt{topics[].\allowbreak subfield.\allowbreak id}, \texttt{.field.\allowbreak id}, \texttt{.domain.\allowbreak id} \\ | ||||
| 360 | +\addlinespace | ||||
| 361 | +\multicolumn{4}{@{}l}{\textit{\texttt{work\_\allowbreak keywords}: one row per OpenAlex keyword}} \\ | ||||
| 362 | +\texttt{keyword} & string & Keyword display name & \texttt{keywords[].\allowbreak display\_\allowbreak name} \\ | ||||
| 363 | +\texttt{score} & float32 & Keyword score & \texttt{keywords[].\allowbreak score} \\ | ||||
| 364 | +\addlinespace | ||||
| 365 | +\multicolumn{4}{@{}l}{\textit{\texttt{references}: one row per referenced work (the citation graph)}} \\ | ||||
| 366 | +\texttt{ref\_\allowbreak id} & int64 & Cited work identifier; may lie outside the CS extraction & \texttt{referenced\_\allowbreak works[]} \\ | ||||
| 367 | +\addlinespace | ||||
| 368 | +\multicolumn{4}{@{}l}{\textit{\texttt{abstracts}: one row per work with an abstract}} \\ | ||||
| 369 | +\texttt{abstract} & string & Abstract text rebuilt by sorting the inverted index by position and joining with spaces & \texttt{abstract\_\allowbreak inverted\_\allowbreak index} \\ | ||||
| 370 | +\texttt{n\_\allowbreak words} & int32 & Number of whitespace-separated tokens & derived \\ | ||||
| 371 | +\addlinespace | ||||
| 372 | +\multicolumn{4}{@{}l}{\textit{\texttt{grants}: one row per grant entry}} \\ | ||||
| 373 | +\texttt{funder\_\allowbreak id} & int64 & Funder & \texttt{grants[].\allowbreak funder} \\ | ||||
| 374 | +\texttt{funder\_\allowbreak name} & string & Funder name & \texttt{grants[].\allowbreak funder\_\allowbreak display\_\allowbreak name} \\ | ||||
| 375 | +\texttt{award\_\allowbreak id} & string & Award identifier & \texttt{grants[].\allowbreak award\_\allowbreak id} \\ | ||||
| 376 | +\addlinespace | ||||
| 377 | +\multicolumn{4}{@{}l}{\textit{\texttt{counts\_\allowbreak by\_\allowbreak year}: one row per (work, calendar year) with citations received}} \\ | ||||
| 378 | +\texttt{year} & int16 & Calendar year of the citing works & \texttt{counts\_\allowbreak by\_\allowbreak year[].\allowbreak year} \\ | ||||
| 379 | +\texttt{cited\_\allowbreak by\_\allowbreak count} & int32 & Citations received in that year & \texttt{counts\_\allowbreak by\_\allowbreak year[].\allowbreak cited\_\allowbreak by\_\allowbreak count} \\ | ||||
| 380 | +\end{longtable} | ||||
| 381 | +\end{footnotesize} | ||||
| 382 | +\end{singlespace} | ||||
| 383 | + | ||||
| 384 | +\begin{singlespace} | ||||
| 385 | +\begin{footnotesize} | ||||
| 386 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 387 | +\caption{\texttt{locations} (view \texttt{oa\_\allowbreak locations}): one row per hosting location of the work.} | ||||
| 388 | +\label{tab:appE-locations}\\ | ||||
| 389 | +\toprule | ||||
| 390 | +Column & Type & Definition & Source \\ | ||||
| 391 | +\midrule | ||||
| 392 | +\endfirsthead | ||||
| 393 | +\caption[]{\texttt{locations} (continued).}\\ | ||||
| 394 | +\toprule | ||||
| 395 | +Column & Type & Definition & Source \\ | ||||
| 396 | +\midrule | ||||
| 397 | +\endhead | ||||
| 398 | +\midrule | ||||
| 399 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 400 | +\endfoot | ||||
| 401 | +\bottomrule | ||||
| 402 | +\endlastfoot | ||||
| 403 | +\texttt{work\_\allowbreak id} & int64 & Work & \texttt{id} of the work \\ | ||||
| 404 | +\texttt{rank} & int16 & Zero-based position in the location list & list index \\ | ||||
| 405 | +\texttt{is\_\allowbreak primary} & bool & Landing-page and PDF URLs equal those of \texttt{primary\_\allowbreak location} & derived \\ | ||||
| 406 | +\texttt{is\_\allowbreak best\_\allowbreak oa} & bool & Landing-page URL equals that of \texttt{best\_\allowbreak oa\_\allowbreak location} & derived \\ | ||||
| 407 | +\texttt{source\_\allowbreak id}, \texttt{source\_\allowbreak name}, \texttt{source\_\allowbreak type} & int64, string & Venue of the location & \texttt{locations[].\allowbreak source.\allowbreak id}, \texttt{.display\_\allowbreak name}, \texttt{.type} \\ | ||||
| 408 | +\texttt{landing\_\allowbreak page\_\allowbreak url}, \texttt{pdf\_\allowbreak url} & string & URLs & \texttt{locations[].\allowbreak landing\_\allowbreak page\_\allowbreak url}, \texttt{.pdf\_\allowbreak url} \\ | ||||
| 409 | +\texttt{is\_\allowbreak oa} & bool & Location is open access & \texttt{locations[].\allowbreak is\_\allowbreak oa} \\ | ||||
| 410 | +\texttt{license} & string & Licence of this copy & \texttt{locations[].\allowbreak license} \\ | ||||
| 411 | +\texttt{version} & string & submittedVersion, acceptedVersion or publishedVersion & \texttt{locations[].\allowbreak version} \\ | ||||
| 412 | +\texttt{is\_\allowbreak accepted}, \texttt{is\_\allowbreak published} & bool & Version flags & \texttt{locations[].\allowbreak is\_\allowbreak accepted}, \texttt{.is\_\allowbreak published} \\ | ||||
| 413 | +\end{longtable} | ||||
| 414 | +\end{footnotesize} | ||||
| 415 | +\end{singlespace} | ||||
| 416 | + | ||||
| 417 | +\subsection{Key tables and linkage tables} | ||||
| 418 | +\label{app:datadict-linkage} | ||||
| 419 | + | ||||
| 420 | +\texttt{link\_\allowbreak sources.\allowbreak py} derives from each source a \emph{key table} that carries only the identifiers used | ||||
| 421 | +for matching, in the shared conventions of Section~\ref{app:datadict-conventions}, and then builds three | ||||
| 422 | +linkage tables by the cascade of Section~\ref{sec:m-cascade}. In every key table \texttt{ntitle} is the title | ||||
| 423 | +in lower case with every run of characters other than \texttt{a--z} and \texttt{0--9} removed, and | ||||
| 424 | +\texttt{tlen} is the length of the original title; title matching requires \texttt{tlen} $\ge 25$ and a | ||||
| 425 | +normalised title that is unique on both sides. | ||||
| 426 | + | ||||
| 427 | +\begin{singlespace} | ||||
| 428 | +\begin{footnotesize} | ||||
| 429 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 430 | +\caption{Key tables (DuckDB tables \texttt{oa\_\allowbreak keys}, \texttt{dblp\_\allowbreak keys}, \texttt{arxiv\_\allowbreak keys}, \texttt{acl\_\allowbreak keys}).} | ||||
| 431 | +\label{tab:appE-keys}\\ | ||||
| 432 | +\toprule | ||||
| 433 | +Column & Type & Definition & Source \\ | ||||
| 434 | +\midrule | ||||
| 435 | +\endfirsthead | ||||
| 436 | +\caption[]{Key tables (continued).}\\ | ||||
| 437 | +\toprule | ||||
| 438 | +Column & Type & Definition & Source \\ | ||||
| 439 | +\midrule | ||||
| 440 | +\endhead | ||||
| 441 | +\midrule | ||||
| 442 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 443 | +\endfoot | ||||
| 444 | +\bottomrule | ||||
| 445 | +\endlastfoot | ||||
| 446 | +\multicolumn{4}{@{}l}{\textit{\texttt{oa\_\allowbreak keys}: every work of \texttt{oa\_\allowbreak works} (both passes)}} \\ | ||||
| 447 | +\texttt{oa\_\allowbreak id} & BIGINT & Work identifier & \texttt{oa\_\allowbreak works.\allowbreak id} \\ | ||||
| 448 | +\texttt{doi} & VARCHAR & DOI & \texttt{oa\_\allowbreak works.\allowbreak doi} \\ | ||||
| 449 | +\texttt{arxiv\_\allowbreak id} & VARCHAR & arXiv identifier, lower case, version suffix removed & \texttt{oa\_\allowbreak works.\allowbreak arxiv\_\allowbreak id} \\ | ||||
| 450 | +\texttt{year} & INTEGER & Publication year & \texttt{oa\_\allowbreak works.\allowbreak publication\_\allowbreak year} \\ | ||||
| 451 | +\texttt{ntitle}, \texttt{tlen} & VARCHAR, INTEGER & Normalised title and original title length & \texttt{oa\_\allowbreak works.\allowbreak title} \\ | ||||
| 452 | +\texttt{type} & VARCHAR & OpenAlex type & \texttt{oa\_\allowbreak works.\allowbreak type} \\ | ||||
| 453 | +\addlinespace | ||||
| 454 | +\multicolumn{4}{@{}l}{\textit{\texttt{dblp\_\allowbreak keys}: DBLP records with year 2018--2026 and type article, inproceedings, incollection, book, phdthesis or data}} \\ | ||||
| 455 | +\texttt{dblp\_\allowbreak key} & VARCHAR & DBLP key & \texttt{dblp\_\allowbreak raw.\allowbreak key} \\ | ||||
| 456 | +\texttt{rec\_\allowbreak type}, \texttt{year} & VARCHAR, INTEGER & Record type and year & \texttt{dblp\_\allowbreak raw} \\ | ||||
| 457 | +\texttt{doi} & VARCHAR & DOI & \texttt{dblp\_\allowbreak raw.\allowbreak doi} \\ | ||||
| 458 | +\texttt{arxiv\_\allowbreak id} & VARCHAR & arXiv identifier extracted from an \texttt{arxiv.\allowbreak org/\allowbreak abs/\allowbreak } URL among \texttt{ee}, version removed; null when none & \texttt{dblp\_\allowbreak raw.\allowbreak ee} \\ | ||||
| 459 | +\texttt{ntitle}, \texttt{tlen} & VARCHAR, INTEGER & Normalised title and length & \texttt{dblp\_\allowbreak raw.\allowbreak title} \\ | ||||
| 460 | +\texttt{stream} & VARCHAR & Venue stream & \texttt{dblp\_\allowbreak raw.\allowbreak stream} \\ | ||||
| 461 | +\addlinespace | ||||
| 462 | +\multicolumn{4}{@{}l}{\textit{\texttt{arxiv\_\allowbreak keys}: every record of \texttt{arxiv\_\allowbreak raw}}} \\ | ||||
| 463 | +\texttt{arxiv\_\allowbreak id} & VARCHAR & Identifier, lower case & \texttt{arxiv\_\allowbreak raw.\allowbreak arxiv\_\allowbreak id} \\ | ||||
| 464 | +\texttt{created}, \texttt{created\_\allowbreak year} & VARCHAR, INTEGER & First-submission date and its year & \texttt{arxiv\_\allowbreak raw.\allowbreak created} \\ | ||||
| 465 | +\texttt{doi} & VARCHAR & Declared DOI & \texttt{arxiv\_\allowbreak raw.\allowbreak doi} \\ | ||||
| 466 | +\texttt{primary\_\allowbreak category}, \texttt{categories} & VARCHAR, list & Categories & \texttt{arxiv\_\allowbreak raw} \\ | ||||
| 467 | +\texttt{ntitle} & VARCHAR & Normalised title & \texttt{arxiv\_\allowbreak raw.\allowbreak title} \\ | ||||
| 468 | +\addlinespace | ||||
| 469 | +\multicolumn{4}{@{}l}{\textit{\texttt{acl\_\allowbreak keys}: entries of type inproceedings or article}} \\ | ||||
| 470 | +\texttt{acl\_\allowbreak id} & VARCHAR & Anthology identifier & \texttt{acl\_\allowbreak raw.\allowbreak acl\_\allowbreak id} \\ | ||||
| 471 | +\texttt{year} & INTEGER & Year (\texttt{try\_\allowbreak cast}; null if not numeric) & \texttt{acl\_\allowbreak raw.\allowbreak year} \\ | ||||
| 472 | +\texttt{doi} & VARCHAR & DOI, lower case & \texttt{acl\_\allowbreak raw.\allowbreak doi} \\ | ||||
| 473 | +\texttt{url}, \texttt{ntitle}, \texttt{entry\_\allowbreak type} & VARCHAR & URL, normalised title, entry type & \texttt{acl\_\allowbreak raw} \\ | ||||
| 474 | +\end{longtable} | ||||
| 475 | +\end{footnotesize} | ||||
| 476 | +\end{singlespace} | ||||
| 477 | + | ||||
| 478 | +\begin{singlespace} | ||||
| 479 | +\begin{footnotesize} | ||||
| 480 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.0cm} p{4.6cm}@{}} | ||||
| 481 | +\caption{Linkage tables and the second-pass identifier list. Each linkage table has one row per matched pair; the cascade inserts a pair only when neither side is already linked by an earlier method (for \texttt{link\_\allowbreak arxiv\_\allowbreak oa} and \texttt{link\_\allowbreak acl\_\allowbreak oa}, when the source-side key is not already linked and, for title matches, the OpenAlex side is not either).} | ||||
| 482 | +\label{tab:appE-links}\\ | ||||
| 483 | +\toprule | ||||
| 484 | +Column & Type & Definition & Source \\ | ||||
| 485 | +\midrule | ||||
| 486 | +\endfirsthead | ||||
| 487 | +\caption[]{Linkage tables (continued).}\\ | ||||
| 488 | +\toprule | ||||
| 489 | +Column & Type & Definition & Source \\ | ||||
| 490 | +\midrule | ||||
| 491 | +\endhead | ||||
| 492 | +\midrule | ||||
| 493 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 494 | +\endfoot | ||||
| 495 | +\bottomrule | ||||
| 496 | +\endlastfoot | ||||
| 497 | +\multicolumn{4}{@{}l}{\textit{\texttt{link\_\allowbreak dblp\_\allowbreak oa}}} \\ | ||||
| 498 | +\texttt{dblp\_\allowbreak key}, \texttt{oa\_\allowbreak id} & VARCHAR, BIGINT & Matched DBLP record and OpenAlex work & key tables \\ | ||||
| 499 | +\texttt{method} & VARCHAR & \texttt{doi} (equal DOIs); \texttt{arxiv} (equal arXiv identifiers, both sides unlinked); \texttt{title\_\allowbreak year} (unique normalised titles, $|\Delta\text{year}| \le 1$, and equal first-author surname when both sides have one) & cascade \\ | ||||
| 500 | +\addlinespace | ||||
| 501 | +\multicolumn{4}{@{}l}{\textit{\texttt{link\_\allowbreak arxiv\_\allowbreak oa}}} \\ | ||||
| 502 | +\texttt{arxiv\_\allowbreak id}, \texttt{oa\_\allowbreak id} & VARCHAR, BIGINT & Matched arXiv record and OpenAlex work & key tables \\ | ||||
| 503 | +\texttt{method} & VARCHAR & \texttt{arxiv\_\allowbreak id} (equal identifiers); \texttt{doi}; \texttt{title\_\allowbreak year} (unique titles, $|\Delta\text{year}| \le 2$ between arXiv creation year and OpenAlex year) & cascade \\ | ||||
| 504 | +\addlinespace | ||||
| 505 | +\multicolumn{4}{@{}l}{\textit{\texttt{link\_\allowbreak acl\_\allowbreak oa}}} \\ | ||||
| 506 | +\texttt{acl\_\allowbreak id}, \texttt{oa\_\allowbreak id} & VARCHAR, BIGINT & Matched ACL entry and OpenAlex work & key tables \\ | ||||
| 507 | +\texttt{method} & VARCHAR & \texttt{doi}; \texttt{url} (an OpenAlex location whose landing page is \texttt{aclanthology.\allowbreak org/\allowbreak <acl\_\allowbreak id>}); \texttt{title\_\allowbreak year} (unique titles, $|\Delta\text{year}| \le 1$) & cascade, \texttt{oa\_\allowbreak locations} \\ | ||||
| 508 | +\addlinespace | ||||
| 509 | +\multicolumn{4}{@{}l}{\textit{\texttt{needed\_\allowbreak oa\_\allowbreak ids}: identifiers extracted in the second snapshot pass}} \\ | ||||
| 510 | +\texttt{oa\_\allowbreak id} & BIGINT & Work present in \texttt{oa\_\allowbreak slim} but absent from the CS extraction & \texttt{oa\_\allowbreak slim.\allowbreak id} \\ | ||||
| 511 | +\texttt{why} & VARCHAR & \texttt{dblp\_\allowbreak doi} (DOI of an unlinked DBLP record) or \texttt{arxiv\_\allowbreak doi} (DOI of an unlinked arXiv record) & derivation \\ | ||||
| 512 | +\end{longtable} | ||||
| 513 | +\end{footnotesize} | ||||
| 514 | +\end{singlespace} | ||||
| 515 | + | ||||
| 516 | +The linkage report (\texttt{data/\allowbreak metadata/\allowbreak linkage\_\allowbreak report.\allowbreak json}) records, for each source, the number of | ||||
| 517 | +keys, the number of keys with a DOI or an arXiv identifier, the links by method, the number of distinct | ||||
| 518 | +linked keys, the unlinked records by type, the unlinked DBLP records whose DOI nevertheless resolves in the | ||||
| 519 | +slim table (the trigger for the second pass), the identifiers needed for the second pass by reason, the size | ||||
| 520 | +of the slim table and the number of slim rows with a CS topic in 2018--2026. These are the quantities quoted | ||||
| 521 | +in Part~III through the \texttt{\textbackslash Link*} macros. | ||||
| 522 | + | ||||
| 523 | +\subsection{The corpus table} | ||||
| 524 | +\label{app:datadict-corpus} | ||||
| 525 | + | ||||
| 526 | +\texttt{define\_\allowbreak corpus.\allowbreak py} creates \texttt{corpus\_\allowbreak works} (also exported to | ||||
| 527 | +\texttt{data/\allowbreak processed/\allowbreak corpus/\allowbreak corpus\_\allowbreak works.\allowbreak parquet}) from \texttt{oa\_\allowbreak works}, adding the membership, | ||||
| 528 | +dating, duplicate and eligibility columns of Table~\ref{tab:appE-corpus}. The constants are those of the | ||||
| 529 | +protocol: the observation window is 2018-01-01 to 2026-11-02 with the boundary 2022-11-30; the kept types | ||||
| 530 | +are article, conference-paper, preprint, book-chapter, book, dissertation, report, review and software-paper; | ||||
| 531 | +the hard-excluded types, which exclude even a DBLP-linked record, are editorial, erratum, retraction, | ||||
| 532 | +peer-review, paratext, conference-abstract, reference-entry, dataset, software, supplementary-materials, | ||||
| 533 | +libguides, letter, grant and standard; the generic deposit platforms of amendment A-002 are Zenodo, Figshare, | ||||
| 534 | +IRDB, LA~Referencia, Dialnet, DOAJ-as-source and OSTI. Every row of \texttt{oa\_\allowbreak works} is present, including | ||||
| 535 | +excluded ones, so that PRISMA counts can be recomputed from the flags. | ||||
| 536 | + | ||||
| 537 | +\begin{singlespace} | ||||
| 538 | +\begin{footnotesize} | ||||
| 539 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.6cm} p{4.0cm}@{}} | ||||
| 540 | +\caption{\texttt{corpus\_\allowbreak works}: columns added to those of \texttt{works} (Table~\ref{tab:appE-works}).} | ||||
| 541 | +\label{tab:appE-corpus}\\ | ||||
| 542 | +\toprule | ||||
| 543 | +Column & Type & Definition & Source \\ | ||||
| 544 | +\midrule | ||||
| 545 | +\endfirsthead | ||||
| 546 | +\caption[]{\texttt{corpus\_\allowbreak works} (continued).}\\ | ||||
| 547 | +\toprule | ||||
| 548 | +Column & Type & Definition & Source \\ | ||||
| 549 | +\midrule | ||||
| 550 | +\endhead | ||||
| 551 | +\midrule | ||||
| 552 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 553 | +\endfoot | ||||
| 554 | +\bottomrule | ||||
| 555 | +\endlastfoot | ||||
| 556 | +\texttt{ntitle} & VARCHAR & Normalised title & \texttt{title} \\ | ||||
| 557 | +\texttt{in\_\allowbreak dblp} & BOOLEAN & The work is linked to a DBLP record & \texttt{link\_\allowbreak dblp\_\allowbreak oa} \\ | ||||
| 558 | +\texttt{in\_\allowbreak acl} & BOOLEAN & The work is linked to an ACL Anthology entry & \texttt{link\_\allowbreak acl\_\allowbreak oa} \\ | ||||
| 559 | +\texttt{arxiv\_\allowbreak id\_\allowbreak linked} & VARCHAR & arXiv identifier of a linked arXiv record (any one, if several) & \texttt{link\_\allowbreak arxiv\_\allowbreak oa}, \texttt{arxiv\_\allowbreak keys} \\ | ||||
| 560 | +\texttt{arxiv\_\allowbreak created} & VARCHAR & Earliest first-submission date among linked arXiv records & \texttt{arxiv\_\allowbreak keys.\allowbreak created} \\ | ||||
| 561 | +\texttt{arxiv\_\allowbreak primary} & VARCHAR & Primary arXiv category of a linked record & \texttt{arxiv\_\allowbreak keys.\allowbreak primary\_\allowbreak category} \\ | ||||
| 562 | +\texttt{arxiv\_\allowbreak cs\_\allowbreak primary} & BOOLEAN & \texttt{arxiv\_\allowbreak primary} starts with \texttt{cs.\allowbreak }; null when no linked record & derived \\ | ||||
| 563 | +\texttt{is\_\allowbreak arxiv\_\allowbreak doi} & BOOLEAN & The DOI is an arXiv DOI (\texttt{10.\allowbreak 48550/\allowbreak arxiv.\allowbreak }) & \texttt{doi} \\ | ||||
| 564 | +\texttt{duplicate\_\allowbreak of} & BIGINT & For a preprint-like record that has a published twin: identifier of the twin; null otherwise & \texttt{dup\_\allowbreak pairs} \\ | ||||
| 565 | +\texttt{preprint\_\allowbreak work\_\allowbreak id} & BIGINT & For a published record that has a preprint twin: identifier of the preprint & \texttt{dup\_\allowbreak pairs} \\ | ||||
| 566 | +\texttt{first\_\allowbreak date} & VARCHAR & First public appearance: the earliest of the publication date, the twin preprint's date and the arXiv creation date; when the publication date is year-only (\texttt{-01-01}) and a preprint or arXiv date exists, the earliest of those two instead & derived (eq.~\ref{eq:first-date}) \\ | ||||
| 567 | +\texttt{version\_\allowbreak relationship} & VARCHAR & On the published twin: \texttt{title\_\allowbreak year} or \texttt{arxiv\_\allowbreak id}, the relation by which the pair was found & \texttt{dup\_\allowbreak pairs.\allowbreak relation} \\ | ||||
| 568 | +\texttt{e1\_\allowbreak outside\_\allowbreak window} & BOOLEAN & Publication date null, before 2018-01-01 or after 2026-11-02 & \texttt{publication\_\allowbreak date} \\ | ||||
| 569 | +\texttt{e2\_\allowbreak type} & BOOLEAN & Type not in the kept list, unless DBLP-linked and not hard-excluded & \texttt{type}, \texttt{in\_\allowbreak dblp} \\ | ||||
| 570 | +\texttt{e3\_\allowbreak xpac} & BOOLEAN & XPAC record not linked to DBLP & \texttt{is\_\allowbreak xpac}, \texttt{in\_\allowbreak dblp} \\ | ||||
| 571 | +\texttt{xpac\_\allowbreak kept\_\allowbreak dblp} & BOOLEAN & XPAC record kept because DBLP-linked (A-001) & \texttt{is\_\allowbreak xpac}, \texttt{in\_\allowbreak dblp} \\ | ||||
| 572 | +\texttt{e7\_\allowbreak generic\_\allowbreak deposit} & BOOLEAN & Primary source is a generic deposit platform and the record is neither DBLP- nor arXiv-linked (A-002) & \texttt{source\_\allowbreak name} \\ | ||||
| 573 | +\texttt{date\_\allowbreak precision} & VARCHAR & \texttt{year} when the publication date is \texttt{YYYY-01-01} and no preprint or arXiv date exists; \texttt{month} otherwise & derived \\ | ||||
| 574 | +\texttt{e4\_\allowbreak no\_\allowbreak title} & BOOLEAN & Title null or blank & \texttt{title} \\ | ||||
| 575 | +\texttt{e5\_\allowbreak duplicate} & BOOLEAN & The record is the preprint member of a duplicate pair & \texttt{duplicate\_\allowbreak of} \\ | ||||
| 576 | +\texttt{e6\_\allowbreak retracted} & BOOLEAN & Retracted (flag, not an exclusion: retracted works stay eligible and are excluded from citation indicators) & \texttt{is\_\allowbreak retracted} \\ | ||||
| 577 | +\texttt{cs\_\allowbreak core} & BOOLEAN & DBLP-linked, or CS primary topic, or CS topic with a \texttt{cs.\allowbreak } arXiv primary category & derived \\ | ||||
| 578 | +\texttt{cs\_\allowbreak penumbra} & BOOLEAN & Any CS topic but not \texttt{cs\_\allowbreak core} & derived \\ | ||||
| 579 | +\texttt{eligible} & BOOLEAN & Not E1--E5, not E7, and \texttt{cs\_\allowbreak core} & derived (eq.~\ref{eq:eligible}) \\ | ||||
| 580 | +\texttt{period} & VARCHAR & \texttt{pre} (before 2018-01-01), \texttt{baseline} (before 2022-11-30), \texttt{post} (to 2026-11-02), \texttt{future}, by \texttt{first\_\allowbreak date} & derived \\ | ||||
| 581 | +\texttt{first\_\allowbreak month} & VARCHAR & \texttt{YYYY-MM} of \texttt{first\_\allowbreak date} & derived \\ | ||||
| 582 | +\end{longtable} | ||||
| 583 | +\end{footnotesize} | ||||
| 584 | +\end{singlespace} | ||||
| 585 | + | ||||
| 586 | +Two auxiliary tables accompany it. \texttt{dup\_\allowbreak pairs} (columns \texttt{preprint\_\allowbreak id}, \texttt{published\_\allowbreak id}, | ||||
| 587 | +\texttt{preprint\_\allowbreak date}, \texttt{published\_\allowbreak date}, \texttt{relation}) holds the one-to-one preprint-to-published | ||||
| 588 | +pairs after the ranking columns \texttt{rn\_\allowbreak pub} and \texttt{rn\_\allowbreak pre} have been used to keep, for each | ||||
| 589 | +published work, the earliest preprint, and for each preprint, the earliest published twin; a candidate pair | ||||
| 590 | +is a preprint-like record (type preprint or arXiv DOI) and a non-preprint record in the window that passes | ||||
| 591 | +E1--E4, with equal normalised titles of at least 25 characters and years at most two apart | ||||
| 592 | +(\texttt{title\_\allowbreak year}), or with the same arXiv identifier (\texttt{arxiv\_\allowbreak id}). \texttt{dblp\_\allowbreak only\_\allowbreak counts} | ||||
| 593 | +(\texttt{year}, \texttt{rec\_\allowbreak type}, \texttt{n}, \texttt{with\_\allowbreak doi}) counts the DBLP records of 2018--2026 that | ||||
| 594 | +have no OpenAlex record after both passes. The PRISMA file \texttt{prisma\_\allowbreak counts.\allowbreak json} stores the identified | ||||
| 595 | +records per source, the links, each exclusion flag counted alone and sequentially in the order E1, E2, E3, | ||||
| 596 | +E4, E5, E7, the remainder, the eligible counts by period, year, type, membership and date precision, and the | ||||
| 597 | +strict core (DBLP-linked and CS primary); its keys are the \texttt{\textbackslash Prisma*} macros of the | ||||
| 598 | +manuscript. | ||||
| 599 | + | ||||
| 600 | +\subsection{Citation windows} | ||||
| 601 | +\label{app:datadict-citations} | ||||
| 602 | + | ||||
| 603 | +\texttt{bibliometrics/\allowbreak 03\_\allowbreak citation\_\allowbreak impact.\allowbreak py} builds \texttt{work\_\allowbreak citation\_\allowbreak windows} (also | ||||
| 604 | +\texttt{data/\allowbreak processed/\allowbreak corpus/\allowbreak work\_\allowbreak citation\_\allowbreak windows.\allowbreak parquet}) for the eligible, non-retracted works. The | ||||
| 605 | +yearly citation counts of a published work and of its merged preprint twin are summed, so that a citation | ||||
| 606 | +to either version counts for the canonical node (Section~\ref{sec:m-citations}); fixed windows are then | ||||
| 607 | +formed from calendar years, and each window is normalised by the mean of the same window over works of the | ||||
| 608 | +same publication year, type and primary subfield. | ||||
| 609 | + | ||||
| 610 | +\begin{singlespace} | ||||
| 611 | +\begin{footnotesize} | ||||
| 612 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.6cm} p{4.0cm}@{}} | ||||
| 613 | +\caption{\texttt{work\_\allowbreak citation\_\allowbreak windows}: one row per eligible, non-retracted work.} | ||||
| 614 | +\label{tab:appE-citwin}\\ | ||||
| 615 | +\toprule | ||||
| 616 | +Column & Type & Definition & Source \\ | ||||
| 617 | +\midrule | ||||
| 618 | +\endfirsthead | ||||
| 619 | +\caption[]{\texttt{work\_\allowbreak citation\_\allowbreak windows} (continued).}\\ | ||||
| 620 | +\toprule | ||||
| 621 | +Column & Type & Definition & Source \\ | ||||
| 622 | +\midrule | ||||
| 623 | +\endhead | ||||
| 624 | +\midrule | ||||
| 625 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 626 | +\endfoot | ||||
| 627 | +\bottomrule | ||||
| 628 | +\endlastfoot | ||||
| 629 | +\texttt{work\_\allowbreak id} & BIGINT & Work & \texttt{corpus\_\allowbreak works.\allowbreak id} \\ | ||||
| 630 | +\texttt{year} & INTEGER & Publication year $y$ & \texttt{publication\_\allowbreak year} \\ | ||||
| 631 | +\texttt{type} & VARCHAR & OpenAlex type, \texttt{unknown} when null & \texttt{type} \\ | ||||
| 632 | +\texttt{sf} & INTEGER & Primary subfield, $-1$ when null & \texttt{primary\_\allowbreak subfield\_\allowbreak id} \\ | ||||
| 633 | +\texttt{cited\_\allowbreak by\_\allowbreak count} & INTEGER & All-time citations at snapshot date (the canonical record only) & \texttt{cited\_\allowbreak by\_\allowbreak count} \\ | ||||
| 634 | +\texttt{fwci} & DOUBLE & OpenAlex FWCI (secondary indicator) & \texttt{fwci} \\ | ||||
| 635 | +\texttt{c12} & BIGINT & Citations received in years $\le y+1$ (publication year and the following year), work plus merged preprint & \texttt{oa\_\allowbreak counts\_\allowbreak by\_\allowbreak year} \\ | ||||
| 636 | +\texttt{c36} & BIGINT & Citations received in years $\le y+2$ & idem \\ | ||||
| 637 | +\texttt{c48} & BIGINT & Citations received in years $\le y+3$ & idem \\ | ||||
| 638 | +\texttt{c12\_\allowbreak complete}, \texttt{c36\_\allowbreak complete}, \texttt{c48\_\allowbreak complete} & BOOLEAN & The window closes in a complete calendar year: $y+1 \le 2025$, $y+2 \le 2025$, $y+3 \le 2025$ & derived \\ | ||||
| 639 | +\texttt{ncs12}, \texttt{ncs36} & DOUBLE & \texttt{c12}, \texttt{c36} divided by the mean of the same quantity over works with the same \texttt{year}, \texttt{type} and \texttt{sf}; null when that mean is zero & derived (eq.~\ref{eq:ncs}) \\ | ||||
| 640 | +\end{longtable} | ||||
| 641 | +\end{footnotesize} | ||||
| 642 | +\end{singlespace} | ||||
| 643 | + | ||||
| 644 | +The accompanying result tables are \texttt{citation\_\allowbreak windows\_\allowbreak yearly.\allowbreak csv} (per year: $n$, completeness | ||||
| 645 | +flags, means and median of the windows, shares uncited at 12 and 36 months, the 99th percentile and maximum | ||||
| 646 | +of \texttt{c36}, Gini coefficients of \texttt{c36} and \texttt{c12}, mean and median FWCI), | ||||
| 647 | +\texttt{top\_\allowbreak cited\_\allowbreak by\_\allowbreak year.\allowbreak csv} (the fifty works with the largest \texttt{c36} per year, with title, venue, | ||||
| 648 | +type and the window counts) and \texttt{fwci\_\allowbreak summary\_\allowbreak yearly.\allowbreak csv}. | ||||
| 649 | + | ||||
| 650 | +\subsection{Citation-graph edge tables} | ||||
| 651 | +\label{app:datadict-graph} | ||||
| 652 | + | ||||
| 653 | +\texttt{knowledge\_\allowbreak graph/\allowbreak 01\_\allowbreak citation\_\allowbreak graph.\allowbreak py} splits the references of eligible works into two edge | ||||
| 654 | +tables according to whether the cited work is itself eligible. External edges are resolved through the slim | ||||
| 655 | +table, which is why the slim table covers the whole snapshot: a reference to a work outside computer | ||||
| 656 | +science is attributed a year, a type and a field without that work ever being extracted in full. References | ||||
| 657 | +whose target is in neither table (identifiers absent from the snapshot) appear in neither edge table and | ||||
| 658 | +are counted only through \texttt{n\_\allowbreak refs}. | ||||
| 659 | + | ||||
| 660 | +\begin{singlespace} | ||||
| 661 | +\begin{footnotesize} | ||||
| 662 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.6cm} p{4.0cm}@{}} | ||||
| 663 | +\caption{Edge tables \texttt{cit\_\allowbreak edges\_\allowbreak internal} and \texttt{cit\_\allowbreak edges\_\allowbreak external} (\texttt{data/\allowbreak processed/\allowbreak graph/\allowbreak }).} | ||||
| 664 | +\label{tab:appE-edges}\\ | ||||
| 665 | +\toprule | ||||
| 666 | +Column & Type & Definition & Source \\ | ||||
| 667 | +\midrule | ||||
| 668 | +\endfirsthead | ||||
| 669 | +\caption[]{Edge tables (continued).}\\ | ||||
| 670 | +\toprule | ||||
| 671 | +Column & Type & Definition & Source \\ | ||||
| 672 | +\midrule | ||||
| 673 | +\endhead | ||||
| 674 | +\midrule | ||||
| 675 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 676 | +\endfoot | ||||
| 677 | +\bottomrule | ||||
| 678 | +\endlastfoot | ||||
| 679 | +\multicolumn{4}{@{}l}{\textit{\texttt{cit\_\allowbreak edges\_\allowbreak internal}: both ends eligible}} \\ | ||||
| 680 | +\texttt{citing\_\allowbreak id}, \texttt{cited\_\allowbreak id} & BIGINT & Citing and cited work & \texttt{oa\_\allowbreak references} \\ | ||||
| 681 | +\texttt{citing\_\allowbreak year}, \texttt{cited\_\allowbreak year} & INTEGER & Publication years of the two ends & \texttt{corpus\_\allowbreak works.\allowbreak publication\_\allowbreak year} \\ | ||||
| 682 | +\addlinespace | ||||
| 683 | +\multicolumn{4}{@{}l}{\textit{\texttt{cit\_\allowbreak edges\_\allowbreak external}: citing work eligible, cited work not eligible but present in the snapshot}} \\ | ||||
| 684 | +\texttt{citing\_\allowbreak id}, \texttt{cited\_\allowbreak id} & BIGINT & Citing and cited work & \texttt{oa\_\allowbreak references} \\ | ||||
| 685 | +\texttt{citing\_\allowbreak year} & INTEGER & Year of the citing work & \texttt{corpus\_\allowbreak works} \\ | ||||
| 686 | +\texttt{cited\_\allowbreak year} & INTEGER & Year of the cited work & \texttt{oa\_\allowbreak slim.\allowbreak publication\_\allowbreak year} \\ | ||||
| 687 | +\texttt{cited\_\allowbreak field\_\allowbreak id} & INTEGER & Primary field of the cited work & \texttt{oa\_\allowbreak slim.\allowbreak primary\_\allowbreak field\_\allowbreak id} \\ | ||||
| 688 | +\texttt{cited\_\allowbreak type} & VARCHAR & Type of the cited work & \texttt{oa\_\allowbreak slim.\allowbreak type} \\ | ||||
| 689 | +\texttt{cited\_\allowbreak cs\_\allowbreak any} & BOOLEAN & The cited work has a CS topic (it is then a non-eligible CS record, e.g.\ outside the window) & \texttt{oa\_\allowbreak slim.\allowbreak cs\_\allowbreak any\_\allowbreak topic} \\ | ||||
| 690 | +\end{longtable} | ||||
| 691 | +\end{footnotesize} | ||||
| 692 | +\end{singlespace} | ||||
| 693 | + | ||||
| 694 | +The derived result tables are \texttt{field\_\allowbreak flows\_\allowbreak yearly.\allowbreak csv} (\texttt{citing\_\allowbreak year}, \texttt{kind} | ||||
| 695 | +internal or external, \texttt{cited\_\allowbreak field\_\allowbreak id}, \texttt{n}), \texttt{cited\_\allowbreak age\_\allowbreak yearly.\allowbreak csv} | ||||
| 696 | +(\texttt{citing\_\allowbreak year}, \texttt{age} = citing year minus cited year, \texttt{n}, over both tables) and | ||||
| 697 | +\texttt{reference\_\allowbreak coverage\_\allowbreak yearly.\allowbreak csv} (\texttt{year}, \texttt{n\_\allowbreak works}, \texttt{n\_\allowbreak refs\_\allowbreak declared}, | ||||
| 698 | +\texttt{n\_\allowbreak internal}, \texttt{n\_\allowbreak external}). Category-level flows, co-classification and modularity tables | ||||
| 699 | +(\texttt{knowledge\_\allowbreak graph/\allowbreak 02\_\allowbreak category\_\allowbreak flows.\allowbreak py}) join the internal edges with the classification tables | ||||
| 700 | +below; the semantic edges of \texttt{04\_\allowbreak semantic\_\allowbreak edges.\allowbreak py} (\texttt{work\_\allowbreak id}, \texttt{neighbour\_\allowbreak id}, | ||||
| 701 | +\texttt{cosine\_\allowbreak similarity}, \texttt{rank}) are computed from the embeddings of | ||||
| 702 | +Section~\ref{app:datadict-embeddings}; and the extracted method, dataset and claim nodes and typed edges of | ||||
| 703 | +\texttt{03\_\allowbreak extract\_\allowbreak entities.\allowbreak py} carry, for every edge, the work, the quoted span from the abstract, a | ||||
| 704 | +model-reported confidence and a verification status (\texttt{unverified}, \texttt{machine-verified} when | ||||
| 705 | +the span is found verbatim, \texttt{human-verified} after review), as the mandate requires. | ||||
| 706 | + | ||||
| 707 | +\subsection{Classification tables} | ||||
| 708 | +\label{app:datadict-classification} | ||||
| 709 | + | ||||
| 710 | +\texttt{taxonomy/\allowbreak classify.\allowbreak py} first exports its input (\texttt{--prepare}) and then scores it | ||||
| 711 | +(\texttt{--run}), with the ontology of Appendix~\ref{app:taxonomy} and the rule of | ||||
| 712 | +equations~\eqref{eq:score}--\eqref{eq:assign}. The population is the eligible corpus together with the | ||||
| 713 | +penumbra, so that sensitivity analyses on the wider set need no second run. | ||||
| 714 | + | ||||
| 715 | +\begin{singlespace} | ||||
| 716 | +\begin{footnotesize} | ||||
| 717 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.6cm} p{4.0cm}@{}} | ||||
| 718 | +\caption{Classification input and output tables (\texttt{data/\allowbreak processed/\allowbreak classification/\allowbreak }).} | ||||
| 719 | +\label{tab:appE-classification}\\ | ||||
| 720 | +\toprule | ||||
| 721 | +Column & Type & Definition & Source \\ | ||||
| 722 | +\midrule | ||||
| 723 | +\endfirsthead | ||||
| 724 | +\caption[]{Classification tables (continued).}\\ | ||||
| 725 | +\toprule | ||||
| 726 | +Column & Type & Definition & Source \\ | ||||
| 727 | +\midrule | ||||
| 728 | +\endhead | ||||
| 729 | +\midrule | ||||
| 730 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 731 | +\endfoot | ||||
| 732 | +\bottomrule | ||||
| 733 | +\endlastfoot | ||||
| 734 | +\multicolumn{4}{@{}l}{\textit{\texttt{input/\allowbreak chunk=N/\allowbreak *.\allowbreak parquet}: one row per work with \texttt{eligible} or \texttt{cs\_\allowbreak penumbra}}} \\ | ||||
| 735 | +\texttt{id} & BIGINT & Work & \texttt{corpus\_\allowbreak works.\allowbreak id} \\ | ||||
| 736 | +\texttt{year} & INTEGER & Publication year & \texttt{publication\_\allowbreak year} \\ | ||||
| 737 | +\texttt{title}, \texttt{abstract} & VARCHAR & Text scored by the keyword rules (abstract null when absent) & \texttt{corpus\_\allowbreak works}, \texttt{oa\_\allowbreak abstracts} \\ | ||||
| 738 | +\texttt{topic\_\allowbreak ids} & list<int> & Topic identifiers in rank order & \texttt{oa\_\allowbreak work\_\allowbreak topics} \\ | ||||
| 739 | +\texttt{subfield\_\allowbreak ids} & list<int> & Subfield identifiers in rank order & \texttt{oa\_\allowbreak work\_\allowbreak topics} \\ | ||||
| 740 | +\texttt{arxiv\_\allowbreak cats} & list<string> & All categories of all linked arXiv records, flattened & \texttt{link\_\allowbreak arxiv\_\allowbreak oa}, \texttt{arxiv\_\allowbreak keys.\allowbreak categories} \\ | ||||
| 741 | +\texttt{dblp\_\allowbreak streams} & list<string> & Distinct streams of the linked DBLP records & \texttt{link\_\allowbreak dblp\_\allowbreak oa}, \texttt{dblp\_\allowbreak keys.\allowbreak stream} \\ | ||||
| 742 | +\texttt{chunk} & INTEGER & Chunk index (default 100{,}000 rows per chunk) & derived \\ | ||||
| 743 | +\addlinespace | ||||
| 744 | +\multicolumn{4}{@{}l}{\textit{\texttt{work\_\allowbreak categories} (\texttt{cats/\allowbreak *.\allowbreak parquet}): one row per (work, assigned category)}} \\ | ||||
| 745 | +\texttt{work\_\allowbreak id} & int64 & Work & input \\ | ||||
| 746 | +\texttt{category\_\allowbreak id} & string & Leaf identifier (Table~\ref{tab:appA-leaves}) & ontology \\ | ||||
| 747 | +\texttt{score} & float32 & Total score $s_k(w)$, rounded to three decimals & eq.~\eqref{eq:score} \\ | ||||
| 748 | +\texttt{s\_\allowbreak topic}, \texttt{s\_\allowbreak subfield}, \texttt{s\_\allowbreak arxiv}, \texttt{s\_\allowbreak dblp}, \texttt{s\_\allowbreak kw} & float32 & Capped contribution of each signal type (each in $[0,1]$) & eq.~\eqref{eq:score} \\ | ||||
| 749 | +\addlinespace | ||||
| 750 | +\multicolumn{4}{@{}l}{\textit{\texttt{work\_\allowbreak primary} (\texttt{primary/\allowbreak *.\allowbreak parquet}): one row per work with at least one category}} \\ | ||||
| 751 | +\texttt{work\_\allowbreak id} & int64 & Work & input \\ | ||||
| 752 | +\texttt{primary\_\allowbreak category} & string & Argmax category with the tie-breaking of Section~\ref{sec:m-score} & derived \\ | ||||
| 753 | +\texttt{n\_\allowbreak categories} & int16 & Number of assigned categories $|C(w)|$ & derived \\ | ||||
| 754 | +\texttt{primary\_\allowbreak score} & float32 & Score of the primary category & derived \\ | ||||
| 755 | +\end{longtable} | ||||
| 756 | +\end{footnotesize} | ||||
| 757 | +\end{singlespace} | ||||
| 758 | + | ||||
| 759 | +A work with no category has no row in either output table; the number of such works is the difference | ||||
| 760 | +between the \texttt{works} and \texttt{classified} fields of \texttt{classification\_\allowbreak report.\allowbreak json}, which | ||||
| 761 | +also records the ontology version and the timestamp of the run. The validation tables | ||||
| 762 | +(\texttt{taxonomy/\allowbreak validation/\allowbreak sample.\allowbreak parquet}, \texttt{ratings\_\allowbreak <model>.\allowbreak jsonl}, \texttt{metrics.\allowbreak json}, | ||||
| 763 | +\texttt{disagreements.\allowbreak csv}) carry, per sampled work, the rule categories and primary, and per rater the label | ||||
| 764 | +set, primary, confidence and rationale returned under the JSON schema of \texttt{VALIDATION.\allowbreak md}. | ||||
| 765 | + | ||||
| 766 | +\subsection{Embedding shards} | ||||
| 767 | +\label{app:datadict-embeddings} | ||||
| 768 | + | ||||
| 769 | +\texttt{embeddings/\allowbreak embed\_\allowbreak corpus.\allowbreak py} exports the text to embed (\texttt{--prepare}) and encodes it | ||||
| 770 | +(\texttt{--run}) with a sentence-embedding model, by default SPECTER2 | ||||
| 771 | +\citep{cohan2020specter,singh2023scirepeval} as served through \texttt{sentence-transformers} | ||||
| 772 | +\citep{reimers2019sbert}, in shards that can be distributed over nodes. | ||||
| 773 | + | ||||
| 774 | +\begin{singlespace} | ||||
| 775 | +\begin{footnotesize} | ||||
| 776 | +\begin{longtable}{@{}p{3.2cm} p{1.6cm} p{6.6cm} p{4.0cm}@{}} | ||||
| 777 | +\caption{Embedding input and output files (\texttt{data/\allowbreak processed/\allowbreak embeddings/\allowbreak }).} | ||||
| 778 | +\label{tab:appE-embeddings}\\ | ||||
| 779 | +\toprule | ||||
| 780 | +File or column & Type & Definition & Source \\ | ||||
| 781 | +\midrule | ||||
| 782 | +\endfirsthead | ||||
| 783 | +\caption[]{Embedding files (continued).}\\ | ||||
| 784 | +\toprule | ||||
| 785 | +File or column & Type & Definition & Source \\ | ||||
| 786 | +\midrule | ||||
| 787 | +\endhead | ||||
| 788 | +\midrule | ||||
| 789 | +\multicolumn{4}{r}{continued on next page}\\ | ||||
| 790 | +\endfoot | ||||
| 791 | +\bottomrule | ||||
| 792 | +\endlastfoot | ||||
| 793 | +\multicolumn{4}{@{}l}{\textit{\texttt{input/\allowbreak shard=K/\allowbreak *.\allowbreak parquet}: eligible works with an abstract, ordered by identifier, 50{,}000 per shard by default}} \\ | ||||
| 794 | +\texttt{work\_\allowbreak id} & BIGINT & Work & \texttt{corpus\_\allowbreak works.\allowbreak id} \\ | ||||
| 795 | +\texttt{text} & VARCHAR & Title, a full stop and a space, then the first 2{,}000 characters of the abstract & \texttt{title}, \texttt{oa\_\allowbreak abstracts.\allowbreak abstract} \\ | ||||
| 796 | +\texttt{shard} & INTEGER & Shard index $K$ & derived \\ | ||||
| 797 | +\addlinespace | ||||
| 798 | +\multicolumn{4}{@{}l}{\textit{\texttt{<model\_\allowbreak tag>/\allowbreak shard\_\allowbreak KKKK.\allowbreak npy}}} \\ | ||||
| 799 | +array & float16, $n \times d$ & One L2-normalised embedding per text, in input order; $d$ is the model's output dimension; maximum sequence length 256 tokens, batch 128 by default & model output \\ | ||||
| 800 | +\addlinespace | ||||
| 801 | +\multicolumn{4}{@{}l}{\textit{\texttt{<model\_\allowbreak tag>/\allowbreak shard\_\allowbreak KKKK.\allowbreak ids.\allowbreak npy}}} \\ | ||||
| 802 | +array & int64, $n$ & The \texttt{work\_\allowbreak id} of each row of the embedding array & input \\ | ||||
| 803 | +\addlinespace | ||||
| 804 | +\multicolumn{4}{@{}l}{\textit{\texttt{<model\_\allowbreak tag>/\allowbreak manifest\_\allowbreak <host>.\allowbreak json}}} \\ | ||||
| 805 | +\texttt{model}, \texttt{max\_\allowbreak seq\_\allowbreak length}, \texttt{device}, \texttt{host} & -- & Model name, sequence limit, \texttt{mps} or \texttt{cpu}, host name & run parameters \\ | ||||
| 806 | +\texttt{shards[K]} & -- & Per shard: number of texts, seconds taken, timestamp & run \\ | ||||
| 807 | +\end{longtable} | ||||
| 808 | +\end{footnotesize} | ||||
| 809 | +\end{singlespace} | ||||
| 810 | + | ||||
| 811 | +The model tag is the model name with \texttt{/\allowbreak } replaced by \texttt{\_\allowbreak \_\allowbreak }; a second model produces a second | ||||
| 812 | +directory with the same layout, which is how the comparison of embedding approaches required by the mandate | ||||
| 813 | +is organised. Shards already present are skipped on resumption, so a shard computed on one node and copied | ||||
| 814 | +to another is never recomputed. | ||||
modified
manuscript/appendices/appH_reproducibility.tex
+315 −1
@@ -1 +1,315 @@ | |||||
| 1 | −% appH_reproducibility — to be written (Phase 7) | 1 | +% ============================================================================ | ||
| 2 | +% Appendix H — Reproducibility | ||||
| 3 | +% Written by Agent L (appendices A/E/H instance), 2026-11-03. Sources of truth: reproducibility/RUNBOOK.md, | ||||
| 4 | +% reproducibility/environment_M3U96b.txt, data/raw/README.md, reports/INCIDENTS.md, protocol/amendments.csv, | ||||
| 5 | +% protocol/PROTOCOL.md §P8, README.md, .gitignore, ingestion/*.py (ledger schemas). Every number here is a | ||||
| 6 | +% provenance fact (file counts, sizes, wall times, versions) taken from those files; no corpus result appears. | ||||
| 7 | +% ============================================================================ | ||||
| 8 | + | ||||
| 9 | +\section{Reproducibility} | ||||
| 10 | +\label{app:repro} | ||||
| 11 | + | ||||
| 12 | +The mandate requires that the corpus be reproducible from immutable raw snapshots and that every quantitative | ||||
| 13 | +claim trace to a script, an input artefact and a result file. This appendix records what a reader needs to | ||||
| 14 | +re-run the pipeline: the computing environment and the locked package versions, the ordered steps with their | ||||
| 15 | +commands, inputs, outputs and measured wall times, the mechanisms by which provenance is kept (manifests, | ||||
| 16 | +hash ledgers, verbatim raw pages, API ledgers), the incidents that occurred during the first run and what | ||||
| 17 | +they imply for a second one, the protocol amendments, and the licence of each source together with what is | ||||
| 18 | +and is not redistributed. The runbook (\texttt{reproducibility/\allowbreak RUNBOOK.\allowbreak md}) is the operational version of | ||||
| 19 | +this appendix and is updated whenever a step changes; where the two differ, the runbook is more recent. | ||||
| 20 | + | ||||
| 21 | +\subsection{Environment} | ||||
| 22 | +\label{app:repro-env} | ||||
| 23 | + | ||||
| 24 | +Everything described in the runbook was executed on one node of the MacLustr cluster, M3U96b (Mac Studio, | ||||
| 25 | +Apple M3~Ultra, 28~cores, 96~GB of unified memory, 103{,}079{,}215{,}104 bytes as recorded by the environment | ||||
| 26 | +capture), in the working directory \texttt{\textasciitilde/\allowbreak wp44\_\allowbreak uqo}, under Python 3.12.15 in a virtual | ||||
| 27 | +environment created with \texttt{uv} from \texttt{requirements.\allowbreak txt} and then frozen to | ||||
| 28 | +\texttt{reproducibility/\allowbreak environment\_\allowbreak M3U96b.\allowbreak txt} (92 lines: 88 pinned packages, the interpreter version, the | ||||
| 29 | +operating-system version string, the processor and the memory size). The operating-system version string | ||||
| 30 | +captured in that file is 26.4, whereas the runbook header states macOS~27.0; the discrepancy is recorded here | ||||
| 31 | +rather than resolved, since the node was updated between captures and neither value affects the computation. | ||||
| 32 | +Two further nodes were used after the incidents of Section~\ref{app:repro-incidents}: M5M36 for the second | ||||
| 33 | +snapshot pass and M4M64c and M3U96a for embedding shards; they run the same locked environment. | ||||
| 34 | +Table~\ref{tab:appH-packages} lists the packages on which results depend; the complete list is the lock file. | ||||
| 35 | +No step requires credentials except the two LLM raters of the taxonomy validation and the knowledge-graph | ||||
| 36 | +extraction (an Anthropic API key read from a file outside the repository, never stored in it). | ||||
| 37 | + | ||||
| 38 | +\begin{table}[htbp] | ||||
| 39 | +\begin{singlespace} | ||||
| 40 | +\centering | ||||
| 41 | +\caption{Key packages of the locked environment (\texttt{reproducibility/\allowbreak environment\_\allowbreak M3U96b.\allowbreak txt}, Python 3.12.15).} | ||||
| 42 | +\label{tab:appH-packages} | ||||
| 43 | +\small | ||||
| 44 | +\begin{tabularx}{\textwidth}{l X l} | ||||
| 45 | +\toprule | ||||
| 46 | +Role & Package & Version \\ | ||||
| 47 | +\midrule | ||||
| 48 | +Analytical database & \texttt{duckdb} \citep{raasveldt2019duckdb} & 2.0.0 \\ | ||||
| 49 | +Columnar I/O & \texttt{pyarrow} & 26.0.0 \\ | ||||
| 50 | +Data frames & \texttt{polars} / \texttt{pandas} & 2.1.0 / 3.1.0 \\ | ||||
| 51 | +Numerics & \texttt{numpy} / \texttt{scipy} & 2.5.4 / 1.18.1 \\ | ||||
| 52 | +Statistics, time series & \texttt{statsmodels} / \texttt{formulaic} & 0.15.0 / 1.2.2 \\ | ||||
| 53 | +Change-point detection & \texttt{ruptures} \citep{truong2020ruptures} & 1.1.10 \\ | ||||
| 54 | +Machine learning & \texttt{scikit-learn} & 1.9.1 \\ | ||||
| 55 | +Embeddings & \texttt{torch} / \texttt{transformers} / \texttt{sentence-transformers} \citep{reimers2019sbert} & 2.15.0 / 5.22.1 / 6.2.0 \\ | ||||
| 56 | +Model hub client & \texttt{huggingface-hub} & 2.4.0 \\ | ||||
| 57 | +Dimensionality reduction, clustering & \texttt{umap-learn} / \texttt{hdbscan} / \texttt{pynndescent} & 0.5.12 / 0.8.44 / 0.7.0 \\ | ||||
| 58 | +Networks & \texttt{networkx} / \texttt{igraph} & 3.7 / 1.0.0 \\ | ||||
| 59 | +Parsing & \texttt{lxml} / \texttt{orjson} / \texttt{zstandard} / \texttt{regex} & 6.1.3 / 3.13.0 / 0.25.0 / 2026.9.29 \\ | ||||
| 60 | +Ontology file & \texttt{pyyaml} & 6.0.3 \\ | ||||
| 61 | +HTTP & \texttt{requests} / \texttt{httpx2} & 2.34.2 / 2.16.0 \\ | ||||
| 62 | +LLM raters, extraction & \texttt{anthropic} & 2.4.0 \\ | ||||
| 63 | +Figures & \texttt{matplotlib} & 3.11.2 \\ | ||||
| 64 | +Tests & \texttt{pytest} & 9.2.0 \\ | ||||
| 65 | +\bottomrule | ||||
| 66 | +\end{tabularx} | ||||
| 67 | +\end{singlespace} | ||||
| 68 | +\end{table} | ||||
| 69 | + | ||||
| 70 | +\subsection{Steps, commands and measured times} | ||||
| 71 | +\label{app:repro-steps} | ||||
| 72 | + | ||||
| 73 | +Table~\ref{tab:appH-steps} reproduces the runbook's step table. Commands are given relative to the repository | ||||
| 74 | +root with the virtual environment active; \texttt{\ldots} abbreviates arguments that are spelled out in the | ||||
| 75 | +runbook. Wall times are those measured during the first run and are reported as orientation, not as | ||||
| 76 | +guarantees: the snapshot download depends on network throughput (about 108~MB/s was sustained from the node | ||||
| 77 | +with eight parallel streams), and the steps marked ``to be measured'' had not completed at the time the | ||||
| 78 | +runbook was last updated because of the node freezes described below. Steps~1--5 are pure acquisition and | ||||
| 79 | +can be run independently and in any order; steps 6--10 are sequential; step~9 is a second acquisition pass | ||||
| 80 | +whose need is discovered at step~8 and which then repeats steps 6--8 and 10--11 with the suffix \texttt{\_\allowbreak p2} | ||||
| 81 | +(\texttt{ingestion/\allowbreak run\_\allowbreak pass2\_\allowbreak chain.\allowbreak sh}, which waits for a flag file written when the pass-2 outputs have | ||||
| 82 | +been copied back to the node). The API increment of Table~\ref{tab:appH-raw} (works published after the | ||||
| 83 | +snapshot cut-off) is converted to the snapshot's part format by \texttt{ingestion/\allowbreak ingest\_\allowbreak api\_\allowbreak increment.\allowbreak py} | ||||
| 84 | +and normalised by the same step~6; it is not yet a numbered step of the runbook. | ||||
| 85 | + | ||||
| 86 | +\begin{singlespace} | ||||
| 87 | +\begin{footnotesize} | ||||
| 88 | +\begin{longtable}{@{}p{0.5cm} p{5.6cm} p{2.3cm} p{4.6cm} p{2.3cm}@{}} | ||||
| 89 | +\caption{Pipeline steps (\texttt{reproducibility/\allowbreak RUNBOOK.\allowbreak md}), with the command, the inputs, the outputs and the wall time measured on M3U96b.} | ||||
| 90 | +\label{tab:appH-steps}\\ | ||||
| 91 | +\toprule | ||||
| 92 | +\# & Step and command & Inputs & Outputs & Wall time \\ | ||||
| 93 | +\midrule | ||||
| 94 | +\endfirsthead | ||||
| 95 | +\caption[]{Pipeline steps (continued).}\\ | ||||
| 96 | +\toprule | ||||
| 97 | +\# & Step and command & Inputs & Outputs & Wall time \\ | ||||
| 98 | +\midrule | ||||
| 99 | +\endhead | ||||
| 100 | +\midrule | ||||
| 101 | +\multicolumn{5}{r}{continued on next page}\\ | ||||
| 102 | +\endfoot | ||||
| 103 | +\bottomrule | ||||
| 104 | +\endlastfoot | ||||
| 105 | +1 & OpenAlex works snapshot, pass~1 (CS filter + slim table): \texttt{python ingestion/\allowbreak openalex\_\allowbreak snapshot.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex/\allowbreak works --manifest data/\allowbreak raw/\allowbreak openalex/\allowbreak works\_\allowbreak manifest\_\allowbreak 2026-10-14.\allowbreak json --workers 8} & public S3 bucket, release 2026-10-14 & \texttt{works/\allowbreak cs/\allowbreak *.\allowbreak jsonl.\allowbreak zst} (15{,}490{,}159 records), \texttt{works/\allowbreak slim/\allowbreak *.\allowbreak parquet} (464{,}765{,}485 rows), \texttt{ledger.\allowbreak sqlite} & 1~h 46~min at $\approx$108~MB/s \\ | ||||
| 106 | +2 & OpenAlex entities: \texttt{python ingestion/\allowbreak openalex\_\allowbreak entities.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex} & S3 & \texttt{topics/\allowbreak }, \ldots, \texttt{institutions/\allowbreak }; \texttt{entities\_\allowbreak ledger.\allowbreak json} & 5~min \\ | ||||
| 107 | +3 & DBLP: \texttt{python ingestion/\allowbreak dblp\_\allowbreak parse.\allowbreak py --xml data/\allowbreak raw/\allowbreak dblp/\allowbreak dblp.\allowbreak xml.\allowbreak gz --out data/\allowbreak raw/\allowbreak dblp} & dump of 2026-11-02, MD5 verified & \texttt{dblp\_\allowbreak publications.\allowbreak parquet} (8{,}840{,}866 records), parse report & 4~min 45~s \\ | ||||
| 108 | +4 & arXiv OAI-PMH: \texttt{python ingestion/\allowbreak arxiv\_\allowbreak oai.\allowbreak py --out data/\allowbreak raw/\allowbreak arxiv --from 2018-01-01 --until 2026-11-02} & OAI feed, set \texttt{cs} & raw XML pages (758), \texttt{arxiv\_\allowbreak cs.\allowbreak parquet} (915{,}115 records) & $\approx$1~h \\ | ||||
| 109 | +5 & ACL Anthology: \texttt{python ingestion/\allowbreak acl\_\allowbreak parse.\allowbreak py --bib data/\allowbreak raw/\allowbreak acl/\allowbreak anthology+abstracts.\allowbreak bib.\allowbreak gz --out data/\allowbreak raw/\allowbreak acl} & BibTeX export & \texttt{acl\_\allowbreak anthology.\allowbreak parquet} (138{,}796 entries) & 10~s \\ | ||||
| 110 | +6 & Normalise: \texttt{python ingestion/\allowbreak build\_\allowbreak corpus.\allowbreak py --cs-dir data/\allowbreak raw/\allowbreak openalex/\allowbreak works/\allowbreak cs --out data/\allowbreak processed --workers 12} & step 1 & \texttt{tables/\allowbreak <table>/\allowbreak *.\allowbreak parquet} & 2~min 10~s \\ | ||||
| 111 | +7 & Compact: \texttt{ulimit -n 65536; python ingestion/\allowbreak compact\_\allowbreak tables.\allowbreak py \textasciitilde/\allowbreak wp44\_\allowbreak uqo} & step 6 & \texttt{compact/\allowbreak <table>.\allowbreak parquet} & 30~s \\ | ||||
| 112 | +8 & Link: \texttt{python ingestion/\allowbreak link\_\allowbreak sources.\allowbreak py --root \textasciitilde/\allowbreak wp44\_\allowbreak uqo} & steps 3--7 & \texttt{wp44.\allowbreak duckdb} with \texttt{link\_\allowbreak *}; \texttt{data/\allowbreak metadata/\allowbreak linkage\_\allowbreak report.\allowbreak json}; \texttt{needed\_\allowbreak oa\_\allowbreak ids.\allowbreak csv} & 10~s \\ | ||||
| 113 | +9 & Second pass (DBLP- and arXiv-linked works outside the CS-topic filter): \texttt{python ingestion/\allowbreak openalex\_\allowbreak snapshot.\allowbreak py --out data/\allowbreak raw/\allowbreak openalex/\allowbreak works\_\allowbreak pass2 --manifest \ldots --workers 8 --id-list data/\allowbreak metadata/\allowbreak needed\_\allowbreak oa\_\allowbreak ids.\allowbreak csv}, then \texttt{ingestion/\allowbreak run\_\allowbreak pass2\_\allowbreak chain.\allowbreak sh} (steps 6--8 with suffix \texttt{\_\allowbreak p2}, then 10--11) & S3, step 8 & \texttt{works\_\allowbreak pass2/\allowbreak cs/\allowbreak *.\allowbreak jsonl.\allowbreak zst}, \texttt{compact/\allowbreak <table>\_\allowbreak p2.\allowbreak parquet} & $\approx$1~h 50~min \\ | ||||
| 114 | +10 & Corpus definition: \texttt{python systematic\_\allowbreak review/\allowbreak define\_\allowbreak corpus.\allowbreak py --root \textasciitilde/\allowbreak wp44\_\allowbreak uqo} & step 8 & \texttt{corpus\_\allowbreak works}, \texttt{data/\allowbreak metadata/\allowbreak prisma\_\allowbreak counts.\allowbreak json} & 2~min \\ | ||||
| 115 | +11 & Volume, collaboration, citations, graph: \texttt{bibliometrics/\allowbreak 01\_\allowbreak volume\_\allowbreak series.\allowbreak py}, \texttt{02\_\allowbreak collaboration.\allowbreak py}, \texttt{03\_\allowbreak citation\_\allowbreak impact.\allowbreak py}, \texttt{knowledge\_\allowbreak graph/\allowbreak 01\_\allowbreak citation\_\allowbreak graph.\allowbreak py} & step 10 & \texttt{results/\allowbreak bibliometrics/\allowbreak *.\allowbreak csv}, \texttt{results/\allowbreak graph/\allowbreak *}, \texttt{data/\allowbreak processed/\allowbreak graph/\allowbreak *} & minutes \\ | ||||
| 116 | +12 & Classification: \texttt{python taxonomy/\allowbreak classify.\allowbreak py --root \ldots --ontology taxonomy/\allowbreak ontology.\allowbreak yaml --prepare --run} (in practice \texttt{taxonomy/\allowbreak run\_\allowbreak classify\_\allowbreak shards.\allowbreak sh}, see incident I-3) & steps 8, 10; \texttt{ontology.\allowbreak yaml} & \texttt{classification/\allowbreak cats/\allowbreak *.\allowbreak parquet}, \texttt{classification/\allowbreak primary/\allowbreak *.\allowbreak parquet}, report & to be measured \\ | ||||
| 117 | +13 & Validation sample and raters: \texttt{python taxonomy/\allowbreak validate\_\allowbreak sample.\allowbreak py --root \ldots --draw}; \texttt{--rate --model claude-haiku-5-5}; \texttt{--rate --model claude-sonnet-5-5}; \texttt{--metrics} & step 12 & \texttt{taxonomy/\allowbreak validation/\allowbreak *} & to be measured \\ | ||||
| 118 | +14 & Embeddings: \texttt{python embeddings/\allowbreak embed\_\allowbreak corpus.\allowbreak py --root \ldots --prepare}; \texttt{--run --model allenai/\allowbreak specter2\_\allowbreak base} (sharded over nodes) & step 10 & \texttt{embeddings/\allowbreak <model>/\allowbreak shard\_\allowbreak *.\allowbreak npy} & $\approx$300 texts/s per M3~Ultra \\ | ||||
| 119 | +15 & Structural change, emerging detection, figures, tables: \texttt{econometrics/\allowbreak run\_\allowbreak structural\_\allowbreak change.\allowbreak py}, \texttt{experiments/\allowbreak emerging\_\allowbreak detection.\allowbreak py}, \texttt{figures/\allowbreak make\_\allowbreak figures.\allowbreak py}, \texttt{tables/\allowbreak make\_\allowbreak numbers.\allowbreak py} & steps 11--14 & \texttt{results/\allowbreak econometrics}, \texttt{results/\allowbreak emerging}, \texttt{figures/\allowbreak }, \texttt{manuscript/\allowbreak tables/\allowbreak numbers.\allowbreak tex} & minutes \\ | ||||
| 120 | +16 & Manuscript: \texttt{cd manuscript \&\& make} (\texttt{latexmk}) & all & \texttt{manuscript/\allowbreak main.\allowbreak pdf} & --- \\ | ||||
| 121 | +\end{longtable} | ||||
| 122 | +\end{footnotesize} | ||||
| 123 | +\end{singlespace} | ||||
| 124 | + | ||||
| 125 | +The test suite (\texttt{python -m pytest -q tests/\allowbreak }) covers the structural-change toolkit and the | ||||
| 126 | +classification scoring rule (\texttt{tests/\allowbreak test\_\allowbreak structural\_\allowbreak change.\allowbreak py}, \texttt{tests/\allowbreak test\_\allowbreak classify\_\allowbreak rules.\allowbreak py}); | ||||
| 127 | +the second fixes the implemented aggregation of equation~\eqref{eq:score} against the toy ontology it | ||||
| 128 | +declares, so that a change in \texttt{classify.\allowbreak py} that altered the rule would fail before any corpus were | ||||
| 129 | +classified. | ||||
| 130 | + | ||||
| 131 | +\subsection{Provenance mechanisms} | ||||
| 132 | +\label{app:repro-provenance} | ||||
| 133 | + | ||||
| 134 | +Provenance is kept at the level of files and jobs, not of individual records, by six mechanisms. | ||||
| 135 | + | ||||
| 136 | +\paragraph{Manifests and hash ledgers for the OpenAlex snapshot.} The snapshot's own manifest | ||||
| 137 | +(\texttt{works\_\allowbreak manifest\_\allowbreak 2026-10-14.\allowbreak json}) lists every part file with its content length and record count. | ||||
| 138 | +The ingestion script keeps a SQLite ledger (\texttt{works/\allowbreak ledger.\allowbreak sqlite}) with a \texttt{files} table (one | ||||
| 139 | +row per part: URL, local name, expected length, manifest record count, status, SHA-256 of the gzip bytes, | ||||
| 140 | +records parsed, records retained by the filter, lines that failed to parse, download and total time, number | ||||
| 141 | +of attempts, start and end timestamps, error text) and a \texttt{runs} table (start time, manifest date, | ||||
| 142 | +manifest record and byte totals, number of workers, host name and command line of every invocation). A | ||||
| 143 | +part is downloaded with resume and retries until its size equals the manifest's, hashed, parsed, and only | ||||
| 144 | +then marked done; outputs are written to a temporary name and renamed atomically, so an interrupted run | ||||
| 145 | +leaves no partial output that a resumed run would mistake for a complete one. The raw gzip files are deleted | ||||
| 146 | +after a successful parse unless \texttt{--keep-raw} is given: the snapshot is a public, versioned release, and | ||||
| 147 | +the manifest together with the per-file hashes allows any later copy to be verified against what was | ||||
| 148 | +ingested. For the first pass the ledger records 1{,}898 of 1{,}898 files done, 0 parse errors, and a sum of | ||||
| 149 | +per-file record counts equal to the manifest total. | ||||
| 150 | + | ||||
| 151 | +\paragraph{Entity ledger.} The small entity snapshots (topics, subfields, fields, domains, sources, | ||||
| 152 | +institutions, publishers, funders, concepts) are listed in \texttt{entities\_\allowbreak ledger.\allowbreak json} with, per file, the | ||||
| 153 | +entity, the local path, the byte count, the SHA-256 and the status. | ||||
| 154 | + | ||||
| 155 | +\paragraph{DBLP checksum.} The DBLP dump is distributed with an \texttt{.md5} file; the downloaded | ||||
| 156 | +\texttt{dblp.\allowbreak xml.\allowbreak gz} was verified against it (\texttt{ff1c424c3b4cef36582f647c36df27c0}). The parse report | ||||
| 157 | +(\texttt{dblp\_\allowbreak parse\_\allowbreak report.\allowbreak json}) additionally records the input path, its size in bytes, an MD5 of its | ||||
| 158 | +first GiB (the file is 1.12~GB, so this digest is not the published one and is a convenience for detecting a | ||||
| 159 | +changed input), the parse timestamp, per-type and per-year record counts, and timings. | ||||
| 160 | + | ||||
| 161 | +\paragraph{Verbatim OAI-PMH pages.} The arXiv harvest stores every XML response page as received under | ||||
| 162 | +\texttt{data/\allowbreak raw/\allowbreak arxiv/\allowbreak raw/\allowbreak <window>/\allowbreak page\_\allowbreak NNNN.\allowbreak xml} (758 pages for the monthly windows from 2018-01 to | ||||
| 163 | +2026-11), together with a harvest state file that allows resumption by window and a parse report; the | ||||
| 164 | +Parquet table can be rebuilt from the pages alone with \texttt{--parse-only}. Because OAI selects by record | ||||
| 165 | +\emph{datestamp}, a record updated in several windows appears in several pages; the parser keeps the latest. | ||||
| 166 | + | ||||
| 167 | +\paragraph{API ledger.} The OpenAlex API increment (works with a CS topic published between 2026-10-01 and | ||||
| 168 | +2026-11-02, fetched after the snapshot) stores every JSON page verbatim under \texttt{api\_\allowbreak increment/\allowbreak pages/\allowbreak } | ||||
| 169 | +and a ledger (\texttt{api\_\allowbreak increment/\allowbreak ledger.\allowbreak json}) with the slices requested (day $\times$ CS-primary or not), | ||||
| 170 | +the number of calls and the cost charged against the public budget; without an API key the harvest is paged | ||||
| 171 | +by day with at most 10{,}000 results per slice and spread over several days. | ||||
| 172 | + | ||||
| 173 | +\paragraph{Step reports and generated numbers.} Each processing step writes a machine-readable report: | ||||
| 174 | +\texttt{linkage\_\allowbreak report.\allowbreak json} (counts of keys, links by method, unlinked records by type, identifiers needed | ||||
| 175 | +for the second pass), \texttt{prisma\_\allowbreak counts.\allowbreak json} (every PRISMA stage, with an identity check that the | ||||
| 176 | +records loaded equal the sum of sequential exclusions and the remainder), \texttt{classification\_\allowbreak report.\allowbreak json} | ||||
| 177 | +(ontology version, works scored, works classified, timestamp), one embedding manifest per host (model, | ||||
| 178 | +maximum sequence length, device, and per shard the number of texts, the seconds taken and the timestamp), | ||||
| 179 | +and the validation metrics. The macros of \texttt{manuscript/\allowbreak tables/\allowbreak numbers.\allowbreak tex} are generated from these | ||||
| 180 | +files by \texttt{tables/\allowbreak make\_\allowbreak numbers.\allowbreak py}; a number whose source file is missing yields an undefined macro | ||||
| 181 | +and therefore a compilation error, which is intended: the prose cannot cite a number that no result file | ||||
| 182 | +contains. | ||||
| 183 | + | ||||
| 184 | +\subsection{Incident log and its implications} | ||||
| 185 | +\label{app:repro-incidents} | ||||
| 186 | + | ||||
| 187 | +Six incidents were recorded during the first run (\texttt{reports/\allowbreak INCIDENTS.\allowbreak md}); none caused data loss, | ||||
| 188 | +because every step is resumable from its ledger or its shard files, but three of them changed how the | ||||
| 189 | +pipeline must be run and are therefore part of the reproduction instructions. | ||||
| 190 | + | ||||
| 191 | +\begin{table}[htbp] | ||||
| 192 | +\begin{singlespace} | ||||
| 193 | +\centering | ||||
| 194 | +\caption{Incident log of the first run (\texttt{reports/\allowbreak INCIDENTS.\allowbreak md}; times in Eastern Time, 2026-11-03).} | ||||
| 195 | +\label{tab:appH-incidents} | ||||
| 196 | +\footnotesize | ||||
| 197 | +\begin{tabularx}{\textwidth}{l l X X} | ||||
| 198 | +\toprule | ||||
| 199 | +\# & When & What happened & Impact and recovery \\ | ||||
| 200 | +\midrule | ||||
| 201 | +I-1 & $\approx$15:45 & Second snapshot pass with \texttt{--id-list}: worker processes had an empty identifier set, because the macOS \texttt{spawn} start method does not inherit module globals & 68 part files marked in error, no data loss; a pool initialiser now loads the identifier set in each worker; ledger reset and rerun \\ | ||||
| 202 | +I-2 & $\approx$15:50 & DuckDB ``too many open files'' when reading 1{,}898 Parquet files per table & Linkage step failed; \texttt{compact\_\allowbreak tables.\allowbreak py} (one file per table) and \texttt{ulimit -n 65536} \\ | ||||
| 203 | +I-3 & $\approx$16:30 & \texttt{multiprocessing.\allowbreak Pool} with pyarrow in spawned workers hung (0\,\% CPU) in the classifier & Test stalled 10~min; shard mode: independent processes started by \texttt{run\_\allowbreak classify\_\allowbreak shards.\allowbreak sh} \\ | ||||
| 204 | +I-4 & $\approx$16:58 & M3U96b froze (no ping, SSH ``host is down'') while running the second pass (8 download-and-parse workers), the SPECTER2 embedding job on the GPU and earlier DuckDB jobs concurrently & Second pass ($\approx$1{,}400 of 1{,}898 files done), the chained corpus v2 and classification, and the node's embedding share interrupted; other embedding nodes unaffected; all jobs resumable; rule adopted: never run ingestion, embeddings and classification concurrently on one node \\ | ||||
| 205 | +I-5 & 17:16 & M3U96b froze again two minutes after the second pass was restarted alone on the rebooted node & Pass~2 moved to the idle node M5M36 (10 workers, ledger copied); M3U96b reserved for DuckDB and classification; the chain now waits for a flag raised when the pass-2 outputs are copied back; Apple diagnostics show performance auto-bug-capture reports, no kernel panic found \\ | ||||
| 206 | +I-6 & 17:17 & M4M64a froze a second time within two minutes of restarting its embedding share (batch 32); M3U96b rebooted in the same minute while running pass~2 alone & M4M64a embedding share lost again; M4M64a and M3U96b withdrawn from GPU and bulk-download work; shard ranges reassigned to M4M64c and M3U96a; pass~2 on M5M36 \\ | ||||
| 207 | +\bottomrule | ||||
| 208 | +\end{tabularx} | ||||
| 209 | +\end{singlespace} | ||||
| 210 | +\end{table} | ||||
| 211 | + | ||||
| 212 | +Three implications follow for anyone reproducing the work. First, the parallelism in the scripts assumes | ||||
| 213 | +the \texttt{spawn} start method of macOS: worker state must be built in initialisers, and the classifier | ||||
| 214 | +should be run as independent shard processes rather than through a pool (\texttt{classify.\allowbreak py --shard i/\allowbreak N}); | ||||
| 215 | +on Linux, where \texttt{fork} is the default, the pool mode may work but has not been exercised. Second, the | ||||
| 216 | +per-part Parquet layout exceeds default file-descriptor limits; the compaction step is mandatory, not an | ||||
| 217 | +optimisation. Third, the two freezes of the primary node occurred under concurrent download, parse, GPU | ||||
| 218 | +and database load, and their cause was not established from the system logs; the operational rule adopted | ||||
| 219 | +afterwards (one heavy job per node at a time) costs wall time but is the configuration under which the | ||||
| 220 | +reported outputs were produced, and the measured times of Table~\ref{tab:appH-steps} assume it. Because | ||||
| 221 | +every job writes its outputs atomically and records what it has completed, the state of the pipeline after | ||||
| 222 | +an interruption is recoverable from the ledgers alone, and the outputs of a resumed run are identical to | ||||
| 223 | +those of an uninterrupted one for all deterministic steps (ingestion, normalisation, linkage, corpus | ||||
| 224 | +definition, classification). The embedding step is the exception: shards were computed on several nodes and | ||||
| 225 | +devices, and the per-host manifests record for each shard where it was computed, so that any numerical | ||||
| 226 | +difference between devices can be traced; the embeddings are stored in half precision and are not claimed | ||||
| 227 | +to be bit-identical across hardware. | ||||
| 228 | + | ||||
| 229 | +\subsection{Protocol amendments} | ||||
| 230 | +\label{app:repro-amendments} | ||||
| 231 | + | ||||
| 232 | +The protocol was frozen on 2026-11-02 before any analysis of the corpus, and every later change is an | ||||
| 233 | +amendment with an identifier, a reason and an approver (\texttt{protocol/\allowbreak amendments.\allowbreak csv}). All six amendments | ||||
| 234 | +of Table~\ref{tab:appH-amendments} are approved by the executing agent and pending the principal | ||||
| 235 | +investigator's signature at the checkpoint of the corresponding phase; the manuscript marks the affected | ||||
| 236 | +conclusions accordingly. | ||||
| 237 | + | ||||
| 238 | +\begin{table}[htbp] | ||||
| 239 | +\begin{singlespace} | ||||
| 240 | +\centering | ||||
| 241 | +\caption{Protocol amendments (\texttt{protocol/\allowbreak amendments.\allowbreak csv}).} | ||||
| 242 | +\label{tab:appH-amendments} | ||||
| 243 | +\footnotesize | ||||
| 244 | +\begin{tabularx}{\textwidth}{l l l X X} | ||||
| 245 | +\toprule | ||||
| 246 | +Id & Date & Phase & Change & Reason \\ | ||||
| 247 | +\midrule | ||||
| 248 | +A-000 & 2026-11-02 & 0 & \S14 of the mandate completed from \S1--\S13 & The pasted mandate was truncated by the terminal \\ | ||||
| 249 | +A-001 & 2026-11-03 & 1 & Eligibility type list corrected: conference-paper and software-paper are kept; DBLP-linked records are kept whatever their type unless hard-excluded; XPAC records kept when DBLP-linked (\texttt{xpac\_\allowbreak kept\_\allowbreak dblp}) & The first run excluded 2.5~M conference-paper records (1.09~M DBLP-linked) through an outdated type list \\ | ||||
| 250 | +A-002 & 2026-11-03 & 1 & Records whose primary source is a generic deposit platform are excluded unless DBLP- or arXiv-linked (E7); \texttt{date\_\allowbreak precision} added, monthly series use month-precision records only & Zenodo alone contributed 138{,}533 otherwise eligible 2026 records (deposits of slides, datasets and reports typed as articles); 18--28\,\% of records carry a year-only date \\ | ||||
| 251 | +A-003 & 2026-11-03 & 2 & LLM rating budget for the taxonomy validation fixed at 25~USD (two raters, structured outputs) & Mandate \S7 requires agent-based validation with human checkpoints; the PI rates 200 works \\ | ||||
| 252 | +A-004 & 2026-11-03 & 3 & Knowledge-graph extraction from abstracts of the high-priority set with \texttt{claude-sonnet-5-5}, provenance spans and machine verification of spans; budget 150~USD & Mandate \S6.4 requires provenance, confidence and verification status on every inferred relationship \\ | ||||
| 253 | +A-005 & 2026-11-03 & 1 & Exclusion code E7 redefined as the generic-deposit exclusion; the CS penumbra is a membership flag, not an exclusion code & The Part~IV writer found the protocol and code labels inconsistent \\ | ||||
| 254 | +\bottomrule | ||||
| 255 | +\end{tabularx} | ||||
| 256 | +\end{singlespace} | ||||
| 257 | +\end{table} | ||||
| 258 | + | ||||
| 259 | +\subsection{Sources, licences and what is redistributed} | ||||
| 260 | +\label{app:repro-licences} | ||||
| 261 | + | ||||
| 262 | +Table~\ref{tab:appH-raw} lists each raw source as obtained, with its size, integrity record and licence. | ||||
| 263 | +Nothing under \texttt{data/\allowbreak raw/\allowbreak } or \texttt{data/\allowbreak processed/\allowbreak } is committed to the repository | ||||
| 264 | +(\texttt{.gitignore} excludes both, together with every Parquet and JSONL file, except the provenance | ||||
| 265 | +\texttt{README} and the metadata directory); the copies live on the compute node. What the released | ||||
| 266 | +repository contains is the code, the ontology and its audit table, the manifests, ledgers and hashes, the | ||||
| 267 | +PRISMA counts and the linkage report, the aggregated result tables, the figures, the generated number macros | ||||
| 268 | +and the manuscript (protocol \S P8). Record-level derived tables may be released where the source licence | ||||
| 269 | +permits: OpenAlex is CC0, DBLP is ODC-BY~1.0, arXiv metadata obtained through OAI-PMH is under the CC0 | ||||
| 270 | +terms of that feed, and the ACL Anthology is CC~BY~4.0; a derived record table therefore carries the | ||||
| 271 | +attribution required by ODC-BY and CC~BY for the DBLP and ACL fields it contains. Full texts are never | ||||
| 272 | +redistributed and were never obtained through paywalls. The Semantic Scholar Academic Graph was not used: | ||||
| 273 | +bulk access requires an API key that was not available and anonymous calls were refused, and the mandate's | ||||
| 274 | +instruction to record inaccessible data explicitly is satisfied by this note; influential-citation and | ||||
| 275 | +citation-intent signals derived from that graph are consequently absent from the analyses. | ||||
| 276 | + | ||||
| 277 | +\begin{table}[htbp] | ||||
| 278 | +\begin{singlespace} | ||||
| 279 | +\centering | ||||
| 280 | +\caption{Raw sources as obtained (\texttt{data/\allowbreak raw/\allowbreak README.\allowbreak md}), with integrity record and licence.} | ||||
| 281 | +\label{tab:appH-raw} | ||||
| 282 | +\footnotesize | ||||
| 283 | +\begin{tabularx}{\textwidth}{p{2.6cm} X p{3.6cm} p{2.0cm}} | ||||
| 284 | +\toprule | ||||
| 285 | +Source & What was obtained, when, size & Integrity & Licence \\ | ||||
| 286 | +\midrule | ||||
| 287 | +OpenAlex works snapshot, release 2026-10-14 & \texttt{data/\allowbreak jsonl/\allowbreak works/\allowbreak }: 1{,}898 part files, 464{,}765{,}485 works, 675.3~GB gzip; streamed 2026-11-02 21:19--23:05~ET over HTTPS with 8 parallel streams; retained: CS subset 20~GB (15{,}490{,}159 records) and slim table 9.3~GB (all works) & manifest; per-file content length and SHA-256 in \texttt{works/\allowbreak ledger.\allowbreak sqlite} (1{,}898/1{,}898 ok, 0 parse errors, record total = manifest total) & CC0 \\ | ||||
| 288 | +OpenAlex entity snapshots & topics 4{,}516; subfields 252; fields 26; domains 4; sources 268{,}629; institutions 143{,}455; publishers 10{,}713; funders 41{,}799; concepts 65{,}026; 2026-11-02; 0.6~GB & \texttt{entities\_\allowbreak ledger.\allowbreak json} (SHA-256 per file) & CC0 \\ | ||||
| 289 | +OpenAlex API increment & works with any CS topic, publication date 2026-10-01 to 2026-11-02, sliced by day and CS-primary status, basic paging $\le$10{,}000 per slice; from 2026-11-03 (public budget 0.10~USD/day, several days) & raw JSON pages; \texttt{api\_\allowbreak increment/\allowbreak ledger.\allowbreak json} (counts, cost) & CC0 \\ | ||||
| 290 | +DBLP & \texttt{dblp.\allowbreak xml.\allowbreak gz} (2026-11-02 22:39~UTC), \texttt{dblp.\allowbreak dtd}, \texttt{.md5}; 1.12~GB & MD5 \texttt{ff1c424c\ldots27c0} verified; parse report (8{,}840{,}866 records) & ODC-BY 1.0 \\ | ||||
| 291 | +arXiv OAI-PMH & set \texttt{cs}, metadata format \texttt{arXiv}, datestamps 2018-01-01 to 2026-11-02 in monthly windows, raw XML pages kept; 2026-11-02/03 & 758 pages, 915{,}115 unique records; \texttt{harvest\_\allowbreak state.\allowbreak json}, parse report & CC0 (metadata) \\ | ||||
| 292 | +ACL Anthology & \texttt{anthology+abstracts.\allowbreak bib.\allowbreak gz}; 2026-11-02; 46.7~MB, 138{,}796 entries & SHA-256 \texttt{44c3f7e9\ldots58e3} & CC BY 4.0 \\ | ||||
| 293 | +\bottomrule | ||||
| 294 | +\end{tabularx} | ||||
| 295 | +\end{singlespace} | ||||
| 296 | +\end{table} | ||||
| 297 | + | ||||
| 298 | +\subsection{Minimal reproduction sequence} | ||||
| 299 | +\label{app:repro-minimal} | ||||
| 300 | + | ||||
| 301 | +A reader with a macOS or Linux machine, disk for the retained outputs (the CS subset and the slim table | ||||
| 302 | +occupy about 30~GB; the 675.3~GB of gzip parts are streamed and deleted after parsing unless | ||||
| 303 | +\texttt{--keep-raw} is given), memory for the DuckDB steps as configured (the scripts set | ||||
| 304 | +\texttt{memory\_\allowbreak limit} to 60--70~GB, to be lowered on smaller machines at the cost of time) and no | ||||
| 305 | +credentials can reproduce the corpus and the | ||||
| 306 | +bibliometric tables by creating the environment (\texttt{uv venv --python 3.\allowbreak 12 .\allowbreak venv \&\& uv pip install -r | ||||
| 307 | +requirements.\allowbreak txt}), downloading the manifest of the 2026-10-14 release, and running steps~1--11 of | ||||
| 308 | +Table~\ref{tab:appH-steps} in order, with the second pass of step~9 between steps~8 and~10. The snapshot | ||||
| 309 | +release is versioned, so a later release will not reproduce the counts; the manifest date is the parameter | ||||
| 310 | +that pins them. Steps~12 and~14 need no credentials but benefit from several nodes; step~13 and the | ||||
| 311 | +knowledge-graph extraction of amendment A-004 need an Anthropic API key and a budget of 25 and 150~USD | ||||
| 312 | +respectively, and their outputs (ratings, extracted edges) are released so that the downstream statistics | ||||
| 313 | +can be recomputed without re-rating. Step~15 regenerates every figure, table and number macro from the | ||||
| 314 | +result files, and step~16 compiles the manuscript; a missing result file surfaces as an undefined macro at | ||||
| 315 | +compilation, which is the intended way of detecting that a number has no source. | ||||
modified
manuscript/merge_bibs.py
+1 −1
@@ -4,7 +4,7 @@ First occurrence wins (main bib first); duplicate keys are reported in reference | |||||
| 4 | DOI differs (a real conflict to resolve by hand). main.tex uses \\bibliography{references_all}.""" | 4 | DOI differs (a real conflict to resolve by hand). main.tex uses \\bibliography{references_all}.""" | ||
| 5 | import glob, re, os | 5 | import glob, re, os | ||
| 6 | H = os.path.dirname(os.path.abspath(__file__)) | 6 | H = os.path.dirname(os.path.abspath(__file__)) | ||
| 7 | −files = [os.path.join(H, "references.bib")] + sorted(glob.glob(os.path.join(H, "references_part*.bib"))) | 7 | +files = [os.path.join(H, "references.bib")] + sorted(glob.glob(os.path.join(H, "references_part*.bib")) + sorted(glob.glob(os.path.join(H, "references_app*.bib")))) | ||
| 8 | seen = {}; out = []; dups = [] | 8 | seen = {}; out = []; dups = [] | ||
| 9 | for f in files: | 9 | for f in files: | ||
| 10 | txt = open(f, encoding="utf-8").read() | 10 | txt = open(f, encoding="utf-8").read() | ||
added
manuscript/references_appAEH.bib
+5 −0
@@ -0,0 +1,5 @@ | |||||
| 1 | +% references_appAEH.bib — Appendices A (taxonomy), E (data dictionary), H (reproducibility), WP44. | ||||
| 2 | +% No new bibliographic entry was needed. The appendices cite only keys that already exist in references.bib | ||||
| 3 | +% (cohan2020specter, singh2023scirepeval, reimers2019sbert, truong2020ruptures) or in references_part04.bib | ||||
| 4 | +% (raasveldt2019duckdb), all of which are merged into references_all.bib by merge_bibs.py. Those keys are not | ||||
| 5 | +% redefined here. See references_verification_appAEH.csv for the record of the keys reused and their verification. | ||||
added
manuscript/references_appBC.bib
+59 −0
@@ -0,0 +1,59 @@ | |||||
| 1 | +% References added by the Appendices B/C writer (Agent L), 2026-11-03. | ||||
| 2 | +% Every entry verified by a single Crossref or OpenAlex lookup; see | ||||
| 3 | +% references_verification_appBC.csv. Keys in references.bib / references_part*.bib | ||||
| 4 | +% are reused and never redefined here. | ||||
| 5 | + | ||||
| 6 | +@article{jackson2005optimal, | ||||
| 7 | + author = {Jackson, Brad and Scargle, Jeffrey D. and Barnes, David and Arabhi, Sundararajan and Alt, Alina and Gioumousis, Peter and Gwin, Elyus and Sangtrakulcharoen, Paungkaew and Tan, Linda and Tsai, Tun Tao}, | ||||
| 8 | + title = {An algorithm for optimal partitioning of data on an interval}, | ||||
| 9 | + journal = {IEEE Signal Processing Letters}, | ||||
| 10 | + year = {2005}, | ||||
| 11 | + volume = {12}, | ||||
| 12 | + number = {2}, | ||||
| 13 | + pages = {105--108}, | ||||
| 14 | + doi = {10.1109/LSP.2001.838216} | ||||
| 15 | +} | ||||
| 16 | + | ||||
| 17 | +@article{lorenz1905concentration, | ||||
| 18 | + author = {Lorenz, M. O.}, | ||||
| 19 | + title = {Methods of Measuring the Concentration of Wealth}, | ||||
| 20 | + journal = {Publications of the American Statistical Association}, | ||||
| 21 | + year = {1905}, | ||||
| 22 | + volume = {9}, | ||||
| 23 | + number = {70}, | ||||
| 24 | + pages = {209--219}, | ||||
| 25 | + doi = {10.1080/15225437.1905.10503443} | ||||
| 26 | +} | ||||
| 27 | + | ||||
| 28 | +@inproceedings{passonneau2006masi, | ||||
| 29 | + author = {Passonneau, Rebecca J.}, | ||||
| 30 | + title = {Measuring Agreement on Set-valued Items ({MASI}) for Semantic and Pragmatic Annotation}, | ||||
| 31 | + booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation ({LREC} 2006)}, | ||||
| 32 | + publisher = {European Language Resources Association (ELRA)}, | ||||
| 33 | + address = {Genoa}, | ||||
| 34 | + year = {2006}, | ||||
| 35 | + pages = {831--836}, | ||||
| 36 | + doi = {10.63317/4nuo6thi27ax} | ||||
| 37 | +} | ||||
| 38 | + | ||||
| 39 | +@article{artstein2008agreement, | ||||
| 40 | + author = {Artstein, Ron and Poesio, Massimo}, | ||||
| 41 | + title = {Inter-Coder Agreement for Computational Linguistics}, | ||||
| 42 | + journal = {Computational Linguistics}, | ||||
| 43 | + year = {2008}, | ||||
| 44 | + volume = {34}, | ||||
| 45 | + number = {4}, | ||||
| 46 | + pages = {555--596}, | ||||
| 47 | + doi = {10.1162/coli.07-034-R2} | ||||
| 48 | +} | ||||
| 49 | + | ||||
| 50 | +@article{hayes2007krippendorff, | ||||
| 51 | + author = {Hayes, Andrew F. and Krippendorff, Klaus}, | ||||
| 52 | + title = {Answering the Call for a Standard Reliability Measure for Coding Data}, | ||||
| 53 | + journal = {Communication Methods and Measures}, | ||||
| 54 | + year = {2007}, | ||||
| 55 | + volume = {1}, | ||||
| 56 | + number = {1}, | ||||
| 57 | + pages = {77--89}, | ||||
| 58 | + doi = {10.1080/19312450709336664} | ||||
| 59 | +} | ||||
added
manuscript/references_verification_appAEH.csv
+6 −0
@@ -0,0 +1,6 @@ | |||||
| 1 | +key,doi_or_arxiv,verified_via,verified_title,match | ||||
| 2 | +cohan2020specter,10.18653/v1/2020.acl-main.207,reused from references.bib (verified there by Crossref),SPECTER: Document-level Representation Learning using Citation-informed Transformers,reused | ||||
| 3 | +singh2023scirepeval,10.18653/v1/2023.emnlp-main.338,reused from references.bib (verified there),SciRepEval: A Multi-Format Benchmark for Scientific Document Representations,reused | ||||
| 4 | +reimers2019sbert,10.18653/v1/D19-1410,reused from references.bib (verified there by Crossref),Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,reused | ||||
| 5 | +truong2020ruptures,10.1016/j.sigpro.2019.107299,reused from references.bib (verified there by Crossref),Selective review of offline change point detection methods,reused | ||||
| 6 | +raasveldt2019duckdb,10.1145/3299869.3320212,reused from references_part04.bib (verified there by Crossref),DuckDB: an embeddable analytical database,reused | ||||
added
manuscript/references_verification_appBC.csv
+6 −0
@@ -0,0 +1,6 @@ | |||||
| 1 | +key,doi_or_arxiv,verified_via,verified_title,match | ||||
| 2 | +jackson2005optimal,10.1109/LSP.2001.838216,crossref,An algorithm for optimal partitioning of data on an interval,yes | ||||
| 3 | +lorenz1905concentration,10.1080/15225437.1905.10503443,crossref,Methods of Measuring the Concentration of Wealth,yes | ||||
| 4 | +passonneau2006masi,10.63317/4nuo6thi27ax,crossref+openalex W1747645360,Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation,yes | ||||
| 5 | +artstein2008agreement,10.1162/coli.07-034-R2,crossref,Inter-Coder Agreement for Computational Linguistics,yes | ||||
| 6 | +hayes2007krippendorff,10.1080/19312450709336664,crossref,Answering the Call for a Standard Reliability Measure for Coding Data,yes | ||||
| 7 | |||||