WP44 — Research plan (operational design, v1.0, 2026-11-02)
This document turns the mandate (CLAUDE.md) into the concrete design implemented by the code, the protocol and the
manuscript. Decisions the mandate left open are listed in §10 with their rationale; they are the executing agent's until
the principal investigator (PI) confirms them (human-review checkpoint H-0).
1. Scientific object and estimands
Population: the computer-science research literature indexed by at least one of DBLP, OpenAlex (any topic in field 17,
Computer Science), arXiv (set cs), or the ACL Anthology, first published between 2018-01-01 and 2026-11-02.
Two windows: baseline B = [2018-01-01, 2022-11-29] and post P = [2022-11-30, 2026-11-02]. The boundary is the public
release of ChatGPT; it is a reference date for description and for interrupted-time-series designs, not a causal
treatment (CLAUDE.md §6.5). Monthly series are the primary temporal unit; years are secondary.
Primary descriptive estimands (RQ1, RQ2): monthly counts of works per category c, N_{c,t}; composition shares s_{c,t} = N_{c,t}/N_t; growth rates and their pre/post structural-break statistics; citation-based influence (field- and age-normalised); collaboration/interdisciplinarity indices (RQ4–RQ6). Inferential estimands (RQ2, RQ10): breakpoint locations and magnitudes with placebo distributions; out-of-time precision/recall of the emerging-topic detector.
2. Sources, access and what each contributes (verified 2026-11-02, see reports/PHASE0_FEASIBILITY.md)
| Source | Access used | Licence | Role in the corpus |
|---|---|---|---|
| OpenAlex public snapshot, release 2026-10-14 | S3 public bucket, JSONL, 1,898 works files (675 GB), ~110 MB/s from M3U96b | CC0 | backbone: works, authorships + institutions + countries, venues, topics (field/subfield/topic), referenced_works (citation graph), abstracts (inverted index), OA status, FWCI, counts_by_year |
| OpenAlex API (no key, $0.10/day shared budget) | REST, polite mailto |
CC0 | incremental works 2026-10-14 → 2026-11-02; spot checks; free single-record lookups for reference verification |
| DBLP XML dump (2026-11-02, 1.12 GB gz, MD5 verified) | HTTPS | ODC-BY 1.0 | CS-scope gold standard, venue streams, author PIDs, DOI links; defines "CS venue" |
arXiv OAI-PMH, set cs, 2018-01 → 2026-11 |
OAI-PMH, 3.5 s/request, raw XML kept | arXiv metadata terms (CC0 for metadata) | preprints, categories (cs.XX, cross-lists), versions, licences; preprint–publication linking |
| ACL Anthology BibTeX (anthology+abstracts.bib, 138,796 entries) | HTTPS | CC BY 4.0 | NLP venue coverage check, abstracts for ACL papers |
| Crossref REST API (polite pool) | REST | open metadata | reference verification, is-referenced-by-count cross-check sample, publisher/type cross-check |
| Semantic Scholar Academic Graph | not available: API key required for bulk/dataset access; anonymous calls return HTTP 429 | — | recorded as unavailable (CLAUDE.md §1: record inaccessible data explicitly); influential-citation and citation-intent signals are therefore not used |
| Papers with Code / benchmark repositories, GitHub | HTTPS, public dumps where available | varies | RQ7/RQ9 (benchmark proliferation, code availability) — scoped in Phase 3 |
| Full texts | arXiv e-prints (rate-limited, licence-respecting), OA locations from OpenAlex/Unpaywall | per-paper licence | ≤ 10,000 high-priority papers; never through paywalls |
No credential was assumed: the OpenAlex key, Semantic Scholar key and Kaggle credentials were tested as absent.
3. Corpus construction (Phase 1) — PRISMA-style stages
S0 Identification: all OpenAlex works with any topic in field 17 and publication_year 2018–2026 (full records kept),
plus DBLP records 2018–2026, arXiv cs records created 2018–2026, ACL entries 2018–2026. A slim table of every
OpenAlex work (465 M rows: id, DOI, year, type, primary topic/subfield/field/domain, cited_by_count, flags) is kept to resolve
cited works outside CS.
S1 Record linkage: OpenAlex ↔ DBLP by DOI (lower-cased), else by arXiv ID (DBLP ee → arxiv.org/abs), else by normalised
title + year + first-author surname (blocking on title hash); OpenAlex ↔ arXiv by locations[].landing_page_url/ids
and DOI (10.48550/arXiv.); ACL by DOI/URL. Linkage precision is estimated on a 500-pair manually reviewed sample (H-1).
S2 Deduplication: one node per work; preprint and published version are merged into a single node with
preprint_date, publication_date, version_relationship kept (CLAUDE.md §6.2: preprint duplication).
S3 Eligibility: (i) publication_year in window; (ii) type in {article, preprint, book-chapter, proceedings-article,
dissertation, report, book} — exclude paratext, editorials, errata, retracted (kept with flag is_retracted), XPAC
expansion records (is_xpac); (iii) CS membership: DBLP-indexed or OpenAlex primary field = CS or (secondary CS
topic and arXiv primary category cs.). Secondary-only CS topics without DBLP/arXiv support form a penumbra set
reported separately. Sensitivity: results re-estimated on the strict core (DBLP ∩ OpenAlex primary CS).
S4 Classification: 40-category hierarchical ontology (taxonomy/ontology.yaml), multi-label, assigned by rules over
OpenAlex topics/subfields, arXiv categories, DBLP venue streams and keyword patterns; validated on manually reviewed
samples (Phase 2, H-2) and by an LLM-based second rater with reported agreement.
S5 Synthesis populations: (a) mapping corpus (all eligible); (b) deeply classified set (≥ 100 k: abstracts + embeddings +
full feature vectors); (c) high-priority set (≥ 10 k: citation-ranked within category × year with OA full text);
(d) landmark set (≥ 1 k: top field-normalised impact, reviewed by specialist agents and flagged for PI review, H-3).
Every stage writes counts to data/metadata/prisma_counts.json, rendered as the PRISMA flow figure.
4. Taxonomy (Phase 2)
taxonomy/ontology.yaml: 40 leaf categories under 8 top-level areas (AI & ML; Vision, Language & Perception; Systems &
Infrastructure; Theory & Foundations; Security, Privacy & Cryptography; Software & Human Factors; Emerging Computing
Paradigms; Computing & Science). Each leaf has: definition, parent, OpenAlex topic IDs (weighted), subfield IDs, arXiv
categories, DBLP streams, keyword regexes with weights, exclusions, and provenance of the mapping. Assignment score per
category = weighted sum of matched signals; multi-label with threshold; primary_category = argmax. Validation: 2,000
stratified works (50 per category) rated by two independent LLM raters with the rubric in taxonomy/VALIDATION.md,
Cohen's κ and per-category precision/recall; a 200-work subsample is flagged for PI review (H-2).
5. Knowledge graph (Phase 3)
Nodes: works, authors (OpenAlex A-ids), institutions, countries, venues (sources), topics, taxonomy categories, arXiv
categories; method/model/dataset/benchmark and claim nodes only for the high-priority set, extracted from abstracts
(and full text where lawfully available) with a schema-constrained LLM extractor; each extracted edge carries
provenance (work id + text span), confidence, verification_status ∈ {unverified, machine-verified, human-verified}.
Citation edges come from referenced_works (verified by construction). Semantic edges from embeddings (k-NN, cosine).
Storage: Parquet tables + DuckDB views; graph analytics with igraph (python-igraph) on M3U96b (96 GB RAM).
6. Statistics (Phase 4) and structural change
- Bibliometrics: monthly/annual counts; shares; CAGR; field-normalised citation score FNCS = c / E[c | field, year, type];
citation velocity (citations within 12/24 months, computed from
counts_by_year); Gini/Hirsch-type inequality; venue concentration (HHI); author/institution/country collaboration networks; Rao–Stirling and Leinster–Cobbold diversity. - Citation-age bias: all cross-period comparisons use fixed citation windows and year-normalisation; FWCI used as a secondary measure with its known limits.
- Structural change: interrupted time series on log monthly counts with seasonality (OLS-HAC, Newey–West), Bai–Perron
multiple-breakpoint estimation, PELT/binary-segmentation changepoints, Bayesian online changepoint detection; pre-trend
diagnostics (placebo breaks at every month of the baseline), alternative event dates (2020-05 GPT-3, 2022-04 DALL·E 2,
2022-11 ChatGPT, 2023-03 GPT-4, 2024-09 o1), donut windows; sensitivity to OpenAlex indexing lag (right-truncation
corrected by indexing-delay profiles estimated from
created_date). - Dynamic networks: yearly topic co-occurrence and citation-flow matrices between taxonomy categories; modularity and cross-area flow shares; ERGM-style descriptives are out of scope.
7. Emerging-research detection (Phase 6, original contribution)
Unit: topic candidates = (i) OpenAlex topics, (ii) semantic clusters (HDBSCAN on embeddings, re-estimated per
training cutoff without look-ahead), (iii) keyword n-grams. Signals at month t using data ≤ t only: growth anomaly,
semantic novelty (distance to pre-t centroid), citation acceleration, interdisciplinary connectivity, methodological
adoption (share of works citing a method node), benchmark proliferation, open-source activity. Label at t+h: top-decile
growth and citation share at horizon h ∈ {12, 24} months. Training cutoffs 2019–2022; test cutoffs 2023–2024 with
outcomes observed up to 2026-11. Models: regularised logistic, gradient boosting, and a simple z-score baseline;
metrics: AUC, precision@k, lead time. Known-emergence validation set: LLMs, diffusion models, NeRF, agents, RAG,
state-space models, vision-language models, mechanistic interpretability (defined before estimation, listed in
protocol/PROTOCOL.md).
8. Agents and artefacts
The executing session is Agent A (Research Director). Subagents are spawned with written briefs (agents/briefs/),
return structured artefacts (JSON/CSV/Markdown with declared schema) and never approve their own findings: every
scientific artefact is checked by a different agent (K or N) before it enters the manuscript (agents/REGISTER.md).
9. Infrastructure
Primary compute/storage: M3U96b (M3 Ultra 28 c, 96 GB, 375 GB free; ~/wp44_uqo/, venv Python 3.12 via uv:
orjson, pyarrow, polars, duckdb, zstandard, lxml, requests). Secondary: M4M64a (540 GB free) for embeddings; other LAN
nodes for embarrassingly parallel embedding/extraction batches when it materially helps. Laptop = controller and git
repository (Recherche-UQO/informatique/wp44, origin gitsrv:srv/git/wp44_uqo.git). Data are not in git; ledgers,
manifests, hashes and PRISMA counts are.
10. Decisions taken by the executing agent (pending PI confirmation, H-0)
| # | Decision | Rationale |
|---|---|---|
| D1 | Snapshot over API for OpenAlex | mandate §3 prefers snapshots; the API budget without a key is $0.10/day (≈ 200 k works/day) |
| D2 | Keep a slim table of all 465 M works | required for interdisciplinary citation flows (RQ4) and cited-work normalisation |
| D3 | Semantic Scholar not used | key required; recorded as unavailable rather than scraped |
| D4 | CS membership rule (§3 S3) with strict-core sensitivity | avoids both AI-only bias and over-inclusion from secondary topics |
| D5 | Manuscript directory manuscript/ (mandate §12) instead of the UQO paper/ convention |
mandate is explicit; deviation noted in README |
| D6 | Historical raw snapshot kept as filtered subset + per-file SHA-256 + manifest, not the 675 GB bucket copy | public versioned release; hashes allow verification; storage |
| D7 | LLM raters (Claude) used for taxonomy validation and knowledge-graph extraction, with PI review checkpoints | mandate §1 requires human checkpoints; PI is the human |