SPB Git

spb/wp44_uqo

Public
11.6 KB

# WP44 — Research plan (operational design, v1.0, 2026-11-02)

This document turns the mandate (CLAUDE.md) into the concrete design implemented by the code, the protocol and the manuscript. Decisions the mandate left open are listed in §10 with their rationale; they are the executing agent's until the principal investigator (PI) confirms them (human-review checkpoint H-0).

# 1. Scientific object and estimands

Population: the computer-science research literature indexed by at least one of DBLP, OpenAlex (any topic in field 17, Computer Science), arXiv (set cs), or the ACL Anthology, first published between 2018-01-01 and 2026-11-02. Two windows: baseline B = [2018-01-01, 2022-11-29] and post P = [2022-11-30, 2026-11-02]. The boundary is the public release of ChatGPT; it is a reference date for description and for interrupted-time-series designs, not a causal treatment (CLAUDE.md §6.5). Monthly series are the primary temporal unit; years are secondary.

Primary descriptive estimands (RQ1, RQ2): monthly counts of works per category c, N_{c,t}; composition shares s_{c,t} = N_{c,t}/N_t; growth rates and their pre/post structural-break statistics; citation-based influence (field- and age-normalised); collaboration/interdisciplinarity indices (RQ4–RQ6). Inferential estimands (RQ2, RQ10): breakpoint locations and magnitudes with placebo distributions; out-of-time precision/recall of the emerging-topic detector.

# 2. Sources, access and what each contributes (verified 2026-11-02, see reports/PHASE0_FEASIBILITY.md)

Source Access used Licence Role in the corpus
OpenAlex public snapshot, release 2026-10-14 S3 public bucket, JSONL, 1,898 works files (675 GB), ~110 MB/s from M3U96b CC0 backbone: works, authorships + institutions + countries, venues, topics (field/subfield/topic), referenced_works (citation graph), abstracts (inverted index), OA status, FWCI, counts_by_year
OpenAlex API (no key, $0.10/day shared budget) REST, polite mailto CC0 incremental works 2026-10-14 → 2026-11-02; spot checks; free single-record lookups for reference verification
DBLP XML dump (2026-11-02, 1.12 GB gz, MD5 verified) HTTPS ODC-BY 1.0 CS-scope gold standard, venue streams, author PIDs, DOI links; defines "CS venue"
arXiv OAI-PMH, set cs, 2018-01 → 2026-11 OAI-PMH, 3.5 s/request, raw XML kept arXiv metadata terms (CC0 for metadata) preprints, categories (cs.XX, cross-lists), versions, licences; preprint–publication linking
ACL Anthology BibTeX (anthology+abstracts.bib, 138,796 entries) HTTPS CC BY 4.0 NLP venue coverage check, abstracts for ACL papers
Crossref REST API (polite pool) REST open metadata reference verification, is-referenced-by-count cross-check sample, publisher/type cross-check
Semantic Scholar Academic Graph not available: API key required for bulk/dataset access; anonymous calls return HTTP 429 — recorded as unavailable (CLAUDE.md §1: record inaccessible data explicitly); influential-citation and citation-intent signals are therefore not used
Papers with Code / benchmark repositories, GitHub HTTPS, public dumps where available varies RQ7/RQ9 (benchmark proliferation, code availability) — scoped in Phase 3
Full texts arXiv e-prints (rate-limited, licence-respecting), OA locations from OpenAlex/Unpaywall per-paper licence ≤ 10,000 high-priority papers; never through paywalls

No credential was assumed: the OpenAlex key, Semantic Scholar key and Kaggle credentials were tested as absent.

# 3. Corpus construction (Phase 1) — PRISMA-style stages

S0 Identification: all OpenAlex works with any topic in field 17 and publication_year 2018–2026 (full records kept), plus DBLP records 2018–2026, arXiv cs records created 2018–2026, ACL entries 2018–2026. A slim table of every OpenAlex work (465 M rows: id, DOI, year, type, primary topic/subfield/field/domain, cited_by_count, flags) is kept to resolve cited works outside CS. S1 Record linkage: OpenAlex ↔ DBLP by DOI (lower-cased), else by arXiv ID (DBLP ee → arxiv.org/abs), else by normalised title + year + first-author surname (blocking on title hash); OpenAlex ↔ arXiv by locations[].landing_page_url/ids and DOI (10.48550/arXiv.); ACL by DOI/URL. Linkage precision is estimated on a 500-pair manually reviewed sample (H-1). S2 Deduplication: one node per work; preprint and published version are merged into a single node with preprint_date, publication_date, version_relationship kept (CLAUDE.md §6.2: preprint duplication). S3 Eligibility: (i) publication_year in window; (ii) type in {article, preprint, book-chapter, proceedings-article, dissertation, report, book} — exclude paratext, editorials, errata, retracted (kept with flag is_retracted), XPAC expansion records (is_xpac); (iii) CS membership: DBLP-indexed or OpenAlex primary field = CS or (secondary CS topic and arXiv primary category cs.). Secondary-only CS topics without DBLP/arXiv support form a penumbra set reported separately. Sensitivity: results re-estimated on the strict core (DBLP ∩ OpenAlex primary CS). S4 Classification: 40-category hierarchical ontology (taxonomy/ontology.yaml), multi-label, assigned by rules over OpenAlex topics/subfields, arXiv categories, DBLP venue streams and keyword patterns; validated on manually reviewed samples (Phase 2, H-2) and by an LLM-based second rater with reported agreement. S5 Synthesis populations: (a) mapping corpus (all eligible); (b) deeply classified set (≥ 100 k: abstracts + embeddings + full feature vectors); (c) high-priority set (≥ 10 k: citation-ranked within category × year with OA full text); (d) landmark set (≥ 1 k: top field-normalised impact, reviewed by specialist agents and flagged for PI review, H-3).

Every stage writes counts to data/metadata/prisma_counts.json, rendered as the PRISMA flow figure.

# 4. Taxonomy (Phase 2)

taxonomy/ontology.yaml: 40 leaf categories under 8 top-level areas (AI & ML; Vision, Language & Perception; Systems & Infrastructure; Theory & Foundations; Security, Privacy & Cryptography; Software & Human Factors; Emerging Computing Paradigms; Computing & Science). Each leaf has: definition, parent, OpenAlex topic IDs (weighted), subfield IDs, arXiv categories, DBLP streams, keyword regexes with weights, exclusions, and provenance of the mapping. Assignment score per category = weighted sum of matched signals; multi-label with threshold; primary_category = argmax. Validation: 2,000 stratified works (50 per category) rated by two independent LLM raters with the rubric in taxonomy/VALIDATION.md, Cohen's κ and per-category precision/recall; a 200-work subsample is flagged for PI review (H-2).

# 5. Knowledge graph (Phase 3)

Nodes: works, authors (OpenAlex A-ids), institutions, countries, venues (sources), topics, taxonomy categories, arXiv categories; method/model/dataset/benchmark and claim nodes only for the high-priority set, extracted from abstracts (and full text where lawfully available) with a schema-constrained LLM extractor; each extracted edge carries provenance (work id + text span), confidence, verification_status ∈ {unverified, machine-verified, human-verified}. Citation edges come from referenced_works (verified by construction). Semantic edges from embeddings (k-NN, cosine). Storage: Parquet tables + DuckDB views; graph analytics with igraph (python-igraph) on M3U96b (96 GB RAM).

# 6. Statistics (Phase 4) and structural change

  • Bibliometrics: monthly/annual counts; shares; CAGR; field-normalised citation score FNCS = c / E[c | field, year, type]; citation velocity (citations within 12/24 months, computed from counts_by_year); Gini/Hirsch-type inequality; venue concentration (HHI); author/institution/country collaboration networks; Rao–Stirling and Leinster–Cobbold diversity.
  • Citation-age bias: all cross-period comparisons use fixed citation windows and year-normalisation; FWCI used as a secondary measure with its known limits.
  • Structural change: interrupted time series on log monthly counts with seasonality (OLS-HAC, Newey–West), Bai–Perron multiple-breakpoint estimation, PELT/binary-segmentation changepoints, Bayesian online changepoint detection; pre-trend diagnostics (placebo breaks at every month of the baseline), alternative event dates (2020-05 GPT-3, 2022-04 DALL·E 2, 2022-11 ChatGPT, 2023-03 GPT-4, 2024-09 o1), donut windows; sensitivity to OpenAlex indexing lag (right-truncation corrected by indexing-delay profiles estimated from created_date).
  • Dynamic networks: yearly topic co-occurrence and citation-flow matrices between taxonomy categories; modularity and cross-area flow shares; ERGM-style descriptives are out of scope.

# 7. Emerging-research detection (Phase 6, original contribution)

Unit: topic candidates = (i) OpenAlex topics, (ii) semantic clusters (HDBSCAN on embeddings, re-estimated per training cutoff without look-ahead), (iii) keyword n-grams. Signals at month t using data ≤ t only: growth anomaly, semantic novelty (distance to pre-t centroid), citation acceleration, interdisciplinary connectivity, methodological adoption (share of works citing a method node), benchmark proliferation, open-source activity. Label at t+h: top-decile growth and citation share at horizon h ∈ {12, 24} months. Training cutoffs 2019–2022; test cutoffs 2023–2024 with outcomes observed up to 2026-11. Models: regularised logistic, gradient boosting, and a simple z-score baseline; metrics: AUC, precision@k, lead time. Known-emergence validation set: LLMs, diffusion models, NeRF, agents, RAG, state-space models, vision-language models, mechanistic interpretability (defined before estimation, listed in protocol/PROTOCOL.md).

# 8. Agents and artefacts

The executing session is Agent A (Research Director). Subagents are spawned with written briefs (agents/briefs/), return structured artefacts (JSON/CSV/Markdown with declared schema) and never approve their own findings: every scientific artefact is checked by a different agent (K or N) before it enters the manuscript (agents/REGISTER.md).

# 9. Infrastructure

Primary compute/storage: M3U96b (M3 Ultra 28 c, 96 GB, 375 GB free; ~/wp44_uqo/, venv Python 3.12 via uv: orjson, pyarrow, polars, duckdb, zstandard, lxml, requests). Secondary: M4M64a (540 GB free) for embeddings; other LAN nodes for embarrassingly parallel embedding/extraction batches when it materially helps. Laptop = controller and git repository (Recherche-UQO/informatique/wp44, origin gitsrv:srv/git/wp44_uqo.git). Data are not in git; ledgers, manifests, hashes and PRISMA counts are.

# 10. Decisions taken by the executing agent (pending PI confirmation, H-0)

# Decision Rationale
D1 Snapshot over API for OpenAlex mandate §3 prefers snapshots; the API budget without a key is $0.10/day (≈ 200 k works/day)
D2 Keep a slim table of all 465 M works required for interdisciplinary citation flows (RQ4) and cited-work normalisation
D3 Semantic Scholar not used key required; recorded as unavailable rather than scraped
D4 CS membership rule (§3 S3) with strict-core sensitivity avoids both AI-only bias and over-inclusion from secondary topics
D5 Manuscript directory manuscript/ (mandate §12) instead of the UQO paper/ convention mandate is explicit; deviation noted in README
D6 Historical raw snapshot kept as filtered subset + per-file SHA-256 + manifest, not the 675 GB bucket copy public versioned release; hashes allow verification; storage
D7 LLM raters (Claude) used for taxonomy validation and knowledge-graph extraction, with PI review checkpoints mandate §1 requires human checkpoints; PI is the human
Files · 5aec0c2a7ebbc962a63ccf850228c53d452071a6
Menu

Theme