# Critique — Parts II, III, IV and Appendices A, B, C, E, H

Agent K (Scientific Critic), 2026-11-03. Artefacts: manuscript/sections/part02_evolution.tex, part03_data.tex, part04_methodology.tex; manuscript/appendices/appA_taxonomy.tex, appB_search_and_sources.tex, appC_derivations.tex, appE_data_dictionary.tex, appH_reproducibility.tex. Machine-readable issue list: reports/critic/issues_parts_02_03_04_appendices.csv (71 issues: 8 blocking, 20 major, 43 minor; issue ids K-nn are cited below). Nothing in the manuscript was edited.

What was checked. Every statement about the pipeline was read against the code (ingestion/link_sources.py, systematic_review/define_corpus.py, systematic_review/linkage_sample.py, taxonomy/classify.py, taxonomy/build_ontology.py, taxonomy/validate_sample.py, taxonomy/ontology.yaml, bibliometrics/01–04, econometrics/structural_change.py, econometrics/run_structural_change.py, experiments/emerging_detection.py, experiments/known_set.py, knowledge_graph/01–03, embeddings/embed_corpus.py, embeddings/semantic_atlas.py, literature_synthesis/select_landmarks.py, tables/make_numbers.py), against the result files (data/metadata/prisma_counts.json, linkage_report.json, linkage_sample_summary.json, linkage_sample.csv, api_increment_ledger.json, manuscript/tables/numbers.tex, reports/*.md, protocol/*), and against the bibliography (references_all.bib, merge report; two cited abstracts fetched from Crossref). The manuscript was compiled from a copy in /tmp to test the macro discipline. Every formula of Part IV and Appendix C was re-derived (ITS design and HAC, placebo rank, Bai–Perron dynamic programme with BIC/LWZ and trimming, PELT penalty, BOCPD conjugate updates and the R(0)=H identity, Gini rank form and its bounds, Leinster–Cobbold/Rao–Stirling identity at q=2, directed modularity, Krippendorff α with MASI, the score rule and tie-break, the filtration argument for the detector); all are correct as stated, and the only mathematical defects are the two statements marked K-42/K-43 (first-date and precision rules that do not match the implemented code) and the minor items K-90–K-93.

Verdict summary.

"REJECT" means the part must go back to its writer because it asserts things about the study's own data or code that are false, not that the design is unsound; most of Part IV is accurate and unusually candid, and the rejection rests on a handful of statements that contradict the implementation.

# 1. Blocking issues (all files, in order of consequence)

B1 — The manuscript does not compile (K-20, K-40, K-80). \LinkNeededOaIdsSecondPassArxivDoi{} is used in Part III l.288, Part IV l.235 and Appendix B l.182 but is not defined in tables/numbers.tex, because linkage_report.json["needed_oa_ids_second_pass"] contains only the key dblp_doi (1,605). A fresh latexmk run on a copy stops at Part III l.288 with "Undefined control sequence". The main.pdf in the repository (17:59) predates the current numbers.tex (18:11). Fix: emit the macro with value 0 from make_numbers.py (or write in prose that no arXiv-DOI identifier was needed in this run) and recompile before any further review.

B2 — The PRISMA flow does not add up (K-21, K-47). after_exclusions = 10,519,027 but core 7,110,289 + penumbra 3,402,475 = 10,512,764. Part III's Figure caption (l.539) states that "the core and the penumbra partition the records that survive the exclusions"; 6,263 surviving records are neither. By construction (define_corpus.py l.71–72) these are records with cs_any = false that are not DBLP-linked, i.e. second-pass works recovered through arXiv DOIs. This exposes a substantive point about the membership rule: its third clause is cs_any ∧ arxiv_cs_primary, so an author-declared cs.* preprint can never enter the core unless OpenAlex already gave the work a CS topic; Part IV l.236–240 ("the union of that set with what DBLP and arXiv call computer science") overstates what the rule does. Required: report the third class in the PRISMA figure and text, explain it, and either reword Part IV or amend the rule (with a logged amendment).

B3 — The identification count is mislabelled across Parts II/III and Appendix B (K-04, K-22, K-81, K-82). PrismaIdentifiedOpenalexCsTopic…PassOne = 16,592,966 is computed on oa_keys, which is built over the union of both snapshot passes; the first pass retained 15,490,159 records (RUNBOOK, Appendix H, slim_cs_any_2018_2026). Part III l.191–193 ("The filter yields … in the first pass"), Part II l.49 ("works classified in computer science"), Appendix B l.111 ("about 20 GB for … records") and Appendix B l.195–200 (which says the PRISMA counts were generated from the first pass and leaves the two-pass total pending) are all wrong in the same direction: about 1.10 M of the quoted works carry no CS topic. Fix in define_corpus.py/make_numbers.py (record the real pass-1 count) and in the four passages.

B4 — Part III contradicts the citation code (K-23). l.356–358: preprint citations "are not added to the published node, a known under-count". bibliometrics/03_citation_impact.py (CTE CY, UNION ALL … ON e.preprint_work_id = c.work_id) adds the merged preprint's counts_by_year to the canonical node, and Appendix E l.604–606 says so. Every c12/c36/Gini figure in numbers.tex was produced with the merge. Correct Part III and state the merge in Part IV §Fixed citation windows.

B5 — Part IV's first-date equation is not the implemented rule (K-42, K-43). Eq. (eq:first-date) takes the minimum of the three dates; define_corpus.py l.58–60 drops a 1-January publication date from the minimum whenever a preprint or arXiv date exists (Appendix B l.511–514 and Appendix E l.566 describe the code correctly). The precision flag (eq. prec) is likewise mis-stated (the code sets month when such a date exists), and the sentence "it does not depend on the period, so it cannot manufacture a break" is unsupported: the rescue depends on arXiv linkage, which grows over the window, and Part III itself reports year-only shares between 18 % and 28 % varying by source and type. The consequence is not cosmetic: a year-only 2022 record with a 2023 arXiv date is dated 2023 and changes period.

B6 — Garbled paragraph in Part IV (K-41). l.766–777 splice two contradictory drafts ("the two diversity indices are not yet produced by a script … their values in Part IX are / The yearly Rao–Stirling … are computed by bibliometrics/04_diversity_cagr.py … / until the computation is added and audited"), leaving an unfinished sentence. The script exists; rewrite the paragraph.

# 2. Part II — part02_evolution.tex — ACCEPT WITH FIXES

The historical synthesis is well sourced and the section on why the date is descriptive (l.291–305) is exactly the posture the mandate requires. Four factual problems and a few superlatives remain.

Major.

• K-01 (l.62): Bornmann & Mutz (2015) is misreported. Their abstract (Crossref, retrieved today) gives three phases of growth in cited references — under 1 % to the mid-18th century, 2–3 % up to the inter-war period, 8–9 % to 2010. The manuscript's "roughly three per cent per year with a doubling time of about a quarter of a century" is the pre-1940s phase presented as the long-run estimate. Report 8–9 % (doubling ≈ 9 years) or attribute the 2–3 % to its phase.
• K-02 (l.151, l.55): "a majority of computer-science preprints in their sample could not be matched to a published version" (Lin et al. 2020). I could not retrieve the abstract (neither Crossref nor OpenAlex carries it) and therefore cannot certify the claim either way; the paper is generally quoted as finding roughly two thirds of arXiv CS preprints published. Verify against the full text before this sentence stays; Part III l.369–370 uses the same reference without a direction and is safe.
• K-03 (l.153): "the seasonal component … is modelled separately for the baseline and the research period". It is not: _design() fits eleven month dummies common to both periods. Either state the common seasonality or implement and report interacted dummies.
• K-04 (l.49): see B3.

Minor: unbalanced parenthesis and a preprint citation for a work that is published and cited under a second key in Part III (K-05); three unsupported superlatives (K-06 "faster than any previous software product", K-07 "the first systems language", K-09 "informative about the mechanism, not merely about measurement"); Kepler-conjecture dating (K-08); Ott et al. 2022 called a preprint although the bib entry is Nature Communications (K-10); the venue-only classification promised for Appendix D does not exist in classify.py (K-11).

Checked and found correct: the eight areas and leaf counts (9/6/4/2/6/5/5/3), the seven specialisation edges, the thirteen keyword-driven categories and the build-time check (build_ontology.py l.157–163), the equation (eq:p2-score) (matches score_row), the timeline table's preprint/report labels, FIPS 203/204/205 (August 2024), BIG-bench (>200 tasks), Sevilla et al. (≈6-month doubling).

# 3. Part III — part03_data.tex — REJECT

The part is the best-written account of a bibliographic pipeline I have reviewed in this project, and its integrity section (l.792–893) is a model. It is rejected because, as it stands, it misdescribes its own data at four points (B1–B4) and is stale against result files that already exist.

Blocking: K-20 (compile), K-21 (partition), K-22 (first-pass count), K-23 (citation merge).

Major.

• K-24 (l.664–665): the truncation sensitivity is said to end in December 2025; the driver and numbers.tex use March and August 2026 (Part IV l.1047 agrees with the code).
• K-25 (l.313–316): "the sample has not been drawn". data/metadata/linkage_sample.csv (450 pairs) and linkage_sample_summary.json exist: DOI stratum 235/250 pass the automatic check (0.94), title–year 200/200. The 15 DOI failures include a book-level DOI matched to a chapter and retitled versions ("Textual Entailment and Token Probability…" vs "Textual Entailment is not a Better Bias Metric…"), which matters because Part IV assumes the DOI stratum is error-free (K-48). Report the automatic rates now, as the text promises to do.
• K-27 (l.443, 600–602): three \pending{} markers for macros that exist in numbers.tex (E7 sequential 473,549; month precision 5,703,718; year precision 1,406,571).
• K-31 (l.336–339): the deduplication rule omits the shared-arXiv-identifier route (13,175 eligible pairs, macro \PrismaVersionRelationshipArxivId) and the conditions imposed on the published twin. Minor: 180 not 200 PI-review pairs (K-26); A-005 already resolves the E7 numbering (K-28); six amendments, not five (K-29); the ACL URL step linked nothing (K-30); first-author condition of the title step omitted (K-32); 138,796 typed although \AclParsedEntries exists (K-33); E1 also removes out-of-window pass-2 records (K-34); date-precision definition incomplete (K-35); duplicate bibliography entries under two keys (K-36 with K-05: culbert 2024/2025, martinmartin coverage/comparison, fortunato science/sos, kinney s2/s2ag, radev acl/aclanthology, openalex 2024/2025 topics; 20 duplicated DOIs in references_all.bib).

Checked and found correct: all \Prisma*/\Link* macros quoted (sequential exclusions sum to the loaded total; year counts sum to the core; arXiv 864,677/890,852; ACL 72,949/85,625), the E1–E7 predicates, the asymmetric type rule (A-001), the generic-deposit list (A-002), the strict core, the "pre" residue of 1,864 works, the created_date diagnosis, the right-truncation rule (last complete month June 2026), the Semantic Scholar record.

# 4. Part IV — part04_methodology.tex — REJECT

Strengths: the part describes the code rather than the plan, declares every deviation (max-vs-sum, c36 vs 24 months, placebo range, no sup-F, topics instead of clusters, volume-only label), states what the look-ahead safeguards do not cover, and uses no causal vocabulary. The mathematics is right. The rejection is for statements that are contradicted by the code or by result files.

Blocking: K-40 (compile), K-41 (garbled diversity paragraph), K-42 (first-date equation).

Major.

• K-43 (l.76–87): precision rule and the "cannot manufacture a break" assertion (see B5).
• K-44 (l.250–261): deduplication rule incomplete (as K-31).
• K-47 (l.236–240): "the union … with what … arXiv call[s] computer science" overstates the membership rule (B2).
• K-48 (l.368–369): "the title–year stratum … is the only one where errors are plausible" — contradicted by the sample (DOI 0.94 vs title 1.00); with weights L_s/L the DOI stratum dominates the precision estimate.
• K-50 (l.1067–1068): "Donut windows … are not implemented". run_structural_change.py --donut (default 3) is implemented and numbers.tex carries the result (n = 94, level change 0.049, p = 0.056). Report it.
• K-51 (l.1568–1572): a 1,000-replication bootstrap of the Gini and the cross-area shares "computed by the table-making script" does not exist anywhere (make_numbers.py has no bootstrap; the only bootstrap is 04_diversity_cagr.py, B = 500 multinomial over categories, as l.772 says). This is a claimed uncertainty quantification without an artefact.
• K-54 (l.1379–1390): internally contradictory training-cutoff description (December 2020 vs December 2021 for h = 24; "label uses data to December 2023"). Code: last_train = 2020-12 for h > 12; cutoffs are months with index ≡ 0 (mod 3) from January 2018, so realised cutoffs run January 2020 → October 2021 (h = 12) / October 2020 (h = 24), test January 2023 → October 2024. Appendix C l.772–776 records the contradiction instead of fixing it.
• K-57 (l.1427–1435): the known-emergence lead time is said to be pending because the topic mapping is empty, but experiments/known_set.py exists and has produced results/emerging/known_set_lead_times.csv under a different design (hand-written regexes; detection = trailing z > 3 ∧ 12-month doubling ∧ ≥ 30 works; peak = full-window maximum of 12-month growth). That is the baseline rule on keyword series, not the trained detector, and the peak month is chosen with full-sample knowledge; it must be described, labelled as a deviation from H8's lead-time definition, and not presented as the H8 test.

Minor (13): period labels pre/future missing (K-45); first-author condition missing (K-46); raw vs normalised titles in the auto check (K-49); CAGR script exists (K-52); HHI without preprints is not computed (K-53); detector --last-month default is June, not October 2026 (K-55); no window overlap (K-56); five vs six checkpoints (K-58); created_date definition (K-59); unsupported seasonality remark in the Bai–Perron section (K-60); Gini/NCS populations exclude retracted works (K-61); the baseline-only placebo rank excludes the true event and can be 0 (K-62); the February 2018 start is asserted immaterial (K-63).

Checked and found correct: linkage cascade order and thresholds (except the omissions above), T_keep/T_hard/G lists, e1–e7 and the membership flags, the score rule with caps and veto, the tie-break, validation sampling (50 × 40 + 200, seed 44, 200 for the PI), κ/Jaccard/α definitions, citation windows and completeness years, NCS reference class, Gini rank form, HHI, counting conventions and thresholds (50, 200), Rao–Stirling/Leinster–Cobbold, modularity, citation-graph edge sets, embedding text/length/precision, atlas parameters (200,000; UMAP 30/0.0/50; HDBSCAN 200/20; split-half 5 ×, 100/20, 5-NN; 12 TF-IDF terms, min_df 5), ITS design and HAC bandwidth 6, placebo range 24/12 and |T'| = 65, Bai–Perron h = 15 and the April 2025 bound, BIC/LWZ forms, PELT penalty and min_size 6, BOCPD hazard and initialisation, alternative event months, detector features and label, model hyper-parameters, landmark selection formulas, extraction schema and verification rule.

# 5. Appendix A — appA_taxonomy.tex — ACCEPT WITH FIXES

All counts of Table tab:appA-signals and its totals (646/93/21/162/1,320/291/273/106/24/236), the 144 unordered related pairs, the seven edges, the thirteen-category build check and the "all keyword patterns on title+abstract" statement were recomputed from ontology.yaml and are exact. Two minor points: the DBLP-coverage numbers (l.396–399, 405, 422–446) are data-derived counts typed from ONTOLOGY.md rather than macros (K-70); "The ontology was validated" (l.675) is premature — taxonomy/validation/ does not exist and Part IV/Appendix H mark the step as not run (K-71).

# 6. Appendix B — appB_search_and_sources.tex — ACCEPT WITH FIXES (one blocking macro)

The cascade table (tab:appB-cascade) is the most accurate description of link_sources.py in the manuscript (it alone states the first-author condition and the asymmetric uniqueness rules), and the first-date/precision/period paragraph (l.510–519) is correct where Part IV is not. Fixes: the undefined macro (K-80); the stale "generated from the first pass / pending after the second pass" paragraph and the 20 GB figure (K-81, K-82, see B3); the API-increment totals now exist (1,002 calls, 193,369 records, 64 slices done, one truncated slice — 2026-10-16 cs_secondary, count 37,712 of which 10,000 retrieved — K-83); three other stale \pending{} (K-84); 110 vs 108 MB/s (K-85); duplicate citation keys (K-86).

# 7. Appendix C — appC_derivations.tex — ACCEPT WITH FIXES

Every derivation was re-done and holds: the centred post-trend parametrisation; Bartlett weights 1 − ℓ/7 and the positive-semi-definiteness argument; statsmodels defaults (no small-sample correction, normal reference); the placebo rank floor 1/65; segment additivity and the DP recursion with limits mh ≤ i ≤ j − h; BIC and LWZ (c₀ = 0.299, exponent 2.1) as coded; the PELT pruning proof with K = 0 for ℓ₂ and the 2σ² log n penalty; the NIG one-step and batch updates and the Student-t predictive (df 2α, scale² β(κ+1)/(ακ)); R_{t+1}(0) = H; the Gini trapezoid derivation, its bounds 0 and 1 − 1/n and the monotone effect of adding uncited works (checked algebraically); ²D^Z = 1/(1 − Δ_RS); the directed-modularity decomposition; MASI with the 0.67/0.33 rounding (max error 0.0034) and the squared-distance caveat; the score rule and the non-transitive tie-break; the filtration argument. Fixes: the realised cutoff months (K-90); the paragraph that documents Part IV's contradiction must disappear once Part IV is fixed (K-91); "at least two post-event months" (K-92); "fifteen columns" (K-93).

# 8. Appendix E — appE_data_dictionary.tex — ACCEPT WITH FIXES

Column lists match the Arrow schemas and the SQL statements (slim, works, authorships, small tables, locations, key tables, link tables, corpus_works, citation windows, edge tables, classification, embeddings). The first_date, date_precision and citation-merge descriptions are the correct ones. Two minor items: the created_date caveat (K-100) and the missing data record type (K-101).

# 9. Appendix H — appH_reproducibility.tex — ACCEPT WITH FIXES

Steps, incidents, amendments (six) and licences agree with RUNBOOK.md, INCIDENTS.md, amendments.csv and the ledgers; both unit-test files exist; the 15,490,159 / 464,765,485 / 1,898 figures are right and expose the mislabel in Part III. Minor: the "28 cores" has no source in the environment capture and conflicts with the cluster inventory (32) (K-110); three provenance counts are typed although macros exist (K-111); the "same locked environment" on the three other nodes is unverified (K-112); arXiv licence wording differs from Part III (K-113).

# 10. Cross-cutting observations for the Director

1. Stale \pending{} markers (Part III, Appendix B) and stale "not yet / not implemented" statements (Part III linkage sample, Part IV donut, diversity, known set) show that the writers worked from an earlier state of results/ and numbers.tex. A rule "regenerate numbers.tex, then grep the sections for \pending{ and for every macro name" before each writing pass would have caught K-27, K-83, K-84 and the undefined macro.
2. The mislabelled first-pass count (B3) originates in define_corpus.py (openalex_cs_topic_2018_2026_pass1 counts the union view). This is a code fix with manuscript consequences in four places; it should carry an entry in reports/DISCREPANCIES_PLAN_VS_CODE.md.
3. The 6,263 orphan records (B2) are the first empirical sign that the arXiv clause of the membership rule is inert outside the topic filter. Before Part IX is written, the Director should decide whether to amend the rule (A-006) or to keep it and reword Part IV; either way the PRISMA figure must show the class.
4. The preprint-citation merge (B4) is a methodological choice that affects every impact statistic and is currently documented only in a data-dictionary sentence; Part IV §Fixed citation windows must own it.
5. No causal language about the reference date was found in any of the eight files; superlatives are limited to the three flagged in Part II; no preprint is called peer-reviewed (the opposite error occurs once, K-10); abstracts are never treated as full text (Part IV §Knowledge-graph extraction is explicit).

# Summary (≤ 20 lines)

• Verdicts: Part II ACCEPT WITH FIXES; Part III REJECT; Part IV REJECT; App A, C, E, H ACCEPT WITH FIXES; App B ACCEPT WITH FIXES plus one compile-blocking macro. 71 issues (8 blocking, 20 major, 43 minor) in the CSV.
• The manuscript does not compile: \LinkNeededOaIdsSecondPassArxivDoi is undefined (Part III l.288, Part IV l.235, App B l.182).
• PRISMA accounting fails: core + penumbra = 10,512,764 ≠ 10,519,027 after exclusions; 6,263 records belong to neither, and they show that the arXiv membership clause never admits works outside the OpenAlex topic filter (Part IV overstates it).
• The "first-pass" identification macro (16,592,966) is the two-pass total; the first pass retained 15,490,159 (Parts II/III and App B wrong, App H right).
• Part III says preprint citations are not merged into the published node; the code merges them (App E agrees with code).
• Part IV's first-date and date-precision equations do not match define_corpus.py (1-January dates are overridden).
• Part IV contradicts the code on: donut windows ("not implemented" — they are), the 1,000-replication bootstrap (does not exist), training cutoffs for h = 24, the known-emergence lead time (a different script exists), the HHI-without-preprints table.
• Part II misreports Bornmann & Mutz 2015 (modern growth is 8–9 %/yr, not 3 %) and claims period-specific seasonality the code does not fit; the Lin et al. 2020 "majority unmatched" claim could not be verified and is probably reversed.
• Linkage precision: the sample exists (DOI 0.94, title 1.00 automatic agreement) and contradicts the assumption that only the title stratum can err; Part III still says the sample was not drawn.
• Three \pending{} markers in Part III and six in App B refer to macros that already exist; amendments are six, not five.
• All mathematics in Part IV and Appendix C re-derives correctly; Appendix A's counts recompute exactly from ontology.yaml.
• Duplicate bibliography entries under two keys for at least six works (20 duplicated DOIs in references_all.bib).
• No causal language, no preprint called peer-reviewed, no abstract treated as full text; three superlatives to soften.