# F4-conf-04 — Verification of a five-item reference list

# Question

For each of five references (R1–R5) cited in a draft manuscript, does (i) the entry's title match the source document, (ii) the entry's year equal the first-submission year or the reproduced-version year given in the extract's "Bibliographic record" line, and (iii) the finding attributed to the reference ("cited_for") appear in, or follow arithmetically from, the provided extract? A reference failing any check is unverifiable.

# Data and exclusions

Inputs: data/reference_list.json (5 references) and five source extracts under data/sources/ plus data/README.md. Line counts are in n_input_rows (README 17 lines; extracts 66, 91, 107, 58, 134 lines; 5 JSON items). SHA-256 values of all files match data/README.md (printed by out/analysis.py; see out/state/data_audit.md). No reference was excluded (out/exclusions.csv: 5 → 5); n_analysed = 5 references, each with three checks. Only the extracts were used as evidence; no remembered paper content or external databases.

# Method

out/analysis.py (deterministic, no randomness, random_seed = null):

1. Title: entry title vs. the extract's Title: header and body # title after normalisation (case, punctuation, diacritics, "&"/"and").
2. Year: entry year vs. the first and the last "submitted" year parsed from the Bibliographic record line.
3. Finding: each element of cited_for is checked against specific table cells / sentences, which the script extracts and prints (magnitudes, SEs/CIs, significance, timing, population, design), plus arithmetic checks (e.g. exp(ATT)−1, coefficient/mean). A naive baseline (header title/year + every number in cited_for appearing verbatim somewhere in the extract) is reported as baseline_unverifiable_references / baseline_n_verified.

# Results

unverifiable_references = ["R3"]; n_verified = 4 (see reference_checks).

• R1 (Armendariz 2026) — all checks pass (reference_checks.R1). Table 6, 5 km column: ATT −0.431** (0.210), −35.0%; text: significant at 5%, staggered DCR introduction, individual-level hospital records.
• R2 (Li, Pearson-Merkowitz & Yokum 2026) — all pass (reference_checks.R2). Table 2 ATE column: $1,153 (95% CI [166, 2,141]), 1.47 pp, −1.99 weeks; abstract: preregistered RCT, N = 23,549.
• R3 (Liu & Zhao) — fails all three (reference_checks.R3). Entry title "Restaurants, Cafés and Neighbourhood Change: Commercial Entry and House Prices in Greater London" differs from "Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London"; entry year 2024 vs. 2026 (v1 and v2). The finding is contradicted: the stacked coefficient is 0.005 at τ=0, rising gradually to 0.037 (τ=4) and 0.034 (τ=5) — 3.4–3.7% is reached after four to five years, not in the entry year.
• R4 (Kanayama, Miyaji & Otani 2025) — all pass (reference_checks.R4); year 2025 = v1 year. Appendix Table 2: employment elasticity w.r.t. MW −0.388 (0.132); own-wage elasticity −0.732 (0.091); Japan's spot labour market.
• R5 (Tossou 2025) — all pass (reference_checks.R5). Table 3 col (4) (stated preferred specification): −0.0962 (0.0129), % change −0.3685, i.e. 36.85% of the mean 0.261.

The naive baseline gives the same set (baseline_unverifiable_references = ["R3"], baseline_n_verified = 4), but only via R3's title/year failure: the numbers "3.4–3.7" do occur in the R3 extract, so number matching alone would not have caught the misattributed timing.

# Uncertainty

The checks are deterministic judgements, so there is no sampling uncertainty. The judgement calls are listed in data_issues: (a) R1's Table A19 prints SE 0.218 for the same estimate (Table 6: 0.210), but 5% significance holds either way (0.431/0.218 ≈ 1.98); (b) R5's printed −0.3685 vs. computed −0.36858 is a last-digit truncation. Neither changes a verdict. The R3 verdict does not depend on judgement: any one of its three failures would be enough.

# Limitations and open issues

• Source-internal inconsistencies recorded in data_issues: R1 (SE 0.210 vs 0.218; 608 vs 640 control regions; fatal-outcome reduction "35%" in the abstract vs "approximately 30%" in Sec. 6.1); R4 (the own-wage elasticity's SE (0.091) is identical to the SE of "Affected employment", which looks wrong for a ratio estimate. The citation copies the printed value, so it passes the "stated in the extract" rule, but whether that SE is reliable cannot be settled from the extract); R5 (abstract headline 0.0855/18.79% refers to a different outcome measure, from Table 2, which is not in the extract).
• The verdicts apply only to the extracts. Omitted sections cannot be checked, and no other versions were consulted.
• Title matching is a normalised exact-string comparison. It would mark a title with any wording change as not matching. That is consistent with the task rule, and the R3 title differs entirely in any case.