# Reference verification — F4-conf-04

# Question

Do the five references (R1–R5) in a draft manuscript's reference list hold up when checked against the supplied arXiv source extracts? For each one I checked (i) the title, (ii) the year, and (iii) whether the finding attributed to it ("cited_for") is stated in, or follows arithmetically from, the extract. A reference is unverifiable if it fails any of the three checks. The extracts are the only admissible evidence.

# Data and exclusions

Inputs: data/reference_list.json (5 items) and five extracts, with line counts (wc -l) given in n_input_rows. The SHA-256 of every extract matches data/README.md (see out/state/data_audit.md). Nothing was excluded (out/exclusions.csv): 5 references were assessed (n_analysed = 5). I found several problems in the inputs (data_issues, 6 entries):

• arXiv:2609.27732: Table 6 and Table A19 print different SEs for the same 5 km ATT (0.210 vs 0.218), and the two tables give different control counts (608 vs 640). The discussion paragraph for A19 is missing, and the abstract's "35%" for fatal outcomes does not match the ~30% in the text.
• arXiv:2603.03260v2: the Table 3 header is garbled. I used the extract editor's column note.
• arXiv:2606.14621: contains a "p ¡ 0.001" rendering defect.
• arXiv:2505.04555v3: the SE printed for the own-wage elasticity is identical to the SE for affected employment.
• arXiv:2510.03658: the abstract headline (Table 2, not in the extract) differs from Table 3. There is also a last-digit rounding difference in Table 3.

None of these changes a verdict (see data_issues[*].handling).

# Method

out/analysis.py is deterministic and uses no randomness (random_seed = null). For each reference it does three things:

• Title: it parses the entry title and compares it with both the header "Title:" line and the body title line, after normalising case, punctuation, diacritics and "&"/"and".
• Year: it extracts the first-submission year and the year of the reproduced version from the "Bibliographic record" line, and accepts the entry year if it equals either one.
• Finding: it locates and prints the exact table cells and sentences that support or contradict each element of the cited finding (magnitudes, SEs/CIs, significance, timing, population, design), and recomputes the implied arithmetic (exp(ATT)−1, the elasticity ratios, coefficient/mean).

Which elements to check is my own reading of each "cited_for" string. The script records the evidence and the decision.

Baseline (P2): a check on bibliographic fields only (title + year) flags baseline_unverifiable_bibliographic_only = ["R3"], so baseline_n_verified_bibliographic_only = 4.

# Results

• R1 (Armendariz 2026), verified. The title matches and 2026 is consistent. Table 6, 5 km column: ATT −0.431** (0.210), −35.0%. This includes 5% significance and the staggered DiD design on hospital records (reference_checks.R1).
• R2 (Li, Pearson-Merkowitz & Yokum 2026), verified. Table 2 ATE column: $1,153 (95% CI [166, 2,141]), 1.47 pp, −1.99 weeks. The abstract gives a preregistered RCT with N = 23,549 (reference_checks.R2).
• R3 (Liu & Zhao, entry year 2024), unverifiable. It fails all three checks:
  • The entry's title ("Restaurants, Cafés and Neighbourhood Change…") is not the document's title ("Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London").
  • 2024 is neither 2026 (v1) nor 2026 (v2).
  • The extract reports a stacked coefficient of 0.005 at τ=0, rising gradually to 0.037/0.034 at τ=4/5 (Section 3.2; Table 3, clean-onset row). It also says "prices do not jump immediately". This contradicts the claim of a 3.4–3.7% jump in the entry year with no later appreciation (reference_checks.R3).
• R4 (Kanayama, Miyaji & Otani 2025), verified. 2025 is the v1 year. Appendix A.1 Table 2 prints −0.388 (0.132) and −0.732 (0.091) (reference_checks.R4).
• R5 (Tossou 2025), verified. Table 3, column (4), which Section 4.1 names as the preferred specification: −0.0962 (0.0129), % Change −0.3685 = 36.85% (reference_checks.R5).

Unverifiable set: unverifiable_references = ["R3"]; n_verified = 4. Feasibility: feasibility = "complete".

# Uncertainty

No statistical estimation is involved. The verdicts are deterministic comparisons against the printed text. Judgement enters in three places:

1. Splitting each "cited_for" into its elements.
2. Deciding which table decides a finding when the source contradicts itself. For R1, Table 6 decides, not Table A19; the R1 claim of 5% significance holds under either SE (|t| = 2.05 or 1.98, printed by the script).
3. Accepting R4's SE as printed even though it may be a copy error at the source.

# Limitations and open issues

• The checks are limited to the extracts. Content omitted from them (other sections, other versions) could not be consulted, by design.
• R4's own-wage elasticity SE (0.091) duplicates the affected-employment SE. If that is an error in the source, the citation faithfully reproduces an erroneous number. The extract cannot settle this.
• R1's source is internally inconsistent about the SE (0.210 vs 0.218) and the number of control regions (608 vs 640). The citation follows the main table.
• Author formatting was not assessed, per the task rules.