# F4-conf-04 — Checking a reference list against the source extracts

# Question

For each of five references (R1–R5) in a draft manuscript, we checked three things against the provided arXiv extract: (i) whether the title matches, (ii) whether the year matches the first-submission year or the year of the reproduced version, and (iii) whether every element of the finding attributed to the reference ("cited_for") is stated in the extract or follows arithmetically from it. A reference that fails any of the three checks is unverifiable.

# Data and exclusions

• Inputs: data/reference_list.json (5 items) and five extracts under data/sources/. data/README.md was read for provenance. Line counts are in n_input_rows.
• The SHA-256 of every extract matches the value in the README (out/state/data_audit.md).
• No references were excluded (out/exclusions.csv: 5 − 0 = 5). All five were assessed (n_analysed = 5).
• Problems found in the inputs are listed in data_issues. None of them changes a verdict:
  • Armendariz: Table 6 gives SE 0.210, Table A19 gives 0.218 for the same estimate. Table A19 also gives two different control counts (608 and 640). The abstract's 35% fatal / "comparable" narcotics figures disagree with Table 7 and Section 6.1, which give about 30% and 47%.
  • Kanayama et al.: the printed SE of 0.091 for the own-wage elasticity looks implausible (see Uncertainty).
  • Liu & Zhao: the Table 3 header is garbled in the extract.
  • Tossou, Li et al.: footnote and table-layout rendering defects.

# Method

out/analysis.py reads only data/ and does the following:

• Header fields. It parses each extract's "Title:" line, its "Bibliographic record:" line and its body title line.
• Title check. Titles are compared after normalisation: case, diacritics, punctuation, and "&" vs "and".
• Year check. The year is compared with the first and the last submission date in the "Bibliographic record" line.
• Finding check. For each reference the script reads the specific table cells and sentences that decide the check, compares them with the cited values exactly as printed, and prints those lines.
  • For R3, the timing claim is tested against the stacked event-time coefficients reported in Section 3.2.
• Baseline. As a first step we ran a bibliographic-only check (title and year only). It is stored under baseline_* keys.
• No randomness is involved (random_seed = null).

# Results

• Unverifiable: only R3 (unverifiable_references = ["R3"]). Four references pass all three checks (n_verified = 4). The bibliographic-only baseline also flags only R3 (baseline_unverifiable_references_bibliographic_only).
• R1, Armendariz 2026: title, year and finding all pass (reference_checks.R1). Table 6, 5 km column: ATT −0.431**, SE (0.210), −35.0%. The text says the estimate is significant at the 5% level and describes a staggered DiD on hospital records.
• R2, Li, Pearson-Merkowitz & Yokum 2026: all checks pass (reference_checks.R2). The Table 2 average-treatment-effect column gives $1,153 (95% CI [166, 2,141]), 1.47 pp and −1.99 weeks. The abstract gives N = 23,549 in a preregistered RCT.
• R3, Liu & Zhao: fails all three checks (reference_checks.R3).
  • Title: the entry's title ("Restaurants, Cafés and Neighbourhood Change: …") is different from the document's title ("Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London").
  • Year: the entry says 2024, but v1 and v2 were both submitted in 2026.
  • Finding: the source reports 0.005 log points (about 0.5%) in the event year, rising gradually to 0.037 and 0.034 at τ = 4 and 5 (Section 3.2; Table 3, clean-onset row). This contradicts "a 3.4–3.7% jump in the year of entry with no further appreciation".
• R4, Kanayama, Miyaji & Otani 2025: all checks pass (reference_checks.R4).
  • Year: 2025 is the v1 year. The reproduced version (v3) is from 2026, and the rule accepts either.
  • Finding: Table 2 prints −0.388 (0.132) and −0.732 (0.091), and the abstract places the study in Japan's spot labour market.
• R5, Tossou 2025: all checks pass (reference_checks.R5). Table 3, column (4), which the text calls the preferred specification, prints −0.0962 (0.0129) and a % change of −0.3685. This equals −0.0962 / 0.261 and matches the stated 36.85% reduction.

# Uncertainty

The checks are deterministic, so there is no sampling uncertainty. Three verdicts depend on judgement:

• R4. The finding counts as supported because the cited SE is printed in the extract. However, the printed SE of 0.091 for the own-wage elasticity is the same as the SE of "Affected employment". A delta-method approximation from the printed components gives about 0.42, which suggests a possible error in the source itself (data_issues). A stricter reviewer might want this SE confirmed before citing it.
• R1. The cited SE of 0.210 comes from Table 6, while Table A19 prints 0.218 for the same estimate. Both values imply significance at the 5% level (t = 2.05 vs 1.98), so the cited claim holds either way.
• R3. The verdict does not depend on judgement: each of its three checks fails on its own.

# Limitations and open issues

• Only the extracts were admissible, so no other versions or bibliographic databases were consulted. Elements outside the extract could not be checked, but none of the cited findings needed them.
• The source inconsistencies listed in data_issues (Armendariz SE and control-count mismatches, Kanayama SE plausibility) cannot be resolved from the extracts. We report them; we did not correct them.
• The task was completed in full (feasibility = "complete").