# Reference verification — F4-conf-04

# Question

For each of the five references in data/reference_list.json, does (i) the title match the provided arXiv extract, (ii) the year agree with the first-submission year or the year of the reproduced version, and (iii) the finding attributed to it ("cited_for") appear in, or follow arithmetically from, the extract? A reference failing any check is unverifiable. Only the extracts under data/sources/ were used as evidence.

# Data and exclusions

Inputs: reference_list.json (5 items) and five extracts (line counts in n_input_rows: 134, 58, 107, 66, 91 lines; README 17 lines). All SHA-256 values match data/README.md (out/state/data_audit.md). No reference was excluded (out/exclusions.csv: 5 before, 5 after, 0 removed); n_analysed = 5. Source-internal issues found and how they were handled are listed in data_issues. None of them changes a verdict. The main ones are: Armendariz Table 6 gives SE 0.210 but Table A19 gives 0.218 for the same ATT; the two tables also give 608 vs 640 control regions. Kanayama's own-wage elasticity SE (0.091) is identical to the affected-employment SE. Liu & Zhao's Table 3 header is garbled. In Tossou, the abstract headline (Table 2, not in the extract) differs from the Table 3 result.

# Method

out/analysis.py does four things for each reference. (1) It takes the extract's header title and checks it equals the body title line. It then compares that title with the entry's title after normalising case, punctuation, diacritics and "&". (2) It reads the first-submission year and the year of the reproduced version from the "Bibliographic record" line and accepts the entry year if it equals either one. (3) It confirms that every evidence string used to decide the finding check appears verbatim in the extract and prints it with its line number. (4) It recomputes the arithmetic relations involved in the check. Whether each element of the finding (magnitude, uncertainty, timing, population) is supported was judged by me from the printed evidence, and the script records that judgement. Baseline (baseline_naive_number_match): a purely mechanical check that every number in "cited_for" occurs somewhere in the extract.

# Results

• unverifiable_references = ["R3"]; n_verified = 4.
• R1 (Armendariz 2026): all three checks pass (reference_checks.R1). Evidence: Table 6, 5 km column, ATT −0.431** (0.210), % change −35.0%. The text says the estimate is significant at the 5% level, and the abstract describes the staggered DiD on Swiss hospital records.
• R2 (Li, Pearson-Merkowitz & Yokum 2026): all checks pass (reference_checks.R2). Table 2's ATE column gives $1,153 (95% CI [166, 2,141]), 1.47 pp and −1.99 weeks. The abstract gives N = 23,549 and a preregistered RCT.
• R3 (Liu & Zhao): fails all three checks (reference_checks.R3). The entry's title ("Restaurants, Cafés and Neighbourhood Change: …") is different from the document's title ("Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London"). The entry year 2024 matches neither 2026 (v1) nor 2026 (v2). The cited timing is contradicted by Section 3.2 and Table 3 (Clean-onset threshold-2 row). The extract reports 0.005 (≈0.5%) in the event year, rising gradually to 0.037/0.034 (3.4–3.7%) by years 4–5, and says "prices do not jump immediately".
• R4 (Kanayama, Miyaji & Otani 2025): all checks pass (reference_checks.R4). The year 2025 equals the v1 submission year. Appendix A.1 Table 2 gives −0.388 (0.132) and −0.732 (0.091), and −0.268/0.366 = −0.732.
• R5 (Tossou 2025): all checks pass (reference_checks.R5). Table 3 column (4), which the text names as the preferred specification, gives −0.0962 (0.0129) and a % change of −0.3685. The text states 36.85%, and 0.0962/0.261 = 0.3686.
• Baseline: a naive number match finds every cited number in all five extracts (baseline_naive_number_match). So number matching alone would not catch R3's misattributed timing. In this case R3 is caught by the title and year checks anyway.

# Uncertainty

There is no sampling uncertainty: the checks are deterministic text comparisons. The judgement calls are these:

• R1: the SE cited (0.210) is printed in Table 6, but Table A19 prints 0.218 for the same estimate. I count the finding as supported because the citation reproduces Table 6 verbatim. The 5% significance holds under either SE (|t| = 2.05 or 1.98).
• R4: the SE 0.091 is printed in the source but may be a source typo. It cannot be checked from the extract.
• R5: 0.0962/0.261 gives 36.86%, while the paper prints 36.85% (−0.3685). The difference comes from rounding in the printed inputs and is accepted.

# Limitations and open issues

• The finding judgements involve my reading of the text. The script makes sure that the evidence exists and that the arithmetic holds, but it does not "understand" the claim.
• The extracts omit most sections. A finding stated only in an omitted section could not have been verified. No reference relied on omitted material, except that the Tossou abstract cites Table 2, which is not in the extract and was not needed.
• The source-internal inconsistencies listed in data_issues are reported but not resolved.
• feasibility = "complete".