1# Reference verification — F4-conf-0423## Question4Do the five references (R1–R5) in a draft manuscript's reference list hold up when checked against the supplied arXiv source extracts? For each one I checked (i) the title, (ii) the year, and (iii) whether the finding attributed to it ("cited_for") is stated in, or follows arithmetically from, the extract. A reference is unverifiable if it fails any of the three checks. The extracts are the only admissible evidence.56## Data and exclusions7Inputs: `data/reference_list.json` (5 items) and five extracts, with line counts (`wc -l`) given in `n_input_rows`. The SHA-256 of every extract matches `data/README.md` (see `out/state/data_audit.md`). Nothing was excluded (`out/exclusions.csv`): 5 references were assessed (`n_analysed` = 5).8I found several problems in the inputs (`data_issues`, 6 entries):9- **arXiv:2609.27732:** Table 6 and Table A19 print different SEs for the same 5 km ATT (0.210 vs 0.218), and the two tables give different control counts (608 vs 640). The discussion paragraph for A19 is missing, and the abstract's "35%" for fatal outcomes does not match the ~30% in the text.10- **arXiv:2603.03260v2:** the Table 3 header is garbled. I used the extract editor's column note.11- **arXiv:2606.14621:** contains a "p ¡ 0.001" rendering defect.12- **arXiv:2505.04555v3:** the SE printed for the own-wage elasticity is identical to the SE for affected employment.13- **arXiv:2510.03658:** the abstract headline (Table 2, not in the extract) differs from Table 3. There is also a last-digit rounding difference in Table 3.1415None of these changes a verdict (see `data_issues[*].handling`).1617## Method18`out/analysis.py` is deterministic and uses no randomness (`random_seed` = null). For each reference it does three things:19- **Title:** it parses the entry title and compares it with both the header "Title:" line and the body title line, after normalising case, punctuation, diacritics and "&"/"and".20- **Year:** it extracts the first-submission year and the year of the reproduced version from the "Bibliographic record" line, and accepts the entry year if it equals either one.21- **Finding:** it locates and prints the exact table cells and sentences that support or contradict each element of the cited finding (magnitudes, SEs/CIs, significance, timing, population, design), and recomputes the implied arithmetic (exp(ATT)−1, the elasticity ratios, coefficient/mean).2223Which elements to check is my own reading of each "cited_for" string. The script records the evidence and the decision.2425Baseline (P2): a check on bibliographic fields only (title + year) flags `baseline_unverifiable_bibliographic_only` = ["R3"], so `baseline_n_verified_bibliographic_only` = 4.2627## Results28- **R1 (Armendariz 2026), verified.** The title matches and 2026 is consistent. Table 6, 5 km column: ATT −0.431** (0.210), −35.0%. This includes 5% significance and the staggered DiD design on hospital records (`reference_checks.R1`).29- **R2 (Li, Pearson-Merkowitz & Yokum 2026), verified.** Table 2 ATE column: $1,153 (95% CI [166, 2,141]), 1.47 pp, −1.99 weeks. The abstract gives a preregistered RCT with N = 23,549 (`reference_checks.R2`).30- **R3 (Liu & Zhao, entry year 2024), unverifiable. It fails all three checks:**31 - The entry's title ("Restaurants, Cafés and Neighbourhood Change…") is not the document's title ("Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London").32 - 2024 is neither 2026 (v1) nor 2026 (v2).33 - The extract reports a stacked coefficient of 0.005 at τ=0, rising gradually to 0.037/0.034 at τ=4/5 (Section 3.2; Table 3, clean-onset row). It also says "prices do not jump immediately". This contradicts the claim of a 3.4–3.7% jump in the entry year with no later appreciation (`reference_checks.R3`).34- **R4 (Kanayama, Miyaji & Otani 2025), verified.** 2025 is the v1 year. Appendix A.1 Table 2 prints −0.388 (0.132) and −0.732 (0.091) (`reference_checks.R4`).35- **R5 (Tossou 2025), verified.** Table 3, column (4), which Section 4.1 names as the preferred specification: −0.0962 (0.0129), % Change −0.3685 = 36.85% (`reference_checks.R5`).3637Unverifiable set: `unverifiable_references` = ["R3"]; `n_verified` = 4. Feasibility: `feasibility` = "complete".3839## Uncertainty40No statistical estimation is involved. The verdicts are deterministic comparisons against the printed text. Judgement enters in three places:411. Splitting each "cited_for" into its elements.422. Deciding which table decides a finding when the source contradicts itself. For R1, Table 6 decides, not Table A19; the R1 claim of 5% significance holds under either SE (|t| = 2.05 or 1.98, printed by the script).433. Accepting R4's SE as printed even though it may be a copy error at the source.4445## Limitations and open issues46- The checks are limited to the extracts. Content omitted from them (other sections, other versions) could not be consulted, by design.47- R4's own-wage elasticity SE (0.091) duplicates the affected-employment SE. If that is an error in the source, the citation faithfully reproduces an erroneous number. The extract cannot settle this.48- R1's source is internally inconsistent about the SE (0.210 vs 0.218) and the number of control regions (608 vs 640). The citation follows the main table.49- Author formatting was not assessed, per the task rules.50