# F4-conf-04 — Verification of a five-item reference list

## Question
For each of five cited arXiv studies, I checked whether (i) the title in the entry matches the document, (ii) the year matches the first-submission year or the year of the version in the extract, and (iii) the finding the manuscript attributes to it is stated in the extract or follows arithmetically from it. A reference that fails any of these checks is unverifiable.

## Data and exclusions
Inputs were `data/reference_list.json` (5 items) and five extracts under `data/sources/`; `data/README.md` was also read. Line counts are in `n_input_rows`. All SHA-256 hashes match the README. No reference was excluded (`out/exclusions.csv`): `n_analysed` = 5. The only evidence used was each extract's header ("Title", "Bibliographic record") and its body. Problems found in the sources are listed under `data_issues`.

## Method
`out/analysis.py` does the following:
- **Entry parsing.** It splits each entry into authors, year, title and arXiv id.
- **Title check.** It compares the entry title with the header title and the body title after normalisation (case, diacritics, punctuation, "&"→"and").
- **Year check.** It reads the first-submission year and the year of the reproduced version from the "Bibliographic record" line.
- **Finding check.** It finds and prints the table cells and sentences that decide each finding, then compares them with every element of `cited_for`: magnitude, SE/CI, significance, timing and population/design.

The decisions on the finding checks were made by me after reading each extract in full. The script encodes those decisions as checks against the exact printed cells. A header-only baseline (title and year checks only) is stored as `baseline_unverifiable_title_year_only`.

## Results
- **R1, Armendariz 2026: verified.** Title and year (2026) match. Table 6, 5 km column, prints ATT -0.431** with SE (0.210) and -35.0%. The text states significance at the 5% level, and the abstract describes the staggered DiD on hospital records (`reference_checks.R1`).
- **R2, Li et al. 2026: verified.** In Table 2's ATE column: $1,153 with 95% CI [166, 2,141], 1.47 pp, and -1.99 weeks. The abstract gives the preregistered RCT and N = 23,549 (`reference_checks.R2`).
- **R3, Liu & Zhao: unverifiable, fails all three checks** (`reference_checks.R3`):
  - The entry title "Restaurants, Cafés and Neighbourhood Change: Commercial Entry and House Prices in Greater London" is a different title from "Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London".
  - The year 2024 does not match either submission year (v1 and v2 are both 2026).
  - The finding is contradicted. The extract reports 0.005 log points at τ=0, rising gradually to 0.037/0.034 at τ=4/5 ("prices do not jump immediately"; Section 3.2 and the Table 3 clean-onset row). This rules out a 3.4–3.7% jump in the entry year followed by no further rise.
- **R4, Kanayama et al. 2025: verified.** Year 2025 equals the v1 year. Table 2 prints -0.388 (0.132) and -0.732 (0.091). The abstract places the study in Japan's spot labour market (`reference_checks.R4`).
- **R5, Tossou 2025: verified.** Table 3 column (4), described as the preferred specification, prints -0.0962 (0.0129) and % change -0.3685, i.e. 36.85% of the mean of 0.261 (`reference_checks.R5`).

Summary: `unverifiable_references` = ["R3"] and `n_verified` = 4. The header-only baseline also flags only R3 (`baseline_unverifiable_title_year_only`).

## Uncertainty
No statistical estimation is involved, and `random_seed` is null. The checks are deterministic, but the following judgement calls affect them:
- **R1 SE conflict.** The same CS-DR 5 km estimate has SE 0.210 in Table 6 but 0.218 in Table A19. I treated Table 6, the main table, as decisive. The 5% significance (**) is printed in both tables.
- **R4 SE may be a copy.** The SE of the own-wage elasticity (0.091) is identical to the SE printed for "Affected employment". That looks like a possible duplication in the original paper. The cited SE matches what is printed, but the extract cannot confirm it is the true SE of the ratio.
- **R5 rounding.** Recomputing -0.0962/0.261 gives -0.3686 against the printed -0.3685. The difference comes from the mean being rounded and does not affect the verdict.

## Limitations and open issues
- Only the excerpts were available, so material outside them was not checked.
- R1 has further internal inconsistencies (608 vs 640 control regions; -30%/+47% in Section 6.1 vs 35% in the abstract). They do not affect the cited finding and are listed in `data_issues`.
- R5's abstract headline (0.0855, 18.79%) comes from a different table that is not reproduced in the extract. A reader citing the abstract would quote a different number.
- `feasibility` = "complete".