# F4-conf-04 — Verification of a five-item reference list

## Question
For each of five cited arXiv studies, does (i) the title in the reference entry match the document, (ii) the year agree with the first-submission year or the year of the version in the extract, and (iii) is the finding the manuscript attributes to it ("cited_for") stated in the extract or does it follow arithmetically from it? A reference that fails any of these checks is **unverifiable**.

## Data and exclusions
- `data/reference_list.json` (5 references) and the five extracts in `data/sources/` (134, 58, 107, 66 and 91 lines), plus `data/README.md` (17 lines) for provenance. The only evidence used is the extracts (header and body). I did not use anything I remember about the papers or any other version of them.
- Nothing was excluded. All 5 references were assessed (15 checks in total).

## Method
`out/analysis.py` does everything with the standard library:
- **Title:** the entry title, the header `Title:` line and the body `# ` title line are compared after normalisation (case, punctuation, diacritics, "&"→"and").
- **Year:** the entry year is compared with the first-submission year and with the year of the reproduced version, both taken from the "Bibliographic record" line.
- **Finding:** for each reference the script pulls out the table cells and text lines that decide the check and prints them. It then compares every element of `cited_for` (magnitude, SE/CI, significance, timing, population or design) with what is printed. Where a value follows arithmetically, the script recomputes it: |t| = ATT/SE; exp(ATT)−1; β/mean; affected employment / affected wage.

## Results
| id | title | year | finding | locator |
|---|---|---|---|---|
| R1 Armendariz 2026 | ✓ | ✓ (2026) | ✓ | Table 6, 5 km column: ATT −0.431**, (0.210), −35.0%; |t| = 2.05 |
| R2 Li et al. 2026 | ✓ | ✓ (2026) | ✓ | Table 2, ATE column: $1,153 [166, 2,141]; 1.47 pp; −1.99 weeks; abstract: N = 23,549, preregistered RCT |
| R3 Liu & Zhao "2024" | ✗ | ✗ (submissions 2026) | ✗ | Section 3.2 and Table 3 (Clean-onset row): 0.005 at τ=0, rising to 0.037 at τ=4 and 0.034 at τ=5 |
| R4 Kanayama et al. 2025 | ✓ | ✓ (v1 2025) | ✓ | App. A.1 Table 2: −0.388 (0.132); −0.732 (0.091) |
| R5 Tossou 2025 | ✓ | ✓ (2025) | ✓ | Table 3 col (4), the preferred specification: −0.0962 (0.0129); % Change −0.3685 |

**Unverifiable references: R3.** It fails all three checks:
- **Title:** the entry's title ("Restaurants, Cafés and Neighbourhood Change: Commercial Entry and House Prices in Greater London") is a different title from the document's ("Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London").
- **Year:** the entry gives 2024, but v1 was submitted in March 2026 and v2 in April 2026.
- **Finding:** the source reports a *gradual* rise, from about 0.5% in the event year to 3.4–3.7% after four to five years, and says explicitly that "prices do not jump immediately". That contradicts the cited "jump of 3.4–3.7% in the year of entry with no further appreciation".

**n_verified = 4.**

Notes on the four verified references:
- **R1:** the abstract and Section 6 confirm a staggered difference-in-differences design on Swiss individual-level hospital records.
- **R4:** the extract itself calls −0.732 the "own-wage employment elasticity". It also equals −0.268/0.366, which the script recomputed.
- **R5:** −0.0962/0.261 = −0.3686, against −0.3685 printed. The gap is consistent with the mean being rounded. The abstract's headline (0.0855, 18.79%) is a different estimate from Table 2, which is not in the extract. The citation correctly uses Table 3 col (4), which the text calls the "most preferred specification".

## Uncertainty
No sampling or statistical uncertainty is involved; these are deterministic comparisons of text. The judgement-dependent points are:
1. **R1 standard error.** The same CS-DR 5 km estimate is printed with SE (0.210) in Table 6 but (0.218) in Table A19. The citation quotes the main table (Table 6), so I count it as supported. Significance at 5% holds under either SE: |t| = 2.05 or 1.98, and both tables print `**`.
2. **R5 rounding.** The 0.0001 gap between −0.0962/0.261 and the printed −0.3685 is treated as rounding. The citation quotes the printed 36.85%.
3. **R3 is not borderline.** Each of its three failures would make it unverifiable on its own.

## Limitations and open issues
- The checks are only as good as the extracts. Content outside the extracts (for example Table 2 in Tossou, or the omitted sections of each paper) could not be consulted, as the task requires.
- Internal inconsistencies in the sources are recorded in `data_issues` but not resolved. In Armendariz:
  - the SE for the same estimate differs between Table 6 (0.210) and Table A19 (0.218);
  - the Table A19 note says 640 control regions, while the table rows say 608;
  - the abstract says fatal hospitalisations fall by 35%, but Table 7 gives −0.354 (≈ −29.8%) and Section 6.1 says ≈30%;
  - the paragraph discussing Table A19 is cut off.

  None of these affects the R1 finding as cited.
- Rendering defects were handled as documented in `data_issues`: re-set tables in Li et al., the garbled Table 3 header in Liu & Zhao, inline footnotes in Tossou, and SEs on separate lines.
- The title comparison is automated with normalisation. A title differing only typographically would pass; the only mismatch found (R3) is a different title, not a typographic variant.