# F4-conf-04 — Verification of a five-item reference list

## Question
For each reference R1–R5 I checked three things against its provided arXiv extract: (i) does the title match, (ii) is the year consistent with the submission dates in the "Bibliographic record" line, and (iii) is the attributed finding stated in the extract, or does it follow arithmetically from it? A reference is unverifiable if it fails any of the three checks.

## Data and exclusions
Inputs are `data/reference_list.json` (5 items) and five extracts under `data/sources/`. Line counts are in `n_input_rows`. All SHA-256 values match `data/README.md`. No reference was excluded (`out/exclusions.csv`: 5 before, 5 after), so all 5 were assessed (`n_analysed` = 5). The anomalies I found in the sources are listed in `data_issues`:
- **R1 source:** SE 0.210 in Table 6 vs 0.218 in Table A19 for the same estimate; 608 vs 640 control regions; the abstract says 35% for fatal outcomes, while Section 6.1 says about 30%.
- **R2 source:** a rendering defect, "p ¡ 0.001".
- **R3 source:** a garbled header in Table 3.
- **R4 source:** the own-wage elasticity has the same SE as affected employment.
- **R5 source:** the headline result comes from an omitted Table 2, and there is a rounding difference of 0.0001 in Table 3.

## Method
`out/analysis.py` works through these steps:
1. Parses each entry into authors, year, title and arXiv id, and parses the extract's header title, body title and submission years.
2. **Title check:** compares titles after normalisation (case, punctuation, diacritics, "&" vs "and").
3. **Year check:** the year is consistent if it equals the v1 year or the year of the reproduced version.
4. **Finding check:** pulls out and prints the exact table cells and sentences that decide each finding. It compares them to the cited values as printed and recomputes simple arithmetic (log-point to percent, coefficient divided by mean).

As a baseline I also ran a check that uses only title and year (`baseline_unverifiable_bibliographic_only`). The analysis has no randomness (`random_seed` = null).

## Results
- **Unverifiable references:** only R3 (`unverifiable_references` = ["R3"]). The other four pass all three checks (`n_verified` = 4). Per-reference records are in `reference_checks`.
- **R1:** passes all checks. Table 6, 5 km column: ATT −0.431**, SE (0.210), −35.0%. The text states 5% significance, and the abstract gives the staggered DiD design on hospital records (`reference_checks.R1`).
- **R2:** passes all checks. Table 2, ATE column: $1,153 [166, 2,141], 1.47 pp, −1.99 weeks. The abstract gives N = 23,549 and a preregistered RCT (`reference_checks.R2`).
- **R3:** fails all three checks (`reference_checks.R3`).
  - Title: the entry's title, "Restaurants, Cafés and Neighbourhood Change…", differs from the document's title, "Does Entry of Food-and-Drink Establishments Raise Local House Prices? Event-Study Evidence from London".
  - Year: the entry says 2024, but v1 and v2 were both submitted in 2026.
  - Finding: the extract reports 0.005 at τ=0, rising gradually to 0.037/0.034 at τ=4/5, and says "prices do not jump immediately". This contradicts the cited 3.4–3.7% event-year jump with no later appreciation.
- **R4:** passes all checks. Appendix A.1 Table 2 prints −0.388 (0.132) and −0.732 (0.091). The entry year 2025 matches v1 (7 May 2025), although the reproduced v3 is from 2026 (`reference_checks.R4`).
- **R5:** passes all checks. Table 3 column (4), which the text calls the preferred specification, prints −0.0962 (0.0129) and a % Change of −0.3685 (36.85%) (`reference_checks.R5`).
- **Baseline:** the title/year-only check also flags only R3 (`baseline_unverifiable_bibliographic_only`).

## Uncertainty
This task is a deterministic check, so there is no sampling uncertainty. The remaining uncertainty comes from judgement calls:
- **R1:** the source is inconsistent about the SE (0.210 vs 0.218). The citation matches the main-results table, and 5% significance holds under either value, so I count it as supported (`data_issues`).
- **R5:** the source's own headline result (0.0855, 18.79%) differs from the Table 3 result that is cited. Since the cited figure is printed exactly and is called preferred, the check passes.

## Limitations and open issues
- The verdicts rest only on the extracts. Anything outside them, such as omitted sections or other versions, was not used and could not be checked.
- I did not resolve the internal inconsistencies in the sources (`data_issues`). I only reported them.
- Author formatting was not checked, following the task rules.
- The task was completed in full (`feasibility` = "complete").