WP38 literature — call log
Scripts: src/literature/verify_refs.py (Crossref query.bibliographic, polite pool, UA WP38-UQO-research/0.1 (mailto:simon-pierre.boucher@uqo.ca)),
src/literature/resolve_dois.py (Crossref /works/{doi} and URL fetches for DOI-less items), src/literature/make_bib.py.
Run with .venv/bin/python src/literature/<script>.py from the project root.
Totals (2026-11-01)
- Seeds in
seed_references.yaml: 129 (streams 1–6). Manual/grey entries inmanual_references.yaml: 17 (3 replace Crossref-unverified seeds, 14 new, of which 12 institutional). - Final: 142 verified / 143 (125 via Crossref bibliographic search, 17 via URL fetch). Unverified and uncited:
kempbenedict2014. - Crossref HTTP calls by scripts: 133 (129 seed queries + 4 type-filtered re-queries) — one cached JSON per call in
crossref_cache/(<key>.json,<key>__type-*.json). - Web fetches by scripts: 17 (
crossref_cache/<key>__url.json, with HTTP status, final URL, content type, page title, text sample). - Ad-hoc exploratory calls outside the scripts (not cached): 1 curl Crossref reachability test, ~27 HTTP probes of candidate grey-literature URLs (several returned 404/403 and were discarded: ISQ document pages, alimentsduquebec.com/fr/a-propos, CRTC monitoring report, JSTOR for Krugman 1980 → replaced by IDEAS/RePEc page), 4 WebSearch queries (Kemp-Benedict MPRA; ISQ Panorama; StatCan Business Counts; Aliments du Québec), 1 WebFetch (lesproduitsduquebec.com).
- Matching rule: title similarity ≥ 0.90 (token-set ratio; strict token-sort fallback when one title has < 60 % of the other's tokens, except when the shorter is the exact main title of the longer), year ±1 (±2 for
rigby2015, online-first 2013), first-author family name present (organisationalnameand swapped given/family tolerated; Crossref records without authors tolerated). Published versions (journal-article, book-chapter, book) are preferred over NBER/SSRN versions when both match. - Two matcher bugs fixed during the run (and visible in the run lines below): a subset false positive ("Reflections" chapter matched Kemp-Benedict) and working-paper versions outranking published ones.
Script run lines
- 2026-11-01T12:52:31 verify_refs.py: 129 seeds, 129 Crossref HTTP calls (rest served from crossref_cache/), 121 verified, 8 unverified
- 2026-11-01T12:53:58 verify_refs.py: 129 seeds, 2 Crossref HTTP calls (rest served from crossref_cache/), 125 verified, 4 unverified
- 2026-11-01T12:54:30 verify_refs.py: 129 seeds, 2 Crossref HTTP calls (rest served from crossref_cache/), 123 verified, 6 unverified
- 2026-11-01T12:54:58 verify_refs.py: 129 seeds, 0 Crossref HTTP calls (rest served from crossref_cache/), 125 verified, 4 unverified
- 2026-11-01T12:56:40 verify_refs.py: 129 seeds, 0 Crossref HTTP calls (rest served from crossref_cache/), 125 verified, 4 unverified
- 2026-11-01T12:58:25 resolve_dois.py: 17 manual entries, 0 Crossref DOI calls, 17 web fetches (rest cached); total 142 verified / 143