1# data/raw — provenance (nothing here is committed except this file; copies live on M3U96b:~/wp44_uqo/data/raw)23| Source | What | Obtained | Size | Integrity |4|---|---|---|---|---|5| OpenAlex works snapshot, release 2026-10-14 | `data/jsonl/works/` 1,898 part files, 464,765,485 works, 675.3 GB gz | 2026-11-02 21:19 → 23:05 ET, HTTPS from `openalex.s3.amazonaws.com`, 8 parallel streams | filtered CS subset `works/cs/*.jsonl.zst` 20 GB (15,490,159 records), slim table `works/slim/*.parquet` 9.3 GB (all works) | manifest (`works_manifest_2026-10-14.json`), per-file content length and SHA-256 in `works/ledger.sqlite` (1,898/1,898 ok, 0 parse errors, record total = manifest total) |6| OpenAlex works snapshot, second pass (id set) | the same 1,898 part files re-scanned for the 1,116,606 OpenAlex ids of DBLP-/arXiv-linked works outside the CS-topic filter (`data/metadata/needed_oa_ids.csv`) | 2026-11-03 15:55 → 17:55 ET (interrupted twice by node freezes, completed on M5M36 with 10 workers) | `works_pass2/cs/*.jsonl.zst`, 1,102,807 records (98.8 % of the ids; the remainder are ids merged or deleted between the slim table and the record scan) | `works_pass2/ledger.sqlite` (1,898/1,898 ok, SHA-256 per file) |7| OpenAlex entity snapshots | topics 4,516; subfields 252; fields 26; domains 4; sources 268,629; institutions 143,455; publishers 10,713; funders 41,799; concepts 65,026 | 2026-11-02 | 0.6 GB | `entities_ledger.json` (SHA-256 per file) |8| OpenAlex API increment | works with any CS topic, publication_date 2026-10-01 → 2026-11-02, sliced by day × (CS primary / non-CS primary), basic paging ≤ 10,000 per slice | 2026-11-03 (public budget $0.10/day → several days) | raw JSON pages `api_increment/pages/` | `api_increment/ledger.json` (counts, cost) |9| DBLP | `dblp.xml.gz` (2026-11-02 22:39 UTC), `dblp.dtd`, `.md5` | 2026-11-02 | 1.12 GB | MD5 `ff1c424c3b4cef36582f647c36df27c0` verified; parse report `dblp_parse_report.json` (8,840,866 records) |10| arXiv OAI-PMH | set `cs`, metadataPrefix `arXiv`, datestamp 2018-01-01 → 2026-11-02, monthly windows, raw XML pages kept | 2026-11-02/03 | 758 pages, 915,115 unique records | `harvest_state.json`, `arxiv_parse_report.json` |11| ACL Anthology | `anthology+abstracts.bib.gz` | 2026-11-02 | 46.7 MB, 138,796 entries | SHA-256 `44c3f7e907f221b740bea09069818a3d7c0dfd88b2da3b750d9f02955efc58b3` |12