# data/raw — provenance (nothing here is committed except this file; copies live on M3U96b:~/wp44_uqo/data/raw) | Source | What | Obtained | Size | Integrity | |---|---|---|---|---| | OpenAlex works snapshot, release 2026-10-14 | `data/jsonl/works/` 1,898 part files, 464,765,485 works, 675.3 GB gz | 2026-11-02 21:19 → 23:05 ET, HTTPS from `openalex.s3.amazonaws.com`, 8 parallel streams | filtered CS subset `works/cs/*.jsonl.zst` 20 GB (15,490,159 records), slim table `works/slim/*.parquet` 9.3 GB (all works) | manifest (`works_manifest_2026-10-14.json`), per-file content length and SHA-256 in `works/ledger.sqlite` (1,898/1,898 ok, 0 parse errors, record total = manifest total) | | OpenAlex entity snapshots | topics 4,516; subfields 252; fields 26; domains 4; sources 268,629; institutions 143,455; publishers 10,713; funders 41,799; concepts 65,026 | 2026-11-02 | 0.6 GB | `entities_ledger.json` (SHA-256 per file) | | OpenAlex API increment | works with any CS topic, publication_date 2026-10-01 → 2026-11-02, sliced by day × (CS primary / non-CS primary), basic paging ≤ 10,000 per slice | 2026-11-03 (public budget $0.10/day → several days) | raw JSON pages `api_increment/pages/` | `api_increment/ledger.json` (counts, cost) | | DBLP | `dblp.xml.gz` (2026-11-02 22:39 UTC), `dblp.dtd`, `.md5` | 2026-11-02 | 1.12 GB | MD5 `ff1c424c3b4cef36582f647c36df27c0` verified; parse report `dblp_parse_report.json` (8,840,866 records) | | arXiv OAI-PMH | set `cs`, metadataPrefix `arXiv`, datestamp 2018-01-01 → 2026-11-02, monthly windows, raw XML pages kept | 2026-11-02/03 | 758 pages, 915,115 unique records | `harvest_state.json`, `arxiv_parse_report.json` | | ACL Anthology | `anthology+abstracts.bib.gz` | 2026-11-02 | 46.7 MB, 138,796 entries | SHA-256 `44c3f7e907f221b740bea09069818a3d7c0dfd88b2da3b750d9f02955efc58b3` |