SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
2.8 KB · 71 lines json
Raw Blame History
1{2  "id": "gcd",3  "displayName": "Grand Comics Database (comics.org catalog)",4  "sourceId": "gcd",5  "sourceName": "Grand Comics Database",6  "sourceType": "catalog",7  "sourceUrl": "https://www.comics.org",8  "module": "api/gcd",9  "enginePriority": [10    "api"11  ],12  "categories": [13    "comics",14    "marvel_comics",15    "dc_comics",16    "independent_comics",17    "manga"18  ],19  "regions": [20    "US",21    "GB",22    "CA",23    "FR",24    "DE",25    "IT",26    "ES",27    "NL",28    "SE",29    "JP",30    "BR"31  ],32  "languages": [33    "en",34    "fr",35    "de",36    "it",37    "es",38    "nl",39    "sv",40    "ja",41    "pt"42  ],43  "currency": [44    "USD"45  ],46  "supportsListings": false,47  "supportsSold": false,48  "supportsAuctions": false,49  "supportsImages": false,50  "supportsCatalog": true,51  "supportsPopulation": false,52  "supportsLookup": true,53  "refreshFrequencyMinutes": 60,54  "priority": "high",55  "trustScore": 0.9,56  "attributionRequired": true,57  "termsUrl": "https://www.comics.org/about/",58  "accessNotes": "Reads only the public, unauthenticated JSON REST API of the Grand Comics Database: https://www.comics.org/api/series/?format=json&page=N (50 series per page, 232,658 series → ~4,650 pages), /api/series/<id>/, /api/issue/<id>/ and /api/publisher/<id>/ (publisher names are cached per run). No login, no cookies; the API root advertises only `series` and `publisher` list endpoints and ignores filter parameters, so discovery is a resumable page walk (cursor = {page, index}). robots.txt (Cloudflare managed): `User-agent: *` Allow: / with Content-Signal search=yes, ai-train=no — we index catalog facts and link back; the HTML pages and the /download/ SQL dumps sit behind a Cloudflare managed challenge and a GCD account and are NOT fetched. GCD data is CC-BY-SA: every record carries the licence in metadata and links to its comics.org issue page. Rate limit (verified live 2026-09-08, twice): the API throttles per client at roughly 60 requests per hour (HTTP 429 with Retry-After up to ~1,741 s; a second burst of 7 requests 30 min later was refused again), so each hourly run is capped at maxRequestsPerRun (30) requests spaced 2.5 s apart, stops cleanly at its last checkpoint on the first 429 (anomaly 'rate_limited'; the comics.org domain policy sets maxRetries 0 so the engine never sleeps on Retry-After) and resumes from the cursor next run. Cover prices are stored as originalMsrp only when GCD prints an ISO currency (e.g. '0.12 USD'); dates with unknown day/month (GCD '00') are kept as year only, never completed.",59  "enabled": true,60  "schemaVersion": "1.0",61  "acquisitionMethod": "official JSON API (comics.org/api, no key)",62  "historicalDepth": "decades",63  "requires": [],64  "config": {65    "seriesPerRun": 5,66    "backfillSeriesPerRun": 5,67    "issuesPerSeries": 6,68    "maxRequestsPerRun": 3069  }70}71