TypeScript 61.9%
HTML 37.2%
SQL 0.7%
1{2 "domains": {3 "psacard.com": {4 "minIntervalMs": 6000,5 "concurrency": 1,6 "timeoutMs": 90000,7 "engines": ["firecrawl", "scrapfly"],8 "firecrawl": { "waitForMs": 4000 },9 "country": "us",10 "currency": "USD",11 "notes": "g2-grading (psa-cert): only public /cert/<n> verification pages, one per known cert number. Plain HTTP incl. robots.txt returns a Cloudflare managed-challenge 403; Firecrawl renders the public page (Scrapfly fallback). APR and /pop are behind a Collectors login and are never fetched."12 },13 "cgccomics.com": {14 "minIntervalMs": 5000,15 "concurrency": 1,16 "timeoutMs": 90000,17 "engines": ["firecrawl", "scrapfly"],18 "country": "us",19 "currency": "USD",20 "notes": "g2-grading (cgc-cert): only public /certlookup/<n>/ pages. robots.txt (200) disallows a list of named scrapers only. Cloudflare managed challenge on plain HTTP → Firecrawl. The CGC census / population report is login-gated and never fetched. The cert page itself warns 'search activity has exceeded our limits' when hammered — keep 1 req / 5 s."21 },22 "cgccards.com": {23 "minIntervalMs": 5000,24 "concurrency": 1,25 "timeoutMs": 90000,26 "engines": ["firecrawl", "scrapfly"],27 "country": "us",28 "currency": "USD",29 "notes": "g2-grading (cgc-cards-cert): only public /certlookup/<n>/ pages. robots.txt: User-agent * Allow: /. Cloudflare managed challenge on plain HTTP → Firecrawl."30 },31 "pcgs.com": {32 "minIntervalMs": 5000,33 "concurrency": 1,34 "timeoutMs": 90000,35 "engines": ["firecrawl", "scrapfly"],36 "country": "us",37 "currency": "USD",38 "notes": "g2-grading (pcgs-cert): only public /cert/<n> pages (plus the price guide read by pcgs-priceguide). Plain HTTP incl. robots.txt returns a Cloudflare managed-challenge 403 → Firecrawl. /pop/pcgsnolookup pages are public but not crawled."39 },40 "ngccoin.com": {41 "minIntervalMs": 5000,42 "concurrency": 1,43 "timeoutMs": 90000,44 "engines": ["firecrawl", "scrapfly"],45 "country": "us",46 "currency": "USD",47 "notes": "g2-grading (ngc-cert): only public /certlookup/<cert>/<grade>/ pages. robots.txt: User-agent * Allow: /. Cloudflare managed challenge on plain HTTP → Firecrawl. Census pages are public but not crawled by this connector."48 },49 "pmgnotes.com": {50 "minIntervalMs": 5000,51 "concurrency": 1,52 "timeoutMs": 90000,53 "engines": ["firecrawl", "scrapfly"],54 "country": "us",55 "currency": "USD",56 "notes": "g2-grading (pmg-cert): only public /certlookup/<cert>/<grade>/ pages. robots.txt: User-agent * Allow: /. Cloudflare managed challenge on plain HTTP → Firecrawl."57 },58 "my.taggrading.com": {59 "minIntervalMs": 4000,60 "concurrency": 1,61 "timeoutMs": 90000,62 "engines": ["firecrawl"],63 "firecrawl": { "waitForMs": 8000 },64 "country": "us",65 "currency": "USD",66 "notes": "g2-grading (tag-cert) + tag-pop: robots.txt explicitly allows /pop-report/ and /card/; sitemaps pop*.xml / dig*.xml public. Client-rendered React app → Firecrawl markdown, 8 s wait."67 },68 "beckett.com": {69 "minIntervalMs": 6000,70 "concurrency": 1,71 "engines": ["firecrawl", "scrapfly"],72 "notes": "g2-grading research 2026-09-08: BGS card lookup (/grading/card-lookup, Next.js → /api/grading/*) and the pop report (/grading/pop-report POST) both require a Google reCAPTCHA token → blocked, no connector. robots.txt: User-agent * (no disallow)."73 },74 "gosgc.com": {75 "minIntervalMs": 5000,76 "concurrency": 1,77 "engines": ["api", "firecrawl", "scrapfly"],78 "notes": "g2-grading research 2026-09-08: Angular SPA; cert lookup uses reCAPTCHA (blocked); the pop-report XHR (prod-customer-sgc-api.azurewebsites.net/v1/pop-report/getPopReportByCardSet) answers 'service is only available through website gosgc.com' to non-browser origins — not called. sgclegacy.com PopulationReport.aspx is gone (404)."79 },80 "cbcscomics.com": {81 "minIntervalMs": 5000,82 "concurrency": 1,83 "engines": ["firecrawl", "scrapfly"],84 "notes": "g2-grading research 2026-09-08: grading-notes (cert) search is behind Google reCAPTCHA and api.cbcscomics.com requires a bearer token → blocked. robots.txt disallows /services, /forum and submission PDFs only."85 },86 "acegrading.com": {87 "minIntervalMs": 6000,88 "concurrency": 1,89 "engines": ["firecrawl", "scrapfly"],90 "notes": "g2-grading research 2026-09-08: /pop and /cert are Livewire pages (results loaded by Livewire XHR after interaction, no URL-addressable cert or set page) behind a Cloudflare managed challenge → not implementable with a stateless fetch; robots.txt allows everything."91 },92 "psavideogames.com": {93 "minIntervalMs": 6000,94 "concurrency": 1,95 "engines": ["firecrawl", "scrapfly"],96 "notes": "g2-grading research 2026-09-08: watagames.com now redirects here (WATA is part of PSA). /verify is a 404; WATA/PSA video-game certs verify through psacard.com/cert (psa-cert). robots.txt: Content-Signal search=yes, ai-train=no, use=reference; Allow: /."97 }98 }99}100