SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
2.7 KB · 42 lines json
Raw Blame History
1{2  "id": "entertainment-earth",3  "displayName": "Entertainment Earth (action figures, Funko, die-cast — retail listings)",4  "sourceId": "entertainment-earth",5  "sourceName": "Entertainment Earth",6  "sourceType": "dealer",7  "sourceUrl": "https://www.entertainmentearth.com",8  "module": "scrapfly/entertainment-earth",9  "enginePriority": ["scrapfly"],10  "categories": ["action_figures", "funko", "model_cars", "designer_toys"],11  "regions": ["US"],12  "languages": ["en"],13  "currency": ["USD"],14  "supportsListings": true,15  "supportsSold": false,16  "supportsAuctions": false,17  "supportsImages": true,18  "supportsCatalog": false,19  "supportsPopulation": false,20  "supportsLookup": false,21  "refreshFrequencyMinutes": 1440,22  "priority": "medium",23  "trustScore": 0.75,24  "attributionRequired": true,25  "termsUrl": "https://www.entertainmentearth.com/s/terms-of-use/s",26  "accessNotes": "Reads only public category listing pages: https://www.entertainmentearth.com/s/action-figures/p, /s/funko/c and /s/vehicles/p (+ ?page=N, 60 product tiles per page, ~73 pages for action figures). Each tile's add-to-cart button exposes data-sku, data-name, data-price, data-company (manufacturer), data-theme (franchise) and data-collect (category); the tile also gives the product URL, image, displayed price, struck list price and the button label (Pre-Order / Add to Cart / Sold Out) which we map to availability. robots.txt (fetched through Scrapfly because plain HTTP gets a Cloudflare challenge): `User-agent: *` disallows only /news/* admin paths; /s/ listing pages and /product/ pages are allowed; a product sitemap index exists at /sitemap/sitemap_index.xml (not used yet). Every request from a non-browser client receives a Cloudflare managed challenge, so pages are fetched through Scrapfly without JS rendering (~1 credit per page, 3 s politeness, concurrency 1). Product detail pages are NOT fetched (not needed; the tiles carry everything we store). Retail asking prices are listings, never sales (SPEC §111); pre-order and sold-out states are kept so MSRP history survives sell-outs. No account, cart, wishlist or personal data is touched.",27  "enabled": true,28  "schemaVersion": "1.0",29  "acquisitionMethod": "Scrapfly (no JS) — public category listing pages",30  "historicalDepth": "none",31  "requires": ["SCRAPFLY_API_KEY"],32  "config": {33    "pagesPerSeed": 2,34    "seeds": [35      { "path": "/s/action-figures/p", "categorySlug": "action_figures" },36      { "path": "/s/funko/c", "categorySlug": "funko" },37      { "path": "/s/vehicles/p", "categorySlug": "model_cars" }38    ],39    "exclude": "gift card|t-shirt|hoodie|mug|tumbler|poster|backpack|keychain|lanyard|blanket|socks|wallet|pin set|enamel pin"40  }41}42