SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
2.3 KB · 41 lines json
Raw Blame History
1{2  "id": "miniature-market",3  "displayName": "Miniature Market (board game retail listings)",4  "sourceId": "miniature-market",5  "sourceName": "Miniature Market",6  "sourceType": "dealer",7  "sourceUrl": "https://www.miniaturemarket.com",8  "module": "api/miniature-market",9  "enginePriority": ["api"],10  "categories": ["board_games"],11  "regions": ["US"],12  "languages": ["en"],13  "currency": ["USD"],14  "supportsListings": true,15  "supportsSold": false,16  "supportsAuctions": false,17  "supportsImages": true,18  "supportsCatalog": false,19  "supportsPopulation": false,20  "supportsLookup": false,21  "refreshFrequencyMinutes": 1440,22  "priority": "low",23  "trustScore": 0.75,24  "attributionRequired": true,25  "termsUrl": "https://www.miniaturemarket.com/terms-and-conditions",26  "accessNotes": "Reads only public category listing pages of the Shopware 6 storefront over plain HTTPS with the RareIndex bot UA (no anti-bot; 200 OK): https://www.miniaturemarket.com/board-games.html?p=N (24 product boxes per page, 379 pages) and /board-games/new-releases.html for freshness. Each product box exposes name, product URL, image, SKU (data-sku), the Shopware product id (buy-widget redirect parameters), 'Retail Price' (publisher list price → originalMsrp) and 'Our Price', plus the buy button (Add to cart / Preorder). robots.txt for `User-agent: *`: disallows /checkout/, /account/, /wishlist/, /search/ and the ?SID=, ?dir=, ?mode=, ?limit= parameters, explicitly allows ?p= pagination, and sets Crawl-delay: 10 — we send at most one request per 10 s, concurrency 1, and never use the disallowed sort/limit parameters. No product detail pages (not needed), no search, no account/cart. Retail asking prices are listings, never sales (SPEC §111); the list price is kept as originalMsrp for board games (taxonomy has no publisher/edition catalog for board games yet — BGG is blocked).",27  "enabled": true,28  "schemaVersion": "1.0",29  "acquisitionMethod": "plain HTTP — public Shopware listing pages",30  "historicalDepth": "none",31  "requires": [],32  "config": {33    "pagesPerSeed": 2,34    "seeds": [35      { "path": "/board-games/new-releases.html", "categorySlug": "board_games", "pages": 3 },36      { "path": "/board-games.html", "categorySlug": "board_games", "pages": 2 }37    ],38    "exclude": "sleeves?|playmat|dice set|token set|insert|organizer|gift card|paint set|brush"39  }40}41