SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
2.5 KB · 39 lines json
Raw Blame History
1{2  "id": "hareruya",3  "displayName": "Hareruya (晴れる屋) MTG singles",4  "sourceId": "hareruya",5  "sourceName": "Hareruya",6  "sourceType": "dealer",7  "sourceUrl": "https://www.hareruyamtg.com/en/",8  "module": "api/hareruya",9  "enginePriority": ["api"],10  "categories": ["magic_the_gathering"],11  "regions": ["JP"],12  "country": "JP",13  "languages": ["en", "ja"],14  "currency": ["JPY"],15  "supportsListings": true,16  "supportsSold": false,17  "supportsAuctions": false,18  "supportsImages": true,19  "supportsCatalog": false,20  "supportsPopulation": false,21  "supportsLookup": false,22  "refreshFrequencyMinutes": 1440,23  "priority": "medium",24  "trustScore": 0.8,25  "attributionRequired": true,26  "termsUrl": "https://www.hareruyamtg.com/en/help/agreement",27  "acquisitionMethod": "the storefront's own search JSON endpoint (unisearch_api) per card set",28  "historicalDepth": "none",29  "capabilities": ["live_listings", "images"],30  "accessNotes": "Hareruya is Japan's largest Magic retailer with an English storefront (EC-CUBE). The HTML search page renders its results through a public, unauthenticated JSON endpoint — GET https://www.hareruyamtg.com/en/products/search/unisearch_api?fq.category_id=1&fq.cardset=<id>&fq.price=1~*&rows=60&page=<n> — which we call exactly as the page does (60 docs per page, capped by the site at 4 000 results per query): product id, English/Japanese product name (《Card》[SET]), card_name, language code (1 JP, 2 EN, 3 CS, 4 CT, 5 FR, 6 DE, 7 IT, 8 KO …), JPY price incl. tax, foil flag, stock, weekly_sales, product_class (per language/condition/foil offer), card_condition code and image. Card-set ids/names are read once per run from the <select name=cardset> of the public search form (https://www.hareruyamtg.com/en/products/search, no query parameters). robots.txt allows / and disallows only /en/products/search?*page=* | *sort=* | *order=* (the HTML result pages), /*.csv$, deck downloads, purchase forwards and news; we never request those HTML pages — only the JSON endpoint the page itself loads, ≥ 4 s apart, 1 concurrent, honest UA. Not fetched: buy-list (purchase) prices, product detail pages (only linked), carts, member areas. Prices are shop asking prices → listings, never sales. The numeric card_condition code is kept in metadata only (Hareruya's NM/EX/PL labels are not exposed by the endpoint, so condition is left null rather than guessed).",31  "enabled": true,32  "schemaVersion": "1.0",33  "config": {34    "setsPerRun": 12,35    "maxPagesPerSet": 67,36    "rows": 6037  }38}39