SPB Git forge

spb/rareindex

Public
54commits 1branches 0releases
7.1 MBsize
maindefault branch
10 days agolast push
TypeScript 61.9% HTML 37.2% SQL 0.7%
5.7 KB · 118 lines json
Raw Blame History
1{2  "id": "turntable-lab",3  "displayName": "Turntable Lab (US vinyl store, USD)",4  "sourceId": "turntable-lab",5  "sourceName": "Turntable Lab",6  "sourceType": "dealer",7  "sourceUrl": "https://www.turntablelab.com",8  "module": "api/turntable-lab",9  "enginePriority": [10    "api"11  ],12  "categories": [13    "music"14  ],15  "regions": [16    "US"17  ],18  "languages": [19    "en"20  ],21  "currency": [22    "USD"23  ],24  "supportsListings": true,25  "supportsSold": false,26  "supportsAuctions": false,27  "supportsImages": true,28  "supportsCatalog": false,29  "supportsPopulation": false,30  "supportsLookup": true,31  "refreshFrequencyMinutes": 720,32  "priority": "medium",33  "trustScore": 0.75,34  "attributionRequired": true,35  "termsUrl": "https://www.turntablelab.com/policies/terms-of-service",36  "accessNotes": "Turntable Lab (turntablelab.com) runs on Shopify. The connector reads ONLY the public, unauthenticated storefront JSON that every Shopify shop serves to any visitor: /collections/<handle>/products.json?limit=250&page=N for the 15 configured collections (colored-vinyl-editions-alpha, deluxe-editions-date, 180g-vinyl-pressings-date, 7-vinyl-date, 12-vinyl-date, 10-vinyl-date, beggars-group-vinyl-records, classic-rock-vinyl … (+7 more, see config.collections)) and /products/<handle>.json for URL lookups (~800 colored-vinyl editions, 415 deluxe editions, 340 180g pressings + genre collections). robots.txt (verified 2026-09-08, User-agent *): standard Shopify rules — Disallow /admin, /cart, /checkout(s), /orders, /account, /services, /sf_*, /cart.js, /recommendations/products, /cdn/wpm/*.js and the filter/sort crawl traps (/collections/*sort_by*, /collections/*+*, /collections/*filter*&*filter*); no Crawl-delay for *. The /collections/<handle>/products.json and /products/<handle>.json paths we read carry none of those patterns and are allowed. Sitemap declared at /sitemap.xml (not needed). Access behaviour: plain HTTP 200 JSON with the honest RareIndexBot UA, no anti-bot challenge, no login, no key. Market pinning: Shopify Markets localises products.json prices by the requester IP as soon as an Accept-Language header is present (observed on Markets-enabled shops: a Canadian crawler receives CAD-converted prices with content-language en-CA while the JSON carries no currency field). Every request therefore sends the public localization=US cookie — the shop's own home market, exactly what a US visitor gets — so prices are always the declared USD (verified 2026-09-08: 30.00 vs 43.00 on a Markets shop). Politeness: 1 request per 1.5 s, concurrency 1 per host (connectors/domains.d/g3-shops-na.json), crawlDepth pages per collection per incremental run, backfill bounded by backfillMaxPages. Data: asking prices in USD (the shop's declared currency) — these are LISTINGS, never sales (§111); one listing per variant (condition/size/edition), variant SKU kept as identifier, out-of-stock variants kept as 'ended' listings, 0.00-priced placeholders dropped. Dates: published_at only; no fetch time is ever used as a sale date. Deliberately NOT fetched: cart/checkout/account/customer endpoints, per-product .js barcode enrichment (fetchBarcodes=false: one extra request per product is not worth it for listings), the whole-shop /products.json, and unmapped collections — audio components, DJ hardware, cartridges/styli, cables, accessories, clothing, clearance umbrella collections. No personal data is collected; seller = the store itself.",37  "enabled": true,38  "schemaVersion": "1.0",39  "acquisitionMethod": "Shopify storefront products.json",40  "historicalDepth": "none",41  "requires": [],42  "config": {43    "currency": "USD",44    "market": "US",45    "seller": "Turntable Lab",46    "location": "New York, NY, USA",47    "collections": [48      {49        "handle": "colored-vinyl-editions-alpha",50        "categorySlug": "music"51      },52      {53        "handle": "deluxe-editions-date",54        "categorySlug": "music"55      },56      {57        "handle": "180g-vinyl-pressings-date",58        "categorySlug": "music"59      },60      {61        "handle": "7-vinyl-date",62        "categorySlug": "music"63      },64      {65        "handle": "12-vinyl-date",66        "categorySlug": "music"67      },68      {69        "handle": "10-vinyl-date",70        "categorySlug": "music"71      },72      {73        "handle": "beggars-group-vinyl-records",74        "categorySlug": "music"75      },76      {77        "handle": "classic-rock-vinyl",78        "categorySlug": "music"79      },80      {81        "handle": "emo-vinyl-records",82        "categorySlug": "music"83      },84      {85        "handle": "essential-electronic-vinyl",86        "categorySlug": "music"87      },88      {89        "handle": "essential-alternative-new-wave-vinyl",90        "categorySlug": "music"91      },92      {93        "handle": "ambient-music-vinyl",94        "categorySlug": "music"95      },96      {97        "handle": "blues-country-vinyl",98        "categorySlug": "music"99      },100      {101        "handle": "blue-note-records",102        "categorySlug": "music"103      },104      {105        "handle": "audiophile-pressings-dss-pure-vinyl",106        "categorySlug": "music"107      }108    ],109    "rules": [],110    "defaultCategory": null,111    "exclude": "gift card|mystery|\\| (Apparel|T-Shirts?|Merch|Accessories|Turntables?|Speakers?|Headphones?|Cartridges?|Cables?|Posters?|Digital) \\||t-shirt|hoodie|\\bshirt\\b|\\bhat\\b|\\btote\\b|\\bmug\\b|poster|slipmat|slip mat|cartridge|stylus|turntable|speaker|headphone|\\bcable\\b|cleaner|cleaning|\\bbrush\\b|record weight|inner sleeves?|outer sleeves?|\\bcrate\\b|\\bstand\\b|\\bpin\\b|enamel pin|digital (album|download)|\\bsocks?\\b|\\bpatch\\b|\\bsticker",112    "keepOutOfStock": true,113    "fetchBarcodes": false,114    "wholeShop": false,115    "pageSize": 250116  }117}118