SPB Git forge
38commits 1branches 0releases
338.7 MBsize
maindefault branch
3 h agolast push
HTML 53.9% TypeScript 44.5% JavaScript 0.6% SQL 0.5%
2.3 KB · 50 lines yaml
Raw Blame History
1# Iron Mountain Data Centers — DISABLED (hard bot-wall; see docs/connectors/ironmountain.md)2id: ironmountain3name: Iron Mountain Data Centers4domain: www.ironmountain.com5kind: operator6mode: hybrid7enabled: false8priority: 19license: "Publicly accessible operator pages; factual data only; © Iron Mountain Incorporated"10attribution: "Iron Mountain Data Centers"11homepage: https://www.ironmountain.com/data-centers12notes: >13  2026-09-11: every direct request (robots.txt, /sitemap.xml, /data-centers, bot or browser UA) returns HTTP 42914  "Vercel Security Checkpoint" (JS attack-challenge). Scrapfly with ASP does pass (200) but is billed 30 credits per15  request (residential proxy 25 + browser 5), and even discovery needs robots + 9 sitemap shards (~300 credits) before16  a single facility page. That exceeds the 150-credit run budget and the "premium only when blocked, never for17  bulk crawl" rule, so no extractor is shipped and the connector is disabled rather than shipping broken or18  budget-breaking extraction. robots.txt (fetched once via Scrapfly) allows /data-centers/ and lists19  https://www.ironmountain.com/sitemap.xml → sitemap1..9.xml. Re-enable when the challenge is lifted or a cheap20  path exists; the facility URL pattern will then need to be confirmed on live pages before adding an extractor.21coverage: { global: true, operators: ["Iron Mountain Data Centers"] }22fetch:23  level: 124  maxLevel: 425  rpm: 1026  concurrency: 127  respectRobots: true28  maxCreditsPerRun: 12029  maxCreditsPerDay: 120   # premium credits per UTC day for this connector (all runs, all workers)30discovery:31  sitemap: true32  include: ["/data-centers/", "/about-us/newsroom/", "/newsroom/"]33  exclude: ["\\?", "/zh-cn/", "/utility/", "/support/", "/resources/landing-pages/"]34  classify:35    - { pattern: "/data-centers/[a-z0-9-]+/[a-z0-9-]+$", group: facility_pages, pageType: facility_page, priority: 60 }36    - { pattern: "/data-centers/?$", group: index, pageType: facility_index, priority: 10 }37    - { pattern: "/newsroom/[a-z0-9-]+", group: newsroom, pageType: press_release, priority: 70 }38  maxUrlsPerRun: 50039schedule:40  facility_pages: weekly41  newsroom: daily42  index: monthly43  sitemap: weekly44extractors: {}45defaults:46  operatorName: Iron Mountain Data Centers47  operatorKind: colocation48  facilityType: colocation49parserVersion: v150