ironmountain — Iron Mountain Data Centers (DISABLED)
| Config | config/connectors/ironmountain.yaml (enabled: false) |
| Parser | none shipped |
| Kind / priority | operator / 1 |
| License / attribution | Publicly accessible operator pages; factual data only; © Iron Mountain Incorporated |
Why it is disabled (2026-09-11)
- Every direct request to
www.ironmountain.com—robots.txt,/sitemap.xml,/data-centers,/data-centers/global-data-center-locations, with the bot UA and with a browser UA — returns HTTP 429 with a "Vercel Security Checkpoint" JavaScript challenge. This is a hard bot wall, not a rate limit we can wait out. - Scrapfly with ASP does pass (HTTP 200), but each request is billed 30 credits (residential proxy 25 + browser 5, even with
render_js=false). Discovery alone needs robots.txt + the sitemap index + 9 sitemap shards (sitemap1.xml … sitemap9.xml) ≈ 330 credits before the first facility page, i.e. more than the whole 150-credit run budget, and every weekly recrawl would cost the same again. That violates the "premium only when blocked, never for bulk crawling" rule and the daily budget. - Two probe requests were spent to establish this (60 credits): robots.txt and the sitemap index. robots.txt (fetched through Scrapfly) allows
/data-centers/, disallows Sitecore/system folders,/zh-cn/,?localize=and some landing pages, and blocks theScrapyandBoogleBot2agents.
What is in the YAML
Discovery/classify rules are drafted from the public URL scheme (/data-centers/…, newsroom) but no extractor is declared — the facility page template could not be inspected without spending credits, and shipping unverified extraction is not allowed. The connector is enabled: false so the scheduler ignores it.
How to re-enable
- Check whether the challenge has been lifted:
curl -A "DataCenterIndexBot/0.1" https://www.ironmountain.com/robots.txtshould return 200 text. - Inspect 2–3 facility pages, confirm the URL pattern (
/data-centers/<region>/<facility>is an assumption) and writeironmountain_facility_v1inapps/worker/src/connectors/operators1/. - Verify with
try-connector(--limit 4, spot-check two sites), then setenabled: true. If the wall stays, an alternative is Iron Mountain's PeeringDB / registry presence via the registry connectors rather than the operator site.