spb/datacenterindex

1# Digital Realty — facility pages (Next.js, __NEXT_DATA__ Drupal node) + press releases2id: digitalrealty3name: Digital Realty4domain: www.digitalrealty.com5kind: operator6mode: hybrid7priority: 18license: "Publicly accessible operator pages; factual data only; © Digital Realty Trust, Inc."9attribution: "Digital Realty"10homepage: https://www.digitalrealty.com/data-centers11notes: >12  Open site, plain HTTP works with the bot UA. Only the English sitemap is crawled (the index lists 6 other locales13  that mirror the same facilities). Facility URLs: /data-centers/<region>/<metro>/<code>; metro pages are indexes.14  No RSS feed exists; press releases are classified from the sitemap (/about/newsroom/press-releases/<id>/<slug>).15  robots.txt disallows /taxonomy/ and /node/ only.16coverage: { global: true, operators: ["Digital Realty"] }17fetch:18  level: 119  maxLevel: 420  rpm: 2021  concurrency: 222  respectRobots: true23  maxCreditsPerRun: 4024  maxCreditsPerDay: 120   # premium credits per UTC day for this connector (all runs, all workers)25discovery:26  sitemap: ["https://www.digitalrealty.com/en-sitemap.xml"]27  include:28    - "^https://www\\.digitalrealty\\.com/data-centers/"29    - "^https://www\\.digitalrealty\\.com/about/newsroom/press-releases/"30  exclude: ["\\?", "/taxonomy/", "/node/"]31  classify:32    - { pattern: "/data-centers/[a-z-]+/[a-z-]+/[a-z0-9-]+$", group: facility_pages, pageType: facility_page, priority: 60 }33    - { pattern: "/data-centers/[a-z-]+/[a-z-]+$", group: index, pageType: facility_index, priority: 20 }34    - { pattern: "/data-centers/[a-z-]+$", group: index, pageType: facility_index, priority: 10 }35    - { pattern: "/about/newsroom/press-releases/\\d+/", group: newsroom, pageType: press_release, priority: 70 }36    - { pattern: "/about/newsroom/press-releases/?$", group: index, pageType: news_index, priority: 10 }37  maxUrlsPerRun: 200038schedule:39  facility_pages: weekly40  newsroom: daily41  index: monthly42  sitemap: weekly43extractors:44  facility:45    # selected by URL only (classifyPage() may mislabel deep facility URLs; the URL is authoritative)46    match: ["/data-centers/[a-z-]+/[a-z-]+/[a-z0-9-]+$"]47    kind: facility48    parser: digitalrealty_facility_v149defaults:50  operatorName: Digital Realty51  operatorKind: colocation52  facilityType: colocation53parserVersion: v154