SPB Git forge
38commits 1branches 0releases
338.7 MBsize
maindefault branch
3 h agolast push
HTML 53.9% TypeScript 44.5% JavaScript 0.6% SQL 0.5%
2.2 KB · 37 lines markdown
Rendered Raw Blame History
1# CDC Data Centres (`cdc`)23- **Domain**: `cdc.com` · **Kind**: operator · **Priority**: 2 · **Enabled**: yes4- **License / attribution**: Publicly accessible operator pages; factual data only; © CDC Data Centres — attribution “CDC Data Centres”5- **Coverage**: AU, NZ67## What it collects89Facility records (`kind: facility`) via parser(s) `cdc_campus_v1` in `apps/worker/src/connectors/operators2/apac.ts`.1011Fields published by the pages and captured: one record per campus: name, city, countryIso2, totalPowerMw (operating) or plannedPowerMw (planned), status, building codes as aliases.1213Every field carries an extraction method in `methods` (e.g. `json-ld:PostalAddress`, `regex:it_mw_v1`, `attr:data-value+MW`, `lookup:city-country`) for provenance. Coordinates are only taken from JSON-LD / embedded JSON / map links published by the operator — never geocoded. Status defaults to `operational` unless the page uses pipeline wording (under construction / coming soon / planned).1415## Discovery1617- Sitemaps: auto (/sitemap.xml, /sitemap_index.xml)18- Seeds: `/locations/auckland/`, `/locations/canberra/`, `/locations/melbourne/`, `/locations/sydney/`, `/locations/perth/`, `/locations/maddington/`19- Include: `/locations/[a-z-]+/$`, `/news/.`20- Exclude: `\?`, `#`21- Classify: `/locations/[a-z-]+/$` → facility_pages (facility_page); `/news/.` → newsroom (press_release)2223Newsroom URLs (when configured) rely on the GenericConnector news fallback (press_release → `news_event` when data-center relevant).2425## Schedule & fetch2627- Schedule: facility_pages weekly, newsroom daily, sitemap weekly28- Fetch: L1→L2, 15 rpm, concurrency 2, robots.txt respected, max 20 premium credits/run. All pages are served at L1 (plain HTTP); `maxLevel` is capped at 2 so false-positive “blocked” heuristics (HubSpot / cookie-consent strings) never spend Scrapfly credits.2930## Quirks3132City pages /locations/<city>/ list campuses with 'Current capacity (operating) NN MW' or 'Total capacity NN MW' (under construction) → one campus-level record each. cdcdatacentres.com.au redirects here.3334## Verification (live, 2026-09-11)3536`try-connector --limit N`: discovered 6 URLs (facility_pages=6); sampled pages → 7 entities, 7 valid, 0 rejected, 0 premium credits.37