SPB Git forge
38commits 1branches 0releases
338.7 MBsize
maindefault branch
3 h agolast push
HTML 53.9% TypeScript 44.5% JavaScript 0.6% SQL 0.5%
5.9 KB

# wikidata — Wikidata SPARQL (data center items)

id / implementation wikidata (apps/worker/src/connectors/datasets/wikidata.ts)
kind / mode / priority dataset / api / 3
endpoint https://query.wikidata.org/sparql (GET, Accept: application/sparql-results+json)
entities NormalizedFacility, key wikidata:Q…, externalIds.wikidata
schedule dataset: weekly
requests per run 1 COUNT + ⌈n / pageSize⌉ pages (currently 1 page of 2 000)
license CC0 1.0
attribution "Data: Wikidata (CC0)" (not legally required, kept for transparency)

# License (verified 2026-09-11)

https://www.wikidata.org/wiki/Wikidata:Licensing:

"All structured data in the main, property and lexeme namespaces is made available under the Creative Commons CC0 License (Public domain); text in other namespaces is made available under the Creative Commons Attribution-ShareAlike 4.0 License."

Item descriptions (schema:description) are part of the structured data (main namespace) → CC0 as well.

# Access policy — why respectRobots: false

https://query.wikidata.org/robots.txt contains Disallow: /sparql and Disallow: /bigdata for User-agent: *. That directive targets web crawlers indexing result pages; WDQS is a public query API whose terms are its own limits (https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#Query_limits, retrieved 2026-09-11):

"There is a hard query deadline configured which is set to 60 seconds. […] One client (user agent + IP) is allowed 60 seconds of processing time each 60 seconds. One client is allowed 30 error queries per minute. Clients exceeding the limits above are throttled with HTTP code 429. Use Retry-After header to see when the request can be repeated. […] access to the service is limited to 5 parallel queries per IP."

and the Wikimedia User-Agent policy (descriptive UA with contact — our bot UA complies). The connector runs ≤ 2 sequential queries a week, each ≈ 1–7 s of server time, and logs 429/Retry-After. Decision to confirm by the owner: the YAML sets fetch.respectRobots: false for this API host only; flipping it back makes the connector refuse to run.

# Query

Class: Q671224 "data center" (?item wdt:P31/wdt:P279* wd:Q671224). The Q-id sometimes quoted for data center, Q1063421, is "Charles-Nicolas Peaucellier" (French general) and returns 0 items — do not use it. Subclasses present on 2026-09-11: high performance computing center (Q25345660), China National Supercomputing Center, colocation centre (Q730132), Kommunales Gebietsrechenzentrum, AI data center (Q137571914), modular data center, green data center, information technology center.

Per page: ordered sub-select (ORDER BY ?item LIMIT/OFFSET) then OPTIONAL properties, grouped one row per item (SAMPLE/GROUP_CONCAT) so multi-valued statements never duplicate a facility:

field property mapping
name label (en,fr,de,es,nl,it,pt,ja,zh,mul) items whose label falls back to the Q-id are skipped (5 on 2026-09-11)
aliases skos:altLabel@en
description schema:description
geo P625 WKT Point(lng lat) → precision street, source dataset:wikidata (Wikidata coordinates are usually the building, but precision is not asserted → never exact)
operatorName / ownerName P137 / P127 labels first value; when several operators, all are listed in the description
countryIso2 P17 → P297 only when exactly one country; multiple → null + listed in description
city P131 label "located in the administrative territorial entity" — usually the municipality, sometimes a county/canton (e.g. "Canton of Berne")
openedOn P571 with wikibase:timePrecision 9 = year → YYYY, 10 → YYYY-MM, 11 → full date; years < 1950 dropped (building or organisation inception, not a data-center opening)
status P576 (dissolved) → closed; inception in the future → announced; otherwise null (Wikidata does not assert operation)
facilityType P31 labels HPC/supercomputing → hpc, "AI data center" → ai, colocation → colocation, else inferFacilityType(name+description) or null
website P856
externalIds wikidata: Q…, wikipedia_en: <enwiki url>
facts openedOn (P571), geo (P625) per-field provenance methods wikidata:P571, wikidata:P625

# Verified live (2026-09-11)

text
node node_modules/tsx/dist/cli.mjs scripts/try-connector.ts config/connectors/wikidata.yaml --limit 3 --json /tmp/wikidata.json
→ COUNT = 195 items → 1 page; 190 entities (5 unlabeled skipped), 190 valid, 0 rejected, ~6 s
   warnings: "no country and no coordinates" on the items that have neither P17 nor P625 (expected; reconcile may attach them via operator/name)

Coverage of the 190: 78 with coordinates, 171 with a country, 80 with an operator, 38 with an opening date (≥ 1950), 103 with a website; facilityType: 31 hpc, 9 research, 7 colocation, 5 government, 4 ai, 1 hyperscale, 133 null; status: 5 closed (P576), 185 null. Samples: wikidata:Q227873 "375 Pearl Street" (US, street-precision geo, openedOn 1976, aliases "Intergate.Manhattan", "Verizon Building"); wikidata:Q872555 "Leibniz Supercomputing Centre" (DE, hpc, operator Bavarian Academy of Sciences and Humanities); wikidata:Q138393192 "Equinix LD4" (GB, operator Equinix, website).

# Quirks / gaps

  • Small source (~200 items) but high-value cross-references (Q-ids that OSM wikidata=* / operator:wikidata=* tags and PeeringDB reconciliation can join on).
  • Some items are organisations rather than buildings ("Deutsches Rechenzentrum", dissolved 2001 → closed).
  • SPARQL results are only as consistent as Wikidata: a "skyscraper" (375 Pearl Street) is legitimately an instance of data center there.
  • pageSize is bounded 100–5 000; the COUNT query failing falls back to a single page and logs a warning.