spb/datacenterindex

1# Claim-first data architecture23Since 2026-09-12 every numerical figure DataCenterIndex extracts is stored as a **claim** before it can become a displayed4value. A claim is one assertion by one document about one subject, with its **scope**, its **semantics**, the **sentence**5that supports it, the **parser** that produced it and the **field-level authority** of its source. Reconciliation then6decides which claim (if any) populates a column. Nothing is deleted when sources disagree.78## Claim910| column | meaning |11|---|---|12| `subject_type`, `subject_id` | facility / campus / project / operator / market / country |13| `predicate` | what is asserted — `it_capacity_mw`, `critical_power_mw`, `utility_capacity_mw`, `grid_connection_mw`, `current_power_mw`, `planned_power_mw`, `ultimate_campus_mw`, `phase_mw`, `project_investment_usd`, `campus_investment_usd`, `company_investment_usd`, `country_program_usd`, `multi_year_capex_usd`, `deal_value_usd`, `status`, … |14| `value`, `value_text`, `unit` | the figure (MW, USD) or text |15| `scope`, `scope_reason` | `building` · `facility` · `campus` · `metro` · `country` · `portfolio` · `company` · `unknown`, and the rule that decided |16| `evidence_text`, `evidence_start`, `evidence_end` | the supporting sentence and its offsets in the parsed text — **no sentence, no numerical claim** (structured spec tables record the extraction method instead) |17| `source_id`, `url`, `document_id`, `published_at`, `retrieved_at` | where and when |18| `authority_tier` | A–E for THIS field (see below) |19| `parser_name`, `parser_version`, `run_id` | who extracted it (reprocessing + rollback) |20| `status` | `current` · `superseded` (same source, same page, new value) · `rejected` (review / rollback) · `review` (critical sanity flag) · `unscoped` (figure describes a portfolio, a company, a country, a metro, or its scope is unknown) |2122## Scope rules2324- Only **site scopes** (`building`, `facility`, `campus`) may populate a facility's or project's capacity / investment25  columns. Portfolio, company, country, metro and unknown scopes are stored and shown in the evidence drawer, never26  summed.27- Scope is classified deterministically from the sentence (`packages/core/src/claims.ts` `classifyScope`): company →28  portfolio → country → metro → campus → building → facility → the record's own scope when the sentence has no signal29  (structured parsers) → unknown.30- A campus figure is never written to a building record; a building's figure never stands for its campus.3132## Capacity semantics3334IT load, critical power, utility capacity, grid connection, current power, planned capacity, ultimate build-out and35phase capacity are different predicates and different columns (`it_capacity_mw`, `total_power_mw`, `planned_power_mw`,36`utility_capacity_mw`, `grid_connection_mw`, `ultimate_campus_mw`). The displayed figure carries `capacity_scope` and37`capacity_semantics` so the UI can say what "300 MW" means. When a sentence gives no cue the predicate defaults from the38field/record status and the claim is flagged `mw_semantics_default` (info).3940## Field-level authority4142One universal source rank is wrong. `FIELD_AUTHORITY` in `packages/core/src/claims.ts` gives a tier per field:4344| field | A | B | C | D | E |45|---|---|---|---|---|---|46| capacity | utility, government, filing | operator, cloud provider | registry, secondary | news | dataset, community |47| geometry | government, community (OSM) | registry, operator, dataset | cloud provider, utility, filing | secondary | news |48| interconnection | registry (PeeringDB) | operator | community, dataset | secondary, news, government, filing | utility, cloud |49| status | operator, government, filing, cloud provider | utility, registry | secondary, news | dataset, community | — |50| investment | filing, government | operator, cloud provider | utility, secondary | news | registry, dataset, community |5152Estimates and LLM-only extractions drop one tier; a human review raises one.5354## Sanity engines5556`capacitySanity` / `investmentSanity` run on every claim. Blocking flags (`blocks: true`) keep the figure out of the57columns: non-site scope, campus figure on a building, ≥ 20 GW (market statistic), invalid value, deal value / capex /58country programme as a project investment, > $500 B. Non-blocking critical flags (single site > 1 000 MW without a campus59designation, > $50 B on one site, money and MW sharing a number in one sentence, 5× change) assign the figure but open a60`quality_flags` row prioritised by impact (`reviewPriority`).6162## Announcement classification (projects)6364Before a news article can create a project, `classifyProjectEvent` labels it: `NEW_BUILD`, `EXPANSION`,65`CONSTRUCTION_START`, `PERMIT`, `LAND_ACQUISITION`, `GRID_CONNECTION` (physical — may create / modify a project);66`POWER_AGREEMENT`, `FINANCING`, `ACQUISITION`, `PARTNERSHIP`, `CUSTOMER_AGREEMENT` (associated — attach an event to an67identifiable project, never create one); `EXECUTIVE_APPOINTMENT`, `SUSTAINABILITY`, `PRODUCT_NEWS`,68`GENERAL_COMPANY_NEWS`, `UNKNOWN` (never a project). Creation additionally needs the evidence threshold: (explicit69name or operator) + location + a development verb. Company domiciles ("Denver-based", "headquartered in") are stripped70before locating the project.7172## Lifecycle state machine7374`projectTransition(from, to)`: forward moves along rumored → proposed → announced → permitting → approved →75under_construction → partially_operational → operational are accepted; `delayed` / `cancelled` are side branches;76leaving `delayed` resumes; a backward move needs a source that outranks the stored one, otherwise it is flagged77(`status_backward`). Stage dates (`permit_filed_on`, `approved_on`, `construction_started_on`, `opened_on`) are set from78the announcement that moved the stage.7980## Winners, history, rollback8182`provenance.is_winner` marks the observation backing each displayed value; `provenance.run_id`, `claims.run_id`,83`events.run_id` and `document_versions.run_id` tie every change to a connector run (`POST /api/admin/runs/:id/rollback`).84Capacity history = winner changes + claims + `capacity_changed` events. Daily `entity_snapshots` keep global totals,85per-operator and per-country totals, rankings and stage counts for "as of" views and the automated regression checks86(facilities −5 %, known MW +20 %, one operator +10 GW/day, one connector > 500 records/day, > 200 location changes/day).8788## Containment (campus vs building)8990`facilities.parent_facility_id` + `record_scope` (building / facility / campus). The matcher's `rule:campus-vs-building`91now links instead of flagging a duplicate. Every aggregate uses `FACILITY_VIEW` (apps/worker/src/rankings.ts): a campus92whose buildings publish figures contributes no MW itself; a building without a figure under a campus with one is93"covered"; a campus with buildings is not counted as an extra facility.9495## Debugging9697`pnpm dci trace <doc-id|url>` (or `GET /api/admin/documents/:id/trace`) shows every stage for one document: raw fetch →98parsed text → structured records → normalized entities → claims with scope/semantics/evidence → match candidates →99dry-run reconciliation → resulting changes. `pnpm dci quality` runs the sweep, `pnpm dci snapshot` the snapshots +100regression checks, `pnpm dci gaps` the data-gap counts, `pnpm dci quarantine <id> on` puts a connector in preview mode.101