# Entity reconciliation

2026-09-12 — claim-first layer. Numeric fields (capacity, investment) now flow through the claim store described in CLAIMS.md: scope + semantics + evidence sentence + field-level authority, sanity engine, then the merge policy below. External-id folding is allowlist-only (IDENTIFYING_EXTERNAL_ID_KEYS), candidates are relevance-ordered (same operator → same name → nearest), and rule:campus-vs-building now links the building to its campus (parent_facility_id, record_scope) instead of opening a duplicate review. provenance.is_winner marks the observation behind each displayed value; run_id ties every write to a connector run (rollback).

How a normalized record coming out of a connector becomes (or updates) a canonical facility, operator, project, cloud region or IXP — and how conflicting values are merged. Implementation: apps/worker/src/ingest/ (match.ts holds the pure scoring/merge logic, facilities.ts wires it to Postgres).

# Guarantees

• Nothing is invented and nothing is lost. Every incoming value is written to provenance even when it does not win the merge; ambiguous matches are kept as new records and queued for review, never discarded.
• Idempotent. Re-ingesting the same document yields 0 created rows, 0 events and the same provenance rows (refreshed last_observed). Connector keys (peeringdb:fac:12) are mapped in entity_keys.
• Atomic per batch, tolerant per entity. A batch runs in one transaction; each entity runs in a savepoint, so a bad record is rolled back and counted as rejected without failing the batch. Dry runs roll back everything.

# Concurrency

Connectors run in parallel, so two batches may describe the same operator at the same moment. Operator resolution takes a transaction-scoped advisory lock on the normalized (canonical) name before looking up or inserting, which serializes creation across batches; mergeOperators() folds duplicates that slipped through before the lock existed. Facilities are not locked (a dataset batch can carry thousands of them and Postgres' lock table is finite): two connectors creating the same unseen facility within the same seconds can still produce a pair of records that only a later near-duplicate sweep or an admin merge (mergeFacilities) will fold.

# Facilities

# Resolution order

1. Connector key — entity_keys.key = record.key → same facility (following merged_into). Always wins.
2. Shared external ids — any externalIds entry (peeringdb_fac, osm, wikidata, …) already stored on a live facility → same facility.
3. Candidate scoring — candidates are live facilities that (a) lie in a 40 km bounding box around the incoming coordinates, or (b) share the normalized name (or an alias) in the same country, or (c) belong to the same operator in the same country (so facility codes like DC12 can be compared). Up to 400 candidates are scored.

# Score

Weighted sum, weights redistributed over the signals available on both sides:

Rules applied on top of the weighted sum:

• Operator + code rule: same operator, matching facility code and geographically compatible (inside the radius or no coordinates) → the name signal is raised to 0.95. This is what merges DC12 into Equinix DC12.
• Code conflict: same operator but conflicting codes (DC12 vs DC13) caps the name signal at 0.45.
• Different countries cap the score at 0.30.
• Too far: precise coordinates more than 4× the match radius apart cap the score at 0.50.
• Different known operators never merge — the score is capped below the pending threshold — unless the normalized name and the normalized address are both exact (an operator change on the same building).

# Decision thresholds

Why "create + pending" rather than "hold": the record is visible immediately (flagged unverified), its provenance is preserved, and approving the match later is a pure fold (mergeFacilities). Holding would hide real facilities for days when the review queue is slow.

# Admin review flow

GET /api/admin/matches?status=pending lists the queue. For each row:

• Approve → mergeFacilities(createdFacilityId, matchedFacilityId): keys, aliases, provenance (deduplicated on entity/field/source/url), tenants, IXPs, projects and events are moved to the surviving facility; the created one gets merged_into; its external ids are folded; derived fields are recomputed; the match row becomes approved.
• Reject → the match row becomes rejected; the created facility simply stays a separate record and its confidence is recomputed on the next observation (it is no longer forced to unverified).

A facility keeps confidence = unverified for as long as a pending match references it as createdFacilityId.

# Field merge policy

Each incoming field is compared with the observation that currently backs the stored value (from provenance, is_current = true, joined with sources.kind):

1. Empty stored value → take the incoming one.
2. Same source re-observing → take it (a source may correct itself).
3. Measured beats estimate (isEstimate), whatever the source.
4. Otherwise higher authority wins: operator / government / filing 4 · utility / cloud provider 3.5 · registry 3 · dataset 2.5 · community 2 · secondary 1.5 · news 1 (estimates −1.5). Ties → most recent observation.

Special cases:

• MW (itCapacityMw, totalPowerMw, plannedPowerMw): a non-primary source (dataset, community, secondary, news) never overwrites a figure from an operator / government / filing / utility / cloud-provider / registry source. mw_is_estimate is true only when every current MW observation is an estimate.
• Coordinates: never replaced by a less precise point (PRECISION_RANK: exact > parcel > street > approximate > city > metro > unknown). Same precision → newest wins (same source always refreshes its own point).
• Flags is_ai, is_hyperscale: true sticks. A facility operated by a hyperscaler is hyperscale.
• Certifications, aliases, external ids: accumulated (set union).
• Name: same policy as other fields; the previous name becomes an alias.
• Status / type: unknown never overwrites a known value.

# Derived fields (recomputed after every write)

• completeness 0–100: geo 15 (exact/parcel/street; 8 for city/metro/approximate), operator 10, address 10, status 10, MW 20 (14 when estimate; planned-only 12/8), type 5, opened 10, website 5, description 5, tenants/IXPs 10.
• confidence = computeConfidence() with the best source kind as base and the number of distinct primary source kinds as corroborations; verified needs ≥ 2 primary kinds, estimated when every observation is an estimate.
• source_count = distinct sources with current provenance; last_verified = latest observation by a primary source.
• metro_id = nearest seeded metro whose radius covers the point (same country), else the metro whose alias matches the city; country_iso2 falls back to the metro's country; geohash (precision 7).

# Operators

Order: connector key → external ids → curated canonical table (canonical-operators.ts: ~175 brands with aliases, e.g. Interxion/Telx/DuPont Fabros → Digital Realty, RagingWire/e-shelter/Gyron/NetMagic → NTT Global Data Centers) → exact normalized name → alias match (case-insensitive) → trigram similarity ≥ 0.92 and the same website domain → create. Kind inference: curated kind, else caller hint (carrier / cloud), else name heuristics. Hyperscalers (AWS, Microsoft, Google, Meta, Oracle, Alibaba Cloud, Tencent Cloud, Apple, IBM, Huawei Cloud) get kind = hyperscaler; is_cloud_provider / is_carrier are set from the table or the role hint and only ever turn on. Incoming names that differ from the canonical name are appended to aliases. News text can only link to curated or already-known operators; it never creates one.

# Projects

Key → external ids → same sourceUrl → (same country, same or unknown operator, normalized-name trigram similarity ≥ 0.85). Timeline rows are deduplicated on (project, date, type, sha256(description)). Status changes emit project_status_changed; planned MW, expected opening, investment and operator changes follow the same event rules as facilities. last_update moves whenever a column or a timeline row changes.

# Cloud regions, IXPs, campuses, tenants

• Cloud regions are unique on (provider, code); the provider is resolved as an operator with the cloud hint.
• IXPs: key → external ids → normalized name (or long name) in the same country. facility_ixps links come from the IXP's facilityKeys (resolved through entity_keys) or from a facility page's ixps list.
• Campuses: key → external ids → normalized name with compatible operator/country.
• Tenants: carriers / cloudProviders on a facility resolve to operators (carrier / cloud hint, AS12345 parsed into asn) and fill facility_tenants; carriers_count / networks_count are recomputed.

# Events

Tracked facility fields: status, IT/total/planned MW, opened/announced/construction-start dates, operator, owner, facility type, name. Significance: status 90 · operator 85 · owner 70 · MW change ≥ 20 % 80 else 50 · dates 60 · others 20 (first-time values are capped at 50; enrichments of name/type/operator are not events). Fingerprint = sha256(entityType|entityId|eventType|JSON(newValue)|day) — the same change observed by several pages on the same day is one event. facility_discovered (65 when MW ≥ 50 or in pipeline, else 40) and project_announced (75 when ≥ 100 MW planned) are emitted for new records. News items keyed by URL emit one event per article (fingerprint on the URL) when they carry an event type with significance ≥ 40.