spb/datacenterindex

1# Announcement → project extraction (`news_article_v1`, `news_planning_pdf_v1`, `news_edgar_fts_v1`)23> **2026-09-12 — announcement classifier first.** Before any project is created, `classifyProjectEvent` (packages/core/src/claims.ts) labels the announcement (NEW_BUILD, EXPANSION, CONSTRUCTION_START, PERMIT, LAND_ACQUISITION, GRID_CONNECTION = physical; POWER_AGREEMENT, FINANCING, ACQUISITION, PARTNERSHIP, CUSTOMER_AGREEMENT = associated events only; EXECUTIVE_APPOINTMENT, SUSTAINABILITY, PRODUCT_NEWS, GENERAL_COMPANY_NEWS, UNKNOWN = never a project). Creation needs (explicit name or operator) + location + development verb; company domiciles ("Denver-based", "headquartered in") are stripped before locating; the headline MW/$ figures are stored as claims with scope and the supporting sentence — portfolio / company / country figures never populate `planned_mw` / `investment_usd`. Lifecycle changes follow `projectTransition`. Details in [CLAIMS.md](CLAIMS.md); debug any article with `pnpm dci trace <url>`.45News, government, utility and filing sources feed the **live change feed** (`news_event`) and the **project pipeline**6(`project`). This document describes how a press release or a planning report becomes structured records, what is7extracted, with which method (provenance), and — above all — what is *not* inferred.89Code: `apps/worker/src/connectors/news/`10(`extract-project.ts` = pure extraction, `article-parser.ts`, `planning-pdf-parser.ts`, `edgar.ts`,11`operators-lexicon.ts`, `locations-lexicon.ts`; tests in `extract-project.test.ts`).12Connector definitions: `config/connectors/*.yaml` with `parser: news_article_v1` (see `docs/connectors/`).1314## 1. Principles1516* **Nothing is invented.** Every field comes from a regex / lexicon hit on the page text, and each carries a `methods`17  entry (`regex:mw_v1:title`, `lexicon:city`, `generated:operator_city`, …) that lands in `provenance.method`.18  A figure that is not in the text is `null`. Currencies are never converted into a stored value.19* **Copyright-aware.** For third-party publishers (`kind: news`) the `news_event` keeps only title, date, a ≤ 400-char20  summary, the link and the extracted facts — `data.text` is `null` (`params.keepText: false`, the default).21  Public-sector and utility sources (`kind: government | utility`) keep the text (`keepText: true`).22* **The URL says nothing about the event.** `/news/…` is classified as `press_release` by the URL rules; the parser23  re-classifies from the *text* (title first) to obtain the semantic page type (`project_announcement`, `expansion`,24  `planning_document`, `construction_update`, `power_infrastructure`, `acquisition`, `closure`, `cloud_region`) and25  the `eventType`. When the classifier only sees a generic release, the inferred status decides26  (approved/permitting → `planning_document`, groundbreaking → `construction_update`, plan with a size →27  `project_announcement`).28* **One event per page, at most one project per page.** Keys are connector-scoped and stable:29  `<connector>:news:<urlFingerprint>` and `<connector>:project:<urlFingerprint>` — re-ingestion is idempotent.3031## 2. Relevance filter3233`relevanceOf(title, text)`:3435| level | rule | effect |36|---|---|---|37| `strong` | data cent(er/re), colocation, hyperscale, cloud region, availability zone, AI factory/infrastructure/campus, GPU cluster, data hall, IBX, Rechenzentrum, centre de données… **or** a MW/GW figure attached to campus/site/facility **or** a known operator + a MW figure in the lead | event (certainty 0.7) + project when qualified |38| `weak` | only the SDK's broad `isDataCenterRelevant` (MW, campus, interconnection…) | event only, certainty 0.35, no project |39| `none` | — | nothing emitted |4041`weak` events are still emitted so that the GenericConnector fallback (which would store the full text) never fires.4243## 3. Fields and methods4445| field | how | method tag |46|---|---|---|47| `title` | `<h1>` (10–220 chars) → `og:title` → `<title>` → RSS title; publisher suffix ("… \| DCD") stripped | `html:h1`, `meta:og:title`, `rss:title`, `pdf:meta-title` |48| `publishedAt` | `publishedDate()` (article:published_time, JSON-LD, `<time>`) → RSS `pubDate` → dateline in the first 1 500 chars (never a future date) | `meta:published`, `rss:published`, `text:dateline` |49| `summary` | `meta description` → RSS description → lead text; cut to 400 chars on a word boundary | `meta:description`, `rss:description`, `text:lead` |50| `plannedMw` | `parseAllMw` on title → title + first 600 chars → first 6 000 chars; **largest** figure of the first scope that has one; figures > 20 GW are market statistics and ignored; "192-MW" is normalised to "192 MW" | `regex:mw_v1:title\|lead\|body` |51| `investmentUsd` | `parseAllMoney` (same scope order, largest figure; per-MWh / per-kWh prices and amounts < $1M ignored). **USD only.** Other currencies stay in `news_event.data.money` and are appended to the project `description` ("Investment: 1.2 billion GBP"). An *approximate* USD equivalent is used **only** for the ≥ $50M gate | `regex:money_v1:*` |52| `acreage` | `parseAreaHa` (acres or hectares) → acres (`ha / 0.40468564`, 1 decimal); > 50 000 acres ignored | `regex:area_v1:*` |53| `phaseCount` | "in three phases", "phase 1 of 3", "two-phase development"; "first phase" alone gives nothing | `regex:phase_v1` |54| `expectedOpening` | sentence-level: (1) sentences about the facility being ready (RFS, come online, open in, operational by, completion, delivery, first phase) → first date token after the keyword; (2) generic "expected / scheduled / slated / targeted" sentences; sentences about construction start or land purchase are skipped. Partial dates (`2027`, `2027-Q2`, `2027-H2`, `2027-06`), year bounded to [published year, now + 15] | `regex:expected_date_v1:opening\|expected` |55| `status` | `STATUS_RULES` with precedence **cancelled > closed > delayed > approved > permitting > under_construction > partially_operational > announced > proposed > rumored > expansion > operational**, evaluated on the **title first**, then the lead (1 500 chars), then the body. "will come online in 2027" is a plan, not `operational` | `regex:status_v1:title\|lead\|body` |56| `operatorName` | curated lexicon (`operators-lexicon.ts`, 150+ operators, hyperscalers, colocation, developers, investors). Word-boundary, case-sensitive; ambiguous names (Meta, Apple, Switch, Aligned, Vantage, STACK, Tract, Lambda, SAP…) need a following verb ("Meta scraps…") so ordinary words never match. Primary = first operator in the title, else first in the body | `lexicon:operator` |57| `city / regionName / countryIso2` | (1) explicit "City, State" (US states, Canadian provinces, abbreviations); (2) ~300 curated data-center cities with region + ISO-2; a proper city beats a county named earlier; (3) standalone state / province; (4) `countryFromText` with traps removed (Latin/North America ≠ US, Georgia = US state unless Tbilisi/Caucasus, "… Jordan" = a person). Title first; a title hit without a city is completed from the lead when consistent | `regex:city_state`, `lexicon:city`, `lexicon:state`, `lexicon:country` |58| `name` | explicit proper-noun project name ending in Campus / Data Center / Digital Gateway / Technology Park / Hub / Cluster… ("Prince William Digital Gateway"), rejecting operator names and generic phrases; else generated `"<Operator> <City> campus"`, `"<Operator> <Region> data center project"`, `"<City> data center project"`; else the cleaned title | `regex:project_name_v1`, `generated:*`, `title` |59| `announcedOn` | = `publishedAt` | as above |60| `description` | the summary (+ non-USD investment sentence) | — |6162## 4. When is a project emitted?6364`qualifiesAsProject()` — all of:65661. relevance `strong`;672. status in the pipeline set (`rumored, proposed, announced, permitting, approved, under_construction,68   partially_operational, delayed`) or `expansion` — never `operational`, `closed`, `cancelled`;693. semantic page type in `project_announcement, expansion, construction_update, planning_document,70   power_infrastructure, press_release, utility_announcement` (not acquisitions, closures, cloud regions, opinion);714. size: `plannedMw ≥ 5` **or** investment ≥ $50M (approximate conversion) **or** site ≥ 100 acres.7273Gates are parameters (`minProjectMw`, `minInvestmentUsd`, `minAcres`). Certainty 0.5 + 0.1 (operator) + 0.174(city or region) + 0.1 (MW or money in the title), capped at 0.8.7576Duplicate avoidance: the GenericConnector normalizer creates its own weak project candidate from77`news_event.data.status + mw`; the parser therefore exposes `status` on the event only when no project could be78derived from it (the inferred status is always available as `data.statusInferred`).7980## 5. Planning documents (`news_planning_pdf_v1`)8182PDF (via `pdfText`) or HTML staff reports / decision notices. In addition to the fields above:8384* **applicant** — "Applicant: …", "on behalf of …", "submitted by …" (canonical lexicon name when it is a known85  operator), else the first operator in the text;86* **decision** — explicit "Decision/Recommendation: Approved|Refused|Pending" line first, then approval / refusal87  keyword counts; → `approved` (eventType `planning_approved`), `pending` → `permitting` (`planning_filed`),88  `refused` → **event only** (a refusal is not a developer cancellation, so no project status is asserted);89* **address** — first street address ("41500 Lovettsville Road"), **reference** ("REZ 2026-0017"), document date90  ("Date of meeting: 8 September 2026").91* A project is emitted as soon as the decision is known and the document gives a MW figure, an acreage or an92  applicant (planning documents describe exactly one application).9394## 6. SEC EDGAR (`news_edgar_fts_v1`)9596`implementation: news_edgar_fts` computes `dateRange=custom&startdt=…&enddt=…` (the API ignores `30d`) for each97`params.queries` phrase; the parser turns the JSON hits into one `news_event` per filing (press-release exhibit98preferred), linking to `https://www.sec.gov/Archives/edgar/data/<cik>/<adsh>/<file>`. Filings are not fetched;99`operators` come from the lexicon match on the registrant name, `countries`/`cities` from `biz_locations`.100101## 7. Known limitations102103* English-first status vocabulary (German / French pages still yield MW, money, operator, location).104* The largest-figure heuristic can pick a market statistic quoted early in an article (mitigated by the 20 GW cap and105  the title-first scope); certainty stays ≤ 0.8 so downstream reconciliation treats these as candidates.106* City lexicon covers ~300 metros; unknown towns are only found through "City, State" (US/CA) patterns.107* `parsePartialDate` in `@dci/core` only knows 3-letter month abbreviations; the parsers pre-normalise full month108  names (`parseDateLoose`).109