# Announcement → project extraction (`news_article_v1`, `news_planning_pdf_v1`, `news_edgar_fts_v1`)

> **2026-09-12 — announcement classifier first.** Before any project is created, `classifyProjectEvent` (packages/core/src/claims.ts) labels the announcement (NEW_BUILD, EXPANSION, CONSTRUCTION_START, PERMIT, LAND_ACQUISITION, GRID_CONNECTION = physical; POWER_AGREEMENT, FINANCING, ACQUISITION, PARTNERSHIP, CUSTOMER_AGREEMENT = associated events only; EXECUTIVE_APPOINTMENT, SUSTAINABILITY, PRODUCT_NEWS, GENERAL_COMPANY_NEWS, UNKNOWN = never a project). Creation needs (explicit name or operator) + location + development verb; company domiciles ("Denver-based", "headquartered in") are stripped before locating; the headline MW/$ figures are stored as claims with scope and the supporting sentence — portfolio / company / country figures never populate `planned_mw` / `investment_usd`. Lifecycle changes follow `projectTransition`. Details in [CLAIMS.md](CLAIMS.md); debug any article with `pnpm dci trace <url>`.

News, government, utility and filing sources feed the **live change feed** (`news_event`) and the **project pipeline**
(`project`). This document describes how a press release or a planning report becomes structured records, what is
extracted, with which method (provenance), and — above all — what is *not* inferred.

Code: `apps/worker/src/connectors/news/`
(`extract-project.ts` = pure extraction, `article-parser.ts`, `planning-pdf-parser.ts`, `edgar.ts`,
`operators-lexicon.ts`, `locations-lexicon.ts`; tests in `extract-project.test.ts`).
Connector definitions: `config/connectors/*.yaml` with `parser: news_article_v1` (see `docs/connectors/`).

## 1. Principles

* **Nothing is invented.** Every field comes from a regex / lexicon hit on the page text, and each carries a `methods`
  entry (`regex:mw_v1:title`, `lexicon:city`, `generated:operator_city`, …) that lands in `provenance.method`.
  A figure that is not in the text is `null`. Currencies are never converted into a stored value.
* **Copyright-aware.** For third-party publishers (`kind: news`) the `news_event` keeps only title, date, a ≤ 400-char
  summary, the link and the extracted facts — `data.text` is `null` (`params.keepText: false`, the default).
  Public-sector and utility sources (`kind: government | utility`) keep the text (`keepText: true`).
* **The URL says nothing about the event.** `/news/…` is classified as `press_release` by the URL rules; the parser
  re-classifies from the *text* (title first) to obtain the semantic page type (`project_announcement`, `expansion`,
  `planning_document`, `construction_update`, `power_infrastructure`, `acquisition`, `closure`, `cloud_region`) and
  the `eventType`. When the classifier only sees a generic release, the inferred status decides
  (approved/permitting → `planning_document`, groundbreaking → `construction_update`, plan with a size →
  `project_announcement`).
* **One event per page, at most one project per page.** Keys are connector-scoped and stable:
  `<connector>:news:<urlFingerprint>` and `<connector>:project:<urlFingerprint>` — re-ingestion is idempotent.

## 2. Relevance filter

`relevanceOf(title, text)`:

| level | rule | effect |
|---|---|---|
| `strong` | data cent(er/re), colocation, hyperscale, cloud region, availability zone, AI factory/infrastructure/campus, GPU cluster, data hall, IBX, Rechenzentrum, centre de données… **or** a MW/GW figure attached to campus/site/facility **or** a known operator + a MW figure in the lead | event (certainty 0.7) + project when qualified |
| `weak` | only the SDK's broad `isDataCenterRelevant` (MW, campus, interconnection…) | event only, certainty 0.35, no project |
| `none` | — | nothing emitted |

`weak` events are still emitted so that the GenericConnector fallback (which would store the full text) never fires.

## 3. Fields and methods

| field | how | method tag |
|---|---|---|
| `title` | `<h1>` (10–220 chars) → `og:title` → `<title>` → RSS title; publisher suffix ("… \| DCD") stripped | `html:h1`, `meta:og:title`, `rss:title`, `pdf:meta-title` |
| `publishedAt` | `publishedDate()` (article:published_time, JSON-LD, `<time>`) → RSS `pubDate` → dateline in the first 1 500 chars (never a future date) | `meta:published`, `rss:published`, `text:dateline` |
| `summary` | `meta description` → RSS description → lead text; cut to 400 chars on a word boundary | `meta:description`, `rss:description`, `text:lead` |
| `plannedMw` | `parseAllMw` on title → title + first 600 chars → first 6 000 chars; **largest** figure of the first scope that has one; figures > 20 GW are market statistics and ignored; "192-MW" is normalised to "192 MW" | `regex:mw_v1:title\|lead\|body` |
| `investmentUsd` | `parseAllMoney` (same scope order, largest figure; per-MWh / per-kWh prices and amounts < $1M ignored). **USD only.** Other currencies stay in `news_event.data.money` and are appended to the project `description` ("Investment: 1.2 billion GBP"). An *approximate* USD equivalent is used **only** for the ≥ $50M gate | `regex:money_v1:*` |
| `acreage` | `parseAreaHa` (acres or hectares) → acres (`ha / 0.40468564`, 1 decimal); > 50 000 acres ignored | `regex:area_v1:*` |
| `phaseCount` | "in three phases", "phase 1 of 3", "two-phase development"; "first phase" alone gives nothing | `regex:phase_v1` |
| `expectedOpening` | sentence-level: (1) sentences about the facility being ready (RFS, come online, open in, operational by, completion, delivery, first phase) → first date token after the keyword; (2) generic "expected / scheduled / slated / targeted" sentences; sentences about construction start or land purchase are skipped. Partial dates (`2027`, `2027-Q2`, `2027-H2`, `2027-06`), year bounded to [published year, now + 15] | `regex:expected_date_v1:opening\|expected` |
| `status` | `STATUS_RULES` with precedence **cancelled > closed > delayed > approved > permitting > under_construction > partially_operational > announced > proposed > rumored > expansion > operational**, evaluated on the **title first**, then the lead (1 500 chars), then the body. "will come online in 2027" is a plan, not `operational` | `regex:status_v1:title\|lead\|body` |
| `operatorName` | curated lexicon (`operators-lexicon.ts`, 150+ operators, hyperscalers, colocation, developers, investors). Word-boundary, case-sensitive; ambiguous names (Meta, Apple, Switch, Aligned, Vantage, STACK, Tract, Lambda, SAP…) need a following verb ("Meta scraps…") so ordinary words never match. Primary = first operator in the title, else first in the body | `lexicon:operator` |
| `city / regionName / countryIso2` | (1) explicit "City, State" (US states, Canadian provinces, abbreviations); (2) ~300 curated data-center cities with region + ISO-2; a proper city beats a county named earlier; (3) standalone state / province; (4) `countryFromText` with traps removed (Latin/North America ≠ US, Georgia = US state unless Tbilisi/Caucasus, "… Jordan" = a person). Title first; a title hit without a city is completed from the lead when consistent | `regex:city_state`, `lexicon:city`, `lexicon:state`, `lexicon:country` |
| `name` | explicit proper-noun project name ending in Campus / Data Center / Digital Gateway / Technology Park / Hub / Cluster… ("Prince William Digital Gateway"), rejecting operator names and generic phrases; else generated `"<Operator> <City> campus"`, `"<Operator> <Region> data center project"`, `"<City> data center project"`; else the cleaned title | `regex:project_name_v1`, `generated:*`, `title` |
| `announcedOn` | = `publishedAt` | as above |
| `description` | the summary (+ non-USD investment sentence) | — |

## 4. When is a project emitted?

`qualifiesAsProject()` — all of:

1. relevance `strong`;
2. status in the pipeline set (`rumored, proposed, announced, permitting, approved, under_construction,
   partially_operational, delayed`) or `expansion` — never `operational`, `closed`, `cancelled`;
3. semantic page type in `project_announcement, expansion, construction_update, planning_document,
   power_infrastructure, press_release, utility_announcement` (not acquisitions, closures, cloud regions, opinion);
4. size: `plannedMw ≥ 5` **or** investment ≥ $50M (approximate conversion) **or** site ≥ 100 acres.

Gates are parameters (`minProjectMw`, `minInvestmentUsd`, `minAcres`). Certainty 0.5 + 0.1 (operator) + 0.1
(city or region) + 0.1 (MW or money in the title), capped at 0.8.

Duplicate avoidance: the GenericConnector normalizer creates its own weak project candidate from
`news_event.data.status + mw`; the parser therefore exposes `status` on the event only when no project could be
derived from it (the inferred status is always available as `data.statusInferred`).

## 5. Planning documents (`news_planning_pdf_v1`)

PDF (via `pdfText`) or HTML staff reports / decision notices. In addition to the fields above:

* **applicant** — "Applicant: …", "on behalf of …", "submitted by …" (canonical lexicon name when it is a known
  operator), else the first operator in the text;
* **decision** — explicit "Decision/Recommendation: Approved|Refused|Pending" line first, then approval / refusal
  keyword counts; → `approved` (eventType `planning_approved`), `pending` → `permitting` (`planning_filed`),
  `refused` → **event only** (a refusal is not a developer cancellation, so no project status is asserted);
* **address** — first street address ("41500 Lovettsville Road"), **reference** ("REZ 2026-0017"), document date
  ("Date of meeting: 8 September 2026").
* A project is emitted as soon as the decision is known and the document gives a MW figure, an acreage or an
  applicant (planning documents describe exactly one application).

## 6. SEC EDGAR (`news_edgar_fts_v1`)

`implementation: news_edgar_fts` computes `dateRange=custom&startdt=…&enddt=…` (the API ignores `30d`) for each
`params.queries` phrase; the parser turns the JSON hits into one `news_event` per filing (press-release exhibit
preferred), linking to `https://www.sec.gov/Archives/edgar/data/<cik>/<adsh>/<file>`. Filings are not fetched;
`operators` come from the lexicon match on the registrant name, `countries`/`cities` from `biz_locations`.

## 7. Known limitations

* English-first status vocabulary (German / French pages still yield MW, money, operator, location).
* The largest-figure heuristic can pick a market statistic quoted early in an article (mitigated by the 20 GW cap and
  the title-first scope); certainty stays ≤ 0.8 so downstream reconciliation treats these as candidates.
* City lexicon covers ~300 metros; unknown towns are only found through "City, State" (US/CA) patterns.
* `parsePartialDate` in `@dci/core` only knows 3-letter month abbreviations; the parsers pre-normalise full month
  names (`parseDateLoose`).