# Connector ncit-evs — NCI Thesaurus (NCIt)

Tier 0 · category terminology · anchor of the CancerIndex cancer ontology (CLAUDE.md §4-§5, §209, §217-§221). Every canonical cancer entity (CI-CAN-…) is minted from an NCIt concept under Neoplasm (C3262); OncoTree and every downstream source reconcile onto these entities.

Code: packages/connectors/src/connectors/ncit-evs/ — manifest.ts, parse.ts (FLAT file), classify.ts (pure ontology rules, unit-tested), index.ts (DB writes), fixtures/, ncit-evs.test.ts.

# Source and access (documentation verified 2026-09-08)

Both evs.nci.nih.gov and api-evsrest.nci.nih.gov accepted the SDK user agent CancerIndex/0.1 and the URL-bearing variant; the connector passes the plain product token anyway (some WAFs reject UAs containing URLs — see sdk/http.ts).

# License evidence

The EVS web site is a JavaScript application (WebFetch/Tavily only see the page title), but the FTP tree carries the authoritative terms page, fetched 2026-09-08:

"The NCI Thesaurus™ is released under the Creative Commons Attribution 4.0 International license (CC BY 4.0). The NCI Thesaurus™ is produced by the Enterprise Vocabulary Services group of the Center for Biomedical Informatics and Information Technology, National Cancer Institute, Maryland, USA. The name "NCI Thesaurus" is trademarked. Only the NCI Thesaurus™ published by the NCI can be released under this name." — https://evs.nci.nih.gov/ftp1/NCI_Thesaurus/ThesaurusTermsofUse.htm

Corroborated by OBO Foundry (obofoundry.org/ontology/ncit.html, "License: CC BY 4.0") and NCBO BioPortal. The wider NCI reuse policy (https://www.cancer.gov/policies/copyright-reuse: "Unless otherwise indicated, all text within National Cancer Institute (NCI) products is free of copyright and may be reused without our permission … Credit the National Cancer Institute as the source") is compatible; the manifest records the stricter, explicit CC BY 4.0 statement, so licenseStatus: approved, redistribution: attribution, commercialUse: allowed. Attribution string and the trademark caveat are in the manifest (attribution, termsNotes). It is not recorded as "public domain": the terms page says CC BY 4.0.

# Fields used

code → cancers.primary_ncit_code, cancer_codes(ncit); parents → subtree closure, depth, cancer_hierarchy(ncit); synonyms[0] → canonical_name (never altered, the top-level display name goes to an alias); synonyms[1..] → cancer_aliases; definition → description + provenance row; display name → alias type display; concept status → retired/obsolete/header handling; semantic type → cancers.semantic_types + disease filter. concept IRI and concept in subset are stored in the raw payload only. UMLS CUIs / ICD-O codes are not derived (the FLAT file does not carry them).

# Construction rules (classify.ts)

1. The whole file (213 083 concepts) is loaded; child lists are built for all concepts so subtree closures are exact. Descendant sets: Neoplasm C3262 (15 682), Malignant Neoplasm C9305 (11 123 → malignant), Hematopoietic and Lymphoid Cell Neoplasm C27134 (2 264 → hematologic, solid_tumor = !hematologic), Childhood Malignant Neoplasm C4005 (638) or preferred name starting with Childhood/Pediatric → pediatric_relevant, Neoplasm by Site C3263, Neoplasm by Morphology C4741. depth = shortest path from C3262 (BFS).
2. Exclusions (in order): outside the Neoplasm subtree; Retired_Concept; Obsolete_Concept; animal/model concept (name matches mouse/murine/rat/canine/feline/zebrafish/drosophila/bovine/ equine/porcine/avian/rodent/hamster/non-human/primate/"animal model"/veterinary, or semantic type Experimental Model of Disease/Mammal/Animal/Vertebrate/Organism — 0 hits in 26.08e, rule kept as a guard); semantic type not disease-like (kept set: Neoplastic Process, Disease or Syndrome, Finding, Congenital Abnormality, Pathologic Function, Cell or Molecular Dysfunction, Anatomical Abnormality, Acquired Abnormality, Sign or Symptom — observed under Neoplasm: 15 656 Neoplastic Process, 24 Disease or Syndrome, 3 Congenital Abnormality, 1 Pathologic Function, 1 Finding, 1 Cell or Molecular Dysfunction; 0 excluded); qualified disease state via isQualifiedState() from @cancerindex/ontology plus a connector-local EXTRA_QUALIFIED_RE for stage patterns the shared rules miss ("Stage 1/2A/4S Neuroblastoma", "Stage A/B/C/D Prostate Cancer", "Stage M0–M4 …", "Stage L1/L2/MS …", "Stage y-I …", "… FIGO 2023", "… by Toronto Guidelines v2"). Qualified states are stored as source_records (entity_kind = qualified_state) with baseLabel = baseDiseaseLabel(name) and, when the base label equals a kept preferred name, baseCode/canonical_id; the same mapping is written to the raw lake (qualified_state_map). They never become cancers or aliases.
3. Header concepts (Header_Concept, e.g. "Neoplasm by Site", "Malignant Neoplasm by Grade") are kept for connectivity with entity_type = other (17 rows) — they are structural, never listed as diseases.
4. Retired concepts: not in the subtree (they hang under "Retired Concept 2025", C223967). The 565 retired concepts with semantic type Neoplastic Process (664 retired rows have an empty semantic type despite the ReadMe's guarantee — accepted by the parser) and the 256 obsolete subtree concepts are recorded as source_records with status = deprecated. When a retired concept's preferred name equals an active kept concept's name, the retired code is mapped to it (cancer_codes system ncit, match_type ONTOLOGY_EXACT, valid_to = 26.08e; 48 such redirects) — e.g. C3556 → C61574 "Malignant Uterine Corpus Neoplasm".
5. Entity type (first match): header → other; depth ≤ 3 and ≥ 15 neoplasm descendants → cancer_family; name matches MOLECULAR_RE (EGFR-Mutant, HER2-Positive, … with NPM1 Mutation, t(8;21), Triple-Negative, Mismatch Repair-Deficient…) → molecular_subtype; not malignant and /in situ|intraepithelial neoplasia|dysplasia|precursor|monoclonal gammopathy|myelodysplastic|premalignant/ → precursor_condition; under Morphology and not under Site → histology; hematologic → hematologic_malignancy; else depth ≥ 4 → subtype, else cancer. Deviation from the brief's order: molecular and precursor are tested before histology so that "Carcinoma In Situ" (a morphology node) is a precursor and molecular subtypes under morphology are not typed as histology. Curated top-level entries typed subtype are promoted to cancer (they are ranking scopes).
6. Aliases (cancer_aliases, source_terminology = NCIt): preferred name → preferred; other synonyms → synonym, or abbreviation when 2–6 upper-case alphanumerics with ≥ 2 letters (GBM, NSCLC, PDAC…); display name → display. Deduplicated per concept on normalizeLabel() (first wins, e.g. "Neoplasms, NOS" collapses into the preferred "Neoplasm"); normalized forms shorter than 2 chars skipped. Batched 500/insert with ON CONFLICT DO NOTHING.
7. Codes: ncit EXACT_IDENTIFIER for every kept concept; for the 36 TOP_LEVEL_CANCERS also icd10 ranges as CURATED_EXACT, top_level = true, and the registry display name (e.g. "Pancreatic Cancer") as a display alias — canonical_name stays the NCIt preferred name.
8. Hierarchy (cancer_hierarchy, hierarchy_type = ncit): every parent inside the kept set; when all parents were excluded, the nearest kept ancestor(s) found by walking up ≤ 10 levels (146 concepts re-attached).
9. Anatomy: the 15 direct children of "Neoplasm by Site" become anatomical_sites (name = preferred name minus trailing "Neoplasm", reused by slug when OncoTree already created one) and every kept descendant is linked via cancer_anatomy(relation = primary).
10. Provenance: one run-level row (dataset "NCI Thesaurus", version 26.08e, expert_curation, open) plus one row per definition (source URL https://ncit.nci.nih.gov/ncitbrowser/ConceptReport.jsp?dictionary=NCI_Thesaurus&code=<code>) referenced by cancers.description_provenance_id; rows are only added when the definition is new or changed.
11. Idempotency / guard: entities are matched on primary_ncit_code; slugs never change; unchanged rows are not rewritten. Anomaly guard: the run refuses to persist when the FLAT zip is < 5 MB, < 100 000 concepts parse, or the kept count drops below 90 % of the previous run (anomalyGuard.minRatioOfPrevious = 0.9, stored in connector_cursors.cursor.keptCount). Only after the guard passes are NCIt-anchored cancers that vanished / were retired / were reclassified as qualified states set to status = deprecated with an explicit deprecated_reason (merged_into set when a same-name replacement exists).

# Counts observed (NCIt 26.08e, 2026-09-08, database cancerindex_a)

# Known limitations

• entity_type is heuristic (depth-based); most organ-level malignancies sit at depth ≥ 4 in NCIt and are typed subtype (e.g. "Breast Carcinoma", "Glioblastoma"). Curated top-level entries are promoted to cancer. A curated layer or descendant-count thresholds per depth would refine this.
• cancer_family also matches large non-malignant groupers ("Lung Neoplasm", 178 of 229).
• Hierarchy edges are never mass-deleted; edges to nodes deprecated by a later run remain (247 on cancerindex_a after the reclassification rerun; a fresh pipeline never creates them). Consumers must filter on cancers.status = 'active'.
• OncoTree runs after NCIt and reuses our site groups by slug (e.g. breast), overwriting anatomical_sites.ncit_code with its own anatomy code (C12971 vs our C2910). Harmless but noted.
• No ICD-O, UMLS or MeSH codes come from the FLAT file; OncoTree contributes UMLS CUIs (650).
• Semantic-type filter and animal regex were validated on 26.08e only; new semantic types under Neoplasm are surfaced in the run log (non-disease concepts under Neoplasm: …).
• unresolved_labels is untouched by this connector (it defines the vocabulary; nothing to resolve).

# Recommendation for downstream reconciliation (trials, literature, registries)

Trial conditions and registry labels are frequently qualified ("Stage IV Pancreatic Cancer", "Recurrent Glioblastoma", "Metastatic HER2-Positive Breast Carcinoma"). Those NCIt concepts are deliberately not canonical entities, so CancerResolver.byLabel() will miss them. The trials and literature connectors should implement this fallback chain (it belongs in @cancerindex/ontology, not in this connector):

1. resolve({ codes }) on any NCIt/UMLS/MeSH identifier the source carries (EXACT_IDENTIFIER).
2. byLabel(label) (ONTOLOGY_EXACT / ALIAS).
3. If it misses and isQualifiedState(label) || EXTRA_QUALIFIED_RE.test(label): strip qualifiers with baseDiseaseLabel(label) (iteratively, also removing "Stage " prefixes and staging suffixes) and retry byLabel(base); record the mapping with match_type = CURATED_BROADER (the trial is about a narrower state of the base disease) and keep the original label + stage in cohort_definitions.stage / attributes.
4. Otherwise ctx.recordUnresolved('cancer', label, normalizeLabel(label), ctx).

The raw-lake file qualified_state_map (code → baseLabel → baseCode, 3 844 resolved) can be loaded as a curated seed for that step. Upstreaming EXTRA_QUALIFIED_RE into packages/ontology/src/qualifiers.ts (with tests) is recommended so both sides share one rule set.