# Connector mesh — MeSH (Medical Subject Headings, NLM)

# Terms (quoted)

NLM Terms and Conditions (https://www.nlm.nih.gov/databases/download/terms_and_conditions.html, fetched 2026-09-08):

acknowledge NLM as the source of the data by including the phrase "Courtesy of the U.S. National Library of Medicine"

not indicate or imply that NLM has endorsed its products/services/applications

Redistributors must "maintain the most current version of all distributed data" or make known "in a clear and conspicuous manner" that their product does not reflect NLM's latest data. "No charges, usage fees or royalties are paid to NLM." NLM web policies (https://www.nlm.nih.gov/web_policies.html): "Works produced by the U.S. government are not subject to copyright protection in the United States."

CancerIndex shows "Courtesy of the U.S. National Library of Medicine", the MeSH year (datasetVersion, e.g. MeSH 2026) and the retrieval date on the /source/mesh page and in the API sources envelope.

# What MeSH is for in CancerIndex

MeSH is the indexing vocabulary of PubMed, not a disease ontology: it never creates canonical cancers (CLAUDE.md §340). The connector gives existing canonical cancers their MeSH descriptor UI so that

1. the PubMed connector can build MeSH-anchored literature queries — ("Breast Neoplasms"[mh]) instead of title/abstract phrases — and
2. publication → cancer edges resolve by UI (resolver.byCode('mesh', ui)) or descriptor name (byLabel(name, {allowMeshInversion: true})).

# Verified endpoint behaviour (2026-09-08)

• Descriptor model (probed on mesh:D001943 "Breast Neoplasms"): rdfs:label, meshv:identifier, meshv:treeNumber → resource whose rdfs:label is the tree number (C04.588.180, C17.800.090.500), meshv:preferredConcept / meshv:concept → Concept (meshv:scopeNote, meshv:term / meshv:preferredTerm) → Term (meshv:prefLabel, meshv:altLabel), meshv:dateIntroduced, meshv:broaderDescriptor.
• The default dataset spans every year graph (<http://id.nlm.nih.gov/mesh/2015> … /2026, 30,512 topical descriptors in 2026): a naive C04 count returns 8,989 rows. Queries are restricted with FROM <http://id.nlm.nih.gov/mesh> (current vocabulary) → 703 descriptors under C04.
• The endpoint answers SPARQL-XML as text/plain when the Accept header lists application/json, ignoring format=JSON; with Accept: */* it honours the parameter. The connector overrides the SDK default header per request.
• One page (LIMIT 1000) of the grouped descriptor query returns in ~1.2 s (≈ 0.8 MB JSON).
• Year graph detection: ASK { GRAPH <…/mesh/2027> { … } } then /2026 → datasetVersion = "MeSH 2026".
• XML fallback: desc2026.xml, Last-Modified 2026-08-12, ~313 MB.

# Scope

Tree-number prefixes fetched (TREE_PREFIXES in sparql.ts):

Total 734 distinct descriptors. The two extra subtrees contain non-neoplastic entries (Infectious Mononucleosis, Hypergammaglobulinemia, Anemia, Refractory…): those are mapped when the resolver knows them and otherwise skipped silently — only C04 misses enter the curation queue.

# Sync design

1. detectYear → datasetVersion.
2. fetchViaSparql: paged grouped query (descriptorsQuery(offset), LIMIT 1000): per descriptor the heading, all tree numbers, the entry terms of the preferred concept (?terms), the entry terms of the other concepts (?related), scope note, dateIntroduced. Rows failing the Descriptor schema are counted (validationFailures, rejected) and logged, never dropped silently. If the SPARQL route throws, fetchViaXml streams the descriptor XML, splits it on </DescriptorRecord> boundaries and parses each record with fast-xml-parser (PreferredConceptYN separates synonyms from other concepts; records outside the prefixes are discarded). dry_run stops after fetching.
3. Anomaly guard: fewer than 500 descriptors → the run fails without writing (CLAUDE.md §171).
4. One provenance row per run (dataset "MeSH descriptors — Neoplasms tree …", evidenceType expert_curation, accessLevel open).
5. Per descriptor: ctx.upsertSourceRecord('descriptor', <UI>, {ui, label, treeNumbers, terms, relatedTerms, scopeNote, dateIntroduced, meshYear, inC04}) (raw lake data/raw/mesh/<date>/descriptor/), then reconciliation (below), then cancer_codes / cancer_aliases writes.
6. Cursor {year, route, descriptors, syncedAt}; the run is idempotent (onConflictDoNothing everywhere, source records unchanged on re-runs).

# Reconciliation (CLAUDE.md §69, §221) — mapDescriptor()

Why preferred-concept terms only: MeSH lists the terms of narrower and related concepts as entry terms of a descriptor ("Adenocarcinoma, Basal Cell" and "Carcinoma, Tubular" under Adenocarcinoma, "Glioblastoma, Giant Cell" under Glioblastoma, "Breast Cancer" under Breast Neoplasms). Using them for mapping sent D000230 Adenocarcinoma to Basal Cell Adenocarcinoma and D005909 Glioblastoma to Giant Cell Glioblastoma in a first run. They are now stored in the raw record (relatedTerms) and never used for mapping or aliases.

# Aliases

• The heading is stored as alias_type 'mesh_heading', source_terminology 'MeSH'. A dedicated type guarantees the row exists even when NCIt already has an identical synonym (unique key is cancer + normalized + type): the PubMed connector selects aliases whose source_terminology matches /mesh/i to build "<heading>"[mh] queries, and its [tiab] term ranking ignores unknown alias types, so mesh_heading never leaks into title/abstract queries.
• Entry terms of the preferred concept are stored as alias_type 'synonym', source_terminology 'MeSH', after aliasWorthyTerms(): ≥ 3 characters, not the heading, not already present for that cancer under any type, and not a permutation of a kept term ("Neoplasm, Breast" / "Neoplasms, Breast" / "Breast Neoplasm" all normalize to the same un-inverted string as "Breast Neoplasms").
• CURATED_BROADER mappings get the code only.

# Observed run (cancerindex_c, 2026-09-08, NCIt 26.08e + OncoTree + CIViC/ClinVar/GDC loaded — 10,192 cancers)

Top 20 unresolved descriptors (alphabetical, each seen once): Abdominal Neoplasms, Aberrant Crypt Foci, ACTH Syndrome Ectopic, Adenocarcinoma in Situ, Adenocarcinoma Bronchiolo-Alveolar, Adenofibroma, Adenoma Acidophil, Adenoma Basophil, Adenoma Chromophobe, Adenomatosis Pulmonary, Adenomatous Polyposis Coli, Anal Gland Neoplasms, Anaplasia, Angiokeratoma, Anti-N-Methyl-D-Aspartate Receptor Encephalitis, Arachnoid Cysts, Atypical Squamous Cells of the Cervix, Avian Leukosis, Barrett Esophagus, Basal Cell Nevus Syndrome. Three groups: (a) non-neoplastic or veterinary descriptors filed under C04 (Anaplasia, Arachnoid Cysts, Avian Leukosis, paraneoplastic syndromes) — correctly unmapped; (b) heading strings that are ambiguous in this development database because OncoTree ran before NCIt and minted duplicates ("Adenocarcinoma, NOS" next to NCIt "Adenocarcinoma", "Sarcoma, NOS", "Teratoma", "Paraganglioma"): the resolver refuses ambiguous aliases, so Adenocarcinoma (D000230), Sarcoma (D012509 → resolved via entry term to Soft Tissue Sarcoma) and Teratoma stay open or land on a sibling; with NCIt loaded first the OncoTree nodes map onto NCIt concepts and these headings resolve exactly; (c) genuine vocabulary gaps for the curation queue (Adenoma, Acidophil; Anal Gland Neoplasms; Adenocarcinoma, Bronchiolo-Alveolar — the WHO-retired term).

# Effect on PubMed

Before this connector literature_counts queries were all [tiab]. After it, pubmed loadTargets() sees 424 cancers with MeSH names; see docs/connectors/pubmed.md and the run report in the integration notes for the [mh] share measured on the development database.

# Limitations / notes

• MeSH descriptors cover benign and malignant neoplasms alike; the descriptor is attached to whichever canonical concept the heading names (Breast Neoplasms → Breast Neoplasm, not Malignant Breast Neoplasm). Literature counts anchored on such headings therefore include benign-tumour literature — the query string is stored with every count (CLAUDE.md §251).
• Tree numbers are stored in the source record only; a hierarchy_type 'mesh' dimension is a future addition.
• Supplementary Concept Records (C-numbers used by PubMed for rare tumours) are not fetched.
• The XML fallback has been exercised on a fixture, not on the 313 MB file; it exists so a SPARQL outage does not block the monthly run.
• Alert bookkeeping in scripts/ci.ts warns when system_alerts is absent on a development database created before that table existed (drizzle-kit push fixes it locally).