SPB Git forge

spb/cancerindex

Public
37commits 1branches 0releases
2.9 MBsize
maindefault branch
10 days agolast push
TypeScript 97.2% SQL 1.5% CSS 0.6% JavaScript 0.5%
14.0 KB

# Connector chembl — ChEMBL (EMBL-EBI)

Source https://www.ebi.ac.uk/chembl/ — European Molecular Biology Laboratory, European Bioinformatics Institute
Tier / category 1 / drugs
Access REST, https://www.ebi.ac.uk/chembl/api/data/<resource>.json, no authentication
Docs verified 2026-09-08 — https://chembl.gitbook.io/chembl-interface-documentation/web-services/chembl-data-web-services + live probes (status, molecule/search, mechanism, target, drug_indication)
License CC BY-SA 3.0 — licenseStatus: approved, redistribution: attribution (share-alike noted)
Code packages/connectors/src/connectors/chembl/ (manifest.ts, api.ts, index.ts, chembl.test.ts, fixtures/)

# License (quoted)

About page (https://chembl.gitbook.io/chembl-interface-documentation/about, fetched 2026-09-08):

The ChEMBL data is made available on a Creative Commons Attribution-Share Alike 3.0 Unported License.

Attribution: "ChEMBL, EMBL-EBI, CC BY-SA 3.0" on /source/chembl and in the API sources envelope; every run records the release (datasetVersion, e.g. ChEMBL_37, release DOIs exist on the Downloads page). Share-alike: the fields and edges derived from ChEMBL carry ChEMBL provenance so a redistributed extract can be published under a compatible licence. Citation: Zdrazil B et al., The ChEMBL Database in 2023, Nucleic Acids Res 2024 (PMID 37933841).

# Verified endpoints (2026-09-08, ChEMBL_37 released 2026-05-01)

Endpoint Used for Notes
status.json health, datasetVersion {status: "UP", chembl_db_version: "ChEMBL_37", chembl_release_date, disinct_compounds: 2921148, targets: 18552}
molecule/search.json?q=<name>&limit=20 identity full molecule objects (pref_name, molecule_type, max_phase, first_approval, molecule_synonyms[{molecule_synonym, syn_type}], molecule_hierarchy.parent_chembl_id, atc_classifications, cross_references, …) + page_meta
mechanism.json?parent_molecule_chembl_id=<id>&limit=200 mechanisms mechanism_of_action, action_type, target_chembl_id, max_phase, mechanism_refs. Attached to the dosed form: molecule_chembl_id=CHEMBL941 (imatinib) returns 0, the parent filter returns the 4 mechanisms recorded on CHEMBL1642 (mesylate)
target/<id>.json targets → genes target_type (SINGLE PROTEIN, CHIMERIC PROTEIN, PROTEIN FAMILY, PROTEIN COMPLEX…), organism, target_components[].accession (UniProt) and target_component_synonyms with syn_type GENE_SYMBOL
drug_indication.json?parent_molecule_chembl_id=<id>&limit=500&offset= indications mesh_id, mesh_heading, efo_id/efo_term (EFO or MONDO), max_phase_for_ind (string "3.0"; mechanism max_phase is a number — both coerced)

No published rate limit ("reasonable use") → manifest 3 req/s, 2 in flight; targets are cached for the run. During development the API returned HTTP 500 for several minutes (EBI outage) — the SDK retries 5xx with back-off and the run fails cleanly if the outage persists; the cursor lets the next run resume.

# Scope — enrichment, not import

Only existing drugs rows (CIViC therapies today, trial interventions later) are looked up; the 2.9 M ChEMBL compounds are never crawled. Names that cannot be molecules — regimens (… Regimen), combinations (A/B), procedures (radiation, surgery, transplantation) — are skipped and counted (isNonMoleculeName).

# Sync design

Drugs are processed in (name, id) order; ctx.cursor = {pass, after: {name, id}, processed, completedAt} is flushed every 25 drugs so a run stopped by the time budget (--max-minutes) or a crash resumes after the last processed drug. A completed pass starts a fresh pass on the next run (full re-check against the current release; every write is idempotent).

Per drug:

  1. Identity — molecule/search.json?q=<name> then pickExactMatch(): the hit whose pref_name, or one of its molecule_synonyms, equals our name after normalizeLabel (case/punctuation-insensitive). Ties: pref_name beats synonym → higher max_phase → parent molecule (molecule_hierarchy.parent_chembl_id = itself) → lowest ChEMBL number. Examples: "Osimertinib" → CHEMBL3353410 (free base, not the mesylate); "Imatinib" → CHEMBL941, "Imatinib Mesylate" → CHEMBL1642 (both exist as CIViC therapies); "Trastuzumab" → CHEMBL1201585 (not the ADCs). No exact hit → unresolved_labels (entity_kind drug, context = total hits + top 5 {chemblId, prefName, maxPhase, type}) — never a fuzzy assignment.
  2. Mechanisms for the parent molecule, deduplicated by (mechanism, target, action type) — ChEMBL repeats a mechanism per salt/dosed form.
  3. Targets: each target_chembl_id → target/<id>.json (cached) → human protein components → GENE_SYMBOL synonyms → GeneCache.ensure(symbol) (shared with CIViC/ClinVar/GDC; HGNC-style symbol check). Chimeric proteins (Bcr/Abl → ABL1 + BCR) and protein families (ABL → ABL1 + ABL2) expand to every component gene. Non-human targets yield nothing.
  4. Indications for the parent molecule (paged), deduplicated by MeSH UI with the highest max_phase_for_ind; EFO/MONDO ids accumulated.
  5. Raw record ctx.upsertSourceRecord('molecule', <ChEMBL id>, {molecule, parentChemblId, matchedBy, firstApproval, atcClassifications, mechanisms, targets, indications}) → raw lake data/raw/chembl/<date>/molecule/. first_approval and ATC codes live only here: a first-approval year is not a jurisdictional approval record (drug_approvals is fed by regulatory connectors, CLAUDE.md §13).
  6. Provenance: one row per (drug, molecule) — sourceRecordId = ChEMBL id, sourceUrl https://www.ebi.ac.uk/chembl/compound_report_card/<id>/, dataset ChEMBL, datasetVersion ChEMBL_37, evidenceType database, accessLevel open, license CC BY-SA 3.0 — reused while the source record is unchanged (looked up from the drug's existing edges).
  7. drugs row: chembl_id; kind from molecule_type (Small molecule → small_molecule, Antibody → monoclonal_antibody, Antibody drug conjugate → adc, Cell → cell_therapy, Gene → gene_therapy, Protein / Enzyme / Oligonucleotide / … → other; Unknown keeps the existing value); mechanism = distinct mechanism_of_action strings joined with "; "; development_status from max_phase (4 → approved, 3 → phase_3, 2 → phase_2, 1 → phase_1, 0.5 → early_phase_1, null/−1 untouched) and never downgraded (a regulatory approved survives a ChEMBL phase 3); target_gene_ids = union with the new gene ids (sql.raw('ARRAY[…]')). A change_events row records the ChEMBL id assignment.
  8. drug_aliases from molecule_synonyms: TRADE_NAME → brand, INN/USAN/BAN/JAN/USP → generic, RESEARCH_CODE → development_code, anything else → synonym; the drug's own name is skipped, duplicates collapse per (normalized, type).
  9. knowledge_edges
    • drug TARGETS gene — one edge per gene, evidenceLevel = ChEMBL action_type(s) joined with | (INHIBITOR, ANTAGONIST, BINDING AGENT…), direction null, cancerContextIds [], evidenceCategory curated_evidence, sourceRecordId = ChEMBL id, provenanceIds [provenance].
    • drug INVESTIGATED_FOR cancer — for each indication whose mesh_id resolves through resolver.byCode('mesh', ui) (i.e. cancer_codes system 'mesh' written by the mesh connector; run mesh first): evidenceLevel = max_phase_for_ind as text ("4", "3", …, never re-scaled), direction 'unknown', cancerContextIds [cancer], evidenceCategory curated_evidence. Unresolved MeSH ids go to unresolved_labels (entity_kind cancer, context {meshId, efoIds, chemblId, drugId, drugName, maxPhaseForIndication}) only when the heading is a neoplasm — known MeSH neoplasm descriptor (mesh source records / codes) or NEOPLASM_HEADING_RE — so rheumatoid arthritis, malaria or pulmonary hypertension indications (imatinib) do not pollute the cancer queue; they are counted as indicationsNonCancer.
    • Upsert on the unique key (source/target/type/source_id/source_record_id); supportCount stays 1, lastSeenAt refreshed.

evidenceCategory note: the task brief suggested database for indication edges; knowledge_edges.evidence_category is the scientific safety label set (observed_data | published_evidence | curated_evidence | regulatory_status | clinical_guideline | computed_metric, CLAUDE.md §3/§12) and ChEMBL indications are manually curated from labels and trial registries, so they are labelled curated_evidence; the provenance row carries evidenceType database. Change evidenceCategory in index.ts if the integrator prefers a new label.

# Tests (chembl.test.ts, fixtures from live responses, trimmed)

Search pages (osimertinib, imatinib, trastuzumab, empty, page with a malformed molecule), mechanisms (osimertinib single, imatinib four mechanisms, duplicate dosed forms), targets (single proteins EGFR/ABL1/KIT/ERBB2/PDGFRB, chimeric Bcr/Abl, ABL family, non-human), indications (dedupe by UI with max phase, string-phase coercion, multi-page merge, malformed rows), vocabulary mappings (kind, phase, synonym type, non-molecule names), manifest licence.

# Observed run (cancerindex_c, 2026-09-08, --max-minutes 15, 661 CIViC drugs)

The EBI API was unstable during the day (HTTP 500 / hung connections for ~40 min; 16 of 2,622 requests failed after retries): the first 15-minute run processed only 14 drugs (each hung request costs the SDK's 60 s timeout × retries). Once the API recovered the resumable cursor was used to cover the whole list in segments (10 runs, 62 min wall-clock, ~1.3 s per drug when healthy — a healthy full pass of 661 drugs takes ≈ 12 min).

Drugs considered 661 (every drug processed at least once across the segments); 66 skipped as non-molecule names (regimens, combinations, procedures)
Matched (exact name) 401 drugs enriched with a ChEMBL id (≈ 91 % by pref_name, 9 % by synonym); 190 names unresolved → unresolved_labels (entity_kind drug, with the top hits)
kind small_molecule 342, monoclonal_antibody 36, adc 10, other 7, gene_therapy 2, cell_therapy 1 (3 kept null — ChEMBL Unknown)
development_status approved 223, phase_3 73, phase_2 67, phase_1 9 (29 without a ChEMBL phase)
mechanism / target_gene_ids 339 drugs with mechanism text; 314 drugs with ≥ 1 target gene
drug_aliases 2,830 — development_code 1,192, synonym 847, brand 741, generic 50
drug TARGETS gene 1,171 edges (314 drugs → 349 genes), evidence_level = ChEMBL action type
drug INVESTIGATED_FOR cancer 6,545 edges (361 drugs → 218 cancers), evidence_level = max phase for the indication
Indication headings not resolved 99 distinct MeSH headings queued as unresolved cancers (e.g. Prostatic Neoplasms, Castration-Resistant [D064129] 41×, Neoplasm Metastasis, Adenocarcinoma, Gliosarcoma, Lymphoma Large B-Cell Diffuse, Glioma, Neuroblastoma) — descriptors the mesh connector could not map (genuine gaps or, on this development database, headings ambiguous because OncoTree minted duplicates before NCIt ran); non-cancer indications (rheumatoid arthritis, malaria, pulmonary hypertension…) are counted, not queued
Samples Osimertinib → CHEMBL3353410, small_molecule, approved, mechanism "Epidermal growth factor receptor erbB1 inhibitor", TARGETS EGFR (INHIBITOR), aliases Tagrisso (brand), AZD-9291 / AZD9291 (development_code), INVESTIGATED_FOR Lung Non-Small Cell Carcinoma (4), Lung Neoplasm (3), Lung Adenocarcinoma (2), Glioblastoma (2)… — Imatinib → CHEMBL941, mechanism "Tyrosine-protein kinase ABL inhibitor; Platelet-derived growth factor receptor beta inhibitor; Stem cell growth factor receptor inhibitor; Bcr/Abl fusion protein inhibitor", TARGETS ABL1, KIT, PDGFRB, BCR, INVESTIGATED_FOR Gastrointestinal Stromal Tumor (4), Chronic Myeloid Leukemia BCR-ABL1 Positive (4), Acute Lymphoblastic Leukemia (4), Dermatofibrosarcoma Protuberans (4)… — Trastuzumab → CHEMBL1201585, monoclonal_antibody, approved, "Receptor protein-tyrosine kinase erbB-2 inhibitor", TARGETS ERBB2, INVESTIGATED_FOR Breast Neoplasm (4), Gastric Neoplasm (4), Esophageal Neoplasm (3)…; Trastuzumab Deruxtecan → adc, TARGETS ERBB2 + TOP1

Idempotency: re-processing a drug reports the source record unchanged, reuses its provenance row and refreshes edges in place. Note for the integrator: CIViC created two drugs rows named "Trastuzumab Deruxtecan"; both received the same ChEMBL id (a merge candidate).

# Resolver note

CancerResolver.byCode() only caches codes with match type EXACT_IDENTIFIER / CURATED_EXACT / ONTOLOGY_EXACT. The mesh connector also writes deliberate ALIAS (entry-term) and CURATED_BROADER codes (206 of 428), so the connector loads cancer_codes system 'mesh' itself (all match types, ambiguous UIs dropped) and uses the resolver as the first lookup. Widening the resolver's filter to ALIAS would make byCode('mesh', …) complete for every consumer (PubMed currently falls back to byLabel on the mesh_heading alias, so its edges are unaffected).

# Limitations / notes

  • Identity is exact-name only: CIViC names that differ from ChEMBL pref_name/synonyms (e.g. investigational codes written differently, salts named "X Hydrochloride" vs ChEMBL "X HYDROCHLORIDE" are fine, but "Anti-PD-1 antibody" is not) stay unresolved and appear in the queue with the top hits for a curator.
  • Mechanisms/indications come from the parent molecule, so a salt row (e.g. "Imatinib Mesylate") receives the same mechanism and indication edges as its parent — both rows keep their own ChEMBL id.
  • max_phase_for_ind reflects the highest trial phase ChEMBL curated for that indication, not an approval; approvals are regulatory records (FDA/EMA connectors).
  • Targets without a human protein component (organisms, viral proteins, "unchecked" targets) produce no gene edge; non-protein targets are skipped.
  • The connector depends on the mesh connector for indication edges; run order: ncit-evs → oncotree → hgnc → civic → mesh → chembl.