# Connector openfda — Drugs@FDA applications and drug labels via openFDA

# Terms, license and disclaimer (quoted)

Terms of service, "Data Rights and Usage" (https://open.fda.gov/terms/):

Unless otherwise noted, the content, data, documentation, code, and related materials on openFDA is public domain and made available with a Creative Commons CC0 1.0 Universal dedication. In short, FDA waives all rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law. You can copy, modify, distribute, and perform the work, even for commercial purposes, all without asking permission. FDA makes no warranties about the work, and disclaims liability for all uses of the work, to the fullest extent permitted by applicable law. Some data on openFDA may not be public domain, such as copies of copyrightable works made available to the FDA by private entities.

License page (https://open.fda.gov/license/): same CC0 1.0 dedication, plus "When using or citing the work, you should not imply endorsement by the author or the affirmer." (The GMDN device-terminology restriction on that page does not apply to Drugs@FDA / SPL data.)

Disclaimer — returned in meta.disclaimer of every API response and repeated on the Drugs@FDA endpoint page ("Responsible use of the data"); stored verbatim in manifest.termsNotes:

Do not rely on openFDA to make decisions regarding medical care. While we make every effort to ensure that data is accurate, you should assume all results are unvalidated. We may limit or otherwise restrict your access to the API in line with our Terms of Service.

Always speak to your health provider about the risks and benefits of FDA-regulated products.

CancerIndex mirrors this: regulatory status is shown as a dated, sourced fact per application (evidence label regulatory), never as a treatment recommendation.

Rate limits (https://open.fda.gov/apis/authentication/, verified 2026-09-08):

With no API key: 240 requests per minute, per IP address. 1,000 requests per day, per IP address. With an API key: 240 requests per minute, per key. 120,000 requests per day, per key.

The connector runs at 3 req/s (180/min) sequentially. Without OPENFDA_API_KEY it stops itself after ~950 requests in a run (OPENFDA_DAILY_BUDGET_NO_KEY) and resumes from its cursor on the next run; a full pass over ~660 drugs needs roughly 900–1,100 requests, so a key (free, https://open.fda.gov/apis/authentication/) is recommended for the weekly schedule. The key is never written to provenance URLs (applicationProvenanceUrl).

# Verified endpoints

Health check: GET /drug/drugsfda.json?limit=1 → healthy when meta.last_updated is present.

# Ingestion

Scope (bounded, CLAUDE.md §227): every row of the local drugs table (CIViC therapies today, trial interventions later), ordered by id; ctx.cursor.lastDrugId is saved after each drug so a run stopped by the time budget or the daily quota resumes exactly where it stopped; a completed pass starts a new one on the next schedule.

1. Skip class / regimen names (ATR Inhibitor, Adjuvant Chemotherapy, Abemaciclib Regimen…, regex CLASS_NAME_RE on multi-word names) — queued in unresolved_labels (entity_kind drug, context.reason class_or_regimen_name) without spending a request.
2. Search openfda.generic_name:"<name>" (paged, ≤ 3 pages of 100); on no match, brand_name OR substance_name for single-token names, then ≤ 2 brand-like drug aliases via openfda.brand_name. Names with no accepted application → unresolved_labels (reason no_drugsfda_match, {source:'openfda'}).
3. Strict post-filter (classifyApplication): an application is accepted only when its generic name / active ingredient is the molecule itself — exact, salt form (IMATINIB MESYLATE), biologic four-letter suffix (TRASTUZUMAB-ANNS) or FDA proper-name prefix (FAM-TRASTUZUMAB DERUXTECAN for "trastuzumab deruxtecan"). Fixed-dose combinations (several substance_names, or "X, Y, and Z" generic names) and other molecules are unmatched. Kinds: reference (NDA/BLA of the molecule), generic_or_biosimilar (ANDA, or every matching name carries a biologic suffix). A drug named after an ADC (e.g. "Trastuzumab Emtansine") matches only under its own name.
4. Source records application (payload = full application) for every accepted application, label (set_id) for approval-bearing ones; drugs.unii filled when empty; openfda.brand_name[] → drug_aliases (alias_type brand).
5. Approval rows (drug_approvals, jurisdiction US, authority FDA, status approved) from the reference applications; when a molecule only exists as generics/biosimilars, the earliest-approved one bears the rows (others remain source records — ten identical ANDA rows would add no regulatory information). Events (approvalEvents): submissions with submission_status = AP and submission_type = ORIG → approvalType 'ORIG'; SUPPL with submission_class_code = EFFICACY → approvalType 'SUPPL'. Labeling / manufacturing supplements and tentative approvals (TA) are not approval events. approvalDate = submission_status_date (YYYYMMDD → YYYY-MM-DD). applicationNumber on every row; raw keeps submission number, review priority, class code, label/letter document URLs and the idempotency key.
   • ORIG rows carry the label's Indications and Usage. The highlights are split into one bullet per ( 1.N ) reference (splitIndications; fallbacks: • bullets, numbered subsections, whole text). Each bullet goes through the conservative dictionary pass (below): a bullet naming exactly one cancer becomes its own row (cancerId, indication = bullet text ≤ 4,000 chars, raw.cancer_match = {matchType:'PROBABILISTIC', via:'label text dictionary', alias}); the remaining bullets (0 or ≥ 2 cancers, tumor-agnostic) are joined into one row with cancerId null (raw.cancer_match.matchType 'UNRESOLVED'). When openFDA has no label for the application the ORIG row says so in indication.
   • SUPPL rows: indication = "Efficacy supplement <date> (see label)", cancerId null (the supplement letter, not the current label, defines what changed — parsing it is out of scope).
   • accelerated stays null unless the bullet text says "accelerated approval" (then true). review_priority = PRIORITY is a review track (priority review), not accelerated approval (21 CFR 314 subpart H) and is recorded in raw.reviewPriority only. Withdrawn accelerated approvals are not detected (limitation).
   • tumorAgnostic is true only when a bullet says "solid tumors" together with a biomarker (NTRK, MSI-H/dMMR, TMB-H, BRAF V600E, RET fusion, HER2, KRAS G12C…); such bullets are never mapped to the generic "Solid Neoplasm" concept.
6. Provenance: one row per application and run when anything changed (sourceUrl https://api.fda.gov/drug/drugsfda.json?search=application_number:"…", dataset 'Drugs@FDA via openFDA', datasetVersion 'drugsfda-<meta.last_updated>', evidenceType regulatory, accessLevel open, publishedAt = first approval date, updatedAtSource = label meta.last_updated, methodology text). Unchanged application + unchanged label + all rows present → the previous provenance row is reused (no duplicate evidence per weekly run).
7. Knowledge edges for rows with a cancerId: drug —APPROVED_FOR→ cancer, evidenceCategory regulatory_status, evidenceLevel 'FDA ORIG' | 'FDA SUPPL', direction supports, cancerContextIds [cancerId], sourceRecordId = application_number, provenanceIds [provenance] (unique per drug/cancer/application, refreshed not incremented).
8. Idempotency: rows of an application are keyed by raw.key (ORIG|<submission>|b<bullet>, ORIG|<submission>|rest, SUPPL|<submission>); existing keys are updated, new keys inserted, keys that no longer derive from the freshly fetched payload are deleted within that application only (never a mass delete on a shrunken response — the payload was fetched successfully).

dry_run probes osimertinib, pembrolizumab and imatinib and logs applications, kinds and event counts without writing.

# Cancer dictionary (conservative, PROBABILISTIC)

Built once per run from cancer_aliases of active, malignant concepts with top_level = true or entity_type IN (cancer, cancer_family, hematologic_malignancy, subtype, histology, molecular_subtype) — the spec's shorter list (cancer / cancer_family / hematologic_malignancy) was widened because NCIt models NSCLC, HNSCC, urothelial carcinoma, glioblastoma, colorectal cancer, CML (BCR-ABL1+) as subtype / molecular_subtype; without them most oncology labels would stay unmapped. Rules:

• alias length ≥ 6 characters, or ≥ 4 for curated abbreviation aliases (NSCLC, HNSCC, GIST, DLBCL); 3-letter abbreviations (AML, CML, ALL) are excluded;
• generic words are stop-listed (cancer, tumor, carcinoma, solid tumor, leukemia, lymphoma, sarcoma, adenocarcinoma, squamous cell carcinoma, carcinoma in situ, metastatic disease…, GENERIC_ALIAS_STOPLIST);
• an alias shared by several concepts is kept only when CancerResolver.byLabel disambiguates it (preferred name, curated display name, broadest concept of one lineage — e.g. "breast cancer" → Malignant Breast Neoplasm); otherwise it is dropped (ambiguousDropped);
• matching is a whole-word n-gram lookup over normalizeLabel(bullet); overlapping mentions resolve to the longest span ("metastatic breast cancer" beats "breast cancer"), then the bullet is mapped only if exactly one distinct cancer remains — after the CancerReconciler has collapsed (a) equivalent rows minted by two terminologies (the canonical name of one is an alias of the other, e.g. OncoTree "Non-Small Cell Lung Cancer" and NCIt "Lung Non-Small Cell Carcinoma"; the NCIt-coded row wins) and (b) candidates on one lineage to the narrowest concept (an indication sentence names the specific disease and uses broader words as context: "Head and Neck Squamous Cell Cancer (HNSCC) … squamous cell cancer" → HNSCC; "HES and/or CEL" → CEL, a descendant of HES in NCIt). "Melanoma or breast cancer" (different diseases) → null. The reconciler also settles aliases shared by equivalent rows at dictionary build time (282 → 19 ambiguous aliases dropped on the dev DB).

Every mapped row is PROBABILISTIC (text mention, not an identifier) and says so in raw.cancer_match (via adds (equivalent) / (narrowest_of_lineage) when the reconciler intervened, distinctMentions counts the raw mentions); curators can review drug_approvals WHERE cancer_id IS NULL (raw.bullets[].distinctCancers).

# Replay without HTTP (--mode backfill)

pnpm cix run openfda --mode backfill re-derives every approval row and edge from the payloads already in the raw lake (source_records.raw_path → RawLake.read), applying the current dictionary / reconciler rules: no request, cursor untouched, ~15 s for 1,500 records. One application can serve several drug rows ("Abiraterone" / "Abiraterone Acetate", "Osimertinib" / "Osimertinib Mesylate"); the source record keeps only the last canonical id, so the backfill also consults existing drug_approvals (drug_id, application_number) pairs. This keeps RAW → NORMALIZED replayable (CLAUDE.md §2) and is the way to refresh mappings when the daily quota is exhausted.

# Observed run (cancerindex_b, 2026-09-08 — 661 drugs from CIViC therapies, NCIt + HGNC + CIViC loaded)

Examples:

• Osimertinib — NDA208065 (reference; TAGRISSO; UNII 3C06JJ0Z2O filled): ORIG 2015-11-13 split into 5 bullets, all → Lung Non-Small Cell Carcinoma; 8 efficacy supplements 2017-03-30 … 2024-09-25 (dated, cancerId null).
• Pembrolizumab — BLA125514 (KEYTRUDA; BLA761467 KEYTRUDA QLEX rejected as a fixed combination with berahyaluronidase alfa): ORIG 2014-09-04 → 36 bullets, 35 mapped (Melanoma ×2, NSCLC ×6, HNSCC ×4, cHL, PMBCL, urothelial carcinoma, bladder ×2, MSI-H colorectal, gastric, esophageal, cervical ×3, HCC, biliary tract, Merkel cell, RCC ×2, endometrial ×3, TNBC ×3, mesothelioma); 1 unmapped row (paediatric cHL wording + the MSI-H/dMMR and TMB-H tumor-agnostic bullets, tumorAgnostic = true, accelerated = true); 108 efficacy supplements 2015-10-02 … 2026-07-10.
• Imatinib — NDA021588 Gleevec (tablets, ORIG 2003-04-18) and NDA219097 Imkeldi (2024-11-22) as reference; 10 ANDAs as source records only. ORIG bullets → Ph+ ALL ×2, GIST (metastatic + adjuvant), DFSP, systemic mastocytosis, MDS/MPN, CEL; the "HES and/or CEL" bullet resolved to CEL by the narrowest-of-lineage rule; the Ph+ CML bullet stays unmapped because "chronic myeloid leukemia" is only a molecular_subtype alias ("…, BCR-ABL1 Positive") whose ambiguity the resolver cannot settle — a curation item. 19 efficacy supplements 2003-05-20 … 2016-08-25.
• Counters: after pnpm cix counters, 407 cancers have approved_drug_count > 0; top-level leaders: Malignant Lung Neoplasm 43 drugs, Malignant Breast Neoplasm 35, Leukemia 22, Malignant Kidney Neoplasm 15, Malignant Prostate Neoplasm 14, Malignant Liver Neoplasm 14.

Idempotent: re-running updates rows in place (raw.key), reuses provenance when neither the application nor the label changed, and only removes rows of an application that no longer derive from its freshly fetched payload.

# Limitations / notes

• One pass costs ~1.5 requests per drug (+1 label per approval-bearing application); without an API key the 1,000/day IP quota is the binding constraint (self-stop at 950, cursor saved).
• Efficacy supplements are dated but not mapped to cancers (no label diff); the current label's bullets are attached to the ORIG event date, so a bullet added by a 2024 supplement is stored under the original approval row set with the ORIG date — the supplement rows carry the actual dates. A future refinement can match application_docs label PDFs per supplement.
• accelerated relies on label wording ("approved under accelerated approval"); conversions to regular approval and withdrawals are not tracked yet (Project Confirm — spec §13 FDA Oncology, later).
• ANDA generics / biosimilars of a molecule that already has a reference NDA/BLA are kept as source records only.
• Line of therapy, disease stage and biomarker ids (lineOfTherapy, diseaseStage, biomarkerIds) are not extracted (Phase 2).
• Only US / FDA. Health Canada, EMA, MHRA, PMDA are separate connectors (spec §13).

# 2026-09-11 — shared-alias mapping fix

A label alias shared by a concept and one of its descendants ("breast cancer" on Breast Carcinoma and on Childhood Breast Carcinoma) used to resolve to the narrowest concept when the dictionary was built (the per-sentence collapse rule leaked into dictionary construction): 67 approval rows and their APPROVED_FOR edges pointed at childhood entities. CancerDictionary.build now resolves a shared alias with the resolver's preferred/display/broadest rule, then equivalence merging, then the broadest of one lineage (CancerReconciler.broadest), and never the narrowest. --mode backfill replays the lake with the corrected dictionary and marks edges the replay no longer asserts as superseded (99 on the 2026-09-10 lake). Regression test: dictionary-shared-alias.test.ts.