SPB Git forge

spb/cancerindex

Public
37commits 1branches 0releases
2.9 MBsize
maindefault branch
10 days agolast push
TypeScript 97.2% SQL 1.5% CSS 0.6% JavaScript 0.5%
12.6 KB

# Connector cbioportal — cBioPortal for Cancer Genomics (public studies)

Source https://www.cbioportal.org — public instance operated by Memorial Sloan Kettering Cancer Center with the cBioPortal consortium (Dana-Farber, Princess Margaret, Children's Hospital of Philadelphia, The Hyve, Bilkent…)
Tier / category 1 / genomics (CLAUDE.md §10.8 — cohorts beyond TCGA, original study IDs preserved)
Access REST https://www.cbioportal.org/api, no authentication, OpenAPI 3 (/api/v3/api-docs, Swagger UI /api/swagger-ui/index.html)
Docs verified 2026-09-08 — OpenAPI document, https://docs.cbioportal.org/user-guide/faq/ (What is cBioPortal, license, citation), live API
License ODC Open Database License (ODC-ODbL) unless otherwise noted per study, attribution to the original studies required — licenseStatus: approved, redistribution: attribution, commercialUse: restricted (some studies restrict commercial use)
Code packages/connectors/src/connectors/cbioportal/ (manifest.ts, normalize.ts, index.ts, cbioportal.test.ts, fixtures/)

# License and citation (quoted)

FAQ, "Do I need permission or a license to use the cBioPortal logo or figures or data?" (https://docs.cbioportal.org/user-guide/faq/):

You are free to use figures from cBioPortal, there is no copyright permission required. However you must provide proper citation for the cBioPortal and the original data source as described above. Unless otherwise noted, data in cBioPortal are available under the ODC Open Database License with no restrictions on the use of the data, as long as you properly give attribution to the original studies. There are some studies that restrict the commercial use of the data, but that will be explicitly mentioned in the study information.

FAQ, "What is the cBioPortal for Cancer Genomics?":

The cBioPortal for Cancer Genomics is an open-access, open-source resource for interactive exploration of multidimensional cancer genomics data sets.

FAQ, "How do I cite the cBioPortal?": Cerami E et al., The cBio Cancer Genomics Portal: An Open Platform for Exploring Multidimensional Cancer Genomics Data, Cancer Discov 2012 (PMID 22588877); Gao J et al., Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal, Sci Signal 2013 (PMID 23550210); de Bruijn I et al., Analysis and Visualization of Longitudinal Genomic and Clinical Data from the AACR Project GENIE Biopharma Collaborative in cBioPortal, Cancer Res 2023 (PMID 37668528). "Remember also to cite the source of the data if you are using a publicly available dataset."

Consequences for CancerIndex: the portal citation is the manifest attribution; every cohort's provenance carries the study's own citation and pmid (417 of 545 studies have one) and a publication stub is created for the PMID; the study description (where commercial-use notices appear) is kept on the source record; only aggregate per-gene counts are stored (no sample- or patient-level data).

# Verified endpoints

Endpoint Verified behaviour (2026-09-08)
GET /api/studies?pageSize=1000&pageNumber=0&projection=DETAILED 545 public studies (one page; pageNumber paging supported). Fields: studyId, cancerTypeId (OncoTree code, lower case; mixed for pan-cancer cohorts — 48 studies), name, description, publicStudy, pmid, citation, importDate "2026-07-28 17:54:40", allSampleCount, sequencedSampleCount (samples with mutation data), cnaSampleCount, referenceGenome (473 hg19, 72 hg38), cancerType{id,name,parent}. 539/545 have sequencedSampleCount > 0.
GET /api/cancer-types?pageSize=10000 897 OncoTree nodes {cancerTypeId, name, shortName, parent}; root tissue; mixed → parent other.
GET /api/studies/{studyId}/molecular-profiles e.g. msk_impact_2017_mutations (molecularAlterationType MUTATION_EXTENDED, datatype MAF), _cna, _structural_variants.
GET /api/sample-lists/{studyId}_sequenced {category all_cases_with_mutation_data, sampleCount, sampleIds[]} — msk_impact_2017_sequenced 10,945 = sequencedSampleCount; acc_tcga_sequenced 90 = sequencedSampleCount. The study-level count is therefore used as the denominator source without downloading sample ids (the SUMMARY projection of /studies/{id}/sample-lists has no sampleCount).
POST /api/mutated-genes/fetch body {"studyIds":["msk_impact_2017"]} (StudyViewFilter) Array of AlterationCountByGene: hugoGeneSymbol, entrezGeneId, numberOfAlteredCases (distinct samples with ≥ 1 mutation in the gene), numberOfProfiledCases (samples profiled for that gene — gene-panel aware: MSK-IMPACT genes on IMPACT341+410 → 10,945; genes only on IMPACT410 → 8,136; 4 genes → 0), totalCount (mutations), matchingGenePanelIds. TP53 4,538/10,945, KRAS 1,643, TERT 1,460, PIK3CA 1,355, APC 1,121 — identical to the study summary "Mutated Genes" table. ~1 s for 10,945 samples.
Alternatives inspected /api/molecular-profiles/{id}/mutations/fetch (per-mutation rows, pageSize ≤ 10,000,000 — hundreds of thousands of rows for large studies) and /api/molecular-profiles/{id}/mutations?sampleListId=…&entrezGeneId=… (one gene at a time) exist but are far more expensive; /api/mutation-counts-by-gene does not exist; /api/studies/{id}/significantly-mutated-genes is MutSig only. /api/mutated-genes/fetch is the cheapest endpoint returning per-gene counts with a denominator.

Health check: GET /api/studies?pageSize=1&projection=SUMMARY → healthy when one study is returned.

# Ingestion

  1. /studies (DETAILED, paged) + /cancer-types. Anomaly guard: < 100 studies → refuse to persist. datasetVersion = cbioportal-<latest importDate>.
  2. Studies are processed in studyId order; ctx.cursor.lastStudyId is saved after each study (restartable, time budget honoured between studies); a completed pass starts over on the next schedule.
  3. Cohort (genomic_cohorts, unique on source_id + study_id): studyId, name, program = first token of the name's trailing parenthetical (MSK-IMPACT Clinical Sequencing Cohort (MSK, Nat Med 2017) → MSK; … (TCGA, PanCancer Atlas) → TCGA), primarySites = OncoTree node just below the tissue root (luad → nsclc → lung → Lung), diseaseTypes = cancer-type name, caseCount = allSampleCount, casesWithSsm = sequencedSampleCount, dataRelease = importDate (date), accessLevel open, url https://www.cbioportal.org/study/summary?id=<studyId>, provenance per study (sourceUrl /api/studies/<id>, dataset 'cBioPortal public studies', pmid, cohortSize, population '<studyId> (<program>)', methodology with the study citation). An unchanged study keeps its previous provenance row.
  4. Cancer reconciliation: resolver.byCode('oncotree', CANCERTYPEID.toUpperCase()) → EXACT_IDENTIFIER; fallback byLabel(cancer-type name) (match type recorded, and the code stored in cancer_codes with that match type); studies filed under a tissue-level node (direct child of the OncoTree root: "Breast", "Prostate", "Bladder/Urinary Tract", "Soft Tissue", "Bowel"…) are rewritten into site-level malignancy labels (tissueLevelCandidates: Breast → Breast Cancer / Malignant Breast Neoplasm / Breast Carcinoma, Soft Tissue → Soft Tissue Sarcoma, Bowel → Colorectal Cancer) and recorded as CURATED_BROADER; cancerTypeId = mixed → cancerId null, cancerMatchType 'UNRESOLVED' (pan-cancer, not queued — by design); other misses → unresolved_labels with the OncoTree lineage.
  5. Publication stub for pmid (publications, publicationTypes ['stub'], title = citation) + publication_entity_edges to the cohort's cancer (method cbioportal_study) — enriched later by the PubMed connector.
  6. Frequencies (cancer_gene_frequencies, unique on cohort_id + gene_symbol + alteration_type) for studies with sequencedSampleCount > 0 and a MUTATION_EXTENDED profile (/molecular-profiles and /mutated-genes/fetch are requested concurrently — concurrency 2): genes ranked by numberOfAlteredCases (desc, symbol asc), top 200 per study (TOP_GENES_PER_STUDY; WES studies return 7,000–18,000 genes), genes with numberOfProfiledCases = 0 dropped. alterationType 'ssm', casesAffected = numberOfAlteredCases, casesProfiled = numberOfProfiledCases (gene-panel aware; equals the <studyId>_sequenced count for exome studies), frequency validated by validateFrequency() (rejected rows counted, never stored), rank, dataRelease, geneId via the shared GeneCache (Entrez id filled when empty), cancerId of the cohort. Provenance per study (sourceUrl /api/mutated-genes/fetch, dataset 'cBioPortal mutated genes by study', pmid, cohortSize = sequencedSampleCount, methodology: distinct samples with ≥ 1 mutation in gene / samples profiled for the gene (numberOfProfiledCases; = samples in <studyId>_sequenced for whole-exome studies), profile id, top-N note). Source record mutated_genes per study with the ranked list.
  7. Studies without a mutation profile are cohorts without frequencies (noMutationProfile in the run summary); no per-mutation paging is ever needed, so no sample cap applies.

TCGA cohorts also present in GDC are kept as separate cohorts (different processing: Firehose Legacy vs PanCancer Atlas vs GDC harmonized) and flagged by program = TCGA; they are never pooled with GDC frequencies.

dry_run fetches studies + cancer types and the top-5 mutated genes of acc_tcga without writing.

# Observed run (cancerindex_b, 2026-09-08 — OncoTree + NCIt + HGNC loaded)

Smoke (--max-records 10) 5 studies (acbc_mskcc_2015 … acc_tcga_pan_can_atlas_2018), 12 requests, 5 s, all EXACT_IDENTIFIER
Full pass (--max-minutes 20) 545 studies / 545 cohorts in 6–8 min, 1,080 requests, 0 HTTP failures, 0 studies rejected
Cancer mapping 489 / 545 (89.7 %): 432 EXACT_IDENTIFIER (OncoTree code), 57 CURATED_BROADER (tissue-level rewrite); 48 mixed pan-cancer cohorts unmappable by design (MSK-IMPACT 2017, GENIE-derived pan-cancer sets, mixed_allen_2018…); 98.4 % of non-mixed studies mapped; 8 unresolved (tissue-level sites without a site-level malignancy alias in the ontology yet)
Programs 139 distinct — MSK 171, TCGA 88, TCGA GDC 33, Broad 15, DFCI 10, CPTAC GDC 10, CPTAC 6…; 17 studies without a parenthetical source (program null)
Frequencies 100,444 rows in 539 cohorts (9,972 distinct genes, all resolved to a genes row — 0 minimal genes created after HGNC), 90,521 rows with a cancerId; denominators 1 – 54,331; 7 rejected by validateFrequency (bowel_colitis_msk_2022: FAT3 11/1, SMOX 4/1… — portal gene-panel matrix inconsistency, never stored)
Publications 378 PMIDs → publication stubs + publication_entity_edges to the cohort's cancer
Counters genomic_cohorts ranking eligible 109 cancers (all scopes) / 9 top-level; e.g. Malignant Breast Neoplasm 7 cohorts, Malignant Lung Neoplasm 5

msk_impact_2017 (10,945 samples, mixed, program MSK, PMID 28481359) top 5 — cases_affected / cases_profiled: TP53 4,538/10,945 (0.415), KRAS 1,643/10,945 (0.150), TERT 1,460/10,945 (0.133), PIK3CA 1,355/10,945 (0.124), APC 1,121/10,945 (0.102) — identical to the portal's study-summary table; genes only on the 410-gene panel carry the panel-aware denominator 8,136.

acc_tcga (90 sequenced) top 3: ZFPM1 47/90, MUC5B 35/90, GARS1 34/90 (Firehose Legacy calls; the PanCancer Atlas cohort acc_tcga_pan_can_atlas_2018 of the same tumours gives TP53 18/91, CTNNB1 14/91 — kept as separate cohorts, as is GDC's TCGA-ACC).

Idempotent: a second full pass (run twice today after rule changes, --reset-cursor) updated cohorts and frequencies in place, reused cohort ids and provenance rows for unchanged studies, and created no duplicate genes or publications.

# Limitations / notes

  • Mutation counts follow the portal's default study-view filters (all mutation types the portal shows in "Mutated Genes"; germline/somatic and driver/VUS filters untouched); counts are per sample, not per patient.
  • Only alteration_type = 'ssm'; CNA (/api/cna-genes/fetch) and structural-variant counts are not ingested yet.
  • Top 200 genes per study; long-tail genes are not stored (the source record keeps the ranked list only up to the cap).
  • mixed studies (48, incl. MSK-IMPACT 2017) have no cancer; their frequencies are cohort-level only and never roll up to a cancer.
  • Some studies restrict commercial use (noted in their description on the portal) — review before any commercial redistribution of those cohorts; commercialUse: restricted at the manifest level reflects this.