spb/cancerindex

1# Connector `cbioportal` — cBioPortal for Cancer Genomics (public studies)23| | |4|---|---|5| Source | https://www.cbioportal.org — public instance operated by Memorial Sloan Kettering Cancer Center with the cBioPortal consortium (Dana-Farber, Princess Margaret, Children's Hospital of Philadelphia, The Hyve, Bilkent…) |6| Tier / category | 1 / genomics (CLAUDE.md §10.8 — cohorts beyond TCGA, original study IDs preserved) |7| Access | REST `https://www.cbioportal.org/api`, no authentication, OpenAPI 3 (`/api/v3/api-docs`, Swagger UI `/api/swagger-ui/index.html`) |8| Docs verified | 2026-09-08 — OpenAPI document, https://docs.cbioportal.org/user-guide/faq/ (What is cBioPortal, license, citation), live API |9| License | **ODC Open Database License (ODC-ODbL)** unless otherwise noted per study, attribution to the original studies required — `licenseStatus: approved`, `redistribution: attribution`, `commercialUse: restricted` (some studies restrict commercial use) |10| Code | `packages/connectors/src/connectors/cbioportal/` (`manifest.ts`, `normalize.ts`, `index.ts`, `cbioportal.test.ts`, `fixtures/`) |1112## License and citation (quoted)1314FAQ, "Do I need permission or a license to use the cBioPortal logo or figures or data?" (https://docs.cbioportal.org/user-guide/faq/):1516> You are free to use figures from cBioPortal, there is no copyright permission required. However you must provide proper citation for the cBioPortal and the original data source as described above. Unless otherwise noted, data in cBioPortal are available under the ODC Open Database License with no restrictions on the use of the data, as long as you properly give attribution to the original studies. There are some studies that restrict the commercial use of the data, but that will be explicitly mentioned in the study information.1718FAQ, "What is the cBioPortal for Cancer Genomics?":1920> The cBioPortal for Cancer Genomics is an open-access, open-source resource for interactive exploration of multidimensional cancer genomics data sets.2122FAQ, "How do I cite the cBioPortal?": Cerami E et al., *The cBio Cancer Genomics Portal: An Open Platform for Exploring Multidimensional Cancer Genomics Data*, Cancer Discov 2012 (PMID 22588877); Gao J et al., *Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal*, Sci Signal 2013 (PMID 23550210); de Bruijn I et al., *Analysis and Visualization of Longitudinal Genomic and Clinical Data from the AACR Project GENIE Biopharma Collaborative in cBioPortal*, Cancer Res 2023 (PMID 37668528). "Remember also to cite the source of the data if you are using a publicly available dataset."2324Consequences for CancerIndex: the portal citation is the manifest `attribution`; every cohort's provenance carries the study's own `citation` and `pmid` (417 of 545 studies have one) and a publication stub is created for the PMID; the study `description` (where commercial-use notices appear) is kept on the source record; only aggregate per-gene counts are stored (no sample- or patient-level data).2526## Verified endpoints2728| Endpoint | Verified behaviour (2026-09-08) |29|---|---|30| `GET /api/studies?pageSize=1000&pageNumber=0&projection=DETAILED` | 545 public studies (one page; `pageNumber` paging supported). Fields: `studyId`, `cancerTypeId` (OncoTree code, lower case; `mixed` for pan-cancer cohorts — 48 studies), `name`, `description`, `publicStudy`, `pmid`, `citation`, `importDate "2026-07-28 17:54:40"`, `allSampleCount`, **`sequencedSampleCount`** (samples with mutation data), `cnaSampleCount`, `referenceGenome` (473 hg19, 72 hg38), `cancerType{id,name,parent}`. 539/545 have `sequencedSampleCount > 0`. |31| `GET /api/cancer-types?pageSize=10000` | 897 OncoTree nodes `{cancerTypeId, name, shortName, parent}`; root `tissue`; `mixed` → parent `other`. |32| `GET /api/studies/{studyId}/molecular-profiles` | e.g. `msk_impact_2017_mutations` (`molecularAlterationType MUTATION_EXTENDED`, datatype MAF), `_cna`, `_structural_variants`. |33| `GET /api/sample-lists/{studyId}_sequenced` | `{category all_cases_with_mutation_data, sampleCount, sampleIds[]}` — `msk_impact_2017_sequenced` 10,945 = `sequencedSampleCount`; `acc_tcga_sequenced` 90 = `sequencedSampleCount`. The study-level count is therefore used as the denominator source without downloading sample ids (the SUMMARY projection of `/studies/{id}/sample-lists` has no `sampleCount`). |34| `POST /api/mutated-genes/fetch` body `{"studyIds":["msk_impact_2017"]}` (StudyViewFilter) | Array of `AlterationCountByGene`: `hugoGeneSymbol`, `entrezGeneId`, **`numberOfAlteredCases`** (distinct samples with ≥ 1 mutation in the gene), **`numberOfProfiledCases`** (samples profiled for that gene — gene-panel aware: MSK-IMPACT genes on IMPACT341+410 → 10,945; genes only on IMPACT410 → 8,136; 4 genes → 0), `totalCount` (mutations), `matchingGenePanelIds`. TP53 4,538/10,945, KRAS 1,643, TERT 1,460, PIK3CA 1,355, APC 1,121 — identical to the study summary "Mutated Genes" table. ~1 s for 10,945 samples. |35| Alternatives inspected | `/api/molecular-profiles/{id}/mutations/fetch` (per-mutation rows, `pageSize` ≤ 10,000,000 — hundreds of thousands of rows for large studies) and `/api/molecular-profiles/{id}/mutations?sampleListId=…&entrezGeneId=…` (one gene at a time) exist but are far more expensive; `/api/mutation-counts-by-gene` does not exist; `/api/studies/{id}/significantly-mutated-genes` is MutSig only. `/api/mutated-genes/fetch` is the cheapest endpoint returning per-gene counts with a denominator. |3637Health check: `GET /api/studies?pageSize=1&projection=SUMMARY` → healthy when one study is returned.3839## Ingestion40411. `/studies` (DETAILED, paged) + `/cancer-types`. Anomaly guard: < 100 studies → refuse to persist. `datasetVersion = cbioportal-<latest importDate>`.422. Studies are processed in `studyId` order; `ctx.cursor.lastStudyId` is saved after each study (restartable, time budget honoured between studies); a completed pass starts over on the next schedule.433. **Cohort** (`genomic_cohorts`, unique on `source_id + study_id`): `studyId`, `name`, `program` = first token of the name's trailing parenthetical (`MSK-IMPACT Clinical Sequencing Cohort (MSK, Nat Med 2017)` → **MSK**; `… (TCGA, PanCancer Atlas)` → **TCGA**), `primarySites` = OncoTree node just below the `tissue` root (luad → nsclc → lung → **Lung**), `diseaseTypes` = cancer-type name, `caseCount = allSampleCount`, `casesWithSsm = sequencedSampleCount`, `dataRelease = importDate (date)`, `accessLevel open`, `url https://www.cbioportal.org/study/summary?id=<studyId>`, provenance per study (`sourceUrl /api/studies/<id>`, `dataset 'cBioPortal public studies'`, `pmid`, `cohortSize`, `population '<studyId> (<program>)'`, methodology with the study citation). An unchanged study keeps its previous provenance row.444. **Cancer reconciliation**: `resolver.byCode('oncotree', CANCERTYPEID.toUpperCase())` → `EXACT_IDENTIFIER`; fallback `byLabel(cancer-type name)` (match type recorded, and the code stored in `cancer_codes` with that match type); studies filed under a **tissue-level node** (direct child of the OncoTree root: "Breast", "Prostate", "Bladder/Urinary Tract", "Soft Tissue", "Bowel"…) are rewritten into site-level malignancy labels (`tissueLevelCandidates`: *Breast → Breast Cancer / Malignant Breast Neoplasm / Breast Carcinoma*, *Soft Tissue → Soft Tissue Sarcoma*, *Bowel → Colorectal Cancer*) and recorded as `CURATED_BROADER`; `cancerTypeId = mixed` → `cancerId null`, `cancerMatchType 'UNRESOLVED'` (pan-cancer, not queued — by design); other misses → `unresolved_labels` with the OncoTree lineage.455. **Publication stub** for `pmid` (`publications`, `publicationTypes ['stub']`, title = citation) + `publication_entity_edges` to the cohort's cancer (`method cbioportal_study`) — enriched later by the PubMed connector.466. **Frequencies** (`cancer_gene_frequencies`, unique on `cohort_id + gene_symbol + alteration_type`) for studies with `sequencedSampleCount > 0` **and** a `MUTATION_EXTENDED` profile (`/molecular-profiles` and `/mutated-genes/fetch` are requested concurrently — concurrency 2): genes ranked by `numberOfAlteredCases` (desc, symbol asc), **top 200 per study** (`TOP_GENES_PER_STUDY`; WES studies return 7,000–18,000 genes), genes with `numberOfProfiledCases = 0` dropped. `alterationType 'ssm'`, `casesAffected = numberOfAlteredCases`, `casesProfiled = numberOfProfiledCases` (gene-panel aware; equals the `<studyId>_sequenced` count for exome studies), `frequency` validated by `validateFrequency()` (rejected rows counted, never stored), `rank`, `dataRelease`, `geneId` via the shared `GeneCache` (Entrez id filled when empty), `cancerId` of the cohort. Provenance per study (`sourceUrl /api/mutated-genes/fetch`, `dataset 'cBioPortal mutated genes by study'`, `pmid`, `cohortSize = sequencedSampleCount`, methodology: *distinct samples with ≥ 1 mutation in gene / samples profiled for the gene (numberOfProfiledCases; = samples in `<studyId>_sequenced` for whole-exome studies)*, profile id, top-N note). Source record `mutated_genes` per study with the ranked list.477. Studies without a mutation profile are cohorts without frequencies (`noMutationProfile` in the run summary); no per-mutation paging is ever needed, so no sample cap applies.4849TCGA cohorts also present in GDC are kept as **separate cohorts** (different processing: Firehose Legacy vs PanCancer Atlas vs GDC harmonized) and flagged by `program = TCGA`; they are never pooled with GDC frequencies.5051`dry_run` fetches studies + cancer types and the top-5 mutated genes of `acc_tcga` without writing.5253## Observed run (cancerindex_b, 2026-09-08 — OncoTree + NCIt + HGNC loaded)5455| | |56|---|---|57| Smoke (`--max-records 10`) | 5 studies (acbc_mskcc_2015 … acc_tcga_pan_can_atlas_2018), 12 requests, 5 s, all `EXACT_IDENTIFIER` |58| Full pass (`--max-minutes 20`) | **545 studies / 545 cohorts** in 6–8 min, 1,080 requests, 0 HTTP failures, 0 studies rejected |59| Cancer mapping | **489 / 545 (89.7 %)**: 432 `EXACT_IDENTIFIER` (OncoTree code), 57 `CURATED_BROADER` (tissue-level rewrite); 48 `mixed` pan-cancer cohorts unmappable by design (MSK-IMPACT 2017, GENIE-derived pan-cancer sets, mixed_allen_2018…); **98.4 % of non-mixed studies mapped**; 8 unresolved (tissue-level sites without a site-level malignancy alias in the ontology yet) |60| Programs | 139 distinct — MSK 171, TCGA 88, TCGA GDC 33, Broad 15, DFCI 10, CPTAC GDC 10, CPTAC 6…; 17 studies without a parenthetical source (`program null`) |61| Frequencies | **100,444 rows** in 539 cohorts (9,972 distinct genes, all resolved to a `genes` row — 0 minimal genes created after HGNC), 90,521 rows with a `cancerId`; denominators 1 – 54,331; **7 rejected** by `validateFrequency` (bowel_colitis_msk_2022: FAT3 11/1, SMOX 4/1… — portal gene-panel matrix inconsistency, never stored) |62| Publications | 378 PMIDs → publication stubs + `publication_entity_edges` to the cohort's cancer |63| Counters | `genomic_cohorts` ranking eligible 109 cancers (all scopes) / 9 top-level; e.g. Malignant Breast Neoplasm 7 cohorts, Malignant Lung Neoplasm 5 |6465`msk_impact_2017` (10,945 samples, `mixed`, program MSK, PMID 28481359) top 5 — `cases_affected / cases_profiled`: TP53 4,538/10,945 (0.415), KRAS 1,643/10,945 (0.150), TERT 1,460/10,945 (0.133), PIK3CA 1,355/10,945 (0.124), APC 1,121/10,945 (0.102) — identical to the portal's study-summary table; genes only on the 410-gene panel carry the panel-aware denominator 8,136.6667`acc_tcga` (90 sequenced) top 3: ZFPM1 47/90, MUC5B 35/90, GARS1 34/90 (Firehose Legacy calls; the PanCancer Atlas cohort `acc_tcga_pan_can_atlas_2018` of the same tumours gives TP53 18/91, CTNNB1 14/91 — kept as separate cohorts, as is GDC's TCGA-ACC).6869Idempotent: a second full pass (run twice today after rule changes, `--reset-cursor`) updated cohorts and frequencies in place, reused cohort ids and provenance rows for unchanged studies, and created no duplicate genes or publications.7071## Limitations / notes7273- Mutation counts follow the portal's default study-view filters (all mutation types the portal shows in "Mutated Genes"; germline/somatic and driver/VUS filters untouched); counts are per **sample**, not per patient.74- Only `alteration_type = 'ssm'`; CNA (`/api/cna-genes/fetch`) and structural-variant counts are not ingested yet.75- Top 200 genes per study; long-tail genes are not stored (the source record keeps the ranked list only up to the cap).76- `mixed` studies (48, incl. MSK-IMPACT 2017) have no cancer; their frequencies are cohort-level only and never roll up to a cancer.77- Some studies restrict commercial use (noted in their description on the portal) — review before any commercial redistribution of those cohorts; `commercialUse: restricted` at the manifest level reflects this.78