Original product specification supplied by the project owner on 2026-09-08. The repository CLAUDE.md is the condensed operational version.
CLAUDE.md — CancerIndex.io
0. PROJECT IDENTITY
Project: CancerIndex.io
Product type: Global cancer intelligence platform, database, search engine, ranking system, knowledge graph, analytics platform, and AI research interface.
Primary language: English.
Domain: cancerindex.io
Mission: Build the most comprehensive, structured, searchable, continuously updated and transparently sourced cancer intelligence database possible.
CancerIndex.io must aim to become:
The global index of cancer.
Think of the product as a combination of:
- Bloomberg Terminal for oncology
- IMDb/Wikipedia-style entity coverage
- Our World in Data for cancer epidemiology
- ClinicalTrials.gov explorer
- cBioPortal genomic exploration
- PubMed intelligence layer
- drug/biomarker intelligence platform
- cancer knowledge graph
- transparent ranking engine
- research assistant
- cancer statistics terminal
The product must not merely list the 20–50 most common cancers.
CancerIndex must attempt to model every identifiable malignant disease, histological subtype, molecular subtype, rare cancer, hematologic malignancy, pediatric malignancy, tumor family, anatomical site and recognized cancer entity that can be responsibly mapped from authoritative taxonomies.
This includes cancers that may affect only a tiny number of people annually.
Do not hardcode a simplistic cancer list.
Build an evolving oncology ontology.
1. THE CORE PRINCIPLE
CancerIndex must answer questions such as:
- What cancers exist?
- How common is each cancer?
- Which cancers kill the most people?
- Which cancers have the highest mortality rate?
- Which cancers have the poorest survival?
- Which cancers are increasing fastest?
- Which cancers affect younger populations?
- Which cancers have the most treatments?
- Which cancers have the fewest treatments?
- Which cancers receive the most research?
- Which cancers receive the least research relative to burden?
- Which cancers currently have the most clinical trials?
- Which cancers have the most actionable mutations?
- Which cancers have the largest unmet need?
- Which cancers are associated with which genes?
- What variants occur in those genes?
- Which therapies target those abnormalities?
- Which drugs are approved?
- Which drugs are experimental?
- Which biomarkers predict response?
- Which trials are recruiting?
- Which publications support each relationship?
- Which countries have the highest incidence?
- How has incidence changed through time?
- How does survival differ by stage?
- How does age influence incidence?
- How does sex influence incidence?
- Which risk factors are associated with each malignancy?
- Which cancers can be screened for?
- Which cancers can be prevented?
- Which cancers are becoming more survivable?
- Where are the largest gaps in oncology research?
Everything must be explorable from the web interface and eventually through an API.
2. ABSOLUTE RULE: PROVENANCE FIRST
CancerIndex must never display an important scientific number without knowing where it originated.
Every imported fact should support metadata such as:
interface Provenance {
sourceId: string
sourceName: string
sourceRecordId?: string
sourceUrl?: string
dataset?: string
datasetVersion?: string
publicationId?: string
pmid?: string
doi?: string
retrievedAt: string
publishedAt?: string
updatedAt?: string
geography?: string
population?: string
cohortSize?: number
methodology?: string
evidenceType:
| "registry"
| "clinical_trial"
| "meta_analysis"
| "systematic_review"
| "cohort"
| "case_control"
| "case_series"
| "case_report"
| "preclinical"
| "regulatory"
| "guideline"
| "expert_curation"
| "database"
| "computed"
accessLevel:
| "open"
| "registration_required"
| "controlled"
| "licensed"
confidence?: number
license?: string
}Every derived statistic must also be reproducible.
Example:
{
"metric": "mortality_to_incidence_ratio",
"value": 0.73,
"computed": true,
"formula_version": "ci-mir-v1",
"inputs": [
"CINDEX-METRIC-102991",
"CINDEX-METRIC-102992"
]
}Never destroy raw source information during normalization.
Architecture:
RAW
↓
NORMALIZED
↓
CANONICAL
↓
DERIVED
↓
RANKED
↓
AI SYNTHESISThese layers must remain separable.
3. SCIENTIFIC SAFETY
CancerIndex is a research/information platform.
It is NOT:
- a physician
- a diagnostic system
- a treatment prescriber
- a replacement for professional medical care
Never produce treatment recommendations based only on an AI-generated inference.
Clearly distinguish:
OBSERVED DATA
PUBLISHED EVIDENCE
CURATED EVIDENCE
REGULATORY STATUS
CLINICAL GUIDELINE
COMPUTED METRIC
AI-GENERATED SYNTHESISNever silently merge these categories.
4. COVERAGE GOAL — EVERY CANCER WE CAN MODEL
CancerIndex needs a hierarchical disease model.
A simple table:
lung cancer
breast cancer
brain cancer
...is unacceptable.
Cancer must be represented through a hierarchy.
Example:
Cancer
└── Solid Tumor
└── Lung Cancer
└── Non-Small Cell Lung Cancer
└── Lung Adenocarcinoma
├── EGFR-mutated LUAD
├── KRAS-mutated LUAD
├── ALK-positive LUAD
├── ROS1-positive LUAD
└── RET-positive LUADAnother:
Cancer
└── Hematologic Malignancy
└── Leukemia
└── Acute Leukemia
└── Acute Myeloid Leukemia
├── AML with NPM1 mutation
├── AML with CEBPA mutation
├── APL
└── therapy-related AMLAnother:
Cancer
└── CNS Tumor
└── Glioma
└── Diffuse Glioma
└── GlioblastomaThe hierarchy needs multiple dimensions.
Do NOT force every entity into only one parent tree.
Support:
anatomical hierarchy
histological hierarchy
molecular hierarchy
WHO-style disease classification
ICD hierarchy
ICD-O morphology
ICD-O topography
NCI Thesaurus
Disease Ontology
OncoTree
SEER classification
pediatric classification
hematologic classification5. CANONICAL CANCER ENTITY
Create:
CancerEntityExample schema:
interface CancerEntity {
id: string
slug: string
canonicalName: string
shortName?: string
aliases: string[]
abbreviations: string[]
entityType:
| "cancer"
| "cancer_family"
| "histology"
| "subtype"
| "molecular_subtype"
| "hematologic_malignancy"
| "precursor_condition"
| "other"
parentIds: string[]
childIds: string[]
anatomyIds: string[]
histologyIds: string[]
ncitCodes: string[]
icd10Codes: string[]
icdoTopographyCodes: string[]
icdoMorphologyCodes: string[]
doidCodes: string[]
oncotreeCodes: string[]
umlsCodes: string[]
meshIds: string[]
mondoIds: string[]
malignant: boolean
solidTumor: boolean
hematologic: boolean
pediatricRelevant: boolean
rareCancer: boolean
description?: string
epidemiology?: CancerEpidemiologySummary
survival?: CancerSurvivalSummary
geneAssociations?: string[]
biomarkerAssociations?: string[]
drugAssociations?: string[]
trialAssociations?: string[]
ranking?: CancerRanking
provenanceIds: string[]
createdAt: string
updatedAt: string
}6. CANCERINDEX IDENTIFIERS
CancerIndex needs its own stable ID namespace.
Examples:
CI-CAN-00000001
CI-CAN-00000002
CI-GENE-00000001
CI-VAR-00000001
CI-DRUG-00000001
CI-TRIAL-00000001
CI-PUB-00000001
CI-BIO-00000001
CI-STUDY-00000001
CI-METRIC-00000001
CI-SOURCE-00000001
CI-ORG-00000001
CI-TRT-00000001Never expose database auto-increment integers as public identifiers.
IDs must remain stable forever.
7. ENTITY UNIVERSE
CancerIndex should eventually contain first-class entities for:
Cancer
Cancer
Cancer subtype
Histology
Molecular subtype
Tumor family
Precancerous condition where relevant
Metastatic disease stateAnatomy
Organ
Tissue
Anatomical site
Primary site
Metastatic siteGenes
Gene
Transcript
Protein
Pathway
Gene familyGenetic alterations
SNV
MNV
Insertion
Deletion
Indel
Fusion
Rearrangement
Amplification
Deletion/CNA
Loss of heterozygosity
Promoter mutation
Splice alteration
Expression change
Epigenetic alteration
Structural variantBiomarkers
Gene mutation
Protein expression
Hormone receptor
PD-L1
MSI
TMB
HRD
ctDNA
methylation
gene signature
expression signature
cell surface marker
immune markerDrugs
small molecule
monoclonal antibody
ADC
bispecific antibody
CAR-T
cell therapy
gene therapy
cancer vaccine
radiopharmaceutical
chemotherapy
hormonal therapy
immunotherapy
targeted therapyTreatment concepts
drug
drug combination
surgery
radiotherapy
brachytherapy
proton therapy
transplantation
cell therapy
watchful waiting
active surveillanceClinical research
clinical trial
trial arm
intervention
cohort
endpoint
study
publication
investigator
institution
sponsorPopulation
country
territory
state/province
region
registry
age group
sex
calendar year8. THE CONNECTOR PHILOSOPHY
CancerIndex must be built around connectors.
Do not manually populate the platform except for explicitly curated metadata.
Every external system must have an independent connector.
Structure:
/connectors
/nci
/gdc
/seer
/iarc
/clinicaltrials
/pubmed
/clinvar
/cbioportal
/civic
/hgnc
/ensembl
/chembl
/opentargets
/dgidb
/fda
...Every connector implements something similar to:
interface Connector {
id: string
name: string
discover(): Promise<void>
fetch(): Promise<void>
normalize(): Promise<void>
reconcile(): Promise<void>
validate(): Promise<void>
persist(): Promise<void>
healthCheck(): Promise<ConnectorHealth>
getCursor(): Promise<ConnectorCursor>
setCursor(cursor: ConnectorCursor): Promise<void>
}9. CONNECTOR MANIFEST
Every connector needs:
id:
name:
organization:
category:
access:
type: api|bulk|rss|ftp|graphql|rest|scrape|manual
auth: none|api_key|oauth|account|controlled
license:
terms_reviewed:
commercial_use_status:
update_frequency:
expected_latency:
supports_incremental_sync:
entities:
metrics:
rate_limits:
retry_policy:
raw_retention:
schema_version:
documentation_verified_at:
owner:
status:10. CONNECTOR PRIORITY TIERS
TIER 0 — FOUNDATIONAL
These must be implemented first.
10.1 NCI Enterprise Vocabulary Services
Purpose:
- cancer terminology
- canonical disease names
- disease aliases
- drug concepts
- biomarkers
- terminology mappings
- controlled oncology concepts
Use NCI terminology as one of the anchors of the CancerIndex normalization system.
Store NCI identifiers on entities.
10.2 NCI Genomic Data Commons — GDC
Use for:
- TCGA
- TARGET
- CPTAC where available through the platform
- case metadata
- tumor metadata
- genomic files
- mutation information
- copy-number information
- expression
- molecular features
- survival-related analyses
- cancer cohorts
Connector must support:
projects
cases
files
annotations
genes
mutations
CNV
metadata
manifestsDo not ingest controlled-access patient-identifiable/low-level data without the proper authorization model.
Prioritize open, aggregated and de-identified data.
10.3 SEER
Purpose:
- US cancer incidence
- mortality where available/appropriate
- survival
- age
- sex
- race/ethnicity where datasets permit
- staging
- disease classification
- registry geography
- trends
Use SEER for American cancer burden and survival analytics.
Never represent a SEER population estimate as a global estimate.
10.4 IARC Global Cancer Observatory / GLOBOCAN
This should be the major global epidemiology layer.
Capture when permitted:
incidence
mortality
prevalence
age-standardized rates
sex
country
region
cancer type
yearBuild CancerIndex country and global rankings from this layer.
Terms and redistribution rights MUST be reviewed before automated bulk ingestion.
Do not assume that because data are publicly viewable they can automatically be republished wholesale.
10.5 ClinicalTrials.gov
Build an extremely robust ClinicalTrials.gov connector.
Capture:
NCT ID
official title
brief title
study type
phase
status
conditions
interventions
arms
sponsor
collaborators
eligibility
sex
age
enrollment
locations
countries
investigators
primary outcomes
secondary outcomes
start date
completion date
study results
references
last updateCancerIndex must map free-text conditions to canonical cancer IDs.
CancerIndex must map:
trial → cancer
trial → drug
trial → biomarker
trial → gene
trial → institution
trial → countryIncrementally synchronize changed records.
10.6 PubMed
Massive literature connector.
Capture:
PMID
title
abstract
authors
affiliations
journal
publication date
publication types
MeSH
DOI
references where accessible
retractions/correctionsBuild mappings:
publication → cancer
publication → gene
publication → variant
publication → biomarker
publication → drug
publication → trialDo not make LLM entity extraction authoritative.
LLM extraction creates candidate relationships.
Those candidates must be labeled accordingly until validated.
10.7 ClinVar
Use for:
variants
clinical significance
conditions
review status
submitter information
variation IDs
HGVS
genes
citations
drug responseMap cancer-associated ClinVar records into the graph.
10.8 cBioPortal
Use for cancer cohort/genomics exploration.
Import where licensing permits:
studies
patients
samples
mutations
CNA
expression
clinical attributes
survival
molecular profilesCancerIndex should preserve original study IDs.
10.9 CIViC
Extremely important for curated clinical interpretation of cancer variants.
Map:
gene
variant
molecular profile
disease
therapy
evidence item
assertion
publication
evidence level
evidence direction
clinical significanceDo not flatten CIViC evidence into a binary:
works / doesn't workPreserve its structured evidence.
11. TIER 1 — MOLECULAR INTELLIGENCE
Implement these after foundational ingest.
HGNC
Canonical human gene nomenclature.
Capture:
HGNC ID
approved symbol
approved name
aliases
previous symbols
chromosomal location
cross referencesHGNC should be authoritative for canonical human gene symbol reconciliation.
Ensembl
Capture:
genes
transcripts
variants
coordinates
assemblies
regulatory information
homology where usefulAlways retain genome assembly.
Never store a coordinate without:
assembly
chromosome
position
reference
alternateNCBI Gene
Use as additional cross-reference and annotation source.
dbSNP
Variant identifiers and genomic cross references.
dbVar
Structural variation.
Sequence Ontology
Normalize variant types.
Gene Ontology
Functional annotation.
UniProt
Protein entities and protein annotation.
Reactome
Pathways.
Build:
gene → pathway
protein → pathway
drug target → pathway
cancer → dysregulated pathwayWikiPathways
Secondary pathway source.
Protein Data Bank
Connect cancer proteins and drug targets to experimental structures.
AlphaFold Protein Structure Database
Optional structural biology layer.
Do not imply predicted structure equals experimental structure.
12. TIER 2 — DRUG INTELLIGENCE
ChEMBL
Capture:
molecules
mechanisms
targets
assays
activities
indications
development phaseOpen Targets
Use as a disease-target-drug evidence layer.
Map:
target ↔ disease
drug ↔ target
evidence source
association scoreNever convert another database's association score directly into a CancerIndex evidence score without documenting transformation.
DGIdb
Drug-gene interactions.
Preserve contributing source information.
DrugBank
Potential connector.
Important: licensing must be verified before implementation or redistribution.
Do not scrape or reproduce licensed content without permission.
PubChem
Use for:
compound IDs
structures
synonyms
chemical identifiersDrugCentral
Candidate drug information source.
Verify current terms and downloadable datasets.
DailyMed
Structured FDA label information.
Potential fields:
drug label
indications
contraindications
warnings
dose language
adverse reactions
manufacturer
label versionNever paraphrase a drug label and then present the paraphrase as the legal label.
OpenFDA
Use where useful for structured FDA data.
Potential:
labels
adverse event aggregates
drug metadataAdverse-event reports must include strong caveats.
Spontaneous reporting cannot be treated as incidence or causal proof.
13. TIER 3 — REGULATORY INTELLIGENCE
Build country-aware regulatory status.
Never use:
approved = truealone.
Use:
interface DrugApproval {
drugId: string
cancerId?: string
biomarkerIds: string[]
jurisdiction:
| "US"
| "CA"
| "EU"
| "UK"
| "AU"
| "JP"
| "OTHER"
authority:
| "FDA"
| "Health Canada"
| "EMA"
| "MHRA"
| "TGA"
| "PMDA"
| string
indication: string
lineOfTherapy?: string
diseaseStage?: string
approvalType?: string
accelerated?: boolean
conditional?: boolean
approvalDate?: string
withdrawalDate?: string
status:
| "approved"
| "conditional"
| "accelerated"
| "withdrawn"
| "superseded"
sourceId: string
}FDA Oncology
Connect:
- oncology approval announcements
- Oncology Center of Excellence
- Project Confirm
- accelerated approvals
- withdrawn accelerated approvals
- verified clinical benefit
- Project Orbis
- labeling
- regulatory reviews
CancerIndex should have an:
Oncology Approval Timeline
Health Canada
CancerIndex is global; Canada must be properly represented.
Potential datasets:
- Drug Product Database
- Notice of Compliance
- Summary Basis of Decision
- Product Monographs
- Project Orbis-related approvals
Review redistribution and API availability before implementation.
EMA
Capture:
European Public Assessment Reports
indications
marketing authorization
authorization dates
withdrawals
safety changesMHRA
UK regulatory layer.
TGA
Australia regulatory layer.
PMDA
Japan regulatory layer.
Swissmedic
Swiss layer.
14. TIER 4 — GLOBAL EPIDEMIOLOGY
Additional country sources should augment IARC rather than blindly override it.
Possible connectors:
WHO
Global health statistics and relevant cancer-related datasets.
CDC
US cancer statistics where useful.
Statistics Canada
Canadian mortality/population data.
Canadian Cancer Statistics
Evaluate licensing and machine-readable availability.
Canadian Cancer Registry
Integrate permitted aggregated data where accessible.
European Cancer Information System
European epidemiology.
EUROCARE
European survival research where permitted.
National cancer registries
Build country-specific connectors where high-quality public data exist.
Examples:
UK
Australia
New Zealand
Nordic countries
France
Germany
Netherlands
Japan
South Korea
Singapore
Canada
United StatesDo NOT mix incompatible epidemiological definitions without harmonization.
15. TIER 5 — LITERATURE
PubMed
Primary.
Europe PMC
Use as a complementary literature graph.
Potential:
abstracts
full-text availability
citations
references
grants
preprintsCrossref
DOI and publication metadata.
OpenAlex
Useful for:
citation graph
institutions
authors
topics
research trendsVerify licensing/current API conditions.
Semantic Scholar
Potential secondary citation/AI-literature layer.
Review API conditions.
bioRxiv
Preprints.
medRxiv
Preprints.
Always label preprints prominently.
Never rank a preprint as equivalent to peer-reviewed evidence.
16. TIER 6 — CLINICAL GUIDELINES
Potential sources:
NCI
ASCO
ESMO
NCCN
Cancer Care Ontario
NICE
other national oncology organizationsCRITICAL:
Guideline copyright and licensing vary substantially.
CancerIndex must NOT automatically scrape and reproduce paid/copyrighted guidelines.
For restricted sources:
store only permitted:
citation
title
publication date
organization
external reference
metadataunless licensing permits more.
17. TIER 7 — PRECISION ONCOLOGY
Potential integrations:
CIViC
ClinVar
OncoKB
Cancer Genome Interpreter
JAX-CKB
MolecularMatch
My Cancer GenomeBut:
Licensing must be checked individually.
Never assume commercial reuse.
Build the CancerIndex precision oncology layer first from sources with clear reuse rights.
18. TIER 8 — CANCER CELL LINES AND PRECLINICAL DATA
Potential sources:
DepMap
cell lines
gene dependencies
CRISPR screens
drug sensitivity
molecular featuresCancer Cell Line Encyclopedia
Integrate where licensing permits.
GDSC
Genomics of Drug Sensitivity in Cancer.
Cell Model Passports
Cancer model information.
PDX resources
Patient-derived xenograft data where publicly available.
Keep:
PRECLINICALclearly separated from human clinical evidence.
19. TIER 9 — IMMUNO-ONCOLOGY
Model:
immune checkpoints
immune cell populations
neoantigens
HLA
PD-1
PD-L1
CTLA-4
LAG-3
TIGIT
TIM-3
TMB
MSI
immune gene signaturesSources can include:
GDC
cBioPortal
CIViC
clinical trials
publications
Open Targets20. TIER 10 — PEDIATRIC ONCOLOGY
CancerIndex must NOT treat pediatric cancers as simply adult cancers in younger people.
Build dedicated pediatric taxonomy.
Sources could include:
TARGET
NCI
SEER
IARC pediatric resources
St. Jude public resources
pediatric clinical trials
literatureAdd:
age at diagnosis
pediatric incidence
AYA incidence
survival
molecular subtype
treatment landscape
late effects evidence21. TIER 11 — RARE CANCERS
Rare cancers are a core differentiator.
CancerIndex should attempt to index cancers even when:
incidence < 1 / 100,000Do not hide them because data are sparse.
Create:
Rare Cancer Explorer
Metrics:
estimated incidence
number of known cases/cohorts
number of publications
number of clinical trials
number of approved therapies
number of targeted therapies
available genomic studies
research activityData scarcity itself should be displayed.
22. CONNECTOR FALLBACK SYSTEM
Preferred connector hierarchy:
1. official API
2. official bulk download
3. official structured feed
4. official database export
5. official static dataset
6. compliant website extraction
7. publication extraction
8. manual curator reviewDo NOT start by scraping if an API exists.
23. FIRECRAWL + SCRAPFLY
CancerIndex may use Firecrawl and Scrapfly for sources without suitable APIs.
Architecture:
official API
↓ unavailable
official bulk
↓ unavailable
Firecrawl
↓ blocked / inadequate
ScrapflyFirecrawl is primarily useful for:
documentation
regulatory pages
research institution pages
structured public pages
public tables
release notesScrapfly should be a fallback for technically difficult public pages when use is permitted.
Do NOT use anti-bot tooling to bypass:
- authentication
- paywalls
- explicit access restrictions
- licensing controls
- patient privacy protections
- controlled genomic datasets
Store source terms/compliance status per connector.
24. CONNECTOR OBSERVABILITY
Every connector receives an admin dashboard.
Display:
status
last successful sync
last attempt
duration
records fetched
records created
records updated
records rejected
schema drift
HTTP failures
rate limit events
validation failures
freshnessExample:
GDC HEALTHY 11 min ago
ClinicalTrials HEALTHY 4 min ago
PubMed HEALTHY 8 min ago
SEER HEALTHY 2 hr ago
FDA DEGRADED 37 min ago
IARC REVIEW license check25. SCHEMA DRIFT DETECTION
External APIs change.
Every connector must detect:
new fields
removed fields
changed enum values
changed types
unexpected nullability
pagination behavior changes
authentication changesWhen drift occurs:
DO NOT silently discard data.Alert the administrator.
26. RAW DATA LAKE
Every source payload should be retained when licensing allows.
Use object storage:
/raw/{source}/{date}/{entity}/{id}.jsonor compressed batch files.
Benefits:
- auditability
- reproducibility
- reprocessing
- parser upgrades
- debugging
- historical snapshots
27. CANONICAL DATA MODEL
Core relational tables:
cancers
cancer_aliases
cancer_hierarchy
cancer_codes
anatomical_sites
genes
gene_aliases
proteins
transcripts
variants
variant_coordinates
variant_aliases
biomarkers
drugs
drug_aliases
drug_targets
drug_indications
drug_approvals
treatments
treatment_regimens
clinical_trials
trial_conditions
trial_interventions
trial_locations
trial_outcomes
trial_eligibility
publications
authors
institutions
studies
cohorts
epidemiology_observations
survival_observations
cancer_gene_edges
cancer_variant_edges
cancer_biomarker_edges
cancer_drug_edges
drug_gene_edges
drug_variant_edges
trial_cancer_edges
publication_entity_edges
sources
source_records
provenance
rankings
ranking_snapshots28. KNOWLEDGE GRAPH
CancerIndex must be graph-native conceptually, even if PostgreSQL remains the primary transactional database.
Graph:
Cancer
├── HAS_SUBTYPE → Cancer
├── OCCURS_IN → Anatomy
├── ASSOCIATED_WITH → Gene
├── HAS_VARIANT → Variant
├── HAS_BIOMARKER → Biomarker
├── TREATED_BY → Drug
├── STUDIED_IN → Trial
├── DESCRIBED_BY → Publication
└── OBSERVED_IN → Cohort
Gene
├── HAS_VARIANT → Variant
├── ENCODES → Protein
├── MEMBER_OF → Pathway
└── TARGETED_BY → Drug
Variant
├── OCCURS_IN → Cancer
├── PREDICTS_RESPONSE_TO → Drug
├── CONFERS_RESISTANCE_TO → Drug
└── SUPPORTED_BY → Evidence
Drug
├── TARGETS → Gene
├── APPROVED_FOR → Cancer
├── INVESTIGATED_FOR → Cancer
└── USED_IN → TrialRelationships need provenance.
29. EDGE MODEL
Never store:
EGFR mutation → osimertinibwithout context.
Use:
interface KnowledgeEdge {
id: string
sourceEntityId: string
targetEntityId: string
relationshipType: string
cancerContextIds?: string[]
predictive?: boolean
prognostic?: boolean
diagnostic?: boolean
predisposing?: boolean
direction?:
| "supports"
| "resistance"
| "sensitivity"
| "neutral"
| "unknown"
evidenceLevel?: string
evidenceScore?: number
provenanceIds: string[]
firstSeenAt: string
lastSeenAt: string
}30. CANCER RANKING ENGINE
This is one of the signature features.
CancerIndex must rank every eligible cancer across MANY metrics.
There must never be one unexplained “danger ranking.”
31. RANKING DIMENSIONS
Every cancer can potentially have:
Burden
global incidence count
global mortality count
global prevalence
age-standardized incidence
age-standardized mortality
DALYs if source available
YLL if source availableLethality
mortality / incidence ratio
1-year survival
5-year survival
10-year survival
stage IV survival
median OS where meaningfulTrend
incidence CAGR
mortality CAGR
survival improvement
age-adjusted incidence trendRarity
global incidence rank
incidence per 100k
estimated annual patientsTreatment Landscape
number of approved therapies
number of targeted therapies
number of immunotherapies
number of biomarker-directed therapies
number of treatment classesClinical Research
active trials
recruiting trials
phase I trials
phase II trials
phase III trials
interventional trial count
trial enrollmentResearch Activity
publications last 12 months
publications last 5 years
publication growth
citations
research institutionsMolecular Knowledge
known recurrent genes
actionable variants
validated biomarkers
genomic studies
sequenced cohortsUnmet Need
Derived carefully from:
mortality burden
poor survival
few approved therapies
few active trials
low research activity
lack of actionable biomarkers32. RANK EVERY CANCER BY DEFAULT
Cancer detail pages should display:
Global incidence rank
Global mortality rank
5-year survival rank
Lethality rank
Research activity rank
Clinical trial rank
Treatment availability rank
Genomic knowledge rank
Unmet need rank
CancerIndex composite rankExample:
Pancreatic Adenocarcinoma
Mortality burden #7
Incidence burden #14
Lethality #3
Five-year survival #4 poorest
Research activity #11
Active trials #15
Treatment options #82
Unmet need #5
CancerIndex Impact #8These are examples only.
Never hardcode examples as actual statistics.
33. RANKING SCOPE
Rankings need scope.
Example:
WORLD
CANADA
UNITED STATES
EUROPE
QUEBEC
MALE
FEMALE
CHILDREN
AYA
AGE 65+
2024
2025
historicalA rank is meaningless without a population and reference year.
Schema:
interface Ranking {
cancerId: string
metricId: string
rank: number
eligibleEntities: number
percentile: number
geography: string
sex?: string
ageGroup?: string
year?: number
value: number
unit: string
sourceIds: string[]
formulaVersion?: string
generatedAt: string
}34. COMPOSITE CANCERINDEX SCORE
Create an optional composite metric.
Do NOT present it as biological truth.
Possible conceptual model:
CancerIndex Impact Score0–100.
Possible components:
25% mortality burden
20% lethality
15% incidence burden
15% unmet treatment need
10% adverse trend
10% research deficit
5% clinical trial deficitWeights must be:
- visible
- versioned
- configurable
- documented
Example:
CancerIndex Impact Score v1.0Display:
Score: 87.4 / 100
Rank: 6 / 412and a breakdown.
Never show only 87.4.
Show:
Mortality burden 93
Lethality 97
Incidence 78
Treatment deficit 84
Research deficit 63
Trend 7135. UNCERTAINTY
Rankings must account for uncertainty.
A rare cancer may have:
n = 22Do not rank survival estimates derived from tiny datasets as equivalent to huge registry datasets.
Store:
sample size
confidence interval
standard error
estimate method
source quality
data completenessOptionally display:
Ranking confidence
HIGH
MEDIUM
LOW
INSUFFICIENT DATA36. DATA COMPLETENESS SCORE
Every cancer gets a completeness profile.
Example:
Epidemiology 92%
Survival 81%
Genomics 97%
Trials 100%
Therapies 93%
Biomarkers 88%
Literature 100%
Pathology 76%This is separate from scientific confidence.
37. RESEARCH GAP INDEX
Create a major CancerIndex innovation:
Research Gap Index
Question:
Which cancers have a large burden but disproportionately little research?
Possible formula:
burden percentile
÷
research activity percentileMore sophisticated version:
expected research activity =
f(
incidence,
mortality,
years_of_life_lost,
lethality
)
research gap =
expected activity - observed activityDisplay:
Most Under-Researched Cancers
This could be extremely compelling.
38. TRIAL GAP INDEX
Another ranking:
disease burden
vs
active interventional trialsIdentify:
High-burden cancers with few active trials.
39. TREATMENT GAP INDEX
Rank cancers based on:
mortality
survival
approved drug count
effective targeted treatment count
biomarker-directed therapiesAgain, label as CancerIndex-derived metric.
40. PROGRESS INDEX
Create:
Cancer Progress Index
Track over 5/10/20 years:
mortality improvement
survival improvement
treatment approvals
trial growth
biomarker growth
research growthShow:
Most rapidly improving cancers
Least improving cancers41. MOMENTUM INDEX
Short-term research momentum.
Components:
new trials
new publications
new drugs
new FDA approvals
new biomarkers
new genomic studiesWindows:
30 days
90 days
1 year
5 years42. CANCER ENTITY DETAIL PAGE
Route:
/cancer/{slug}Example layout:
┌─────────────────────────────────────────────┐
│ Pancreatic Ductal Adenocarcinoma │
│ PDAC │
│ CI-CAN-0000342 │
└─────────────────────────────────────────────┘
CancerIndex Score
89.2
Global Rank
#5
Tabs:
Overview
Statistics
Survival
Stages
Genomics
Genes
Variants
Biomarkers
Treatments
Drugs
Trials
Research
Publications
Risk Factors
Screening
Prevention
Countries
Trends
Sources43. CANCER OVERVIEW HERO
Show:
Global annual cases
Global annual deaths
5-year survival
median diagnosis age
male/female distribution
Impact rank
Mortality rank
Lethality rank
Research rank
Unmet need rankNever display unsupported values.
44. CANCER SUMMARY
AI-generated summary should contain:
What it is
Where it originates
Major subtypes
Epidemiology
Typical molecular features
Major treatment modalities
Current research landscapeEvery paragraph needs citations.
AI summaries must be cached with:
model
prompt version
source snapshot
generation date45. GLOBAL CANCER RANKING PAGE
Route:
/rankingsFilters:
metric
year
country
region
sex
age
cancer category
minimum cases
data confidenceColumns:
Rank
Cancer
Score/value
Cases
Deaths
Mortality/incidence
5-year survival
Active trials
Publications
Trend46. RANKING PRESETS
Routes or presets:
/rankings/incidence
/rankings/mortality
/rankings/lethality
/rankings/survival
/rankings/research
/rankings/trials
/rankings/treatment-gap
/rankings/research-gap
/rankings/momentum
/rankings/progress
/rankings/rare-cancers47. COUNTRY PAGES
Route:
/country/canada
/country/united-states
/country/franceDisplay:
population
annual cancer cases
annual cancer deaths
ASIR
ASMR
Top cancers by incidence
Top cancers by mortality
male
female
historical trend
age distributionMap visualization.
48. GLOBAL CANCER MAP
Interactive world map.
Filters:
cancer
incidence
mortality
ASR
sex
year
ageClick country → country dashboard.
49. GENE PAGES
Route:
/gene/TP53Display:
gene overview
HGNC identity
chromosome
protein
pathways
cancers
variants
mutation frequencies
biomarkers
therapies
clinical trials
publications50. VARIANT PAGES
Example:
/variant/BRAF-V600EDisplay:
gene
HGVS
protein change
coordinates by assembly
ClinVar
CIViC
cancers
frequencies
drug sensitivity evidence
drug resistance evidence
clinical trials
publicationsSeparate evidence by cancer.
BRAF V600E in one cancer must not automatically inherit evidence from another cancer.
51. BIOMARKER PAGES
Examples:
PD-L1
MSI-H
TMB-high
HER2
HRD
ER
PR
PSMA
ctDNADisplay:
definition
measurement method
cancers
therapies
FDA-approved indications
clinical evidence
trials
publications52. DRUG PAGES
Route:
/drug/osimertinibHero:
Generic name
Brand names
Drug class
Targets
Mechanism
Developer
First approval
Current jurisdictionsTabs:
Overview
Mechanism
Targets
Cancer indications
Biomarkers
Approvals
Clinical trials
Publications
Combinations
Resistance
Safety
Sources53. DRUG COMBINATION ENTITY
Do not treat:
Drug A + Drug Bas two unrelated drugs.
Create:
TreatmentRegimenExamples:
FOLFOX
FOLFIRINOX
R-CHOP
ABVD
drug A + drug B54. CLINICAL TRIAL PAGES
Route:
/trial/NCT...Display:
status
phase
title
cancers
biomarkers
interventions
enrollment
sponsor
locations
eligibility
dates
outcomes
publications
results55. TRIAL MATCH EXPLORER
Research use only.
Filters:
cancer
stage
gene
variant
biomarker
drug
phase
country
recruiting status
age
sexDo not claim a patient is eligible solely from automated filtering.
Use:
Potentially relevant trials — verify full eligibility criteria with the study team.
56. PUBLICATION PAGES
Route:
/publication/{pmid}Show:
title
authors
journal
date
abstract
DOI
publication type
linked cancers
linked genes
linked variants
linked drugs
linked trialsAI:
structured research summaryOnly where legally permitted from available text.
57. RESEARCHER PAGES
Optional later stage:
/researcher/{id}Metrics:
oncology publications
cancers studied
genes studied
clinical trials
citations
institutionsAvoid misleading researcher ranking based on raw citation count alone.
58. INSTITUTION PAGES
Examples:
MD Anderson
Memorial Sloan Kettering
Dana-Farber
Princess Margaret
Mayo Clinic
Gustave RoussyAutomatically derived from:
trials
authors
affiliations
publicationsRank institutions by transparent criteria, not prestige claims.
59. CANCER RESEARCH DASHBOARD
Route:
/researchShow:
publications/year
trials/year
new drugs/year
new targets/year
new biomarkers/year
research funding when reliable data exists60. RESEARCH TRENDS
Detect emerging topics.
Examples:
KRAS G12D
T-cell engagers
ADC
ctDNA
personalized vaccines
radioligand therapy
CAR-T in solid tumorsDo not hardcode trends.
Calculate from publication/trial growth.
61. CANCER NEWS
Potential later connector layer:
FDA
NCI
NIH
major journals
cancer centers
regulators
clinical trial updatesUse original source and publication timestamp.
AI clustering:
multiple reports → one story cluster62. AI — ASK CANCERINDEX
CancerIndex should contain a research assistant.
Route:
/askExample questions:
Which cancers have the highest mortality-to-incidence ratio?
What are the most frequent genomic alterations in LUAD?
Compare KRAS G12C in lung and colorectal cancer.
Which recruiting Phase III trials are testing therapies for pancreatic cancer?
Which rare cancers have the fewest active clinical trials relative to incidence?
What cancers have seen the largest improvement in survival over 20 years?63. AI MUST QUERY STRUCTURED DATA FIRST
Never do:
question
↓
LLM general knowledge
↓
answerDo:
question
↓
intent parser
↓
CancerIndex query plan
↓
SQL / graph / search
↓
source records
↓
LLM synthesis
↓
citations64. AI ANSWER CONTRACT
Every answer returns:
{
"answer": "...",
"entities": [],
"citations": [],
"data_as_of": "...",
"confidence": "...",
"limitations": []
}65. AI CITATIONS
Every important assertion must point to:
source
dataset
publication
or regulatory recordClick citation → source drawer.
Source drawer:
Source
Organization
Dataset
Version
Record
Retrieved
Raw value
Normalized value
Transformation66. AI MODEL PROVIDER ABSTRACTION
Do not couple the application to one LLM.
Interface:
interface LLMProvider {
generate()
stream()
structuredOutput()
embed()
}Support configurable providers.
Possible:
OpenAI
Anthropic
Google
xAI
local OpenAI-compatible endpointCancerIndex should operate without requiring AI for core database functionality.
67. EMBEDDINGS
Generate embeddings for:
cancer descriptions
publication abstracts
trial descriptions
drug mechanisms
biomarker descriptions
evidence summariesUse pgvector initially.
Store embedding model/version.
Never mix embeddings generated by incompatible models in one vector column without model metadata.
68. SEARCH ENGINE
Global search should support:
cancers
subtypes
genes
variants
biomarkers
drugs
trials
publications
institutions
researchersExamples:
panc
→ Pancreatic Cancer
→ Pancreatic Ductal Adenocarcinoma
G12C
→ KRAS G12C
HER2 low
→ HER2-low Breast Cancer
→ HER2-low biomarker conceptImplement:
exact
alias
prefix
fuzzy
semantic
cross-entity69. ENTITY RECONCILIATION ENGINE
This is one of the hardest parts.
Example source names:
NSCLC
Non-small-cell lung cancer
Non Small Cell Lung Carcinoma
non-small cell carcinoma of lungmust resolve appropriately.
Use:
exact IDs
ontology mappings
canonical aliases
normalized strings
context
LLM only as fallback candidate generator
human reviewNever merge two cancer entities solely because embeddings are similar.
70. ENTITY MERGE QUEUE
Admin system:
Possible duplicate
Cancer A
Cancer B
Evidence:
name similarity: 0.94
NCIt match: yes
OncoTree match: yes
[MERGE]
[KEEP SEPARATE]
[REVIEW]Every merge must be auditable and reversible.
71. TEMPORAL DATA
Every observation is time-aware.
Never overwrite:
incidence 2022with:
incidence 2024Store both.
Core observation:
interface EpidemiologyObservation {
cancerId: string
geographyId: string
year: number
sex?: string
ageGroup?: string
metric:
| "incidence_count"
| "incidence_rate"
| "as_incidence_rate"
| "mortality_count"
| "mortality_rate"
| "as_mortality_rate"
| "prevalence"
value: number
unit: string
lowerCI?: number
upperCI?: number
sourceId: string
}72. SURVIVAL DATA MODEL
Survival requires context.
Never store simply:
survival = 32%Use:
interface SurvivalObservation {
cancerId: string
geographyId?: string
stage?: string
sex?: string
ageGroup?: string
diagnosisPeriod?: string
survivalType:
| "overall"
| "relative"
| "cause_specific"
| "progression_free"
| "disease_free"
durationMonths: number
probability?: number
medianMonths?: number
cohortSize?: number
lowerCI?: number
upperCI?: number
sourceId: string
}73. STAGING
CancerIndex must support multiple staging systems.
Do not pretend all cancers use identical Stage I–IV systems.
Model:
AJCC/TNM
FIGO
Ann Arbor
Lugano
Durie-Salmon
ISS/R-ISS
Binet
Rai
disease-specific systemsLicensing must be checked before reproducing proprietary staging definitions.
74. RISK FACTORS
Risk factor entities:
smoking
alcohol
UV
obesity
infection
occupational exposure
radiation
genetic predisposition
hormonal factors
ageRelations require evidence.
Example:
RiskFactor → ASSOCIATED_WITH → CancerStore:
relative risk
odds ratio
hazard ratio
population attributable fraction
confidence interval
studyDo not translate association into causality automatically.
75. HEREDITARY CANCER
Dedicated hereditary layer.
Entities:
germline gene
syndrome
variant
cancer risk
penetrance estimateExamples conceptually:
BRCA1
BRCA2
Lynch syndrome
TP53/Li-Fraumeni
APC/FAP
VHLUse trusted genetic sources.
Strong warning:
CancerIndex must not interpret a user's personal germline result as medical advice.
76. SCREENING
Store:
screening method
eligible population
cancer
country
organization
recommendation date
evidence levelGuidelines are geography and organization specific.
Never display a universal screening recommendation when there isn't one.
77. PREVENTION
Represent prevention evidence separately.
Potential:
vaccination
smoking cessation
UV protection
risk-reducing surgery
screening
infection prevention
occupational exposure reduction78. PATHOLOGY
Future module.
Data:
histology
pathology images
stains
IHC
morphology
gradeUse public datasets with explicit image usage rights.
79. RADIOLOGY
Future module.
Potential public datasets:
TCIA and other properly licensed collectionsSeparate:
CT
MRI
PET
X-ray
ultrasoundDo not expose patient-identifiable DICOM metadata.
80. CANCER INDEX API
Public API:
api.cancerindex.ioVersion:
/v1/Possible endpoints:
GET /v1/cancers
GET /v1/cancers/{id}
GET /v1/cancers/{id}/statistics
GET /v1/cancers/{id}/survival
GET /v1/cancers/{id}/genes
GET /v1/cancers/{id}/variants
GET /v1/cancers/{id}/drugs
GET /v1/cancers/{id}/trials
GET /v1/cancers/{id}/publications
GET /v1/genes
GET /v1/genes/{symbol}
GET /v1/variants/{id}
GET /v1/drugs
GET /v1/drugs/{id}
GET /v1/trials/{nct}
GET /v1/rankings81. GRAPHQL
Consider later:
/graphqlExample conceptual query:
cancer(id: "CI-CAN-...") {
name
ranking {
incidence
mortality
}
genes {
gene {
symbol
}
frequency
}
trials(status: RECRUITING) {
nctId
phase
}
}82. MCP SERVER
CancerIndex should eventually expose an MCP server.
Purpose:
Allow AI agents to query CancerIndex directly.
Tools:
search_cancers
get_cancer
rank_cancers
get_epidemiology
get_survival
get_gene
get_variant
get_drug
search_trials
search_publications
query_knowledge_graphRead-only initially.
83. BULK DATA
Eventually provide permitted CancerIndex-derived datasets.
Formats:
CSV
JSON
JSONL
ParquetNever redistribute restricted upstream source material.
84. ARCHITECTURE
Recommended:
Next.js
TypeScript
React
PostgreSQL
pgvector
ClickHouse
Redis
OpenSearch or Elasticsearch
MinIO/S3
Python ingestion workers
FastAPI scientific services
Temporal or durable job orchestrationGraph:
Start with relational edge tables.
Introduce Neo4j/Memgraph only if graph workloads justify operational complexity.
Do not add infrastructure merely because it sounds sophisticated.
85. POSTGRESQL
Primary store for:
canonical entities
relationships
users
API metadata
source registry
provenance
admin
ranking snapshots86. CLICKHOUSE
Use for large analytical observations:
epidemiology
variant frequencies
publication timelines
trial timelines
ranking datasets
event logs87. OPENSEARCH
Use for global full-text search.
Indexes:
cancers
genes
variants
drugs
trials
publications88. OBJECT STORAGE
Use for:
raw connector snapshots
bulk source archives
large datasets
export files
images where licensed89. REDIS
Use for:
hot cache
rate limiting
jobs
distributed locks
temporary AI streamsDo not use Redis as canonical storage.
90. INGESTION JOB SYSTEM
Every ingest must be restartable.
Use:
connector
↓
discovery
↓
fetch
↓
raw persist
↓
parse
↓
validate
↓
normalize
↓
reconcile
↓
canonical persist
↓
index
↓
derived metrics
↓
ranking recompute91. IDEMPOTENCY
Running a connector twice must not duplicate data.
Use source-native IDs.
Example:
source = PubMed
source_record_id = 12345678Unique constraint.
92. SOFT DELETION
Sources can retract or remove records.
Never immediately hard-delete.
Use:
active
deprecated
retracted
withdrawn
source_missingRetain history.
93. PUBLICATION RETRACTIONS
CancerIndex must track retracted publications when data permit.
Relationships based only on retracted evidence should be flagged.
94. DATA FRESHNESS
Every page needs:
Data updated
Source updated
CancerIndex synchronizedExample:
Clinical trials updated: today
Genomics updated: Aug 2026
Global incidence dataset: 2024 estimate95. SOURCE PAGE
Route:
/source/{source}Display:
provider
description
dataset
access method
last sync
records
coverage
license status
data version
connector healthTransparency is a feature.
96. CHANGE HISTORY
Every entity should support change history.
Example:
Aug 19:
FDA approval added
Aug 14:
3 new trials
Aug 10:
GDC mutation frequency refreshed
Aug 03:
CancerIndex score changed 84.1 → 84.797. CANCER WATCH
Users can follow:
cancer
gene
variant
drug
trialNotifications:
new clinical trial
trial status change
FDA approval
publication
new genomic finding
ranking change98. USER ACCOUNTS
Account system:
email
password
email verification
password reset
session managementOptional:
Google
Apple
ORCIDDo not store sensitive health profiles by default.
99. RESEARCH WORKSPACE
Users can save:
cancers
genes
variants
drugs
trials
papers
queries
chartsCreate collections:
"My KRAS research"
"Rare sarcomas"
"Pancreatic cancer trials"100. COMPARISON ENGINE
Route:
/compareCompare up to several cancers.
Example:
Pancreatic cancer
Glioblastoma
Lung adenocarcinoma
MelanomaCompare:
incidence
mortality
survival
trends
genes
biomarkers
treatments
trials
research101. VISUALIZATION SYSTEM
CancerIndex should be visually exceptional.
Visualizations:
ranked bar charts
time-series
survival curves
heatmaps
world maps
genomic frequency plots
co-occurrence matrices
oncoprints
trial timelines
drug approval timelines
knowledge graphs
Sankey charts
bubble plotsCharts need:
source
unit
population
time period
download102. KNOWLEDGE GRAPH UI
Users can start from:
KRASand visually explore:
KRAS
├── G12C
│ ├── NSCLC
│ ├── colorectal cancer
│ ├── therapies
│ └── trials
├── G12D
├── G12V
└── pathwaysClick nodes dynamically.
Avoid rendering thousands of nodes at once.
103. DESIGN SYSTEM
CancerIndex must NOT look like a generic SaaS dashboard.
Target aesthetic:
scientific
editorial
premium
institutional
modern
high-information-density
trustworthyThink:
Nature
Bloomberg
Our World in Data
high-end scientific visualizationAvoid:
giant gradients everywhere
dozens of rounded cards
cartoon health icons
generic AI sparkle graphics104. COLOR
Base:
off-white / white
deep charcoal
muted scientific neutralsCancer-specific color coding can exist but must not compromise accessibility.
Never rely on color alone.
105. HOME PAGE
Hero:
CancerIndex
The global index of cancer.
Explore every cancer.
Rank global burden.
Follow treatments.
Search genomics.
Track clinical research.Global search immediately visible.
Below:
Cancer burden today
Global rankings
Fastest-rising cancers
Highest mortality
Poorest survival
Most active research
Largest treatment gaps
Rare cancers
Latest oncology approvals
New clinical trials106. LIVE DATA TICKER
Tasteful top-line statistics:
Cancer entities indexed
Genes indexed
Variants indexed
Clinical trials
Publications
Drug indications
Countries
SourcesValues must come from database counts.
107. "ALL CANCERS" EXPLORER
Route:
/cancersDo not show only a few dozen cards.
Build a powerful explorer.
Filters:
anatomical system
histology
solid/hematologic
adult/pediatric
rare/common
molecular subtype
incidence
mortality
survival
research level
trial count
treatment availabilitySupport thousands of entities.
108. TAXONOMY EXPLORER
Tree/browser:
Blood
Breast
CNS
Digestive
Endocrine
Gynecologic
Head & Neck
Lung
Skin
Soft tissue
Urinary
...Also:
histology view
molecular view
WHO view
NCI view109. RARE CANCER DISCOVERY
Feature:
Random Rare Cancer
Useful for discovery.
Shows:
what it is
annual incidence
known cases/data
research count
trials
genes
treatments110. DATA QUALITY ENGINE
Every normalized record runs validation.
Examples:
incidence >= 0
deaths >= 0
survival between 0 and 1
year reasonable
country valid
gene symbol canonical
variant syntax valid where possible
trial phase enum recognized111. CROSS-SOURCE CONFLICTS
Sources will disagree.
Never silently average everything.
Store each observation.
Example:
Source A:
5-year survival = 31%
Source B:
5-year survival = 36%CancerIndex may compute a harmonized estimate only with a documented method.
Show:
Why estimates differ112. EVIDENCE ENGINE
Create CancerIndex evidence hierarchy.
Possible dimensions:
study design
sample size
replication
publication quality
clinical relevance
regulatory validation
expert curation
recencyDo NOT reduce all scientific truth to one score.
Use multi-dimensional evidence badges.
113. CLINICAL EVIDENCE LABELS
Example:
REGULATORY APPROVED
GUIDELINE SUPPORTED
PHASE III
PHASE II
PHASE I
RETROSPECTIVE CLINICAL
CASE SERIES
CASE REPORT
PRECLINICAL
COMPUTATIONAL114. STATISTICAL INTEGRITY
Never calculate survival by dividing unrelated values.
Never compare crude incidence with age-standardized incidence without labeling.
Never mix:
incidence
prevalence
mortality
case fatality
overall survival
relative survivalEvery metric needs precise definition.
115. CANCER "DEADLINESS"
Avoid an undefined “deadliest cancer” metric.
The interface should let users choose:
Most deaths
Highest mortality rate
Highest mortality/incidence ratio
Lowest 5-year survival
Highest CancerIndex ImpactThis distinction is important.
116. AGE STANDARDIZATION
For international comparison prioritize appropriately standardized rates.
Store standard population used if source provides it.
Do not present crude rates as directly comparable across countries with radically different age structures.
117. GEOGRAPHIC NORMALIZATION
Canonical geography entity:
ISO country
ISO subdivision
region
continent
WHO region
IARC region if appropriateKeep source geography separately.
118. CURRENCY
Not central initially.
If later adding:
drug cost
economic burden
research fundingalways store:
currency
year
country
nominal/real
source119. RESEARCH FUNDING
Future innovation:
Connect:
NIH RePORTER
CIHR
EU grants
UKRI
other public grantsThen create:
funding by cancer
funding per annual death
funding per incident casePotential:
Funding Gap Index
But methodology must be transparent.
120. NIH REPORTER CONNECTOR
Potential high-priority future connector.
Map grant:
project
principal investigator
institution
funding amount
year
cancer
gene
topic
publication121. PATENTS
Potential future module.
Sources:
USPTO
EPO
Google Patents metadata where appropriateUse to map therapeutic innovation.
Not required for MVP.
122. COMPANY PIPELINE
Potential future module:
biotech
pharma
drug candidate
target
phase
indicationSources must be verified.
Public company claims should not override trial registries/regulatory sources.
123. DRUG DEVELOPMENT PIPELINE
Statuses:
preclinical
Phase I
Phase I/II
Phase II
Phase II/III
Phase III
submitted
approved
discontinued
withdrawnStatus may be disease-specific.
124. FAILURE DATABASE
Extremely valuable.
Track oncology programs that fail or stop.
Sources:
ClinicalTrials.gov status
regulatory documents
company releases
publicationsCreate:
Drug → Cancer → Development outcomeAvoid inferring failure solely from stale trial status.
125. RESISTANCE DATABASE
Track mechanisms:
primary resistance
acquired resistanceRelations:
Variant → confers resistance → Drug
Pathway → resistance mechanism → DrugEvidence-backed only.
126. METASTASIS DATABASE
Map:
primary cancer
→ common metastatic locationsStore frequency only with cohort context.
Do not generalize from small cohorts.
127. MULTI-OMICS
Future coverage:
genome
transcriptome
epigenome
proteome
metabolome
single-cell
spatialGDC and other public research repositories can seed this layer.
128. SINGLE-CELL CANCER DATA
Future connector candidates:
CELLxGENE
Human Tumor Atlas Network resources
public scRNA-seq studiesMust support:
study
sample
cell type
cancer
gene expressionLarge matrices should not live in PostgreSQL.
129. HUMAN TUMOR ATLAS
Potential high-value research connector where datasets and terms permit.
130. PROTEOMICS
CPTAC-related data should connect:
cancer
protein
phosphoprotein
genomic alteration
clinical outcome131. MICROBIOME / CANCER
Future experimental research category.
Clearly label exploratory evidence.
132. ENVIRONMENTAL EXPOSURES
Possible integration:
IARC carcinogen classifications
occupational exposure datasets
air pollution dataDo not infer personal cancer risk.
133. CARCINOGEN ENTITY
Create:
CarcinogenRelations:
Carcinogen → evidence of association → CancerStore classification authority.
134. INFECTIOUS ONCOLOGY
Entities:
HPV
HBV
HCV
EBV
H. pylori
HHV-8
etc.Map to cancer evidence.
135. CANCER PREVALENCE FORECASTS
Can later model forecasts.
But clearly label:
OBSERVED
ESTIMATED
PROJECTEDNever make projections visually indistinguishable from observed registry data.
136. FORECAST ENGINE
Potential:
incidence forecast
mortality forecast
trial activity forecast
research momentumVersion each model.
Display uncertainty intervals.
137. DATA SNAPSHOTS
Monthly immutable snapshots:
CancerIndex 2026-09
CancerIndex 2026-10Allows reproducibility.
138. DATA RELEASES
Publish:
CancerIndex Data Release 1With:
new sources
updated sources
entity changes
ranking methodology changes
known limitations139. API VERSIONING
Never break existing clients casually.
Use:
/v1
/v2Data release version separate from API version.
140. ADMIN CONTROL CENTER
Route:
/adminSections:
Overview
Connectors
Ingestion
Entities
Reconciliation
Rankings
Evidence
Sources
Licensing
Users
AI
Jobs
Search
System141. CONNECTOR ADMIN
For every connector:
Run now
Pause
Resume
Backfill
Incremental sync
Dry run
View raw records
View parser
View errors
View schema changes142. LICENSE REGISTRY
Create internal table:
source_licenseFields:
source
license
commercial use
redistribution
derivative works
attribution requirements
API terms
review date
notes
approved for productionNo new connector becomes public until licensing status is reviewed.
143. SOURCE PRIORITY
When sources conflict, do not blindly implement a global precedence.
Precedence depends on field.
Examples:
Gene official symbol → HGNC
Clinical trial registration → ClinicalTrials.gov
US regulatory status → FDA
global burden estimate → selected IARC dataset
US registry survival → SEER
variant clinical curation → retain multiple curated sources144. ENTITY LINEAGE
Every canonical field may need:
derivedFromSourceRecordIdsExample:
canonical name:
"Lung Adenocarcinoma"
supported by:
NCIt
SEER
OncoTree
GDC145. CACHING
Cache expensive:
rankings
global aggregates
country dashboards
AI answers
knowledge graph layoutsInvalidation should be event-driven when possible.
146. PERFORMANCE
Targets:
homepage < 2 sec meaningful render
search suggestions < 200 ms cached target
common API reads < 300 ms target
ranking query < 500 ms targetDo not block page rendering on AI generation.
147. SEO
CancerIndex has enormous programmatic SEO potential.
Pages:
/cancer/{cancer}
/cancer/{cancer}/survival
/cancer/{cancer}/statistics
/cancer/{cancer}/genes
/cancer/{cancer}/trials
/gene/{gene}
/drug/{drug}
/variant/{variant}
/country/{country}Every generated page must contain substantive sourced information.
No thin spam pages.
148. STRUCTURED DATA
Use relevant Schema.org structured metadata where appropriate:
MedicalCondition
Drug
Dataset
ScholarlyArticle
OrganizationVerify current specifications before implementation.
149. ACCESSIBILITY
WCAG-minded implementation.
Requirements:
keyboard navigation
screen reader labels
contrast
chart text alternatives
color-independent state
reduced motion150. INTERNATIONALIZATION
English first.
Architecture must support:
French
Spanish
German
Portuguese
Japanese
etc.Canonical scientific entity remains language-independent.
Translations are attributes.
151. LOCALIZATION
Important distinction:
language != geographyFrench Canadian user can view Canadian data.
French user can view France data.
152. TESTING REQUIREMENTS
Claude must create:
unit tests
integration tests
connector fixture tests
schema tests
ranking tests
reconciliation tests
API contract tests
UI tests
end-to-end tests153. CONNECTOR FIXTURES
Never run all connector tests against production APIs.
Store sanitized fixtures.
Test:
normal response
empty response
pagination
rate limit
server error
schema change
malformed record
duplicate record154. SCIENTIFIC REGRESSION TESTS
Create invariant tests.
Examples:
survival >= 0
survival <= 1
incidence >= 0
mortality >= 0
lowerCI <= estimate
estimate <= upperCI155. RANKING TESTS
Given fixed fixture inputs, ranking output must be deterministic.
Snapshot:
ranking methodology version
input snapshot
output156. RECONCILIATION TESTS
Known aliases:
NSCLC
non-small cell lung cancershould behave correctly.
Known distinct diseases must never merge accidentally.
Build a large gold-standard mapping fixture.
157. AI EVALUATION SUITE
Create fixed questions:
What is PDAC?
Compare LUAD and SCLC.
What cancers are associated with BRAF V600E?
What recruiting trials exist for X?Evaluate:
citation correctness
entity correctness
numerical correctness
unsupported statements
source freshness158. HALLUCINATION DEFENSE
AI must say:
CancerIndex does not currently have sufficient sourced data to answer this.instead of guessing.
159. NO SILENT FALLBACK TO MODEL KNOWLEDGE
If CancerIndex retrieval finds no evidence:
do not silently answer using model memory.
Model knowledge may only be used as clearly labeled supplementary context if product policy explicitly allows it.
Default:
database-grounded answers only.
160. SECURITY
Protect:
API keys
database credentials
connector credentials
LLM keys
admin routes
worker endpointsUse environment variables/secrets.
Never commit secrets.
161. USER PRIVACY
CancerIndex does not require personal health data to be useful.
Avoid collecting:
diagnosis
genetic results
treatment history
medical documentsunless a future clearly separated healthcare feature has proper privacy architecture.
162. ANALYTICS PRIVACY
Do not log sensitive search queries unnecessarily.
Provide privacy-preserving analytics.
163. AUTHORIZATION
Roles:
USER
RESEARCHER
CURATOR
ADMIN
SUPERADMINCurators can edit scientific metadata.
Every curator action is logged.
164. CURATION PLATFORM
Allow expert curators to:
merge entities
split entities
add aliases
correct mappings
flag evidence
resolve conflicts
add citations
approve AI extractions165. AI CURATION QUEUE
LLM pipeline may discover candidate:
publication → gene
publication → cancer
publication → drugConfidence:
>0.98 auto-accept only for low-risk deterministic mappings
0.80–0.98 review
<0.80 reject/manualThresholds must be evaluated empirically.
Do not use these exact numbers blindly.
166. EXTRACTION ENGINE
For publications:
abstract
↓
NER
↓
ontology mapping
↓
relationship extraction
↓
confidence
↓
validation
↓
graph edgePrefer deterministic identifiers when present.
167. PDF INGESTION
Some sources publish PDFs.
Pipeline:
PDF
↓
native text extraction
↓
layout understanding
↓
table extraction
↓
OCR only when necessary
↓
structured JSON
↓
validationStore page-level citations.
168. TABLE EXTRACTION
AI-extracted numeric tables must pass validations.
Do not accept:
OCR number → production statisticwithout confidence checks.
169. DATA DIFFS
On source refresh:
previous snapshot
vs
current snapshotGenerate:
new records
removed records
changed values
new enumsStore diff.
170. ALERTS
Internal alerts:
connector failure
stale source
ranking anomaly
mass entity deletion
schema drift
unexpected record drop
license review due171. ANOMALY DETECTION
Example:
GDC records yesterday: 2,430,000
today: 214Do NOT publish a destructive update.
Pause ingest and alert.
172. BACKUPS
Automated:
PostgreSQL backups
object storage versioning
search reindex ability
configuration backupsTest restore process.
173. INFRASTRUCTURE / CLUSTER DEPLOYMENT
CancerIndex should be containerized.
Use:
DockerServices should be independently deployable.
Suggested:
web
api
worker-ingest
worker-ai
worker-ranking
postgres
redis
clickhouse
opensearch
minioIf deploying to an existing cluster, keep configuration portable.
174. DOMAIN
Production:
www.cancerindex.io
cancerindex.io
api.cancerindex.ioOptional:
status.cancerindex.io
docs.cancerindex.io175. OBSERVABILITY
Use:
structured logs
metrics
distributed traces
error monitoring
job monitoringEvery request gets correlation ID.
Every ingest gets run ID.
176. INGEST RUN ID
Example:
ING-CLINICALTRIALS-20260908-000019Every created/updated record can reference ingest run.
177. CANCERINDEX SCORE VERSIONING
Example:
CI-IMPACT-v1.0
CI-RESEARCH-GAP-v1.0
CI-TRIAL-GAP-v1.0
CI-PROGRESS-v1.0
CI-MOMENTUM-v1.0Never silently change formula.
178. METHODOLOGY PAGE
Route:
/methodologyExplain:
sources
normalization
ranking
age standardization
survival
research metrics
trial metrics
composite indexes
uncertainty
limitationsThe methodology page should be exceptionally detailed.
179. PUBLIC REPRODUCIBILITY
For each ranking:
button:
MethodologyDisplay formula.
Potential later:
Download input dataset
Download ranking datasetwhere licensing permits.
180. DATA SOURCE BADGES
On values:
IARC
SEER
GDC
FDA
ClinicalTrials.gov
CIViCHover → metadata.
181. CITATION UX
Citation:
[1]click opens side panel rather than sending user away immediately.
Panel:
Source
Original title
Dataset
Record
Date
Method
Open source182. CONFIDENCE UX
Examples:
High confidence
Moderate confidence
Limited evidence
Sparse dataDo not hide uncertainty.
183. "WHY THIS RANK?"
Every CancerIndex rank gets:
Why #4?Click:
Mortality burden +23.3
Lethality +18.9
Treatment gap +14.1
Research gap +9.7
Trend +8.2
...184. HISTORICAL RANKS
Store ranking snapshots.
Graph:
2015 #12
2018 #11
2021 #9
2024 #8Important: methodology consistency must be maintained or explicitly annotated.
185. USER-CUSTOM RANKINGS
Advanced feature.
Allow users to set weights:
Mortality 40%
Incidence 20%
Survival 20%
Research gap 20%Generate:
Custom Cancer IndexDo not overwrite official CancerIndex ranking.
186. DATA EXPLORER
Advanced SQL-like analytics UI without exposing raw SQL.
Dimensions:
cancer
country
year
sex
ageMeasures:
cases
deaths
ASIR
ASMR
survival
trials
publications187. CHART BUILDER
Users choose:
X = year
Y = mortality
Group = cancer
Country = CanadaGenerate shareable chart.
188. EMBEDDABLE CHARTS
Future:
embed.cancerindex.io/chart/{id}Attribution required.
189. SHAREABLE RESEARCH CARDS
Generate beautiful cards:
Pancreatic Cancer
#3 Lethality
#7 Global Mortality
5-year survival ...Always include date/source.
190. PUBLIC DATA API KEYS
API account:
free
research
pro
institutionalDo not monetize third-party data contrary to source licenses.
Value can come from CancerIndex aggregation, normalization and infrastructure where permitted.
191. RATE LIMITING
API:
anonymous
authenticated
paid/institutionalReturn standard rate-limit headers.
192. DEVELOPER PORTAL
Route:
/developersInclude:
API docs
OpenAPI
authentication
examples
schema
changelog
status193. DATA DOWNLOAD CENTER
Route:
/dataList datasets CancerIndex is legally allowed to redistribute.
194. SOURCE LICENSE AUTOMATION
Crawler can periodically detect source terms changes.
But:
AI cannot make final legal determination.
Flag for human review.
195. RELEASE BOT
Weekly report:
CancerIndex Weekly Data Report
+43 cancers/subtypes
+12,328 publications
+184 trials
+2 FDA approvals
+91,224 variant relations
3 connector warnings196. FRONT PAGE DAILY UPDATE
Show:
Updated X minutes agoonly for sources actually refreshed that recently.
Do not imply global dataset freshness because ClinicalTrials updated today.
197. CANCERINDEX DAILY
Potential editorial product:
CancerIndex Daily
Automatically identify:
important approvals
practice-changing trials
major publications
new trial openings
large dataset releasesAI summarizes with citations.
198. TREND DETECTOR
Calculate abnormal increases in:
publication volume
trial creation
drug development
gene mentionsPotential:
"KRAS G12D research activity +74% YoY"Only publish after methodology validation.
199. TOPIC GRAPH
Search:
ADCGraph:
ADC
→ HER2
→ TROP2
→ HER3
→ cancers
→ drugs
→ trials
→ publications200. RELATIONSHIP TEMPORALITY
Relationships evolve.
Store:
first evidence
most recent evidence
current statusA therapy-cancer relationship may change from:
experimental
→ Phase III
→ approved201. REAL-WORLD EVIDENCE
Future module.
Possible sources:
public registries
regulatory RWE reports
published cohortsDo not attempt to ingest private medical records casually.
202. PATIENT-REPORTED OUTCOMES
When published:
quality of life
symptom burden
functional outcomesStore separately from survival.
203. ENDPOINT ENTITY
Clinical endpoints should become structured concepts:
OS
PFS
DFS
EFS
ORR
DOR
pCR
MRD
QoLMap trial results.
204. TRIAL RESULTS EXTRACTION
When results are available:
capture structured registry results first.
Publication-derived results must cite paper.
Store:
endpoint
population
arm
estimate
CI
p-value
follow-up205. TREATMENT EFFECT MODEL
Do not store:
Drug X improves survival by 40%Store:
endpoint
effect measure
HR/RR/OR
estimate
CI
population
comparator
trial
follow-up206. CROSS-CANCER ANALYSIS
Enable questions:
Which cancers share KRAS mutations?
Which cancers have HER2 amplification?
Which cancers respond to tissue-agnostic therapies?
Which cancers share immune biomarkers?207. TUMOR-AGNOSTIC INDICATIONS
Support cancer-agnostic drug approvals.
Drug indication entity may reference:
biomarker
without single cancer restrictionDo not force every approval to one cancer ID.
208. CANCER OF UNKNOWN PRIMARY
Include CUP properly.
Do not force primary anatomical site where unknown.
209. BENIGN / BORDERLINE TUMORS
CancerIndex may index clinically relevant nonmalignant/borderline tumors if useful for taxonomy.
They must be clearly marked:
malignant = falseNever count them in cancer rankings unless methodology explicitly includes them.
210. SKIN CANCER COUNTING
Be careful with:
non-melanoma skin cancersSome global datasets treat them differently.
Ranking engine must preserve inclusion/exclusion rules.
211. HEMATOLOGIC MALIGNANCIES
Do not model solely by anatomical organ.
Dedicated structure for:
leukemia
lymphoma
myeloma
myelodysplastic neoplasms
myeloproliferative neoplasms212. SARCOMAS
Build fine-grained taxonomy.
Examples categories:
soft tissue
bone
GIST
leiomyosarcoma
liposarcoma
angiosarcoma
synovial sarcoma
Ewing sarcoma
osteosarcomaDo not group all rare sarcomas when subtype data exists.
213. BRAIN/CNS TUMORS
Molecular classification is essential.
Model modern molecular subtypes.
Taxonomies change over time.
Store classification version.
214. BREAST CANCER
Support:
histology
ER
PR
HER2
HER2-low where applicable
triple negative
molecular subtypes
germline contextDo not collapse all breast cancers.
215. LUNG CANCER
Support:
SCLC
NSCLC
adenocarcinoma
squamous
large cell
molecular alterations216. COLORECTAL CANCER
Support:
colon
rectal
left/right sided context where evidence requires
MSI
RAS
BRAF
HER2217. PRECISION TAXONOMY
CancerIndex needs overlapping labels.
One patient cohort may conceptually be:
lung
adenocarcinoma
metastatic
EGFR-mutated
exon 19 deletionDo not create a unique canonical cancer entity for every arbitrary combination.
Use attributes/biomarker cohort definitions appropriately.
218. COHORT ENTITY
Create:
CohortDefinitionExample:
Metastatic EGFR exon 19 deletion lung adenocarcinomaThis is not necessarily a globally recognized cancer taxonomy node.
219. ONTOLOGY VERSIONING
Taxonomies evolve.
Store:
ontology
version
concept
valid_from
valid_toNever lose historical mappings.
220. CROSSWALK TABLES
Build:
NCIt ↔ ICD-O
NCIt ↔ ICD-10
NCIt ↔ OncoTree
NCIt ↔ Disease Ontology
NCIt ↔ MONDO
SEER ↔ canonical CancerIndexMappings may be:
exact
broader
narrower
related
ambiguous221. MATCH CONFIDENCE
Entity mapping:
EXACT_IDENTIFIER
CURATED_EXACT
ONTOLOGY_EXACT
CURATED_BROADER
CURATED_NARROWER
ALIAS
PROBABILISTIC
UNRESOLVED222. UNRESOLVED ENTITY QUEUE
Never discard unknown disease labels.
Store:
source text
source ID
context
countAdmin can map later.
223. DATA DISCOVERY AGENT
Build an AI-assisted internal agent that searches for:
new official APIs
new dataset releases
schema changes
new registries
new cancer ontologiesIt produces proposals.
It cannot automatically onboard sources into production without compliance review.
224. CONNECTOR DOCUMENTATION REQUIREMENT
Before Claude implements ANY connector:
- locate current official documentation;
- verify API/bulk mechanism;
- verify authentication;
- inspect pagination;
- inspect rate limits;
- inspect license/terms;
- inspect update schedule;
- inspect identifiers;
- save source schema;
- create tests.
Do not implement an API from memory.
225. CURRENT-DOC REQUIREMENT
Because CancerIndex depends on external systems:
Claude MUST always verify current documentation before coding an integration.
Do not trust:
old blog posts
random GitHub examples
Stack Overflow
cached knowledgePrefer:
official documentation
official repositories
official OpenAPI specs
official release notes226. CONNECTOR SOURCE TEST
Before production:
curl/API smoke test
↓
small fixture
↓
parser
↓
normalization
↓
reconciliation
↓
integration test
↓
full sync227. HUGE IMPORT SAFETY
Never begin a million-record import before proving the pipeline on:
10
100
1,000records.
228. BULK-FIRST STRATEGY
For massive datasets:
prefer bulk downloads over millions of API calls when terms and official access support it.
229. RATE LIMIT RESPECT
Implement:
token bucket
exponential backoff
Retry-After
jitter
max concurrencySource-specific.
230. CHECKSUMS
Bulk file:
SHA-256Store:
source URL
timestamp
checksum
size231. ETL LANGUAGE
Use Python heavily for scientific ETL.
TypeScript can orchestrate web/application systems.
Do not force complex bioinformatics normalization into TypeScript if mature Python packages are appropriate.
232. DATAFRAMES
For large ETL:
consider:
Polars
PyArrow
DuckDBinstead of blindly using pandas for everything.
233. PARQUET
Use Parquet for large analytical snapshots.
Partition by sensible dimensions.
Example:
source
year
entity type234. BIOINFORMATICS LIBRARIES
Before choosing packages:
verify active maintenance/current documentation.
Potential functionality:
HGVS normalization
VCF parsing
genomic liftover
sequence handlingNever implement complex genomics standards from scratch unless necessary.
235. GENOME BUILD
Canonical support:
GRCh37
GRCh38Where available.
Never silently convert coordinates.
Store original + normalized.
236. LIFTOVER
If performing liftover:
original assembly
original coordinate
target assembly
converted coordinate
tool/version
status237. VARIANT NORMALIZATION
Store:
genomic HGVS
coding HGVS
protein HGVS
gene
transcript
assembly
dbSNP
ClinVar ID
CIViC IDNot every variant will have all identifiers.
238. FUSIONS
Dedicated structure:
5' gene
3' gene
breakpoint
fusion name
orientationDo not model only as free text.
239. COPY NUMBER
Model:
amplification
gain
loss
deep deletionKeep source-specific thresholds.
240. EXPRESSION
Keep units/platform.
Never compare raw expression values from incompatible platforms directly.
241. BIOMARKER THRESHOLDS
Example PD-L1.
Store:
assay
clone
scoring system
threshold
cancer
indicationDo not reduce to positive/negative globally.
242. TMB
Store:
assay
unit
threshold
panel
cancer243. MSI
Map:
MSI-H
MSS
MSI-L
dMMR
pMMRbut preserve differences.
244. EVIDENCE CROSS-CANCER CONTEXT
A variant may be:
predictive in cancer A
prognostic in cancer B
unknown in cancer CRelationship context is mandatory.
245. RANKING DATA ELIGIBILITY
For a cancer to enter a ranking:
define explicit inclusion rules.
Example survival ranking:
minimum cohort size
accepted survival type
accepted diagnosis period
geography
minimum source qualityDo not rank sparse estimates unfairly.
246. PARENT VS SUBTYPE RANKING
Avoid double-counting.
If global incidence gives:
Lung Cancer = 2.4Mand subtype estimates separately:
LUAD
SCCdo not sum all three.
Rank scope must define entity level.
Allow:
Top-level cancer ranking
Histology ranking
Subtype ranking
Rare entity ranking247. GLOBAL MASTER RANKING
Default broad global ranking should use mutually exclusive or carefully defined top-level cancer categories.
Fine-grained ranking is separate.
248. CANCERINDEX COVERAGE COUNT
Homepage may say:
4,812 cancer entities indexedonly if entity model genuinely contains them.
Do not market every alias as a separate cancer.
249. DUPLICATE CONTROL
Alias count ≠ cancer count.
Subtype count ≠ top-level cancer count.
Be explicit.
250. METRIC CATALOG
Create first-class:
MetricDefinitionFields:
id
name
description
formula
unit
higherIsWorse
aggregation
validDimensions
sources
methodologyVersion251. FORMULA ENGINE
Derived metrics should not live as random code functions.
Create versioned formulas.
Example:
id: CI-METRIC-MIR
name: Mortality-to-Incidence Ratio
formula: mortality_count / incidence_count
version: 1.0252. DATA LINEAGE GRAPH
Every ranked score should be traceable:
CancerIndex score
↓
component
↓
normalized metric
↓
source observation
↓
raw source record253. ADMIN "TRACE VALUE"
Admin button:
TRACEFor any number.
Shows full lineage.
This will save massive debugging time.
254. CANCERINDEX LABS
Experimental section:
/labsFor:
forecasting
experimental indexes
novel network analysis
AI research toolsClearly separate experimental metrics from main product.
255. NETWORK CENTRALITY
Interesting research feature:
rank genes by:
number of cancers
number of actionable variants
number of approved drugs
number of trials
network centralityDo not imply biological importance solely from graph centrality.
256. DRUG TARGET LANDSCAPE
Visual:
targets × cancersHeatmap:
approved
clinical
preclinical257. ONCOLOGY PIPELINE MAP
Interactive:
Cancer
→ Target
→ Drug
→ Phase
→ Company258. BIOMARKER LANDSCAPE
Interactive matrix:
Cancer × BiomarkerColor:
frequency
clinical actionabilityDifferent toggles.
259. CANCER GENOMIC LANDSCAPE
Cancer page:
Top mutated genes
CNAs
fusions
pathwaysAllow study selection.
Do not merge frequencies from incompatible cohorts without method.
260. COHORT SELECTOR
Example:
TCGA
MSK cohort
CPTAC
study XUser can switch data source.
261. FREQUENCY DENOMINATORS
Every genomic frequency must include denominator.
Example:
KRAS mutation: 31.4%
214 / 681 profiled samplesNever show 31.4% without cohort context.
262. MISSINGNESS
Genomic studies frequently have different profiling coverage.
Store:
tested
not tested
unknownDo not assume missing = wild type.
263. SURVIVAL CURVES
Where permissible/raw aggregate data allow:
Kaplan-Meier visualization.
Display:
n at risk
CI
censoring
cohort
endpointDo not fabricate curves from summary survival percentages.
264. INCIDENCE TREND CHART
Use:
annual estimates
ASIR
confidence intervals when available265. GLOBAL BURDEN BUBBLE CHART
Axes:
X = incidence
Y = mortality/incidence
bubble = deathsGreat discovery visualization.
266. RESEARCH GAP QUADRANT
Axes:
X = disease burden
Y = research activityQuadrants:
high burden / high research
high burden / low research
low burden / high research
low burden / low research267. CLINICAL TRIAL MAP
World map of recruiting trial sites.
Filters:
cancer
drug
phase
biomarker268. FACILITY ENTITY
Normalize trial locations.
Challenge:
same hospital with slightly different namesUse:
name
address
city
country
geocode
organization ID269. ORGANIZATION RESOLUTION
Potential IDs:
ROR
GRID historical mappings
OpenAlex institutionUse ROR where appropriate.
270. RESEARCHER RESOLUTION
Use ORCID when explicitly linked.
Never assume identical author names are same person.
271. PUBLICATION ENTITY EXTRACTION
Use pipeline:
dictionary match
ontology mapping
NER
LLM structured extraction
cross validationStore extraction method.
272. EVIDENCE SENTENCE
When permitted, keep sentence-level evidence reference.
Do not violate publication copyright.
Store small extracted evidence snippets within legal limits where appropriate, otherwise store location/reference only.
273. FULL-TEXT RIGHTS
Open access ≠ automatically unrestricted redistribution in every context.
Track article license.
274. AUTOMATED CITATION
Generated text should prefer primary source where possible.
Example:
trial result:
prefer:
peer-reviewed trial publicationplus registry.
Regulatory claim:
prefer regulator.
275. SOURCE TRUST DOES NOT MEAN UNIVERSAL AUTHORITY
A source can be excellent for one question and inappropriate for another.
Example:
FDA → US approval
SEER → US registry statistics
IARC → global estimates
HGNC → gene symbols276. NO SOURCE MONOCULTURE
Important cancer facts should ideally cross-reference multiple sources where appropriate.
277. DATA FUSION
Do not create a fused value unless scientifically justified.
Often the right UX is:
SEER estimate
IARC estimate
Canadian estimateside by side.
278. DOCUMENT EVERYTHING
Repository:
/docs
architecture.md
data-model.md
ranking-methodology.md
source-policy.md
connectors.md
reconciliation.md
evidence.md
ai.md
security.md279. ADRs
Use Architecture Decision Records.
Example:
ADR-001 PostgreSQL as canonical database
ADR-002 pgvector
ADR-003 source-native raw retention
ADR-004 CancerIndex identifiers280. CLAUDE WORKFLOW
Before editing code:
1. inspect repository
2. inspect CLAUDE.md
3. inspect current architecture
4. inspect existing tests
5. inspect database schema
6. inspect connector framework
7. verify current external documentation
8. create plan
9. implement
10. test
11. verify
12. document281. DO NOT FAKE FEATURES
Never create UI with hardcoded fake metrics merely to make screenshots look complete.
If backend data doesn't exist:
show:
Data not yet availableor use explicitly labeled fixtures only in tests/dev.
282. NO MOCK DATA IN PRODUCTION
Absolutely no hidden mock values.
283. SOURCE AVAILABILITY
If a source connector fails:
CancerIndex continues using last known valid snapshot.
Display freshness.
Do not replace missing real data with AI guesses.
284. DATABASE MIGRATIONS
Every schema change:
migration
rollback strategy
testNever manually mutate production schema without migration.
285. QUERY OPTIMIZATION
Use indexes on:
canonical IDs
external IDs
slugs
gene symbol
NCT ID
PMID
DOI
year
geography
metric286. PARTITIONING
For huge observations:
partition or use ClickHouse.
Do not create billions of rows in poorly indexed PostgreSQL tables.
287. ENTITY COUNTERS
Precompute:
trial count
publication count
gene count
drug countwhere necessary.
But source them from canonical relations and refresh deterministically.
288. RANKING MATERIALIZATION
Rankings should be precomputed per common scope.
Do not run huge window functions for every homepage request.
289. INCREMENTAL RANK REBUILD
When ClinicalTrials changes:
recalculate trial-dependent ranks only.
Do not recompute global genomic rankings unnecessarily.
290. EVENT BUS
Useful events:
CancerUpdated
TrialUpdated
DrugApprovalAdded
PublicationAdded
ConnectorCompleted
RankingInvalidated291. STATIC + DYNAMIC
Use Next.js rendering strategy intelligently.
SEO pages can be statically regenerated.
Live trials/rankings can refresh dynamically.
292. MOBILE
Mobile is first-class.
High-density information must remain usable.
Use:
sticky tabs
collapsible sources
horizontal ranking tables
full-screen charts293. DESKTOP
Desktop should feel like a professional research terminal.
Allow:
split panels
dense tables
compare mode
persistent filters
keyboard search294. COMMAND PALETTE
Shortcut:
⌘KSearch any entity/action.
295. SHAREABLE URL STATE
Filters reflected in URL.
Example:
/rankings?metric=mortality&year=2024&sex=all296. DOWNLOAD CHART DATA
Every chart:
Download CSV
Download PNG/SVG where appropriate
Copy citationsubject to source redistribution terms.
297. CITATION EXPORT
Support:
BibTeX
RIS
plain citationfor publications.
298. RESEARCH NOTEBOOK
Future:
Users can create notes linking CancerIndex entities.
Not needed for initial release.
299. COMPLIANCE CHECKLIST
Before new source goes live:
[ ] official source verified
[ ] access method verified
[ ] license reviewed
[ ] attribution requirements stored
[ ] rate limits implemented
[ ] parser tests
[ ] raw snapshot
[ ] entity mappings
[ ] quality tests
[ ] production health monitoring300. PHASE 1
Build foundation.
Deliver:
Cancer taxonomy
NCI EVS
GDC
ClinicalTrials.gov
PubMed
ClinVar
SEER
IARC integration plan/compliance
HGNC
Cancer pages
Gene pages
Trial pages
Publication pages
Search
Provenance
Basic rankings301. PHASE 2
Add:
cBioPortal
CIViC
ChEMBL
Open Targets
DGIdb
FDA
DailyMed
Ensembl
Drug pages
Variant pages
Biomarker pages
Knowledge graph
Advanced rankings302. PHASE 3
Add:
global country data
European sources
Canadian sources
additional regulators
research funding
research gap
trial gap
treatment gap303. PHASE 4
Add:
DepMap
cell lines
preclinical
single cell
pathology
imaging
multi-omics304. PHASE 5
Add:
CancerIndex AI
MCP
advanced analytics
custom rankings
developer ecosystem
public data releases305. INITIAL CONNECTOR TARGET
Do not stop at five connectors.
Long-term target:
50+ high-quality connectorsBut quality > arbitrary connector count.
One reliable bulk/API integration is worth more than ten fragile scrapers.
306. CANCER COVERAGE TARGET
Do not set a fixed target such as:
200 cancersInstead:
Index every distinct malignant disease entity that can be mapped from selected authoritative oncology classifications.
Then expose:
top-level cancers
families
histologies
subtypes
molecular subtypesseparately.
307. PUBLIC SOURCE CATALOG
CancerIndex should publicly list connectors.
Example:
NCI GDC Genomics
SEER Epidemiology
IARC Global burden
ClinicalTrials Trials
PubMed Literature
ClinVar Variants
cBioPortal Cancer genomics
CIViC Clinical variants
HGNC Genes
ChEMBL Drugs
FDA Regulation
...308. SOURCE COVERAGE MATRIX
Table:
Epidemiology Genomics Trials Drugs Research
GDC · ✓ · · ✓
SEER ✓ · · · ·
ClinicalTrials · · ✓ ✓ ✓
PubMed · ✓ ✓ ✓ ✓
CIViC · ✓ · ✓ ✓309. CANCER DATA CARD
Every cancer card should contain at minimum:
name
parent category
annual burden if available
mortality
survival
CancerIndex rank
research activity
data confidence310. CANCER INDEX BADGES
Examples:
Rare
Pediatric
Hematologic
High Mortality
Rapidly Rising
High Research Activity
Low Trial Activity
Treatment GapGenerated from rules, not editorial opinion.
311. RANKING CHANGE EXPLANATION
If rank changes:
#12 → #8show why:
2024 global mortality estimate updated
+4 new active trials
methodology unchanged312. SEARCH RESULT EXPLANATION
Search result should show type:
EGFR
GENE
EGFR L858R
VARIANT
EGFR-mutated NSCLC
MOLECULAR COHORT
Osimertinib
DRUG313. SYNONYM MANAGEMENT
Alias examples:
GBM
glioblastoma
glioblastoma multiforme [historical usage]Preserve historical terminology without necessarily using it as preferred current name.
314. DEPRECATED TERMINOLOGY
Fields:
deprecated
deprecated_reason
replacement_entity
classification_version315. VERSION-SENSITIVE MEDICINE
Cancer classification changes.
Never rewrite historical publication terminology as if author used modern classification.
Map:
source_term
→ modern CancerIndex entitywhile preserving original term.
316. DATA CITATION
CancerIndex itself should produce dataset citations:
CancerIndex Data Release YYYY-MMfor researchers.
317. ABOUT PAGE
Explain:
what CancerIndex is
what it is not
where data comes from
how rankings work
limitations318. TRUST CENTER
Route:
/trustInclude:
data provenance
methodology
AI policy
privacy
security
source policy
corrections319. CORRECTIONS
Public correction mechanism.
Researchers can report:
wrong mapping
outdated statistic
incorrect citation
entity duplicationCorrections tracked publicly where appropriate.
320. SCIENTIFIC ADVISORY MODEL
Future:
expert contributors can have verified profiles.
No anonymous modification of clinical evidence without review.
321. API SOURCE ATTRIBUTION
API responses include:
{
"data": {},
"sources": [],
"dataRelease": "..."
}322. AI SOURCE SNAPSHOT
AI answer stores exactly which record versions were used.
This allows reproducibility after database updates.
323. NO BLACK-BOX SCORE
CancerIndex's major promise:
Every score can be decomposed.
No proprietary mystery scoring methodology.
324. RESEARCH GAP ETHICS
Low research activity does not inherently mean neglect.
Reasons can include:
rarity
disease biology
classification changes
small populations
successful preventionDisplay as quantitative signal, not accusation.
325. SURVIVAL ETHICS
Population survival statistics do not predict an individual's outcome.
Always show contextual note.
326. EPIDEMIOLOGY ESTIMATES
Global cancer statistics are frequently estimates, not literal complete counts.
Label:
estimated cases
estimated deathswhere appropriate.
327. CANCERINDEX "TODAY"
Avoid saying:
today X people have cancerunless the methodology genuinely supports it.
Prefer annual/latest dataset statistics.
328. CANCERINDEX DISCOVERY HOMEPAGE
Interesting modules:
Most common
Most lethal
Most researched
Most under-researched
Most trials
Largest treatment gaps
Fastest improving
Fastest rising
Rare cancer spotlight329. "CANCER UNIVERSE"
Create a visual:
Cancer UniverseThousands of cancer/subtype nodes arranged by anatomical/histological family.
Size could represent:
incidenceColor could represent:
survivalFilterable.
Potential signature visualization.
330. "CANCER MATRIX"
Rows:
cancersColumns:
genesCell:
alteration frequencyFilter by study.
331. "DRUG MATRIX"
Rows:
cancersColumns:
drugsCell:
approved
clinical
preclinical332. "TRIAL PULSE"
Live research activity chart.
new oncology trials / weekBreak down by:
cancer
phase
country
target333. "RESEARCH PULSE"
new publications / weekTopic clustering.
334. "APPROVAL PULSE"
Timeline of oncology regulatory decisions.
335. "GENOMIC PULSE"
Newly curated variant/drug evidence from authoritative sources.
336. DATA QUALITY BADGE
Pages can show:
Data Quality: HighDerived from:
source coverage
freshness
agreement
sample size
missingnessDocument formula.
337. SPARSE CANCER UX
For ultra-rare cancers:
do NOT show empty giant sections.
Instead:
Very limited epidemiological data are currently available.
What CancerIndex knows:
- 7 publications
- 1 active trial
- 2 reported genomic associationsSparse-data visibility is valuable.
338. ENTITY DISCOVERY PRIORITY
Nightly job:
find source disease labels not mapped to canonical CancerIndex cancer.
Rank by frequency.
This continuously expands coverage.
339. AUTOMATIC CANCER DISCOVERY
Candidate disease discovery:
NCIt additions
SEER updates
ClinicalTrials condition strings
GDC disease terms
CIViC diseases
PubMed MeSHCandidate → curator queue.
340. NEVER LET LLM INVENT A NEW CANONICAL DISEASE
LLM can suggest.
Canonical entity creation requires:
recognized ontology
trusted source
or human curator approval341. SOURCE COUNT IS NOT EVIDENCE QUALITY
Displaying:
14 sourcesdoes not mean stronger evidence if all sources repeat one paper.
Track evidence lineage and primary evidence.
342. DUPLICATE PUBLICATION DETECTION
Resolve:
PubMed
DOI
Crossref
Europe PMCinto one publication entity.
343. TRIAL ↔ PUBLICATION RESOLUTION
Use:
NCT IDs in publication
registry references
PubMed linksThen probabilistic matching only as fallback.
344. DRUG SYNONYMS
Drug naming is messy.
Support:
generic
brand
development code
salt
active moiety
combinationDo not treat brand names as separate molecules.
345. TARGET SYNONYMS
Resolve gene/protein target nomenclature carefully.
346. CANCERINDEX INTERNAL ONTOLOGY
Do not reinvent all external ontologies.
CancerIndex ontology should function primarily as a stable reconciliation layer.
347. EXTERNAL IDS ARE FIRST-CLASS
Never place IDs in an unstructured JSON dump only.
Create searchable cross-reference tables.
348. AUDIT LOG
Record:
who
what
before
after
when
whyfor curator/admin changes.
349. SOFT LAUNCH CRITERIA
CancerIndex is not ready merely because home page looks good.
Minimum:
credible taxonomy
multiple foundational connectors
provenance
search
cancer pages
ranking methodology
ranking reproducibility
connector monitoring
scientific disclaimers350. LAUNCH DATA QUALITY TARGETS
Before launch:
- no unexplained metrics
- no production mock data
- no duplicate canonical cancers from obvious aliases
- no broken source links
- no AI answers without citations
- no ranking without scope/year/source
- no unsupported treatment claims
351. PRIORITY IMPLEMENTATION ORDER
Claude should execute approximately:
1. repository foundation
2. database
3. source/provenance framework
4. canonical IDs
5. cancer ontology
6. NCI terminology connector
7. HGNC
8. GDC
9. ClinicalTrials
10. PubMed
11. ClinVar
12. SEER
13. IARC compliance/integration
14. cBioPortal
15. CIViC
16. FDA
17. ChEMBL
18. Open Targets
19. ranking engine
20. web experience
21. AIExact order may change based on API/data dependencies.
352. FIRST RANKINGS TO SHIP
Ship these first:
Global Incidence
Global Mortality
Age-standardized Incidence
Age-standardized Mortality
Mortality/Incidence Ratio
5-Year Survival where comparable
Active Trial Count
Research Publication Count
CancerIndex Research Gap
CancerIndex Trial GapThen composite score.
353. COMPOSITE SCORE MUST COME LATER
Do not launch composite score until individual metrics are correct.
Composite rankings amplify bad data.
354. DOCUMENT CONNECTORS AS CODE
Connector documentation should be generated/validated from manifests where practical.
355. REPOSITORY STRUCTURE
Suggested:
/apps
/web
/api
/admin
/services
/ingestion
/ranking
/ai
/search
/packages
/database
/ontology
/connectors
/schemas
/ui
/analytics
/provenance
/connectors
/nci-evs
/gdc
/seer
/iarc
/clinicaltrials
/pubmed
/clinvar
/cbioportal
/civic
/hgnc
/ensembl
/chembl
/opentargets
/dgidb
/fda
/docs
/infra
/tests356. TYPES
Share canonical TypeScript/Python schemas where possible.
Use generated JSON Schema/OpenAPI contracts to prevent divergence.
357. OPENAPI
API documentation generated from real API definitions.
Never manually maintain docs that drift from endpoints.
358. DATABASE SEEDS
Only seed:
system roles
configuration
metric definitions
source definitionsScientific data comes from connectors.
359. DEVELOPMENT FIXTURES
Label dev fixtures clearly.
Environment must make production fixture insertion impossible.
360. FINAL PRODUCT STANDARD
CancerIndex.io should feel like a serious piece of scientific infrastructure.
Not:
"Here are 30 cancers and some AI summaries."
But:
"Here is a continuously updated, provenance-aware, globally ranked ontology of cancer connecting epidemiology, genomics, biomarkers, therapies, trials, regulatory evidence and scientific literature."
The user should be able to start at any node:
a cancer
a gene
a mutation
a drug
a clinical trial
a paper
a countryand move through the entire oncology knowledge graph.
361. NORTH STAR
The ultimate CancerIndex graph should conceptually support:
ALL CANCERS
↓
ALL RECOGNIZED SUBTYPES
↓
EPIDEMIOLOGY
↓
GENES
↓
VARIANTS
↓
BIOMARKERS
↓
PATHWAYS
↓
DRUGS
↓
TREATMENTS
↓
APPROVALS
↓
CLINICAL TRIALS
↓
PUBLICATIONS
↓
RESEARCHERS
↓
INSTITUTIONS
↓
COUNTRIESEvery connection:
traceable
versioned
source-backed
queryable
rankable362. NON-NEGOTIABLE CLAUDE INSTRUCTION
Claude must NOT rush this project.
Before building substantial portions of CancerIndex:
- read this entire
CLAUDE.md; - inspect all existing code;
- research the latest official documentation;
- identify source licensing constraints;
- construct the canonical ontology carefully;
- test every connector independently;
- retain raw source provenance;
- build reconciliation before massive ingestion;
- validate all scientific computations;
- ensure ranking formulas are reproducible;
- never fabricate missing data;
- never substitute AI memory for missing database evidence;
- preserve uncertainty;
- optimize the product for both ordinary users and serious researchers.
Whenever there is a choice between:
faster implementationand:
scientifically defensible implementationchoose the scientifically defensible implementation.
Whenever there is a choice between:
more connectorsand:
reliable connectors with provenancechoose reliability first — then expand aggressively.
Whenever there is a choice between:
one broad cancer categoryand:
accurately modeling recognized subtypespreserve the detailed taxonomy.
Whenever an upstream source provides an identifier:
keep it.
Whenever an upstream source provides provenance:
keep it.
Whenever an upstream source provides uncertainty:
keep it.
Whenever CancerIndex computes something:
make the formula visible.
Whenever CancerIndex ranks something:
explain the ranking.
Whenever CancerIndex AI says something:
cite the evidence.
363. FINAL VISION
CancerIndex.io should eventually make it possible to ask:
Rank every indexed cancer in the world by unmet need, weighting global mortality, 5-year survival, number of approved systemic therapies, active Phase II/III trials and publication activity.
and receive a reproducible result.
Or:
Show every known cancer subtype associated with BRAF V600E, the frequency reported in each major cohort, regulatory-approved treatments by jurisdiction, active trials and supporting clinical evidence.
Or:
Which rare cancers have fewer than five recruiting trials worldwide despite high mortality?
Or:
What malignancies experienced the largest improvement in survival during the last two decades?
Or:
Which cancer research areas are accelerating fastest this year?
Or:
Show the entire treatment-development landscape for KRAS G12D.
CancerIndex should answer these from structured, sourced data rather than model memory.
That is the standard.
364. BUILD PRINCIPLE
Do not build a cancer website.
Build:
the structured global intelligence layer for cancer.
That is CancerIndex.io.