> Original product specification supplied by the project owner on 2026-09-08. The repository CLAUDE.md is the condensed operational version. # CLAUDE.md — CancerIndex.io ## 0. PROJECT IDENTITY **Project:** CancerIndex.io **Product type:** Global cancer intelligence platform, database, search engine, ranking system, knowledge graph, analytics platform, and AI research interface. **Primary language:** English. **Domain:** `cancerindex.io` **Mission:** Build the most comprehensive, structured, searchable, continuously updated and transparently sourced cancer intelligence database possible. CancerIndex.io must aim to become: > **The global index of cancer.** Think of the product as a combination of: * Bloomberg Terminal for oncology * IMDb/Wikipedia-style entity coverage * Our World in Data for cancer epidemiology * ClinicalTrials.gov explorer * cBioPortal genomic exploration * PubMed intelligence layer * drug/biomarker intelligence platform * cancer knowledge graph * transparent ranking engine * research assistant * cancer statistics terminal The product must not merely list the 20–50 most common cancers. CancerIndex must attempt to model **every identifiable malignant disease, histological subtype, molecular subtype, rare cancer, hematologic malignancy, pediatric malignancy, tumor family, anatomical site and recognized cancer entity that can be responsibly mapped from authoritative taxonomies.** This includes cancers that may affect only a tiny number of people annually. Do not hardcode a simplistic cancer list. Build an evolving oncology ontology. --- # 1. THE CORE PRINCIPLE CancerIndex must answer questions such as: * What cancers exist? * How common is each cancer? * Which cancers kill the most people? * Which cancers have the highest mortality rate? * Which cancers have the poorest survival? * Which cancers are increasing fastest? * Which cancers affect younger populations? * Which cancers have the most treatments? * Which cancers have the fewest treatments? * Which cancers receive the most research? * Which cancers receive the least research relative to burden? * Which cancers currently have the most clinical trials? * Which cancers have the most actionable mutations? * Which cancers have the largest unmet need? * Which cancers are associated with which genes? * What variants occur in those genes? * Which therapies target those abnormalities? * Which drugs are approved? * Which drugs are experimental? * Which biomarkers predict response? * Which trials are recruiting? * Which publications support each relationship? * Which countries have the highest incidence? * How has incidence changed through time? * How does survival differ by stage? * How does age influence incidence? * How does sex influence incidence? * Which risk factors are associated with each malignancy? * Which cancers can be screened for? * Which cancers can be prevented? * Which cancers are becoming more survivable? * Where are the largest gaps in oncology research? Everything must be explorable from the web interface and eventually through an API. --- # 2. ABSOLUTE RULE: PROVENANCE FIRST CancerIndex must never display an important scientific number without knowing where it originated. Every imported fact should support metadata such as: ```ts interface Provenance { sourceId: string sourceName: string sourceRecordId?: string sourceUrl?: string dataset?: string datasetVersion?: string publicationId?: string pmid?: string doi?: string retrievedAt: string publishedAt?: string updatedAt?: string geography?: string population?: string cohortSize?: number methodology?: string evidenceType: | "registry" | "clinical_trial" | "meta_analysis" | "systematic_review" | "cohort" | "case_control" | "case_series" | "case_report" | "preclinical" | "regulatory" | "guideline" | "expert_curation" | "database" | "computed" accessLevel: | "open" | "registration_required" | "controlled" | "licensed" confidence?: number license?: string } ``` Every derived statistic must also be reproducible. Example: ```json { "metric": "mortality_to_incidence_ratio", "value": 0.73, "computed": true, "formula_version": "ci-mir-v1", "inputs": [ "CINDEX-METRIC-102991", "CINDEX-METRIC-102992" ] } ``` Never destroy raw source information during normalization. Architecture: ```text RAW ↓ NORMALIZED ↓ CANONICAL ↓ DERIVED ↓ RANKED ↓ AI SYNTHESIS ``` These layers must remain separable. --- # 3. SCIENTIFIC SAFETY CancerIndex is a research/information platform. It is NOT: * a physician * a diagnostic system * a treatment prescriber * a replacement for professional medical care Never produce treatment recommendations based only on an AI-generated inference. Clearly distinguish: ```text OBSERVED DATA PUBLISHED EVIDENCE CURATED EVIDENCE REGULATORY STATUS CLINICAL GUIDELINE COMPUTED METRIC AI-GENERATED SYNTHESIS ``` Never silently merge these categories. --- # 4. COVERAGE GOAL — EVERY CANCER WE CAN MODEL CancerIndex needs a hierarchical disease model. A simple table: ```text lung cancer breast cancer brain cancer ... ``` is unacceptable. Cancer must be represented through a hierarchy. Example: ```text Cancer └── Solid Tumor └── Lung Cancer └── Non-Small Cell Lung Cancer └── Lung Adenocarcinoma ├── EGFR-mutated LUAD ├── KRAS-mutated LUAD ├── ALK-positive LUAD ├── ROS1-positive LUAD └── RET-positive LUAD ``` Another: ```text Cancer └── Hematologic Malignancy └── Leukemia └── Acute Leukemia └── Acute Myeloid Leukemia ├── AML with NPM1 mutation ├── AML with CEBPA mutation ├── APL └── therapy-related AML ``` Another: ```text Cancer └── CNS Tumor └── Glioma └── Diffuse Glioma └── Glioblastoma ``` The hierarchy needs multiple dimensions. Do NOT force every entity into only one parent tree. Support: ```text anatomical hierarchy histological hierarchy molecular hierarchy WHO-style disease classification ICD hierarchy ICD-O morphology ICD-O topography NCI Thesaurus Disease Ontology OncoTree SEER classification pediatric classification hematologic classification ``` --- # 5. CANONICAL CANCER ENTITY Create: ```ts CancerEntity ``` Example schema: ```ts interface CancerEntity { id: string slug: string canonicalName: string shortName?: string aliases: string[] abbreviations: string[] entityType: | "cancer" | "cancer_family" | "histology" | "subtype" | "molecular_subtype" | "hematologic_malignancy" | "precursor_condition" | "other" parentIds: string[] childIds: string[] anatomyIds: string[] histologyIds: string[] ncitCodes: string[] icd10Codes: string[] icdoTopographyCodes: string[] icdoMorphologyCodes: string[] doidCodes: string[] oncotreeCodes: string[] umlsCodes: string[] meshIds: string[] mondoIds: string[] malignant: boolean solidTumor: boolean hematologic: boolean pediatricRelevant: boolean rareCancer: boolean description?: string epidemiology?: CancerEpidemiologySummary survival?: CancerSurvivalSummary geneAssociations?: string[] biomarkerAssociations?: string[] drugAssociations?: string[] trialAssociations?: string[] ranking?: CancerRanking provenanceIds: string[] createdAt: string updatedAt: string } ``` --- # 6. CANCERINDEX IDENTIFIERS CancerIndex needs its own stable ID namespace. Examples: ```text CI-CAN-00000001 CI-CAN-00000002 CI-GENE-00000001 CI-VAR-00000001 CI-DRUG-00000001 CI-TRIAL-00000001 CI-PUB-00000001 CI-BIO-00000001 CI-STUDY-00000001 CI-METRIC-00000001 CI-SOURCE-00000001 CI-ORG-00000001 CI-TRT-00000001 ``` Never expose database auto-increment integers as public identifiers. IDs must remain stable forever. --- # 7. ENTITY UNIVERSE CancerIndex should eventually contain first-class entities for: ## Cancer ```text Cancer Cancer subtype Histology Molecular subtype Tumor family Precancerous condition where relevant Metastatic disease state ``` ## Anatomy ```text Organ Tissue Anatomical site Primary site Metastatic site ``` ## Genes ```text Gene Transcript Protein Pathway Gene family ``` ## Genetic alterations ```text SNV MNV Insertion Deletion Indel Fusion Rearrangement Amplification Deletion/CNA Loss of heterozygosity Promoter mutation Splice alteration Expression change Epigenetic alteration Structural variant ``` ## Biomarkers ```text Gene mutation Protein expression Hormone receptor PD-L1 MSI TMB HRD ctDNA methylation gene signature expression signature cell surface marker immune marker ``` ## Drugs ```text small molecule monoclonal antibody ADC bispecific antibody CAR-T cell therapy gene therapy cancer vaccine radiopharmaceutical chemotherapy hormonal therapy immunotherapy targeted therapy ``` ## Treatment concepts ```text drug drug combination surgery radiotherapy brachytherapy proton therapy transplantation cell therapy watchful waiting active surveillance ``` ## Clinical research ```text clinical trial trial arm intervention cohort endpoint study publication investigator institution sponsor ``` ## Population ```text country territory state/province region registry age group sex calendar year ``` --- # 8. THE CONNECTOR PHILOSOPHY CancerIndex must be built around connectors. Do not manually populate the platform except for explicitly curated metadata. Every external system must have an independent connector. Structure: ```text /connectors /nci /gdc /seer /iarc /clinicaltrials /pubmed /clinvar /cbioportal /civic /hgnc /ensembl /chembl /opentargets /dgidb /fda ... ``` Every connector implements something similar to: ```ts interface Connector { id: string name: string discover(): Promise fetch(): Promise normalize(): Promise reconcile(): Promise validate(): Promise persist(): Promise healthCheck(): Promise getCursor(): Promise setCursor(cursor: ConnectorCursor): Promise } ``` --- # 9. CONNECTOR MANIFEST Every connector needs: ```yaml id: name: organization: category: access: type: api|bulk|rss|ftp|graphql|rest|scrape|manual auth: none|api_key|oauth|account|controlled license: terms_reviewed: commercial_use_status: update_frequency: expected_latency: supports_incremental_sync: entities: metrics: rate_limits: retry_policy: raw_retention: schema_version: documentation_verified_at: owner: status: ``` --- # 10. CONNECTOR PRIORITY TIERS ## TIER 0 — FOUNDATIONAL These must be implemented first. ### 10.1 NCI Enterprise Vocabulary Services Purpose: * cancer terminology * canonical disease names * disease aliases * drug concepts * biomarkers * terminology mappings * controlled oncology concepts Use NCI terminology as one of the anchors of the CancerIndex normalization system. Store NCI identifiers on entities. --- ### 10.2 NCI Genomic Data Commons — GDC Use for: * TCGA * TARGET * CPTAC where available through the platform * case metadata * tumor metadata * genomic files * mutation information * copy-number information * expression * molecular features * survival-related analyses * cancer cohorts Connector must support: ```text projects cases files annotations genes mutations CNV metadata manifests ``` Do not ingest controlled-access patient-identifiable/low-level data without the proper authorization model. Prioritize open, aggregated and de-identified data. --- ### 10.3 SEER Purpose: * US cancer incidence * mortality where available/appropriate * survival * age * sex * race/ethnicity where datasets permit * staging * disease classification * registry geography * trends Use SEER for American cancer burden and survival analytics. Never represent a SEER population estimate as a global estimate. --- ### 10.4 IARC Global Cancer Observatory / GLOBOCAN This should be the major global epidemiology layer. Capture when permitted: ```text incidence mortality prevalence age-standardized rates sex country region cancer type year ``` Build CancerIndex country and global rankings from this layer. Terms and redistribution rights MUST be reviewed before automated bulk ingestion. Do not assume that because data are publicly viewable they can automatically be republished wholesale. --- ### 10.5 ClinicalTrials.gov Build an extremely robust ClinicalTrials.gov connector. Capture: ```text NCT ID official title brief title study type phase status conditions interventions arms sponsor collaborators eligibility sex age enrollment locations countries investigators primary outcomes secondary outcomes start date completion date study results references last update ``` CancerIndex must map free-text conditions to canonical cancer IDs. CancerIndex must map: ```text trial → cancer trial → drug trial → biomarker trial → gene trial → institution trial → country ``` Incrementally synchronize changed records. --- ### 10.6 PubMed Massive literature connector. Capture: ```text PMID title abstract authors affiliations journal publication date publication types MeSH DOI references where accessible retractions/corrections ``` Build mappings: ```text publication → cancer publication → gene publication → variant publication → biomarker publication → drug publication → trial ``` Do not make LLM entity extraction authoritative. LLM extraction creates candidate relationships. Those candidates must be labeled accordingly until validated. --- ### 10.7 ClinVar Use for: ```text variants clinical significance conditions review status submitter information variation IDs HGVS genes citations drug response ``` Map cancer-associated ClinVar records into the graph. --- ### 10.8 cBioPortal Use for cancer cohort/genomics exploration. Import where licensing permits: ```text studies patients samples mutations CNA expression clinical attributes survival molecular profiles ``` CancerIndex should preserve original study IDs. --- ### 10.9 CIViC Extremely important for curated clinical interpretation of cancer variants. Map: ```text gene variant molecular profile disease therapy evidence item assertion publication evidence level evidence direction clinical significance ``` Do not flatten CIViC evidence into a binary: ```text works / doesn't work ``` Preserve its structured evidence. --- # 11. TIER 1 — MOLECULAR INTELLIGENCE Implement these after foundational ingest. ## HGNC Canonical human gene nomenclature. Capture: ```text HGNC ID approved symbol approved name aliases previous symbols chromosomal location cross references ``` HGNC should be authoritative for canonical human gene symbol reconciliation. --- ## Ensembl Capture: ```text genes transcripts variants coordinates assemblies regulatory information homology where useful ``` Always retain genome assembly. Never store a coordinate without: ```text assembly chromosome position reference alternate ``` --- ## NCBI Gene Use as additional cross-reference and annotation source. --- ## dbSNP Variant identifiers and genomic cross references. --- ## dbVar Structural variation. --- ## Sequence Ontology Normalize variant types. --- ## Gene Ontology Functional annotation. --- ## UniProt Protein entities and protein annotation. --- ## Reactome Pathways. Build: ```text gene → pathway protein → pathway drug target → pathway cancer → dysregulated pathway ``` --- ## WikiPathways Secondary pathway source. --- ## Protein Data Bank Connect cancer proteins and drug targets to experimental structures. --- ## AlphaFold Protein Structure Database Optional structural biology layer. Do not imply predicted structure equals experimental structure. --- # 12. TIER 2 — DRUG INTELLIGENCE ## ChEMBL Capture: ```text molecules mechanisms targets assays activities indications development phase ``` --- ## Open Targets Use as a disease-target-drug evidence layer. Map: ```text target ↔ disease drug ↔ target evidence source association score ``` Never convert another database's association score directly into a CancerIndex evidence score without documenting transformation. --- ## DGIdb Drug-gene interactions. Preserve contributing source information. --- ## DrugBank Potential connector. **Important:** licensing must be verified before implementation or redistribution. Do not scrape or reproduce licensed content without permission. --- ## PubChem Use for: ```text compound IDs structures synonyms chemical identifiers ``` --- ## DrugCentral Candidate drug information source. Verify current terms and downloadable datasets. --- ## DailyMed Structured FDA label information. Potential fields: ```text drug label indications contraindications warnings dose language adverse reactions manufacturer label version ``` Never paraphrase a drug label and then present the paraphrase as the legal label. --- ## OpenFDA Use where useful for structured FDA data. Potential: ```text labels adverse event aggregates drug metadata ``` Adverse-event reports must include strong caveats. Spontaneous reporting cannot be treated as incidence or causal proof. --- # 13. TIER 3 — REGULATORY INTELLIGENCE Build country-aware regulatory status. Never use: ```text approved = true ``` alone. Use: ```ts interface DrugApproval { drugId: string cancerId?: string biomarkerIds: string[] jurisdiction: | "US" | "CA" | "EU" | "UK" | "AU" | "JP" | "OTHER" authority: | "FDA" | "Health Canada" | "EMA" | "MHRA" | "TGA" | "PMDA" | string indication: string lineOfTherapy?: string diseaseStage?: string approvalType?: string accelerated?: boolean conditional?: boolean approvalDate?: string withdrawalDate?: string status: | "approved" | "conditional" | "accelerated" | "withdrawn" | "superseded" sourceId: string } ``` --- ## FDA Oncology Connect: * oncology approval announcements * Oncology Center of Excellence * Project Confirm * accelerated approvals * withdrawn accelerated approvals * verified clinical benefit * Project Orbis * labeling * regulatory reviews CancerIndex should have an: **Oncology Approval Timeline** --- ## Health Canada CancerIndex is global; Canada must be properly represented. Potential datasets: * Drug Product Database * Notice of Compliance * Summary Basis of Decision * Product Monographs * Project Orbis-related approvals Review redistribution and API availability before implementation. --- ## EMA Capture: ```text European Public Assessment Reports indications marketing authorization authorization dates withdrawals safety changes ``` --- ## MHRA UK regulatory layer. --- ## TGA Australia regulatory layer. --- ## PMDA Japan regulatory layer. --- ## Swissmedic Swiss layer. --- # 14. TIER 4 — GLOBAL EPIDEMIOLOGY Additional country sources should augment IARC rather than blindly override it. Possible connectors: ## WHO Global health statistics and relevant cancer-related datasets. ## CDC US cancer statistics where useful. ## Statistics Canada Canadian mortality/population data. ## Canadian Cancer Statistics Evaluate licensing and machine-readable availability. ## Canadian Cancer Registry Integrate permitted aggregated data where accessible. ## European Cancer Information System European epidemiology. ## EUROCARE European survival research where permitted. ## National cancer registries Build country-specific connectors where high-quality public data exist. Examples: ```text UK Australia New Zealand Nordic countries France Germany Netherlands Japan South Korea Singapore Canada United States ``` Do NOT mix incompatible epidemiological definitions without harmonization. --- # 15. TIER 5 — LITERATURE ## PubMed Primary. ## Europe PMC Use as a complementary literature graph. Potential: ```text abstracts full-text availability citations references grants preprints ``` ## Crossref DOI and publication metadata. ## OpenAlex Useful for: ```text citation graph institutions authors topics research trends ``` Verify licensing/current API conditions. ## Semantic Scholar Potential secondary citation/AI-literature layer. Review API conditions. ## bioRxiv Preprints. ## medRxiv Preprints. Always label preprints prominently. Never rank a preprint as equivalent to peer-reviewed evidence. --- # 16. TIER 6 — CLINICAL GUIDELINES Potential sources: ```text NCI ASCO ESMO NCCN Cancer Care Ontario NICE other national oncology organizations ``` CRITICAL: Guideline copyright and licensing vary substantially. CancerIndex must NOT automatically scrape and reproduce paid/copyrighted guidelines. For restricted sources: store only permitted: ```text citation title publication date organization external reference metadata ``` unless licensing permits more. --- # 17. TIER 7 — PRECISION ONCOLOGY Potential integrations: ```text CIViC ClinVar OncoKB Cancer Genome Interpreter JAX-CKB MolecularMatch My Cancer Genome ``` But: **Licensing must be checked individually.** Never assume commercial reuse. Build the CancerIndex precision oncology layer first from sources with clear reuse rights. --- # 18. TIER 8 — CANCER CELL LINES AND PRECLINICAL DATA Potential sources: ## DepMap ```text cell lines gene dependencies CRISPR screens drug sensitivity molecular features ``` ## Cancer Cell Line Encyclopedia Integrate where licensing permits. ## GDSC Genomics of Drug Sensitivity in Cancer. ## Cell Model Passports Cancer model information. ## PDX resources Patient-derived xenograft data where publicly available. Keep: ```text PRECLINICAL ``` clearly separated from human clinical evidence. --- # 19. TIER 9 — IMMUNO-ONCOLOGY Model: ```text immune checkpoints immune cell populations neoantigens HLA PD-1 PD-L1 CTLA-4 LAG-3 TIGIT TIM-3 TMB MSI immune gene signatures ``` Sources can include: ```text GDC cBioPortal CIViC clinical trials publications Open Targets ``` --- # 20. TIER 10 — PEDIATRIC ONCOLOGY CancerIndex must NOT treat pediatric cancers as simply adult cancers in younger people. Build dedicated pediatric taxonomy. Sources could include: ```text TARGET NCI SEER IARC pediatric resources St. Jude public resources pediatric clinical trials literature ``` Add: ```text age at diagnosis pediatric incidence AYA incidence survival molecular subtype treatment landscape late effects evidence ``` --- # 21. TIER 11 — RARE CANCERS Rare cancers are a core differentiator. CancerIndex should attempt to index cancers even when: ```text incidence < 1 / 100,000 ``` Do not hide them because data are sparse. Create: **Rare Cancer Explorer** Metrics: ```text estimated incidence number of known cases/cohorts number of publications number of clinical trials number of approved therapies number of targeted therapies available genomic studies research activity ``` Data scarcity itself should be displayed. --- # 22. CONNECTOR FALLBACK SYSTEM Preferred connector hierarchy: ```text 1. official API 2. official bulk download 3. official structured feed 4. official database export 5. official static dataset 6. compliant website extraction 7. publication extraction 8. manual curator review ``` Do NOT start by scraping if an API exists. --- # 23. FIRECRAWL + SCRAPFLY CancerIndex may use Firecrawl and Scrapfly for sources without suitable APIs. Architecture: ```text official API ↓ unavailable official bulk ↓ unavailable Firecrawl ↓ blocked / inadequate Scrapfly ``` Firecrawl is primarily useful for: ```text documentation regulatory pages research institution pages structured public pages public tables release notes ``` Scrapfly should be a fallback for technically difficult public pages when use is permitted. Do NOT use anti-bot tooling to bypass: * authentication * paywalls * explicit access restrictions * licensing controls * patient privacy protections * controlled genomic datasets Store source terms/compliance status per connector. --- # 24. CONNECTOR OBSERVABILITY Every connector receives an admin dashboard. Display: ```text status last successful sync last attempt duration records fetched records created records updated records rejected schema drift HTTP failures rate limit events validation failures freshness ``` Example: ```text GDC HEALTHY 11 min ago ClinicalTrials HEALTHY 4 min ago PubMed HEALTHY 8 min ago SEER HEALTHY 2 hr ago FDA DEGRADED 37 min ago IARC REVIEW license check ``` --- # 25. SCHEMA DRIFT DETECTION External APIs change. Every connector must detect: ```text new fields removed fields changed enum values changed types unexpected nullability pagination behavior changes authentication changes ``` When drift occurs: ```text DO NOT silently discard data. ``` Alert the administrator. --- # 26. RAW DATA LAKE Every source payload should be retained when licensing allows. Use object storage: ```text /raw/{source}/{date}/{entity}/{id}.json ``` or compressed batch files. Benefits: * auditability * reproducibility * reprocessing * parser upgrades * debugging * historical snapshots --- # 27. CANONICAL DATA MODEL Core relational tables: ```text cancers cancer_aliases cancer_hierarchy cancer_codes anatomical_sites genes gene_aliases proteins transcripts variants variant_coordinates variant_aliases biomarkers drugs drug_aliases drug_targets drug_indications drug_approvals treatments treatment_regimens clinical_trials trial_conditions trial_interventions trial_locations trial_outcomes trial_eligibility publications authors institutions studies cohorts epidemiology_observations survival_observations cancer_gene_edges cancer_variant_edges cancer_biomarker_edges cancer_drug_edges drug_gene_edges drug_variant_edges trial_cancer_edges publication_entity_edges sources source_records provenance rankings ranking_snapshots ``` --- # 28. KNOWLEDGE GRAPH CancerIndex must be graph-native conceptually, even if PostgreSQL remains the primary transactional database. Graph: ```text Cancer ├── HAS_SUBTYPE → Cancer ├── OCCURS_IN → Anatomy ├── ASSOCIATED_WITH → Gene ├── HAS_VARIANT → Variant ├── HAS_BIOMARKER → Biomarker ├── TREATED_BY → Drug ├── STUDIED_IN → Trial ├── DESCRIBED_BY → Publication └── OBSERVED_IN → Cohort Gene ├── HAS_VARIANT → Variant ├── ENCODES → Protein ├── MEMBER_OF → Pathway └── TARGETED_BY → Drug Variant ├── OCCURS_IN → Cancer ├── PREDICTS_RESPONSE_TO → Drug ├── CONFERS_RESISTANCE_TO → Drug └── SUPPORTED_BY → Evidence Drug ├── TARGETS → Gene ├── APPROVED_FOR → Cancer ├── INVESTIGATED_FOR → Cancer └── USED_IN → Trial ``` Relationships need provenance. --- # 29. EDGE MODEL Never store: ```text EGFR mutation → osimertinib ``` without context. Use: ```ts interface KnowledgeEdge { id: string sourceEntityId: string targetEntityId: string relationshipType: string cancerContextIds?: string[] predictive?: boolean prognostic?: boolean diagnostic?: boolean predisposing?: boolean direction?: | "supports" | "resistance" | "sensitivity" | "neutral" | "unknown" evidenceLevel?: string evidenceScore?: number provenanceIds: string[] firstSeenAt: string lastSeenAt: string } ``` --- # 30. CANCER RANKING ENGINE This is one of the signature features. CancerIndex must rank every eligible cancer across MANY metrics. There must never be one unexplained “danger ranking.” --- # 31. RANKING DIMENSIONS Every cancer can potentially have: ## Burden ```text global incidence count global mortality count global prevalence age-standardized incidence age-standardized mortality DALYs if source available YLL if source available ``` ## Lethality ```text mortality / incidence ratio 1-year survival 5-year survival 10-year survival stage IV survival median OS where meaningful ``` ## Trend ```text incidence CAGR mortality CAGR survival improvement age-adjusted incidence trend ``` ## Rarity ```text global incidence rank incidence per 100k estimated annual patients ``` ## Treatment Landscape ```text number of approved therapies number of targeted therapies number of immunotherapies number of biomarker-directed therapies number of treatment classes ``` ## Clinical Research ```text active trials recruiting trials phase I trials phase II trials phase III trials interventional trial count trial enrollment ``` ## Research Activity ```text publications last 12 months publications last 5 years publication growth citations research institutions ``` ## Molecular Knowledge ```text known recurrent genes actionable variants validated biomarkers genomic studies sequenced cohorts ``` ## Unmet Need Derived carefully from: ```text mortality burden poor survival few approved therapies few active trials low research activity lack of actionable biomarkers ``` --- # 32. RANK EVERY CANCER BY DEFAULT Cancer detail pages should display: ```text Global incidence rank Global mortality rank 5-year survival rank Lethality rank Research activity rank Clinical trial rank Treatment availability rank Genomic knowledge rank Unmet need rank CancerIndex composite rank ``` Example: ```text Pancreatic Adenocarcinoma Mortality burden #7 Incidence burden #14 Lethality #3 Five-year survival #4 poorest Research activity #11 Active trials #15 Treatment options #82 Unmet need #5 CancerIndex Impact #8 ``` These are examples only. Never hardcode examples as actual statistics. --- # 33. RANKING SCOPE Rankings need scope. Example: ```text WORLD CANADA UNITED STATES EUROPE QUEBEC MALE FEMALE CHILDREN AYA AGE 65+ 2024 2025 historical ``` A rank is meaningless without a population and reference year. Schema: ```ts interface Ranking { cancerId: string metricId: string rank: number eligibleEntities: number percentile: number geography: string sex?: string ageGroup?: string year?: number value: number unit: string sourceIds: string[] formulaVersion?: string generatedAt: string } ``` --- # 34. COMPOSITE CANCERINDEX SCORE Create an optional composite metric. Do NOT present it as biological truth. Possible conceptual model: ```text CancerIndex Impact Score ``` 0–100. Possible components: ```text 25% mortality burden 20% lethality 15% incidence burden 15% unmet treatment need 10% adverse trend 10% research deficit 5% clinical trial deficit ``` Weights must be: * visible * versioned * configurable * documented Example: ```text CancerIndex Impact Score v1.0 ``` Display: ```text Score: 87.4 / 100 Rank: 6 / 412 ``` and a breakdown. Never show only 87.4. Show: ```text Mortality burden 93 Lethality 97 Incidence 78 Treatment deficit 84 Research deficit 63 Trend 71 ``` --- # 35. UNCERTAINTY Rankings must account for uncertainty. A rare cancer may have: ```text n = 22 ``` Do not rank survival estimates derived from tiny datasets as equivalent to huge registry datasets. Store: ```text sample size confidence interval standard error estimate method source quality data completeness ``` Optionally display: ```text Ranking confidence HIGH MEDIUM LOW INSUFFICIENT DATA ``` --- # 36. DATA COMPLETENESS SCORE Every cancer gets a completeness profile. Example: ```text Epidemiology 92% Survival 81% Genomics 97% Trials 100% Therapies 93% Biomarkers 88% Literature 100% Pathology 76% ``` This is separate from scientific confidence. --- # 37. RESEARCH GAP INDEX Create a major CancerIndex innovation: ## Research Gap Index Question: > Which cancers have a large burden but disproportionately little research? Possible formula: ```text burden percentile ÷ research activity percentile ``` More sophisticated version: ```text expected research activity = f( incidence, mortality, years_of_life_lost, lethality ) research gap = expected activity - observed activity ``` Display: **Most Under-Researched Cancers** This could be extremely compelling. --- # 38. TRIAL GAP INDEX Another ranking: ```text disease burden vs active interventional trials ``` Identify: > High-burden cancers with few active trials. --- # 39. TREATMENT GAP INDEX Rank cancers based on: ```text mortality survival approved drug count effective targeted treatment count biomarker-directed therapies ``` Again, label as CancerIndex-derived metric. --- # 40. PROGRESS INDEX Create: **Cancer Progress Index** Track over 5/10/20 years: ```text mortality improvement survival improvement treatment approvals trial growth biomarker growth research growth ``` Show: ```text Most rapidly improving cancers Least improving cancers ``` --- # 41. MOMENTUM INDEX Short-term research momentum. Components: ```text new trials new publications new drugs new FDA approvals new biomarkers new genomic studies ``` Windows: ```text 30 days 90 days 1 year 5 years ``` --- # 42. CANCER ENTITY DETAIL PAGE Route: ```text /cancer/{slug} ``` Example layout: ```text ┌─────────────────────────────────────────────┐ │ Pancreatic Ductal Adenocarcinoma │ │ PDAC │ │ CI-CAN-0000342 │ └─────────────────────────────────────────────┘ CancerIndex Score 89.2 Global Rank #5 Tabs: Overview Statistics Survival Stages Genomics Genes Variants Biomarkers Treatments Drugs Trials Research Publications Risk Factors Screening Prevention Countries Trends Sources ``` --- # 43. CANCER OVERVIEW HERO Show: ```text Global annual cases Global annual deaths 5-year survival median diagnosis age male/female distribution Impact rank Mortality rank Lethality rank Research rank Unmet need rank ``` Never display unsupported values. --- # 44. CANCER SUMMARY AI-generated summary should contain: ```text What it is Where it originates Major subtypes Epidemiology Typical molecular features Major treatment modalities Current research landscape ``` Every paragraph needs citations. AI summaries must be cached with: ```text model prompt version source snapshot generation date ``` --- # 45. GLOBAL CANCER RANKING PAGE Route: ```text /rankings ``` Filters: ```text metric year country region sex age cancer category minimum cases data confidence ``` Columns: ```text Rank Cancer Score/value Cases Deaths Mortality/incidence 5-year survival Active trials Publications Trend ``` --- # 46. RANKING PRESETS Routes or presets: ```text /rankings/incidence /rankings/mortality /rankings/lethality /rankings/survival /rankings/research /rankings/trials /rankings/treatment-gap /rankings/research-gap /rankings/momentum /rankings/progress /rankings/rare-cancers ``` --- # 47. COUNTRY PAGES Route: ```text /country/canada /country/united-states /country/france ``` Display: ```text population annual cancer cases annual cancer deaths ASIR ASMR Top cancers by incidence Top cancers by mortality male female historical trend age distribution ``` Map visualization. --- # 48. GLOBAL CANCER MAP Interactive world map. Filters: ```text cancer incidence mortality ASR sex year age ``` Click country → country dashboard. --- # 49. GENE PAGES Route: ```text /gene/TP53 ``` Display: ```text gene overview HGNC identity chromosome protein pathways cancers variants mutation frequencies biomarkers therapies clinical trials publications ``` --- # 50. VARIANT PAGES Example: ```text /variant/BRAF-V600E ``` Display: ```text gene HGVS protein change coordinates by assembly ClinVar CIViC cancers frequencies drug sensitivity evidence drug resistance evidence clinical trials publications ``` Separate evidence by cancer. BRAF V600E in one cancer must not automatically inherit evidence from another cancer. --- # 51. BIOMARKER PAGES Examples: ```text PD-L1 MSI-H TMB-high HER2 HRD ER PR PSMA ctDNA ``` Display: ```text definition measurement method cancers therapies FDA-approved indications clinical evidence trials publications ``` --- # 52. DRUG PAGES Route: ```text /drug/osimertinib ``` Hero: ```text Generic name Brand names Drug class Targets Mechanism Developer First approval Current jurisdictions ``` Tabs: ```text Overview Mechanism Targets Cancer indications Biomarkers Approvals Clinical trials Publications Combinations Resistance Safety Sources ``` --- # 53. DRUG COMBINATION ENTITY Do not treat: ```text Drug A + Drug B ``` as two unrelated drugs. Create: ```text TreatmentRegimen ``` Examples: ```text FOLFOX FOLFIRINOX R-CHOP ABVD drug A + drug B ``` --- # 54. CLINICAL TRIAL PAGES Route: ```text /trial/NCT... ``` Display: ```text status phase title cancers biomarkers interventions enrollment sponsor locations eligibility dates outcomes publications results ``` --- # 55. TRIAL MATCH EXPLORER Research use only. Filters: ```text cancer stage gene variant biomarker drug phase country recruiting status age sex ``` Do not claim a patient is eligible solely from automated filtering. Use: > Potentially relevant trials — verify full eligibility criteria with the study team. --- # 56. PUBLICATION PAGES Route: ```text /publication/{pmid} ``` Show: ```text title authors journal date abstract DOI publication type linked cancers linked genes linked variants linked drugs linked trials ``` AI: ```text structured research summary ``` Only where legally permitted from available text. --- # 57. RESEARCHER PAGES Optional later stage: ```text /researcher/{id} ``` Metrics: ```text oncology publications cancers studied genes studied clinical trials citations institutions ``` Avoid misleading researcher ranking based on raw citation count alone. --- # 58. INSTITUTION PAGES Examples: ```text MD Anderson Memorial Sloan Kettering Dana-Farber Princess Margaret Mayo Clinic Gustave Roussy ``` Automatically derived from: ```text trials authors affiliations publications ``` Rank institutions by transparent criteria, not prestige claims. --- # 59. CANCER RESEARCH DASHBOARD Route: ```text /research ``` Show: ```text publications/year trials/year new drugs/year new targets/year new biomarkers/year research funding when reliable data exists ``` --- # 60. RESEARCH TRENDS Detect emerging topics. Examples: ```text KRAS G12D T-cell engagers ADC ctDNA personalized vaccines radioligand therapy CAR-T in solid tumors ``` Do not hardcode trends. Calculate from publication/trial growth. --- # 61. CANCER NEWS Potential later connector layer: ```text FDA NCI NIH major journals cancer centers regulators clinical trial updates ``` Use original source and publication timestamp. AI clustering: ```text multiple reports → one story cluster ``` --- # 62. AI — ASK CANCERINDEX CancerIndex should contain a research assistant. Route: ```text /ask ``` Example questions: ```text Which cancers have the highest mortality-to-incidence ratio? What are the most frequent genomic alterations in LUAD? Compare KRAS G12C in lung and colorectal cancer. Which recruiting Phase III trials are testing therapies for pancreatic cancer? Which rare cancers have the fewest active clinical trials relative to incidence? What cancers have seen the largest improvement in survival over 20 years? ``` --- # 63. AI MUST QUERY STRUCTURED DATA FIRST Never do: ```text question ↓ LLM general knowledge ↓ answer ``` Do: ```text question ↓ intent parser ↓ CancerIndex query plan ↓ SQL / graph / search ↓ source records ↓ LLM synthesis ↓ citations ``` --- # 64. AI ANSWER CONTRACT Every answer returns: ```json { "answer": "...", "entities": [], "citations": [], "data_as_of": "...", "confidence": "...", "limitations": [] } ``` --- # 65. AI CITATIONS Every important assertion must point to: ```text source dataset publication or regulatory record ``` Click citation → source drawer. Source drawer: ```text Source Organization Dataset Version Record Retrieved Raw value Normalized value Transformation ``` --- # 66. AI MODEL PROVIDER ABSTRACTION Do not couple the application to one LLM. Interface: ```ts interface LLMProvider { generate() stream() structuredOutput() embed() } ``` Support configurable providers. Possible: ```text OpenAI Anthropic Google xAI local OpenAI-compatible endpoint ``` CancerIndex should operate without requiring AI for core database functionality. --- # 67. EMBEDDINGS Generate embeddings for: ```text cancer descriptions publication abstracts trial descriptions drug mechanisms biomarker descriptions evidence summaries ``` Use pgvector initially. Store embedding model/version. Never mix embeddings generated by incompatible models in one vector column without model metadata. --- # 68. SEARCH ENGINE Global search should support: ```text cancers subtypes genes variants biomarkers drugs trials publications institutions researchers ``` Examples: ```text panc → Pancreatic Cancer → Pancreatic Ductal Adenocarcinoma G12C → KRAS G12C HER2 low → HER2-low Breast Cancer → HER2-low biomarker concept ``` Implement: ```text exact alias prefix fuzzy semantic cross-entity ``` --- # 69. ENTITY RECONCILIATION ENGINE This is one of the hardest parts. Example source names: ```text NSCLC Non-small-cell lung cancer Non Small Cell Lung Carcinoma non-small cell carcinoma of lung ``` must resolve appropriately. Use: ```text exact IDs ontology mappings canonical aliases normalized strings context LLM only as fallback candidate generator human review ``` Never merge two cancer entities solely because embeddings are similar. --- # 70. ENTITY MERGE QUEUE Admin system: ```text Possible duplicate Cancer A Cancer B Evidence: name similarity: 0.94 NCIt match: yes OncoTree match: yes [MERGE] [KEEP SEPARATE] [REVIEW] ``` Every merge must be auditable and reversible. --- # 71. TEMPORAL DATA Every observation is time-aware. Never overwrite: ```text incidence 2022 ``` with: ```text incidence 2024 ``` Store both. Core observation: ```ts interface EpidemiologyObservation { cancerId: string geographyId: string year: number sex?: string ageGroup?: string metric: | "incidence_count" | "incidence_rate" | "as_incidence_rate" | "mortality_count" | "mortality_rate" | "as_mortality_rate" | "prevalence" value: number unit: string lowerCI?: number upperCI?: number sourceId: string } ``` --- # 72. SURVIVAL DATA MODEL Survival requires context. Never store simply: ```text survival = 32% ``` Use: ```ts interface SurvivalObservation { cancerId: string geographyId?: string stage?: string sex?: string ageGroup?: string diagnosisPeriod?: string survivalType: | "overall" | "relative" | "cause_specific" | "progression_free" | "disease_free" durationMonths: number probability?: number medianMonths?: number cohortSize?: number lowerCI?: number upperCI?: number sourceId: string } ``` --- # 73. STAGING CancerIndex must support multiple staging systems. Do not pretend all cancers use identical Stage I–IV systems. Model: ```text AJCC/TNM FIGO Ann Arbor Lugano Durie-Salmon ISS/R-ISS Binet Rai disease-specific systems ``` Licensing must be checked before reproducing proprietary staging definitions. --- # 74. RISK FACTORS Risk factor entities: ```text smoking alcohol UV obesity infection occupational exposure radiation genetic predisposition hormonal factors age ``` Relations require evidence. Example: ```text RiskFactor → ASSOCIATED_WITH → Cancer ``` Store: ```text relative risk odds ratio hazard ratio population attributable fraction confidence interval study ``` Do not translate association into causality automatically. --- # 75. HEREDITARY CANCER Dedicated hereditary layer. Entities: ```text germline gene syndrome variant cancer risk penetrance estimate ``` Examples conceptually: ```text BRCA1 BRCA2 Lynch syndrome TP53/Li-Fraumeni APC/FAP VHL ``` Use trusted genetic sources. Strong warning: CancerIndex must not interpret a user's personal germline result as medical advice. --- # 76. SCREENING Store: ```text screening method eligible population cancer country organization recommendation date evidence level ``` Guidelines are geography and organization specific. Never display a universal screening recommendation when there isn't one. --- # 77. PREVENTION Represent prevention evidence separately. Potential: ```text vaccination smoking cessation UV protection risk-reducing surgery screening infection prevention occupational exposure reduction ``` --- # 78. PATHOLOGY Future module. Data: ```text histology pathology images stains IHC morphology grade ``` Use public datasets with explicit image usage rights. --- # 79. RADIOLOGY Future module. Potential public datasets: ```text TCIA and other properly licensed collections ``` Separate: ```text CT MRI PET X-ray ultrasound ``` Do not expose patient-identifiable DICOM metadata. --- # 80. CANCER INDEX API Public API: ```text api.cancerindex.io ``` Version: ```text /v1/ ``` Possible endpoints: ```text GET /v1/cancers GET /v1/cancers/{id} GET /v1/cancers/{id}/statistics GET /v1/cancers/{id}/survival GET /v1/cancers/{id}/genes GET /v1/cancers/{id}/variants GET /v1/cancers/{id}/drugs GET /v1/cancers/{id}/trials GET /v1/cancers/{id}/publications GET /v1/genes GET /v1/genes/{symbol} GET /v1/variants/{id} GET /v1/drugs GET /v1/drugs/{id} GET /v1/trials/{nct} GET /v1/rankings ``` --- # 81. GRAPHQL Consider later: ```text /graphql ``` Example conceptual query: ```graphql cancer(id: "CI-CAN-...") { name ranking { incidence mortality } genes { gene { symbol } frequency } trials(status: RECRUITING) { nctId phase } } ``` --- # 82. MCP SERVER CancerIndex should eventually expose an MCP server. Purpose: Allow AI agents to query CancerIndex directly. Tools: ```text search_cancers get_cancer rank_cancers get_epidemiology get_survival get_gene get_variant get_drug search_trials search_publications query_knowledge_graph ``` Read-only initially. --- # 83. BULK DATA Eventually provide permitted CancerIndex-derived datasets. Formats: ```text CSV JSON JSONL Parquet ``` Never redistribute restricted upstream source material. --- # 84. ARCHITECTURE Recommended: ```text Next.js TypeScript React PostgreSQL pgvector ClickHouse Redis OpenSearch or Elasticsearch MinIO/S3 Python ingestion workers FastAPI scientific services Temporal or durable job orchestration ``` Graph: Start with relational edge tables. Introduce Neo4j/Memgraph only if graph workloads justify operational complexity. Do not add infrastructure merely because it sounds sophisticated. --- # 85. POSTGRESQL Primary store for: ```text canonical entities relationships users API metadata source registry provenance admin ranking snapshots ``` --- # 86. CLICKHOUSE Use for large analytical observations: ```text epidemiology variant frequencies publication timelines trial timelines ranking datasets event logs ``` --- # 87. OPENSEARCH Use for global full-text search. Indexes: ```text cancers genes variants drugs trials publications ``` --- # 88. OBJECT STORAGE Use for: ```text raw connector snapshots bulk source archives large datasets export files images where licensed ``` --- # 89. REDIS Use for: ```text hot cache rate limiting jobs distributed locks temporary AI streams ``` Do not use Redis as canonical storage. --- # 90. INGESTION JOB SYSTEM Every ingest must be restartable. Use: ```text connector ↓ discovery ↓ fetch ↓ raw persist ↓ parse ↓ validate ↓ normalize ↓ reconcile ↓ canonical persist ↓ index ↓ derived metrics ↓ ranking recompute ``` --- # 91. IDEMPOTENCY Running a connector twice must not duplicate data. Use source-native IDs. Example: ```text source = PubMed source_record_id = 12345678 ``` Unique constraint. --- # 92. SOFT DELETION Sources can retract or remove records. Never immediately hard-delete. Use: ```text active deprecated retracted withdrawn source_missing ``` Retain history. --- # 93. PUBLICATION RETRACTIONS CancerIndex must track retracted publications when data permit. Relationships based only on retracted evidence should be flagged. --- # 94. DATA FRESHNESS Every page needs: ```text Data updated Source updated CancerIndex synchronized ``` Example: ```text Clinical trials updated: today Genomics updated: Aug 2026 Global incidence dataset: 2024 estimate ``` --- # 95. SOURCE PAGE Route: ```text /source/{source} ``` Display: ```text provider description dataset access method last sync records coverage license status data version connector health ``` Transparency is a feature. --- # 96. CHANGE HISTORY Every entity should support change history. Example: ```text Aug 19: FDA approval added Aug 14: 3 new trials Aug 10: GDC mutation frequency refreshed Aug 03: CancerIndex score changed 84.1 → 84.7 ``` --- # 97. CANCER WATCH Users can follow: ```text cancer gene variant drug trial ``` Notifications: ```text new clinical trial trial status change FDA approval publication new genomic finding ranking change ``` --- # 98. USER ACCOUNTS Account system: ```text email password email verification password reset session management ``` Optional: ```text Google Apple ORCID ``` Do not store sensitive health profiles by default. --- # 99. RESEARCH WORKSPACE Users can save: ```text cancers genes variants drugs trials papers queries charts ``` Create collections: ```text "My KRAS research" "Rare sarcomas" "Pancreatic cancer trials" ``` --- # 100. COMPARISON ENGINE Route: ```text /compare ``` Compare up to several cancers. Example: ```text Pancreatic cancer Glioblastoma Lung adenocarcinoma Melanoma ``` Compare: ```text incidence mortality survival trends genes biomarkers treatments trials research ``` --- # 101. VISUALIZATION SYSTEM CancerIndex should be visually exceptional. Visualizations: ```text ranked bar charts time-series survival curves heatmaps world maps genomic frequency plots co-occurrence matrices oncoprints trial timelines drug approval timelines knowledge graphs Sankey charts bubble plots ``` Charts need: ```text source unit population time period download ``` --- # 102. KNOWLEDGE GRAPH UI Users can start from: ```text KRAS ``` and visually explore: ```text KRAS ├── G12C │ ├── NSCLC │ ├── colorectal cancer │ ├── therapies │ └── trials ├── G12D ├── G12V └── pathways ``` Click nodes dynamically. Avoid rendering thousands of nodes at once. --- # 103. DESIGN SYSTEM CancerIndex must NOT look like a generic SaaS dashboard. Target aesthetic: ```text scientific editorial premium institutional modern high-information-density trustworthy ``` Think: ```text Nature Bloomberg Our World in Data high-end scientific visualization ``` Avoid: ```text giant gradients everywhere dozens of rounded cards cartoon health icons generic AI sparkle graphics ``` --- # 104. COLOR Base: ```text off-white / white deep charcoal muted scientific neutrals ``` Cancer-specific color coding can exist but must not compromise accessibility. Never rely on color alone. --- # 105. HOME PAGE Hero: ```text CancerIndex The global index of cancer. Explore every cancer. Rank global burden. Follow treatments. Search genomics. Track clinical research. ``` Global search immediately visible. Below: ```text Cancer burden today Global rankings Fastest-rising cancers Highest mortality Poorest survival Most active research Largest treatment gaps Rare cancers Latest oncology approvals New clinical trials ``` --- # 106. LIVE DATA TICKER Tasteful top-line statistics: ```text Cancer entities indexed Genes indexed Variants indexed Clinical trials Publications Drug indications Countries Sources ``` Values must come from database counts. --- # 107. "ALL CANCERS" EXPLORER Route: ```text /cancers ``` Do not show only a few dozen cards. Build a powerful explorer. Filters: ```text anatomical system histology solid/hematologic adult/pediatric rare/common molecular subtype incidence mortality survival research level trial count treatment availability ``` Support thousands of entities. --- # 108. TAXONOMY EXPLORER Tree/browser: ```text Blood Breast CNS Digestive Endocrine Gynecologic Head & Neck Lung Skin Soft tissue Urinary ... ``` Also: ```text histology view molecular view WHO view NCI view ``` --- # 109. RARE CANCER DISCOVERY Feature: **Random Rare Cancer** Useful for discovery. Shows: ```text what it is annual incidence known cases/data research count trials genes treatments ``` --- # 110. DATA QUALITY ENGINE Every normalized record runs validation. Examples: ```text incidence >= 0 deaths >= 0 survival between 0 and 1 year reasonable country valid gene symbol canonical variant syntax valid where possible trial phase enum recognized ``` --- # 111. CROSS-SOURCE CONFLICTS Sources will disagree. Never silently average everything. Store each observation. Example: ```text Source A: 5-year survival = 31% Source B: 5-year survival = 36% ``` CancerIndex may compute a harmonized estimate only with a documented method. Show: ```text Why estimates differ ``` --- # 112. EVIDENCE ENGINE Create CancerIndex evidence hierarchy. Possible dimensions: ```text study design sample size replication publication quality clinical relevance regulatory validation expert curation recency ``` Do NOT reduce all scientific truth to one score. Use multi-dimensional evidence badges. --- # 113. CLINICAL EVIDENCE LABELS Example: ```text REGULATORY APPROVED GUIDELINE SUPPORTED PHASE III PHASE II PHASE I RETROSPECTIVE CLINICAL CASE SERIES CASE REPORT PRECLINICAL COMPUTATIONAL ``` --- # 114. STATISTICAL INTEGRITY Never calculate survival by dividing unrelated values. Never compare crude incidence with age-standardized incidence without labeling. Never mix: ```text incidence prevalence mortality case fatality overall survival relative survival ``` Every metric needs precise definition. --- # 115. CANCER "DEADLINESS" Avoid an undefined “deadliest cancer” metric. The interface should let users choose: ```text Most deaths Highest mortality rate Highest mortality/incidence ratio Lowest 5-year survival Highest CancerIndex Impact ``` This distinction is important. --- # 116. AGE STANDARDIZATION For international comparison prioritize appropriately standardized rates. Store standard population used if source provides it. Do not present crude rates as directly comparable across countries with radically different age structures. --- # 117. GEOGRAPHIC NORMALIZATION Canonical geography entity: ```text ISO country ISO subdivision region continent WHO region IARC region if appropriate ``` Keep source geography separately. --- # 118. CURRENCY Not central initially. If later adding: ```text drug cost economic burden research funding ``` always store: ```text currency year country nominal/real source ``` --- # 119. RESEARCH FUNDING Future innovation: Connect: ```text NIH RePORTER CIHR EU grants UKRI other public grants ``` Then create: ```text funding by cancer funding per annual death funding per incident case ``` Potential: **Funding Gap Index** But methodology must be transparent. --- # 120. NIH REPORTER CONNECTOR Potential high-priority future connector. Map grant: ```text project principal investigator institution funding amount year cancer gene topic publication ``` --- # 121. PATENTS Potential future module. Sources: ```text USPTO EPO Google Patents metadata where appropriate ``` Use to map therapeutic innovation. Not required for MVP. --- # 122. COMPANY PIPELINE Potential future module: ```text biotech pharma drug candidate target phase indication ``` Sources must be verified. Public company claims should not override trial registries/regulatory sources. --- # 123. DRUG DEVELOPMENT PIPELINE Statuses: ```text preclinical Phase I Phase I/II Phase II Phase II/III Phase III submitted approved discontinued withdrawn ``` Status may be disease-specific. --- # 124. FAILURE DATABASE Extremely valuable. Track oncology programs that fail or stop. Sources: ```text ClinicalTrials.gov status regulatory documents company releases publications ``` Create: ```text Drug → Cancer → Development outcome ``` Avoid inferring failure solely from stale trial status. --- # 125. RESISTANCE DATABASE Track mechanisms: ```text primary resistance acquired resistance ``` Relations: ```text Variant → confers resistance → Drug Pathway → resistance mechanism → Drug ``` Evidence-backed only. --- # 126. METASTASIS DATABASE Map: ```text primary cancer → common metastatic locations ``` Store frequency only with cohort context. Do not generalize from small cohorts. --- # 127. MULTI-OMICS Future coverage: ```text genome transcriptome epigenome proteome metabolome single-cell spatial ``` GDC and other public research repositories can seed this layer. --- # 128. SINGLE-CELL CANCER DATA Future connector candidates: ```text CELLxGENE Human Tumor Atlas Network resources public scRNA-seq studies ``` Must support: ```text study sample cell type cancer gene expression ``` Large matrices should not live in PostgreSQL. --- # 129. HUMAN TUMOR ATLAS Potential high-value research connector where datasets and terms permit. --- # 130. PROTEOMICS CPTAC-related data should connect: ```text cancer protein phosphoprotein genomic alteration clinical outcome ``` --- # 131. MICROBIOME / CANCER Future experimental research category. Clearly label exploratory evidence. --- # 132. ENVIRONMENTAL EXPOSURES Possible integration: ```text IARC carcinogen classifications occupational exposure datasets air pollution data ``` Do not infer personal cancer risk. --- # 133. CARCINOGEN ENTITY Create: ```text Carcinogen ``` Relations: ```text Carcinogen → evidence of association → Cancer ``` Store classification authority. --- # 134. INFECTIOUS ONCOLOGY Entities: ```text HPV HBV HCV EBV H. pylori HHV-8 etc. ``` Map to cancer evidence. --- # 135. CANCER PREVALENCE FORECASTS Can later model forecasts. But clearly label: ```text OBSERVED ESTIMATED PROJECTED ``` Never make projections visually indistinguishable from observed registry data. --- # 136. FORECAST ENGINE Potential: ```text incidence forecast mortality forecast trial activity forecast research momentum ``` Version each model. Display uncertainty intervals. --- # 137. DATA SNAPSHOTS Monthly immutable snapshots: ```text CancerIndex 2026-09 CancerIndex 2026-10 ``` Allows reproducibility. --- # 138. DATA RELEASES Publish: ```text CancerIndex Data Release 1 ``` With: ```text new sources updated sources entity changes ranking methodology changes known limitations ``` --- # 139. API VERSIONING Never break existing clients casually. Use: ```text /v1 /v2 ``` Data release version separate from API version. --- # 140. ADMIN CONTROL CENTER Route: ```text /admin ``` Sections: ```text Overview Connectors Ingestion Entities Reconciliation Rankings Evidence Sources Licensing Users AI Jobs Search System ``` --- # 141. CONNECTOR ADMIN For every connector: ```text Run now Pause Resume Backfill Incremental sync Dry run View raw records View parser View errors View schema changes ``` --- # 142. LICENSE REGISTRY Create internal table: ```text source_license ``` Fields: ```text source license commercial use redistribution derivative works attribution requirements API terms review date notes approved for production ``` No new connector becomes public until licensing status is reviewed. --- # 143. SOURCE PRIORITY When sources conflict, do not blindly implement a global precedence. Precedence depends on field. Examples: ```text Gene official symbol → HGNC Clinical trial registration → ClinicalTrials.gov US regulatory status → FDA global burden estimate → selected IARC dataset US registry survival → SEER variant clinical curation → retain multiple curated sources ``` --- # 144. ENTITY LINEAGE Every canonical field may need: ```text derivedFromSourceRecordIds ``` Example: ```text canonical name: "Lung Adenocarcinoma" supported by: NCIt SEER OncoTree GDC ``` --- # 145. CACHING Cache expensive: ```text rankings global aggregates country dashboards AI answers knowledge graph layouts ``` Invalidation should be event-driven when possible. --- # 146. PERFORMANCE Targets: ```text homepage < 2 sec meaningful render search suggestions < 200 ms cached target common API reads < 300 ms target ranking query < 500 ms target ``` Do not block page rendering on AI generation. --- # 147. SEO CancerIndex has enormous programmatic SEO potential. Pages: ```text /cancer/{cancer} /cancer/{cancer}/survival /cancer/{cancer}/statistics /cancer/{cancer}/genes /cancer/{cancer}/trials /gene/{gene} /drug/{drug} /variant/{variant} /country/{country} ``` Every generated page must contain substantive sourced information. No thin spam pages. --- # 148. STRUCTURED DATA Use relevant Schema.org structured metadata where appropriate: ```text MedicalCondition Drug Dataset ScholarlyArticle Organization ``` Verify current specifications before implementation. --- # 149. ACCESSIBILITY WCAG-minded implementation. Requirements: ```text keyboard navigation screen reader labels contrast chart text alternatives color-independent state reduced motion ``` --- # 150. INTERNATIONALIZATION English first. Architecture must support: ```text French Spanish German Portuguese Japanese etc. ``` Canonical scientific entity remains language-independent. Translations are attributes. --- # 151. LOCALIZATION Important distinction: ```text language != geography ``` French Canadian user can view Canadian data. French user can view France data. --- # 152. TESTING REQUIREMENTS Claude must create: ```text unit tests integration tests connector fixture tests schema tests ranking tests reconciliation tests API contract tests UI tests end-to-end tests ``` --- # 153. CONNECTOR FIXTURES Never run all connector tests against production APIs. Store sanitized fixtures. Test: ```text normal response empty response pagination rate limit server error schema change malformed record duplicate record ``` --- # 154. SCIENTIFIC REGRESSION TESTS Create invariant tests. Examples: ```text survival >= 0 survival <= 1 incidence >= 0 mortality >= 0 lowerCI <= estimate estimate <= upperCI ``` --- # 155. RANKING TESTS Given fixed fixture inputs, ranking output must be deterministic. Snapshot: ```text ranking methodology version input snapshot output ``` --- # 156. RECONCILIATION TESTS Known aliases: ```text NSCLC non-small cell lung cancer ``` should behave correctly. Known distinct diseases must never merge accidentally. Build a large gold-standard mapping fixture. --- # 157. AI EVALUATION SUITE Create fixed questions: ```text What is PDAC? Compare LUAD and SCLC. What cancers are associated with BRAF V600E? What recruiting trials exist for X? ``` Evaluate: ```text citation correctness entity correctness numerical correctness unsupported statements source freshness ``` --- # 158. HALLUCINATION DEFENSE AI must say: ```text CancerIndex does not currently have sufficient sourced data to answer this. ``` instead of guessing. --- # 159. NO SILENT FALLBACK TO MODEL KNOWLEDGE If CancerIndex retrieval finds no evidence: do not silently answer using model memory. Model knowledge may only be used as clearly labeled supplementary context if product policy explicitly allows it. Default: **database-grounded answers only.** --- # 160. SECURITY Protect: ```text API keys database credentials connector credentials LLM keys admin routes worker endpoints ``` Use environment variables/secrets. Never commit secrets. --- # 161. USER PRIVACY CancerIndex does not require personal health data to be useful. Avoid collecting: ```text diagnosis genetic results treatment history medical documents ``` unless a future clearly separated healthcare feature has proper privacy architecture. --- # 162. ANALYTICS PRIVACY Do not log sensitive search queries unnecessarily. Provide privacy-preserving analytics. --- # 163. AUTHORIZATION Roles: ```text USER RESEARCHER CURATOR ADMIN SUPERADMIN ``` Curators can edit scientific metadata. Every curator action is logged. --- # 164. CURATION PLATFORM Allow expert curators to: ```text merge entities split entities add aliases correct mappings flag evidence resolve conflicts add citations approve AI extractions ``` --- # 165. AI CURATION QUEUE LLM pipeline may discover candidate: ```text publication → gene publication → cancer publication → drug ``` Confidence: ```text >0.98 auto-accept only for low-risk deterministic mappings 0.80–0.98 review <0.80 reject/manual ``` Thresholds must be evaluated empirically. Do not use these exact numbers blindly. --- # 166. EXTRACTION ENGINE For publications: ```text abstract ↓ NER ↓ ontology mapping ↓ relationship extraction ↓ confidence ↓ validation ↓ graph edge ``` Prefer deterministic identifiers when present. --- # 167. PDF INGESTION Some sources publish PDFs. Pipeline: ```text PDF ↓ native text extraction ↓ layout understanding ↓ table extraction ↓ OCR only when necessary ↓ structured JSON ↓ validation ``` Store page-level citations. --- # 168. TABLE EXTRACTION AI-extracted numeric tables must pass validations. Do not accept: ```text OCR number → production statistic ``` without confidence checks. --- # 169. DATA DIFFS On source refresh: ```text previous snapshot vs current snapshot ``` Generate: ```text new records removed records changed values new enums ``` Store diff. --- # 170. ALERTS Internal alerts: ```text connector failure stale source ranking anomaly mass entity deletion schema drift unexpected record drop license review due ``` --- # 171. ANOMALY DETECTION Example: ```text GDC records yesterday: 2,430,000 today: 214 ``` Do NOT publish a destructive update. Pause ingest and alert. --- # 172. BACKUPS Automated: ```text PostgreSQL backups object storage versioning search reindex ability configuration backups ``` Test restore process. --- # 173. INFRASTRUCTURE / CLUSTER DEPLOYMENT CancerIndex should be containerized. Use: ```text Docker ``` Services should be independently deployable. Suggested: ```text web api worker-ingest worker-ai worker-ranking postgres redis clickhouse opensearch minio ``` If deploying to an existing cluster, keep configuration portable. --- # 174. DOMAIN Production: ```text www.cancerindex.io cancerindex.io api.cancerindex.io ``` Optional: ```text status.cancerindex.io docs.cancerindex.io ``` --- # 175. OBSERVABILITY Use: ```text structured logs metrics distributed traces error monitoring job monitoring ``` Every request gets correlation ID. Every ingest gets run ID. --- # 176. INGEST RUN ID Example: ```text ING-CLINICALTRIALS-20260908-000019 ``` Every created/updated record can reference ingest run. --- # 177. CANCERINDEX SCORE VERSIONING Example: ```text CI-IMPACT-v1.0 CI-RESEARCH-GAP-v1.0 CI-TRIAL-GAP-v1.0 CI-PROGRESS-v1.0 CI-MOMENTUM-v1.0 ``` Never silently change formula. --- # 178. METHODOLOGY PAGE Route: ```text /methodology ``` Explain: ```text sources normalization ranking age standardization survival research metrics trial metrics composite indexes uncertainty limitations ``` The methodology page should be exceptionally detailed. --- # 179. PUBLIC REPRODUCIBILITY For each ranking: button: ```text Methodology ``` Display formula. Potential later: ```text Download input dataset Download ranking dataset ``` where licensing permits. --- # 180. DATA SOURCE BADGES On values: ```text IARC SEER GDC FDA ClinicalTrials.gov CIViC ``` Hover → metadata. --- # 181. CITATION UX Citation: ```text [1] ``` click opens side panel rather than sending user away immediately. Panel: ```text Source Original title Dataset Record Date Method Open source ``` --- # 182. CONFIDENCE UX Examples: ```text High confidence Moderate confidence Limited evidence Sparse data ``` Do not hide uncertainty. --- # 183. "WHY THIS RANK?" Every CancerIndex rank gets: ```text Why #4? ``` Click: ```text Mortality burden +23.3 Lethality +18.9 Treatment gap +14.1 Research gap +9.7 Trend +8.2 ... ``` --- # 184. HISTORICAL RANKS Store ranking snapshots. Graph: ```text 2015 #12 2018 #11 2021 #9 2024 #8 ``` Important: methodology consistency must be maintained or explicitly annotated. --- # 185. USER-CUSTOM RANKINGS Advanced feature. Allow users to set weights: ```text Mortality 40% Incidence 20% Survival 20% Research gap 20% ``` Generate: ```text Custom Cancer Index ``` Do not overwrite official CancerIndex ranking. --- # 186. DATA EXPLORER Advanced SQL-like analytics UI without exposing raw SQL. Dimensions: ```text cancer country year sex age ``` Measures: ```text cases deaths ASIR ASMR survival trials publications ``` --- # 187. CHART BUILDER Users choose: ```text X = year Y = mortality Group = cancer Country = Canada ``` Generate shareable chart. --- # 188. EMBEDDABLE CHARTS Future: ```text embed.cancerindex.io/chart/{id} ``` Attribution required. --- # 189. SHAREABLE RESEARCH CARDS Generate beautiful cards: ```text Pancreatic Cancer #3 Lethality #7 Global Mortality 5-year survival ... ``` Always include date/source. --- # 190. PUBLIC DATA API KEYS API account: ```text free research pro institutional ``` Do not monetize third-party data contrary to source licenses. Value can come from CancerIndex aggregation, normalization and infrastructure where permitted. --- # 191. RATE LIMITING API: ```text anonymous authenticated paid/institutional ``` Return standard rate-limit headers. --- # 192. DEVELOPER PORTAL Route: ```text /developers ``` Include: ```text API docs OpenAPI authentication examples schema changelog status ``` --- # 193. DATA DOWNLOAD CENTER Route: ```text /data ``` List datasets CancerIndex is legally allowed to redistribute. --- # 194. SOURCE LICENSE AUTOMATION Crawler can periodically detect source terms changes. But: AI cannot make final legal determination. Flag for human review. --- # 195. RELEASE BOT Weekly report: ```text CancerIndex Weekly Data Report +43 cancers/subtypes +12,328 publications +184 trials +2 FDA approvals +91,224 variant relations 3 connector warnings ``` --- # 196. FRONT PAGE DAILY UPDATE Show: ```text Updated X minutes ago ``` only for sources actually refreshed that recently. Do not imply global dataset freshness because ClinicalTrials updated today. --- # 197. CANCERINDEX DAILY Potential editorial product: **CancerIndex Daily** Automatically identify: ```text important approvals practice-changing trials major publications new trial openings large dataset releases ``` AI summarizes with citations. --- # 198. TREND DETECTOR Calculate abnormal increases in: ```text publication volume trial creation drug development gene mentions ``` Potential: ```text "KRAS G12D research activity +74% YoY" ``` Only publish after methodology validation. --- # 199. TOPIC GRAPH Search: ```text ADC ``` Graph: ```text ADC → HER2 → TROP2 → HER3 → cancers → drugs → trials → publications ``` --- # 200. RELATIONSHIP TEMPORALITY Relationships evolve. Store: ```text first evidence most recent evidence current status ``` A therapy-cancer relationship may change from: ```text experimental → Phase III → approved ``` --- # 201. REAL-WORLD EVIDENCE Future module. Possible sources: ```text public registries regulatory RWE reports published cohorts ``` Do not attempt to ingest private medical records casually. --- # 202. PATIENT-REPORTED OUTCOMES When published: ```text quality of life symptom burden functional outcomes ``` Store separately from survival. --- # 203. ENDPOINT ENTITY Clinical endpoints should become structured concepts: ```text OS PFS DFS EFS ORR DOR pCR MRD QoL ``` Map trial results. --- # 204. TRIAL RESULTS EXTRACTION When results are available: capture structured registry results first. Publication-derived results must cite paper. Store: ```text endpoint population arm estimate CI p-value follow-up ``` --- # 205. TREATMENT EFFECT MODEL Do not store: ```text Drug X improves survival by 40% ``` Store: ```text endpoint effect measure HR/RR/OR estimate CI population comparator trial follow-up ``` --- # 206. CROSS-CANCER ANALYSIS Enable questions: ```text Which cancers share KRAS mutations? Which cancers have HER2 amplification? Which cancers respond to tissue-agnostic therapies? Which cancers share immune biomarkers? ``` --- # 207. TUMOR-AGNOSTIC INDICATIONS Support cancer-agnostic drug approvals. Drug indication entity may reference: ```text biomarker without single cancer restriction ``` Do not force every approval to one cancer ID. --- # 208. CANCER OF UNKNOWN PRIMARY Include CUP properly. Do not force primary anatomical site where unknown. --- # 209. BENIGN / BORDERLINE TUMORS CancerIndex may index clinically relevant nonmalignant/borderline tumors if useful for taxonomy. They must be clearly marked: ```text malignant = false ``` Never count them in cancer rankings unless methodology explicitly includes them. --- # 210. SKIN CANCER COUNTING Be careful with: ```text non-melanoma skin cancers ``` Some global datasets treat them differently. Ranking engine must preserve inclusion/exclusion rules. --- # 211. HEMATOLOGIC MALIGNANCIES Do not model solely by anatomical organ. Dedicated structure for: ```text leukemia lymphoma myeloma myelodysplastic neoplasms myeloproliferative neoplasms ``` --- # 212. SARCOMAS Build fine-grained taxonomy. Examples categories: ```text soft tissue bone GIST leiomyosarcoma liposarcoma angiosarcoma synovial sarcoma Ewing sarcoma osteosarcoma ``` Do not group all rare sarcomas when subtype data exists. --- # 213. BRAIN/CNS TUMORS Molecular classification is essential. Model modern molecular subtypes. Taxonomies change over time. Store classification version. --- # 214. BREAST CANCER Support: ```text histology ER PR HER2 HER2-low where applicable triple negative molecular subtypes germline context ``` Do not collapse all breast cancers. --- # 215. LUNG CANCER Support: ```text SCLC NSCLC adenocarcinoma squamous large cell molecular alterations ``` --- # 216. COLORECTAL CANCER Support: ```text colon rectal left/right sided context where evidence requires MSI RAS BRAF HER2 ``` --- # 217. PRECISION TAXONOMY CancerIndex needs overlapping labels. One patient cohort may conceptually be: ```text lung adenocarcinoma metastatic EGFR-mutated exon 19 deletion ``` Do not create a unique canonical cancer entity for every arbitrary combination. Use attributes/biomarker cohort definitions appropriately. --- # 218. COHORT ENTITY Create: ```ts CohortDefinition ``` Example: ```text Metastatic EGFR exon 19 deletion lung adenocarcinoma ``` This is not necessarily a globally recognized cancer taxonomy node. --- # 219. ONTOLOGY VERSIONING Taxonomies evolve. Store: ```text ontology version concept valid_from valid_to ``` Never lose historical mappings. --- # 220. CROSSWALK TABLES Build: ```text NCIt ↔ ICD-O NCIt ↔ ICD-10 NCIt ↔ OncoTree NCIt ↔ Disease Ontology NCIt ↔ MONDO SEER ↔ canonical CancerIndex ``` Mappings may be: ```text exact broader narrower related ambiguous ``` --- # 221. MATCH CONFIDENCE Entity mapping: ```text EXACT_IDENTIFIER CURATED_EXACT ONTOLOGY_EXACT CURATED_BROADER CURATED_NARROWER ALIAS PROBABILISTIC UNRESOLVED ``` --- # 222. UNRESOLVED ENTITY QUEUE Never discard unknown disease labels. Store: ```text source text source ID context count ``` Admin can map later. --- # 223. DATA DISCOVERY AGENT Build an AI-assisted internal agent that searches for: ```text new official APIs new dataset releases schema changes new registries new cancer ontologies ``` It produces proposals. It cannot automatically onboard sources into production without compliance review. --- # 224. CONNECTOR DOCUMENTATION REQUIREMENT Before Claude implements ANY connector: 1. locate current official documentation; 2. verify API/bulk mechanism; 3. verify authentication; 4. inspect pagination; 5. inspect rate limits; 6. inspect license/terms; 7. inspect update schedule; 8. inspect identifiers; 9. save source schema; 10. create tests. Do not implement an API from memory. --- # 225. CURRENT-DOC REQUIREMENT Because CancerIndex depends on external systems: **Claude MUST always verify current documentation before coding an integration.** Do not trust: ```text old blog posts random GitHub examples Stack Overflow cached knowledge ``` Prefer: ```text official documentation official repositories official OpenAPI specs official release notes ``` --- # 226. CONNECTOR SOURCE TEST Before production: ```text curl/API smoke test ↓ small fixture ↓ parser ↓ normalization ↓ reconciliation ↓ integration test ↓ full sync ``` --- # 227. HUGE IMPORT SAFETY Never begin a million-record import before proving the pipeline on: ```text 10 100 1,000 ``` records. --- # 228. BULK-FIRST STRATEGY For massive datasets: prefer bulk downloads over millions of API calls when terms and official access support it. --- # 229. RATE LIMIT RESPECT Implement: ```text token bucket exponential backoff Retry-After jitter max concurrency ``` Source-specific. --- # 230. CHECKSUMS Bulk file: ```text SHA-256 ``` Store: ```text source URL timestamp checksum size ``` --- # 231. ETL LANGUAGE Use Python heavily for scientific ETL. TypeScript can orchestrate web/application systems. Do not force complex bioinformatics normalization into TypeScript if mature Python packages are appropriate. --- # 232. DATAFRAMES For large ETL: consider: ```text Polars PyArrow DuckDB ``` instead of blindly using pandas for everything. --- # 233. PARQUET Use Parquet for large analytical snapshots. Partition by sensible dimensions. Example: ```text source year entity type ``` --- # 234. BIOINFORMATICS LIBRARIES Before choosing packages: verify active maintenance/current documentation. Potential functionality: ```text HGVS normalization VCF parsing genomic liftover sequence handling ``` Never implement complex genomics standards from scratch unless necessary. --- # 235. GENOME BUILD Canonical support: ```text GRCh37 GRCh38 ``` Where available. Never silently convert coordinates. Store original + normalized. --- # 236. LIFTOVER If performing liftover: ```text original assembly original coordinate target assembly converted coordinate tool/version status ``` --- # 237. VARIANT NORMALIZATION Store: ```text genomic HGVS coding HGVS protein HGVS gene transcript assembly dbSNP ClinVar ID CIViC ID ``` Not every variant will have all identifiers. --- # 238. FUSIONS Dedicated structure: ```text 5' gene 3' gene breakpoint fusion name orientation ``` Do not model only as free text. --- # 239. COPY NUMBER Model: ```text amplification gain loss deep deletion ``` Keep source-specific thresholds. --- # 240. EXPRESSION Keep units/platform. Never compare raw expression values from incompatible platforms directly. --- # 241. BIOMARKER THRESHOLDS Example PD-L1. Store: ```text assay clone scoring system threshold cancer indication ``` Do not reduce to positive/negative globally. --- # 242. TMB Store: ```text assay unit threshold panel cancer ``` --- # 243. MSI Map: ```text MSI-H MSS MSI-L dMMR pMMR ``` but preserve differences. --- # 244. EVIDENCE CROSS-CANCER CONTEXT A variant may be: ```text predictive in cancer A prognostic in cancer B unknown in cancer C ``` Relationship context is mandatory. --- # 245. RANKING DATA ELIGIBILITY For a cancer to enter a ranking: define explicit inclusion rules. Example survival ranking: ```text minimum cohort size accepted survival type accepted diagnosis period geography minimum source quality ``` Do not rank sparse estimates unfairly. --- # 246. PARENT VS SUBTYPE RANKING Avoid double-counting. If global incidence gives: ```text Lung Cancer = 2.4M ``` and subtype estimates separately: ```text LUAD SCC ``` do not sum all three. Rank scope must define entity level. Allow: ```text Top-level cancer ranking Histology ranking Subtype ranking Rare entity ranking ``` --- # 247. GLOBAL MASTER RANKING Default broad global ranking should use mutually exclusive or carefully defined top-level cancer categories. Fine-grained ranking is separate. --- # 248. CANCERINDEX COVERAGE COUNT Homepage may say: ```text 4,812 cancer entities indexed ``` only if entity model genuinely contains them. Do not market every alias as a separate cancer. --- # 249. DUPLICATE CONTROL Alias count ≠ cancer count. Subtype count ≠ top-level cancer count. Be explicit. --- # 250. METRIC CATALOG Create first-class: ```text MetricDefinition ``` Fields: ```text id name description formula unit higherIsWorse aggregation validDimensions sources methodologyVersion ``` --- # 251. FORMULA ENGINE Derived metrics should not live as random code functions. Create versioned formulas. Example: ```yaml id: CI-METRIC-MIR name: Mortality-to-Incidence Ratio formula: mortality_count / incidence_count version: 1.0 ``` --- # 252. DATA LINEAGE GRAPH Every ranked score should be traceable: ```text CancerIndex score ↓ component ↓ normalized metric ↓ source observation ↓ raw source record ``` --- # 253. ADMIN "TRACE VALUE" Admin button: ```text TRACE ``` For any number. Shows full lineage. This will save massive debugging time. --- # 254. CANCERINDEX LABS Experimental section: ```text /labs ``` For: ```text forecasting experimental indexes novel network analysis AI research tools ``` Clearly separate experimental metrics from main product. --- # 255. NETWORK CENTRALITY Interesting research feature: rank genes by: ```text number of cancers number of actionable variants number of approved drugs number of trials network centrality ``` Do not imply biological importance solely from graph centrality. --- # 256. DRUG TARGET LANDSCAPE Visual: ```text targets × cancers ``` Heatmap: ```text approved clinical preclinical ``` --- # 257. ONCOLOGY PIPELINE MAP Interactive: ```text Cancer → Target → Drug → Phase → Company ``` --- # 258. BIOMARKER LANDSCAPE Interactive matrix: ```text Cancer × Biomarker ``` Color: ```text frequency clinical actionability ``` Different toggles. --- # 259. CANCER GENOMIC LANDSCAPE Cancer page: ```text Top mutated genes CNAs fusions pathways ``` Allow study selection. Do not merge frequencies from incompatible cohorts without method. --- # 260. COHORT SELECTOR Example: ```text TCGA MSK cohort CPTAC study X ``` User can switch data source. --- # 261. FREQUENCY DENOMINATORS Every genomic frequency must include denominator. Example: ```text KRAS mutation: 31.4% 214 / 681 profiled samples ``` Never show 31.4% without cohort context. --- # 262. MISSINGNESS Genomic studies frequently have different profiling coverage. Store: ```text tested not tested unknown ``` Do not assume missing = wild type. --- # 263. SURVIVAL CURVES Where permissible/raw aggregate data allow: Kaplan-Meier visualization. Display: ```text n at risk CI censoring cohort endpoint ``` Do not fabricate curves from summary survival percentages. --- # 264. INCIDENCE TREND CHART Use: ```text annual estimates ASIR confidence intervals when available ``` --- # 265. GLOBAL BURDEN BUBBLE CHART Axes: ```text X = incidence Y = mortality/incidence bubble = deaths ``` Great discovery visualization. --- # 266. RESEARCH GAP QUADRANT Axes: ```text X = disease burden Y = research activity ``` Quadrants: ```text high burden / high research high burden / low research low burden / high research low burden / low research ``` --- # 267. CLINICAL TRIAL MAP World map of recruiting trial sites. Filters: ```text cancer drug phase biomarker ``` --- # 268. FACILITY ENTITY Normalize trial locations. Challenge: ```text same hospital with slightly different names ``` Use: ```text name address city country geocode organization ID ``` --- # 269. ORGANIZATION RESOLUTION Potential IDs: ```text ROR GRID historical mappings OpenAlex institution ``` Use ROR where appropriate. --- # 270. RESEARCHER RESOLUTION Use ORCID when explicitly linked. Never assume identical author names are same person. --- # 271. PUBLICATION ENTITY EXTRACTION Use pipeline: ```text dictionary match ontology mapping NER LLM structured extraction cross validation ``` Store extraction method. --- # 272. EVIDENCE SENTENCE When permitted, keep sentence-level evidence reference. Do not violate publication copyright. Store small extracted evidence snippets within legal limits where appropriate, otherwise store location/reference only. --- # 273. FULL-TEXT RIGHTS Open access ≠ automatically unrestricted redistribution in every context. Track article license. --- # 274. AUTOMATED CITATION Generated text should prefer primary source where possible. Example: trial result: prefer: ```text peer-reviewed trial publication ``` plus registry. Regulatory claim: prefer regulator. --- # 275. SOURCE TRUST DOES NOT MEAN UNIVERSAL AUTHORITY A source can be excellent for one question and inappropriate for another. Example: ```text FDA → US approval SEER → US registry statistics IARC → global estimates HGNC → gene symbols ``` --- # 276. NO SOURCE MONOCULTURE Important cancer facts should ideally cross-reference multiple sources where appropriate. --- # 277. DATA FUSION Do not create a fused value unless scientifically justified. Often the right UX is: ```text SEER estimate IARC estimate Canadian estimate ``` side by side. --- # 278. DOCUMENT EVERYTHING Repository: ```text /docs architecture.md data-model.md ranking-methodology.md source-policy.md connectors.md reconciliation.md evidence.md ai.md security.md ``` --- # 279. ADRs Use Architecture Decision Records. Example: ```text ADR-001 PostgreSQL as canonical database ADR-002 pgvector ADR-003 source-native raw retention ADR-004 CancerIndex identifiers ``` --- # 280. CLAUDE WORKFLOW Before editing code: ```text 1. inspect repository 2. inspect CLAUDE.md 3. inspect current architecture 4. inspect existing tests 5. inspect database schema 6. inspect connector framework 7. verify current external documentation 8. create plan 9. implement 10. test 11. verify 12. document ``` --- # 281. DO NOT FAKE FEATURES Never create UI with hardcoded fake metrics merely to make screenshots look complete. If backend data doesn't exist: show: ```text Data not yet available ``` or use explicitly labeled fixtures only in tests/dev. --- # 282. NO MOCK DATA IN PRODUCTION Absolutely no hidden mock values. --- # 283. SOURCE AVAILABILITY If a source connector fails: CancerIndex continues using last known valid snapshot. Display freshness. Do not replace missing real data with AI guesses. --- # 284. DATABASE MIGRATIONS Every schema change: ```text migration rollback strategy test ``` Never manually mutate production schema without migration. --- # 285. QUERY OPTIMIZATION Use indexes on: ```text canonical IDs external IDs slugs gene symbol NCT ID PMID DOI year geography metric ``` --- # 286. PARTITIONING For huge observations: partition or use ClickHouse. Do not create billions of rows in poorly indexed PostgreSQL tables. --- # 287. ENTITY COUNTERS Precompute: ```text trial count publication count gene count drug count ``` where necessary. But source them from canonical relations and refresh deterministically. --- # 288. RANKING MATERIALIZATION Rankings should be precomputed per common scope. Do not run huge window functions for every homepage request. --- # 289. INCREMENTAL RANK REBUILD When ClinicalTrials changes: recalculate trial-dependent ranks only. Do not recompute global genomic rankings unnecessarily. --- # 290. EVENT BUS Useful events: ```text CancerUpdated TrialUpdated DrugApprovalAdded PublicationAdded ConnectorCompleted RankingInvalidated ``` --- # 291. STATIC + DYNAMIC Use Next.js rendering strategy intelligently. SEO pages can be statically regenerated. Live trials/rankings can refresh dynamically. --- # 292. MOBILE Mobile is first-class. High-density information must remain usable. Use: ```text sticky tabs collapsible sources horizontal ranking tables full-screen charts ``` --- # 293. DESKTOP Desktop should feel like a professional research terminal. Allow: ```text split panels dense tables compare mode persistent filters keyboard search ``` --- # 294. COMMAND PALETTE Shortcut: ```text ⌘K ``` Search any entity/action. --- # 295. SHAREABLE URL STATE Filters reflected in URL. Example: ```text /rankings?metric=mortality&year=2024&sex=all ``` --- # 296. DOWNLOAD CHART DATA Every chart: ```text Download CSV Download PNG/SVG where appropriate Copy citation ``` subject to source redistribution terms. --- # 297. CITATION EXPORT Support: ```text BibTeX RIS plain citation ``` for publications. --- # 298. RESEARCH NOTEBOOK Future: Users can create notes linking CancerIndex entities. Not needed for initial release. --- # 299. COMPLIANCE CHECKLIST Before new source goes live: ```text [ ] official source verified [ ] access method verified [ ] license reviewed [ ] attribution requirements stored [ ] rate limits implemented [ ] parser tests [ ] raw snapshot [ ] entity mappings [ ] quality tests [ ] production health monitoring ``` --- # 300. PHASE 1 Build foundation. Deliver: ```text Cancer taxonomy NCI EVS GDC ClinicalTrials.gov PubMed ClinVar SEER IARC integration plan/compliance HGNC Cancer pages Gene pages Trial pages Publication pages Search Provenance Basic rankings ``` --- # 301. PHASE 2 Add: ```text cBioPortal CIViC ChEMBL Open Targets DGIdb FDA DailyMed Ensembl Drug pages Variant pages Biomarker pages Knowledge graph Advanced rankings ``` --- # 302. PHASE 3 Add: ```text global country data European sources Canadian sources additional regulators research funding research gap trial gap treatment gap ``` --- # 303. PHASE 4 Add: ```text DepMap cell lines preclinical single cell pathology imaging multi-omics ``` --- # 304. PHASE 5 Add: ```text CancerIndex AI MCP advanced analytics custom rankings developer ecosystem public data releases ``` --- # 305. INITIAL CONNECTOR TARGET Do not stop at five connectors. Long-term target: ```text 50+ high-quality connectors ``` But quality > arbitrary connector count. One reliable bulk/API integration is worth more than ten fragile scrapers. --- # 306. CANCER COVERAGE TARGET Do not set a fixed target such as: ```text 200 cancers ``` Instead: > Index every distinct malignant disease entity that can be mapped from selected authoritative oncology classifications. Then expose: ```text top-level cancers families histologies subtypes molecular subtypes ``` separately. --- # 307. PUBLIC SOURCE CATALOG CancerIndex should publicly list connectors. Example: ```text NCI GDC Genomics SEER Epidemiology IARC Global burden ClinicalTrials Trials PubMed Literature ClinVar Variants cBioPortal Cancer genomics CIViC Clinical variants HGNC Genes ChEMBL Drugs FDA Regulation ... ``` --- # 308. SOURCE COVERAGE MATRIX Table: ```text Epidemiology Genomics Trials Drugs Research GDC · ✓ · · ✓ SEER ✓ · · · · ClinicalTrials · · ✓ ✓ ✓ PubMed · ✓ ✓ ✓ ✓ CIViC · ✓ · ✓ ✓ ``` --- # 309. CANCER DATA CARD Every cancer card should contain at minimum: ```text name parent category annual burden if available mortality survival CancerIndex rank research activity data confidence ``` --- # 310. CANCER INDEX BADGES Examples: ```text Rare Pediatric Hematologic High Mortality Rapidly Rising High Research Activity Low Trial Activity Treatment Gap ``` Generated from rules, not editorial opinion. --- # 311. RANKING CHANGE EXPLANATION If rank changes: ```text #12 → #8 ``` show why: ```text 2024 global mortality estimate updated +4 new active trials methodology unchanged ``` --- # 312. SEARCH RESULT EXPLANATION Search result should show type: ```text EGFR GENE EGFR L858R VARIANT EGFR-mutated NSCLC MOLECULAR COHORT Osimertinib DRUG ``` --- # 313. SYNONYM MANAGEMENT Alias examples: ```text GBM glioblastoma glioblastoma multiforme [historical usage] ``` Preserve historical terminology without necessarily using it as preferred current name. --- # 314. DEPRECATED TERMINOLOGY Fields: ```text deprecated deprecated_reason replacement_entity classification_version ``` --- # 315. VERSION-SENSITIVE MEDICINE Cancer classification changes. Never rewrite historical publication terminology as if author used modern classification. Map: ```text source_term → modern CancerIndex entity ``` while preserving original term. --- # 316. DATA CITATION CancerIndex itself should produce dataset citations: ```text CancerIndex Data Release YYYY-MM ``` for researchers. --- # 317. ABOUT PAGE Explain: ```text what CancerIndex is what it is not where data comes from how rankings work limitations ``` --- # 318. TRUST CENTER Route: ```text /trust ``` Include: ```text data provenance methodology AI policy privacy security source policy corrections ``` --- # 319. CORRECTIONS Public correction mechanism. Researchers can report: ```text wrong mapping outdated statistic incorrect citation entity duplication ``` Corrections tracked publicly where appropriate. --- # 320. SCIENTIFIC ADVISORY MODEL Future: expert contributors can have verified profiles. No anonymous modification of clinical evidence without review. --- # 321. API SOURCE ATTRIBUTION API responses include: ```json { "data": {}, "sources": [], "dataRelease": "..." } ``` --- # 322. AI SOURCE SNAPSHOT AI answer stores exactly which record versions were used. This allows reproducibility after database updates. --- # 323. NO BLACK-BOX SCORE CancerIndex's major promise: > Every score can be decomposed. No proprietary mystery scoring methodology. --- # 324. RESEARCH GAP ETHICS Low research activity does not inherently mean neglect. Reasons can include: ```text rarity disease biology classification changes small populations successful prevention ``` Display as quantitative signal, not accusation. --- # 325. SURVIVAL ETHICS Population survival statistics do not predict an individual's outcome. Always show contextual note. --- # 326. EPIDEMIOLOGY ESTIMATES Global cancer statistics are frequently estimates, not literal complete counts. Label: ```text estimated cases estimated deaths ``` where appropriate. --- # 327. CANCERINDEX "TODAY" Avoid saying: ```text today X people have cancer ``` unless the methodology genuinely supports it. Prefer annual/latest dataset statistics. --- # 328. CANCERINDEX DISCOVERY HOMEPAGE Interesting modules: ```text Most common Most lethal Most researched Most under-researched Most trials Largest treatment gaps Fastest improving Fastest rising Rare cancer spotlight ``` --- # 329. "CANCER UNIVERSE" Create a visual: ```text Cancer Universe ``` Thousands of cancer/subtype nodes arranged by anatomical/histological family. Size could represent: ```text incidence ``` Color could represent: ```text survival ``` Filterable. Potential signature visualization. --- # 330. "CANCER MATRIX" Rows: ```text cancers ``` Columns: ```text genes ``` Cell: ```text alteration frequency ``` Filter by study. --- # 331. "DRUG MATRIX" Rows: ```text cancers ``` Columns: ```text drugs ``` Cell: ```text approved clinical preclinical ``` --- # 332. "TRIAL PULSE" Live research activity chart. ```text new oncology trials / week ``` Break down by: ```text cancer phase country target ``` --- # 333. "RESEARCH PULSE" ```text new publications / week ``` Topic clustering. --- # 334. "APPROVAL PULSE" Timeline of oncology regulatory decisions. --- # 335. "GENOMIC PULSE" Newly curated variant/drug evidence from authoritative sources. --- # 336. DATA QUALITY BADGE Pages can show: ```text Data Quality: High ``` Derived from: ```text source coverage freshness agreement sample size missingness ``` Document formula. --- # 337. SPARSE CANCER UX For ultra-rare cancers: do NOT show empty giant sections. Instead: ```text Very limited epidemiological data are currently available. What CancerIndex knows: - 7 publications - 1 active trial - 2 reported genomic associations ``` Sparse-data visibility is valuable. --- # 338. ENTITY DISCOVERY PRIORITY Nightly job: find source disease labels not mapped to canonical CancerIndex cancer. Rank by frequency. This continuously expands coverage. --- # 339. AUTOMATIC CANCER DISCOVERY Candidate disease discovery: ```text NCIt additions SEER updates ClinicalTrials condition strings GDC disease terms CIViC diseases PubMed MeSH ``` Candidate → curator queue. --- # 340. NEVER LET LLM INVENT A NEW CANONICAL DISEASE LLM can suggest. Canonical entity creation requires: ```text recognized ontology trusted source or human curator approval ``` --- # 341. SOURCE COUNT IS NOT EVIDENCE QUALITY Displaying: ```text 14 sources ``` does not mean stronger evidence if all sources repeat one paper. Track evidence lineage and primary evidence. --- # 342. DUPLICATE PUBLICATION DETECTION Resolve: ```text PubMed DOI Crossref Europe PMC ``` into one publication entity. --- # 343. TRIAL ↔ PUBLICATION RESOLUTION Use: ```text NCT IDs in publication registry references PubMed links ``` Then probabilistic matching only as fallback. --- # 344. DRUG SYNONYMS Drug naming is messy. Support: ```text generic brand development code salt active moiety combination ``` Do not treat brand names as separate molecules. --- # 345. TARGET SYNONYMS Resolve gene/protein target nomenclature carefully. --- # 346. CANCERINDEX INTERNAL ONTOLOGY Do not reinvent all external ontologies. CancerIndex ontology should function primarily as a stable reconciliation layer. --- # 347. EXTERNAL IDS ARE FIRST-CLASS Never place IDs in an unstructured JSON dump only. Create searchable cross-reference tables. --- # 348. AUDIT LOG Record: ```text who what before after when why ``` for curator/admin changes. --- # 349. SOFT LAUNCH CRITERIA CancerIndex is not ready merely because home page looks good. Minimum: ```text credible taxonomy multiple foundational connectors provenance search cancer pages ranking methodology ranking reproducibility connector monitoring scientific disclaimers ``` --- # 350. LAUNCH DATA QUALITY TARGETS Before launch: * no unexplained metrics * no production mock data * no duplicate canonical cancers from obvious aliases * no broken source links * no AI answers without citations * no ranking without scope/year/source * no unsupported treatment claims --- # 351. PRIORITY IMPLEMENTATION ORDER Claude should execute approximately: ```text 1. repository foundation 2. database 3. source/provenance framework 4. canonical IDs 5. cancer ontology 6. NCI terminology connector 7. HGNC 8. GDC 9. ClinicalTrials 10. PubMed 11. ClinVar 12. SEER 13. IARC compliance/integration 14. cBioPortal 15. CIViC 16. FDA 17. ChEMBL 18. Open Targets 19. ranking engine 20. web experience 21. AI ``` Exact order may change based on API/data dependencies. --- # 352. FIRST RANKINGS TO SHIP Ship these first: ```text Global Incidence Global Mortality Age-standardized Incidence Age-standardized Mortality Mortality/Incidence Ratio 5-Year Survival where comparable Active Trial Count Research Publication Count CancerIndex Research Gap CancerIndex Trial Gap ``` Then composite score. --- # 353. COMPOSITE SCORE MUST COME LATER Do not launch composite score until individual metrics are correct. Composite rankings amplify bad data. --- # 354. DOCUMENT CONNECTORS AS CODE Connector documentation should be generated/validated from manifests where practical. --- # 355. REPOSITORY STRUCTURE Suggested: ```text /apps /web /api /admin /services /ingestion /ranking /ai /search /packages /database /ontology /connectors /schemas /ui /analytics /provenance /connectors /nci-evs /gdc /seer /iarc /clinicaltrials /pubmed /clinvar /cbioportal /civic /hgnc /ensembl /chembl /opentargets /dgidb /fda /docs /infra /tests ``` --- # 356. TYPES Share canonical TypeScript/Python schemas where possible. Use generated JSON Schema/OpenAPI contracts to prevent divergence. --- # 357. OPENAPI API documentation generated from real API definitions. Never manually maintain docs that drift from endpoints. --- # 358. DATABASE SEEDS Only seed: ```text system roles configuration metric definitions source definitions ``` Scientific data comes from connectors. --- # 359. DEVELOPMENT FIXTURES Label dev fixtures clearly. Environment must make production fixture insertion impossible. --- # 360. FINAL PRODUCT STANDARD CancerIndex.io should feel like a serious piece of scientific infrastructure. Not: > "Here are 30 cancers and some AI summaries." But: > "Here is a continuously updated, provenance-aware, globally ranked ontology of cancer connecting epidemiology, genomics, biomarkers, therapies, trials, regulatory evidence and scientific literature." The user should be able to start at any node: ```text a cancer a gene a mutation a drug a clinical trial a paper a country ``` and move through the entire oncology knowledge graph. --- # 361. NORTH STAR The ultimate CancerIndex graph should conceptually support: ```text ALL CANCERS ↓ ALL RECOGNIZED SUBTYPES ↓ EPIDEMIOLOGY ↓ GENES ↓ VARIANTS ↓ BIOMARKERS ↓ PATHWAYS ↓ DRUGS ↓ TREATMENTS ↓ APPROVALS ↓ CLINICAL TRIALS ↓ PUBLICATIONS ↓ RESEARCHERS ↓ INSTITUTIONS ↓ COUNTRIES ``` Every connection: ```text traceable versioned source-backed queryable rankable ``` --- # 362. NON-NEGOTIABLE CLAUDE INSTRUCTION Claude must NOT rush this project. Before building substantial portions of CancerIndex: * read this entire `CLAUDE.md`; * inspect all existing code; * research the latest official documentation; * identify source licensing constraints; * construct the canonical ontology carefully; * test every connector independently; * retain raw source provenance; * build reconciliation before massive ingestion; * validate all scientific computations; * ensure ranking formulas are reproducible; * never fabricate missing data; * never substitute AI memory for missing database evidence; * preserve uncertainty; * optimize the product for both ordinary users and serious researchers. Whenever there is a choice between: ```text faster implementation ``` and: ```text scientifically defensible implementation ``` choose the scientifically defensible implementation. Whenever there is a choice between: ```text more connectors ``` and: ```text reliable connectors with provenance ``` choose reliability first — then expand aggressively. Whenever there is a choice between: ```text one broad cancer category ``` and: ```text accurately modeling recognized subtypes ``` preserve the detailed taxonomy. Whenever an upstream source provides an identifier: **keep it.** Whenever an upstream source provides provenance: **keep it.** Whenever an upstream source provides uncertainty: **keep it.** Whenever CancerIndex computes something: **make the formula visible.** Whenever CancerIndex ranks something: **explain the ranking.** Whenever CancerIndex AI says something: **cite the evidence.** --- # 363. FINAL VISION CancerIndex.io should eventually make it possible to ask: > Rank every indexed cancer in the world by unmet need, weighting global mortality, 5-year survival, number of approved systemic therapies, active Phase II/III trials and publication activity. and receive a reproducible result. Or: > Show every known cancer subtype associated with BRAF V600E, the frequency reported in each major cohort, regulatory-approved treatments by jurisdiction, active trials and supporting clinical evidence. Or: > Which rare cancers have fewer than five recruiting trials worldwide despite high mortality? Or: > What malignancies experienced the largest improvement in survival during the last two decades? Or: > Which cancer research areas are accelerating fastest this year? Or: > Show the entire treatment-development landscape for KRAS G12D. CancerIndex should answer these from structured, sourced data rather than model memory. That is the standard. --- # 364. BUILD PRINCIPLE **Do not build a cancer website.** Build: > **the structured global intelligence layer for cancer.** That is CancerIndex.io.