SPB Git

spb/cancerindex

Public
4.7 KB

# CancerIndex.io — the global index of cancer

CancerIndex is a provenance-first, continuously updated, transparently sourced oncology knowledge platform. It connects a canonical cancer ontology (NCIt-anchored, OncoTree and registry dimensions) to epidemiology, genomics, biomarkers, therapies, clinical trials, regulatory evidence and literature — for every recognised malignant disease entity — and publishes reproducible rankings with their scope, formula version and lineage down to the raw record.

It is not a physician, not a diagnostic tool, and gives no treatment recommendations.

# Status

Phase 1 (2026-09). Foundation, connectors, public API, worker and web app are being built in parallel. Data currently ingested locally: OncoTree (865 cancer entities), with NCIt EVS, HGNC, CIViC, GDC, ClinVar, ClinicalTrials.gov, PubMed and CDC WONDER connectors in progress. GLOBOCAN remains under license review and SEER awaits credentials — the platform says so rather than showing numbers it cannot source.

# Repository

text
apps/web         Next.js 16 public site (port 8250), proxies /api/v1/* to the API
apps/api         Fastify 5 public API /v1 (port 8251), OpenAPI at /v1/docs
workers/         pg-boss scheduler: connector cron, counters, rankings, health probes
packages/        shared · database (Drizzle) · ontology · connectors (SDK + connectors) · ranking
scripts/ci.ts    operator CLI (pnpm cix …)
deploy/          MacLustr mld manifest, first-run bootstrap
docs/            architecture, data model, methodology, API, security, AI policy, ADRs, connector docs

Start with CLAUDE.md (operational rules), then docs/ARCHITECTURE.md, docs/DATA-MODEL.md and docs/METHODOLOGY.md.

# Quickstart

Requirements: Node ≥ 22, pnpm 11, PostgreSQL 17 with pg_trgm, unaccent and vector.

bash
createdb cancerindex && cp .env.example .env      # set NCBI_EMAIL, ADMIN_TOKEN
pnpm install
pnpm db:migrate && pnpm db:seed && pnpm cix sources:sync
pnpm cix run oncotree --mode dry_run              # smoke test against the live API (no writes)
pnpm cix run oncotree                             # first real ingest
pnpm cix counters && pnpm cix rank
pnpm dev:api &                                    # http://127.0.0.1:8251/v1/docs
pnpm worker &                                     # schedules from connector manifests
pnpm dev:web                                      # http://localhost:8250

Verify: curl -s localhost:8251/healthz, curl -s "localhost:8251/v1/search?q=glio".

# CLI

text
pnpm cix connectors                 list connectors, license status, health
pnpm cix run <id> [--mode full|incremental|dry_run] [--max-records N] [--max-minutes M] [--reset-cursor]
pnpm cix run-all [--max-minutes M]  every active connector in registry order
pnpm cix health <id>                source liveness probe
pnpm cix sources:sync               manifests → sources table
pnpm cix counters                   rebuild entity_counters
pnpm cix rank                       recompute ranking snapshots
pnpm cix stats                      table counts
pnpm cix trace <table> <id>         lineage of a value

tsx workers/cli.ts run <id> | counters [--then-rank] | rank | health | schedules | queues
pnpm --filter @cancerindex/api create-key -- --label "Lab" --tier research --rpm 600

# Quality gates

bash
pnpm typecheck        # every workspace
pnpm -r test          # vitest; connector tests use fixtures, API smoke tests skip without a DB

# Deployment

Production runs on the MacLustr cluster through mld (gateway M1M32): manifest deploy/mld-manifest.cancerindex.json, mld stage <dir> cancerindex, mld deploy cancerindex, then deploy/first-run.sh for the ordered first ingestion. PM2 processes cancerindex-web, cancerindex-api, cancerindex-worker; ngrok www.cancerindex.io → 8250. See deploy/README.md.

# Principles (short form)

  1. No scientific number without provenance; derived values carry a formula version and inputs.
  2. Layers stay separable: RAW → NORMALIZED → CANONICAL → DERIVED → RANKED → AI.
  3. Never fake data: missing is missing ("Data not yet available"), never zero.
  4. Identifiers are first-class: every upstream id is kept; public ids are CI-<NS>-00000001.
  5. Reconciliation before ingestion; unknown labels go to a curation queue, never dropped.
  6. Time-aware observations: a new year is a new row.
  7. Every edge has context: cancer, direction, evidence level, provenance.
  8. Licensing gate on every source; scientific safety labels are never merged.

# License and attribution

Code: private (Groupe/SPB). Data: each source keeps its own license and attribution, surfaced in every API response (sources) and on /sources. See docs/source-policy.md.

Files · f2c4fe3d71ef358edbf079200a52ea335cc5b3f6
Menu

Theme