spb/airiskindex Public

1# CLAUDE.md — AI Risk Index Platform (www.airiskindex.io)23This file provides guidance to Claude Code when working in this repository.45---67## 1. Project Overview89**AI Risk Index** (airiskindex.io) is a public platform that scores occupations on their exposure to AI-driven automation, using a **task-based methodology** (not whole-occupation scoring). Every score must be transparent, versioned, and reproducible.1011**Core product promise:** "The most methodologically rigorous, fully transparent AI job-exposure index."1213### The three concepts (never confuse them)14- **Exposure** — AI is technically capable of performing the task15- **Substitution** — AI actually replaces the human performing it16- **Augmentation** — AI assists the human, increasing productivity1718The composite index reports all three as separate sub-scores. Never collapse them into a single undifferentiated "risk" number in code, UI copy, or API responses without also exposing the sub-scores.1920### Scoring model (v1)21Composite score 0–100 per occupation, computed from task-level scores weighted by task importance/frequency (O*NET weights). Dimensions:2223| Dimension | Key | Weight (v1) |24|---|---|---|25| Task automatability | `automatability` | 0.35 |26| Current technical feasibility | `feasibility` | 0.20 |27| Cost of substitution vs. wage | `cost_ratio` | 0.15 |28| Adoption barriers (regulation, liability, human-contact requirement) | `barriers` | 0.20 |29| Sector adoption velocity | `adoption_velocity` | 0.10 |3031- Weights live in `packages/scoring/src/weights.ts` — **never hardcode weights anywhere else**.32- Every scoring release gets a semver version (`INDEX_VERSION` in `packages/scoring/src/version.ts`) and a changelog entry in `docs/methodology/CHANGELOG.md`.33- Scores are stored with confidence intervals (`score_low`, `score`, `score_high`). UI must always be able to display uncertainty.3435### Data sources36- **O*NET 30.x** database dumps (occupations + tasks + importance ratings) → `data/raw/onet/` — 30.3 as of May 2026, 31.0 expected late Aug 2026. ⚠️ 30.x renamed *Technology Skills → Software Skills* and split *Skills* into Essential/Transferable; ETL loaders must target the 30.x schema. License CC BY 4.0 (attribution required).37- ESCO crosswalk for EU/France occupations (ROME codes) → `data/raw/esco/`38- LLM-as-evaluator task ratings (see §6) → `data/derived/ratings/`39- Expert panel (Delphi) validation overrides → `data/derived/expert_overrides/`4041Raw data is **never** edited in place. All transformations go through the ETL pipeline (`apps/etl`).4243---4445## 2. Tech Stack4647- **Monorepo:** pnpm workspaces + Turborepo48- **Language:** TypeScript everywhere (strict mode). Python 3.12 only inside `apps/etl` for data science steps.49- **Frontend:** Next.js 14 (App Router), React 18, Tailwind CSS, shadcn/ui, Recharts for visualizations50- **API:** Next.js route handlers for public API (`/api/v1/*`) + tRPC for internal app calls51- **Database:** PostgreSQL 16 (Prisma ORM). Read-heavy → materialized views for score lookups.52- **Cache:** Redis (score lookups, rate limiting)53- **Jobs:** BullMQ workers in `apps/worker` (score recomputation, LLM rating batches)54- **Auth:** Auth.js (email magic link + OAuth). Public browsing requires no auth.55- **Testing:** Vitest (unit), Playwright (e2e), pytest (ETL)56- **Lint/format:** ESLint + Prettier, ruff for Python. CI fails on warnings.5758## 3. Repository Layout5960```61airiskindex/62├── apps/63│   ├── web/          # Next.js app (site + public API routes)64│   ├── worker/       # BullMQ background workers65│   └── etl/          # Python data pipeline (raw → derived → DB)66├── packages/67│   ├── scoring/      # Pure TS scoring engine — NO I/O, fully deterministic68│   ├── db/           # Prisma schema + client + seeds69│   ├── ui/           # Shared React components70│   └── config/       # Shared eslint/ts/tailwind configs71├── data/72│   ├── raw/          # Immutable source dumps (gitignored, fetched by script)73│   └── derived/      # Pipeline outputs (versioned manifests committed)74├── docs/75│   └── methodology/  # Public methodology doc, changelog, sensitivity analyses76├── infra/            # Deployment scripts, ngrok config, systemd units77└── CLAUDE.md78```7980## 4. Commands8182```bash83pnpm install                  # install all workspaces84pnpm dev                      # run web app on :3000 (+ worker with --filter)85pnpm build                    # turbo build all86pnpm test                     # vitest across packages87pnpm test:e2e                 # playwright (requires pnpm dev running)88pnpm lint && pnpm typecheck   # must pass before any commit89pnpm db:migrate               # prisma migrate dev90pnpm db:seed                  # seed occupations + demo scores91pnpm score:recompute          # full index recomputation (writes new INDEX_VERSION run)92cd apps/etl && make pipeline  # full ETL: raw → derived → DB load93```9495Single test file: `pnpm vitest run packages/scoring/src/composite.test.ts`9697## 5. Coding Conventions9899- `packages/scoring` must stay **pure and deterministic**: no network, no DB, no `Date.now()`, no randomness. It takes typed inputs and returns typed scores. This is what makes the methodology auditable.100- Every scoring function has property-based tests (fast-check) + snapshot tests against the published methodology examples in `docs/methodology/examples/`.101- All user-facing risk language follows the tone guide: the product frames results as **adaptation guidance, not doom**. Avoid copy like "your job will disappear"; prefer "X% of tasks in this occupation are highly exposed".102- API responses are versioned (`/api/v1/...`) and include `index_version` in every payload.103- Money/wages: store as integer cents + ISO currency. Percentages: store as 0–1 floats, format only at the UI layer.104- Never commit anything under `data/raw/`. Derived data commits only the manifest JSON (hashes + row counts), not the payloads.105- Migrations: additive-first. Destructive migrations require a `-- DESTRUCTIVE` comment and a manual approval in PR review.106- Accessibility: all charts need a data-table fallback (`<VisuallyHidden>` table) — this is a public-interest tool.107108## 6. LLM-as-Evaluator (task rating pipeline)109110Task automatability ratings are produced by an LLM rater in `apps/worker/src/raters/`, then validated by human experts.111112- Use the Anthropic API via the official SDK. Model names come from the `RATER_MODELS` env var (comma-separated list — **multi-model rating is required**: single-model LLM exposure ratings show up to 19× spread across frontier raters; see `docs/research/01-existing-indices.md` §7.2). Never hardcode model IDs in source.113- Newer models (Sonnet 5 / Opus 5) reject the `temperature` parameter — rating variance comes from the multi-model panel, not sampling temperature.114- Prompts live in `apps/worker/src/raters/prompts/*.md` and are versioned; a prompt change bumps `RATER_PROMPT_VERSION` and invalidates cached ratings.115- Every rating stores: model, prompt version, raw response, parsed score, timestamp. Full audit trail, always.116- Ratings are sampled (5%) for human review; disagreement > 1 point on the 5-point scale flags the task for the expert panel queue.117- Batch jobs must be idempotent and resumable (BullMQ job IDs = deterministic hash of task_id + prompt version).118- For current API details (batch endpoints, rate limits, model names), check https://platform.claude.com/docs (docs.claude.com redirects there) rather than relying on memory. A vetted snapshot lives in `docs/research/03-llm-rater-api.md`.119120## 7. Environments & Deployment121122### Environments123- `local` — developer machine, SQLite optional shortcut is **not** allowed; always Postgres via Docker (`infra/docker-compose.dev.yml`).124- `staging` — node **m3u96b**, exposed via **ngrok** (see below).125- `production` — airiskindex.io (target: VPS/managed later; staging setup is the current deployment).126127### Deploying to node m3u96b (staging/current prod)128129The app currently runs on the self-hosted node `m3u96b` and is exposed publicly through an ngrok tunnel mapped to `www.airiskindex.io`.130131**Process layout on m3u96b (macOS node — PM2, not systemd; deployed 2026-08-05):**132- App dir: `~/apps/airiskindex` (deployed by rsync from the laptop — no git remote yet)133- `airiskindex-web` (PM2) → `./run-web.sh` → `next start -p 3000` (cwd `apps/web`)134- `airiskindex-ngrok` (PM2) → `./run-ngrok.sh` → `ngrok http 3000 --url=https://www.airiskindex.io`135- `postgresql@16` + `redis` via `brew services` (NOT Docker on this node)136- Worker (`airiskindex-worker`) not started yet — starting it submits a live Anthropic rating batch137- `.env` at repo root (chmod 600), symlinked into `apps/web/.env`, `packages/db/.env`, `apps/worker/.env`138  (Next.js and Prisma each load env from their own directory in a monorepo)139- Registered in `~/Desktop/cluster-skill/cluster-deployments.json`140141**Deploy procedure:**142```bash143# from the laptop144rsync -az --delete --exclude node_modules --exclude .next --exclude .turbo \145  --exclude .env --exclude "data/raw/*" ~/Desktop/airiskindex/ M3U96b:apps/airiskindex/146ssh M3U96b147cd ~/apps/airiskindex148pnpm install --no-frozen-lockfile149pnpm build150set -a && . ./.env && set +a && pnpm db:migrate:deploy151pm2 restart airiskindex-web && pm2 save152curl -fsS http://127.0.0.1:3000/api/v1/health      # must return {"ok":true,...}153curl -fsS https://www.airiskindex.io/api/v1/health # ALWAYS check the public URL too154```155156**ngrok configuration (`infra/ngrok.yml`, copied to `/etc/ngrok/ngrok.yml` on the node):**157```yaml158version: 3159agent:160  authtoken: ${NGROK_AUTHTOKEN}     # from env, never committed161endpoints:162  - name: airiskindex163    url: https://www.airiskindex.io  # requires custom domain configured in ngrok dashboard + CNAME164    upstream:165      url: 3000166```167- The custom domain must be added in the ngrok dashboard and DNS `CNAME` for `www.airiskindex.io` pointed at the ngrok edge target they provide. Verify with `dig CNAME www.airiskindex.io`.168- ngrok runs as `ngrok.service` (systemd, `Restart=always`). Logs: `journalctl -u ngrok -f`.169- Health rule: after every deploy, hit the public URL, not just localhost — tunnel failures are the most common outage cause.170- Because the app sits behind ngrok, trust `X-Forwarded-*` headers: Next.js config already sets trusted proxy handling; do not remove it. Rate limiting keys off `x-forwarded-for` first IP.171172**Environment variables** (`/srv/airiskindex/.env`, template in `infra/.env.example`):173```174DATABASE_URL=postgresql://...175REDIS_URL=redis://...176ANTHROPIC_API_KEY=...        # rater pipeline177RATER_MODELS=...             # comma-separated model IDs (multi-model rating panel)178NGROK_AUTHTOKEN=...179NEXTAUTH_URL=https://www.airiskindex.io180NEXTAUTH_SECRET=...181PUBLIC_BASE_URL=https://www.airiskindex.io182```183Never print, log, or commit secrets. `infra/.env.example` lists keys with empty values only.184185### Rollback186```bash187ssh m3u96b188cd /srv/airiskindex189git checkout <previous-tag>190pnpm install --frozen-lockfile && pnpm build191sudo systemctl restart airiskindex-web airiskindex-worker192```193DB rollbacks: only via forward-fix migrations. Nightly `pg_dump` to `/srv/backups` (retained 14 days) — verify the cron is alive when touching infra.194195## 8. Public API rules196197- `/api/v1/occupations` — list/search (paginated, cached 1h)198- `/api/v1/occupations/:code` — full score breakdown incl. sub-scores, CI bounds, task list, `index_version`199- `/api/v1/methodology` — machine-readable weights + version metadata200- Rate limit: 60 req/min unauthenticated, 600 with API key. Return `429` with `Retry-After`.201- Breaking changes require a new `/api/v2` — never mutate v1 response shapes.202203## 9. Methodology Integrity Rules (non-negotiable)2042051. Any change to weights, formulas, or rater prompts ⇒ bump `INDEX_VERSION`, add changelog entry, regenerate `docs/methodology/sensitivity/` outputs.2062. Scores shown anywhere must be traceable to a stored computation run (`score_runs` table) — no ad-hoc numbers.2073. Historical scores are immutable; recomputations create new runs, old runs remain queryable.2084. The public methodology doc (`docs/methodology/METHODOLOGY.md`) is the source of truth; code comments link to its section anchors. If code and doc disagree, stop and flag it — do not silently pick one.209210## 10. When Working in This Repo, Claude Should211212- Run `pnpm typecheck && pnpm lint && pnpm test` before declaring any task done.213- Touch `packages/scoring` only with accompanying tests and a methodology changelog note.214- Ask before: destructive migrations, changing index weights, editing ngrok/systemd config, or anything that alters public API shapes.215- Prefer small, reviewable commits with conventional-commit messages (`feat(scoring): ...`, `fix(api): ...`, `infra(deploy): ...`).216- Keep UI copy aligned with the "adaptation, not doom" tone guide.217- When uncertain about Anthropic API specifics (models, batch API, limits), consult the official docs instead of guessing.218