SPB Git

spb/airiskindex Public

The most methodologically rigorous, fully transparent AI job-exposure index.

TypeScript 88% Python 6.1% SQL 2.7% CSS 1.2% JavaScript 0.9% Shell 0.8%
4.5 KB

# Research Corpus — AI Risk Index (compiled 2026-08-05)

Comprehensive web research to ground the v1 methodology, ETL pipeline, LLM rater, and product positioning. Each document carries full source URLs; claims are dated.

Doc Contents
01-existing-indices.md 20+ AI job-exposure indices & studies (Frey–Osborne → Eloundou → ILO/IMF/OECD → Anthropic Economic Index → 2025–26 frontier), comparison table, validation literature, implications for our 5 dimensions
02-data-sources.md 20 raw data sources with verified URLs, versions, licenses, cadences: O*NET, ESCO/ROME, OEWS wages, employment projections, AI-adoption surveys, benchmark exposure datasets; ingestion & manifest plan
03-llm-rater-api.md Current Anthropic model lineup/pricing, Message Batches API, structured outputs (GA), prompt caching, BullMQ integration pattern, cost estimates for the full rating job
04-landscape-and-evidence.md Competitor analysis (US + FR/EU), measured 2024–26 labor-market effects, adoption stats, regulatory barriers; proposed parameterization of adoption_velocity and barriers

# Headline takeaways

Methodology (01)

  • Single-number occupation scores have a poor empirical record: individual indices explain <11% of realized unemployment risk; ensembles reach 30–75% (Frank et al., PNAS Nexus 2025). Our sub-score + CI design is the right call.
  • Yin et al. (2026) found a 19× spread in Eloundou-style exposure headlines depending on which frontier LLM rates the tasks (2.7%–51.5%). ⇒ Rate with multiple models (or multiple samples) and derive score_low/score_high from rater disagreement; publish rater identity per run.
  • Rate automation vs. augmentation separately per task (ILO WP140 and the Anthropic Economic Index both discriminate on this; the Stanford "Canaries" employment effects concentrate in automation-exposed occupations).
  • cost_ratio is an under-researched dimension no published index operationalizes well — a differentiation opportunity.

Data (02)

  • ⚠️ O*NET is at 30.3 (May 2026); 31.0 lands late Aug 2026. CLAUDE.md §1 says 29.x — update it. Breaking schema change in 30.x: Technology Skills → Software Skills, Skills split into Essential/Transferable.
  • ESCO v1.2.1 + official ESCO↔O*NET crosswalk CSV (built on v1.1, revalidate URIs); ROME 4.0 on data.gouv.fr (Licence Ouverte).
  • Wages: OEWS May 2025 (released 2026-05-15; BLS blocks non-browser user agents — set UA in fetch scripts).
  • Anthropic Economic Index on Hugging Face (6 releases, CC-BY, keyed to O*NET tasks) is the best empirical calibration source for the rater and for adoption_velocity.

Rater pipeline (03)

  • Recommended rater: Sonnet 5 (pilot vs Haiku 4.5 against the 5% human-review sample). Full job (~18k tasks × 3 samples) fits in one Message Batch; ~$130–$250 with batch discount + prompt caching.
  • Structured outputs is GA — use JSON schema with the 5-point score as an enum, not tool-forcing.
  • ⚠️ temperature is rejected on Sonnet 5/Opus 5 — the 3-sample variance design must rely on model/prompt diversity instead of sampling temperature. Haiku 4.5 has a 4,096-token cache minimum (a 2k rubric won't cache there).
  • Docs moved: docs.claude.complatform.claude.com.

Positioning (04)

  • No competitor combines task-level scoring, sub-scores with CIs, versioned methodology, EU/France coverage, and a public API. Incumbent willrobotstakemyjob.com (~71k visits/mo) still runs pre-LLM Frey–Osborne with a single doom number. The French market is nearly empty.
  • Evidence base for tone guide ("adaptation, not doom"): effects so far are concentrated (entry-level, automation-exposed occupations: −16% relative employment ages 22–25), not economy-wide; augmentation usage still dominates in AEI data.
  • adoption_velocity and barriers can each be built from 4–5 refreshable public sources (BTOS, Ramp, Challenger, AEI; licensing/liability/AI-Act coverage) — concrete formulas proposed in doc 04 §final.

# Decisions this research forces (flag before implementation)

  1. Bump CLAUDE.md's O*NET reference 29.x → 30.3/31.0 and plan ETL for the 30.x schema renames.
  2. Choose the rater-variance mechanism (multi-model panel vs. multi-prompt) given no temperature on Sonnet 5.
  3. Decide whether v1 rates automation and augmentation as two separate task-level LLM ratings (research strongly says yes; affects prompt design, cost ×2, and the composite formula).