IRT Specification (2PL)
Author: Simon-Pierre Boucher — contact@spboucher.ai
Model
For model m and item i: P(x_mi = 1) = σ(a_i (θ_m − b_i)).
Estimation
MAP via Adam gradient ascent on the penalized log-likelihood (apps/psychometrics/llmindex_psycho/irt.py):
- Priors: θ ~ N(0, 1); b ~ N(0, 1.5); log a ~ N(0, 0.5) (a > 0 by construction).
- Warm start: θ from row-accuracy logits (centered), b from inverted column-accuracy logits.
- Identification: θ recentered to mean 0 each step (shift absorbed into b); scale pinned by priors.
- Missing cells masked (models need not share identical item sets).
- Convergence: relative objective change <
tolerance(default 1e-6), max 500 iterations.
Uncertainty
SE(θ_m) = 1/√(Σ_i a_i² P(1−P) + 1/σ_θ²) over observed items (Fisher information + prior
precision). Published as 95% CIs on every score.
Hygiene
After each refit: items with a < 0.3 or |b| > 3 are flagged flagged_for_retirement in the
item bank and reviewed before the next run. Fit diagnostics (iterations, convergence, final
log-likelihood, matrix sizes) are stored on the score_runs row.
Deviation note
The spec allows py-irt / PyTorch; v1 ships a dependency-light custom numpy implementation with the
same 2PL likelihood and priors — verified by parameter-recovery tests
(apps/psychometrics/tests/test_psycho.py).