1# Human rating protocol (prepared, not executed)23Raters: ≥ 2 PhD-level finance researchers, blind to condition; each rates all outputs of a (task, document) set presented in random order under random letters.4Scale: the six primary dimensions of `prompts/evaluation/judge_rubric_v1.md` (1–7) + n_invented_facts + free comment. Calibration: raters first score two anchor sets together and discuss.5Reliability: Krippendorff's alpha (interval) per dimension; disagreement > 2 points resolved by discussion, original scores kept for analysis.6Outputs to rate are in `experiments/outputs/heldout_v1/outputs/`, blinded sets are generated by `scripts/06_make_rating_packets.py` (to be written when raters are available). Store ratings as `experiments/outputs/heldout_v1/human_ratings.csv` (rater, task, doc, rep, letter, dimension, score).7