1# Evaluator validation (seeded artefacts)23Generated 2026-10-30T04:34:23+00:00 by `evaluators/run_validations.py`. Each hidden oracle ships a `validation/` set with ≥1 correct_out (an independently written solution whose analysis.py is re-run) and ≥2 wrong_out artefacts (planted scientific errors, fabrication, false completion or leakage). The evaluator must accept the former and reject the latter; a PASS means every seeded case received the expected verdict (false acceptance = 0 and false rejection = 0 on the seeded set). The seeded set is small and designer-authored; it bounds gross evaluator errors, not the full space of valid solutions.45| task_id | result | last line of validation output |6|---|---|---|7| F1-conf-01 | PASS | ALL PASS |8| F1-conf-02 | PASS | ALL PASS |9| F1-conf-03 | PASS | ALL PASS |10| F1-conf-04 | PASS | ALL PASS |11| F1-dev-01 | PASS | ALL PASS |12| F2-conf-01 | PASS | ALL OK |13| F2-conf-02 | PASS | ALL OK |14| F2-conf-03 | PASS | ALL OK |15| F2-conf-04 | PASS | ALL OK |16| F2-dev-01 | PASS | ALL OK |17| F2-smoke-01 | no validate.py | |18| F3-conf-01 | PASS | ALL PASS |19| F3-conf-02 | PASS | ALL PASS |20| F3-conf-03 | PASS | ALL PASS |21| F3-conf-04 | PASS | ALL PASS |22| F3-dev-01 | PASS | ALL PASS |23| F4-conf-01 | PASS | VALIDATION PASSED |24| F4-conf-02 | PASS | VALIDATION PASSED |25| F4-conf-03 | PASS | VALIDATION PASSED |26| F4-conf-04 | PASS | VALIDATION PASSED |27| F4-dev-01 | PASS | VALIDATION PASSED |28| F4-sources | no validate.py | |29| F5-conf-01 | PASS | ALL PASS |30| F5-conf-02 | PASS | ALL PASS |31| F5-conf-03 | PASS | ALL PASS |32| F5-conf-04 | PASS | ALL PASS |33| F5-dev-01 | PASS | ALL PASS |34| F6-conf-01 | PASS | VALIDATION PASSED |35| F6-conf-02 | PASS | VALIDATION PASSED |36| F6-conf-03 | PASS | VALIDATION PASSED |37| F6-conf-04 | PASS | VALIDATION PASSED |38| F6-dev-01 | PASS | VALIDATION PASSED |39| T1-forecast | PASS | ALL PASS |40| T2-simulation | PASS | ALL PASS |41| T3-synthesis | PASS | ALL PASS |4243Summary: 33 PASS, 0 FAIL, 2 without validation set, of 35 oracles.44