SPB Git

spb/wp29_uqo

Public
1.4 KB · 4 lines csv
1amendment_id,timestamp_utc,phase,before_or_after_results,document,description,rationale,previous_version_ref2A-001,2026-10-30T03:00:00Z,02,before_results,configs/budgets.yaml,"Provisional live-inference envelope set by the executor: per benchmark run 4.00 USD / 1500 s / 80 turns per session; per transfer run 12.00 USD / 3600 s; total study envelope 800 USD (pilot + main + transfer + recovery + ablations) with the confirmatory scale to be chosen after the pilot.","The mandate requires a precise budget decision but none was stated; the researcher is not available mid-run; prior UQO studies spent ≈1,079 USD of API inference (WP26), so 800 USD is a conservative envelope. The researcher may raise or lower it; any change will be a new amendment.",n/a3A-002,2026-10-30T04:40:00Z,06,before_confirmatory_results,src/pgar/evaluation.py,"Final evaluators are now executed on a composed workspace (public bundle + the run's collected out/) instead of the run directory alone, which lacked data/ and made source-grounded checks (F4 'numbers_in_sources') fail for correct answers. All pilot runs evaluated before the fix are re-evaluated with the same oracles; original receipts are kept in runs/pilot/<run>/evaluation_receipt_pre-A002.json.","Pilot diagnosis: F4-dev-01 failed in all four arms with a spurious fabrication violation although every required criterion passed. Harness defect, not an oracle change.",runs/pilot/*/evaluation_receipt_pre-A002.json4
Files · 90bf126799e046545d1bf5210fce451071dce757
Menu

Theme