SPB Git

spb/wp42_uqo

Public
1.8 KB · 9 lines csv
1id,utc,phase,decision,reason,status2A-001,2026-11-02T23:40:00Z,0,Study envelope 800 USD; 4.00 USD per session; 1800 s wall; 150 turns,The mandate gives no budget; same envelope as WP29; PI may change it,pending PI confirmation3A-002,2026-11-02T23:40:00Z,0,Human arms H0/H1 designed but not executed,No participants nor ethics approval exist; the mandate forbids LLM imitations of humans,pending PI confirmation4A-003,2026-11-02T23:40:00Z,0,A2 = headless macOS automation layer (osascript/System Events/macOS utilities) without GUI app control,GUI Apple Events time out over SSH (TCC consent cannot be answered on headless nodes),recorded5A-004,2026-11-02T23:40:00Z,0,Primary worker model claude-opus-5-5; Sonnet 5.5 and Fable 5.1 as sensitivities on A3,Cost: Fable 2.5x Opus per token in the probe; Opus was the WP29 subject,recorded6A-005,2026-11-02T23:40:00Z,0,Blind professional evaluators are LLM judges (3 per run); human expert review left to the PI,No human evaluators available in-session,recorded7A-006,2026-11-03T01:05:00Z,2,Per-scenario caps = max(global 4 USD / 1800 s; task.json budget_usd / time_limit_s) — ACC-01 (workflow 150 invoices) 6 USD / 3600 s; RES-01 6 USD / 2700 s; BANK-01 5 USD / 2400 s; LEGAL-01 2400 s; A4 recovery gets the remainder,Scenario authors sized the caps to the designed workload; set before any pilot run and before the freeze,recorded8A-007,2026-11-03T02:40:00Z,3,Freeze v1.0 of the 10 validated scenarios + oracles + harness (protocol/preregistration/freeze_manifest.json) after validator fixes (evaluators 1.1; RE-01/HR-01/RES-01/LEGAL-01/ADM-01 bundles regenerated at the same seed),All 10 validator solutions pass their evaluator and empty outputs fail (scripts check at freeze); no pilot run executed before this freeze except the dev run HR-01_A3_r1 on the pre-validation HR-01 bundle (stage dev; excluded),recorded9
Files · 97305fbe27600ca2c360f7f7baf413fe199a39ab
Menu

Theme