WP42: freeze v1.0 before the pilot (A-007)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Simon-Pierre Boucher committed 14 h ago (Nov 3, 2026) parent b4ee165 Browse files

2 changed files +33 −0

modified protocol/amendments.csv +1 −0

		`@@ -5,3 +5,4 @@ A-003,2026-11-02T23:40:00Z,0,A2 = headless macOS automation layer (osascript/Sys`
5	5	A-004,2026-11-02T23:40:00Z,0,Primary worker model claude-opus-5-5; Sonnet 5.5 and Fable 5.1 as sensitivities on A3,Cost: Fable 2.5x Opus per token in the probe; Opus was the WP29 subject,recorded
6	6	A-005,2026-11-02T23:40:00Z,0,Blind professional evaluators are LLM judges (3 per run); human expert review left to the PI,No human evaluators available in-session,recorded
7	7	A-006,2026-11-03T01:05:00Z,2,Per-scenario caps = max(global 4 USD / 1800 s; task.json budget_usd / time_limit_s) — ACC-01 (workflow 150 invoices) 6 USD / 3600 s; RES-01 6 USD / 2700 s; BANK-01 5 USD / 2400 s; LEGAL-01 2400 s; A4 recovery gets the remainder,Scenario authors sized the caps to the designed workload; set before any pilot run and before the freeze,recorded
	8	+A-007,2026-11-03T02:40:00Z,3,Freeze v1.0 of the 10 validated scenarios + oracles + harness (protocol/preregistration/freeze_manifest.json) after validator fixes (evaluators 1.1; RE-01/HR-01/RES-01/LEGAL-01/ADM-01 bundles regenerated at the same seed),All 10 validator solutions pass their evaluator and empty outputs fail (scripts check at freeze); no pilot run executed before this freeze except the dev run HR-01_A3_r1 on the pre-validation HR-01 bundle (stage dev; excluded),recorded

added protocol/preregistration/freeze_manifest.json +32 −0

		`@@ -0,0 +1,32 @@`
	1	+{
	2	+ "frozen_utc": "2026-11-03T00:32:05.568635+00:00",
	3	+ "git_head": "b4ee165b1a115d2a9e70eb178b5ccda08553f810",
	4	+ "scenarios": {
	5	+ "ACC-01 ADM-01 BANK-01 BI-01 FIN-01 HR-01 LEGAL-01 PROC-01 RE-01 RES-01": {}
	6	+ },
	7	+ "oracles": {
	8	+ "ACC-01 ADM-01 BANK-01 BI-01 FIN-01 HR-01 LEGAL-01 PROC-01 RE-01 RES-01": {}
	9	+ },
	10	+ "harness": {
	11	+ "src/wp42/arms.py": "72f1c834ad573abbe960f49a41f347e9893fa5cbd70fc870e4314616bd6a9a01",
	12	+ "src/wp42/launcher.py": "4fe88b2ca934c51a7e6045d51b1169f869c990fae47fc50724efa5be01114133",
	13	+ "src/wp42/bundle.py": "1182092b120f974d394bad636327c6324690e49306ecd0fd0fd7ea21c88d8f10",
	14	+ "src/wp42/scheduler.py": "42f13a087ea5eb290d4e39535a6d7e2872aa31684027f56243940fa181759326",
	15	+ "src/wp42/judge.py": "6deb8032bc6d3fb34170f0e8daf66c2b39b8c4ef83bf5ae14a19efd75c31aa5e",
	16	+ "src/wp42/evaluate.py": "996f91ca2db957fb7c81fbe633238f98ff1c6a64bb89075a0e65127fe3db1bca",
	17	+ "src/wp42/metrics.py": "c0e3b5aa7616c918ba039230f3b14cec4c1c9b7d1b9ed95b27e014de552a2c02",
	18	+ "src/wp42/config.py": "25020277bb3e3772d2ecb43b1c1d02e06d6ba5dab5df04d3ae08d460546ead36",
	19	+ "src/wp42/portal.py": "98f0981eea6604ef65dbe180c4daa2f8a9e50e16ec05dc5492d1a212275f2a9d",
	20	+ "src/wp42/office.py": "fbe9a2cb4c9bb640fd6df383d2367879caef87cfba90d453704a8d7e2c72076b",
	21	+ "benchmark/tools/selfcheck.py": "fe0c14daab875da9e0509df378166f8f9e4b5c3e04725f4df4c2c0c9b0c3eae2",
	22	+ "benchmark/validators/common.py": "a639b77b5cb99b09f307a1386b66b250997e7cb031a0e92841b1aed4edb04f4a",
	23	+ "benchmark/BRIEF_FOOTER.md": "67433becb7262bb5714186d1b0e9a17f0dd97cb67ea5192c785f09407d95f562",
	24	+ "benchmark/COMPLETION_SCHEMA.json": "bffb459f612a192f694ac9fd67a25192ccc0d484c0dc698fcfbe7a2db55cce03"
	25	+ },
	26	+ "protocol": {
	27	+ "protocol/preregistration/PREREGISTRATION.md": "5e6a92280ca700a2d237fbb37fe70164923e16ce30922d8f10ae03d862c235bf",
	28	+ "RESEARCH_PLAN.md": "23f1e647d2f41fb658c33b8661c7a1c34c6d434cb8a240d535aa3ff608a7c600",
	29	+ "analysis/econometrics/estimate.py": "bd9c6df4a414da84804c9294034a5b18d95f91d1d86ceee3676bf470bd41c853"
	30	+ },
	31	+ "claude_code_controller": "2.1.316 (Claude Code)"
	32	+}
		\ No newline at end of file
	33

spb/wp42_uqo

WP42: freeze v1.0 before the pilot (A-007)

Theme