1# Pilot report (development tasks, 1 repeat per arm)23Generated 2026-10-30T03:57:27+00:00 from `results/analysis_data/*.csv` by `analysis/pilot_report.py`.45Runs assigned 20, evaluated 20, tasks 5 (F1-dev-01, F2-dev-01, F4-dev-01, F5-dev-01, F6-dev-01). Model reported by the client: claude-opus-5-5; client 2.1.314 (Claude Code).67## Outcomes by arm89| arm | runs | Y (task-weighted) | success (any) | runs with violation | reproducible | quality mean | mean cost USD | median cost | max cost | mean wall s | max wall s | sessions/run |10|---|---|---|---|---|---|---|---|---|---|---|---|---|11| A | 5 | 1.00 | 1.00 | 0 | 1.00 | 1.000 | 0.212 | 0.194 | 0.353 | 66 | 114 | 1.0 |12| B | 5 | 1.00 | 1.00 | 0 | 1.00 | 1.000 | 0.345 | 0.312 | 0.452 | 103 | 140 | 1.0 |13| C | 5 | 1.00 | 1.00 | 0 | 1.00 | 1.000 | 0.388 | 0.404 | 0.506 | 110 | 151 | 1.0 |14| D | 5 | 1.00 | 1.00 | 0 | 1.00 | 1.000 | 0.558 | 0.462 | 0.840 | 153 | 229 | 2.0 |1516## Outcomes by task1718| task | Y A | Y B | Y C | Y D | cost A | cost B | cost C | cost D |19|---|---|---|---|---|---|---|---|---|20| F1-dev-01 | 1 | 1 | 1 | 1 | 0.19 | 0.30 | 0.40 | 0.46 |21| F2-dev-01 | 1 | 1 | 1 | 1 | 0.14 | 0.31 | 0.26 | 0.39 |22| F4-dev-01 | 1 | 1 | 1 | 1 | 0.21 | 0.31 | 0.42 | 0.67 |23| F5-dev-01 | 1 | 1 | 1 | 1 | 0.16 | 0.36 | 0.35 | 0.43 |24| F6-dev-01 | 1 | 1 | 1 | 1 | 0.35 | 0.45 | 0.51 | 0.84 |2526## Sessions2728| | n | cost_mean | turns_mean | duration_s |29|:------------------------|----:|------------:|-------------:|-------------:|30| ('executor', 'initial') | 20 | 0.34 | 10.75 | 100.11 |31| ('reviewer', 'review') | 5 | 0.13 | 5.8 | 29.71 |3233## Gate receipts (controller-side)3435| origin | passed | n |36|:----------------------------|---------:|----:|37| controller | 1 | 10 |38| final_common_contract_check | 1 | 20 |3940## Reviews (arm D)4142| verdict | n | issues_mean |43|:----------|----:|--------------:|44| accept | 5 | 2.8 |4546- Projected 24 tasks × 4 arms × 3 repeats: 288 runs ≈ 124 USD (1.15 × mean pilot cost 0.376), ≈ 1.1 h at 8 concurrent runs (mean pilot wall 108 s).47- Projected 24 × 4 × 2: 192 runs ≈ 83 USD (1.15 × mean pilot cost 0.376), ≈ 0.7 h at 8 concurrent runs (mean pilot wall 108 s).48- Projected 12 × 4 × 2: 96 runs ≈ 41 USD (1.15 × mean pilot cost 0.376), ≈ 0.4 h at 8 concurrent runs (mean pilot wall 108 s).4950Scale decision rule (preregistration §6): choose the largest configuration whose projected cost fits the remaining envelope after a 150 USD reserve and whose projected time is ≤ 12 h.51