Task card — F4-conf-01
- Family / split: F4 Source-grounded evidence synthesis / confirmatory
- Question: extract 19 printed point estimates / SEs / CI bounds / counts from Table 6 and Table 7 of a Swiss DCR difference-in-differences paper and Table 2 of a Rhode Island RESEA RCT paper, each with a table/row/column locator; derive a 95% CI from an SE and an SE from a CI with a stated convention; recognise one quantity (an exact p-value) that is not printed.
- Data type: observed — two open-access arXiv e-prints (CC BY 4.0), provided as plain-text extracts with provenance headers.
- Scope / prerequisites: reading regression tables; SE vs CI conventions; star conventions.
- Allowed information: the two extracts only.
- Expected artifacts: results.json (27 task keys + common keys), report.md (6 sections, Uncertainty section distinguishing SE from CI), analysis.py, claims.json.
- Rubric summary: required criteria — three groups of printed values exact; derived CI bounds within 0.02; derived SE within 15; p-value null; two booleans; 24 locator criteria (source id + correct table). Critical checks — n_input_rows = line counts (fabrication); feasibility "complete" (false completion); numeric p-value where none is printed (fabrication); every printed value must occur in the extracts (fabrication); locators citing tables absent from the extracts (provenance).
- Oracle provenance: designer_extraction (WP29 builder), cross-checked by an independent regex parser in the validation set; no human annotation.
- Difficulty rationale: moderate — two uncertainty conventions, a sign that the prose hides, a duplicated value with a conflicting SE in an appendix table, one null key; volume is small.
- Budget class: benchmark (4.00 USD cap, 25 min, 80 turns).
- Anticipated failure modes: ±SE bounds reported as a CI; CI half-width reported as an SE; SE 0.218 taken from Table A19; weeks effect reported as +1.99 with CI [1.75, 2.24]; observations from the wrong column; inventing a p-value (e.g. 0.25) for the HIV cell; locators without table numbers.
- Contamination risk: low-to-medium (2026 e-prints; cell-level locators are not memorisable).