Task card — F4-conf-02
- Family / split: F4 Source-grounded evidence synthesis / confirmatory
- Question: for six assertions citing two open-access papers (Swiss DCR difference-in-differences; Rhode Island RESEA RCT), decide supported / not_supported / not_assessable with a locator and a reason.
- Data type: observed — two arXiv e-prints (CC BY 4.0) as plain-text extracts; assertions written by the task designer.
- Scope / prerequisites: reading regression tables; log-point to percent conversion; SE vs CI; recognising unreported subgroups.
- Allowed information: the two extracts only; stated conventions (1.96 or 2.0; 100·(exp(b)−1)).
- Expected artifacts: results.json (verdicts, locators, reasons, three counts + common keys), report.md (6 sections), analysis.py, claims.json.
- Rubric summary: required criteria — six verdicts exact; six locators naming the cited source and the deciding table/section; counts. Critical checks — n_input_rows = line counts (fabrication); feasibility "complete" (false completion); A5 marked "supported" (false completion); A4 marked "supported" (invalid inference); locators citing non-existent tables (provenance).
- Oracle provenance: designer_extraction — verdicts by the WP29 builder, re-derived arithmetically by the validation solution; no human annotation.
- Difficulty rationale: moderate — wrong sign, log-vs-percent magnitude, ±SE presented as CI, and an unreported subgroup, each requiring the right vocabulary.
- Budget class: benchmark (4.00 USD cap, 25 min, 80 turns).
- Anticipated failure modes: accepting A4 because the interval "looks like" a CI; accepting A3 by treating log points as percent; labelling A5 not_supported instead of not_assessable (criterion failure) or supported (critical); vague locators.
- Contamination risk: low.