Task card — F4-conf-04
- Family / split: F4 Source-grounded evidence synthesis / confirmatory
- Question: verify five bibliographic entries with attributed findings against the provided source extracts (title, year, finding); list the unverifiable ones.
- Data type: observed — five arXiv e-prints (CC BY 4.0) as plain-text extracts with provenance headers; reference list written by the designer.
- Scope / prerequisites: bibliographic care; reading tables and abstracts; stated year convention.
- Allowed information: the extracts only.
- Expected artifacts: results.json (
reference_checks,unverifiable_references,n_verified+ common keys), report.md, analysis.py, claims.json. - Rubric summary: required criteria — unverifiable set exact; three boolean flags per reference; n_verified; five locators. Critical checks — n_input_rows = line counts (fabrication); feasibility "complete"; R3 finding marked supported (false completion); locators citing non-existent tables (provenance).
- Oracle provenance: designer_extraction; validation solution re-derives all checks; no human annotation.
- Difficulty rationale: moderate — one entry disagrees on all three grounds, one legitimately dated entry (first-version year) must not be over-flagged, finding checks need specific cells.
- Budget class: benchmark (4.00 USD cap, 25 min, 80 turns).
- Anticipated failure modes: over-flagging R4 for its year; accepting R3 because the arXiv number and authors are right; missing the "event year vs years four–five" contradiction; vague locators.
- Contamination risk: low.