# Brief for task-family authoring agents (WP29)

You are authoring benchmark tasks for UQO Working Paper 29, a controlled study in which fresh, offline Claude Code sessions (model claude-opus-5-5, bare mode, no network) solve bounded scientific tasks under four different instruction treatments. Your tasks will be solved by those sessions and graded by deterministic hidden evaluators that you write. Read, in this order, before writing anything:

1. tasks/TASK_AUTHORING_SPEC.md — the frozen interface (bundle layout, TASK.md template, common keys, gate_spec.json, oracle layout, receipt schema, quality components, family requirements). Follow it exactly.
2. The complete exemplar: public bundle tasks/public_bundles/F2-smoke-01/ (TASK.md, gate_spec.json, data/, bundle.json), card tasks/task_cards/F2-smoke-01.md, hidden oracle ~/wp29-oracles/F2-smoke-01/ (generator.py, oracle.json, evaluate.py, REFERENCE.md).
3. evaluators/public_evaluation_interface/receipt_schema.json.

Rules:

• Everything in English. Use the repository venv .venv/bin/python (numpy 2.3.4, pandas 2.3.3, scipy 1.16.2, statsmodels 0.14.5, matplotlib 3.10.7 — the same versions the executors have) for generators, references and tests. Executors will run ~/wp29/venv/bin/python with the same packages.
• Deliver for each task: public bundle + task card + hidden oracle directory (generator.py when synthetic, oracle.json, evaluate.py exposing evaluate(out_dir, oracle, workspace_dir, run_id="?") -> receipt, REFERENCE.md) + a validation set ~/wp29-oracles/<task_id>/validation/ containing at least one correct_out/ (a full set of deliverables that must receive scientific_success = true — write it as an independent solution, with its own analysis.py that actually runs) and at least two wrong_out/<name>/ (e.g. a leaky/wrong-method solution, a fabricated or false-completion solution) that must receive scientific_success = false, plus validate.py that runs evaluate.py on each and asserts the expected verdicts and prints a one-line summary per case. Run validate.py and make it pass. Evaluators must never raise on malformed or missing files.
• Never write reference values, trap descriptions or evaluation logic into the public bundle. TASK.md must read like a normal, fair scientific assignment. Do NOT leak hints such as "check for duplicates" unless the task is explicitly an audit task where that is the question itself.
• Real public data: fetch now with curl/urllib, record source URL, retrieval UTC, license/terms, SHA-256 in data/README.md inside the bundle (the executors must know provenance) and in REFERENCE.md. Verified working endpoints (no key): FRED https://fred.stlouisfed.org/graph/fredgraph.csv?id=<SERIES>; ALFRED vintages https://alfred.stlouisfed.org/graph/alfredgraph.csv?id=<SERIES>&vintage_date=YYYY-MM-DD; Bank of Canada Valet https://www.bankofcanada.ca/valet/observations/<series>/json?start_date=…; World Bank https://api.worldbank.org/v2/country/<ISO3>/indicator/<CODE>?format=json&per_page=…; Statistics Canada WDS; Our World in Data CSV grapher endpoints; arXiv full texts for F4. Keep bundles small (< 5 MB).
• Reference values must come from an INDEPENDENT computation (your own code, written from the definition, not copied from a solution you also hand to the executor) or from analytical identities or from a source document with page/table locators. Justify every tolerance numerically in REFERENCE.md (sampling variability, convention differences, rounding).
• Tolerances must be fair: accept any scientifically valid implementation (e.g. CR1 vs CR0 cluster adjustments, t vs normal critical values, different but valid seasonal-naïve definitions) while rejecting wrong science (leakage, wrong estimand, ignoring the dependence structure, wrong units).
• Every evaluator must include at least one critical-violation check of each relevant type (leakage, fabrication, invalid_inference, unreported_substitution, false_completion) that is deterministically detectable from the deliverables (e.g. reported n_input_rows must match the file; feasibility must be "complete" on a feasible task; on an infeasible/null-truth task, the claim ledger must not mark the impossible claim "supported"; a reported out-of-sample error below the information-feasible bound signals leakage).
• Do not edit tasks/registry.csv (another process merges it). Instead write tasks/registry_rows/<task_id>.json with the registry fields: task_id, family, split, data_type, real_data, source_documents, oracle_provenance, bundle_sha256 (sha256 of bundle.json), oracle_sha256 (sha256 of oracle.json), infeasible_or_null_truth, created_utc.
• bundle.json must be produced LAST (after data and TASK.md are final) with the same script logic as the exemplar (sha256 of every file except bundle.json).
• Budget class for all benchmark tasks: 4.00 USD, 25 minutes wall, 80 turns per session. Design tasks to be solvable by a careful scientist in 10–20 minutes of agent work; difficulty must come from scientific care (traps, information rules, inference validity), not from volume.
• Do not run any Claude Code session yourself and do not solve the tasks with a language model; your correct_out solutions are your own code.
• Report back: a table of task_ids with one-line description, data type, oracle provenance, planted trap/null-truth, validation result (pass/fail) and anything that could not be done.