SPB Git

spb/wp29_uqo

Public
4.4 KB

Complete the task described in TASK.md by following this research protocol. The protocol adds no scientific information to the task; it organizes how you work and what evidence you leave behind.

PHASES (execute in order; a phase is eligible only when its dependencies are satisfied):

P0 Scope and contract (depends on: nothing)

  • Read TASK.md completely. In out/state/contract.md, restate in your own words: the scientific question; the quantity or quantities to be estimated (the estimands) and the assumptions under which they are meaningful; every required output file and key; the information-availability rules; what would make the task impossible or the result invalid.
  • Write the initial phase ledger out/state/phases.json: a JSON object mapping each phase id (P0 … P5) to {"status": "pending"|"running"|"passed"|"failed"|"blocked"|"stale"|"skipped_optional", "checks": [], "notes": ""}. Keep it current as you work. A phase may only be set to "passed" when the checks listed for it in this protocol have actually been executed and their outcome recorded in "checks".

P1 Data audit and provenance (depends on: P0)

  • Inspect every input file: compute and record its SHA-256, row and column counts, schema, units, date ranges, missing values, duplicates and anomalies in out/state/data_audit.md.
  • Keep an exclusion ledger in out/exclusions.csv (columns: step, rule, n_before, n_after, n_removed, rationale) for every filtering step you apply, so that n_raw − total removed = n_analysed.
  • Identify any input that violates the information-availability rules and decide, in writing, how you will handle it.

P2 Baseline (depends on: P1)

  • Before any elaborate method, compute the simplest valid baseline that answers the question (for example a naïve or unconditional estimate), with the same evaluation rule used later. Record it in out/results.json under the keys the task requires, or under baseline_* keys when the task does not name them.

P3 Method and computation (depends on: P2)

  • Implement the analysis in out/analysis.py so that it reads only data/, writes out/results.json deterministically (fix every random seed you use and record it), and prints nothing that contradicts the written results. Any figure or table must be generated by this script.

P4 Verification (depends on: P3)

  • Re-run out/analysis.py from a clean state and confirm it regenerates out/results.json with identical values. Record the check in out/state/phases.json under P4.
  • Check internal consistency: identities that must hold (for example counts that must add up, standard errors that must be positive, confidence intervals that must contain their estimates, date orderings that must respect the information rules), units, and magnitudes against the baseline. Record every check and its outcome in out/state/checks.md. A failed check must be fixed or honestly reported, never silently ignored.

P5 Reporting (depends on: P4)

  • Write out/report.md with the sections: Question; Data and exclusions; Method; Results; Uncertainty; Limitations and open issues. Every quantitative statement must cite the key of out/results.json it comes from.
  • Write out/claims.json: a list of the central claims of your report, each as {"id": "C1", "text": "...", "results_keys": ["..."], "status": "supported"|"unsupported"|"not_evaluated", "caveat": "..."}. Do not state a claim in the report that is not in this ledger.
  • Set the final statuses in out/state/phases.json.

STOPPING RULES

  • Stop and declare the task complete only when every phase P0–P5 is "passed" with recorded checks. If a phase is "blocked" (missing information, impossible requirement, violated availability rule), stop, set the status, explain in out/report.md (section Limitations and open issues) and still write every required file with null for values you could not legitimately produce.
  • Do not spend effort on extensions that the task does not require.

RECOVERY RULES

  • Append a short dated entry to out/state/log.md after each phase transition and after any important decision, so that work can be resumed by reading out/state/.
  • If you discover that an upstream input or decision changed after a downstream phase passed, mark the downstream phases "stale" and redo them; do not reuse stale results.
  • If you are interrupted and later resumed, first read out/state/ and continue from the last consistent state; never re-run expensive steps that already have recorded, unchanged outputs.
Files · 502400bef152b80194d2a72e83992531749d18e1
Menu

Theme