# WP29 — Content-coding protocol for the mandate corpus (codebook)

Version 1.0 — 2026-10-29. Applies to `research/corpus/*.md` (13 files, hashes in `research/corpus_manifest.csv`).

## 1. Scope, provenance and limits of the coding

- **Corpus.** Thirteen files retrieved on 2026-10-29 (UTC 2026-10-30T03:01). They contain **eleven distinct CLAUDE.md mandates** (WP15, WP18, WP19, WP20, WP21, WP22, WP23, WP24, WP25, WP26, WP28), **one supplementary design brief** (WP27 `DESIGN-BRIEF.md`, which is not a CLAUDE.md) and **one byte-identical duplicate** (`WP15_PROMPT_PROMPT_INITIAL_CLAUDE.md`, SHA-256 `2dd19c59…1b77`, identical to `WP15_CLAUDE.md`). The duplicate is recorded but coded once. WP27's own CLAUDE.md returned HTTP 404 and is therefore absent; nothing is inferred about it.
- **Who coded.** A **single automated + LLM-analyst pass** (Claude Code subagent, model Claude Fable 5.1, 2026-10-29). The automated variables were computed by `research/code_corpus.py`; the analyst variables were produced by the same subagent after reading each document in full. **There was no human second coder, so no intercoder reliability (kappa, agreement rate) is reported and none should be inferred.** The analyst columns are the subagent's reading of the text and may contain judgement errors; every yes/partial/no judgement is accompanied by a verbatim quote so that it can be re-checked against the source.
- **What the coding can and cannot support.** The corpus is **small (11 mandates) and researcher-selected** (the author's own instruction files for earlier UQO working papers). It is used to **motivate design patterns** for WP29's protocol, **not to estimate the prevalence** of practices across autonomous-research mandates in general. Denominators are reported as "x of 11 mandates" and no inferential statistics are computed.
- **What is coded.** Only what the instruction documents say. The coding does **not** assess whether the earlier working papers were good, whether the instructions were followed, or whether the papers were produced without human assistance. A mandate that *requires* a provenance manifest is coded as requiring one; whether such a manifest exists is outside this coding.
- **Verbatim vs interpretation.** In `corpus_coding.csv`, verbatim material is always enclosed in double quotes `"…"`; everything outside the quotes is analyst interpretation. Quotes are copied exactly (including capitalisation, spelling and Markdown markup) and truncated with `…` only when the sentence exceeds roughly 40 words; nothing is paraphrased inside quotation marks.

## 2. Files produced

| File | Produced by | Content |
|---|---|---|
| `research/code_corpus.py` | script (stdlib only) | computes the automated variables |
| `research/corpus_automated.csv` | script | 13 rows (one per file, duplicate flagged) |
| `research/corpus_coding.csv` | analyst (written with `csv` module, RFC-4180) | 13 rows: 11 mandates + 1 design brief + 1 duplicate-note row |
| `research/corpus_summary.md` | analyst | counts, patterns, excerpts |

## 3. Variable types

- **(A) automated** — computed by `code_corpus.py` from the file bytes; fully reproducible by re-running the script.
- **(I) analyst-interpreted** — a categorical judgement made after reading the whole document; always accompanied by a supporting verbatim quote or by `none`.
- **(V) verbatim extraction** — text copied from the document, or metadata copied from `corpus_manifest.csv`.

## 4. Automated variables (`corpus_automated.csv`) — type (A)

| Variable | Definition |
|---|---|
| `file` | file name in `research/corpus/` |
| `wp` | leading `WP\d+` token of the file name |
| `document_type` | `claude_md`; `design_brief` if the name contains `BRIEF`; `duplicate_claude_md` if the SHA-256 equals that of an earlier file in sorted order |
| `duplicate_of` | file name of the first file with the same SHA-256, else empty |
| `sha256`, `sha256_matches_manifest` | SHA-256 of the bytes; `yes` if equal to the manifest value |
| `manifest_url`, `manifest_retrieved_utc` | copied from `corpus_manifest.csv` |
| `bytes`, `lines`, `words` | byte length; newline count (+1 if no trailing newline); `len(text.split())` |
| `n_headings` | lines matching `^#{1,6}\s+\S` (ATX Markdown headings) |
| `n_phase_labels_distinct` | number of distinct N in matches of `\bphase\s*(\d+)` (case-insensitive) |
| `n_stage_labels_distinct` | distinct capital letters X in `(?i:\bstage)\s+([A-Z])\b` |
| `n_gate_labels_distinct` | distinct N in `\bgate\s*(\d+)` (case-insensitive) |
| `n_numbered_items_in_phase_sections` | numbered list items (`^\s*\d+\.\s`) located inside a section whose heading contains one of *phase(s), stage(s), sequence, execution order, work sequence, execution protocol, execution phases, milestones*; a section extends to the next heading of equal or higher level |
| `n_numbered_items_total` | all numbered list items in the file |
| `n_numbered_phases` | mechanical phase count. Rule, in order: distinct `Phase N` labels if any; else distinct `Stage X` labels; else distinct `Gate N` labels; else numbered items inside phase/sequence sections. `n_numbered_phases_rule` records which branch fired. Known limits: WP20 states a 6-step "Mandatory sequence: (1) … (6)" inline with parentheses (not matched) and 7 numbered gates (matched → 7); WP21 lists steps "A … G" inline (→ 0); WP28's "Stage A–G" are message-protocol steps, not phases, but its 13 `Phase N` labels take precedence. The analyst variable `n_phases_declared` is authoritative. |
| `n_tables` | Markdown table separator rows (`|---|---|`): one per table |
| `n_code_blocks` | pairs of ``` fences |
| `language_of_document`, `fr_token_share` | heuristic: share of French function-word tokens (le, la, les, des, est, une, ce, cette, dans, pour, sur, avec, pas, aux, qui, que, ci-dessous, dossier, appelle, correspond, auteur) among French + English function-word tokens; `en` < 0.015 ≤ `mixed` < 0.50 ≤ `fr`. Tokens shared with English in this corpus (*conventions, journal, du, et*) are excluded. Result: all files `en` except WP18 and WP20 (`mixed`: a French prefatory paragraph on an English body). |
| `mentions_<family>` / `mentions_<family>_n` | boolean / count of case-insensitive regex matches. Families: `preregist` `pre-?regist\w*`; `estimand` `\bestimands?\b`; `hypothes` `\bhypothes\w*`; `falsif` `\bfalsif\w*`; `baseline` `\bbaselines?\b`; `leakage` `leakage|look-?ahead|look ahead|point-in-time|point in time`; `provenance` `provenance|hash(es|ed|ing)?|sha-?256|checksums?`; `reproduc` `\breproduc\w*`; `stopping` `stopping|stop rules?|stop conditions?|termination|stop and|stop if|stop only`; `recovery` `recover\w*|resum\w*|checkpoint\w*`; `gate` `gates?|acceptance|criteria`; `exhibit` `screenshots?|figures?|tables?`; `disclos` `disclos\w*|AI-use|AI use|AI contribution`; `budget` `budgets?|costs?|USD|\$`; `license` `licen[cs]e[sd]?|licensing`; `seed` `seeds?|seeded`. **Caveat:** these are mention counts, not judgements. The `budget` family is inflated where "cost" is a research object (WP20: the *cost approach* to appraisal, 42 hits; WP24: adjustment *costs*); the `gate` family also catches "criteria" used in "inclusion criteria" (WP21). The analyst variables, not these counts, carry the substantive coding. |

## 5. Analyst variables (`corpus_coding.csv`)

Allowed values for every yes/no-type variable are **`yes` / `partial` / `no`** (plus `n/a` for the duplicate row). `partial` means the mandate addresses the feature but only indirectly, conditionally, or without an operational mechanism (e.g. it *requires* that assumptions be stated but does not state them). Each value is followed, in the same cell, by a supporting verbatim quote `"…"` or by an analyst remark outside quotes.

| Variable | Type | Definition and allowed values |
|---|---|---|
| `wp` | V | WP15 … WP28; `WP27_BRIEF` for the design brief; `WP15_PROMPT` for the duplicate row |
| `file` | V | corpus file name |
| `document_type` | I | `claude_md` (an instruction file addressed to Claude Code); `design_brief` (WP27, an internal description of the simulator, not a mandate); `duplicate` |
| `source_url`, `retrieved_utc`, `sha256` | V | copied from `corpus_manifest.csv` |
| `research_domain` | I | free label: real estate economics; finance / market microstructure; macro-finance theory; econometric theory / statistical computing; applied econometrics (meta-analysis); etc. |
| `research_type` | I | `empirical`, `theoretical`, `simulation`, `design-science`, or `mixed (…)` with the components in parentheses |
| `explicit_question` | V | the research question verbatim (≤ 40 words; the first RQ when several are listed), or `none` when the document states objectives but no question |
| `explicit_estimand_or_hypotheses` | I + V | `yes` = named estimand(s) / hypotheses / theorems to prove, or a pre-specified primary outcome; `partial` = research questions or candidate contributions treated "as hypotheses" without formal statements; `no` |
| `stated_assumptions` | I + V | `yes` = the mandate itself states substantive model/identification assumptions or requires a numbered assumptions register; `partial` = requires that assumptions be stated with each result but does not state them; `no` |
| `falsification_or_null_result_conditions` | I + V | `yes` = explicit instruction to attempt disproof, required counterexamples/placebos, or a declared acceptable null outcome; `partial` = failures/negative results must be reported but no disproof condition is defined; `no` |
| `literature_or_novelty_requirement` | I + V | `yes` = verified literature review and/or novelty gate required before claiming a contribution; `no` |
| `data_access_controls` | I + V | `yes` = explicit rules on credentials, secrets, scraping, licensing, privacy or permission; `partial` = only generic "do not pretend access / log unavailable files"; `no` |
| `temporal_information_controls` | I + V | `yes` = look-ahead / as-of-date / point-in-time / vintage / chronological-split rules; `partial` = vintage awareness without a leakage rule; `no` (including not applicable) |
| `baseline_requirement` | I + V | `yes` = named comparison baselines/benchmark methods; `partial` = a "baseline scenario" or benchmark model, not comparison methods; `no` |
| `n_phases_declared` | I | integer count of execution phases/stages the document declares, or `none`; the labelling scheme is noted in parentheses |
| `dependency_or_gate_structure` | I + V | `explicit` = named gates or "do not start X until Y" conditions; `implicit` = ordering only; `none` |
| `stopping_rules` | I + V | `yes` = conditions under which work must stop / a verdict be issued; `partial` = escalation or fallback rules only ("if blocked, document and deliver partial result"); `no` |
| `recovery_or_resume_instructions` | I + V | `yes` = resumable/idempotent/checkpointed execution required; `partial` = phase logs, caching or retry limits that enable resumption only indirectly; `no` |
| `artifact_requirements` | V (list) | semicolon-separated list of required deliverables (paper, tables, figures, code, data, README, logs, audits …) |
| `provenance_requirements` | I + V | `yes` = hashes/manifests/pinned revisions/traceable number-to-file mapping/versioned outputs; `partial` = "every number must map to a result file" or logs without hashes/manifests; `no` |
| `review_or_self_audit_stage` | I + V | `yes` = a dedicated audit/referee/validation stage; `partial` = audit items inside a final checklist only; `no` |
| `disclosure_or_AI_use_statement` | I + V | `yes` = an AI-use / authorship-disclosure instruction; `no` (statements about prose style, e.g. "not AI-generated prose", are **not** disclosure) |
| `budget_or_resource_limits` | I + V | `yes` = an explicit budget (API cost tiers, token budgets) or numeric resource cap; `partial` = resource estimation/profiling or "within available resources" without a cap; `no` |
| `integrity_prohibitions` | I + V | `yes` = explicit "never invent / fabricate …" prohibitions; `no` |
| `contradictory_or_non_operational_requirements` | I | free text listing instructions that cannot be verified from the document's own criteria, or that conflict with another instruction; `none found` otherwise |
| `statements_added_during_implementation` | V | passages that the document itself marks as added later or reconstructed (headers such as "added 2026-10-27", bracketed reconstruction notes, prefatory mapping notes dated after the programme); `none identified` otherwise |
| `analyst_notes` | I | anything else relevant |

## 6. Procedure

1. Compute hashes and automated variables (`python3 -I research/code_corpus.py`); confirm every hash equals the manifest (`sha256_matches_manifest = yes` for all 13 files) and identify duplicates.
2. Read each distinct document in full (the two long documents, WP15 at 2 165 lines and WP28 at 1 055 lines, were read in consecutive line ranges).
3. For each analyst variable, locate the supporting passage, copy it verbatim, then assign the value. Where no passage exists, assign `no`/`none` without quote.
4. Write `corpus_coding.csv` with Python's `csv` module (`QUOTE_ALL`), re-read it with the same module and check the row count (13) and column count.
5. Tabulate counts over the 11 mandates only (brief and duplicate excluded) for `corpus_summary.md`.

## 7. Known weaknesses of this coding

- Single coder; no reliability statistics (see §1).
- Keyword counts are crude (see the `budget`/`gate` caveats in §4) and are reported only as descriptive context.
- The `partial` category absorbs heterogeneous cases; the quotes allow a reader to re-cut the categories.
- The design brief (WP27) is a different genre; its row is kept separate and excluded from all "of 11" counts.