# Procedure in detail

Eight stages. Do not skip stage 4: an unverified finding in a referee report costs more credibility than a missed one.

# Stage 1: intake

Record, before reading:

• Files. Main file, included files, bibliography, supplementary material, code, data. List what was provided and what was not.
• Version. A commit hash, a file checksum or the modification date of the main file. The review is about this version only.
• Request. Referee report, pre-submission check by the author, editor's assessment, check of a revision. The deliverable differs (see templates.md).
• Venue and field, when known: they set the bar for contribution, the expected reporting standards and the length.
• Constraints. Confidentiality (an unpublished manuscript is not sent to external services or quoted outside the review), conflicts, deadline.

If the manuscript is a PDF or a word-processor file, the scripts cannot read it. Convert it to text for reading and say in the report that the mechanical audit was not run, or run it on the source if the author can provide it.

# Stage 2: deterministic audit

node scripts/manuscript.mjs audit paper.tex --root . --text

1. Check the first lines of the output: how many files were read, and whether any include was not read (outline-include-not-read). Fix the root and rerun before going further.
2. Go through the errors. They are few and usually real: a missing bibliography entry, an undefined reference, a percentage that does not match its counts, a leftover TODO.
3. Go through the warnings, opening each line. Dismiss a finding when the context answers it (the test is named in the previous sentence; the "inconsistent" sample sizes are two subsamples). Keep a one-line reason for each dismissal.
4. Skim the notes (info level). They are prompts for the role reviews, rarely findings.

The confirmed audit findings enter the ledger with the prefix A (A-01, A-02 ...). They are almost always minor comments. A few are not: a number in the abstract that appears nowhere in the paper, or a sample size that changes between sections, can be major.

# Stage 3: role reviews

Five independent readings, defined in roles.md. For each role:

1. Read the whole manuscript once without writing.
2. Answer the role's questions, writing a finding whenever an answer is "no" or "cannot tell".
3. End with "Not assessed": what the role could not evaluate and why.

Budget: a role review that produces more than about fifteen findings has usually mixed major and minor. Rank and cut.

When delegating to subagents, the brief for each role contains: the role description from roles.md, the paths of the manuscript files, the finding format, and the rules of SKILL.md. It does not contain the audit findings (except for the role they belong to) or your opinion of the paper.

# Stage 4: adversarial verification

For each finding, the verifier's job is to refute it.

1. Locate. Open the location. Is the quote exact? If the quote cannot be found, the finding is refuted (and the role that produced it needs a second look).
2. Widen. Read the paragraph before and after, the table notes, the footnotes, the appendix section that the passage points to.
3. Search. If the finding says something is missing, search the manuscript for it under other names (a robustness table may be called "sensitivity"; a data statement may be in a title-page footnote).
4. Recompute. If the finding is about arithmetic or consistency, redo it from the numbers in the manuscript and write the calculation down.
5. Classify. confirmed / not shown / refuted / unverifiable, with one line of reason.

Typical refutations: the support is in the next sentence; the "undefined" acronym is defined in a table note; the two "inconsistent" numbers are before and after a stated filter; the cross-reference is defined in a chapter that was not part of the audit.

Count and report the outcome: how many findings were raised, confirmed, refuted. A refuted share near zero means the verification was not adversarial enough.

# Stage 5: synthesis

The editor role takes all confirmed and not-shown findings and:

1. Merges duplicates. Two roles reporting the same problem is one comment; keep the better-evidenced wording and note that it was found independently.
2. Assigns severity. Major: the paper's main conclusions depend on it, or it would mislead a reader about what was done. Minor: everything else.
3. Orders major comments by how much of the conclusion depends on them.
4. Groups minor comments by section or by exhibit, so the authors can work through them.
5. Cuts. Matters of taste that the authors may reasonably decline are dropped or clearly marked as suggestions.

# Stage 6: scoring

Use rubric.md. Score each dimension from its anchors and write one or two sentences that tie the score to specific findings. Apply the caps. Report the total with the per-dimension scores, never alone.

For a revision, score both versions with the same anchors and report the two totals side by side, with the findings that moved each dimension.

# Stage 7: the report

Write from templates.md. Principles:

• Start with a summary of the paper that the authors would accept as fair. If you cannot write it, you have not understood the paper well enough to review it.
• Each major comment states: what the problem is, where, why it matters for the conclusions, and what would resolve it (an analysis, a clarification, a narrower claim).
• Say what is good, specifically. It tells the authors what to keep.
• Separate what you ask for from what you suggest.
• State what was not reviewed.
• Keep the recommendation and any confidential remarks for the editor's letter, unless the venue's format asks otherwise.

# Stage 8: revision and response letter

1. Run compare on the two versions and the audit on the new one.
2. Build the response table (templates.md): one row per point of the previous report.
3. For each row, find the change in the new version. Classify: resolved, partly resolved, not resolved, resolved differently than asked (and whether that is acceptable), new problem introduced.
4. Check the changed numbers that compare lists: is each change explained in the response letter? An unexplained change in a headline number is a major comment.
5. Do not raise new comments on unchanged material unless they are serious: say explicitly that they are new and why they were missed.