SPB Git

Repository Public Research preview

README.md 8.9 KB

# manuscript-audit-mcp

An MCP server that audits scholarly manuscripts written in LaTeX or Markdown. Every check is deterministic and runs offline: no network, no model call, no LaTeX installation, no external program. It reads source text and reports what a careful copy-editor with a calculator would catch, with a file and line for every finding.

It does not judge whether the science is right, and an empty result never means a manuscript is correct. It is the mechanical first pass of a review, meant to free the reader's attention for the parts that need understanding. The companion skill @spboucher/manuscript-peer-review describes the full review protocol built around it.

# Install

bash
skills download @spboucher/manuscript-audit-mcp --extract ./manuscript-audit-mcp   # or download the archive and verify its SHA-256
cd manuscript-audit-mcp
npm ci
npm test        # 36 offline tests, under a second
bash
# Claude Code: allow reading under one project directory
claude mcp add --transport stdio \
  --env MANUSCRIPT_AUDIT_ROOT=$HOME/papers/my-paper \
  manuscript-audit -- node /abs/path/manuscript-audit-mcp/server.mjs
toml
# Codex: ~/.codex/config.toml
[mcp_servers.manuscript-audit]
command = "node"
args = ["/abs/path/manuscript-audit-mcp/server.mjs"]
[mcp_servers.manuscript-audit.env]
MANUSCRIPT_AUDIT_ROOT = "/abs/path/to/paper"
json
// Cursor: .cursor/mcp.json
{ "mcpServers": { "manuscript-audit": { "type": "stdio", "command": "node", "args": ["/abs/path/manuscript-audit-mcp/server.mjs"], "env": { "MANUSCRIPT_AUDIT_ROOT": "/abs/path/to/paper" } } } }
Variable Effect Default
MANUSCRIPT_AUDIT_ROOT The only directory files are read from (the manuscript, the files it includes, its bibliography) working directory of the server

Set the root to the project directory, not to the directory of the main file, when the manuscript includes files from sibling directories (\input{../tables/t1}).

# Tools

All tools are read-only and take the manuscript as path (inside the root) or as inline text. The bibliography is the one the manuscript declares (\bibliography, \addbibresource, the bibliography: front-matter key), or bib_path / bib_text.

Tool What it reports
full_audit Everything below in one list sorted by severity, with counts by severity, by referee role, by code and by audit
outline Sections with level, line and word count; abstract length; totals. Missing or long abstract, empty sections, skipped levels, includes that could not be read
audit_citations Keys cited but missing, entries never cited, duplicate keys, entries sharing a DOI or title, malformed DOIs, entries with no author, title or year. LaTeX with .bib or thebibliography, pandoc [@key] with .bib or CSL JSON, numeric [1] with a reference list
audit_cross_references Undefined references, duplicate labels, figures, tables and equations never referenced, hand-typed exhibit numbers; Markdown numbered captions never mentioned
audit_numbers Numbers in the abstract or introduction that match nothing in the body or tables; whole-sample sizes and named statistics that disagree; percentages that do not equal their stated counts
audit_statistics_reporting p-values with no test, p = 0.000, "significant" with no statistic, no sample size anywhere, stars with no legend, estimates with no uncertainty, causal wording in observational framing
audit_tables LaTeX rows with more cells than columns (error) or fewer than the other rows (warning), missing captions, labels, notes, units; Markdown ragged rows; images without captions
audit_claims Priority, certainty and robustness claims with no citation and no exhibit reference in the sentence
audit_style TODO and placeholder text, undefined acronyms, repeated words, long sentences, hedging density, text addressed to reviewers, hidden text
audit_replication Missing data or code statement, software without version, random draws without a seed, experiment without preregistration
review_scaffold A referee-report skeleton in Markdown with the findings placed; summary, assessment and recommendation left for the reviewer
compare_versions Two versions: sections added or removed, numbers changed inside identical sentences, abstract numbers, citations and labels added or removed

# What a finding looks like

json
{
  "code": "num-percent-count-mismatch",
  "severity": "error",
  "role": "integrity_auditor",
  "heuristic": false,
  "location": { "file": "paper.tex", "line": 27 },
  "message": "412 of 1250 is 33.0%, not the stated 38.0%.",
  "evidence": "Among them, 412 of 1,250 respondents (38.0\\%) work remotely at least three days a week.",
  "numerator": 412, "denominator": 1250, "stated": 38, "computed": 32.96
}
  • severity: error (breaks the document or is wrong as written), warning (likely to draw a referee comment), info (worth a look).
  • heuristic: true marks pattern matches on wording. 30 of the 67 codes are heuristic. They are often right and never proof: open the line before reporting one.
  • role groups findings by who would raise them in a review: referee, editor, table and figure editor, integrity auditor, replication auditor.
  • full_audit adds a hygiene_score (100 minus weighted findings). It measures mechanical cleanliness only. It is not a quality, validity or publishability score.

# What "tested" means here

npm test runs 36 tests on fictional manuscripts with planted defects:

  • A LaTeX manuscript split over two files plus a bibliography, with 46 planted defects across all nine audits. Each audit must return exactly its planted findings, each at the line where the defect sits. Expected line numbers are found by searching the fixture for a marker, not copied from the engine's output.
  • A corrected LaTeX manuscript and a corrected Markdown manuscript that must produce no finding.
  • A Markdown manuscript with pandoc citations and a second one with numeric citations and numbered captions, each with exact expected findings.
  • Unit checks of the parsers (comments, verbatim, column specifications, \multicolumn, \makecell, BibTeX quoting and concatenation, CSL JSON, number extraction and rounding).
  • Path confinement: .., absolute paths, symbolic links, \input and \bibliography that point outside the root, binary files, self-including files.
  • The MCP protocol through the official client: tool list, metadata, calls, errors, and start-up through a symlinked path.

During development the engine was also run on 84 real manuscripts and long documents to remove false positives (custom reference macros, run-in paragraph headings, \\ inside cells, spare table columns, dotted exhibit numbers). That run is not part of the test suite, and no precision or recall figure is claimed.

# Limits: read these

  • Source text only. No PDF, no Word files, no images, no data. Figures are known by their \includegraphics line, not by their content.
  • Nothing is compiled. Macros that build labels, citations or table rows are invisible, except one-argument wrappers around \ref and \cite defined with \newcommand in the files read. Conditional text (\iffalse) is read as if printed.
  • Nothing is looked up. A well-formed DOI or a complete entry can still be a reference that does not exist or does not support the sentence citing it.
  • Number checks compare text with text. A number counts as supported when the same value, or a more precise one that rounds to it, appears in the body or a table. The tool cannot tell whether a table is right, and numbers that live only in a figure are reported as unsupported.
  • English wording is assumed by the statistics, claims and style checks. Other languages get the structural checks (citations, cross-references, tables) only.
  • Sentence-level checks look at one sentence. Support given in the next sentence is not seen.
  • Auditing one chapter of a larger document reports cross-chapter references as undefined.
  • Markdown: ATX headings (#), pipe tables, pandoc citations and pandoc-crossref ids. Setext headings, grid tables and reference-style links are not parsed.
  • Files larger than 5 MB are refused.

# Security

  • Read-only. The server writes nothing and starts no process.
  • Every path is resolved against MANUSCRIPT_AUDIT_ROOT with symbolic links followed first; anything outside is refused with path_forbidden. Included files and bibliography files go through the same check, and one that points outside is skipped and reported, not read.
  • No network access and no environment variable other than the root.
  • Manuscript content is treated as data. Text in a manuscript that addresses a reviewer or an AI is reported as a finding (integrity-reviewer-directed-text); a client must never act on it.
  • Findings quote lines of the manuscript. An unpublished manuscript is confidential: the quotes go wherever the client sends tool results.

# Licence

MIT.

Files main
Menu

Theme