1# LIMITATIONS (verified, 2026-11-03)231. **No human evaluation.** The mandate's H1 (blind human-rated) could not be tested; all quality outcomes are LLM-judge ratings (claude-sonnet-5-5) with a measured self-consistency check; judge biases (verbosity, style preference, same-vendor affinity) are not eliminated.42. **Single journal, four volumes.** Profiles describe JoF 2017–2020 practice (263 articles, 4 presidential addresses, 8 corrigenda/replications, 1 retracted article); they are not finance-wide norms and not editorial requirements.53. **Annotation is model-based.** The rhetorical annotations come from claude-opus-5-5 with schema validation and deterministic cross-checks (roadmap regex, contribution regex, magnitude regex, section-role rules). Model–rule agreement is reported; it is not human inter-rater reliability. ~10% of first-pass annotations failed strict schema validation and were re-run (see PROGRESS.md).64. **Parser is heuristic.** Layout heuristics tuned to the JoF LaTeX template; cross-parser agreement on a stratified sample is reported in `data/derived/parser_qa.md` (sections exact 85%, tables within ±1 96%, figures exact 92%, references within 10% 92%; numbered-equation counts are approximate). Corrigenda and one-page notes parse with low confidence and are excluded from rhetoric profiles by genre.75. **Pretraining contamination.** The generator and judge may have seen the held-out articles; mitigations (withheld introductions, overlap screening, citation verification against the reference list) reduce but do not remove this.86. **Small experimental sample.** Two held-out finance documents × 8 tasks (+ one out-of-domain manuscript), one to two repetitions: a pilot of the protocol, powered to detect only large effects; results are reported with CIs and Holm correction and must not be over-read.97. **Conditions differ in token budget.** E makes two generation calls (draft + referee) and therefore uses more tokens than A–D; cost and latency are reported as outcomes, not controlled.108. **Engine scope.** Numeric-preservation checks compare number sets, not meanings; "unsupported claim" detection is a regex heuristic; citation verification checks keys and DOIs, not whether the cited work supports the claim.119. **Platform**: tested on one Mac; the Claude Code integration targets version 2.1.317's command/agent/skill formats.1210. **H5 metric as preregistered was mis-specified**: "max overlap with any corpus document" counts overlap with the task's own source article (unavoidable for abstract/literature/specification rewrites). Both the preregistered value (max 0.36) and the refined value excluding the source document (max 0.006) are reported; the refinement is post hoc.1311. **H4 citation metric did not discriminate**: every condition received the reference list and an instruction not to invent; zero unverifiable author–year citations were produced in all conditions.1412. **The benchmark judge saw up to nine outputs per set** (conditions + ablations for the same task/document), which may induce relative rather than absolute scoring; the second-pass consistency (Spearman 0.60–0.91) is measured under the same design.1513. **Run 1 discarded (A-004).** The first benchmark run was invalidated by a Markdown-loader defect that dropped most of the fixture material and by a contradictory framing of two tasks; it is archived, not used, and the defects were found only by reading judge justifications — aggregate scores did not reveal them.1614. **Rated usefulness trade-off.** The referee pass improves rigor and fidelity but produces more hedged text that the judge rates slightly less useful; the engine's output is a verified proposal with flags, not finished prose.17