# 📝 JoF-250: Learning Scientific Writing Conventions from Pre-LLM Financial Research

UQO Working Paper No. 45 — A measured-convention writing engine for Claude Code, built from 276 Journal of Finance articles (2017–2020), with a preregistered evaluation

$Paper$

TL;DR — We parse 276 pre-ChatGPT Journal of Finance articles (vol. 72–75), annotate their rhetorical structure with a schema-validated LLM ontology, turn the train split into measured writing profiles (medians, IQRs, bootstrap CIs), index structural units for hybrid retrieval with provenance, and expose the result as a Claude Code writing/review/audit engine (/jof-write, /jof-review, /jof-audit, …). A preregistered held-out experiment compares five conditions (baseline → full engine) with a blind LLM judge and deterministic fidelity audits. See docs/DELIVERY_REPORT.md for measured results.

# 📌 Status

Item	Value
Version	1.0 (November 2026)
SSRN	not yet submitted
Journal	—
Code & data	pipeline reproducible from `make` (see Quick start); PDFs not redistributed

# 🎯 What this paper does

It asks whether abstract, domain-specific writing conventions extracted from a pre-LLM corpus improve LLM-assisted scholarly writing and review beyond strong prompting and conventional RAG, while preserving originality and factual fidelity. It delivers both the study and the tool (the JoF-250 Writing Engine), and the paper studies the tool.

# 🏆 Headline results (protocol heldout_v1; 2 held-out JoF articles × 8 tasks; blind LLM judge, 1–7 scale; n = 24 paired units)

Object	Estimate
Corpus	276 JoF documents (2017–2020), 276/276 Crossref-verified, 276/276 parsed and annotated
H1: profiles + structured retrieval (D) vs strong prompt (B), argument logic	+0.04 [−0.16, 0.22], Holm p = 1.0 → not supported
H2: D vs conventional RAG (C), argument logic	−0.12 [−0.22, 0.03] → not supported
H3: full engine (E) vs D, methodological rigor	+0.96 [0.44, 1.47], Holm p = 0.0085
H3: E vs D, fidelity	+0.46 [0.03, 0.69], Holm p = 0.024 (usefulness −0.33 [−0.78, 0.06], n.s.)
Ablation: E − E without referee pass, rigor	+1.25 [0.63, 1.88] (retrieval/profile/fact-check ablations ≈ 0)
Unsupported-claim sentences, E − C	−1.00 [−1.59, −0.16] (unverifiable citations ≈ 0 in every condition)
Max 8-gram overlap with any corpus article other than the task's source	0.007 (100% of outputs < 0.02)
Cost / latency per task, E vs D	0.32 vs 0.15 USD; 98 vs 49 s

Run 2 of the protocol (run 1 discarded after two implementation defects, Amendment A-004, archived in experiments/outputs/heldout_v1_run1/). Full tables: experiments/analysis/; limitations: docs/LIMITATIONS.md (pilot scale, LLM judge, no human raters).

# 🗂 Repository layout

CLAUDE.md (mandate) · config/ · src/jof250/ · prompts/ · profiles/ · .claude/{commands,agents,skills} · experiments/{preregistration.md,protocols,fixtures,outputs,analysis} · paper/ · tests/ · scripts/ · docs/.

# 🚀 Quick start

bash

uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -e ".[dev]"
cp .env.example .env   # API key file
make audit parse profiles index test
.venv/bin/jof250 write --paper /path/to/paper --section introduction --output /tmp/intro.tex
cd paper && make        # compiles main.pdf

# 📊 Data

277 PDFs under ~/Desktop/Journal-of-Finance-2018-2020 (UQO library access, 2026-11-03); 276 research documents included; metadata verified against Crossref. Derived full texts are private (docs/RIGHTS_AND_ETHICS.md).

# 🧪 Methodology

Phases 1–9 of CLAUDE.md; preregistration in experiments/preregistration.md; limitations in docs/LIMITATIONS.md.

# 📈 Figures & tables inventory

figures/ (regenerated by make profiles and scripts/05_analyze_benchmark.py), paper/tables/ (generated LaTeX fragments).

# ⚠️ Limitations

docs/LIMITATIONS.md.

# 📚 Citation

bibtex

@techreport{boucher2026jof250,
  title  = {JoF-250: Learning Scientific Writing Conventions from Pre-LLM Financial Research},
  author = {Boucher, Simon-Pierre}, institution = {Université du Québec en Outaouais, Département des sciences administratives},
  type = {UQO Working Paper}, number = {45}, year = {2026}, month = {11}
}

# 👤 Author & contact

Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca

# 📝 Notes

Folder named wp45-jof250 as required by the mandate; corpus scope amended to volumes 72–75 (A-002); global Claude Code install authorized by the author (A-003).

spb/wp45_uqo