# 📝 JoF-250: Learning Scientific Writing Conventions from Pre-LLM Financial Research **UQO Working Paper No. 45** — *A measured-convention writing engine for Claude Code, built from 276 Journal of Finance articles (2017–2020), with a preregistered evaluation* [](paper/main.pdf) [](pyproject.toml) [](docs/RIGHTS_AND_ETHICS.md) [](docs/REPRODUCIBILITY.md) [](https://uqo.ca) [](mailto:simon-pierre.boucher@uqo.ca) > **TL;DR** — We parse 276 pre-ChatGPT *Journal of Finance* articles (vol. 72–75), annotate their rhetorical structure with a schema-validated LLM ontology, turn the train split into measured writing profiles (medians, IQRs, bootstrap CIs), index structural units for hybrid retrieval with provenance, and expose the result as a Claude Code writing/review/audit engine (`/jof-write`, `/jof-review`, `/jof-audit`, …). A preregistered held-out experiment compares five conditions (baseline → full engine) with a blind LLM judge and deterministic fidelity audits. See `docs/DELIVERY_REPORT.md` for measured results. ## 📌 Status | Item | Value | |---|---| | Version | 1.0 (November 2026) | | SSRN | not yet submitted | | Journal | — | | Code & data | pipeline reproducible from `make` (see Quick start); PDFs not redistributed | ## 🎯 What this paper does It asks whether *abstract, domain-specific writing conventions* extracted from a pre-LLM corpus improve LLM-assisted scholarly writing and review beyond strong prompting and conventional RAG, while preserving originality and factual fidelity. It delivers both the study and the tool (the JoF-250 Writing Engine), and the paper studies the tool. ## 🏆 Headline results (protocol heldout_v1; 2 held-out JoF articles × 8 tasks; blind LLM judge, 1–7 scale; n = 24 paired units) | Object | Estimate | |---|---| | Corpus | 276 JoF documents (2017–2020), 276/276 Crossref-verified, 276/276 parsed and annotated | | H1: profiles + structured retrieval (D) vs strong prompt (B), argument logic | +0.33 [−0.22, 0.66], Holm p = 0.45 → not supported | | H2: D vs conventional RAG (C), argument logic | +0.08 [−0.44, 0.47] → not supported | | H3: full engine (E) vs D, methodological rigor | **+1.33 [1.00, 1.84], Holm p = 0.0003** | | H3: E vs D, fidelity | **+0.62 [0.31, 1.03], Holm p = 0.004** | | Ablation: E − E without referee pass, rigor | +1.63 (retrieval/profile/fact-check ablations ≈ 0) | | New numbers absent from the source, E − A | −0.88 [−1.94, −0.03] | | Max 8-gram overlap with any corpus article other than the task's source | 0.006 (100% of outputs < 0.02) | | Cost / latency per task, E vs D | 0.20 vs 0.10 USD; 80 vs 38 s | Full tables: `experiments/analysis/`; limitations: `docs/LIMITATIONS.md` (pilot scale, LLM judge, no human raters). ## 🗂 Repository layout `CLAUDE.md` (mandate) · `config/` · `src/jof250/` · `prompts/` · `profiles/` · `.claude/{commands,agents,skills}` · `experiments/{preregistration.md,protocols,fixtures,outputs,analysis}` · `paper/` · `tests/` · `scripts/` · `docs/`. ## 🚀 Quick start ```bash uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -e ".[dev]" cp .env.example .env # API key file make audit parse profiles index test .venv/bin/jof250 write --paper /path/to/paper --section introduction --output /tmp/intro.tex cd paper && make # compiles main.pdf ``` ## 📊 Data 277 PDFs under `~/Desktop/Journal-of-Finance-2018-2020` (UQO library access, 2026-11-03); 276 research documents included; metadata verified against Crossref. Derived full texts are private (`docs/RIGHTS_AND_ETHICS.md`). ## 🧪 Methodology Phases 1–9 of `CLAUDE.md`; preregistration in `experiments/preregistration.md`; limitations in `docs/LIMITATIONS.md`. ## 📈 Figures & tables inventory `figures/` (regenerated by `make profiles` and `scripts/05_analyze_benchmark.py`), `paper/tables/` (generated LaTeX fragments). ## ⚠️ Limitations `docs/LIMITATIONS.md`. ## 📚 Citation ```bibtex @techreport{boucher2026jof250, title = {JoF-250: Learning Scientific Writing Conventions from Pre-LLM Financial Research}, author = {Boucher, Simon-Pierre}, institution = {Université du Québec en Outaouais, Département des sciences administratives}, type = {UQO Working Paper}, number = {45}, year = {2026}, month = {11} } ``` ## 👤 Author & contact Simon-Pierre Boucher — simon-pierre.boucher@uqo.ca ## 📝 Notes Folder named `wp45-jof250` as required by the mandate; corpus scope amended to volumes 72–75 (A-002); global Claude Code install authorized by the author (A-003).