SPB Git

Repository Public Research preview

abstract.tex 2.3 KB 22 lines latex

1% Abstract — confirmatory numbers from artifacts/metrics/conf_test/report.json and docs/gates/GATE_E.md; exploratory2% HTR 2.0 numbers from artifacts/metrics/medium_test_htr2_exploratory/report.json and docs/STATUS.md (D22).3Can a learned model recover characteristics of pre-2018 human-authored scientific prose from a passage that a large4language model has rewritten? We build 103\,934 validated (rewrite, original) pairs from rights-cleared English5journal articles and books published before 2018, rewritten by Claude Haiku~5.5 under five instructions, and test6pre-registered hypotheses with document-clustered bootstrap inference. On 15\,649 test pairs, a 33.5\,M-parameter7encoder--decoder pretrained as a denoiser and fine-tuned on the pairs reduces the stylometric distance to the8originals by 32.5\% [29.6\%, 37.6\%], a reduction that holds on held-out disciplines, periods and rewriting models.9Its gain in chrF++ over returning the rewrite unchanged is $+0.44$ [$+0.30$, $+0.58$], statistically detectable but10below the pre-registered threshold of $+2.0$; it beats a generic paraphrase baseline by 3.1 points. Number recall11falls by 0.9 points, so non-inferiority on numbers is not established, and the lexical gain does not transfer to12held-out prompts. Metadata conditioning and a dual-latent content/style objective bring no gain. In exploratory,13post hoc tracks, scaling the same architecture to 335\,M parameters raises the gain only to $+0.69$ and a 1.07\,B14model is unstable; sampling, minimum-Bayes-risk reranking and minimum-risk training add at most 0.2 points; a15non-generative editing model is negative. The lever is a pretrained English prior: flan-T5-xl and Qwen2.5-3B16fine-tuned on the same pairs gain $+2.39$ and $+2.27$ chrF++ over identity on test, carry a comparable gain to17unseen disciplines and to pre-1900 books, and split the objectives between them (Qwen: closest wording and18near-perfect preservation of numbers; flan-T5: closest stylometric distribution). The LLM trace is thus largely19stylistic and reversible in distribution, while the original wording is recoverable only in part, from a prior20over English, and never verbatim. Reconstructions also removed most or all flags of a commercial AI-text detector21in a small sample; we report this as a dual-use risk, not as evidence of human authorship.22
Files main
Menu

Theme