# Pre-registration — WP-AI-0002 (v1.0, FROZEN 2026-11-09 17:59 UTC)

Status: v1.0, frozen 2026-11-09 17:59 UTC, pre-specified in-session by the operating assistant (the SESOI values are the v0.1 proposals; the author had not yet confirmed them at freeze time — this is stated in the paper as pre-specified in-session, not as author-reviewed pre-registration). Version 1.0 must be frozen before any confirmatory split is scored. The confirmatory splits are test, heldout_domain, heldout_period, held-out prompt and held-out model.

# 1. Units, systems, data

Unit of analysis. A validated pair: an LLM rewrite as source and the original passage as target, both in placeholder form.
Clustering. All inference is clustered by document (doc_id), which covers the multiple variants and samples per passage.
Systems.
- B0 identity and B1 generic paraphrase (evaluation/baselines.py).
- The best HTR model, M*. M* is selected on validation only, among M1/M2/M3 and the ablations, and is fixed in the freeze record (run id + checkpoint sha256).
- B2, if implemented, is a secondary comparator.
Primary test set. Document-safe, author-group-safe test split of the corpus that is current at freeze time. Its build summary and splits hash are recorded at freeze.
Decoding. Greedy, unless beam width is chosen on validation and recorded at freeze.

# 2. Confirmatory hypotheses

ID	Hypothesis	Primary metric	SESOI (proposed)	Test
H1	M* reconstructs the original better than B0 and better than B1	chrF++ (sacrebleu `CHRF(word_order=2)`, sentence level, mean over items)	+2.0 chrF++ points vs each baseline	one-sided: the 95 % CI lower bound of the paired difference must be > 0; practical relevance: the point estimate must be ≥ SESOI
H2	Same comparison as H1	normalised Levenshtein similarity (`edit_sim`)	+0.02	as H1
H3	M* does not lose numerical facts relative to B0 (non-inferiority)	number recall vs the original (`metrics.factual`)	margin −0.01	the lower CI bound of (M* − B0) must be > −0.01
H4	Reconstructions are distributionally closer to the originals than the rewrites are	`W1_mean` over 18 standardised stylometric features (`stylometry.distribution_distances`)	≥ 10 % relative reduction vs W1(rewrite, original)	document-clustered bootstrap of the W1 difference: resample documents, then recompute both W1 values
H5	The H1 gain over B0 persists for held-out prompts (H1/H2) and held-out models (Sonnet 5.5, Haiku 5.5)	chrF++ gain vs B0 on each held-out set	+1.0	one-sided, per held-out set
H6	The dual-latent model (M3) beats token-loss M1 at the same size, data and step budget (mean of 3 seeds)	chrF++ on test	+1.0	paired bootstrap on seed-averaged item scores

Multiplicity.

Holm–Bonferroni at family-wise α = 0.05 over the primary family {H1-vs-B0, H1-vs-B1, H2-vs-B0, H2-vs-B1, H3, H4}.
H5 is a separate family with Holm across the four held-out sets.
H6 is tested alone.

Interpretation rules.

A hypothesis is "supported" only if it is significant after Holm and its point estimate reaches the SESOI.
A significant result below the SESOI is reported as "statistically detectable but practically small".
Non-significant results are reported with their CIs as null findings.

# 3. Statistical procedure

Bootstrap. Document-clustered paired bootstrap (evaluation/bootstrap.py). Documents are resampled with replacement, and every item of a selected document enters the resample.
- Confirmatory analyses use 10 000 resamples with seed 0. The current code default in runner.py is 1 000, so it must be raised before freeze.
p-values. Two-sided p = 2·min(P(d̄* ≤ 0), P(d̄* > 0)). One-sided decisions use the CI bounds above.
Reporting. Means, 95 % percentile CIs, the number of items and the number of documents.

# 4. Exploratory (not confirmatory) analyses

These analyses are reported clearly labelled as exploratory:

Feature shifts. RQ1 per-feature shifts, with Holm over 18 features as a descriptive guard.
Heterogeneity (RQ3). Breakdowns by discipline, era, section role, variant, rewrite strength and input length.
Other metrics. Exact match, BLEU, ROUGE-L, placeholder exactness, negation and hedge deltas.
Diagnostics. The ablations (E5), the scaling curves (E10, E11), calibration (E18) and the error taxonomy (E17).
Style probe. The style probe's AUC and mean P(original) on reconstructions are exploratory only. They are never evidence of human authorship. The probe must be fitted on a split that is disjoint from the evaluated items.
RQ6 analyses. Non-identifiability and memorisation are descriptive.

# 5. Freeze procedure

The PREREGISTRATION v1.0 must state:
- the final SESOIs;
- the M* selection rule;
- the decoding settings;
- the corpus and splits hashes;
- the held-out sets.
Commit v1.0, record the commit hash in docs/DECISION_LOG.md and in a new section, "§6 Freeze record", of this file, and push it to spbgit. Optionally, also deposit it on OSF.
Until the freeze commit exists:
- only train and val may be scored;
- scripts/predict.py / evaluate.py may not be run on confirmatory splits, except B0/B1 sanity checks that report no model comparison.
After freeze, run the confirmatory scoring once. Any deviation is reported as a dated amendment with its reason. Results obtained after a deviation are labelled exploratory.

# 6. Freeze record (v1.0)

Date: 2026-11-09 17:59 UTC
M* selection rule: the candidate run (same size, data and step budget) with the lowest validation token cross-entropy, recorded with its checkpoint sha256 before any confirmatory scoring
Decoding: greedy, KV-cached, placeholder protocol of the pair's protect_version, max_new_tokens = 1.6 x input + 16
Confirmatory bootstrap: 10 000 resamples, seed 0 (scripts/evaluate.py --n-boot 10000 --confirmatory)
Corpus: data/parsed/pilot_pmc · splits: data/splits/pilot_pmc/splits.jsonl · pairs: data/pairs/pilot_opus/validated/pairs.jsonl
sha256 corpus build_summary.json: d028b5aec44b10c3db5c5ae9a2249b6e64b3c5043175257a2e3c22884b0a3dcd
sha256 documents.jsonl: 9a244f007974226557c08d61f5dd6ececf1969a9ec883838d2ea158dcbc1232d
sha256 splits.jsonl: f0088cccbd5e94a6926e81fc2dd4e342a60fe9cc2d23245d62973ed871d6e6e0
sha256 validated pairs.jsonl: 695fefdf9f6b1e755b90a63e7ed830dd9bc00f79781de891521092d5436f1b8f
sha256 held-out pairs pilot_heldout_models: 3eebdbdbdbf453867cf11dcd294a4a52b2346617c84d91158333f5e87b591d20
sha256 held-out pairs pilot_heldout_prompts: 49592652ce4a94085bf6752498b422627224d7f3ef2f4350b1ca34c450b39b36
Held-out sets: test, heldout_domain (chemistry as labelled at split time), heldout_period (pub_year < 1950; 4 passages only), held-out prompts H1/H2, held-out models Sonnet 5.5 / Haiku 5.5
Commit hash: e74b71266d5c0e8c20f181a50df139f52c771d0f (the freeze commit; this line was added by the following commit)

# 7. Amendments

A1 (2026-11-09, before any confirmatory scoring). Implementation correction of H4: runner.py now standardises every stylometric W1 comparison with ONE common scale (pooled SD of originals ∪ rewrites, floored), so W1(recon, orig) and W1(rewrite, orig) are comparable. The hypothesis, metric and SESOI are unchanged. Reason: the paper-writing review found that the two distances previously used different scales.

spb wpai0002_uqo