# WP-AI-0002 — Research plan

Human Text Reconstruction: Learning the Inverse of LLM-Induced Transformations in Scientific Writing Version 0.1 · 2026-11-09 · binds to docs/MANDATE.md (phases 0–8, gates A–F, RQ1–RQ6). Every number below comes from a file in the repository. A number that has not been measured yet is written "pending (run id …)".

# 1. Mission and narrowed contribution

Mission. Build a reproducible research programme and a model family that is usable after release. The programme asks whether a Transformer can reconstruct characteristics of pre-2018 human-authored scientific prose from LLM-rewritten versions of the same passages. Two deliverables are mandatory: a paper and an independently loadable model.

What is not new. The direct antecedents are Rivera Soto et al.:

• Rivera Soto et al. (2024, riverasoto2024paraphrase) introduced paraphrase inversion. They trained paraphrase→original models on synthetic pairs and showed that these models generalise to paraphrasers unseen in training.
• Rivera Soto et al. (2026, riverasoto2026unsupervised) trained a style encoder that reconstructs human text from its machine paraphrase while a semantic encoder is frozen. This is architecturally close to our M3.

WP-AI-0002 therefore does not claim to be the first to invert LLM paraphrases (docs/RESEARCH_GAP.md §5).

What the contribution is. Our claim is restricted to four points:

1. Reconstruction fidelity is the object of study. Prior work used inversion as a means to detection. We measure fidelity against identity (B0) and generic-paraphrase (B1) baselines.
2. The originals are rights-verified, pre-2018 scientific prose. Each passage has controlled rewrite variants A–E, which allows paired, per-passage measurement for RQ1.
3. Factual integrity, non-identifiability and memorisation are evaluated. This covers numbers, citations, equations, negation and hedging, plus a test of how many distinct originals a rewrite is compatible with.
4. Results are broken down by discipline, era and section role, and generalisation is tested. Held-out prompts and held-out models are a primary research question (RQ4), not a robustness footnote.

Whether Rivera Soto et al. stratify by discipline or era is marked to check in full text.

# 2. Research questions → hypotheses → metrics → tests

Confirmatory hypotheses H1–H6 are pre-registered in docs/PREREGISTRATION.md. The others are exploratory.

Lower lexical overlap is never read as failure to recover style (mandate §6). Style results are distributional (W1). The original-vs-rewrite probe is exploratory only.

# 3. Data pipeline

flowchart LR
  subgraph Sources[Official channels, metadata first]
    P[PMC OA · AWS bucket pmc-oa-opendata<br/>JSON license_code + JATS]
    X[arXiv OAI-PMH arXivRaw + e-prints<br/>CC licence, v1 date · in progress]
    C[Crossref CC BY econ/finance journals<br/>in progress]
    B[Books: Gutenberg PD · NCBI Bookshelf BITS ·<br/>OpenStax-IA only if dated]
  end
  Sources --> S[raw file + meta.json stub<br/>licence evidence, sha256, provider md5]
  S --> PA[parse: JATS/BITS · LaTeX · txt/PDF<br/>pub-date verified < 2018-01-01]
  PA --> L{licence policy<br/>htr.licenses.policy}
  L -- tier C/X --> M[metadata only]
  L -- training set --> SEG[paragraphs → passages<br/>60–400 words, ≥2 sentences,<br/>en ≥ 0.9, quality ≥ 0.6]
  SEG --> DD[MinHash dedup<br/>5-word shingles, 128 perm, J ≥ 0.8]
  DD --> SP[splits BEFORE rewriting<br/>cluster ∪ author group (∪ venue);<br/>held-out discipline / period]
  SP --> RW[protect: citations/math/URLs → ⟦C/M/U n⟧<br/>Claude rewrite A–E (+H1/H2 held out)]
  RW --> V[validate val-1.0<br/>pass → pairs · fail → quarantine]
  V --> T[tokenizer (train split only) → M1/M2/M3]
  T --> E[evaluation · release]

Measured so far.

The economics and finance strata depend on the arXiv q-fin/econ adapter and the Crossref adapter, both in progress.

Scale target (author, 2026-11-09). Medium stage: ≥ 10 000 documents and ≥ 200 000 eligible passages, with no single archive dominating.

# 4. Rewrite protocol (src/htr/rewrite/, prompt version rw-1.0)

Prompts.

• System prompt. Forbids invented information. Requires each placeholder ⟦C n⟧, ⟦M n⟧, ⟦U n⟧ to be copied verbatim, exactly once per occurrence.
• PRESERVE block. Requires preservation of numbers, units, entities, citations, math, URLs, negation, hedges and uncertainty. Output must be a single English paragraph.

Variants. A neutral paraphrase · B polished/standardised · C concise (~70 % length) · D structure-preserving (same sentence count and order) · E diverse sampling (2 samples).

Held-out prompts. H1 (plain-language expert) and H2 (journal copy-editor) are never used in training.

Placeholder protocol.

• protect() replaces math, URLs/DOIs and numeric or author–year citations by indexed placeholders before any LLM call.
• Both rewrite and target are stored in placeholder form.
• Numbers stay as text. They are checked, not masked (D9).

Provider (anthropic_provider.py).

• Model ids are pinned: Opus 5.5 (claude-opus-5-5) is the primary rewriter. Sonnet 5.5 and Haiku 5.5 are held-out models for RQ4 (configs/rewriting/pilot.json).
• output_config.effort = "low". Thinking is adaptive (provider default). No temperature is set.
• Each response logs model, request id, usage, stop reason, cost and latency.
• Retries use exponential backoff.
• Message Batches are billed at 50 %.
• Request ids are deterministic (passage, variant, sample, provider, model, prompt version). This deduplicates requests and makes runs resumable. The output is an append-only rewrites.jsonl plus ledger.json.
• The default provider is the zero-cost mock. Real calls require --live. The budget ceiling is enforced on recorded spend (pilot: 30 USD).

Validation (validate/checks.py, val-1.0). A pair passes only if all of the following hold:

• length ratio within [0.5, 1.8] (0.4 for C);
• normalised edit similarity within [0.05, 0.95] (above 0.95 counts as a near-verbatim copy);
• language is English;
• the placeholder multiset is identical;
• no numbers are missing and at most 1 is extra;
• proxy entity recall ≥ 0.6;
• |Δ negation| ≤ 2 and |Δ hedge| ≤ 4;
• for D, sentence count is within ±1;
• there are no meta-text artefacts;
• optionally, MiniLM cosine ≥ 0.70.

Failed pairs are quarantined, not dropped. Bidirectional entailment (mandate Phase 3) is not yet implemented.

# 5. Model family (src/htr/models/)

Shared backbone. A pre-LN encoder–decoder with RoPE, SwiGLU feed-forward and tied embeddings. The tokenizer is a from-scratch SentencePiece unigram model trained on train-split text only, with atomic control and placeholder tokens.

M1 is the plain backbone with an optional copy gate.

M2 adds learned discipline, era, section-role and source embeddings to the encoder and decoder inputs, plus one <meta> memory token.

M3 (dual-latent) is defined as follows:

• Content and style. E_c is the encoder. The content vector is c = W_c · mean-pool(H). The style branch E_s = MLP(mean;max + meta) predicts a distribution over K style classes. K = observed (discipline | era) buckets with ≥ 20 training pairs, plus "other".
• Style memory. Style memory = Σ_k p_k · prototype_k. It contributes 4 memory tokens, which are prepended to the cross-attention memory.
• Adversary. A gradient-reversal adversary predicts the style class from c.
• Copy gate. p(w) = (1−g_t)·softmax(W h_t)w + g_t · Σ{i: x_i=w} a_{t,i}, where a is the head-averaged last-layer cross-attention over source positions, renormalised, and g_t = σ(w_g·h_t).
• No target at inference. The original target is never an input at inference time. Metadata, if used, is supplied by the caller.

How the counts were obtained. Parameter counts come from instantiating HTRModel on the meta device. Trained checkpoint counts are logged in manifest.json and are pending, except for the smoke run: vs_smoke-m1-nano has 8 326 401 parameters at tokenizer vocab 4 842.

Mismatch with the mandate. The design points exceed the mandate's approximate sizes by 40–60 %. Either resize them or relabel them before Gate D.

# 6. Objective (src/htr/models/losses.py)

Notation.

Full objective.

L = λ_ce L_token + λ_fact L_factual + λ_style L_style + λ_adv L_adv + λ_con L_contrastive + λ_sem L_semantic + λ_orth L_orth

Terms.

Differentiability.

• All terms are differentiable with respect to model parameters.
• f_t and k are constants.
• The copy mixture is differentiable through the gate and the attention weights.
• c′ receives no gradient.
• Through the GRL, the adversary minimises L_adv while the encoder receives −λ times its gradient.
• Logged accuracies (argmax) are not in the graph.

Where the terms apply. L_style, L_adv, L_con, L_sem and L_orth are active only for arch = m3. The original is encoded only inside the loss (training).

Weights. Preset m3: ce 1, fact 0.5, style 0.3, adv 0.3, con 0.2, sem 0.1, orth 0.1. Weights are tuned on validation only.

Controls. M1 and M2 are token-loss only (ce 1). The ablation presets are m3_tokenonly, m3_nostyle, m3_noadv, m3_nofact and m3_nocon.

Disentanglement claims. Chance-level adversary accuracy is not taken as evidence of disentanglement (elazar2018adversarial). A post-hoc probe on frozen c is required.

# 7. Baselines

All trained systems are compared at matched data and with FLOPs reported.

# 8. Evaluation plan

Instance level (metrics.py):

• exact match, normalised edit similarity, chrF++, sentence BLEU, ROUGE-L, length ratio;
• factual: placeholder exactness and recall, number recall and precision, negation, hedge and sentence-count deltas.

All instance metrics are reported next to B0.

Corpus level.

• Document-clustered bootstrap CIs (bootstrap.py), by variant, discipline, era and role (runner.py).
• Paired differences vs B0.

Stylometric fidelity.

• 18 features: sentence length mean/sd, TTR, MTLD, word length, long-word, comma, semicolon, parenthesis, discourse, passive, nominalisation, first person, subordination, hedge, negation, digit rates, Flesch.
• W1 is the 1-D Wasserstein distance per feature, standardised by the reference spread, then averaged (W1_mean).
• Comparisons: rewrite vs original, and each system vs original.

Exploratory probe. A logistic regression (original = 1 vs rewrite = 0), with a 70/30 split. We report AUC and mean P(original) on reconstructions. It is never used as evidence of origin.

• Current limitation: the probe is fitted on the originals and rewrites of the evaluated pairs. A disjoint probe-training split is required before any reported probe number.

Non-identifiability (planned, not implemented). Three measurements:

• (a) Consistency: for the six rewrites of one original, compare the agreement among their reconstructions with their distance to the original.
• (b) Confusable originals: take near-neighbour originals from different documents (embedding and MinHash neighbours). Measure identification accuracy, i.e. whether the reconstruction is closer to its own original.
• (c) An "oracle" ceiling: the chrF++ of the best available rewrite.

The ambiguity rate goes to RQ6.

Memorisation audit (planned). Four checks:

• longest common token span and 13-gram overlap between each reconstruction and the training originals;
• the excess overlap of training neighbours over the test original;
• synthetic canaries inserted into training, then measured for exposure (lawful because they are synthetic);
• for B2, a contamination audit of the pretraining data where it is documented.

Human evaluation (planned). See docs/experiments/EXPERIMENT_MATRIX.md (E16).

# 9. Gates (acceptance criteria → evidence)

# 10. Roadmap and decision points

1. Corpus expansion (this week): arXiv q-fin/econ/math/cs/stat/physics, Crossref econ/finance, books. Rebuild the splits with venue-safe and author-safe options.
   • Decision point DP1: is the mix balanced, i.e. no archive above roughly 50 %?
2. Gate B pilot: estimate the cost, then submit batches.
   • DP2: is acceptance ≥ 70 % per variant? If not, revise the prompts (rw-1.1) before scaling.
3. Freeze the pre-registration (DP3), then run baselines.
4. M1/M2/M3 at tiny size on the pilot pairs.
   • DP4: does any model beat B0 on validation? If not, report the null result. Scale only to test RQ5.
5. Medium-scale rewriting (20 000 passages and up). Requires an explicit budget approval (docs/reproducibility/COMPUTE_BUDGET.md).
   • DP5: small/base on M4M64a, M3U96a/b or the laptop, depending on throughput measured at tiny size.
6. Final confirmatory scoring. Run once on the test and held-out sets.
7. Release candidate: audit, model card, paper.

# 11. Release plan

Planned layout of the Hugging Face-compatible model directory:

API and CLI. The usable API is HTRReconstructor.from_pretrained(dir).reconstruct(text, metadata) (inference/pipeline.py). Decoding is deterministic, greedy or beam. Placeholders are restored. Preservation warnings are produced, along with a notice that the output is synthetic. The CLI entry point htr = htr.cli:main is declared in pyproject.toml, but htr/cli.py does not exist yet.

Demo. A small demo showing the input, the reconstruction, a diff and the warnings. demo/ is empty.

Release conditions. Weights are released only for models trained on the release set, or after legal review of NC/SA inheritance (LICENSE_MATRIX.md). Nothing is uploaded and the namespace is not chosen without the author's approval.

# 12. Honest-claims policy

• Outputs are synthetic reconstructions. They do not restore authorship, they are not evidence of human authorship, and they are not an AI detector.
• Verbatim reconstruction, stylistic distribution matching and factual fidelity are reported separately.
• Every gain is stated against B0 and B1, with document-clustered CIs.
• Null and negative findings are reported in the paper and the gate reports.
• Probe and classifier scores are exploratory.
• Only rights-verified data and verified citations are used, and every number traces to a run manifest.