# STATUS — WP-AI-0002 (weights published on Hugging Face, D23; updated 2026-11-11, 03:30 UTC)
# Gates
| Gate |
State |
Evidence |
| A — repository, governance, 100-paper smoke, quality audit |
GO |
docs/gates/GATE_A.md |
| B — 500–1 000 validated Claude pairs, spend, human review |
GO (974 passages, 5 383 validated pairs, 29.68 USD total spend) |
docs/gates/GATE_B.md |
| C — leakage-safe splits, reproducible baselines, benchmark + statistical plan |
GO (splits frozen; B0/B1 run on val; pre-registration v1.0 frozen at e74b7126 + A1) |
docs/gates/GATE_C.md |
| D — M1/M2/M3 training, one-batch overfit, pilot results |
in progress — exploratory val (502 pairs): B0 chrF 71.2 · B1 68.4 · B2 T5-small 68.8 (W1 to originals 0.112 < 0.170, i.e. closer style distribution but lower lexical fidelity) · M1-tiny from scratch 32.9 (degenerate). Best val CE over seeds 1–3: M1 3.69/4.09/4.31 · M2 3.72/5.03/3.71 · M3 token-only 3.80/3.74/3.58 · M3 with auxiliary losses ≈ 5.4 → diagnosed (D18): the L_fact normalisation, not the architecture; fixed and re-running. Denoising pretraining (D17) works: pilot denoiser (12 000 steps on 17 k train passages) val CE 1.58; fine-tuned on the pilot pairs → val chrF 63.6 [62.2, 64.9] (from scratch 32.9), number recall 0.93, placeholders 0.88, W1 0.150 (< identity 0.170); still −7.6 chrF below identity on this small pair set (B2 T5-small −2.4). Denoiser alone (no pair fine-tuning) chrF 58.1. Medium denoisers tiny (M3U96b) and small 130 M (M4M64a) running, fine-tuning queued after them. |
artifacts/metrics/pilot_val_exploratory/, artifacts/runs/, docs/DECISION_LOG.md D17–D18 |
| E — larger training |
GO (Medium pairs, confirmatory pass run once; H1/H2 vs B0 detectable but below SESOI, H1/H2 vs B1 and H4 supported, H3 not established, H5 not supported) |
docs/gates/GATE_E.md |
| F — release + paper |
GO (2026-11-11): paper v1.0 compiled (79 pages, all \\pending{} markers resolved, new §10 "Beyond the vanilla Transformer" with the HTR 2.0 tracks, figures/tables generated from artefacts by scripts/build_figures.py --tag HtrTwoTest and scripts/build_htr2_figures.py, val/domain/period deltas by scripts/htr2_split_deltas.py); release candidates artifacts/releases/HTR-Tiny-0.3.0, HTR-Base-0.1.0, flan-t5-xl-htr load independently; six models served on htr.spboucher.ai with model cards. Author to do: confirm SESOIs; decide weights release vs Anthropic terms; decide the platform's default model; independent rater study = future work. |
docs/gates/GATE_F.md, paper/main.pdf |
# HTR 2.0 — exploratory tracks after Gate E (D22, 2026-11-10; all numbers are direct paired chrF++ differences vs B0 identity, ± 1.96 SE, no bootstrap yet; the Gate E confirmatory numbers are unchanged)
| System (track) |
val (3 000) |
test (15 649) |
held-out domain (2 493) |
held-out period (2 574) |
numbers / placeholders (test) |
| M* HTR-Tiny (Gate E, reference) |
+0.42 ± 0.14 |
+0.44 [+0.30, +0.58] (confirmatory) |
−0.10 |
−0.53 |
0.991 / 0.992 |
HTR-Base 335 M, H100 (medium_m1_base_lambda; denoiser 30 k steps val CE 1.225; FT best at step 1 000, CE 1.266 — lr 1e-4 overfits afterwards) |
+0.83 ± 0.13 |
+0.69 ± 0.05 |
+0.29 ± 0.13 |
−0.12 ± 0.10 |
0.992 / 0.993 |
T3: flan-t5-large 780 M, STRAP-style inverse paraphrase (medium_b2_flan_t5_large, system B2_flan-t5-large; best val CE 0.753 at 5 k of 11 k steps) |
+2.04 ± 0.15 |
+1.79 ± 0.07 |
+1.27 ± 0.18 |
+1.09 ± 0.14 |
0.986 / 0.992 |
T3b: Qwen2.5-1.5B SFT, decoder-only LLM backbone (medium_llm_qwen25_15b, system LLM_qwen2.5-1.5b; best val CE 0.733 at 4.5 k of 6 k steps, lr 1e-5) |
+2.19 ± 0.12 |
+1.97 ± 0.06 |
+1.78 ± 0.13 |
+1.30 ± 0.14 |
0.997 / 0.998 |
HTR-Base seed 2 (medium_m1_base_lambda-s2, denoiser CE 1.231, FT best at step 1 000 CE 1.267) — replicates Base |
+0.67 ± 0.14 |
+0.63 ± 0.06 |
+0.32 ± 0.14 |
−0.07 ± 0.10 |
0.991 / 0.992 |
| HTR-Large 1.07 B from the half-trained stage-2 denoiser (CE 1.628; stages 1–2 diverged) — under-trained, negative; stage 3 also diverged (INCIDENTS I4), no further Large runs |
−0.64 ± 0.12 |
−0.71 ± 0.05 |
−1.15 ± 0.13 |
−1.62 ± 0.12 |
0.987 / 0.987 |
HTR-Base FT v2, lr 2e-5 (medium_m1_base_lambda_lr2e5, from the seed-2 denoiser; best CE 1.259 then rising — Base over-fits the pairs at any lr) |
+0.68 ± 0.13 |
+0.55 ± 0.06 |
|
|
|
HTR-Base on MPS (M4M64c, medium_m1_base_mediumdenoiseinit; denoiser 24 k small-batch steps, CE 1.959 vs 1.225 on the H100) — under-trained, superseded by the H100 Base |
−0.21 ± 0.13 |
−0.26 ± 0.06 |
|
|
|
Qwen2.5-1.5B SFT, seed 2 (medium_llm_qwen25_15b_s2, best val CE 0.7332) — replicates seed 1 |
+2.17 ± 0.12 |
+1.97 ± 0.06 |
+1.75 ± 0.13 |
+1.28 ± 0.14 |
|
T3: flan-t5-xl 2.85 B (medium_b2_flan_t5_xl, system B2_flan-t5-xl; best val CE 0.688 at 5 k of 9 k steps, lr 5e-5, grad-ckpt) |
+2.56 ± 0.16 |
+2.39 ± 0.07 (CI above the +2.0 SESOI) |
+1.78 ± 0.18 |
+2.12 ± 0.15 |
0.985 / 0.992 |
T3b: Qwen2.5-3B SFT (medium_llm_qwen25_3b, system LLM_qwen2.5-3b; best val CE 0.701 at 6 k steps, lr 8e-6) — best fidelity of all systems |
+2.42 ± 0.13 |
+2.27 ± 0.06 (CI above the +2.0 SESOI) |
+2.05 ± 0.13 |
+1.91 ± 0.14 |
0.998 / 0.999 |
HTR-Large, final attempt: FT lr 1.5e-5 from the stage-3 denoiser (medium_m1_large_lambda_lr15e6, best CE 1.522, stable) — still negative; Large track closed (I4) |
−0.60 ± 0.10 |
−0.79 ± 0.05 |
−1.05 ± 0.11 |
−1.10 ± 0.10 |
0.988 / 0.989 |
| HTR-Small v2 (M3U96b; denoiser 22.5 k steps CE 1.849, FT CE 1.520) |
−0.13 ± 0.14 |
— |
— |
— |
0.987 / 0.993 |
| HTR-Edit (non-generative tagger, M4M64a) |
−2.27 [−2.54, −2.03] |
not scored |
— |
— |
1.000 / 1.000 |
| T1 decoding (Tiny M*, 1 000 val pairs, K = 8 nucleus samples): greedy +0.54 → MBR +0.66 → MBR + source anchor +0.70; oracle best-of-9 +1.74 |
|
|
|
|
artifacts/metrics/decode_search_tiny_val/ |
| T2 MRT on Tiny M* (α 0.5 and 0.005, lr 1e-5/2e-5, 750–800 steps): val chrF 68.96 → 68.80–68.86 |
negative (abandoned) |
|
|
|
mrt_tiny_mstar_a005, mrt_tiny_mstar_a05_abandoned |
T4 discriminator RoBERTa-large original-vs-rewrite (disc_roberta_large) |
val AUROC 0.994, acc 0.976 |
|
|
|
reward/reranking signal; not yet used in a reported system |
Reading: the pretrained English prior is the lever (the T5 backbone gains 4× M* and, unlike every HTR model, its gain transfers to the held-out
disciplines and to the pre-1900 books); scale of our own Transformer helps modestly (Base +0.69 vs Tiny +0.44 on test). Running / queued:
flan-t5-xl 3 B (instance 1), Qwen2.5-1.5B SFT (instance 3), HTR-Base FT v2 lr 2e-5 (instance 2), HTR-Large stage 3 with the fp32 copy mixture
(instance 2; stages 1–2 diverged at lr 2e-4 / 8e-5, see INCIDENTS I3). Pangram (same 54-pair test sample, artifacts/metrics/pangram_test_htr2/, 7.52 USD): flan-t5-large 54/54 "Human" (mean ai_likelihood 0.000), HTR-Base 53/54, Qwen2.5-1.5B 48/54 (6 "Mixed", mean 0.032; pangram_test_qwen, 3.64 USD) — the lexically closest system is not the most detector-clean; reported as detector behaviour only (D21). RQ6 memorisation audits on validation (artifacts/metrics/audit_qwen15b_val, audit_flant5large_val): max Jaccard to HTR training originals 0.000, no same-passage leaks, for both backbones — these audits cover the HTR pairs only, not the backbones' own pretraining corpora.
Exploratory test report with all systems (artifacts/metrics/medium_test_htr2_exploratory/, 15 649 pairs, 2 000 document-clustered resamples):
chrF++ B0 68.91 [68.35, 69.48] · Tiny M* 69.35 · Base 69.60 · flan-t5-large 70.71 [70.19, 71.24] · Qwen2.5-1.5B 70.88 [70.33, 71.42].
Stylometric distance W1 to the originals (rewrite 0.154): flan-t5-large 0.083 (−46 %) · Tiny 0.104 (−32 %) · Base 0.108 · Qwen 0.123 (−20 %). Final report (2026-11-11 01:31 UTC, 27 systems, report.json): flan-t5-xl +2.39 [+2.20, +2.57], W1 0.081 · Qwen2.5-3B +2.27 [+2.13, +2.40], W1 0.100 · Qwen 1.5B s2 +1.97 [+1.83, +2.11] · Base MPS −0.26 · Large lr 1.5e-5 −0.79 · Large stage-3 FT −1.25 · Small v2 −0.09 · HTR-Edit −2.20 [−2.36, −2.05].
Reading: the two backbones split the objectives — Qwen is the closest lexically and keeps the most numbers/placeholders, flan-t5-large moves the
text furthest towards the pre-2018 stylometric distribution and is the only backbone with 0 % Pangram flags; the paper reports both, no single "best".
# Medium-stage pairs (Gate E data, 2026-11-10; data/pairs/medium_v1_haiku/validated/validation_summary.json)
| Item |
Value |
| Rewriter |
claude-haiku-5-5 (author decision D20), prompts rw-1.0, protocol 1.1, Message Batches |
| Requests |
120 000 on 20 000 passages (train 14 000 · val 2 000 · test 3 000 · held-out domain 500 · held-out period 500) |
| Outcome |
119 724 ok · 88 refused · 188 truncated · 13.52 USD (project total 43.20 USD) |
Validation (val-1.0) |
103 934 passed (86.6 %), 16 066 quarantined (numbers missing 10 700 · entity recall 2 682 · formatting 784 · extra numbers 454 · placeholders 451 · negation 434 · near-verbatim 426) |
| By variant |
A 17 558 · B 16 815 · C 17 881 · D 18 043 · E 33 637 (of 20 000 / 40 000) |
| Pairs by source |
PMC 63 726 · Crossref econ/finance 20 028 · Gutenberg 12 862 · Bookshelf 4 948 · arXiv 1 398 · OpenStax 972 |
| Pairs by discipline (top) |
biomedicine 12 043 · economics 11 818 · life sciences 10 789 · finance 10 737 · social sciences 10 716 · engineering 9 379 · CS 8 098 · chemistry 6 047 |
Medium validation (exploratory, 3 000 val pairs, 2 000 bootstrap; artifacts/metrics/medium_val_exploratory/): |
|
| System |
chrF++ |
edit sim |
number recall |
placeholders |
W1 to originals ↓ |
paired chrF vs B0 |
| B0 identity |
68.42 [67.60, 69.23] |
0.678 |
1.000 |
1.000 |
0.164 |
— |
| B1 generic paraphrase |
65.67 |
0.566 |
1.000 |
1.000 |
0.226 |
−2.75 |
HTR-Tiny, Medium denoiser init (medium_m1_tiny_mediumdenoiseinit, val CE 1.47) |
68.75 [67.94, 69.54] |
0.678 |
0.991 |
0.992 |
0.100 |
+0.33 [+0.09, +0.58], p_Holm 0.015 |
HTR-Tiny, pilot denoiser init (medium_m1_tiny_pilotdenoiseinit, 16 k tokenizer) |
67.87 |
0.674 |
0.985 |
0.988 |
0.126 |
−0.55 |
| HTR-Tiny M3 architecture, token-only (medium_m3tokenonly_tiny_mediumdenoiseinit, val CE 1.49) | 68.78 | 0.680 | | | 0.106 | +0.35 [+0.13, +0.59], p 0.002 |
| HTR-Tiny M2 metadata-conditioned (val CE 1.49) | 68.57 | 0.677 | 0.988 | | 0.098 | +0.15 [−0.08, +0.39], n.s. |
| HTR-Small 130 M (medium_m1_small_mediumdenoiseinit, denoiser 15 k steps val CE 1.85 → FT val CE 1.50) | 68.62 | 0.675 | 0.990 | 0.996 | 0.095 | +0.20 [−0.04, +0.45], n.s. |
| HTR-Tiny M3 token-only, seed 2 (val CE 1.48) | 68.73 | | 0.990 | | 0.105 | +0.31 [+0.09, +0.56], p 0.020 |
| HTR-Tiny M3-light (style/adv/con/sem/orth/fact at small weights; val CE 1.49) | 68.48 | | 0.991 | | 0.098 | +0.06 [−0.18, +0.32], n.s. |
| HTR-Tiny M1 seed 2 (val CE 1.474) | 68.81 | | | | 0.097 | +0.38 [+0.14, +0.63], p 0.003 |
| HTR-Tiny M1 seed 3 (val CE 1.470, lowest → M* by the frozen rule) | 68.84 | | 0.991 | 0.995 | 0.100 | +0.42 [+0.20, +0.66], p 0.003 |
| B2 T5-small (60 M) fine-tuned on the Medium pairs, 3 000 steps | 68.38 [67.60, 69.14] | 0.680 | 0.989 | 0.713 | 0.117 | −0.04 [−0.40, +0.30], n.s. |
Reading: every Medium-denoiser-initialised HTR model sits at or just above identity on chrF (detectable gains of +0.15 to +0.35, all
below the pre-registered +2.0 SESOI) while halving the stylometric distance to the originals (W1 0.095–0.106 vs 0.164) at a cost of
≈ 1 pp in number/placeholder preservation. Metadata conditioning (M2) and model size (Small vs Tiny, same step budget) do not change the
picture; the Small model is under-trained relative to its size. Running: M3-light and M1 seed 2 (M4M64a), M1 seed 3 (laptop), B2 T5-small
on the Medium pairs (M3U96b). M* selection follows the frozen rule (lowest validation CE among same-budget candidates).
# Incident (2026-11-09 ~23:30 UTC)
M3U96b and M4M64a froze (no ping) under Medium-scale denoising pretraining (MPS memory near the limit) — docs/INCIDENTS.md I1.
The Medium tiny denoiser was relaunched on M4M64c with PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.6; the HTR-Small pretraining is on
hold until a 64 GB+ node is back. If the two nodes do not come back by themselves, they need a power cycle.
# API spend (ledgers, Message Batches = 50 % price)
| Job |
Model |
Requests |
ok / refused |
USD |
pilot_opus |
claude-opus-5-5 |
5 844 |
5 801 / 43 |
26.85 |
pilot_heldout_models |
claude-sonnet-5-5, claude-haiku-5-5 |
600 |
600 / 0 |
1.43 |
pilot_heldout_prompts |
claude-opus-5-5 (H1, H2) |
300 |
297 / 3 |
1.40 |
| Total |
|
6 744 |
|
29.68 |
| Measured mean tokens per request on Opus: 691 in / 323 out → 0.0046 USD per request in batch mode. |
|
|
|
|
# Medium corpus v1 (final acquisition, built 2026-11-09 night; data/parsed/medium_v1/build_summary.json, splits data/splits/medium_v1/ with v0 assignments preserved)
| Item |
Value |
| Documents kept |
14 470 (14 900 stubs; 2 parse errors, 35 post-2017, 61 undated, 36 licence-excluded, 16 non-English, 234 exact + 46 near duplicates) |
| By source |
PMC 10 911 · Crossref econ/finance 1 730 · Bookshelf 1 286 chapters · Gutenberg 448 · arXiv 90 · OpenStax 5 |
| By licence |
CC-BY 9 702 · CC-BY-NC 1 849 · PUBLIC-DOMAIN 1 737 · CC-BY-NC-SA 1 070 · CC0 109 · CC-BY-SA 3 |
| Disciplines |
biomedicine 8 828 · life sciences 2 222 · economics 1 289 · finance 564 · social sciences 363 · engineering 252 · chemistry 174 · CS 168 · earth 147 · physics 102 · statistics 84 · general reference 58 · humanities 56 · mathematics 45 · other 118 |
| Paragraphs / eligible passages |
897 419 / 451 072 |
| Splits |
train 11 035 · val 1 270 · test 1 261 · heldout_domain 253 · heldout_period 510 · quarantine 141 (fixed-split conflicts) · 0 leaks |
# Medium corpus v0 (built 2026-11-09 evening; data/parsed/medium_v0/build_summary.json, splits data/splits/medium_v0/)
| Item |
Value |
| Stubs parsed / documents kept |
13 773 → 13 372 (2 parse errors, 35 post-2017, 42 undated, 36 licence-excluded, 16 non-English, 228 exact + 42 near duplicates) |
| By source |
PMC 10 911 · Bookshelf 1 184 chapters · Crossref econ/finance 895 · Gutenberg 287 · arXiv 90 · OpenStax 5 |
| By licence |
CC-BY 8 867 · CC-BY-NC 1 849 · PUBLIC-DOMAIN 1 478 · CC-BY-NC-SA 1 070 · CC0 105 · CC-BY-SA 3 |
| Disciplines |
biomedicine 8 706 · life sciences 2 208 · economics 689 · finance 319 · social sciences 318 · engineering 237 · chemistry 173 · CS 168 · earth 128 · physics 100 · statistics 84 · mathematics 45 · humanities 42 · general reference 37 · other 118 |
| Paragraphs / eligible passages |
660 108 / 371 419 (chapter 85 657 · other 57 689 · results 55 116 · methods 53 319 · discussion 44 692 · introduction 27 984 · abstract 13 252 · background 8 851 · conclusion 8 074 · data 6 632 · theory 6 202 · related work 1 999 · limitations 1 304 · preface 648) |
| Splits (pilot assignments preserved; held-out physics + earth sciences; pub_year < 1900) |
train 10 268 · val 1 190 · test 1 166 · heldout_domain 257 · heldout_period 472 · quarantine 19 (fixed-split conflicts); 0 leaks |
| Tokenizer |
artifacts/tokenizers/medium_v0 — SentencePiece unigram 32 000 on 258 264 train-split passages |
| Pretraining |
denoise_m1_tiny_medium (M3U96b, 30 000 steps, no API cost) running |
| Acquisition still running when v0 was built: Crossref (895/1 730), Gutenberg (287/500), Bookshelf (1 184 chapters); v1 will be rebuilt when they finish. |
|
# Corpus (rights-verified, pre-2018, English)
| Source |
Status |
Documents / eligible passages |
PMC Open Access (AWS pmc-oa-opendata) |
pilot built; Medium-stage ingestion running (seed 3, target 10 000 more docs; 5 100+ fetched so far) |
pilot 1 180 / 23 658 |
| arXiv q-fin + econ (CC-licensed, v1 < 2018) |
adapter written; live smoke 12 docs; pool count in progress |
see data/manifests/discover_arxiv_* |
| Crossref CC-BY economics/finance journals (16 verified journals) |
adapter written; 1 730 eligible articles counted; 15 fetched → 566 eligible passages |
data/manifests/discover_crossref_count.json |
| Books: Project Gutenberg (public domain, nonfiction LoCC Q/R/T/H/B/G/L/Z) |
adapter written; pool 7 799 English nonfiction texts; 12 fetched → 2 037 eligible passages, 10/12 dated from front matter |
data/manifests/discover_books_gutenberg_* |
| Books: NCBI Bookshelf (BITS XML via FTP, 3 941 pre-2018 books) |
adapter written; 3 books → 17 chapter stubs, 299 passages |
configs/data/books_sources.json |
| Books: OpenStax / CNX pre-2018 editions (Internet Archive, dates from the PDF itself) |
adapter written; 5 items |
idem |
# Open decisions for the author
- Publisher robots.txt: MDPI and Springer disallow named AI crawlers; our fetcher uses a research user agent and the
Crossref text-mining links for CC BY content. Confirm this policy (documented in
docs/RISK_REGISTER.md).
- Anthropic terms: publishing weights trained on Claude rewrites — recorded as a risk to confirm, not a conclusion.
- SESOI thresholds in
docs/PREREGISTRATION.md v0.1 must be confirmed before the v1.0 freeze.
- DECIDED (author, 2026-11-10): Haiku 5.5. Job
medium_v1_haiku submitted: 60 batches, 120 000 requests, pre-flight
estimate ≈ 13 USD (batch); collection + validation run automatically when the batches end (D20). The earlier plan for reference:
20 000 passages of medium_v1 (train 14 000 · val 2 000 · test 3 000 · held-out domain 500 · held-out period 500) × 6
generations = 120 000 requests. Pilot-calibrated estimate (Medium passages are longer than the PMC pilot's):
≈ 1 070 USD with Opus 5.5 (batch), ≈ 535 USD with Sonnet 5.5, ≈ 27 USD with Haiku 5.5; a 10 000-passage plan halves these.
Per-request lower bound from the pilot's actual spend: 551 / 276 / 14 USD. Nothing has been submitted.
# Known limitations at this point
- Discipline labels for PMC are heuristic (journal title / subject categories); relabelled once (165/1 180 changed).
- Pre-1950 PMC material is OCR and yields almost no eligible passages; historical style will come from books.
- Validator heuristics count spelled-out numbers as missing; entity matching is a proxy.
https://htr.spboucher.ai — research preview serving HTR-Tiny 0.3.0 (M*, 33.5 M params, CPU inference ≈ 0.1 s per paragraph; the pilot model HTR-Tiny-pilot-rc0 was served until 2026-11-10 morning): single-page UI (examples, metadata, word-level diff, preservation chips, warnings), JSON API POST /api/reconstruct, GET /api/info, /model-card, /health; 30 requests/min per client, 8 000 characters max. Deployed with mld on m4mm:8540 (PM2 htr-web, tunnel route on BHS64); source = m4mm:~/apps/htr (git htr-platform.git on spbgit; laptop staging copy ~/Desktop/Cluster/htr-platform). To ship a better model: copy a new release dir to m4mm:~/apps/htr/model/ and pm2 restart htr-web.
# Cluster usage (2026-11-10, author: "exploite le cluster encore plus")
All training runs on the Apple GPUs (MPS) of the nodes — there is no CUDA hardware in MacLustr. In flight: M3U96b HTR-Small v2 (denoiser resumed 15 k → 45 k steps, then 30 k fine-tuning steps), M4M64c HTR-Base 335 M (denoiser 24 k steps + fine-tune), M4M64a HTR-Edit non-generative tagger (agent), m4mn (new node, torch installed) M3 token-only seed 3 + M2 seed 2, laptop: confirmatory scoring + Pangram evaluation (agent). Memory caps on every node (PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.7, LOW=0.5).