Data card draft

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Simon-Pierre Boucher committed 4 h ago (Nov 9, 2026) parent a6d9576 Browse files

3 changed files +242 −5

added docs/data_governance/DATA_CARD.md +54 −0

++# Data card — HTR corpus and HTR-Benchmark (draft, 2026-11-09)
++
++## Summary
++Paired data: **source** = a passage rewritten by a large language model (Claude Opus 5.5 primary; Sonnet 5.5 / Haiku 5.5
++held-out), **target** = the original English passage from a document published before 2018-01-01 whose licence permits
++the operations we perform. Both sides are stored in *placeholder form* (citations ⟦Cn⟧, math ⟦Mn⟧, links ⟦Un⟧; see
++`src/htr/parse/protect.py`, protocol version recorded per pair). Metadata per pair: discipline, publication year/era,
++section role, source archive, licence code, venue, author group, rewriting model, prompt version, protocol version.
++
++## Sources (all acquired through official channels, rate-limited, rights recorded per document)
++| Source | Access route | Date evidence | Licence evidence | Scale observed |
++|---|---|---|---|---|
++| PMC Open Access Subset | AWS Open Data bucket `pmc-oa-opendata` (per-article JSON + JATS XML; MD5 verified) | JATS `<pub-date>` (ppub → collection → epub) | JSON `license_code` cross-checked with JATS `<license xlink:href>` | pilot 1 180 docs; Medium ingestion ~10 000 |
++| arXiv (q-fin, econ; others possible) | OAI-PMH `arXivRaw` metadata + e-print source of the last pre-2018 version (3 s between requests) | v1 submission date | OAI `license` URI (only CC licences kept; the arXiv non-exclusive licence is excluded) | 162 eligible q-fin papers 2008–2017 |
++| CC BY economics & finance journals (16 verified: MDPI Econometrics, JRFM, IJFS, Risks, Economies, Games; SpringerOpen Financial Innovation, J. Economic Structures, IZA journals, LAER, SJES, JIE) | Crossref REST (polite pool) + publisher PDF via Crossref text-mining links | Crossref `published-print` → `published-online` → `issued` | Crossref `license[]` (content-version vor) | 1 730 eligible articles |
++| Project Gutenberg (reference / scholarly non-fiction, LoCC Q R T H B G L Z) | catalog feed + official mirror `gutenberg.pglaf.org` | front matter year (imprint / copyright / roman numerals); else author death year as an **upper bound** (flagged) | public domain in the US (catalog + licence header) | pool 7 799; 500 requested |
++| NCBI Bookshelf open-access books | FTP `pub/litarch` (BITS XML) | BITS `<pub-date>` | BITS `<permissions>` (CC BY / public domain NIH works; UNKNOWN rejected) | 3 941 pre-2018 books; 300 requested |
++| OpenStax / CNX pre-2018 editions | Internet Archive items; date and CC statement read **inside the PDF** | PDF copyright page | PDF licence statement | 5 trial items |
++
++Excluded by policy: `TDM` author manuscripts, `CC-BY-ND` / `CC-BY-NC-ND`, `UNKNOWN`/`OTHER-OA`, the arXiv non-exclusive
++licence, retracted articles, non-English text, near-duplicates (MinHash Jaccard ≥ 0.8), references/acknowledgements/
++funding/ethics sections, captions and list items (kept in the parsed store, never eligible).
++
++## Processing
++JATS/BITS, LaTeX, PDF and plain-text parsers → paragraphs with offsets into the normalised document text → eligibility
++(60–400 words, ≥ 2 sentences, English ≥ 0.9, quality ≥ 0.6) → protected-span placeholders → LLM rewrite (variants A–E,
++prompts `rw-1.0`) → automated validation (`val-1.0`: language, length ratio, near-copy, placeholder multiset, numbers,
++entities, negation/hedging deltas, structure for D, formatting artefacts, MiniLM semantic similarity) → quarantine of
++failures (kept, never deleted) → leakage-safe splits (document + near-duplicate cluster + author group; held-out discipline
++and period; pilot assignments frozen).
++
++## Release policy (HTR-Benchmark)
++- **Public subset**: pairs whose document licence is `CC0`, `PUBLIC-DOMAIN` or `CC-BY` — redistributed with attribution
++  (title, authors, venue, DOI/PMCID/arXiv id/Gutenberg id, licence URI, "modified: segmented and placeholder-protected").
++- **Research subset** (`CC-BY-SA`, `CC-BY-NC`, `CC-BY-NC-SA`): metadata + identifiers + scripts to rebuild locally; text
++  shared only under the matching licence terms (SA → CC BY-SA; NC → non-commercial research).
++- **Never released**: documents of tiers C/X, quarantined pairs' rewrites of non-releasable originals, provider request
++  payloads beyond what the schema records.
++- LLM rewrites are our outputs under Anthropic's commercial terms; their redistribution and use for training are recorded as a
++  compliance item to confirm before publication (`docs/RISK_REGISTER.md`).
++- Takedown: a retraction or licence change at the source triggers removal from the next release and a dated note in
++  `docs/DECISION_LOG.md`.
++
++## Known biases and limitations
++- Biomedicine dominates PMC; economics/finance, mathematics, physics and books are deliberate strata but smaller.
++- Discipline labels are heuristic (journal title / subject categories / arXiv categories / LoCC classes).
++- Pre-1950 PMC material is OCR and rarely passes quality; historical variation comes mostly from books, whose dates may
++  be upper bounds.
++- Validation heuristics: spelled-out numbers count as missing; entity matching is a proxy.
++- The rewrites reflect a specific model generation (Claude 5.5 family, late 2026) and six prompt variants.
++
++## Citation and acknowledgement
++Acknowledge NLM/NCBI (PMC Article Datasets on AWS, accessed 2026-11), arXiv, Crossref, the publishers of the CC BY
++journals, Project Gutenberg, the Internet Archive and OpenStax (Rice University). See `LICENSE_MATRIX.md`.

modified paper/appendix/appendix.tex +188 −5

++% ============================================================================
++% Appendices
++% ============================================================================
++\providecommand{\evid}[1]{\texttt{\detokenize{#1}}}
++\newenvironment{promptblock}{\begin{quote}\small\ttfamily\raggedright}{\end{quote}}
++
+ \appendix
++
++\section{Rewrite Prompts (version \texttt{rw-1.0})}
++\label{app:prompts}
++
++The prompts below are reproduced verbatim from \evid{src/htr/rewrite/prompts.py}. Two typographic substitutions
++are made for this document only. (i) In the code, placeholders are delimited by the mathematical white square
++brackets U+27E6 and U+27E7 (for example, the citation placeholder is the letter C and the index 1 between these two
++glyphs); here they are written \texttt{[[C1]]}, \texttt{[[M1]]} and \texttt{[[U1]]}. (ii) LaTeX special characters
++(\texttt{\%}, \texttt{\$}) are escaped. Line breaks inside a string are not significant.
++
++\paragraph{System message (all variants).}
++\begin{promptblock}
++You rewrite passages of scientific writing for a research study on how language models transform prose. You never
++invent information. Placeholders of the form [[C1]], [[M1]] and [[U1]] stand for citations, equations and links;
++copy each of them verbatim, exactly once per occurrence in the source, in the position where it belongs.
++\end{promptblock}
++
++\paragraph{User message template.} The user message is the variant instruction, a blank line, the preservation
++block, a blank line, and the passage in placeholder form:
++\begin{promptblock}
++\{instruction\}\\[0.5ex]
++\{PRESERVE\}\\[0.5ex]
++<passage>\\
++\{passage\_text\}\\
++</passage>
++\end{promptblock}
++
++\paragraph{Preservation block (\texttt{PRESERVE}, all variants).}
++\begin{promptblock}
++Preserve every scientific proposition exactly: all numbers, units, statistical values, named entities, species,
++genes, drugs, places, dates, citation markers (e.g. [12], (Smith et al., 2015), [[C1]]), equations and math tokens
++(e.g. \$x\$, [[M1]]), URLs/DOIs ([[U1]]), negations, hedges and expressions of uncertainty. Do not add, remove or
++reorder facts. Do not add commentary, headings, bullet points or notes. Keep the passage as a single paragraph in
++English. Output only the rewritten passage.
++\end{promptblock}
++
++\paragraph{Variant A (neutral paraphrase).}
++\begin{promptblock}
++Paraphrase the following passage in neutral academic English. Change the wording and sentence structure freely
++while keeping the meaning identical.
++\end{promptblock}
++
++\paragraph{Variant B (polished, standardised).}
++\begin{promptblock}
++Rewrite the following passage as polished, standardized contemporary academic prose: improve clarity, flow and
++concision, use conventional modern scientific phrasing, and correct awkward constructions.
++\end{promptblock}
++
++\paragraph{Variant C (concise).}
++\begin{promptblock}
++Rewrite the following passage more concisely (aim for about 70\% of the original length) without dropping any fact,
++number, citation or qualification.
++\end{promptblock}
++
++\paragraph{Variant D (structure-preserving).}
++\begin{promptblock}
++Rewrite the following passage sentence by sentence: keep the same number of sentences and the same order of ideas,
++but rephrase each sentence in your own words.
++\end{promptblock}
++
++\paragraph{Variant E (diverse sampling; two independent samples per passage).}
++\begin{promptblock}
++Rewrite the following passage in your own words as a fluent academic paragraph. Vary the wording naturally.
++\end{promptblock}
++
++\paragraph{Held-out prompt \texttt{H1} (plain-language expert; never used in training).}
++\begin{promptblock}
++Rewrite the following passage for an expert reader in plain, direct language with short sentences.
++\end{promptblock}
++
++\paragraph{Held-out prompt \texttt{H2} (editorial polish; never used in training).}
++\begin{promptblock}
++You are a copy-editor at a scientific journal. Edit the following passage for publication: fix style, tighten the
++prose, keep the author's meaning.
++\end{promptblock}
++
++\paragraph{Request settings.} Model \texttt{claude-opus-5-5} (held-out models \texttt{claude-sonnet-5-5},
++\texttt{claude-haiku-5-5}); effort \texttt{low}; \texttt{max\_tokens} 1\,500; no temperature set; thinking at the
++provider default; Message Batches; samples per variant A--D: 1, E: 2; pilot budget ceiling 30~USD
++(\evid{configs/rewriting/pilot.json}). A fingerprint of the prompt set (SHA-256 over version, system message,
++preservation block and variants A--E) is logged with every job.
++
+ \section{Hyper-parameters}
++\label{app:hyper}
++
++Table~\ref{tab:hyper} lists the training and model hyper-parameters. The ``code default'' column gives the values
++in the repository at this version (\evid{src/htr/training/trainer.py}, \evid{scripts/train.py},
++\evid{src/htr/models/config.py}, \evid{src/htr/models/losses.py}); where the training script overrides a library
++default, both are shown. Values used for the reported models are read from the run manifests and are pending.
++
++\begin{table}[ht]
++\centering
++\caption{Hyper-parameters. Script default / trainer default where they differ.}
++\label{tab:hyper}
++\small
++\begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{0.3\textwidth}l@{}}
++\toprule
++Hyper-parameter & Code default & Reported models \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{Optimisation}} \\
++Optimiser & AdamW, $\beta = (0.9, 0.98)$, $\epsilon = 10^{-8}$ & \pending{manifest} \\
++Peak learning rate & $3 \times 10^{-4}$ & \pending{manifest} \\
++Weight decay & 0.01 (not on biases, norms) & \pending{manifest} \\
++Warm-up steps & 300 / 500 & \pending{manifest} \\
++Maximum steps & 5\,000 / 20\,000 & \pending{manifest} \\
++Schedule & linear warm-up, cosine to 10\% of peak & \pending{manifest} \\
++Gradient clipping (norm) & 1.0 & \pending{manifest} \\
++Gradient accumulation & 1 & \pending{manifest} \\
++Batch budget & 8\,192 tokens, at most 48 pairs & \pending{manifest} \\
++Evaluation interval & 250 / 500 steps & \pending{manifest} \\
++Early-stopping patience & 6 evaluations (validation token CE) & \pending{manifest} \\
++Precision & bf16 on CUDA, fp32 on MPS & \pending{manifest} \\
++Seeds & 1, 2, 3 & \pending{manifest} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{Model}} \\
++Size design point & tiny (Table~\ref{tab:sizes}) & \pending{manifest} \\
++Dropout & 0.1 & \pending{manifest} \\
++Positional encoding & RoPE (self-attention) & \pending{manifest} \\
++Maximum source / target length & 512 / 512 tokens & \pending{manifest} \\
++Tokenizer & SentencePiece unigram, train split only; design vocabulary 32\,000 & \pending{vocabulary size} \\
++Copy gate & on & \pending{manifest} \\
++Style tokens $T_s$ (M3) & 4 & \pending{manifest} \\
++Style classes $K$ (M3) & (discipline, era) buckets with $\geq 20$ training pairs, plus ``other'' & \pending{$K$} \\
++Gradient-reversal $\lambda$ (M3) & 0.1 & \pending{manifest} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{Loss}} \\
++Label smoothing $\varepsilon$ & 0.1 (0 in overfit test and validation) & \pending{manifest} \\
++Contrastive temperature $\tau$ & 0.07 & \pending{manifest} \\
++Loss weights & Table~\ref{tab:loss-weights} & \pending{manifest} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{Decoding and evaluation}} \\
++Decoding & greedy (beam width chosen on validation) & \pending{freeze record} \\
++Bootstrap replicates & 1\,000 (development) / 10\,000 (confirmatory) & \pending{freeze record} \\
++\bottomrule
++\end{tabularx}
++\end{table}
++
++\section{Reproducibility Checklist}
++\label{app:checklist}
++
++The checklist follows the structure of the ACL Responsible NLP Research checklist. ``Yes'' means the item exists in
++the repository at this version; ``Partial'' that it exists for the completed gates only; ``Pending'' that it
++depends on runs not yet made.
++
++\begin{center}
++\small
++\begin{tabularx}{\textwidth}{@{}>{\raggedright\arraybackslash}X>{\raggedright\arraybackslash}p{0.1\textwidth}>{\raggedright\arraybackslash}p{0.33\textwidth}@{}}
++\toprule
++Item & Status & Where \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{A. Claims, limitations and risks}} \\
++Main claims stated and scoped (no first-inversion, authorship or detection claim) & Yes & Sections~\ref{sec:intro}, \ref{sec:related} \\
++Limitations discussed & Pending & Limitations section (after results) \\
++Potential risks and misuse discussed & Yes & Section~\ref{sec:ethics} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{B. Scientific artefacts}} \\
++Sources of data cited and described & Yes & Section~\ref{sec:data} \\
++Licence of every document recorded and respected & Yes & licence policy, per-document metadata \\
++Intended use of artefacts stated & Partial & Section~\ref{sec:ethics}; model card template \\
++Personal or offensive content checked & Partial & published scholarly text only; no systematic screen \\
++Documentation of artefacts (data card, model card) & Pending & model-card template exists \\
++Dataset statistics (counts, splits) reported & Partial & pilot corpus and splits; Medium stage \pending{counts} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{C. Computational experiments}} \\
++Parameter counts, compute budget, infrastructure & Partial & design points (Table~\ref{tab:sizes}); per-run counts \pending{manifests} \\
++Experimental setup and hyper-parameters & Partial & Section~\ref{sec:design}, Table~\ref{tab:hyper} \\
++Descriptive statistics with error bars over seeds & Pending & 3 seeds, clustered bootstrap planned \\
++Packages and versions for pre-processing and evaluation & Yes & \evid{pyproject.toml}; versions logged per run \\
++Code for data acquisition, training and evaluation & Yes & \evid{src/htr/}, \evid{scripts/} \\
++Random seeds, data hashes, run manifests & Yes & per-run manifests; split seed 20261109 \\
++Pre-registration of confirmatory tests & Partial & draft v0.1; freeze \pending{commit} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{D. Human annotators and participants}} \\
++Instructions, recruitment, payment, consent & Pending & protocol in Section~\ref{sec:design} \\
++Ethics review & Pending & UQO \pending{approval} \\
++\midrule
++\multicolumn{3}{@{}l}{\emph{E. Use of AI assistants and paid APIs}} \\
++Model identifiers, prompts and settings of the rewriting LLM & Yes & Section~\ref{sec:rewrite}, Appendix~\ref{app:prompts} \\
++API spend disclosed & Yes & 29.68 USD (Table~\ref{tab:pilot-jobs}) \\
++Use of AI assistance in research and writing disclosed & Yes & Section~\ref{sec:ethics} \\
++\bottomrule
++\end{tabularx}
++\end{center}

modified paper/main.pdf +0 −0

Binary file not shown.

spb/wpai0002_uqo

Data card draft

Theme