SPB Git

Repository Public

references.md 21.3 KB

# References engine (DocReferences)

DocReferences reads and writes bibliography files (BibTeX / BibLaTeX, RIS, CSL-JSON), merges them into a document's library, and formats in-text citations and the generated bibliography. It depends on DocModel and Foundation only, and never touches the network (no DOI lookup, no style download).

This is not a CSL processor. The six built-in styles are hand-written Swift code that follows the published rules of each guide. A .csl file cannot be loaded, and a style that is not in the list below does not exist in ORIX Docs.

# Why no citeproc

Option evaluated Why it was not embedded
A Swift CSL processor None is maintained. The Swift packages that exist are partial experiments without the CSL test-suite coverage a word processor needs.
citeproc-js in JavaScriptCore Works, but adds a large JavaScript runtime dependency plus the CSL style and locale repositories to an engine package whose rule is "no third-party dependency except SwiftMath". Output is HTML or RTF that would have to be converted back to runs.
citeproc-rs (Rust) or Pandoc citeproc (Haskell) through a C interface Adds a second toolchain and a binary artefact to the build, and the same style and locale payload.
Writing a full CSL 1.0.2 processor here Several months of work on its own (name disambiguation, substitution, cite collapsing, locales, about 2 500 conformance tests). Out of proportion for six styles.

The price of this choice is stated in "Known deviations" and "Limitations" below. If arbitrary CSL styles become a requirement, CitationFormatting (in DocModel) is the seam: a citeproc-backed implementation can replace CitationEngine without touching the model.

# Files

File Content
BibTeXParser.swift BibTeX / BibLaTeX syntax, @string, #, @preamble, @comment, crossref and xdata inheritance.
BibTeXMapping.swift Entry types and fields ⇄ CSL types and variables; BibTeXWriter.
LaTeXText.swift LaTeX ⇄ Unicode for field values.
NameParser.swift BibTeX name lists ⇄ PersonName.
RIS.swift RISParser, RISWriter, type tables.
CSLJSON.swift CSL-JSON reader and writer.
BibliographyImporter.swift Format detection, library merge, ReferenceChecks.
CitationEngine.swift Public engine: style catalogue, in-text citations, ordering, disambiguation, bibliography paragraphs.
StyleFormatter.swift Reference-list formatting of the six styles, in-text author labels, sort keys.
ReferenceView.swift, RichText.swift, ReferenceSupport.swift Entry view and name formatting; run builder and localised terms; dates, DOI, text folding.

# How a record is stored

BibEntry is the frozen model type: a CSL type, authors, editors, year / month / day and a [String: String] of CSL variables. What does not fit is kept by convention, so nothing is dropped on import:

Information Where it lives
Name particle ("van", "de la") In PersonName.family ("van Gogh"). A CSL dropping particle goes after the given name ("Ludwig van").
Name suffix ("Jr.", "3rd") After the given name, comma-separated: given = "Martin Luther, Jr.". PersonName.givenAndSuffix splits it.
"and others" PersonName.others (literal "others"). Formatters print "et al.".
Institutional author PersonName.literal.
Other name lists (translator, container-author, …) fields[variable] as a BibTeX name list: "Goldstein, Ann and {Some Collective}".
Date that is not a plain year ("in press") fields["issued-literal"]; year stays nil.
Date range, season, approximate date year/month/day hold the start; the original EDTF string stays in fields["date"]; a season name in fields["season"].
Other dates (accessed, event-date, original-date) fields[variable] as ISO 8601 / EDTF text.
Article number without pages fields["article-number"].
Italic, bold, superscript, subscript, small caps inside a title CSL-JSON markup: <i>…</i>, <b>…</b>, <sup>…</sup>, <sub>…</sub>, <span style="font-variant:small-caps;">…</span>.
Unknown BibTeX entry type type = "document" and fields["entrytype"].
A source field with no CSL equivalent fields[<original name>] (eprint, mycustomfield, RIS L1, …). A source field whose name collides with a variable already filled is stored as x-<name>.

# Parsers and writers

Every parser returns BibliographyParseResult(entries, issues). issues are sentences for the user, with a line number for BibTeX and RIS and an item number for CSL-JSON. A parser never fills a gap with invented data. The one thing it generates is a citation key when the source has none (ReferenceChecks.suggestedKey), and it says so in issues.

# BibTeX / BibLaTeX

  • Syntax: @type{…} and @type(…), @string macros (case-insensitive), # concatenation, numbers, brace and quote delimiters with nesting, @preamble and @comment (skipped), % comments between tokens, text outside entries ignored.
  • Month macros jan…dec; a month may carry a day ("15~" # jan). French and English month names are understood.
  • crossref follows BibLaTeX inheritance: a container's title becomes the child's booktitle (maintitle for mv* parents, journaltitle for periodicals), the rest is copied when missing. Chains and cycles are handled. xdata is inlined and @xdata entries are not returned.
  • Names follow BibTeX rules: "First von Last", "von Last, First", "von Last, Jr, First", "and others", fully braced names as institutions.
  • LaTeX decoding: all standard accent commands in every spelling ({\'e}, \'{e}, \'e, \"\i), special letters (\ss, \ae, \o, \l, \aa, …), --, ---, ~, `` and '', escaped specials, text symbols, \emph / \textit / \textbf / \textsc / super- and subscript (to the markup above), old-style {\em …}, \url, \href. Case-protecting braces are removed.
  • Left untouched: math ($…$, \(…\), \[…\]) and unknown commands with their arguments.
  • Errors: reported with the line number; the fields read before the error are kept and parsing resumes at the next line starting with @. Duplicate keys keep the first entry; duplicate fields keep the first value.
  • BibTeXWriter writes UTF-8 text, escapes only what BibTeX treats specially, writes months as macros, protects words with inner capitals in titles ({NASA}, {iPhone}), and writes every unmapped field back under its name. parse(write(x)) == x is tested.

Type mapping (BibTeX → CSL): article → article-journal (article-magazine / article-newspaper with entrysubtype), book mvbook collection reference proceedings manual → book, inbook incollection → chapter, inproceedings conference → paper-conference, inreference → entry-encyclopedia, phdthesis mastersthesis thesis → thesis (genre "PhD thesis" / "Master's thesis" or the type field), techreport report → report, unpublished → manuscript, online electronic www → webpage, misc → document, plus patent, dataset, software, periodical, booklet, audio, video, artwork, letter, legislation, jurisdiction, standard, performance, review. The writer uses the classic type where one exists; when a CSL type has no BibTeX type it writes @misc with a csltype field, which the parser reads back.

Field mapping: journal / journaltitle / booktitle → container-title, subtitle joined to the title, series → collection-title, number → issue (articles), collection-number (books, chapters, proceedings) or number (reports, theses), pages → page, eid → article-number, school / institution / organization / publisher → publisher, address / location → publisher-place, type → genre, chapter → chapter-number, date / year / month / day → the date slots, urldate → accessed, eventtitle / eventdate / venue, origdate, pubstate → status, pagetotal, volumes, shortjournal, shorttitle, bookauthor, translator, doi (resolver prefix removed), url, isbn, issn, pmid, pmcid, language, keywords, abstract, note, annote.

# RIS

Tags read: TY, ID, AU / A1, A2 / ED, A3, A4, TI / T1 / BT, T2, T3, JO / JF / J1 / J2, JA, PY / Y1 / DA, Y2, VL, IS, SP / EP, DO, UR, PB, CY, SN (ISSN for periodicals, ISBN otherwise), AB / N2, KW (repeatable), N1, ET, M1, M3, LA, ST, NV, CN, AN, DB, DP, C7, ER. Continuation lines are joined to the previous value. Any other tag is kept under its two-character name and written back. 54 TY values are mapped to CSL types and 36 CSL types are mapped back.

# CSL-JSON

Reads an array, a single item, or an {"items": […]} wrapper. Names: family, given, literal, non-dropping-particle, dropping-particle, suffix. Dates: date-parts (numbers or strings, one or two parts), raw, literal, season, circa, or a plain EDTF string. Numbers and strings are interchangeable everywhere. Structured values under unknown keys are kept as JSON text and reported. The writer produces a sorted, pretty-printed standard array.

# Import and merge

BibliographyImporter.importEntries detects the format from the content first (the content is what gets parsed), then from the extension. Files that are not UTF-8 are read as UTF-16 (with a byte-order mark) or ISO Latin-1, and the user is told.

merge(_:into:) treats an incoming entry as a duplicate when, in this order: its key exists and it is the same work; its DOI matches (case-insensitive, resolver prefix ignored); its normalised title (letters and digits only, diacritics folded) and year match and the two do not carry different DOIs. A key used by a different work is renamed key-2, key-3, … Existing entries are never modified. The log has one line per decision.

# Citation engine

swift
let engine = CitationEngine(document: doc)             // doc.settings.citationStyle, doc.meta.language
engine.citationText(citation, document: doc, numbering: numbering)
engine.bibliographyParagraphs(document: doc, numbering: numbering)
engine.formattedEntry(entry, number: nil)              // preview, no document context

Only cited works that exist in DocumentModel.bibliography appear in the bibliography. What depends on the whole set of cited works (order, numbers, year suffixes) is computed once per document value and cached. The cache key is the cited-key list and the bibliography array, so an unchanged document costs one array-identity comparison per call, and an edited one is recomputed.

# Styles

Id Guide In text Reference list order
apa Publication Manual of the APA, 7th ed. (Grady et al., 2019, p. 4) Alphabetical
chicago-author-date Chicago Manual of Style, 17th ed., author-date (Grazer and Fishman 2015, 12) Alphabetical
mla MLA Handbook, 9th ed. (Baron 194) Alphabetical, by author then title
ieee IEEE Reference Guide (2023 / 2025) [1], [3]–[5] First citation
numeric Generic: IEEE reference format, full journal titles [1, 3–5] First citation
vancouver ICMJE recommendations / NLM sample references (1,3–5) First citation

An unknown style id falls back to apa.

# Rules covered

  • Authors: 1, 2, 3+ authors with each style's "et al." threshold: APA lists up to 20 and uses "first 19, . . . last" from 21; Chicago lists up to 10, then 7 and "et al."; MLA uses "et al." from 3; IEEE lists up to 6, then the first and "et al."; Vancouver lists 6 then "et al.". In-text: APA "et al." from 3, Chicago from 4, MLA from 3. Institutional authors, suffixes and particles are handled. An incomplete list ("and others") always ends with "et al.".
  • No author: editors take the author position for whole works ("(Ed.)", "ed.", "editor", "Ed."); otherwise the title does, in the list and in the citation.
  • Dates: year; year-month-day for web pages, newspapers and magazines (APA, Chicago, MLA); "n.d." (APA, Chicago), nothing (MLA); literal dates ("in press"); retrieval or access date for undated web pages.
  • Same author, same year: suffixes 2012a, 2012b, n.d.-a, assigned in reference-list order (by title). Consecutive works with the same author label collapse: "(Judge & Kammeyer-Mueller, 2012a, 2012b)". MLA adds a short title instead.
  • Types: journal, magazine and newspaper articles (volume, issue, pages or article number), books (edition, volume, translators, editors), chapters in edited books, conference papers with or without proceedings, theses, reports with number, web pages, and a book-like fallback for everything else.
  • Titles: italics, or quotation marks with the style's closing punctuation inside; no doubled punctuation after a title ending in "?" or "!"; inner double quotation marks become single inside a quoted title; title markup is rendered as run properties.
  • Pages: en dash in simple ranges (APA, Chicago, IEEE), hyphen (Vancouver), as stored (MLA); "p." / "pp.".
  • DOI and URL: https://doi.org/… (APA, Chicago, MLA), doi: … (IEEE, Vancouver), "[Online]. Available:" (IEEE), "Available from:" (Vancouver). DOI and URL runs carry CharProps.link.
  • Sorting: author-date styles sort letter by letter on the authors ("nothing precedes something", one-author entries before multi-author ones), then year (undated first, "in press" last), then title without its leading article; the key is the last tie-breaker so the order is deterministic.
  • Citations: locators ("12", "pp. 12-14", "ch. 3" kept verbatim), prefix, suffix, suppressAuthor, several items in one citation (APA orders them as in the reference list unless a prefix or suffix is attached).
  • Numeric styles: numbers follow the first citation; three or more consecutive numbers collapse: [1]–[3], [5] (IEEE), [1–3, 5], (1–3,5).
  • Missing key: (?key) or [?key]. No entry is fabricated. Numbers are assigned to the works that exist, so citations and the list always agree.
  • French (meta.language starting with fr): "et" for "&" and "and", no comma before it, "s. d.", "Dans", "(dir.)", "2e éd.", "p." for "pp.", "sous la direction de", "traduit par", "Consulté le … à l’adresse", French month names.

# Title capitalisation

Titles are printed exactly as stored, in every style. APA and IEEE want sentence case for article and book titles, Chicago and MLA want headline case; converting one into the other needs to know which words are proper nouns, so the engine does not attempt it. The reversible rules that are applied to a title are: the closing full stop goes inside a final closing quotation mark (English only), and double quotation marks become single ones inside a quoted title.

# Known deviations

Each one is asserted in the tests next to the published string (expectKnownDeviation) or stated in the suite's header.

Style Deviation
All Titles are not re-cased (see above).
All Terminal punctuation after an italic title is roman. The Chicago and IEEE guides italicise it.
APA When several works reduce to the same in-text form but have different author lists, APA spells out more names; the engine adds year suffixes instead (as CSL does when name disambiguation is off).
APA No bracketed description is invented: an unpublished manuscript has no "[Unpublished manuscript]" and a conference talk no "[Conference session]" unless genre says so.
APA A title used as in-text author is plain text; citationText returns a String, so it cannot be italic.
APA Retrieval dates are printed only for undated web pages.
Chicago No "Last modified" or "Published" statement for web pages: the model has one date. Published: Google. 2017. “Privacy Policy.” Privacy & Terms. Last modified April 17, 2017. https://… Ours omits the "Last modified" sentence.
Chicago Page ranges are printed as stored, not abbreviated to Chicago's "167–95" form.
Chicago Reference-list entries by the same author do not use the 3-em dash (the 17th edition itself discourages it).
MLA Publisher names are not abbreviated ("University Press" → "UP"), URLs keep their scheme if stored with one, and page ranges are printed as stored.
MLA Only one container; a database or archive is supported as the second container through the archive variable.
IEEE Journal titles are abbreviated only when container-title-short is filled; no abbreviation list is embedded.
IEEE The guide prints "USA,Tech. Rep." without a space in its report example (a typo); the engine writes the space.
IEEE Web pages follow the 2023 / 2025 guide ("Accessed: … [Online.] Available: …"), not the 2018 form.
Vancouver Copyright dates ("c2016") and "[updated …]" are not produced. Published: eatright.org [Internet]. Chicago: Academy of Nutrition and Dietetics; c2016 [cited 2016 Dec 27]. … Ours prints 2016.
Vancouver Journal titles use container-title-short when present; NLM abbreviations are not embedded. Page ranges are printed as stored, not shortened ("284-7").
Vancouver In-text numbers are in parentheses on the line; superscript numbers are not offered.
French Only the connecting terms are localised. Quotation marks stay “ ” (no guillemets), and edition ordinals are limited to "1re" and "2e", "3e"…

# Sources of the test strings

Style Source Status
APA 7 apastyle.apa.org, reference examples pages 16 reference strings and their in-text forms checked against archived copies of the official pages (2022–2025).
APA 7 Printed manual: 21+ authors (Kalnay et al.), same author and year (Judge & Kammeyer-Mueller) Not checked online. Asserted as rules, and labelled so in the tests.
Chicago 17 chicagomanualofstyle.org, author-date sample citations 12 strings checked against archived copies of the 17th-edition page; 1 known deviation. The live page now shows the 18th edition.
IEEE IEEE Reference Guide 9 strings checked against the 2023 and 2025 versions; 1 known deviation.
MLA 9 style.mla.org, citations by format 6 strings checked. Two-author, "et al." and in-text rules are asserted as rules from the handbook, not as quoted examples.
Vancouver NLM, Samples of Formatted References 7 strings checked on the live page; 1 known deviation.

# Limitations

  • Six styles, no CSL import, no note-based styles (Chicago notes-bibliography, footnote citations), no "ibid.".
  • Title case conversion, journal abbreviation, publisher abbreviation and page-range abbreviation are not done.
  • citationText returns plain text: no italics and no hyperlink inside an in-text citation.
  • Contributor roles other than author, editor and translator are stored but not printed. Translators are printed by Chicago and MLA only.
  • Original publication dates, series titles, reviewed works, legal and audiovisual types have no dedicated layout; they use the book-like fallback.
  • BibTeX: @preamble content is discarded; the writer does not reproduce @string macros, crossref structure (inherited fields are written in full) or the original entry type when several map to one CSL type (@mvbook comes back as @book).
  • BibTeX: a title that contains math or a raw LaTeX command is written with its <i> markup as literal text and without case protection, to keep the braces balanced.
  • BibTeX: $…$ is recognised as math only when it has no white space next to the delimiters and is not followed by a digit; anything else is a literal dollar sign.
  • RIS: variables without an RIS tag are not written, because RIS has no free-form field (mycustomfield, eprint, …). A continuation line that itself looks like XX - text is read as a tag.
  • CSL-JSON: structured values under unknown keys come back as strings.
  • French localisation is limited to the terms listed above; other languages use English terms.
  • A year suffix needs the whole document, so formattedEntry (the preview) never shows one.

# Performance

Measured by EngineTests.performanceWith5000Entries on 5 000 entries (2.7 MiB of BibTeX), debug build, Apple Silicon:

Step Time
BibTeX parse 0.71 s
BibTeX write 0.74 s
CSL-JSON write / parse 0.09 s / 0.14 s
Merge 5 000 into 5 000 0.10 s
APA bibliography of 5 000 cited works 0.36 s
5 000 APA in-text citations 0.01 s
IEEE bibliography / 5 000 citations 0.20 s / 0.01 s

The test prints the measured times on every run.

# Model changes proposed

None is required; these would remove the conventions above.

  1. PersonName: add suffix and droppingParticle / nonDroppingParticle.
  2. BibEntry: add a structured date (end of range, season, circa, literal) and accessed, instead of the issued-literal, date, season and accessed strings.
  3. BibEntry: a dictionary of extra name lists (translator, container-author, …) as [String: [PersonName]].
  4. CitationFormatting.citationText returning [Run] so an in-text title can be italic and a citation can link to its entry.
  5. CitationItem.locator: a separate locator label (page, chapter, section) instead of parsing the text.
  6. DocumentSettings.citationStyle documentation: list vancouver as a valid identifier.
Files main
Menu

Theme