Testing
There are four kinds of automated check:
| Check | Where | Runs |
|---|---|---|
| Engine tests | Packages/ORIXDocsKit/Tests/ (Swift Testing) |
swift test --no-parallel, no window |
| Interaction self-test | App/Sources/App/SelfTest.swift |
inside the built app, -ORIXSelfTest <report.json> |
| Recovery test | Scripts/recovery-test.sh |
three launches of the built app |
| XCUITest suite | UITests/EditorUITests.swift |
Xcode UI testing — not run, see below |
Scripts/test.sh [engines|app|all] runs the first three and writes docs/reports/test-results.json (date, number of test targets, tests and suites with the overall result, self-test scenario counts, last line of the recovery script). That file is the record of the last run; this document does not restate its numbers as a status.
Engine tests
How to run
cd Packages/ORIXDocsKit
swift test --no-parallel # everything
swift test --no-parallel --filter KeepRuleTests
ORIX_ONLY=DocModel,DocModelTests swift test # only the listed targets are part of the packageWhy serial. Swift Testing runs tests in parallel by default. Tests that lay out documents from many threads at once can stall in the system font registry (the reason FontCache reads the installed-family list once, see layout-engine.md), so the suite is run with --no-parallel, as Scripts/test.sh does.
Environment variables read by the tests:
| Variable | Effect |
|---|---|
ORIX_ONLY |
Comma-separated target names; Package.swift drops every other target. Lets one engine be tested while another does not compile. |
ORIX_SLOW_TESTS=1 |
Enables the LibreOffice conversions in DocDOCXTests (ExternalValidationTests, the compatibility report). Without it they print ORIX-DOCX-SKIP: … and return. |
ORIX_ARTIFACTS=<dir> |
TemplateRenderingTests writes each template's PDF and PNG renderings of its first three pages there. |
ORIX_DOCX_REPORT_DIR=<dir> |
Where CompatibilityReportTests writes the DOCX compatibility report (set by Scripts/docx-compat.sh). |
ORIX_AI_TESTS=1 |
Enables OnDeviceModelTests in DocAutomationTests, which need the system's on-device model. |
ORIX_AUTOMATION_HOME=<dir> |
Moves the automation history and workflow folder; the tests point it at a temporary directory. |
Tests that depend on an external tool (Poppler, pandoc, LibreOffice, textutil) return early with a printed …-SKIP line when the tool is missing. A skipped check still counts as a passed test in the totals.
Targets
The counts are source counts: the number of @Test attributes and @Suite attributes in each target's files (grep -cE '^\s*@Test'), taken when this document was written. A parameterised test counts once here, whatever its number of arguments.
| Target | @Test |
@Suite |
Covers |
|---|---|---|---|
DocModelTests |
74 | 9 | Typing and typing attributes; composed-character safety (UnicodeTests); paragraph splitting and joining; character and paragraph formatting, style inheritance; lists and numbering, captions, cross-references; copy/paste fragments and JSON round trip; track changes including a randomised accept/reject invariant; comments, find and replace; table editing, sorting and cell formulas. |
DocContainerTests |
5 | 1 | The ZIP reader and writer (ZipTests). |
DocDOCXTests |
73 | 9 | DOCX import of a pandoc/LibreOffice corpus and of hand-authored packages; export → reimport round trips; export checked by independent tools; hostile input; scale; LaTeX ↔ OMML math conversion; the compatibility report. See docx.md. |
DocFormatsTests |
110 | 13 | Native container (persistence.md), Markdown, HTML and RTF import and export, plain text, PDF text import, the DocumentIO facade, Unicode torture cases, hostile input, cross-checks against pandoc and textutil. |
DocLayoutTests |
60 | 11 | Page geometry, breaks and sections, keep rules, footnotes, table layout, generated content, typesetting, geometry queries (pagination.md); PDF publishing (pdf.md); math robustness (hostile and random LaTeX never crashes); every template lays out and publishes. |
DocReferencesTests |
56 | 10 | BibTeX, RIS and CSL-JSON parsing; the citation engine; APA, Chicago, IEEE, MLA and Vancouver formatting. See references.md. |
DocReviewTests |
48 | 9 | Document comparison: content, formatting, moves, tables, inline objects and stories, Unicode, invariants, reports, performance. See review.md. |
DocAuditTests |
49 | 9 | Auditor checks (headings, references, formatting and review, pages, language), behaviour, search, acronyms, research tools. See audit.md. |
DocTemplatesTests |
43 | 6 | Template catalogue, integrity, features, context, language, genre. |
DocAutomationTests |
64 | 8 | The orixdocs command line, workflow steps and runner, text-intelligence support and runner, the on-device-only guarantee, confinement of images referenced by imported documents. See automation.md. |
| Total | 581 | 84 |
There are no unit tests in the app target; the app's behaviour is covered by the self-test below.
Interaction self-test
SelfTest is compiled into the application and started by the launch option -ORIXSelfTest <report.json> (setup.md). It synthesises NSEvent key and mouse events and sends them to the document window (window.sendEvent, and NSApp.mainMenu.performKeyEquivalent for ⌘ shortcuts), so each scenario goes through the same path as a person at the keyboard: window → canvas → input context → NSTextInputClient / doCommand(by:) → editor → layout. After each step it compares the document state. Some steps call DocumentSession.run or perform directly (inserting a table, toggling tracking); they are not key presses.
It writes a JSON report (scenarios with name, passed, failures, milliseconds; totals; date; system version) and the process exits.
Scripts/build.sh
"build/DerivedData/Build/Products/Debug/ORIX Docs.app/Contents/MacOS/ORIX Docs" \
-ApplePersistenceIgnoreState YES -ORIXTemplate blank \
-ORIXSelfTest "$PWD/build/selftest.json" -checkSpelling NO -AppleLanguages "(en)"The 19 scenarios, in order:
| # | Scenario | Checks |
|---|---|---|
| 1 | typing, Return and Backspace | Two typed paragraphs; five Backspaces; caret position. |
| 2 | Unicode typing deletes whole characters | A ZWJ family emoji and CJK text; Backspace removes one composed character at a time. |
| 3 | arrow keys, word and line movement | ⌥←, ⌥⌫, ⌘←, ⌘→, ⇧← selection replaced by typing, forward delete. |
| 4 | vertical movement keeps the column | ↓ and ↑ through a short line restore the column. |
| 5 | undo, redo and select all | ⌘A + Delete, undo, redo, a second undo. |
| 6 | typing is coalesced into one undo step per burst | One undo removes a typed word. |
| 7 | bold shortcut applies to typed text | ⌘B with a caret formats only what is typed next. |
| 8 | smart quotes and dashes | "Hi" -- it's becomes “Hi” — it’s. |
| 9 | mouse: click, shift-click, double and triple click, drag | Caret placement, extension, word and paragraph selection, drag across paragraphs, typing over the selection. |
| 10 | copy, cut and paste through the pasteboard | ⌘C/⌘V keep bold; ⌘X then ⌘V restores; plain text and RTF flavours are present. |
| 11 | lists: Tab indents, Return on an empty item leaves the list | Labels "1." and "a."; leaving the list. |
| 12 | table: insert, type, Tab between cells, new row at the end | Cell contents, added row, ⇧Tab, Delete Rows. |
| 13 | footnote: insert, type in the note, Escape returns | Note text, return to body, deleting the mark deletes the note. |
| 14 | header: edit, page field, return to body | Header text and page field stored and laid out. |
| 15 | track changes: insert, delete, accept and reject | Two tracked changes, Reject All, undo, Accept All. |
| 16 | comment anchored on the selection | A comment anchored on the double-clicked word. |
| 17 | find and replace | Two matches, Find Next, Replace All. |
| 18 | page breaks create pages; many pages scroll with the caret | Four pages; the caret's page is visible. |
| 19 | save, reopen and autosave through NSDocument | Save to a file and read it back; edit and autosave; an external overwrite stays on disk (persistence.md). |
Limits of the self-test: events are synthesised inside the process, so it does not prove that the window server delivers events, nor exercise an input method, dictation, VoiceOver or drag and drop from another application. The run sets UndoManager.groupsByEvent = false, which is why DocumentSession.registerUndo opens explicit undo groups (text-engine.md).
Recovery test
Scripts/build.sh
Scripts/recovery-test.sh # ORIX_APP=<path to .app> and ORIX_RECOVERY_WAIT=<seconds> are optionalCreates a saved document through the app, types into it with -ORIXType and a two-second autosave delay, sends SIGKILL, reopens the file and compares the text; prints PASS or FAIL. What it does and does not prove is set out in persistence.md.
XCUITest suite
UITests/EditorUITests.swift holds ten XCUITest methods that drive the application through the accessibility interface (the canvas is found by its identifier document-canvas and its text read from its accessibility value): typing and Backspace, Unicode typing, undo/redo and Select All, arrow keys and word deletion, the Bold shortcut and toolbar state, modes, inserting a table from the menu, footnotes and track changes, save and reopen, and autosave surviving a killed process. The target is ORIXDocsUITests in project.yml, attached to the ORIXDocs scheme.
This suite has not been run. XCUITest needs macOS Automation Mode (UI automation permission) to be authorised on the machine by its owner, and that authorisation was not given. No result is claimed for it. The interaction self-test was written to cover the same ground without that permission, and it is the check that was used instead; the two overlap but are not equivalent — the XCUITest suite goes through the real accessibility and event system, the self-test does not.
To run it once Automation Mode is authorised:
xcodegen generate
xcodebuild -project ORIXDocs.xcodeproj -scheme ORIXDocs -derivedDataPath build/DerivedData testOther reports
Scripts/docx-compat.shruns the DOCX corpus cycle and writesdocs/reports/docx-compatibility.mdand.json.swift run -c release orixbenchwrites the benchmark figures (performance.md).