Repository Public
1<!--2ORIX Docs3File: docs/architecture/persistence.md4Description: The native .orixdoc file format: container layout, manifest, integrity checks, recovery and schema migrations.5Author: Simon-Pierre Boucher6Contact: contact@spboucher.ai7-->8# Persistence910## The native format: `.orixdoc`1112Code: `Packages/ORIXDocsKit/Sources/DocFormats/NativeContainer.swift`, `SchemaMigration.swift`.13Tests: `Tests/DocFormatsTests/NativeContainerTests.swift`.1415### Why a single ZIP file1617An `.orixdoc` document is one ZIP archive, not a directory package.18A package looks like a folder to everything that is not macOS: mail clients, most non-Apple cloud folders and web uploads split it, zip it behind the user's back or refuse it.19A single file survives all of them, and the ZIP layer is the defensive reader the DOCX engine already uses (`DocContainer.ZipReader`: entry-count, size and compression-ratio limits, no path traversal, no ZIP64, no encryption).2021### Layout2223| Entry | Stored as | Content |24|---|---|---|25| `mimetype` | first entry, uncompressed | `application/vnd.orix.docs+zip`. Being first and stored, the string sits at byte offset 38 of the file, so the type can be recognised without a ZIP reader (the OpenDocument / EPUB convention). |26| `manifest.json` | deflate | Container version, schema version, generator, creation date, and one record per other entry. |27| `document.json` | deflate | `JSONEncoder` output of `DocumentModel`: sorted keys, ISO-8601 dates. Assets and preserved parts are **not** in it. |28| `assets/<id>.<ext>` | stored for PNG / JPEG / GIF / HEIC / WebP, deflate otherwise | One entry per `DocumentModel.assets` value. |29| `preserved/<part name>` | deflate | One entry per `DocumentModel.preservedParts` value (uninterpreted parts of an imported DOCX), under the original part name. |3031Asset ids and DOCX part names are not always valid, portable entry names (a part name starts with `/`, an id may contain a space or a slash).32The entry name is therefore only a readable hint: the **manifest** maps each entry to its asset id or original part name (`key`) and records the media type of assets.33Ids are hex-escaped outside `[A-Za-z0-9_-]`; a part name that cannot be an entry name becomes `preserved/_partN.bin`.3435### The manifest3637(Sizes and dates below are illustrative.)3839```json40{41 "containerVersion": 1,42 "created": "2026-11-21T06:30:51.169Z",43 "entries": [44 { "name": "mimetype", "role": "mimetype", "sha256": "…", "size": 29 },45 { "name": "document.json", "role": "document", "sha256": "…", "size": 18342 },46 { "key": "photo", "mimeType": "image/jpeg", "name": "assets/photo.jpg", "role": "asset", "sha256": "…", "size": 707 },47 { "key": "/word/vbaProject.bin", "name": "preserved/word/vbaProject.bin", "role": "preserved", "sha256": "…", "size": 600 }48 ],49 "generator": { "name": "ORIX Docs", "version": "1.0" },50 "schemaVersion": 151}52```5354Every entry other than the manifest itself is listed with its byte size and the SHA-256 (CryptoKit) of its uncompressed bytes.55Two versions are recorded and checked independently:5657* `containerVersion` — the layout described on this page (currently 1);58* `schemaVersion` — the version of `document.json`, equal to `DocumentModel.schemaVersion`.5960### Dates6162Dates are ISO-8601 strings in UTC with a fractional part of up to nine digits (`2026-11-21T06:30:51.123456789Z`), trailing zeros removed.63Nine digits are more than a `Date` can resolve, so a date read back is bit-identical to the one written; a plain second-precision ISO-8601 date is accepted on reading.6465### Reading6667`NativeContainer.read` is strict. In order:68691. files larger than `FormatLimits.maxContainerBytes` (1 GiB) are refused (`FormatError.tooLarge`), as is anything `ZipLimits` refuses;702. the first entry must be `mimetype` with the expected content, otherwise the file is "not an ORIX Docs document" (`FormatError.unreadable`);713. `manifest.json` must exist and decode; a `containerVersion` or `schemaVersion` newer than this build throws `FormatError.unsupportedVersion`;724. every manifest entry must exist, have the recorded size and the recorded SHA-256 (the ZIP CRC-32 is checked by the reader underneath);735. `document.json` is parsed, migrated (below) and decoded.7475Any failure in steps 3–5 that is not a version problem throws `FormatError.integrityCheckFailed`.76Entries present in the archive but absent from the manifest are **tolerated and ignored** (reported as one `.info` warning): a future version, or another tool, may add a thumbnail or a signature without breaking older readers. An unlisted `assets/…` entry is not loaded.7778`NativeContainer.validate` runs the same checks and returns every problem as a sentence, instead of stopping at the first.7980### Recovery8182`NativeContainer.recover` is the best-effort path offered when `read` fails. It does not need the ZIP central directory (the part a truncated download loses): it walks the local file headers from the start of the file and keeps every entry whose data is complete, noting whether its CRC-32 holds.8384* `document.json` is used when it is intact, and also when its checksum fails but it still parses and decodes — with a warning asking the user to check the text;85* a manifest that fails its own CRC is ignored (it cannot vouch for anything), and the loss of checksums is reported;86* assets and preserved parts that are missing, incomplete or fail their checksum are skipped, one `.dropped` warning each; images whose asset is gone are counted in a `Missing images` warning;87* a file whose `document.json` cannot be recovered throws: recovery never invents a document.8889A sound file recovers to exactly the document `read` returns, with no warning.9091### Schema versions and migrations9293**Only schema version 1 exists today.** There is no older format in the wild, so `SchemaMigrator.standard` contains no migration step.9495The pipeline is nevertheless in place and runs on every read, strict or recovering:9697* `document.json` is first parsed as an untyped JSON tree (`[String: Any]`);98* its `schemaVersion` is read; a version newer than the build throws `FormatError.unsupportedVersion`;99* for each version from the document's up to the current one, the step registered for that source version transforms the tree into the next version; the pipeline itself updates `schemaVersion` after each step;100* the result is decoded into `DocumentModel`; when at least one step ran, an `.info` warning tells the user the document was upgraded.101102Steps work on the JSON tree, not on Swift types, so a step written today keeps working after the model has moved on.103A missing step for an old version is an error ("no migration is available from document schema N"), never a silent best guess.104105Because no real older schema exists, the tests exercise the pipeline with an injected migration table (`SchemaMigrator(steps:targetVersion:)`, internal) and deliberately invented "version 0" and "version −1" layouts: a renamed key, a moved key, steps run in sequence, a failing step, a missing step.106107When schema 2 is introduced:1081091. bump `DocumentModel.currentSchemaVersion`;1102. add `1: { tree in … }` to `SchemaMigrator.standard`;1113. add a fixture written by the last schema-1 build and a test that reads it.112113### What the format does not do114115* No encryption, no digital signature. SHA-256 here detects corruption, not tampering: someone who can rewrite the file can rewrite the manifest.116* No ZIP64: a document is limited to what a classic ZIP holds (4 GiB per entry and for the archive), and the reader's own limits are lower.117* No incremental save: the whole file is rewritten on each save.118* Writing is deterministic for a given document, generator and creation date (fixed ZIP timestamps, sorted keys and entries), which makes files diffable and cacheable.119120## Autosave, versions and recovery121122Code: `App/Sources/Document/ORIXDocument.swift`, `DocumentSession.swift` (`ModelStore`), `App/Sources/App/AppDelegate.swift`.123Checks: the scenario "save, reopen and autosave through NSDocument" in `App/Sources/App/SelfTest.swift`; `Scripts/recovery-test.sh`.124125### NSDocument126127The application is a classic AppKit document application: `ORIXDocument` subclasses `NSDocument`, and `NSDocumentController` owns opening and the list of recent documents. The reason is that autosave in place, versions, revert, duplicate, rename, move and file coordination then come from the system instead of being rewritten. The document class itself is small:128129| Override | Value or behaviour |130|---|---|131| `autosavesInPlace` | `true` |132| `preservesVersions` | `true` |133| `autosavesDrafts` | `true` |134| `writableTypes(for:)` | The native type only (`ai.spboucher.orix.docs.document`); other formats are written through Export. |135| `canAsynchronouslyWrite(to:ofType:for:)` | `true` |136| `data(ofType:)` | The native container of the current model. |137| `read(from:ofType:)` | Imports any supported format (below). |138139### What AppKit provides140141The File menu items are wired straight to `NSDocument` actions in `MainMenu.build` (`AppDelegate.swift`): Save (`save(_:)`), Duplicate (`duplicate(_:)`), Rename (`rename(_:)`), Move To (`move(_:)`), Revert To ▸ Last Saved Version (`revertToSaved(_:)`) and Browse All Versions (`browseVersions(_:)`). The versions browser, the storage of versions, renaming and moving are provided by AppKit; ORIX Docs adds no code to them beyond supplying and reading the file data.142143### Autosave delay144145`applicationWillFinishLaunching` sets `NSDocumentController.shared.autosavingDelay` to 5 seconds, or to the value of the `ORIXAutosaveDelay` default when it is set (never less than 1 second). AppKit autosaves that long after a change, and at other moments it chooses.146147### Background writing from a snapshot148149`DocumentSession` runs on the main actor and owns the editor. `NSDocument` may call `data(ofType:)` on a background thread, so the two meet in `ModelStore`, a lock-protected holder of the latest `DocumentModel`:150151* `DocumentSession.documentChanged()` stores the model after every edit;152* `data(ofType:)` reads it under the lock, stamps `meta.modified` on its copy and encodes the container.153154Because the model is a value type, what the save thread receives is a consistent snapshot; later edits on the main thread do not affect it. The provisional text of an input-method composition is not stored until it is committed, so an autosave during a composition writes the document as it was before the composition.155156### Foreign files open as untitled documents157158`read(from:ofType:)` chooses the format from the file extension (falling back to the type), imports the file with `DocumentConverter.importDocument`, and records the import warnings and the source format in the store. When the window is created (`makeWindowControllers`), a document that came from a foreign format (DOCX, RTF, Markdown, HTML, plain text, PDF) has its `fileURL` cleared, its type set to the native type and its change count marked (`.changeReadOtherContents`), keeping the original's name as display name. The result is an untitled, edited document: saving asks for a new `.orixdoc` location and **the original file is never written**. The document types other than the native one are declared with the `Viewer` role in `project.yml`.159160Import warnings are kept in the session and shown from the status bar; the warnings sheet opens by itself when at least one warning is more than informational.161162### Damaged native files163164For `.orixdoc`, `DocumentConverter.importDocument` first calls the strict `NativeContainer.read`. If it throws, it tries `NativeContainer.recover` (see "Recovery" above); when recovery succeeds, the document opens with an extra warning, "Damaged document: the file failed its integrity check (…). What could be read was recovered; save it under a new name.", which is of the `.dropped` kind and therefore opens the warnings sheet. When recovery also fails, the strict error is shown and nothing opens.165166A recovered document is still a native document: unlike an imported foreign file it **stays bound to the damaged file**, so autosave in place will rewrite that file after the next edit. The warning text asks the user to save under a new name; the code does not enforce it.167168### Revert and version restore169170Revert and restoring a version make AppKit call `read(from:ofType:)` on a document that already has a window. The read fills the store, then, on the main thread, `DocumentSession.reload(_:)` replaces the editor, empties the layout cache and lays the document out again.171172### External modification173174ORIX Docs contains no file-presenter code of its own: detecting that another program changed the file, and asking the user what to do when saving over it, is `NSDocument`'s standard behaviour.175176What the repository shows is one check in the self-test: after a save and an autosave, the test overwrites the file from outside with different content, lets the run loop turn for 1.5 seconds, and asserts that the file still holds the external content — that is, the open document did not silently write over it during that interval. The test does not make a further edit, does not trigger a save afterwards, and does not exercise the conflict alert.177178### Recovery after a kill179180`Scripts/recovery-test.sh` checks that work survives a process that gets no chance to save:1811821. launch the built app with `-ORIXTemplate blank -ORIXSaveTo <file>` so the application itself creates a saved document, and record the file's SHA-256;1832. launch it again with `-ORIXOpen <file> -ORIXAutosaveDelay 2 -ORIXType "Unsaved but recoverable 42"`: the text is typed through real key events (`SelfTest.type`), with no explicit save;1843. wait (9 seconds by default, `ORIX_RECOVERY_WAIT`), then `kill -9` the process;1854. launch a third time with `-ORIXOpen <file> -ORIXDumpText <out>` and compare the text read back.186187It prints whether the file changed on disk before the kill, the text found, and `PASS` or `FAIL`.188189What it proves when it passes: with autosave in place, text typed into a saved document is on disk within the autosave delay, in a file that opens normally, without any save command or orderly quit.190191What it does not prove: anything about an **untitled** document that was never saved (AppKit's draft autosave and crash restoration; the script disables state restoration with `-ApplePersistenceIgnoreState YES`); the fate of text typed less than the autosave delay before the kill; a kill in the middle of a write; power loss; or the versions browser. `Scripts/test.sh` runs the script and copies its last line to `docs/reports/test-results.json` (key `recovery`), which is the only record of a run in the repository. The XCUITest `testAutosaveSurvivesAKilledProcess` (`UITests/EditorUITests.swift`) describes the same scenario; see [testing.md](../development/testing.md) for why that suite was not run.192