SPB Git

Repository Public

NATIVE-FORMAT.md 10.9 KB 207 lines markdown

1# The `.orixsheet` document format — version 123`.orixsheet` is the native, lossless format of ORIX Sheet. A document is a **package**: a directory4that macOS presents as a single file. The application reads and writes it by URL5(`NSDocument.read(from:ofType:)` / `write(to:ofType:)` calling `WorkbookReader.load` and6`WorkbookWriter.save`); a `FileWrapper` API exists as well and is tested, but the app does not use it.7Everything the SheetCore model holds is stored; nothing is recomputed on load.89Implementation: `Sources/SheetIO/Native/` (`NativeFormat`, `NativeSchema`, `NativeCells`).1011```12Budget.orixsheet/13├── manifest.json            format id, versions, dates, SHA-256 and size of every other part14├── workbook.json            sheet list, defined names, style table, properties, attachment index15├── sheets/16│   ├── 1.cells              cells of the sheet whose id is 1 (binary, chunked, zlib)17│   ├── 1.meta.json          everything in SheetMeta for that sheet18│   ├── 4.cells19│   └── 4.meta.json20├── attachments/21│   └── 0-clean.py           Workbook.attachments (Python scripts, workflows, queries…)22├── preserved.plist          optional: parts kept from an imported XLSX (charts, VBA, pivots…)23└── preview.png              optional: thumbnail passed in `NativeWriteOptions.preview` (the app passes none)24```2526Sheet parts are named after `Sheet.id`, which is stable across renames and reordering. The order of27the sheets is the order of `workbook.json › sheets`.2829## API3031```swift32NativeFormat.write(workbook, to: url, options:, preserved:)      // atomic, returns [CompatibilityIssue]33NativeFormat.read(url:, options:, progress:) -> LoadResult34NativeFormat.fileWrappers(for: workbook, options:, preserved:) -> FileWrapper   // not used by the app35NativeFormat.read(fileWrapper:, options:, progress:) -> LoadResult             // not used by the app36NativeFormat.verify(url:) -> [String]                           // paths failing the integrity check37```3839`WorkbookReader.load` / `WorkbookWriter.save` with `.orixsheet` call the same code; this is the path40the application takes (`App/Sources/WorkbookDocument.swift`). The two `FileWrapper` functions build41and read the same parts in memory for a host that prefers `NSDocument.fileWrapper(ofType:)`; they are42covered by `NativeFormatTests.fileWrappersPreviewAttachmentsAndPreservedParts` only.4344`preview.png` is written when the caller sets `NativeWriteOptions.preview`. The application does not45set it, so documents saved by ORIX Sheet contain no preview; a reader must not expect one.4647## manifest.json4849```json50{51  "format": "orixsheet",52  "formatVersion": 1,53  "minReaderVersion": 1,54  "app": "ORIX Sheet",55  "appVersion": "1.0",56  "created": 721692800.0,57  "modified": 785000000.5,58  "parts": {59    "workbook.json":      { "sha256": "9f2c…", "size": 1830 },60    "sheets/1.cells":     { "sha256": "41aa…", "size": 5242880 },61    "sheets/1.meta.json": { "sha256": "c07e…", "size": 912 }62  }63}64```6566* `parts` lists every file of the package except the manifest itself, with its SHA-256 (lower-case67  hex) and byte size.68* Dates everywhere in the JSON parts are **seconds since 2001-01-01 00:00:00 UTC as a double**69  (`Date.timeIntervalSinceReferenceDate`). This is the only encoding that round-trips a Swift `Date`70  bit for bit; ISO 8601 would lose sub-millisecond precision.71* `created` is `Workbook.properties.created` when set, otherwise the time of the first save.7273## workbook.json7475| Key | Content |76|---|---|77| `sheets` | array, in tab order: `id`, `name`, `cells` (part path), `meta` (part path), `cellCount` |78| `names` | defined names: `key` (dictionary key in `Workbook.names`), `name`, `formula`, `scopeSheetID`, `comment`, `hidden` |79| `styles` | the `StyleTable` in index order; cells refer to styles by index, index 0 is the default style |80| `date1904`, `activeSheet`, `defaultFontName`, `defaultFontSize` | workbook settings |81| `properties` | `WorkbookProperties` (title, author, subject, keywords, comments, created, modified) |82| `attachments` | `name` (key in `Workbook.attachments`) → `path` inside the package |83| `hasPreservedParts` | present and true when `preserved.plist` exists |8485Attachment names are arbitrary strings; the file name inside `attachments/` is derived from the86index and a sanitised suffix (`[A-Za-z0-9._-]`), so a name can never escape the package.8788## sheets/&lt;id&gt;.meta.json8990A direct image of `SheetMeta`: `colWidths`, `rowHeights` (points, keyed by index), default sizes,91`hiddenRows` / `hiddenCols` / `filteredRows` / collapsed sets (index ranges), outline levels,92row/column styles, `merges`, frozen panes, `tables`, `conditionalFormats`, `validations`,93`comments` and `hyperlinks` (arrays of `{row, col, value}`), `autoFilter`, `objects`94(`SheetObject`, payload as base64), `print`, view flags, `tabColor`, `isHidden`, `isProtected`.9596Number values are written with the shortest representation that reads back to the same double.97Non-finite numbers are written as the strings `"+inf"`, `"-inf"`, `"nan"`.9899## sheets/&lt;id&gt;.cells100101Binary, little-endian. Designed for millions of cells: column-major order (the order of the in-memory102store), independent zlib-compressed chunks of 50,000 cells, per-chunk string table.103104```105header   4 bytes  "OXCL"106         u32      version (1)107         u64      number of cells108         u32      number of chunks109chunk    u32      raw (uncompressed) length110         u32      compressed length111         u32      number of cells in the chunk112         u32      CRC-32 of the raw bytes113         bytes    zlib stream (RFC 1950)114```115116Raw chunk content (`varint` = LEB128 unsigned, `zigzag` = zigzag-encoded signed varint):117118```119varint stringCount, then stringCount × (varint byteLength, UTF-8 bytes)120then, for every cell:121  varint  column delta from the previous cell (the first cell of a chunk: absolute column)122  varint  row: delta from the previous cell when the column is unchanged, absolute otherwise123  u8      flags: bits 0–3 value kind, 0x10 has formula, 0x20 isSpill, 0x40 has style124  zigzag  style index                              (when 0x40)125  …       value payload (see below)126  when 0x10:127    u8      formula flags: 1 = spill range present, 2 = volatile128    varint  string index of the formula text (canonical text, no leading "=")129    u8      kind of the cached value, then its payload130    4 × varint  r0, c0, r1, c1 of the spill range  (when flag 1)131```132133Value kinds and payloads:134135| Kind | Meaning | Payload |136|---|---|---|137| 0 | blank | – |138| 1 | number | 8 bytes, IEEE-754 double (keeps −0.0, subnormals, NaN payloads) |139| 2 | integral number | zigzag (exact integers below 9·10¹⁵) |140| 3 | text | varint string index |141| 4 / 5 | FALSE / TRUE | – |142| 6 | error | u8: Excel `ERROR.TYPE` code (1 #NULL!, 2 #DIV/0!, 3 #VALUE!, 4 #REF!, 5 #NAME?, 6 #NUM!, 7 #N/A, 9 #SPILL!, 14 #CALC!, 15 #CIRC!) |143| 7 | array | varint rows, varint cols, then rows × cols × (u8 kind, payload) |144| 8 | LAMBDA value | – (cannot be stored; read back as #CALC!, reported as a loss on save) |145146Formulas are stored as **canonical formula text** (`Expr.formulaText()`, the OOXML en-US syntax) and147parsed again on load; identical texts inside a chunk are parsed once. The cached value, the spill148range and the volatile flag are stored next to it so a document opens without recalculation149(`CalcEngine(workbook:trustCachedValues: true)`).150151## preserved.plist152153Binary property list of `PreservedParts`: raw bytes of the XLSX parts ORIX Sheet does not model154(charts, drawings, images, pivot tables and caches, VBA project, external links, custom XML, theme)155with the sheet ids they belong to. It lets an imported Excel workbook be saved natively and exported156to XLSX later without losing those parts. Its content is never interpreted or executed.157158## Integrity and recovery159160On load every part listed in the manifest is hashed and compared.161162| Situation | Default behaviour | `LoadOptions.strictIntegrity` |163|---|---|---|164| `workbook.json` missing or undecodable | `SheetIOError.corrupted(part: "workbook.json", …)` | same |165| `workbook.json` hash mismatch but decodable | loads, warning | error |166| a `.cells` part: bad chunk (CRC, zlib, structure) | that chunk is skipped, the other chunks and sheets load; `loss` issue naming the sheet, the part and the chunk | `corrupted(part: "sheets/4.cells", …)` |167| a `.cells` part missing or truncated | the sheet opens with what could be read (possibly empty); `loss` issue | error |168| a `.meta.json` part missing or undecodable | cells load, layout of that sheet is reset to defaults; `loss` issue | error |169| attachment missing / changed | `loss` / `warning` issue | error |170| manifest missing or unreadable | document opens without verification; `warning` issue | error |171172Issues are returned in `LoadResult.issues` with `location` set to the part path. The reader never173trusts sizes found in the file: chunk lengths are bounded (256 MiB raw), decompression stops at the174declared size, every index is range-checked, and `LoadOptions.maxCells` caps the total.175176## Atomic save177178`NativeFormat.write` builds the complete package in a hidden sibling directory179(`.<name>.saving-<uuid>`), then swaps it in with `FileManager.replaceItemAt` (or a rename when the180destination does not exist). If anything fails before the swap — disk full, crash, power cut — the181previous document is untouched and the temporary directory is removed. A successful save replaces182the whole package, so parts of deleted sheets never linger.183184## Versioning185186* `formatVersion` is the schema version the writer used; `minReaderVersion` is the oldest reader187  that can open the file without misreading it.188* Reader rules: `formatVersion` ≤ ours → migrate if older, then read. `formatVersion` > ours and189  `minReaderVersion` ≤ ours → read, ignore what is unknown, warn that unknown content would be lost190  on save. `minReaderVersion` > ours → `SheetIOError.unsupportedVersion`.191* **Forward compatibility**: unknown JSON keys are ignored; unknown files in the package are192  ignored; lists whose elements may gain new variants (`tables`, `conditionalFormats`,193  `validations`, `objects`) are decoded element by element, so one unknown element is skipped and194  reported instead of making the sheet unreadable.195* **Migration hook**: `NativeFormat.migrations[v]` upgrades the parts of a version-`v` package to196  `v + 1` before decoding; `migrate(_:from:)` chains the steps. Version 1 is the first released197  schema, so the table is empty. A future version that changes a part bumps `formatVersion`,198  registers a step, and raises `minReaderVersion` only when old readers would misread the file.199* The `.cells` part has its own version field (1) and magic number, checked independently.200201## What is not stored202203* LAMBDA values held by a cell (closures): saved as `#CALC!`, recomputed on load, reported as a204  `loss` issue by `write` and `preflight`.205* Engine state that is derived from the model (dependency graph, dirty flags) is rebuilt.206* The private sheet-id counter of `Workbook` is restored to "largest id + 1".207
Files main
Menu

Theme