SP Notes Core
The shared, UI-free Swift core of the native SP Notes apps for iOS, iPadOS and macOS — a lossless ProseMirror document model, a durable local SQLite store, a typed API client with passkeys, and a synchronisation engine whose first rule is that no local modification is ever lost.
Author: Simon-Pierre Boucher
Table of contents
- Overview
- Features
- Architecture
- Stack
- Repository layout
- Getting started
- Configuration
- Module: SPNotesDocument
- Module: SPNotesStore
- Module: SPNotesAPI
- Module: SPNotesSync
- Local data model
- Security and privacy
- Testing
- Troubleshooting and gotchas
- Known limitations
- Documentation
- Author
- License
🎯 Overview
SPNotesCore is a Swift package with no user interface and no third-party dependency. It contains everything the
two native SP Notes apps have in common: how a note is represented, where it is stored on the device, how the app talks
to the backend, and how the device and the server are kept in step.
The backend is note-spboucher-ai (production: https://note.spboucher.ai), which stores a note as ProseMirror / Tiptap JSON and exposes a change feed for native clients. The core must stay byte-compatible with it: plain text, word count and Markdown export reproduce the server's own helpers, and every document the core emits is validated against the real web schema.
Two rules run through the whole package:
- No note is ever lost (store): local edits are never overwritten by the feed; every mutation is one atomic SQLite transaction; nothing waits for the network.
- No local modification is lost or silently overwritten (sync): concurrent edits are merged when that is safe; otherwise both versions are kept and the user decides.
Family
| Repository | Role | Platform |
|---|---|---|
| note-spboucher-ai | Web app, API, database, sync feed | Web / PWA — note.spboucher.ai |
| sp-notes-core (this one) | Shared Swift package | iOS 18+ · macOS 15+ |
| sp-notes-macos | SP Notes for Mac | macOS |
| sp-notes-ios | SP Notes for iPhone and iPad | iOS / iPadOS |
Both apps expect this repository to be cloned next to them as SPNotesCore (they reference ../SPNotesCore).
✨ Features
| Module | What it gives |
|---|---|
| SPNotesDocument | ProseMirror JSON ⇄ a flat list of lines an editor can drive, without losing anything (unknown nodes included); editing commands (blocks, lists, marks, links, tables); plain text, word count, preview, Markdown import/export, HTML export; three-way merge by block id; schema validation |
| SPNotesStore | Durable local database on the system SQLite: pages, tags, attachments, ops queue, conflicts, local versions, reminders, meta; FTS5 search that ignores case and diacritics; change notification streams; multi-process access (app, share extension, read-only widget) |
| SPNotesAPI | The only module that talks to the backend: hardened URLSession transport, hand-written cookies, typed errors, one method per route, multipart uploads, downloads, server-sent events for the AI routes, passkey ceremonies through AuthenticationServices, Keychain token store |
| SPNotesSync | SyncEngine actor: push then pull in bounded rounds, idempotent retries, lost-acknowledgement recovery, three-way merge, conflict records, trash/purge/server-restore handling, background downloads of attachments |
| SPNotesCore | Umbrella module: one import SPNotesCore re-exports the four others |
🏗 Architecture
flowchart TB
subgraph APPS["Native apps (own the UI and the attachment directory)"]
MAC["SP Notes macOS<br/>AppKit · TextKit 2"]
IOS["SP Notes iOS / iPadOS<br/>SwiftUI · UIKit · TextKit 2"]
end
subgraph CORE["SPNotesCore (this package)"]
SYNC["SPNotesSync<br/>SyncEngine (actor)"]
DOC["SPNotesDocument<br/>Foundation only · value types"]
STORE["SPNotesStore<br/>NotesStore (actor) · SQLite WAL · FTS5"]
API["SPNotesAPI<br/>HTTPClient (actor) · NotesAPI · passkeys"]
end
BACK["note-spboucher-ai<br/>https://note.spboucher.ai"]
DB[("SQLite file<br/>on the device")]
KC["Keychain<br/>session token"]
MAC --> CORE
IOS --> CORE
SYNC --> DOC
SYNC --> STORE
SYNC --> API
STORE --> DB
API --> KC
API -- "HTTPS · cookie written by hand" --> BACK
The three lower modules are independent of one another:
SPNotesStoredoes not importSPNotesDocument: note content is opaque JSONData, and the caller supplies the derived plain text, word count and preview.SPNotesAPIdepends on Foundation, Security and AuthenticationServices only: notecontentandpropertiestravel asRawJSONand are interpreted bySPNotesDocument.- Only
SPNotesSyncdepends on the three others.
One sync cycle
sequenceDiagram
autonumber
participant A as App
participant E as SyncEngine
participant S as NotesStore
participant B as Backend
A->>S: local mutation (one SQLite transaction)
A->>E: schedule() — debounced, 2 s by default
loop up to 4 rounds
E->>B: POST /api/pages (creations, parents first, client-chosen id)
E->>B: POST /api/attachments (uploads)
E->>B: PATCH with baseVersion (title · content · properties)
alt 409 conflict
E->>S: reconcile: adopt · move base · three-way merge · or record a conflict
end
E->>B: queued actions with their idempotency key, then reminders
E->>B: GET /api/sync from the stored cursor, until more is false
E->>S: apply each answer in one transaction with its cursor
end
E->>B: download up to 40 wanted files
E-->>A: statusUpdates() — phase, pending, conflicts, needsReauth
🧱 Stack
| Layer | Technology | Version | Role |
|---|---|---|---|
| Language | Swift, language mode 6 on every target | tools 6.0 | Strict concurrency: actors, Sendable value types |
| Package | Swift Package Manager | — | 4 libraries + the umbrella; defaultLocalization: "fr" |
| Platforms | iOS, macOS | 18 · 15 (minimum) | Deployment targets of the package |
| Storage | system SQLite (linkedLibrary("sqlite3")) |
— | WAL, synchronous=FULL, FTS5 |
| Network | Foundation URLSession (ephemeral) |
— | TLS ≥ 1.2, no cookie storage, no cache, redirects refused |
| Sign-in | AuthenticationServices | — | Platform passkeys |
| Secrets | Security (Keychain) | — | Session token, this device only, not synchronised |
| Tests | Swift Testing (@Test) and XCTest |
— | 301 + 99 tests |
| Third-party code | none | — | — |
🗂 Repository layout
87 tracked files.
sp-notes-core/ # clone it as SPNotesCore next to the apps
├── Package.swift # products, SPN_ONLY switch
├── Sources/
│ ├── SPNotesDocument/ # 14 files · 3 467 lines
│ │ ├── Model.swift JSONValue.swift Schema.swift SchemaValidation.swift BlockIds.swift
│ │ ├── Commands.swift InlineCommands.swift Inline.swift Tables.swift
│ │ ├── Text.swift MarkdownImport.swift MarkdownExport.swift HTMLExport.swift
│ │ └── Merge.swift
│ ├── SPNotesStore/ # 9 files · 2 966 lines
│ │ ├── NotesStore.swift SQLite.swift Schema.swift Models.swift
│ │ └── NotesStore+Pages / +Queries / +Sync / +Queue / +History .swift
│ ├── SPNotesAPI/ # 14 files · 3 479 lines
│ │ ├── APIConfiguration.swift HTTPClient.swift APIError.swift JSONSupport.swift
│ │ ├── NotesAPI.swift PageModels.swift ServiceModels.swift AIModels.swift
│ │ ├── MultipartForm.swift SSEParser.swift
│ │ └── AuthService.swift PasskeyCeremony.swift PasskeyAuthorizer.swift SessionTokenStore.swift
│ ├── SPNotesSync/ # 1 file · 1 014 lines
│ │ └── SyncEngine.swift
│ └── SPNotesCore/ # umbrella: @_exported import of the four modules
├── Tests/
│ ├── SPNotesDocumentTests/ # 5 suites + Fixtures/ (6 JSON files)
│ ├── SPNotesStoreTests/ # 5 suites
│ ├── SPNotesAPITests/ # 6 suites (URLProtocol stub + live tests)
│ └── SPNotesSyncTests/ # 6 suites + FakeBackend.swift (in-memory backend, 1 072 lines)
└── docs/
├── DOCUMENT_API.md STORE_API.md API_CLIENT.md SYNC_ENGINE.md
├── reference/ # read-only copies of the web sources the core must match + BACKEND_CONTRACT.md
└── brand/🚀 Getting started
Build and test
swift build
swift testOne module in isolation (parallel development — each with its own scratch directory):
SPN_ONLY=Document swift test --scratch-path .build-document
SPN_ONLY=Store swift test --scratch-path .build-store
SPN_ONLY=API swift build --scratch-path .build-api && SPN_ONLY=API swift test --scratch-path .build-api
SPN_ONLY=Sync swift test --scratch-path .build-syncSPN_ONLY=Sync also declares the three modules it depends on. The umbrella product SPNotesCore only exists when
SPN_ONLY is unset.
Use it from an app
Clone this repository next to the app as SPNotesCore and reference it as a local package:
.package(path: "../SPNotesCore")
// then depend on the product "SPNotesCore" (everything) or on a single moduleimport SPNotesCore
let configuration = APIConfiguration.production(appVersion: "1.0.0") // https://note.spboucher.ai
let tokens = KeychainTokenStore(configuration: configuration, accessGroup: "<team>.<keychain group>")
let client = HTTPClient(configuration: configuration, tokenStore: tokens)
let store = try NotesStore(url: databaseURL)
let engine = SyncEngine(store: store, api: NotesAPI(client: client), files: locator, options: SyncOptions())
var doc = try NoteDocument(json: data) // ProseMirror JSON → lines
doc.toggleMark(.bold, in: 0..<5, at: 2) // edit
doc.ensureBlockIds() // before saving: merges match blocks by id
let body = try doc.jsonData() // lines → canonical JSON (sorted keys)
await engine.schedule() // after every local mutation
await engine.sync() // app start, foreground, network back, pull-to-refresh⚙️ Configuration
There is no configuration file. Behaviour is set by types and by a few environment variables used at build or test time.
| Name | Where | Purpose |
|---|---|---|
SPN_ONLY |
Package.swift |
Document, Store, API or Sync: build and test a single module |
SPN_LIVE_URL, SPN_LIVE_TOKEN |
API and sync tests | enable the live tests against the test server; any *.spboucher.ai host is refused |
SPN_LIVE_TOKEN_DISPOSABLE, SPN_LIVE_TOKEN_DISPOSABLE2 |
API tests | optional: two more tokens (one is revoked, one logs out) |
SPN_RANDOM_SEEDS, SPN_RANDOM_ONLY |
sync tests | extra seeds for the randomised convergence test, or a single seed to replay |
SPN_DUMP |
document tests | enables the dump test |
| Type | Settings |
|---|---|
APIConfiguration |
baseURL, userAgent, requestTimeout, resourceTimeout, streamIdleTimeout, sessionCookieName, ceremonyCookieName; origin and relyingPartyID are derived from the base URL |
SyncOptions |
autoDownloadLimit (25 MiB), includeDatabaseRows (false), feedPageSize (200) |
KeychainTokenStore |
service, account (defaults to the origin, so a test server never shares a token with production), accessGroup, useDataProtectionKeychain |
📝 Module: SPNotesDocument
Foundation only, value types, all Sendable and Hashable. Full reference: docs/DOCUMENT_API.md.
| Area | API (excerpt) |
|---|---|
| Model | JSONValue, PMNode, PMMark, NoteDocument (lines), Line (displayString, inlineLength) |
| Reading / writing | NoteDocument(json:), jsonData() (canonical JSON, sorted keys), ensureBlockIds(), isStructurallyEqual(to:) |
| Commands | public mutating func on NoteDocument; they return whether the document changed, never trap on out-of-range input, and leave a document valid for the web schema |
| Marks | MarkKind: bold, italic, underline, strike, code, highlight, subscript, superscript; links through setLink / removeLink |
| Lists | ListKind: bullet, ordered, task |
| Text | plainText(), wordCount, preview(maxLength:), pageReferences(), attachmentReferences(), taskItems() |
| Export / import | markdown(), html(title:), htmlBody(), Markdown import |
| Merge | NoteDocument.merge(base:mine:theirs:) → .merged(doc) or .conflict |
| Validation | schemaViolations(strict:) |
The 30 node types of the web schema known to the core
doc, text, paragraph, heading, codeBlock, footnote, detailsSummary, bulletList, orderedList,
taskList, listItem, taskItem, blockquote, callout, aiBlock, details, detailsContent, table,
tableRow, tableHeader, tableCell, image, fileBlock, blockMath, horizontalRule, youtube, hardBreak,
mention, inlineMath, footnoteRef.
Unknown nodes are kept as they are and written back unchanged.
Conventions worth knowing before writing an editor on top of it:
- Inline offsets are UTF-16 code units (
NSString/NSRangecompatible). An inline atom (hard break, mention, inline math, footnote reference, unknown inline node) counts as exactly 1 unit: display it as oneU+FFFCattachment character. - New nodes are created with every schema attribute (defaults included) and a fresh block id, exactly as the web editor serializes them.
plainText,wordCountandmarkdownreproduce the server byte for byte (checked on the fixtures and on 400 random trees against outputs recorded from the web application's own helpers).- Three-way merge: top-level blocks are matched by block id. A block changed, inserted, deleted or moved on one side only is taken from that side; the merge never drops a change silently. A top-level block is the whole node (an entire list, quote or table): two edits inside the same list conflict.
- HTML export uses the same markup as the server's print view, without KaTeX — math is exported as its LaTeX source.
🗄 Module: SPNotesStore
The durable local database. Full reference: docs/STORE_API.md.
public actor NotesStore {
public init(url: URL, readOnly: Bool = false) throws
public static func inMemory() throws -> NotesStore
public func integrityCheck() throws -> [String] // empty = healthy
public func checkpoint() throws // wal_checkpoint(TRUNCATE)
public func rebuildSearchIndex() throws
public func wipeAll() throws // "erase this device" only
}| Aspect | Behaviour |
|---|---|
| Connection | SQLITE_OPEN_FULLMUTEX, journal_mode=WAL, synchronous=FULL, foreign_keys=ON, busy_timeout 8 s |
| Transactions | every mutation is ONE BEGIN IMMEDIATE transaction; an error rolls it back entirely (index, tags, queue and cursor included) and publishes no change event |
| Multi-process | the same file may be opened by the app (read/write), a share extension (read/write) and a widget (readOnly: true); a read-only store never creates nor migrates the file |
| Schema | PRAGMA user_version, one transaction per migration step; a file newer than the build is refused (StoreError.schema) |
| Ids and dates | lowercase UUID strings; timestamps stored as integer milliseconds since the Unix epoch |
| Dirty state | needsCreate, dirtyContent, dirtyMeta (parent, section, position, pinned, archived, icon, cover, inbox), tagsDirty, editSeq (+1 on every local change to push) |
| Base copy | PageBase { title, content, properties, version }: the last synced copy of the version-guarded fields, captured at the first content edit and dropped when the content is clean again |
| Sync state | conflict · error · pending · synced |
| Search | FTS5 (title, body, tags), tokenizer unicode61 remove_diacritics 2; every term must match, the last one as a prefix; terms are quoted so no input can cause a syntax error; bm25 weights title 10, tags 4, body 1; incremental index |
| Ops queue | trash, restore, purge, deleteAttachment, createTask, updateTask, deleteTask, triage; leased for 120 s; per-target order never broken; backoff 5 s, 10 s, 20 s … capped at 15 min; one idempotency key per op, never changed across retries |
| Change notification | changes() -> AsyncStream<StoreChange>: pages, tags, attachments, queue, conflicts, reminders, meta, reset; emitted after the commit; noteExternalChange() when another process wrote the file |
| Errors | StoreError: sqlite, notFound, invalid, cycle, readOnly, closed, schema |
🌐 Module: SPNotesAPI
The only module that talks to the backend. Full reference: docs/API_CLIENT.md; contract:
docs/reference/BACKEND_CONTRACT.md.
| Type | Role |
|---|---|
APIConfiguration |
base URL, Origin, User-Agent, timeouts, cookie names |
SessionTokenStore, KeychainTokenStore, InMemoryTokenStore |
where the session token lives |
HTTPClient (actor) |
transport: cookies by hand, headers, uploads, downloads, event streams, session events |
APIError |
every failure, typed |
NotesAPI |
one method per route |
PasskeyCeremony, PasskeyAuthorizing, SystemPasskeyAuthorizer, AuthService |
passkeys and sign-in flows |
SSEParser, AIEvent |
server-sent events of the AI routes |
Transport rules implemented by HTTPClient:
- One dedicated ephemeral
URLSession: no cookie storage, no URL cache, no credential storage, TLS ≥ 1.2. Redirects are refused (the cookie never follows one). Cookie: __Host-note_session=<token>is written by hand from the token store; aSet-Cookieon any answer (sign-in, step-up rotation) replaces the token and saves it.- A cleared cookie, or a
401answering a request that carried the current token, clears the store and emitsSessionEvent.signedOutexactly once (a late 401 for a token that has since been rotated is ignored). - The WebAuthn ceremony cookie is captured from the
optionscall and sent only on the matchingverifycall, once. OriginandIdempotency-Keyonly on POST / PATCH / PUT / DELETE;x-request-idon every request.- JSON bodies are encoded with sorted keys (same request → same bytes); answers are decoded with ISO 8601 dates with or
without fractional seconds, and the PostgreSQL text form that
/api/searchreturns. - User-Agent
SPNotes/<x.y.z> (iPhone|iPad|Macintosh; <os>) native→ the server labels the session "SP Notes · iPhone|iPad|Mac".
Routes covered by NotesAPI:
| Group | Routes |
|---|---|
| Change feed | GET /api/sync?since=&limit= |
| Pages | GET/POST /api/pages, GET/PATCH/DELETE/POST /api/pages/<id>, resolve, versions, tags |
| Attachments | POST/GET/DELETE /api/attachments, GET /api/files/<id> |
| Authentication | login, reauth (step-up), passkeys, sessions, logout, state |
| AI | chat and inline actions as server-sent events (data: <json>, terminator data: [DONE]) |
| Other | search, tags, tasks (reminders), trash, inbox and triage, capture, health |
🔄 Module: SPNotesSync
SyncEngine is an actor that keeps one NotesStore in step with the backend through NotesAPI. Full reference:
docs/SYNC_ENGINE.md.
What the apps call, and when
| Moment | Call |
|---|---|
| after every local mutation of the store (edit, tag, pin, move, trash, attachment, reminder…) | await engine.schedule() (debounced, 2 s by default) and, for the badge, await engine.refreshCounts() |
| app start, return to foreground, network back, pull-to-refresh, push notification, after sign-in | await engine.sync() |
after a successful passkey step-up, when status.needsReauth was true |
await engine.sync() (the queued permanent deletions go at once) |
| the user opens a file that is not on the device | try await engine.download(attachmentId:) |
| the user settles a conflict | try await engine.resolveConflict(pageId:choice:) |
| sign-out, app about to be suspended for good | await engine.stop() |
| UI state (spinner, offline banner, counters) | for await status in await engine.statusUpdates() |
sync() returns .completed, .offline, .signedOut or .failed(message); nothing is lost in any of these cases.
It is re-entrant: a call made while a cycle runs waits for the next cycle, so its outcome always covers the state of
the store at the time of the call.
Guarantees
- A local edit is acknowledged only for the exact state that was sent (
editSeq): what is typed while a request is in flight stays dirty and goes with the next round or cycle. - Local text is replaced in exactly three cases, and each time the replaced text is first saved as a local version: an automatic merge, the user choosing "theirs" or "both", a server copy identical to the local one.
- The server's text is never overwritten without
baseVersion, and only on top of a copy the local text derives from. - A page purged on the server while this device holds unsynced text is created again under the same id.
- Nothing is duplicated by retries (page, file, task, conflict copy).
- Killing the app at any point loses nothing: all state is in the store.
Idempotency
| Request | What makes a retry safe |
|---|---|
| create page, upload attachment | the client-chosen id (200 + x-existing) — deliberately no idempotency key |
guarded PATCH |
baseVersion; a retry gets 409 with a server copy equal to the local one → adopted |
unguarded PATCH, PUT tags |
idempotent by nature |
| trash, restore, delete attachment, triage | the op's own key (one key per op, never reused for another payload) |
| create task | spn-task-<reminder id> |
| resolve conflict | spn-res-<page>-<server version>-<choice>-<fingerprint of the local text> |
Conflict policy
Reconciliation happens when a guarded write is refused (409) or when the feed brings a newer version of a note with
unsynced text:
- server copy = local copy → adopted, nothing to push;
- server copy = what this device sent before a lost answer → base moved, local text pushed next;
- server version lower than the local base (restored server) → the local text is sent again on top, with a snapshot;
- three-way merge with the stored base copy: title (one side changed), content (top-level blocks by id), properties (key by key);
- otherwise a conflict is recorded: the local text is untouched and stays editable, the server copy is stored beside it, the note leaves the push list until the user decides.
| Choice | Note on the server | Where the other text survives |
|---|---|---|
.keepMine |
local title and content | server history |
.keepTheirs |
unchanged | server history (not applied) and local versions |
.keepBoth |
unchanged; the local text becomes a sibling page "… (conflict copy)" | both are notes |
Flags, tags, moves, icon, cover and position are not versioned: last writer wins, they never conflict.
Error classification, trash, purge and server restore
- Stops the cycle: 401 (
.signedOut); no connectivity, connection failures, timeouts (.offline); 429, the third 5xx of the cycle, a feed that breaks the contract, a closed store (.failed); cancellation (.offline). - Concerns one item: everything else. A page gets
syncError; an op gets the store's backoff; an upload refused with a 4xx is not sent again before a backoff. 403 reauth_requiredon a purge is not a failure: the op stays first in line without counting an attempt andneedsReauthstays true.- Loops are bounded: 4 rounds per cycle; the feed must advance, never go backwards, and may ask for a reset twice at most.
- Trash is not a conflict: if a note is trashed elsewhere while this device edits it, the edit stays pending; restoring the note lets the edit through.
- Purge on the server: a clean page disappears; a page with unsynced text is created again with the same id.
- Purge on the device: queued; needs a fresh passkey verification.
resetfeed (the server database was restored): everything is downloaded again; dirty notes are kept and pushed; synced notes the restored server no longer has are created again on it. Nothing is deleted on the device in this mode.
🗃 Local data model
Schema version 1 (Sources/SPNotesStore/Schema.swift): 10 tables and one FTS5
virtual table.
| Table | Content |
|---|---|
pages |
the notes: server fields, opaque content and properties, dirty flags, edit_seq, sync state, plain text and preview |
page_bases |
last synced copy of the version-guarded fields of a page being edited |
tags, page_tags |
tag names and their pages |
pages_fts |
FTS5 index (title, body, tags) |
attachments |
file metadata, upload state, local path, last access |
ops |
queue of actions that are not dirty state, with their idempotency key and backoff |
conflicts |
the server copy stored beside a note in conflict |
local_versions |
local history, including every text replaced by a merge or a resolution |
reminders |
reminders (pushed to the server as tasks) |
meta |
key/value: feed cursor, sync.sent.*, sync.fullCopyPending… |
Logical relations (most tables are keyed by the page id rather than by a declared foreign key):
erDiagram
pages ||--o| page_bases : "base copy"
pages ||--o{ page_tags : tagged
pages ||--o{ attachments : holds
pages ||--o| conflicts : "server copy"
pages ||--o{ local_versions : history
pages ||--o{ reminders : has
pages ||--|| pages_fts : indexed
Data conventions shared by the native apps: favourite = properties.favorite === true; folder = a page with
properties.folder === true; reminder = a task { pageId, dueAt }; a PencilKit drawing = an image node whose alt
is drawing-<uuid>.png plus a companion attachment drawing-<uuid>.pkdrawing on the same page (editable source).
🔒 Security and privacy
- The session token lives in the Keychain (
kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly, not synchronised). It is never logged;APIErrordescriptions never contain request data. - The Keychain account defaults to the origin, so a test server never shares a token with production.
- The
URLSessionis ephemeral, stores no cookie, no cache and no credential, requires TLS ≥ 1.2 and refuses redirects. - Mutating requests carry the exact
Originthe backend expects; permanent deletions need a passkey verification younger than 5 minutes (reauth_required). - Passkeys go through the system (
AuthenticationServices);PasskeyCeremonyonly does the JSON work. - Link and image URLs of the HTML export go through the same allow-list as the server;
javascript:links are dropped by the Markdown import. - The live tests refuse any
*.spboucher.aihost and clean up what they create.
🧪 Testing
400 tests (2.0.0, branch next): 301 with Swift Testing (@Test) and 99 with XCTest (the API module; the 11 live tests are
skipped without SPN_LIVE_URL). Per module: Document 131, Store 82, API 99, Sync 67, Design 21. The table below lists the 1.0
suites; 2.0 adds NextNodeExportTests (9, one of them over the 4 fixtures published by the backend) and NextNodeModelTests (13), InkTests (12), CanvasModelTests (4),
CanvasMergeTests (8), CanvasLayoutTests (4) in Document; MigrationV2Tests (17), EntityTests (15) and the fixture generator
(1, inert without SPN_MAKE_V1_FIXTURE) in Store; RetryTests (11) in API; NextNodesSyncTests (1) and one more queue test in
Sync; SPNotesDesignTests (21). What 2.0 changes is described in CHANGELOG.md, docs/MIGRATION-V2.md
and docs/DESIGN.md.
| Module | Suite | Tests |
|---|---|---|
| Document (81) | CommandTests |
35 |
ModelTests |
22 | |
TextTests |
16 | |
MergeTests |
7 | |
DumpTests |
1 | |
| Store (49) | SyncTests |
15 |
PageTests |
12 | |
EngineTests |
11 | |
SearchTests |
6 | |
QueueHistoryTests |
5 | |
| API (88) | HTTPClientTests |
22 |
ErrorAndModelTests |
17 | |
PasskeyAndAuthTests |
15 | |
RoutesAndMultipartTests |
12 | |
SSEAndAITests |
11 | |
LiveBackendTests (only with SPN_LIVE_URL) |
11 | |
| Sync (65) | FaultTests |
26 |
MergeConflictTests |
16 | |
TrashAttachmentQueueTests |
14 | |
SyncEngineTests |
7 | |
RandomConvergenceTests (seeded) |
1 | |
LiveSmokeTests (only with the live variables) |
1 |
- Document: fixtures recorded from the web application (
Tests/SPNotesDocumentTests/Fixtures);swift testwrites every document the module emits to/tmp/spn-docs.jsonso they can be validated against the real web schema. - API: a
URLProtocolstub (Tests/SPNotesAPITests/TestSupport.swift). - Sync: an in-memory fake of the backend written from the server's route handlers
(
Tests/SPNotesSyncTests/FakeBackend.swift), real store files, 2–3 devices, fault injection, and a seeded randomised convergence test.
Live tests against the test server (never production):
ssh -f -N -L 8482:localhost:8482 M4M64c
TOKEN=$(ssh M4M64c 'bash -lc "cd ~/dev/note-native && set -a && . ./.env && set +a && node_modules/.bin/tsx scripts/dev-session.ts"')
SPN_LIVE_URL=http://localhost:8482 SPN_LIVE_TOKEN=$TOKEN SPN_ONLY=API swift test --scratch-path .build-apiA development token is "fresh" for 5 minutes: within that window the purge and revocation paths run for real,
afterwards the tests check the typed reauth_required error instead.
Validating emitted documents against the real web schema (the dev worktree on the backend node, never production):
scp /tmp/spn-docs.json M4M64c:/tmp/spn-docs.json
ssh M4M64c 'bash -lc "cd ~/dev/note-v2 && node_modules/.bin/tsx .spn-schema.mts validate /tmp/spn-docs.json"'Every line must read ok … identical.
🧯 Troubleshooting and gotchas
| Trap | Explanation |
|---|---|
| Never send an idempotency key with a page creation or an upload | The server replays the stored answer of the first attempt for 48 h, whatever the new body, even if the page was purged since. The client-chosen id is what makes the retry safe. |
| The answer of a creation is never trusted blindly | The returned row is always compared with the local one. |
Call ensureBlockIds() before saving |
Merges match blocks by id. Notes without ids still work but merge less well. |
KeychainTokenStore fails in an unsigned tool on macOS |
The data-protection keychain needs a signed app with a keychain access group; pass useDataProtectionKeychain: false. |
| A widget cannot open the store | A read-only store never creates nor migrates the file, and the directory must be writable by the reader (SQLite needs the -shm file). |
| The share extension wrote but the app shows nothing | Only changes made through the same NotesStore instance are observed: signal the app yourself (Darwin notification) and call noteExternalChange(). |
After markAttachmentDeleted(id:) |
The app deletes the returned file itself: attachment files belong to the caller. |
removeSyncedPages(notIn:) |
Scans every clean page: use it only after a full re-download. |
== on NoteDocument |
Compares container keys too; use isStructurallyEqual(to:) to compare content only. |
| Two parallel module builds | Give each its own --scratch-path. |
| Test server | Do not modify its worktree and do not reset its database. |
Differences observed between the contract and the test server (2026-11-18)
GET /api/syncwith a cursor ahead of the server answers{reset: true, cursor: 0, more: true, …}withoutuser.limitbounds each list separately. A non-numeric or negativesince→400 bad_cursor.GET /api/search?mode=quickreturnsupdatedAtin PostgreSQL text form, not ISO 8601.- Page rows of
POST/GET /api/pages…carryworkspaceId,plainText,tsvand notags(the feed has them). POST /api/pages/<id>?action=duplicateanswers200(not 201).- Task
priorityislow | medium | high(400otherwise). POST /api/inbox/<id>also returnscreated;POST /api/inbox/undoreturns{ok, already}; triage actions arekeep | task | database | reference | archive | trash.DELETE /api/auth/sessions/<id>needs a fresh verification (403 reauth_required), like the global revocation.- Idempotency: a replayed answer has
idempotent-replay: truebut notx-existing; a key reused with a different body on the same method and path is replayed as well. - Extra error codes:
bad_request,challenge_missing,no_instruction/bad_language,last_passkey(409),bad_cursor.not_enrolledis a 409. Validation errors are400without a code. A malformed UUID in a path answers500. GET /api/healthis public and does not echox-request-id.GET /api/tagsreportspages: 0for tags carried by live pages.DELETE /api/attachments?id=is a soft delete:/api/files/<id>keeps serving the bytes. Purging a page leaves its attachments alive withpageId: null(no tombstone).
🚧 Known limitations
Documented in the module references.
Sync (docs/SYNC_ENGINE.md)
- Reminders are pushed, not pulled: tasks changed or created on the web do not come back.
- Folders purged on the device: the store queues one purge for the root and the server's purge does not cascade, so the notes inside stay on the server.
- Moves are last-writer-wins and the server does not check cycles.
- The store has no compare-and-set on its conflict helpers: an edit landing in the few microseconds between the read and the write of a merge is replaced by the merge (it is kept as a local version, not lost).
- Undetectable restore: if a restored server has already received enough new writes for its head to pass a device's
cursor, no
resetis sent. - A page that turns into a database row on the web stops being updated on the device (unless
includeDatabaseRows). - Pending pages are pushed 500 per round, 4 rounds per cycle; a larger backlog needs another
sync(). - Ops refused for good by the server are retried with backoff for ever rather than dropped.
- An edit of the title only, or of the same block, on two devices is a conflict by design.
SyncOutcomehas no "cancelled" case:stop()reads as.offline.
Store (docs/STORE_API.md)
trash/restorecancellation ignores pages moved into or out of the subtree between the two calls.- Properties equality ignores key order but is otherwise structural; content equality is byte equality.
Document (docs/DOCUMENT_API.md)
- Two documents differing only by Unicode normalization are
==. - A lone UTF-16 surrogate escape in JSON is read as U+FFFD.
- Marks are toggled on text only; inline atoms keep their own marks.
- Table helpers address cells by their position in each row; they do not resolve
colspan/rowspangeometry. - Reference-style links, footnotes and HTML blocks are imported from Markdown as text.
📚 Documentation
| Document | Content |
|---|---|
docs/DOCUMENT_API.md |
public API of SPNotesDocument: conventions, model, commands, tables, text/Markdown/HTML, merge, validation |
docs/STORE_API.md |
public API and exact semantics of SPNotesStore (12 sections) |
docs/API_CLIENT.md |
SPNotesAPI: set-up, transport rules, errors, endpoints, passkeys, server-sent events, tests |
docs/SYNC_ENGINE.md |
the cycle, its guarantees, the conflict policy, what the apps must call, known limits |
docs/reference/BACKEND_CONTRACT.md |
the backend contract used by the native apps |
docs/reference/README.md |
read-only copies of the web sources the core must stay compatible with (text.ts, markdown.ts, html.ts, validate.ts, same-json.ts, service.ts, web-editor-schema.txt) |
👤 Author
Simon-Pierre Boucher — design, development, operations
📄 License
All rights reserved © 2026 Simon-Pierre Boucher.