SP Mail Next — audit of the existing system
Date: 2026-11-20. Baseline: backend + web 46817b4 (version 0.3.0, in production on M4M64c, identical to spbgit main), Apple apps 102facc (1.1.0 build 4, notarised macOS build installed, iOS build on TestFlight). Both working trees were clean and in sync with their remotes. Method: the backend (5 600 lines), schema, documentation and test reports were read directly; the web client (≈ 32 700 lines with tests) and the Apple apps (≈ 10 800 lines of Swift) were each read in full by a dedicated read-only audit pass. Line references are to the baseline commits.
1. What exists and works (verified by the existing suites or read in code)
| Area | State at baseline |
|---|---|
| Mail edge | Stalwart 1.1.1 on BHS64: direct SMTP in/out, DKIM (ed25519 + RSA), SPF/DMARC/iprev checks, ClamAV milter, antispam to Junk, encrypted nightly backup + off-machine copy, restore drill passed (test-report T10) |
| Identity | WebAuthn passkeys only; per-device sessions (web cookie / native bearer, SHA-256 at rest), remote revocation, 5-minute passkey re-authentication for sensitive operations, one-time enrolment/recovery links from the node CLI |
| Gateway | Standard JMAP (core, mail, submission, sieve, vacation, blob, quota, contacts, calendars) relayed per mailbox with a sealed per-mailbox secret; management capability and x:* methods refused; streamed upload/download with attachment + CSP: sandbox; EventSource relay |
/api/v1 |
Scoped API keys with address allow-lists, rate and daily limits, external-send approval, idempotency, scheduled send + cancel, drafts, files, unified list/search, incremental changes, signed webhooks with retries and replay, OpenAPI generated from the route registry, SDKs (TS, Python, Swift example), MCP server |
| Workers | In-process, PostgreSQL-backed leases (FOR UPDATE SKIP LOCKED): scheduled dispatcher, delivery tracker, inbound watcher (persistent JMAP state, events written in the same transaction as the state), webhook dispatcher, expiry, retention, pruning |
| Templates studio | Library, versions, AI generation and retouch (Anthropic), sanitised preview, send by template |
| Admin console (web) | Overview, live DNS check, users/mailboxes/aliases, addresses, API keys + usage, approvals, webhooks, API explorer, automation (Sieve-backed), metrics, queue, audit |
| Web client | Three-pane mail, threads, virtualised list (10 000 messages tested), optimistic actions with undo, keyboard shortcuts, search operators (6), Sieve rule builder, vacation, identities/aliases with signature per sender, unified inbox, remote-image blocking with proxy, authentication badges, link-mismatch warnings, IndexedDB read cache, device/passkey management, export, FR/EN |
| Apple apps | Shared SwiftUI for macOS and iOS/iPadOS on SPMailCore: encrypted SQLite cache, delta sync with Email/changes, durable pending-operation queue and outbox, offline compose/send, isolated WKWebView reader (JavaScript off), Quick Look, drag and drop, swipe actions, command menus, app lock gate, Spotlight (opt-in), unified inbox, templates studio, passkey sign-in, FR/EN |
| Tests | Backend 66 integration tests against real PostgreSQL + real Stalwart (staging domain); web 177 unit + 83 Playwright (desktop, mobile, admin, studio, axe); SDK/MCP 31; Apple core 67 (unit + real integration) + in-app self-test 40/40 on macOS and iPhone simulator |
2. Partially implemented
| Capability | Web | macOS / iOS | Backend |
|---|---|---|---|
| Push notifications | none (title counter only) | token registration code + aps-environment entitlement; local notifications without categories, actions or tap routing |
push_devices table and POST /api/push/register; no sender (test-report A10) |
| Incremental sync | Email/changes then full re-query of loaded pages; no queryChanges |
real delta; full sync wipes bodies and pending ops | — |
| Offline | read-only IndexedDB cache of first pages and 40 threads; no service worker, no action queue | reading limited to bodies already opened; attachments never cached | — |
| Composer | execCommand rich text, one signature per sender, autocomplete from the last 150 sent messages, single draft at a time |
bold/italic/strike/code on OS 26+ only, text signatures | inline images (cid), scheduling and cancel exist in /api/v1 |
| Scheduled send | only from the templates dialog | only for template sends | complete |
| Search | current mailbox only, 6 operators | free text + two scopes; filter fields without UI | unified GET /api/v1/messages |
| Threading | complete | SQL window over cached mail only | — |
| Multi-window / iPad | n/a | macOS: main + composer + studio; iPad: shared split view, multiple scenes declared but one shared model | — |
| Spotlight | n/a | indexing without continuation or per-item deletion | — |
| Themes | system / light / dark | system / light / dark | — |
| Monitoring | panel shows live state | — | no alerts (deployment.md: « Non fait ») |
| Background refresh (iOS) | n/a | BGAppRefreshTask only; may end before sync |
— |
3. Missing
Undo send · snooze · follow-up reminders · labels · saved searches / smart folders · contacts and contact groups · snippets · multiple signatures · inline images in the composers · missing-attachment and sensitive-recipient warnings · attachment gallery and cross-mailbox attachment search · search across all mailboxes in the clients · command palette · resizable/collapsible panes · midnight theme, typography and layout preferences · dashboard · any AI assistance for reading or writing mail (the only AI today generates templates) · semantic search · post-delivery automation beyond Sieve · APNs delivery, notification actions, quiet hours, priority senders · widgets, share extension, menu bar extra, App Intents · service worker / installable web app · server-side alerting · storage and delivery trends · device overview for administrators.
4. Differences between clients (baseline)
| Feature | Web | macOS | iOS / iPadOS |
|---|---|---|---|
| Sieve filters, vacation, quota | yes | no | no |
| Administration | yes | no | no |
Pin ($pinned) |
yes | no | no |
| Search operators | 6 | none (scopes only) | none |
| Bulk selection | yes | yes | Edit mode |
| Unified inbox search / multi-select | no | no | no |
| Offline send queue | no | yes | yes |
| Encrypted local cache | no (IndexedDB clear text) | yes (bodies/previews sealed; subjects, addresses, search text clear) | same |
| Rich text | basic | basic, OS 26+ | basic, OS 26+ |
| Scheduled send in composer | no | no | no |
| App lock | no | yes (cosmetic gate) | yes (cosmetic gate) |
mailto: handling |
n/a | declared | handler unreachable (no URL type) |
5. Defects found (to fix in Phase 1)
Data loss and correctness
| # | Where | Defect |
|---|---|---|
| R1 | web state/composer.ts:383-391 |
Closing the composer (Esc) clears the draft even when the server save failed or was deferred — closing while offline loses the draft |
| R2 | web state/mail.ts:396-401 |
refreshList() refreshes the four lowest loaded pages, not the visible ones: rows shown after deep scrolling go stale |
| R3 | web state/mail.ts:331-337 |
Page loading leaves stale ids after deletions (duplicate keys possible) |
| R4 | web state/mail.ts:886-913, 658 |
List state written after awaits without re-checking the active mailbox: a switch mid-flight can show one account's data in another's view |
| R5 | web state/session.ts:164-169 |
Removed mailboxes keep their JMAP client and EventSource; routes use positional mailbox indices |
| R6 | Apple Core/Compose.swift:237, HTMLSanitizer.swift:194-200 |
Reply/forward runs the display sanitiser on the quoted original: outgoing mail contains spmail-cid:// sources and data-spm-remote attributes |
| R7 | Apple Core/Outbox.swift:199-201, 105 |
A lost response re-queues the message and resubmits it without idempotency: duplicate send possible |
| R8 | Apple Core/Outbox.swift:167 |
Retrying an accepted item destroys the sent copy |
| R9 | Apple Core/SyncEngine.swift:242 vs 217, 387 |
loadOlder assumes the cache is the newest prefix; sparse old rows create pagination gaps |
| R10 | Apple Core/SyncEngine.swift:264, 477-482 |
One rejected pending operation blocks every later sync pass forever |
| R11 | Apple Core/MailStore.swift:435 |
Full resynchronisation wipes cached bodies and pending operations |
| R12 | Apple Shared/Compose.swift:52, Core/Outbox.swift:73 |
Reopening a draft flattens its HTML and quote |
| R13 | Apple Shared/MainViews.swift:507, Reader.swift:160 |
« Move to » lists the folders of the wrong account for a message opened from the unified inbox |
| R14 | Apple Shared/AppModel.swift:150-173 |
Removed mailboxes are never detached or wiped |
| R15 | Apple Core/SyncEngine.swift:258, AppModel.swift:805-809 |
Background refresh can complete before the sync it started |
Security and privacy
| # | Where | Weakness |
|---|---|---|
| S1 | web state/session.ts:241-262 |
Cached mail in IndexedDB survives a remote session revocation (cleared only on voluntary sign-out) |
| S2 | web ui/Reader.tsx:71-72 |
The height-measuring shadow root injects sanitised message HTML into the top-level document, outside the sandboxed frame — one defence layer fewer than documented |
| S3 | web ui/Reader.tsx:92-94 |
« Trusted sender » auto-loads remote images on the From address alone, without DKIM/DMARC alignment |
| S4 | Apple macOS/SPMailApp.swift:59-71, AppModel.swift:314 |
App lock does not gate composer and settings windows on macOS; « lock on leaving » relies on a scene phase change that may not come |
| S5 | Apple Core/MailStore.swift |
Subjects, addresses, search text and correspondents are stored unsealed; decrypted attachments stay in a temporary folder until next launch |
| S6 | backend util.ts |
Rate limits are in memory (single process) — acceptable for one instance, documented |
| S7 | backend | No alerting: a stuck queue, a full disk, an expiring certificate or a failed backup is visible only if someone opens the admin panel |
Performance
| # | Where | Finding |
|---|---|---|
| P1 | web MessageList.tsx:150, 405, mail.ts:696 |
O(total) selector work on every store update (each composer keystroke with a 10 000-row list) |
| P2 | web state/composer.ts, Apple Outbox.swift:28-58 |
Draft autosave = create + destroy every 1.5–2 s: id churn, push storm, list refetches |
| P3 | web mail.ts:812-940 |
Any change re-queries up to four pages and refetches the open thread |
| P4 | web composer.ts:495-500 |
Alias send downloads and re-uploads every attachment, again on retry |
| P5 | Apple Shared/Reader.swift:256 |
HTML sanitiser runs inside body on the main thread at every render |
| P6 | Apple Core/MailStore.swift:260, 339 |
No prepared-statement cache, per-id loops, offline search decrypts up to 5 000 rows on the store actor |
| P7 | web build | Entry chunk 498 kB raw / 155 kB gzip, no manual chunking |
Technical debt
Single global store and 988-line state/mail.ts without unit tests; execCommand editor; preferences written both by PUT and PATCH; AppModel god object (816 lines); no schema versioning in the Apple cache; duplicated dark-theme block in tokens.css; release logs on disk still show 1.0.0 for the notarisation step although the 1.1.0 DMG validates.
6. Opportunities taken by SP Mail Next
- The inbound watcher already writes events and JMAP state in one transaction: push, rules, contact harvesting and semantic indexing are added as hooks in that transaction instead of new polling loops (see ARCHITECTURE-NEXT.md §2).
/api/v1already schedules and cancels sends and supports inline images: undo send and send-later in the composers are client work plus one optional field, not a new subsystem.- JMAP keywords already carry pin state on the web: labels, snooze and follow-up markers reuse the same channel and therefore the same offline queues.
- The three clients each have an operator-free or partial search: one server-side parser removes the divergence.
- Everything not modelled by JMAP is currently either absent or per-client: it moves to one server-side store used by all clients.