Repository Public
1# SP Mail Next — audit of the existing system23Date: 2026-11-20. Baseline: backend + web `46817b4` (version 0.3.0, in production on M4M64c, identical to spbgit `main`), Apple apps `102facc` (1.1.0 build 4, notarised macOS build installed, iOS build on TestFlight). Both working trees were clean and in sync with their remotes. Method: the backend (5 600 lines), schema, documentation and test reports were read directly; the web client (≈ 32 700 lines with tests) and the Apple apps (≈ 10 800 lines of Swift) were each read in full by a dedicated read-only audit pass. Line references are to the baseline commits.45## 1. What exists and works (verified by the existing suites or read in code)67| Area | State at baseline |8|---|---|9| Mail edge | Stalwart 1.1.1 on BHS64: direct SMTP in/out, DKIM (ed25519 + RSA), SPF/DMARC/iprev checks, ClamAV milter, antispam to Junk, encrypted nightly backup + off-machine copy, restore drill passed (test-report T10) |10| Identity | WebAuthn passkeys only; per-device sessions (web cookie / native bearer, SHA-256 at rest), remote revocation, 5-minute passkey re-authentication for sensitive operations, one-time enrolment/recovery links from the node CLI |11| Gateway | Standard JMAP (core, mail, submission, sieve, vacation, blob, quota, contacts, calendars) relayed per mailbox with a sealed per-mailbox secret; management capability and `x:*` methods refused; streamed upload/download with `attachment` + `CSP: sandbox`; EventSource relay |12| `/api/v1` | Scoped API keys with address allow-lists, rate and daily limits, external-send approval, idempotency, scheduled send + cancel, drafts, files, unified list/search, incremental changes, signed webhooks with retries and replay, OpenAPI generated from the route registry, SDKs (TS, Python, Swift example), MCP server |13| Workers | In-process, PostgreSQL-backed leases (`FOR UPDATE SKIP LOCKED`): scheduled dispatcher, delivery tracker, inbound watcher (persistent JMAP state, events written in the same transaction as the state), webhook dispatcher, expiry, retention, pruning |14| Templates studio | Library, versions, AI generation and retouch (Anthropic), sanitised preview, send by template |15| Admin console (web) | Overview, live DNS check, users/mailboxes/aliases, addresses, API keys + usage, approvals, webhooks, API explorer, automation (Sieve-backed), metrics, queue, audit |16| Web client | Three-pane mail, threads, virtualised list (10 000 messages tested), optimistic actions with undo, keyboard shortcuts, search operators (6), Sieve rule builder, vacation, identities/aliases with signature per sender, unified inbox, remote-image blocking with proxy, authentication badges, link-mismatch warnings, IndexedDB read cache, device/passkey management, export, FR/EN |17| Apple apps | Shared SwiftUI for macOS and iOS/iPadOS on `SPMailCore`: encrypted SQLite cache, delta sync with `Email/changes`, durable pending-operation queue and outbox, offline compose/send, isolated WKWebView reader (JavaScript off), Quick Look, drag and drop, swipe actions, command menus, app lock gate, Spotlight (opt-in), unified inbox, templates studio, passkey sign-in, FR/EN |18| Tests | Backend 66 integration tests against real PostgreSQL + real Stalwart (staging domain); web 177 unit + 83 Playwright (desktop, mobile, admin, studio, axe); SDK/MCP 31; Apple core 67 (unit + real integration) + in-app self-test 40/40 on macOS and iPhone simulator |1920## 2. Partially implemented2122| Capability | Web | macOS / iOS | Backend |23|---|---|---|---|24| Push notifications | none (title counter only) | token registration code + `aps-environment` entitlement; local notifications without categories, actions or tap routing | `push_devices` table and `POST /api/push/register`; **no sender** (test-report A10) |25| Incremental sync | `Email/changes` then full re-query of loaded pages; no `queryChanges` | real delta; full sync wipes bodies and pending ops | — |26| Offline | read-only IndexedDB cache of first pages and 40 threads; no service worker, no action queue | reading limited to bodies already opened; attachments never cached | — |27| Composer | `execCommand` rich text, one signature per sender, autocomplete from the last 150 sent messages, single draft at a time | bold/italic/strike/code on OS 26+ only, text signatures | inline images (`cid`), scheduling and cancel exist in `/api/v1` |28| Scheduled send | only from the templates dialog | only for template sends | complete |29| Search | current mailbox only, 6 operators | free text + two scopes; filter fields without UI | unified `GET /api/v1/messages` |30| Threading | complete | SQL window over cached mail only | — |31| Multi-window / iPad | n/a | macOS: main + composer + studio; iPad: shared split view, multiple scenes declared but one shared model | — |32| Spotlight | n/a | indexing without continuation or per-item deletion | — |33| Themes | system / light / dark | system / light / dark | — |34| Monitoring | panel shows live state | — | no alerts (deployment.md: « Non fait ») |35| Background refresh (iOS) | n/a | `BGAppRefreshTask` only; may end before sync | — |3637## 3. Missing3839Undo send · snooze · follow-up reminders · labels · saved searches / smart folders · contacts and contact groups · snippets · multiple signatures · inline images in the composers · missing-attachment and sensitive-recipient warnings · attachment gallery and cross-mailbox attachment search · search across all mailboxes in the clients · command palette · resizable/collapsible panes · midnight theme, typography and layout preferences · dashboard · any AI assistance for reading or writing mail (the only AI today generates templates) · semantic search · post-delivery automation beyond Sieve · APNs delivery, notification actions, quiet hours, priority senders · widgets, share extension, menu bar extra, App Intents · service worker / installable web app · server-side alerting · storage and delivery trends · device overview for administrators.4041## 4. Differences between clients (baseline)4243| Feature | Web | macOS | iOS / iPadOS |44|---|---|---|---|45| Sieve filters, vacation, quota | yes | no | no |46| Administration | yes | no | no |47| Pin (`$pinned`) | yes | no | no |48| Search operators | 6 | none (scopes only) | none |49| Bulk selection | yes | yes | Edit mode |50| Unified inbox search / multi-select | no | no | no |51| Offline send queue | no | yes | yes |52| Encrypted local cache | no (IndexedDB clear text) | yes (bodies/previews sealed; subjects, addresses, search text clear) | same |53| Rich text | basic | basic, OS 26+ | basic, OS 26+ |54| Scheduled send in composer | no | no | no |55| App lock | no | yes (cosmetic gate) | yes (cosmetic gate) |56| `mailto:` handling | n/a | declared | handler unreachable (no URL type) |5758## 5. Defects found (to fix in Phase 1)5960### Data loss and correctness61| # | Where | Defect |62|---|---|---|63| R1 | web `state/composer.ts:383-391` | Closing the composer (Esc) clears the draft even when the server save failed or was deferred — closing while offline loses the draft |64| R2 | web `state/mail.ts:396-401` | `refreshList()` refreshes the four lowest loaded pages, not the visible ones: rows shown after deep scrolling go stale |65| R3 | web `state/mail.ts:331-337` | Page loading leaves stale ids after deletions (duplicate keys possible) |66| R4 | web `state/mail.ts:886-913, 658` | List state written after awaits without re-checking the active mailbox: a switch mid-flight can show one account's data in another's view |67| R5 | web `state/session.ts:164-169` | Removed mailboxes keep their JMAP client and EventSource; routes use positional mailbox indices |68| R6 | Apple `Core/Compose.swift:237`, `HTMLSanitizer.swift:194-200` | Reply/forward runs the display sanitiser on the quoted original: outgoing mail contains `spmail-cid://` sources and `data-spm-remote` attributes |69| R7 | Apple `Core/Outbox.swift:199-201, 105` | A lost response re-queues the message and resubmits it without idempotency: duplicate send possible |70| R8 | Apple `Core/Outbox.swift:167` | Retrying an accepted item destroys the sent copy |71| R9 | Apple `Core/SyncEngine.swift:242` vs `217, 387` | `loadOlder` assumes the cache is the newest prefix; sparse old rows create pagination gaps |72| R10 | Apple `Core/SyncEngine.swift:264, 477-482` | One rejected pending operation blocks every later sync pass forever |73| R11 | Apple `Core/MailStore.swift:435` | Full resynchronisation wipes cached bodies and pending operations |74| R12 | Apple `Shared/Compose.swift:52`, `Core/Outbox.swift:73` | Reopening a draft flattens its HTML and quote |75| R13 | Apple `Shared/MainViews.swift:507`, `Reader.swift:160` | « Move to » lists the folders of the wrong account for a message opened from the unified inbox |76| R14 | Apple `Shared/AppModel.swift:150-173` | Removed mailboxes are never detached or wiped |77| R15 | Apple `Core/SyncEngine.swift:258`, `AppModel.swift:805-809` | Background refresh can complete before the sync it started |7879### Security and privacy80| # | Where | Weakness |81|---|---|---|82| S1 | web `state/session.ts:241-262` | Cached mail in IndexedDB survives a remote session revocation (cleared only on voluntary sign-out) |83| S2 | web `ui/Reader.tsx:71-72` | The height-measuring shadow root injects sanitised message HTML into the top-level document, outside the sandboxed frame — one defence layer fewer than documented |84| S3 | web `ui/Reader.tsx:92-94` | « Trusted sender » auto-loads remote images on the From address alone, without DKIM/DMARC alignment |85| S4 | Apple `macOS/SPMailApp.swift:59-71`, `AppModel.swift:314` | App lock does not gate composer and settings windows on macOS; « lock on leaving » relies on a scene phase change that may not come |86| S5 | Apple `Core/MailStore.swift` | Subjects, addresses, search text and correspondents are stored unsealed; decrypted attachments stay in a temporary folder until next launch |87| S6 | backend `util.ts` | Rate limits are in memory (single process) — acceptable for one instance, documented |88| S7 | backend | No alerting: a stuck queue, a full disk, an expiring certificate or a failed backup is visible only if someone opens the admin panel |8990### Performance91| # | Where | Finding |92|---|---|---|93| P1 | web `MessageList.tsx:150, 405`, `mail.ts:696` | O(total) selector work on every store update (each composer keystroke with a 10 000-row list) |94| P2 | web `state/composer.ts`, Apple `Outbox.swift:28-58` | Draft autosave = create + destroy every 1.5–2 s: id churn, push storm, list refetches |95| P3 | web `mail.ts:812-940` | Any change re-queries up to four pages and refetches the open thread |96| P4 | web `composer.ts:495-500` | Alias send downloads and re-uploads every attachment, again on retry |97| P5 | Apple `Shared/Reader.swift:256` | HTML sanitiser runs inside `body` on the main thread at every render |98| P6 | Apple `Core/MailStore.swift:260, 339` | No prepared-statement cache, per-id loops, offline search decrypts up to 5 000 rows on the store actor |99| P7 | web build | Entry chunk 498 kB raw / 155 kB gzip, no manual chunking |100101### Technical debt102Single global store and 988-line `state/mail.ts` without unit tests; `execCommand` editor; preferences written both by PUT and PATCH; `AppModel` god object (816 lines); no schema versioning in the Apple cache; duplicated dark-theme block in `tokens.css`; release logs on disk still show 1.0.0 for the notarisation step although the 1.1.0 DMG validates.103104## 6. Opportunities taken by SP Mail Next1051061. The inbound watcher already writes events and JMAP state in one transaction: push, rules, contact harvesting and semantic indexing are added as hooks in that transaction instead of new polling loops (see ARCHITECTURE-NEXT.md §2).1072. `/api/v1` already schedules and cancels sends and supports inline images: undo send and send-later in the composers are client work plus one optional field, not a new subsystem.1083. JMAP keywords already carry pin state on the web: labels, snooze and follow-up markers reuse the same channel and therefore the same offline queues.1094. The three clients each have an operator-free or partial search: one server-side parser removes the divergence.1105. Everything not modelled by JMAP is currently either absent or per-client: it moves to one server-side store used by all clients.111