SPB Git

Repository Public

AUDIT-NEXT.md 12.1 KB

# SP Mail Next — audit of the existing system

Date: 2026-11-20. Baseline: backend + web 46817b4 (version 0.3.0, in production on M4M64c, identical to spbgit main), Apple apps 102facc (1.1.0 build 4, notarised macOS build installed, iOS build on TestFlight). Both working trees were clean and in sync with their remotes. Method: the backend (5 600 lines), schema, documentation and test reports were read directly; the web client (≈ 32 700 lines with tests) and the Apple apps (≈ 10 800 lines of Swift) were each read in full by a dedicated read-only audit pass. Line references are to the baseline commits.

# 1. What exists and works (verified by the existing suites or read in code)

Area State at baseline
Mail edge Stalwart 1.1.1 on BHS64: direct SMTP in/out, DKIM (ed25519 + RSA), SPF/DMARC/iprev checks, ClamAV milter, antispam to Junk, encrypted nightly backup + off-machine copy, restore drill passed (test-report T10)
Identity WebAuthn passkeys only; per-device sessions (web cookie / native bearer, SHA-256 at rest), remote revocation, 5-minute passkey re-authentication for sensitive operations, one-time enrolment/recovery links from the node CLI
Gateway Standard JMAP (core, mail, submission, sieve, vacation, blob, quota, contacts, calendars) relayed per mailbox with a sealed per-mailbox secret; management capability and x:* methods refused; streamed upload/download with attachment + CSP: sandbox; EventSource relay
/api/v1 Scoped API keys with address allow-lists, rate and daily limits, external-send approval, idempotency, scheduled send + cancel, drafts, files, unified list/search, incremental changes, signed webhooks with retries and replay, OpenAPI generated from the route registry, SDKs (TS, Python, Swift example), MCP server
Workers In-process, PostgreSQL-backed leases (FOR UPDATE SKIP LOCKED): scheduled dispatcher, delivery tracker, inbound watcher (persistent JMAP state, events written in the same transaction as the state), webhook dispatcher, expiry, retention, pruning
Templates studio Library, versions, AI generation and retouch (Anthropic), sanitised preview, send by template
Admin console (web) Overview, live DNS check, users/mailboxes/aliases, addresses, API keys + usage, approvals, webhooks, API explorer, automation (Sieve-backed), metrics, queue, audit
Web client Three-pane mail, threads, virtualised list (10 000 messages tested), optimistic actions with undo, keyboard shortcuts, search operators (6), Sieve rule builder, vacation, identities/aliases with signature per sender, unified inbox, remote-image blocking with proxy, authentication badges, link-mismatch warnings, IndexedDB read cache, device/passkey management, export, FR/EN
Apple apps Shared SwiftUI for macOS and iOS/iPadOS on SPMailCore: encrypted SQLite cache, delta sync with Email/changes, durable pending-operation queue and outbox, offline compose/send, isolated WKWebView reader (JavaScript off), Quick Look, drag and drop, swipe actions, command menus, app lock gate, Spotlight (opt-in), unified inbox, templates studio, passkey sign-in, FR/EN
Tests Backend 66 integration tests against real PostgreSQL + real Stalwart (staging domain); web 177 unit + 83 Playwright (desktop, mobile, admin, studio, axe); SDK/MCP 31; Apple core 67 (unit + real integration) + in-app self-test 40/40 on macOS and iPhone simulator

# 2. Partially implemented

Capability Web macOS / iOS Backend
Push notifications none (title counter only) token registration code + aps-environment entitlement; local notifications without categories, actions or tap routing push_devices table and POST /api/push/register; no sender (test-report A10)
Incremental sync Email/changes then full re-query of loaded pages; no queryChanges real delta; full sync wipes bodies and pending ops —
Offline read-only IndexedDB cache of first pages and 40 threads; no service worker, no action queue reading limited to bodies already opened; attachments never cached —
Composer execCommand rich text, one signature per sender, autocomplete from the last 150 sent messages, single draft at a time bold/italic/strike/code on OS 26+ only, text signatures inline images (cid), scheduling and cancel exist in /api/v1
Scheduled send only from the templates dialog only for template sends complete
Search current mailbox only, 6 operators free text + two scopes; filter fields without UI unified GET /api/v1/messages
Threading complete SQL window over cached mail only —
Multi-window / iPad n/a macOS: main + composer + studio; iPad: shared split view, multiple scenes declared but one shared model —
Spotlight n/a indexing without continuation or per-item deletion —
Themes system / light / dark system / light / dark —
Monitoring panel shows live state — no alerts (deployment.md: « Non fait »)
Background refresh (iOS) n/a BGAppRefreshTask only; may end before sync —

# 3. Missing

Undo send · snooze · follow-up reminders · labels · saved searches / smart folders · contacts and contact groups · snippets · multiple signatures · inline images in the composers · missing-attachment and sensitive-recipient warnings · attachment gallery and cross-mailbox attachment search · search across all mailboxes in the clients · command palette · resizable/collapsible panes · midnight theme, typography and layout preferences · dashboard · any AI assistance for reading or writing mail (the only AI today generates templates) · semantic search · post-delivery automation beyond Sieve · APNs delivery, notification actions, quiet hours, priority senders · widgets, share extension, menu bar extra, App Intents · service worker / installable web app · server-side alerting · storage and delivery trends · device overview for administrators.

# 4. Differences between clients (baseline)

Feature Web macOS iOS / iPadOS
Sieve filters, vacation, quota yes no no
Administration yes no no
Pin ($pinned) yes no no
Search operators 6 none (scopes only) none
Bulk selection yes yes Edit mode
Unified inbox search / multi-select no no no
Offline send queue no yes yes
Encrypted local cache no (IndexedDB clear text) yes (bodies/previews sealed; subjects, addresses, search text clear) same
Rich text basic basic, OS 26+ basic, OS 26+
Scheduled send in composer no no no
App lock no yes (cosmetic gate) yes (cosmetic gate)
mailto: handling n/a declared handler unreachable (no URL type)

# 5. Defects found (to fix in Phase 1)

# Data loss and correctness

# Where Defect
R1 web state/composer.ts:383-391 Closing the composer (Esc) clears the draft even when the server save failed or was deferred — closing while offline loses the draft
R2 web state/mail.ts:396-401 refreshList() refreshes the four lowest loaded pages, not the visible ones: rows shown after deep scrolling go stale
R3 web state/mail.ts:331-337 Page loading leaves stale ids after deletions (duplicate keys possible)
R4 web state/mail.ts:886-913, 658 List state written after awaits without re-checking the active mailbox: a switch mid-flight can show one account's data in another's view
R5 web state/session.ts:164-169 Removed mailboxes keep their JMAP client and EventSource; routes use positional mailbox indices
R6 Apple Core/Compose.swift:237, HTMLSanitizer.swift:194-200 Reply/forward runs the display sanitiser on the quoted original: outgoing mail contains spmail-cid:// sources and data-spm-remote attributes
R7 Apple Core/Outbox.swift:199-201, 105 A lost response re-queues the message and resubmits it without idempotency: duplicate send possible
R8 Apple Core/Outbox.swift:167 Retrying an accepted item destroys the sent copy
R9 Apple Core/SyncEngine.swift:242 vs 217, 387 loadOlder assumes the cache is the newest prefix; sparse old rows create pagination gaps
R10 Apple Core/SyncEngine.swift:264, 477-482 One rejected pending operation blocks every later sync pass forever
R11 Apple Core/MailStore.swift:435 Full resynchronisation wipes cached bodies and pending operations
R12 Apple Shared/Compose.swift:52, Core/Outbox.swift:73 Reopening a draft flattens its HTML and quote
R13 Apple Shared/MainViews.swift:507, Reader.swift:160 « Move to » lists the folders of the wrong account for a message opened from the unified inbox
R14 Apple Shared/AppModel.swift:150-173 Removed mailboxes are never detached or wiped
R15 Apple Core/SyncEngine.swift:258, AppModel.swift:805-809 Background refresh can complete before the sync it started

# Security and privacy

# Where Weakness
S1 web state/session.ts:241-262 Cached mail in IndexedDB survives a remote session revocation (cleared only on voluntary sign-out)
S2 web ui/Reader.tsx:71-72 The height-measuring shadow root injects sanitised message HTML into the top-level document, outside the sandboxed frame — one defence layer fewer than documented
S3 web ui/Reader.tsx:92-94 « Trusted sender » auto-loads remote images on the From address alone, without DKIM/DMARC alignment
S4 Apple macOS/SPMailApp.swift:59-71, AppModel.swift:314 App lock does not gate composer and settings windows on macOS; « lock on leaving » relies on a scene phase change that may not come
S5 Apple Core/MailStore.swift Subjects, addresses, search text and correspondents are stored unsealed; decrypted attachments stay in a temporary folder until next launch
S6 backend util.ts Rate limits are in memory (single process) — acceptable for one instance, documented
S7 backend No alerting: a stuck queue, a full disk, an expiring certificate or a failed backup is visible only if someone opens the admin panel

# Performance

# Where Finding
P1 web MessageList.tsx:150, 405, mail.ts:696 O(total) selector work on every store update (each composer keystroke with a 10 000-row list)
P2 web state/composer.ts, Apple Outbox.swift:28-58 Draft autosave = create + destroy every 1.5–2 s: id churn, push storm, list refetches
P3 web mail.ts:812-940 Any change re-queries up to four pages and refetches the open thread
P4 web composer.ts:495-500 Alias send downloads and re-uploads every attachment, again on retry
P5 Apple Shared/Reader.swift:256 HTML sanitiser runs inside body on the main thread at every render
P6 Apple Core/MailStore.swift:260, 339 No prepared-statement cache, per-id loops, offline search decrypts up to 5 000 rows on the store actor
P7 web build Entry chunk 498 kB raw / 155 kB gzip, no manual chunking

# Technical debt

Single global store and 988-line state/mail.ts without unit tests; execCommand editor; preferences written both by PUT and PATCH; AppModel god object (816 lines); no schema versioning in the Apple cache; duplicated dark-theme block in tokens.css; release logs on disk still show 1.0.0 for the notarisation step although the 1.1.0 DMG validates.

# 6. Opportunities taken by SP Mail Next

  1. The inbound watcher already writes events and JMAP state in one transaction: push, rules, contact harvesting and semantic indexing are added as hooks in that transaction instead of new polling loops (see ARCHITECTURE-NEXT.md §2).
  2. /api/v1 already schedules and cancels sends and supports inline images: undo send and send-later in the composers are client work plus one optional field, not a new subsystem.
  3. JMAP keywords already carry pin state on the web: labels, snooze and follow-up markers reuse the same channel and therefore the same offline queues.
  4. The three clients each have an operator-free or partial search: one server-side parser removes the divergence.
  5. Everything not modelled by JMAP is currently either absent or per-client: it moves to one server-side store used by all clients.
Files main
Menu

Theme