Repository Public
1# SP Mail Next — test report23Statuses: **PASS** (executed, result observed), **FAIL**, **BLOCKED** (access, credential or hardware missing), **NOT TESTED**. A line is PASS only if the command was run on the stated commit and its output read; counts are copied from the runners' summaries. Secrets and message contents are absent.45Commits under test — backend + web: `ead8d18` (branch `next`, repository `sp-mail`); Apple: `b414110` (branch `next`, repository `sp-mail-apple`). Dates: 2026-11-20.67Environments: *dev* = backend process started from the checkout on the development laptop, real PostgreSQL 17, the **real mail server** (Stalwart 1.1.1 on BHS64) reached through an SSH tunnel and used only through the staging domain `stg.spboucher.ai`; browsers = Chromium (Playwright 1.65.1) with a virtual WebAuthn authenticator; Apple = macOS 27 on the laptop, iOS 26.5 simulators (iPhone 17 Pro, iPad Pro 13-inch). **Production was not touched and nothing was deployed.** No test sent mail outside `stg.spboucher.ai`.89Important context: the staging domain lives on the production mail server. Test mailboxes are created and purged through the backend's administration API; the registry added in this release refuses any address that is not a test prefix on `stg.spboucher.ai`.1011## 1. Backend1213| # | Suite | Command | Result | Status |14|---|---|---|---|---|15| B1 | Complete integration (existing 0.3.0 tests + Next) | `SPMAIL_TEST_PORT=8981 SPMAIL_TEST_DB=…/sp_mail_int_test node --import tsx --test --test-concurrency=1 tests/integration/api.test.ts tests/integration/templates.test.ts "tests/integration/v1-*.test.ts" "tests/integration/next-*.test.ts"` | 145 tests: 143 pass, 0 fail, 2 skipped (opt-in real-provider AI) | PASS |16| B2 | Pure unit tests | `node --import tsx --test "server/src/**/*.test.ts"` | 44 / 44 | PASS |17| B3 | SDK (TypeScript, Python, Swift example) and MCP server, generated types up to date | `pnpm test:sdk` | 31 / 31 | PASS |18| B4 | Type check | `pnpm typecheck` | clean | PASS |19| B5 | Real provider — Anthropic (`claude-haiku-5-5`), 5 calls, structured output validated, injected message not obeyed | `SPMAIL_TEST_AI_REAL=1 … next-ai.test.ts` (run once on the AI branch) | 9 / 9 | PASS (on an earlier commit) |20| B6 | Real provider — local MacLustr gateway | — | no key was created | NOT TESTED |21| B7 | Real embedding model (`multilingual-e5-small`, in process): three queries without keyword overlap rank the right message first | one-off script, not in the suite | observed once | PASS (manual, earlier commit) |22| B8 | Real APNs | — | no APNs credential configured; the pipeline was exercised against a local HTTP/2 fake only | BLOCKED |2324What B1 covers for Next, with real mail between staging mailboxes: push token lifecycle and cross-user isolation, one alert per device per message (also across a server restart), JWT verified against the test key, preview levels, quiet hours and priority senders, per-mailbox mute, retry after 503, token deletion on 410, badge value and silent badge push, device removal on session revocation, nothing enqueued when unconfigured; snooze (folder, keyword, wake by the worker, cancel, missing message) and reminders (fire, resolved by a real reply); labels, saved searches, snippets, signatures with isolation between users; operator search across two mailboxes with stable pagination; attachment index and ZIP (bytes compared); contact harvest from real mail, vCard round trip, merge; undo send (held, cancelled with the draft and attachments kept, delivered once when not cancelled), sensitive-recipient confirmation; dashboard including the `partial` list; AI consent gates, rate limits, JSON validation and retry, prompt-injection containment (the model request carries the untrusted block and no tools; nothing is sent, moved or deleted whatever the model answers); rules (label, star, mark read, snooze, remind, event, draft reply; idempotent across a restart; never an outbound message; never applied to mail received before the rule was enabled); health, alert raised then resolved, notification dedup, administrator-only and re-authentication enforcement; watcher properties accepted by the real server; mailbox order; upstream 401/403 mapped to 502.2526Not provoked by any test: alert conditions `queue`, `disk`, `cert:*`, `delivery`, `webhooks`, `mail_server` (threshold functions are unit-tested); APNs give-up after 6 attempts, 24 h expiry and 7-day prune; shared mailboxes (`mailbox_grants`) with push, contacts and search; two concurrent server instances; rule actions `pin`, `notify_app`, `extract` and the `ai_category` condition on real mail.2728## 2. Web client2930| # | Suite | Command | Result | Status |31|---|---|---|---|---|32| W1 | Unit (pure logic, editor model, offline queue, list model, sanitiser) | `pnpm test:web` | 376 / 376 | PASS |33| W2 | Main browser suite: desktop, mobile, shell, composer, productivity + intelligence, performance | `playwright test -c tests/e2e/playwright.config.ts` (port 8982) | 124 passed, 0 failed | PASS |34| W3 | Administration | `playwright test -c tests/e2e/admin/playwright.config.ts` (port 8983) | 39 passed | PASS |35| W4 | Templates studio | `playwright test -c tests/e2e/studio/playwright.config.ts` (port 8984) | 24 passed, 1 skipped (real AI, opt-in) | PASS |36| W5 | Offline and service worker | `playwright test -c tests/e2e/offline/playwright.config.ts` (port 8985) | 5 passed (the update-flow test last ran one CSS-only commit before the final one) | PASS |37| W6 | Test-mailbox registry | `node --import tsx --test tests/e2e/mailboxes.test.ts` | 3 / 3 | PASS |38| W7 | Accessibility (axe-core inside W2–W4) | — | 0 serious or critical violation on every screen checked, in light, dark and midnight; 172 checks on the productivity screens alone | PASS (automated rules only) |39| W8 | Browsers other than Chromium; installed web app (incl. iOS Safari); real touch devices; English interface in a browser | — | — | NOT TESTED |4041Performance (W2, project `perf`; 10 000-message mailbox, median of 5, browser and backend on the same machine — no network latency): cold load to interactive list 164 ms, mailbox switch 106 ms, thread open 116 ms, search 790 ms (remarkably constant, 787–793 ms, which suggests a fixed delay rather than compute; not investigated), worst scroll frame 18.6 ms, JS heap after scrolling the whole list 7.8 MB. Budgets = measured + 25 % (`tests/e2e/perf-budgets.json`, `PERFORMANCE.md`). Bundle: entry path 628 kB raw / 201 kB gzip (was 498 / 155 at 0.3.0), post-login Next modules 201 / 61, everything else lazy.4243Cross-device scenarios executed: read state changed in a second browser context appears in the first without a list refetch; a draft replaced from another device keeps both versions; actions taken offline reach the server exactly once after reconnect and the server's concurrent change wins; mailbox removed on the server disappears without a reload.4445## 3. Apple apps4647| # | Suite | Command | Result | Status |48|---|---|---|---|---|49| A1 | Core unit tests (sync hardening, Next client and store, rich text, composer support, helpers) + system package | `Scripts/test-core.sh` | 248 in 49 suites (39 integration tests skipped in this mode) + 26 in 7 suites | PASS |50| A2 | Core integration against the dev backend and real staging mail | `Scripts/test-core.sh --integration` | 248 passed, then 26 passed (run on commit `d9b68e4` + the serial-run change, not re-run on the last UI commits) | PASS (earlier commit) |51| A3 | « Fails before » evidence for the audited core defects | nine expectations run against an unmodified 1.1.0 copy | 9 fail on 1.1.0; 8 pass on the new core, the ninth passes its redefined bound | PASS |52| A4 | Migration of a real 1.1.0 cache file to schema 2 | `opensACacheFileWrittenByTheReal110Core` | messages, sealed bodies, pending operations and outbox items readable | PASS |53| A5 | Builds, Debug, both targets + six extension targets | `Scripts/build.sh mac Debug`, `Scripts/build.sh ios Debug` | succeeded, 0 errors | PASS |54| A6 | In-app self-test, macOS | `Scripts/run-dev.sh --selftest` | 146 / 146 | PASS |55| A7 | In-app self-test, iPhone simulator | `Scripts/run-sim.sh` | 145 / 145 | PASS |56| A8 | In-app self-test, iPad simulator | `Scripts/run-sim.sh --device …` | 145 / 145 | PASS |57| A9 | Signed macOS Release build 2.0.0 (1), universal, three extensions | `Scripts/release.sh`, `Scripts/verify-release.sh` | built; verifier exit 0 (27 checks) | PASS |58| A10 | macOS notarisation of the build made from `b414110` | `Scripts/release.sh`, `verify-release.sh`, `notarize.sh`; then `stapler validate`, `spctl -a -vv` | submission `fae01265-6381-4b3e-abef-4bfd6d7a65a4` Accepted; app and DMG stapled; `spctl`: accepted, `Notarized Developer ID` | PASS |59| A11 | iOS archive and export | `Scripts/ios-archive.sh archive`, `export` | succeeded; IPA 18 MB, Apple Distribution, `aps-environment: production`, no App Group entitlement | PASS |60| A12 | Shortcut clashes | `Scripts/check-shortcuts.py` | 58 menu shortcuts, 0 clash | PASS |61| A13 | Localisation | `Scripts/make-strings.py --sync` | 1 348 + 4 + 68 keys, all with English | PASS |62| A14 | Push pipeline end to end in development: the app's registration reaches the backend, the backend's payloads reach a fake APNs server; same payloads delivered with `simctl push`: banner, actions, « mark as read » changes server state (iPhone), tap opens the conversation (iPad) | manual + self-test | observed | PASS (simulator) |63| A15 | Anything on a physical device: passkey sign-in with Face ID / Touch ID, biometric lock, real APNs, notification service extension running, widgets on a Home or Lock Screen, share extension from a real app, Handoff, background refresh timing, haptics, VoiceOver and Dynamic Type passes, Low Data / Low Power behaviour | — | — | NOT TESTED |64| A16 | The signed Release app launched; macOS widgets, share-menu extension, Services, Dock menu, menu bar extra interaction, window restoration after a real relaunch | — | compiled and signed only | NOT TESTED |65| A17 | 2.0.0 apps against a real 0.3.0 server (capabilities absent) | — | unit-tested gating only | NOT TESTED |6667The self-test runs against the real dev backend with a scripted AI stub and a fake APNs server: sign-in by development token, sync, compose, real sends between two staging mailboxes, labels, pin ordering, snooze and wake, reminders, operator search and saved searches, unified bulk actions, contacts, attachment gallery and ZIP, dashboard, AI summary and reply suggestion opening the composer without sending, writing assistant, rich text with inline image received with its `cid` part, lossless drafts, send later and cancel, undo send inside and outside the window, sensitive-recipient confirmation, delivery tracking in the outbox, notification routing and actions, device removal at sign-out, widget snapshot by privacy level, share hand-off, `mailto:`, lock gating of every window (simulated leave), mailbox removal keeping unsent drafts.6869Text editing was exercised through the same delegate paths as typing, not by a person typing; drag and drop, Quick Look in the composer, snippet and template pickers, vCard import/export and « send and remind » are compiled only.7071## 4. Failure injection and recovery7273| Scenario | Where | Status |74|---|---|---|75| Server restart with scheduled/held sends, pending webhooks, queued pushes, due snoozes, rule jobs: nothing lost, nothing done twice | BI | PASS |76| APNs 503 then success; 410 → token removed; unconfigured → nothing enqueued | BI | PASS |77| Lost HTTP response after the server accepted a submission → not sent twice (fake server, 6 capability sets; and once on the real server) | CU, CI | PASS |78| 5xx bursts, `cannotCalculateChanges`, rejected pending operation followed by valid ones, interruption between delta pages | CU | PASS |79| Browser offline: shell boots, queued actions replay once, rejected operation reported, sign-out empties caches and queue | WE | PASS |80| Mail server unreachable (health, alert after three failures) | BU only | NOT TESTED end to end |81| Disk full, certificate expiry, stuck SMTP queue, antivirus down | — | NOT TESTED |82| Real network cut on a device (not emulation) | — | NOT TESTED |8384## 5. Release artefacts8586| Artefact | State |87|---|---|88| Backend + web | 1.0.0 in production since 2026-11-20 (§7); `main` = `next`, tag `v1.0.0` on spbgit |89| macOS app | `dist/SP Mail.app` 2.0.0 (1) and `dist/SPMail-2.0.0.dmg` (19.4 MB, SHA-256 `d56625536a2c8a9ee8129dc120cee907aa5a4954e609d1a775e80b8503800f39`) notarised and stapled; installed in `/Applications` (§7) |90| iOS app | 2.0.0 (1) on App Store Connect, `VALID` (TestFlight) |91| Production | 1.0.0, `https://mail.spboucher.ai/api/health/ready` → ok, HTTP/2 |9293## 6. Housekeeping done949540 `web-*` and 3 `api-*` leftover test mailboxes purged from `stg.spboucher.ai`; `recette@stg.spboucher.ai` kept; the nine real `spboucher.ai` accounts counted before and after: unchanged.9697## 7. Production deployment — 2026-11-20 (owner's instruction: « continue until everything is done and deployed »)9899Deployed commit `3b4d6ec` (version 1.0.0) on M4M64c, following `RELEASE-NEXT.md`.100101| # | Step / check | Observed | Status |102|---|---|---|---|103| D1 | Database backup with restore verification | `db-20261120T192451Z.dump.enc` (96 K), « restauration vérifiée » | PASS |104| D2 | Mail-store backup on BHS64 | `edge-20261120T192452Z.tar.zst.enc` (52 M), mail server stopped 1 s, `active` afterwards | PASS |105| D3 | `git pull --ff-only` (46817b4 → 3b4d6ec), `pnpm install --frozen-lockfile`, `pnpm build` | build ok, service worker with 110 precached files | PASS |106| D4 | Restart with the new environment; migrations | `migration applied` for 004, 005, 006, 007; `workers started`; no level-50 line in the log | PASS |107| D5 | Health | `/api/health/ready` → `{"ok":true,"version":"1.0.0","mail_server":"reachable"}` locally and through `https://mail.spboucher.ai`; `/api/v1/health` database ok, workers on | PASS |108| D6 | API compatibility with an existing key (read-only MCP key) | `/api/v1/me` 8 addresses; `/api/v1/messages` returns messages with the new `pinned` / `snoozed` / `followup` fields; `openapi.json` 200 | PASS |109| D7 | Web | login page renders in headless Chromium over HTTP/2, no console error other than the expected 401 of `/api/me`; `sw.js` and `manifest.webmanifest` 200 | PASS (signed-in screens not checked: needs the owner's passkey) |110| D8 | Real send and receive in production: `notifications@` → `info@` through the `spboucher-site` key (one clearly labelled control message in the owner's inbox) | `sent`, then `delivered` with `250 2.1.5 OK`; watcher recorded the reception (`watched_messages` +1) | PASS |111| D9 | Existing webhook consumer (www.spboucher.ai) | `message.sent` and `message.delivered` delivered, HTTP 200 | PASS |112| D10 | Alerts raised by the deploy | none (`alerts` empty) | PASS |113| D11 | Push configuration: existing team APNs key `Z9M555KPUY` probed against `api.push.apple.com` and the sandbox with a dummy token for both SP Mail topics | `400 BadDeviceToken` three times — the key and topics are accepted by Apple | PASS (authentication only; no notification delivered to a device) |114| D12 | Local AI provider: key created on the MacLustr gateway; real-provider integration test with Anthropic **and** the local model | 9 / 9, including summary, replies, writing and the ignored injection | PASS |115| D13 | Semantic search | deliberately off (`SPMAIL_AI_EMBED_BACKEND=off`): the in-process model has not been run on the node and the process is capped at 800 MB | NOT ENABLED |116| D14 | Signed-in web session, passkey, real device push, external delivery to several providers | — | NOT TESTED (owner) |117118Apple: macOS 2.0.0 (1) notarised (submission `fae01265-6381-4b3e-abef-4bfd6d7a65a4`), installed in `/Applications` in place of 1.1.0, `spctl` accepted, launched for 14 s without crash report and quit cleanly (window content not inspected; 1.1.0 DMG kept in `dist/`). iOS: 2.0.0 (1) rebuilt from the final commit, uploaded with `Scripts/ios-archive.sh upload` (« UPLOAD SUCCEEDED », delivery `80816845-e303-43ba-b576-6693d30af30f`), state on App Store Connect: `VALID` (app `6836619195`); never installed on a device.119