SPB Git

Repository Public

RELEASE-NEXT.md 12.2 KB

# SP Mail Next — release and rollback

Scope: backend + webmail (repository sp-mail, branch next, version 1.0.0) and Apple apps (repository sp-mail-apple, branch next, version 2.0.0 build 1). Production today: backend/web 0.3.0 at commit 46817b4 on M4M64c (~/apps/sp-mail, PM2 sp-mail, port 8730), mail server Stalwart 1.1.1 on BHS64, Apple apps 1.1.0 (4).

Status: executed on 2026-11-20 on the owner's instruction — production runs 1.0.0 at 3b4d6ec; evidence in TEST-REPORT-NEXT.md §7. Rollback target: 46817b4 (0.3.0). Configuration applied: version string, APNs (existing team key Z9M555KPUY, validated against Apple), local AI provider, semantic search off, disk path. The manifest on M1M32 was updated to match (backup sp-mail.json.bak-20261120-next).

# 0. What the release changes in production

Area Change Reversible by
PostgreSQL sp_mail Migrations 004–007, all additive: new tables; new nullable/defaulted columns on push_devices, outbound_messages, rules Rolling the code back; 0.3.0 ignores the new tables and columns. No down-migration is needed or provided
Backend process New routes /api/next/*, extra optional fields on /api/v1/messages/send and /api/v1/messages, new workers (push, snooze, rules, semantic, monitor, usage snapshots), watcher hooks git checkout 46817b4 && pnpm install --frozen-lockfile && pnpm build && pm2 restart sp-mail
Mail server (Stalwart) No configuration change. New behaviour seen by Stalwart: a per-mailbox folder named Snoozed is created the first time a user snoozes; keywords $snoozed, $followup, $pinned, lbl_* on messages; internal alert messages are written into the administrator's inbox by JMAP Folder and keywords are ordinary mail data; they stay harmlessly if the code is rolled back
Web client New build served from dist/web; a service worker is registered in browsers Old build + the service worker unregisters itself on sign-out and updates on the next load; a rolled-back build does not ship a worker, so add the kill-switch described in §5
DNS, Caddy, firewall, BHS64 None —
Apple developer account Already done during development through the App Store Connect API: six extension bundle identifiers, three Developer ID profiles for the macOS extensions, App Groups capability enabled on the iOS App ID Identifiers cannot be deleted once used by an uploaded build; profiles can be revoked in the portal

Sender authorisation, API keys, scopes, webhooks signatures, passkeys and sessions are unchanged. No user is signed out by the upgrade.

# 1. Preconditions

  1. Working trees clean, next pushed to spbgit for both repositories (not done yet).
  2. Suites green on the commit to be released — see TEST-REPORT-NEXT.md.
  3. Decide the configuration (all optional; without them the corresponding feature reports itself disabled and nothing else is affected):
Feature Variables (manifest M1M32:~/dispatch/apps/sp-mail.json, process sp-mail) Note
Version string SPMAIL_VERSION=1.0.0 Currently 0.1.0 in the manifest although 0.3.0 is deployed
Push (APNs) SPMAIL_APNS_KEY_FILE (path to a .p8, mode 600, outside the repository), SPMAIL_APNS_KEY_ID, SPMAIL_APNS_TEAM_ID=3YM54G49SN Needs an APNs key valid for the SP Mail bundle identifiers. A team key exists for another project (Z9M555KPUY); whether it may be reused for SP Mail is the owner's decision. Outbound TCP 443 to api.push.apple.com required from M4M64c
Local AI SPMAIL_AI_LOCAL_URL=https://llm.spboucher.ai/v1, SPMAIL_AI_LOCAL_KEY (create with maclustr-llm key create on m4mm), SPMAIL_AI_LOCAL_MODEL=maclustr-qwen3 Never tested against the real gateway — only against a scripted stub
Anthropic ANTHROPIC_API_KEY is already in the manifest (templates studio). With Next it also makes the provider selectable; it stays off for every user until that user gives explicit consent. Optional: SPMAIL_AI_ANTHROPIC_MODEL To make it unavailable, run the Next code with a separate variable for the studio or leave as is
Semantic search optional dependency @huggingface/transformers installed by pnpm install; first use downloads the embedding model (HF_HUB_DISABLE_XET=1, SPMAIL_AI_EMBED_CACHE_DIR outside node_modules) ; SPMAIL_AI_EMBED_BACKEND=off hides the feature Recommended for the first release: off (not exercised end to end)
Monitoring SPMAIL_BACKUP_STATUS already set; SPMAIL_OPS_DISK_PATH = data volume; SPMAIL_OPS_MONITOR=off disables alerts Alerts are written into the administrator's inbox and pushed to administrators' devices

# 2. Backup (blocking)

On M4M64c:

bash
cd ~/apps/sp-mail
pm2 trigger sp-mail-backup 2>/dev/null || /bin/zsh scripts/backup.sh        # dump of sp_mail, encrypted
ls -lt ~/backups/sp-mail | head -3                                           # a file dated now
# verify the dump restores: restore into a scratch database and count tables
createdb sp_mail_restorecheck && <decrypt per scripts/backup.sh header> | psql -q sp_mail_restorecheck \
  && psql -Atc "select count(*) from users" sp_mail_restorecheck && dropdb sp_mail_restorecheck

On BHS64: sudo /usr/local/sbin/sp-mail-edge-backup then check /var/backups/sp-mail (the mail store is not modified by the release, this is the standing precaution of docs/deployment.md). Record both file names and sizes in the release log. Do not continue if either step fails.

# 3. Deploy the backend and the web client (M4M64c)

bash
ssh M4M64c
cd ~/apps/sp-mail && export PATH=/opt/homebrew/bin:$PATH
git status --short                     # must be empty (mld guard)
git rev-parse HEAD                     # note it: rollback target (expected 46817b4)
git fetch origin && git checkout main && git merge --ff-only origin/main      # after `next` was merged into `main` on spbgit
pnpm install --frozen-lockfile
pnpm build
# manifest edits (variables of §1) are made on M1M32 and applied with:  mld deploy sp-mail   — or, without manifest change:
pm2 restart sp-mail --update-env

Migrations run at start, each in one transaction (schema_migrations records them). Watch the first lines: pm2 logs sp-mail --lines 40 --nostream must show migration applied: 004_push_snooze.sql … 007_next_integration.sql, workers started, and no unhandled error.

# 4. Verification (in this order; stop and roll back on the first failure)

# Check Expected
1 curl -s https://mail.spboucher.ai/api/health/ready {"ok":true,"version":"1.0.0","mail_server":"reachable"}
2 curl -s https://mail.spboucher.ai/api/v1/health database, mail server and workers ok
3 curl -s https://mail.spboucher.ai/api/next/capabilities (signed-in session) feature list
4 API compatibility: an existing API key (e.g. the claude-code-mcp read key) lists messages; GET /api/v1/openapi.json loads; the MCP server answers spmail_list_accounts unchanged behaviour, new optional fields only
5 Web: sign in with the owner's passkey, inbox loads, open a message, existing unread counts unchanged —
6 Real mail on the staging domain only: tests/integration/jmap_send.py (recette@stg.spboucher.ai → the owner's iCloud address, the one test recipient authorised) and a reply back; jmap_inbox.py shows it accepted, delivered: yes, reply received
7 Inbound watcher: within 10 s of check 6, select count(*) from watched_messages where received_at > now() - interval '5 minutes' ≥ 1 and /api/admin/health → workers.data.watcher_stalled = [] —
8 Admin › Santé: all checks ok or an explained unknown; no alert raised by the deploy itself —
9 Undo send between two owned mailboxes (10 s hold, cancel): nothing delivered —
10 pm2 logs sp-mail --lines 200 --nostream | grep -c '"level":50' 0

The existing webhooks consumer (spboucher-site key: accounts and newsletter of www.spboucher.ai) must be checked explicitly: send one internal notification through that key and confirm the site still receives its webhook.

# 5. Rollback

Trigger: any failed check of §4, mail not flowing, or an error rate in the logs.

bash
ssh M4M64c 'cd ~/apps/sp-mail && export PATH=/opt/homebrew/bin:$PATH && git checkout 46817b4 && pnpm install --frozen-lockfile && pnpm build && pm2 restart sp-mail --update-env'
curl -s https://mail.spboucher.ai/api/health/ready     # version 0.3.0
  • Database: leave it as is. 0.3.0 runs on the migrated schema (additive changes only; schema_migrations rows for 004–007 are ignored by the old runner, which only applies files it finds on disk). Restoring the dump of §2 is needed only if data was damaged, and would lose identities/keys created since the dump — last resort.
  • Mail store: untouched by the release; nothing to roll back. Messages snoozed before the rollback remain in the Snoozed folder and no worker will wake them under 0.3.0: before rolling back, wake them from the Next build (select id from snoozes where status = 'active', then DELETE /api/next/snoozes/:id for each), or move them back to the inbox by hand afterwards. Held (undo) and scheduled sends are in outbound_messages and are still dispatched by 0.3.0.
  • Service worker: browsers that loaded the Next build keep its worker and precached shell until they load a build that tells them otherwise. The 0.3.0 build has no worker file, so after a rollback serve a one-line kill-switch at /sw.js (self.addEventListener("install",()=>self.skipWaiting());self.addEventListener("activate",e=>e.waitUntil(self.registration.unregister().then(()=>self.clients.matchAll()).then(cs=>cs.forEach(c=>c.navigate(c.url)))))) by placing it in dist/web/sw.js of the rolled-back build.
  • Apple apps 2.0.0 against a 0.3.0 server: supported. The apps probe /api/next/capabilities; on 404 every Next entry is hidden and the 1.1 behaviour remains. This path is compiled and unit-tested but was not run against a real 0.3.0 server.
  • Apple apps downgrade: the local cache schema of 2.0.0 (version 2) is refused by 1.1.0. Going back to 1.1.0 on a device means deleting the app's cache (sign out or reinstall); mail itself is on the server.

# 6. Apple apps

All commands in ~/Desktop/Cluster/apple/SPMail (branch next). Signing identities, the notarisation keychain profile MacLustr-Notarize and the App Store Connect API key are the existing ones.

bash
Scripts/release.sh            # signed universal build: dist/SP Mail.app 2.0.0 (1) with its three extensions
Scripts/verify-release.sh     # signature, entitlements per bundle, no dev switch, no test server
Scripts/notarize.sh           # notarytool + staple of the app, then DMG built, signed, notarised, stapled: dist/SPMail-2.0.0.dmg
Scripts/ios-archive.sh archive && Scripts/ios-archive.sh export       # dist/ios-export/SP Mail.ipa
Scripts/ios-archive.sh upload                                         # TestFlight — owner's instruction required
  • macOS: installing 2.0.0 over 1.1.0 migrates the local cache in place (tested on a real 1.1.0 cache file). Keep the 1.1.0 DMG (dist/SPMail-1.1.0.dmg) as the fallback.
  • iOS: the release build ships without the App Group (the API key cannot register it). To enable it: create group.ai.spboucher.spmail in the developer portal, attach it to the four iOS identifiers, then SPMAIL_IOS_APP_GROUP=1 Scripts/ios-archive.sh archive — an untested path (docs/SYSTEM-INTEGRATION.md in the Apple repository).
  • Push needs the server variables of §1 and the App ID's push capability (already enabled by the existing provisioning).
  • First launch on a real device is the first time the following run for real: passkey sign-in, biometric lock, APNs registration, widgets, share extension, background refresh.

# 7. After the release

  1. Merge next into main in both repositories, tag v1.0.0 (backend/web) and v2.0.0 (Apple), push to spbgit.
  2. Update docs/deployment.md, docs/test-report.md (link to TEST-REPORT-NEXT.md) and the manifest's SPMAIL_VERSION.
  3. Remove the development worktrees (~/dev/spmail-next/*), the next-* branches, the SPMailNext … simulators and the scratch databases (sp_mail_next*_test, sp_mail_w*_dev, sp_mail_apple*_dev, sp_mail_int*).
  4. Watch Admin › Santé and the alert list for 24 hours.
Files main
Menu

Theme