SP Vault native apps — test report (2026-11-19)
Xcode 27.2, macOS 27.0, simulators "SPVault iPhone" (iPhone 17 Pro, iOS 26.5) and "SPVault iPad" (iPad Pro 11-inch M5).
All runs below use the Dev variant (SPV_DEV: software passkey authenticator, file-based software device keys, no
biometric prompt) against the REAL server started locally by Scripts/dev-server.sh (port 8781, database
sp_vault_native_test). Nothing below ran against production.
1. Verified by automated test
| What | How | Result |
|---|---|---|
| Debug builds, zero warnings: macOS app + extension, iOS app + extension | Scripts/build.sh mac, Scripts/build.sh ios |
BUILD SUCCEEDED, no warning line |
| Unit tests (27): AutoFill matching, confirmation rule, lookalike, password / OTP results, passkey assertion and registration bytes, pending-items round trip, identities, invitation parsing | Scripts/t.sh mac, Scripts/t.sh ios |
27 tests, 0 failures on each platform |
| iPhone journey 01 (XCUITest, real taps): join with invitation → recovery kit → wrong group refused → verification → main; create login with TOTP; masked → reveal → hide; copy (toast); edit; favorite; search; trash / restore; lock / unlock; generator | Scripts/ui-tests.sh iphone |
passed |
| iPhone journey 02: CSV import (4 items; the Dev build reads the prepared file instead of the system file picker) → Security Center shows weak / reused / insecure-URL findings | same | passed |
| iPhone journey 03: recovery with the recovery key on a fresh profile (a damaged key is refused locally by the checksum) → items are back | same | passed |
| iPad journey 04: join, two logins, split layout | Scripts/ui-tests.sh ipad |
passed |
| Cross-device pairing: the iPhone simulator (XCUITest, journey 11) signs in with the account's passkey, shows the code + fingerprint; the Mac Dev app (scripted scenario, see §2) verifies the code, compares the 6-digit fingerprint and approves; the iPhone opens the vault, sees the Mac's items, creates "From New Device"; the Mac receives it through the live event stream and creates "From Trusted Device"; the iPhone shows it | Scripts/mac-scenarios.sh pair |
passed. First run: iPhone → Mac direction proven live; the Mac → iPhone assertion failed on a test navigation bug. Second run (after fixing the test): passed, but the items of the first run already existed, so the Mac → iPhone direction is proven by sync after pairing, not by a live push |
| Release macOS archive, Developer ID + both provisioning profiles | Scripts/release-mac.sh, Scripts/verify-release.sh |
ARCHIVE SUCCEEDED, 0 compiler warnings, codesign --verify --deep --strict valid, hardened runtime, secure timestamp, no SPV_DEV artefact, no ATS exception, no WebKit linked |
Release iOS archive for generic/platform=iOS, automatic provisioning |
Scripts/release-ios.sh archive |
archive 1.0.0 (1) built and signed (development signing, re-signed at export); entitlements as expected; no SPV_DEV artefact |
2. Verified by me with evidence (not click-level)
macOS XCUITest could not run: the runner failed with "Timed out while enabling automation mode" — this Mac requires
an administrator authentication to enable Automation Mode (automationmodetool), which I did not change. The same
journeys exist for macOS (Scripts/ui-tests.sh mac) and are ready to run once Automation Mode is enabled.
Instead the macOS Dev app ran scripted scenarios (App/Shared/Model/DevScenario.swift, SPV_DEV only): each step calls
the same AppModel / engine function the corresponding button calls, against the real server, with the real window on
screen. Evidence: /tmp/spvault-ui/scenario-*.log at run time and the screenshots (real screencapture of the window).
Scripts/mac-scenarios.sh main: join → recovery kit → seed items → select → copy password (pasteboard holds the password and carriesorg.nspasteboard.ConcealedType) → search → favorite / trash / restore → CSV import → analysis has weak + reused → lock (no decrypted item left in the model) → unlock → dark and light windows → all steps ok.Scripts/mac-scenarios.sh recover: recovery key on a fresh profile → items back → ok.- Screenshots looked at: macOS main dark / light, item detail, Security Center, Settings, Devices, locked, recovery kit, generator; iPhone list / detail / locked / recovery kit / Security Center / pairing; iPad split. Fixed after looking: wrapped "SPV1" prefix, cramped iPhone detail header and metadata, iPad duplicate title + unreadable selection + missing "+" button, sidebar footer overlapping the list, search tab bouncing back (search role removed), list rows not tappable on iOS (root tap gesture replaced by a passive recogniser).
mac-menubar.pngis the menu bar view drawn off screen by the app (the real panel only exists while open).
3. NOT verified (needs a physical device, the owner, or Automation Mode)
- Everything on macOS at click level (menus, keyboard shortcuts ⌘N ⌘F ⌘L ⌘C ⇧⌘C ⌥⌘C ⌘, , the real menu bar extra panel, the editor sheet, toolbar) — compiled and partly screenshotted, never clicked by a test.
- Production path of the keys: Secure Enclave keys with the
.userPresencepolicy, Touch ID / Face ID prompts, oneLAContextcovering both keys, step-up signature prompting again after the reuse window, key invalidation. - The Release app at run time on either platform (sandbox, app-group container, shared Keychain group between app and extension). Only signing and entitlements were checked.
- System passkey sheet with Associated Domains (
webcredentials:vault.spboucher.ai) — registration and sign-in. - The AutoFill extension being loaded by the OS on either platform: enabling the provider, QuickType suggestions,
filling in Safari and apps, passkey assertion / registration from a real site, one-time codes, the
ASCredentialIdentityStorecontent, the pending-items hand-over from the real extension to the app. The logic behind it is unit-tested; the view controller glue is compile-checked only. The macOS Dev extension is not functional by design (sandboxed, cannot read the unsandboxed Dev profile). - Camera: QR scan of a pairing code and of an otpauth secret (not available in the simulator).
- System file pickers (import, attachments, backup export / restore), Quick Look preview, printing the recovery kit.
- Attachments upload / download, vault sharing with a second account, relay shares, vault key rotation, account rekey, plaintext export, backup / restore, passkey add / remove, Connect & Agents, Activity chain view: screens are written and compile; the engine calls are covered by SharedVaultKit's own tests, the screens were not driven.
- Auto-lock timer expiry, lock on sleep / screen lock (macOS), background lock delay and privacy cover in the app switcher (iOS), hide-from-screen-capture, clipboard expiry on iOS, notifications, Spotlight opt-in, App Intents.
- VoiceOver and Dynamic Type passes (labels are set in code; not audited with the assistive technologies on).
- Notarization and App Store Connect upload (intentionally not run).
SharedVaultKitwas not modified; its test suite was not re-run in this session.