SPB Git

Repository Public

TEST_REPORT.md 7.1 KB 67 lines markdown

1# SP Vault native apps — test report (2026-11-19)23Xcode 27.2, macOS 27.0, simulators "SPVault iPhone" (iPhone 17 Pro, iOS 26.5) and "SPVault iPad" (iPad Pro 11-inch M5).4All runs below use the Dev variant (`SPV_DEV`: software passkey authenticator, file-based software device keys, no5biometric prompt) against the REAL server started locally by `Scripts/dev-server.sh` (port 8781, database6`sp_vault_native_test`). Nothing below ran against production.78## 1. Verified by automated test910| What | How | Result |11|---|---|---|12| Debug builds, zero warnings: macOS app + extension, iOS app + extension | `Scripts/build.sh mac`, `Scripts/build.sh ios` | BUILD SUCCEEDED, no warning line |13| Unit tests (27): AutoFill matching, confirmation rule, lookalike, password / OTP results, passkey assertion and registration bytes, pending-items round trip, identities, invitation parsing | `Scripts/t.sh mac`, `Scripts/t.sh ios` | 27 tests, 0 failures on each platform |14| iPhone journey 01 (XCUITest, real taps): join with invitation → recovery kit → wrong group refused → verification → main; create login with TOTP; masked → reveal → hide; copy (toast); edit; favorite; search; trash / restore; lock / unlock; generator | `Scripts/ui-tests.sh iphone` | passed |15| iPhone journey 02: CSV import (4 items; the Dev build reads the prepared file instead of the system file picker) → Security Center shows weak / reused / insecure-URL findings | same | passed |16| iPhone journey 03: recovery with the recovery key on a fresh profile (a damaged key is refused locally by the checksum) → items are back | same | passed |17| iPad journey 04: join, two logins, split layout | `Scripts/ui-tests.sh ipad` | passed |18| Cross-device pairing: the iPhone simulator (XCUITest, journey 11) signs in with the account's passkey, shows the code + fingerprint; the Mac Dev app (scripted scenario, see §2) verifies the code, compares the 6-digit fingerprint and approves; the iPhone opens the vault, sees the Mac's items, creates "From New Device"; the Mac receives it through the live event stream and creates "From Trusted Device"; the iPhone shows it | `Scripts/mac-scenarios.sh pair` | passed. First run: iPhone → Mac direction proven live; the Mac → iPhone assertion failed on a test navigation bug. Second run (after fixing the test): passed, but the items of the first run already existed, so the Mac → iPhone direction is proven by sync after pairing, not by a live push |19| Release macOS archive, Developer ID + both provisioning profiles | `Scripts/release-mac.sh`, `Scripts/verify-release.sh` | ARCHIVE SUCCEEDED, 0 compiler warnings, `codesign --verify --deep --strict` valid, hardened runtime, secure timestamp, no SPV_DEV artefact, no ATS exception, no WebKit linked |20| Release iOS archive for `generic/platform=iOS`, automatic provisioning | `Scripts/release-ios.sh archive` | archive 1.0.0 (1) built and signed (development signing, re-signed at export); entitlements as expected; no SPV_DEV artefact |2122## 2. Verified by me with evidence (not click-level)2324macOS XCUITest could not run: the runner failed with "Timed out while enabling automation mode" — this Mac requires25an administrator authentication to enable Automation Mode (`automationmodetool`), which I did not change. The same26journeys exist for macOS (`Scripts/ui-tests.sh mac`) and are ready to run once Automation Mode is enabled.2728Instead the macOS Dev app ran scripted scenarios (`App/Shared/Model/DevScenario.swift`, SPV_DEV only): each step calls29the same `AppModel` / engine function the corresponding button calls, against the real server, with the real window on30screen. Evidence: `/tmp/spvault-ui/scenario-*.log` at run time and the screenshots (real `screencapture` of the window).3132* `Scripts/mac-scenarios.sh main`: join → recovery kit → seed items → select → copy password (pasteboard holds the33  password and carries `org.nspasteboard.ConcealedType`) → search → favorite / trash / restore → CSV import → analysis34  has weak + reused → lock (no decrypted item left in the model) → unlock → dark and light windows → all steps ok.35* `Scripts/mac-scenarios.sh recover`: recovery key on a fresh profile → items back → ok.36* Screenshots looked at: macOS main dark / light, item detail, Security Center, Settings, Devices, locked, recovery kit,37  generator; iPhone list / detail / locked / recovery kit / Security Center / pairing; iPad split. Fixed after looking:38  wrapped "SPV1" prefix, cramped iPhone detail header and metadata, iPad duplicate title + unreadable selection +39  missing "+" button, sidebar footer overlapping the list, search tab bouncing back (search role removed), list rows40  not tappable on iOS (root tap gesture replaced by a passive recogniser).41* `mac-menubar.png` is the menu bar view drawn off screen by the app (the real panel only exists while open).4243## 3. NOT verified (needs a physical device, the owner, or Automation Mode)4445* Everything on macOS at click level (menus, keyboard shortcuts ⌘N ⌘F ⌘L ⌘C ⇧⌘C ⌥⌘C ⌘, , the real menu bar extra46  panel, the editor sheet, toolbar) — compiled and partly screenshotted, never clicked by a test.47* Production path of the keys: Secure Enclave keys with the `.userPresence` policy, Touch ID / Face ID prompts, one48  `LAContext` covering both keys, step-up signature prompting again after the reuse window, key invalidation.49* The Release app at run time on either platform (sandbox, app-group container, shared Keychain group between app and50  extension). Only signing and entitlements were checked.51* System passkey sheet with Associated Domains (`webcredentials:vault.spboucher.ai`) — registration and sign-in.52* The AutoFill extension being loaded by the OS on either platform: enabling the provider, QuickType suggestions,53  filling in Safari and apps, passkey assertion / registration from a real site, one-time codes, the54  `ASCredentialIdentityStore` content, the pending-items hand-over from the real extension to the app. The logic55  behind it is unit-tested; the view controller glue is compile-checked only. The macOS Dev extension is not56  functional by design (sandboxed, cannot read the unsandboxed Dev profile).57* Camera: QR scan of a pairing code and of an otpauth secret (not available in the simulator).58* System file pickers (import, attachments, backup export / restore), Quick Look preview, printing the recovery kit.59* Attachments upload / download, vault sharing with a second account, relay shares, vault key rotation, account60  rekey, plaintext export, backup / restore, passkey add / remove, Connect & Agents, Activity chain view: screens are61  written and compile; the engine calls are covered by SharedVaultKit's own tests, the screens were not driven.62* Auto-lock timer expiry, lock on sleep / screen lock (macOS), background lock delay and privacy cover in the app63  switcher (iOS), hide-from-screen-capture, clipboard expiry on iOS, notifications, Spotlight opt-in, App Intents.64* VoiceOver and Dynamic Type passes (labels are set in code; not audited with the assistive technologies on).65* Notarization and App Store Connect upload (intentionally not run).66* `SharedVaultKit` was not modified; its test suite was not re-run in this session.67
Files main
Menu

Theme