SPB Git

Repository Public License

TESTING.md 6.2 KB

# Testing

State on 2026-11-22, Xcode 27.2, iOS 26.5 simulator (iPhone 17 Pro). Only results that were actually obtained are listed.

# Summary

Suite Command Result
AlarmCore unit tests cd Packages/AlarmCore && swift test 100 tests, 0 failures
App integration tests Scripts/build.sh test 27 tests, 0 failures
Interface tests Scripts/build.sh uitest 17 tests run, 0 failures (5 probe tests skipped by default)
Real AlarmKit probe Scripts/probe-alarmkit.sh <test> 2 of 5 scenarios pass; 3 cannot be driven reliably (below)
Release archive Scripts/release.sh archive export validate archive built, App Store export signed, "VERIFY SUCCEEDED with no errors"

# Unit tests (AlarmCore)

Recurrence (one-time, weekly, nearest day, exactly-now, spring-forward gap, repeated autumn hour, time-zone change), snooze policy, intensity curves, reconciliation (missing, mismatched, orphan, disabled, unreadable system, running snooze), stable identifiers, the occurrence state machine, storage (round trip, corrupt file recovered from backup, corrupt file never overwriting a good backup, both corrupt, migration, newer schema preserved), export and import, event deduplication and cap, statistics (only recorded facts, streaks, consistency), adaptive difficulty and suggestions, templates and catalogs, progressive wake timeline, sleep planner (midnight wrap, noon rollover, DST), dashboard schedule, and every puzzle engine over thousands of generated puzzles per level.

# Integration tests (Tests/AppModelTests.swift)

The real AppModel runs against FakeAlarmService with an injected clock and a temporary directory:

  • creation, modification without duplicates, disabling, deletion, duplication;
  • permission denied (error returned, never shown as scheduled, scheduled after the user allows), not-determined, system limit then retry;
  • an alarm stays scheduled when its mission is invalid;
  • reconciliation: lost alarm restored, orphan removed, relaunch without rescheduling, relaunch after the system lost everything, unreadable system state, corrupted library;
  • time-zone change;
  • stop of a recurring alarm, retirement of a one-time alarm, full snooze chain with decreasing durations and difficulty escalation, snooze beyond the limit, snooze follow-up failure;
  • mission completion, confirmation asked again after the app is closed, emergency exit, preview recording nothing;
  • alarm that rang while the app was not running (recorded as derived from the schedule, one-time alarm retired, never counted twice);
  • QR digest storage, export / import / erase, focus timer, settings persistence.

# Interface tests

ORIXAlarmUITests: onboarding then alarm creation showing the verified status; denied permission showing the banner and the "saved but not scheduled" error; all five tabs with demo data; a full simulated wake (the fake system rings and stops the alarm, the mission is solved, "I'm up" is confirmed, Insights shows it, and a relaunch does not propose it again); the emergency exit. MathMissionUITests, PuzzleMissionsUITests: missions played to the summary.

# Real AlarmKit on the simulator

UITests/AlarmKitProbeUITests.swift launches the app with the real AlarmKitService (-ORIXProbeAlarm), answers the system permission prompt, schedules an alarm for the next minute, sends the app to the background and reads what AlarmKit reports. Evidence is kept in docs/evidence/.

Scenario Result
Permission prompt shown with the usage description and accepted observed
Alarm scheduled and read back as scheduled observed
Alarm alerting at the set minute with the app in the background; system alert shown over the Home Screen observed (alarmkit-simulator-alert.png)
Stop issued through AlarmKit is observed by the app, recorded as acknowledged, and a one-time alarm is retired pass (testObservedStopRetiresOneTimeAlarm)
After that stop the wake mission is presented when the app comes forward pass (testObservedStopPresentsMission)
Tapping Stop on the alert inconclusive: the simulator shows the alert as a banner that collapses into the Dynamic Island, exposes no accessibility element, and blocks XCTest screenshots. In the one run where the tap landed, the alarm left the system but the stop intent did not reach the app
Tapping Snooze on the alert, follow-up countdown and Live Activity not verified
Custom alert sound audible not verified (no audio check)

Consequence in the code: a stop is detected by observing AlarmManager.alarmUpdates and, at the next activation, from the system state — it does not depend on the stop intent. Snooze does depend on its intent being delivered; if it is not, the alert offers a Snooze button that only stops the alarm. This must be checked on an iPhone before relying on snooze.

# Not verified

  • Any physical iPhone: Lock Screen alert, silent switch, Focus modes, device restart, app terminated by the system, Bluetooth and other audio routes, alarm sound loudness.
  • Camera (QR missions), pedometer (step mission), tilt control, Core Haptics patterns.
  • Audio by ear: the eighteen files were measured (peak, RMS, clipping, loop seam) and three were played for two seconds.
  • VoiceOver, Reduce Motion and large Dynamic Type paths (implemented, not exercised).
  • Widgets on the Home Screen and Lock Screen, the Live Activity, Siri shortcuts (they compile and their logic is shared with tested code; none was seen running).
  • Sleep reminder delivery, import flow, share sheet.
  • Low-memory behaviour.

# Reproducing

bash
cd Packages/AlarmCore && swift test
Scripts/build.sh test
Scripts/build.sh uitest
Scripts/probe-alarmkit.sh testObservedStopRetiresOneTimeAlarm

Launch arguments: -ORIXUITest YES (fake alarm service, isolated storage), -ORIXReset YES, -ORIXDemo YES, -ORIXSkipOnboarding YES, -ORIXDenyAlarms YES, -ORIXTab <tab>, -ORIXMission <kind> -ORIXLevel n -ORIXSeed n, -ORIXSimulateWake YES, -ORIXNightstandDemo YES, -ORIXEditor YES, and in Debug builds -ORIXFakeQR <text>, -ORIXFakeSteps YES, -ORIXProbeAlarm plain|mission|snooze[-autostop].

Files main
Menu

Theme