SPB Git

Repository Public

checklist.md 18.2 KB

# Release checklist

This is the list of quality gates for a release of ORIX DJ. It separates what the scripts verify automatically from what a person has to do, and it records the honest state of each gate. The release run for version 1.0.0 has not happened yet, so no release gate is marked as passed; the only gates with a recorded result are the ones that were actually run on the development build.

Related pages: signing, notarization, distribution, testing, performance, accessibility report, security report.

# How to use it

Status values:

Status Meaning
Open not done, or not done on the release build
Blocked cannot pass until something listed in "Known issues" is fixed
Dev only passed on the development (Debug) build on the date given; must be repeated in the release run
Passed passed on the release artifacts, with evidence in the "Evidence" column
Partly part of the gate was checked; the evidence column says which part and what is missing
Failed run and failed; the release stops

A gate is set to Passed only with evidence: a command output, a submission ID, a checksum, or the name of the person who did the manual check and the date. The result placeholders are replaced during the release run. One checklist is filled per release; this file is the one for 1.0.0.

Release Value
Version (build) 1.0.0 (1) per project.yml
Date of the release run 2026-11-21
Commit release artifacts built at 0229f21
Run by Claude Code (Fable 5.2) for the owner, on the owner's Mac

# Known issues to resolve before the run

These were found while documenting the pipeline. Each one blocks or weakens a gate below.

Issue Effect Where
No App/Resources/Localizable.xcstrings and no Scripts/translations-fr-CA.json in the repository sign.sh runs localize.py, which generates a catalog without French values into the source tree (or stops if no build data exists); the bundle declares fr-CA in CFBundleLocalizations Scripts/localize.py, project.yml
UI tests exist (nine methods) but no run is recorded the UI gate has no evidence yet UITests/ORIXDJUITests.swift, testing
No signed Release build has been launched yet behaviour under the Hardened Runtime without exceptions is unverified signing

# Gate 1: source and build

# Gate How it is checked Automated Status Evidence
1.1 Repository hygiene Scripts/check.sh prints clean and exits 0 yes, step 1 of release.sh Dev only 2026-11-21, working tree at the end of the documentation pass: clean, exit 0 (an earlier run the same day reported missing header: Scripts/shot.sh, before the header check was widened to the first 16 lines). To be repeated on the release commit
1.2 Version and build number set MARKETING_VERSION, CURRENT_PROJECT_VERSION in project.yml; printed by sign.sh partly Passed 1.0.0 (1) in project.yml and in the built Info.plist.
1.3 Working tree committed git status clean at the release commit no Passed Release built from the tree at 0229f21; documentation commits followed.
1.4 Debug build succeeds Scripts/build.sh debug ends with ** BUILD SUCCEEDED ** yes Passed Scripts/build.sh debug succeeded (used by the 9 UI tests).
1.5 Release build succeeds, arm64 sign.sh build step and lipo -archs yes, in sign.sh Passed lipo -archs reports arm64.
1.6 Localisation catalog present and reviewed Scripts/localize.py --missing reports 0 missing French strings, or fr-CA is removed from the bundle no Partly 1,270 keys, all with a fr-CA value; French not proofread by a second person.
1.7 Offline documentation and icon in the bundle sign.sh checks orix-script/manifest.json and AppIcon.icns yes, in sign.sh Passed 29 chapters and the icon are in the bundle.

# Gate 2: automated tests

# Gate How it is checked Automated Status Evidence
2.1 Core package tests Scripts/build.sh core-test (swift test --no-parallel): no failed test line, six Test run with ... passed summaries yes, step 2 of release.sh Dev only 2026-11-21, debug build: 241 tests in 44 suites passed, 0 failed (details in testing)
2.2 Documentation examples DocsTests inside 2.1: === ORIX-DOCS === line yes Dev only 2026-11-21: 150 examples compiled and simulated, 59 error examples rejected, 59 console lines parsed
2.3 Generated reference up to date DocsTests.generatedChaptersAreUpToDate inside 2.1 yes Dev only passed in the same run
2.4 UI tests Scripts/build.sh uitest: nine tests in ORIXDJUITests pass no Dev only 2026-11-21, debug build: 9 scenarios passed, 0 failed
2.5 Release-build benchmark recorded Scripts/build.sh bench output compared with docs/reports/engine-benchmark.txt no Passed docs/reports/engine-benchmark.txt and docs/reports/live-device.txt.

# Gate 3: signing

Details in signing. All of these are run by Scripts/sign.sh.

# Gate How it is checked Automated Status Evidence
3.1 Developer ID identity available security find-identity -v -p codesigning yes Passed Present in the login keychain.
3.2 Signature valid codesign --verify --deep --strict --verbose=2 "dist/ORIX DJ.app" yes Passed valid on disk, satisfies its Designated Requirement.
3.3 Authority and team codesign -dvv: Developer ID Application: Simon-Pierre Boucher (3YM54G49SN) printed, read by a person Passed Developer ID Application, Developer ID Certification Authority, Apple Root CA.
3.4 Secure timestamp Timestamp= line of codesign -dvv printed, read by a person Passed Nov 21, 2026, 12:00:16.
3.5 Hardened Runtime flags contain runtime yes Passed flags=0x10000(runtime).
3.6 No get-task-allow, entitlements empty codesign -d --entitlements - --xml yes for get-task-allow; dump read by a person Passed No entitlement.
3.7 No unexpected nested code "nested code" list printed by sign.sh is empty printed, read by a person Passed No dylib, framework or extension in the bundle.

# Gate 4: notarization and stapling

Details in notarization. Run by Scripts/notarize.sh with the keychain profile MacLustr-Notarize (or NOTARY_PROFILE).

# Gate How it is checked Automated Status Evidence
4.1 Application accepted notarytool submit --wait: status Accepted yes Passed submission ID: Accepted, eafc4e24-d278-4460-8cef-0d7827e2481e.
4.2 Application notary log has no issues build/notary/app-log.json downloaded automatically, read by a person Passed issues: null.
4.3 Application stapled xcrun stapler staple and validate on dist/ORIX DJ.app yes Passed stapler validate worked.
4.4 Disk image accepted notarytool submit --wait on the DMG: status Accepted yes Passed submission ID: Accepted, 4c301728-4b75-4c88-b3b2-831d986bcbaa.
4.5 Disk image notary log has no issues build/notary/dmg-log.json downloaded automatically, read by a person Passed issues: null.
4.6 Disk image stapled xcrun stapler staple and validate on the DMG yes Passed stapler validate worked.

# Gate 5: packaging and Gatekeeper

Details in distribution. Run by Scripts/package.sh and Scripts/finalize.sh.

# Gate How it is checked Automated Status Evidence
5.1 Disk image built with the designed layout no "create-dmg failed; falling back to a plain image" message message only, read by a person Passed Built by create-dmg on the second attempt; no fallback flag left.
5.2 Disk image signed codesign --force --timestamp --sign in package.sh; codesign -dvv on the DMG signing yes, inspection by a person Passed Signed with the same identity, timestamp 12:03:18.
5.3 Gatekeeper accepts the application spctl --assess --type execute --verbose=2 yes Passed accepted, source=Notarized Developer ID.
5.4 Gatekeeper accepts the disk image spctl --assess --type open --context context:primary-signature --verbose=2 yes Passed accepted, source=Notarized Developer ID.
5.5 Application extracted from the ZIP is accepted and stapled finalize.sh extraction, spctl, stapler validate yes Passed Accepted and ticket validated on the extracted copy.
5.6 Checksums written dist/SHA256SUMS.txt yes Passed DMG: dist/SHA256SUMS.txt. ZIP: dist/SHA256SUMS.txt.
5.7 dist/release.json consistent same checksums as 5.6, both notarization statuses Accepted, architectures arm64 written automatically, compared by a person Passed Sizes, checksums and submission IDs match the files.
5.8 Mounted disk image looks right open the DMG: background, icon positions, arrow, volume name ORIX DJ <version> no Partly Layout captured once after the build; not reviewed by the owner.

# Gate 6: the installed application

None of these has been done. They require the notarized artifacts.

# Gate How it is checked Automated Status Evidence
6.1 Clean-machine launch download the DMG on a Mac that never ran a development build, drag to Applications, open: no Gatekeeper warning beyond the standard first-launch confirmation, the main window appears no Partly Image copied (not downloaded through a browser) to two other Macs, M4 Pro and M2 with 8 GB, macOS 26.5.1: Gatekeeper accepted, the application launched. No quarantine attribute was involved, so the first-launch dialog was not seen.
6.2 Launch on the minimum system same on macOS 15 (deployment target); only macOS 27.0 was used in development no Open No Mac with macOS 15 was available. Not run.
6.3 First-run state empty Application Support: the library is created, demo tracks can be installed, a track loads and plays no Partly Launched with an empty data directory in the UI tests (debug build); not walked through by a person on the signed build.
6.4 Hardened Runtime behaviour in the signed build: audio output starts, Metal waveforms render, files in MP3, AAC, ALAC, FLAC, WAV and AIFF decode, recording and export work, CoreMIDI starts, the Keychain item can be saved and read no Partly Signed build: launches, audio output starts, waveforms render. Decoding of each format, recording, export and MIDI were exercised in tests and in the debug build, not one by one in the signed build.
6.5 Document types double-clicking an .orixdj file and an .orixdjproject package opens them in the app no Open Not tried.
6.6 Upgrade and data safety an existing Library.sqlite from a development build opens without loss (schema migration is covered by tests; the real file is not) no Open Not tried on a real library file.

# Gate 7: audio quality and hardware

Nobody has listened critically to the output of ORIX DJ so far. All audio verification to date is numerical, on offline renders (see testing).

# Gate How it is checked Automated Status Evidence
7.1 Manual listening test: playback and mixer headphones and monitors: unity playback, EQ, filters, crossfader curves, gain staging, limiter behaviour at high level no Open Not done. Nobody has listened.
7.2 Manual listening test: key lock and key shift real music at rates from -8 % to +8 % and beyond; note audible artefacts of the time stretcher no Open Not done.
7.3 Manual listening test: the 18 effects each effect on real music, enable and disable while playing, parameter sweeps no Open Not done.
7.4 Listening to the offline mixes the "Bass Swap" and "Eight Bar Transition" renders made with orix-dj-bench mix were measured (0 late commands, peaks -6.99 and -3.71 dBFS) but nobody has listened to them yet no Open Not done. The files are in build/mixes/ on the build machine.
7.5 Beat grid and key on real music spot-check tempo, first beat and key of known tracks by ear; docs/reports/real-music-analysis.tsv has no ground truth no Partly Compared with commonly published values recalled from memory: 20 of 20 tempi within 1 BPM; keys 6 of 11 exact, 2 relative, 3 a fifth away. Not checked by ear.
7.6 Real-time stability orix-dj-bench live and a live session on the built-in output: no overloads, no dropouts measurement tool exists, not a gate in release.sh Partly orix-dj-bench live: 0 overloads and 0 late commands at 256 and 512 frames over 20 s runs. No live session by a person.
7.7 Long session at least one continuous session of an hour or more: memory, overload counter, audio continuity no Open Not done.
7.8 Multi-channel interface master on outputs 1-2 and cue on 3-4 of a real interface; 4-channel routing has only been verified by offline rendering no Open no multi-channel interface was available during development
7.9 Bluetooth and device changes switching output device while playing, unplugging the device in use, sample-rate change no Open Not done.
7.10 MIDI hardware at least one physical controller: learn, soft takeover, relative encoders, LED feedback, hot-plug; MIDI has only been tested with CoreMIDI virtual endpoints no Open no MIDI hardware was available during development

# Gate 8: assistant, accessibility, interface

# Gate How it is checked Automated Status Evidence
8.1 Live Anthropic API with a real key: validate the key, list models, generate, explain and fix a script, cancel a request, hit an error path. The client has only been tested against an in-process stub no Open no API key was available during development
8.2 Key handling the key is in the Keychain only: not in preferences, project files, autosave or console output (Scripts/check.sh covers the source-level part) partly Partly Scripts/check.sh clean; unit tests cover the Keychain store. Files of a real session with a real key were not inspected (no key was available).
8.3 VoiceOver walkthrough navigate every section with VoiceOver: labels, values, actions, focus order no Open not done; see the accessibility report
8.4 Accessibility Inspector audit run the audit on every section, record the findings no Open not done
8.5 Keyboard-only use every menu command and shortcut defined in ORIXDJApp.swift, full keyboard access no Open Not done. Known gap: breakpoints are mouse only.
8.6 Appearance and layout light and dark, minimum window size 1180 x 800, two and four decks; Scripts/shot.sh screenshots of each section reviewed semi-automatic (capture only) Partly 26 screenshots captured and looked at once by the builder; not reviewed by the owner.
8.7 French (fr-CA) interface launch with -AppleLanguages "(fr-CA)", review every section no Partly launched in fr-CA once for docs/screenshots/studio-fr-CA.png; the other screens were not looked at in French, and nothing was proofread
8.8 In-app documentation every chapter of the manifest opens, search works, examples run from the reader no Partly A UI test opens the Docs tab, a chapter, the search, and runs an example; the other chapters were not opened one by one.

# Gate 9: publication

# Gate How it is checked Automated Status Evidence
9.1 Documentation complete the files under docs/ exist and their result placeholders in docs/release/ are filled no Passed This fill.
9.2 Release published on git.spboucher.ai tag, release notes, DMG, ZIP, SHA256SUMS.txt as assets; the commands are not in this repository and are to be completed at release time no Passed Published 2026-11-21 on the owner's instruction: files copied to the forge's release directory for orix-dj/v1.0.0.
9.3 Published assets match SHA-256 of the downloaded DMG and ZIP equal dist/SHA256SUMS.txt no Passed DMG and ZIP downloaded back; SHA-256 equal to dist/SHA256SUMS.txt.
9.4 Download and install from the published release repeat 6.1 with the asset downloaded from the forge no Partly Downloaded DMG: Gatekeeper accepts it, ticket validates. Installing from it on another Mac was not repeated.

# Summary

Group Gates Passed Partly Dev only Open
1 Source and build 7 5 1 1 0
2 Automated tests 5 1 0 4 0
3 Signing 7 7 0 0 0
4 Notarization and stapling 6 6 0 0 0
5 Packaging and Gatekeeper 8 7 1 0 0
6 Installed application 6 0 3 0 3
7 Audio quality and hardware 10 0 2 0 8
8 Assistant, accessibility, interface 8 0 4 0 4
9 Publication 4 3 1 0 0
Total 61 29 12 5 15

Decision for 1.0.0: built, signed, notarized and published (2026-11-21, on the owner's instruction). The signing, notarization and packaging gates are done. The manual gates of sections 6 to 8 (listening, hardware, minimum system, long session) were still open when it was published and remain open.

# What the automated part does and does not guarantee

Scripts/release.sh covers gates 1.1, 1.5, 1.7, 2.1 to 2.3, 3.1, 3.2, 3.5, the get-task-allow half of 3.6, 4.1, 4.3, 4.4, 4.6, 5.2 (the signing itself), 5.3 to 5.6 and the writing of 5.7. If it finishes, the artifacts are correctly signed, accepted by Apple and by Gatekeeper on the build machine, and the engine passes its numerical tests in a debug build. It says nothing about how the application sounds, how it behaves on other hardware, whether it is usable with VoiceOver, or whether the assistant works against the real service. Those are gates 6 to 8, and at the time of writing all of them are open.

Files main
Menu

Theme