Release checklist
The quality gates of a release, in the order they are run, each with the command that proves it. A gate is passed when the command exits 0 and prints the stated evidence; a statement in a document is not evidence. All commands are run from the repository root.
Details of the last three stages: signing, notarization, distribution. Test strategy: testing. Build requirements: setup.
flowchart LR
A["1 Source<br/>clean and scanned"] --> B["2 Tests<br/>all packages"]
B --> C["3 Strings and docs<br/>consistent"]
C --> D["4 Release build<br/>signed and verified"]
D --> E["5 Notarized<br/>and stapled"]
E --> F["6 Artifacts<br/>checked"]
F --> G["7 Publication<br/>owner only"]
SVG fallback:
Gates 1 to 4 run offline (after the one-time DuckDB download) and change nothing outside build/,
dist/ and docs/reports/. Gate 5 uploads the application to Apple. Gate 7 is never automated.
1. Source
- Working tree is what will be released.
git status --shortprints nothing (or only files deliberately left out), andgit log --oneline -1is the intended commit. - No secrets in tracked or untracked, non-ignored files.
Scripts/secret-scan.sh→secret scan: clean (<n> files), exit 0. The patterns cover private-key blocks, AWS/Anthropic/OpenAI/GitHub/Slack token shapes, genericpassword|secret|api_key|token = <16+ chars>assignments andnotarytool store-credentials … --password; it also fails on.p8,.p12,.pem,.mobileprovision,.cer,.keychainand.envfiles. Test fixtures,.xcstringscatalogs and documentation lines containing "example" or "placeholder" are excluded. It is a pattern scan, not a proof of absence. - Version is set.
grep -E "MARKETING_VERSION|CURRENT_PROJECT_VERSION" project.ymlshows the version and build number to release. - The project file matches
project.yml.Scripts/build.sh generate→ORIXSheet.xcodeproj generated(also runsScripts/fetch-duckdb.sh, which verifies the pinned SHA-256 when it downloads).
2. Tests
- Every package test suite passes.
Scripts/test-all.sh(orScripts/build.sh test) → last lineTOTAL … 0 failed package(s), exit 0; the machine-readable result is written todocs/reports/tests.json("failedPackages": 0). Per-package logs are inbuild/test-logs/<Package>.log. A package is counted as failed if its log lacks a "passed" summary line or containserror:orsignal code. The on-device model evaluation is opt-in (ORIX_AI_EVAL=1) and is not part of this run. Recorded run: 22 packages, 1,270 test functions,"failedPackages": 0(2026-11-21T08:47:01Z). - Counts are taken from the file, not from memory. Quote test numbers only from
docs/reports/tests.jsonof this run. - Optional, env-gated suites that the default run skips (run when the area changed):
cd Packages/SheetIntelligence && ORIX_AI_EVAL=1 swift test(model evaluation, minutes long, needs the on-device model, rewritesEVALUATION.md);ORIX_DATA_LARGE_TESTS,ORIX_BENCH(see testing and performance for what each enables). - App integration tests.
xcodebuild test -project ORIXSheet.xcodeproj -scheme ORIXSheet -destination 'platform=macOS,arch=arm64'→** TEST SUCCEEDED **. Recorded run of theORIXSheetTeststarget (App/Tests): 9 tests, 0 failures (docs/reports/app-tests.json, 2026-11-21T08:48:35Z). One of the nine asserts that the menu bar has no duplicate key equivalent. - Debug app builds.
Scripts/build.sh debug→** BUILD SUCCEEDED **and noerror:line. (The script filters warnings out of its output.)
3. Strings and documentation
- String catalogs of the shared UI and the app are complete and regenerated.
python3 Scripts/localize.py --check→ one(checked)line per catalog, exit 0. It fails on a key without translation, mismatched format specifiers, a stale catalog or an interpolated key. Observed on 2026-11-21 after the template chooser and the new Settings privacy text were added:sheetui: 346 keys(checked),app: 218 keys,9 problem(s), exit 1 — eight keys ofApp/Sources/TemplateChooser.swiftandApp/Sources/AppSettings.swifthave no fr-CA translation andApp/Resources/Localizable.xcstringsis out of date. This gate is not passed untilScripts/localize.pyis run with the missing translations added. - Feature packages' catalogs. Each feature package has its own string tool
(
Packages/FeatureCharts/Scripts/strings.py,Packages/FeatureAnalysis/Scripts/strings.py --all, tools underPackages/FeatureAudit/Tools,FeatureIntelligence/Tools,FeatureAutomation/Tools). The Charts and Analysis tools exit 1 on a missing or mismatched translation, but they also rewrite their catalog; there is no verify-only mode. Run them, then check thatgit status --shortshows no change. - Documentation links and images resolve.
python3 Scripts/check-docs.py→0 problem(s). - Every diagram has an up-to-date SVG fallback.
python3 docs/assets/diagrams/make_diagrams.py --check→ nostale:orerror:line, exit 0. - Documents quote measured values only. Benchmarks referenced by the documents exist in
docs/benchmarks/and in the packageBENCHMARKS.mdfiles with their machine, build and date.
4. Release build and signature
-
Signed universal build.
Scripts/sign.sh(orScripts/build.sh release) → ends withRESULT: dist/ORIX Sheet.app is built and signed.The script itself enforces, and prints:codesign --verify --deep --strictpasses;- architectures
x86_64 arm64; - authority
Developer ID Application: …, aTimestamp=line,flags=0x10000(runtime); - no
get-task-allowentitlement; - every Mach-O in the bundle signed by team
3YM54G49SNwith the hardened runtime; - the embedded tool answers: a line
-- embedded CLI: orix-sheet <version> (…).
-
Independent re-check of the bundle (does not rebuild):
bashcodesign --verify --deep --strict --verbose=2 "dist/ORIX Sheet.app" codesign -dvv "dist/ORIX Sheet.app" 2>&1 | grep -E "Authority|TeamIdentifier|Timestamp|flags" codesign -d --entitlements - --xml "dist/ORIX Sheet.app" # expect an empty <dict/> -
The bundle contains what the current project links.
ls "dist/ORIX Sheet.app/Contents/Frameworks"listslibduckdb.dylib, andcodesign -dvv "dist/ORIX Sheet.app/Contents/Frameworks/libduckdb.dylib" 2>&1 | grep TeamIdentifierprints3YM54G49SN. -
The command-line tool is embedded, signed and runs.
"dist/ORIX Sheet.app/Contents/MacOS/orix-sheet" versionprints the version line, andcodesign -dvv "dist/ORIX Sheet.app/Contents/MacOS/orix-sheet" 2>&1 | grep -E "TeamIdentifier|flags"prints3YM54G49SNandflags=0x10000(runtime). Scheduled workflows depend on this file (automation). -
The signed app launches and opens a document. Launch
dist/ORIX Sheet.appby hand; the launch arguments used for automated screenshots (-ORIXOpen <file> -ORIXShot <png>) are described in setup. No script asserts this for the Release bundle.
5. Notarization (uploads to Apple; owner's credentials)
-
App and DMG accepted and stapled.
Scripts/notarize.sh→ exit 0 andRESULT: dist/ORIX Sheet.app and dist/ORIXSheet-<version>.dmg are notarized by Apple and stapled.Exit 2 = app not accepted, exit 3 = DMG not accepted; logs inbuild/notary/. -
Tickets validate.
xcrun stapler validate "dist/ORIX Sheet.app"andxcrun stapler validate dist/ORIXSheet-<version>.dmg→The validate action worked! -
Gatekeeper accepts both.
bashspctl --assess --type execute --verbose=2 "dist/ORIX Sheet.app" spctl --assess --type open --context context:primary-signature --verbose=2 dist/ORIXSheet-<version>.dmg→
accepted,source=Notarized Developer ID.
6. Artifacts
- Checksums match.
cd dist && shasum -a 256 -c ORIXSheet-<version>.sha256→OKtwice. - The DMG has the designed layout (background, app at the left, Applications link at the
right). If
package.shprintedcreate-dmg unavailable or failed: building a plain disk image, the fallback image was produced: decide whether that is acceptable. Checked by opening the DMG. - The DMG is signed.
codesign -dvv dist/ORIXSheet-<version>.dmg 2>&1 | grep Authority. - First-launch test from a download on a Mac other than the build machine (quarantine set
by the browser): the app opens without a Gatekeeper warning beyond the standard "downloaded from
the Internet" confirmation. Manual; see
RELEASE-REPORT.mdfor whether it was done. - Manual accessibility pass, if claimed.
docs/development/accessibility.mdlists what was not verified (VoiceOver was not listened to, display options not switched on during a run). Do not describe these as verified unless they have been.
7. Publication (owner's authorization required)
-
Scripts/secret-scan.shrun again immediately before pushing. -
git push origin mainto the private repository (gitsrv:srv/git-private/orix-sheet.git). - Tag, release notes and any download location: decided and performed by the owner. Nothing in the repository automates them; see distribution.
Gates observed on 2026-11-21
| Gate | Command | Result |
|---|---|---|
| Package tests | Scripts/test-all.sh |
22 packages, 1,270 test functions, 0 failed packages (docs/reports/tests.json, 2026-11-21T08:47:01Z) |
| App integration tests | xcodebuild test -project ORIXSheet.xcodeproj -scheme ORIXSheet -destination 'platform=macOS,arch=arm64' |
9 tests, 0 failures (docs/reports/app-tests.json, 2026-11-21T08:48:35Z) |
| Signed universal build | Scripts/sign.sh, then codesign -dvv "dist/ORIX Sheet.app" and codesign --verify --deep --strict |
Developer ID Application, team 3YM54G49SN, flags=0x10000(runtime), x86_64 arm64, timestamp Nov 21, 2026 at 3:50:32 AM, empty entitlements (no get-task-allow), verification passes |
| Bundle content | ls, codesign -dvv, lipo -archs on the nested files |
Contents/MacOS/orix-sheet and Contents/Frameworks/libduckdb.dylib present, universal, signed by the same team with the hardened runtime |
| Secret scan | Scripts/secret-scan.sh |
secret scan: clean (1071 files), exit 0 |
| String catalogs | python3 Scripts/localize.py --check |
fails: sheetui: 346 keys checked, app: 218 keys, 9 problems (8 missing fr-CA translations, app catalog out of date), exit 1 |
| Documentation | python3 Scripts/check-docs.py, python3 docs/assets/diagrams/make_diagrams.py --check |
run after the documentation refresh; see the tools' output |
| Notarization, stapling, checksums, Gatekeeper | Scripts/notarize.sh and the commands of gates 5 and 6 |
see RELEASE-REPORT.md |
Source: docs/reports/tests.json, docs/reports/app-tests.json, and read-only inspection of
dist/ORIX Sheet.app, Scripts/secret-scan.sh and Scripts/localize.py --check run on 2026-11-21
after the release commit 1a190fd.