spb/zyquo-mlx Public MIT

1<!--2  BUILD.md3  Zyquo MLX45  Author: Simon-Pierre Boucher6  Mail: contact@spboucher.ai7-->89# Zyquo MLX — Build Recipe (No Xcode IDE)1011> Phase 0 deliverable. The Metal-toolchain question was **resolved empirically12> on this machine on 2026-07-30** and cross-checked against upstream sources.13> This document is the contract for the Phase 1 `Makefile`.1415---1617## 1. The Metal Toolchain Question — Resolved1819### 1.1 Empirical test (this Mac)2021Host: Apple M5 Max, 48 GB, macOS 27.0, Swift 6.422(swiftlang-6.4.0.25.4), `xcode-select -p` →23`/Library/Developer/CommandLineTools`, no Xcode.app installed anywhere.2425Test: fresh SPM package depending on `mlx-swift` (resolved 0.31.6),26`swift build`. Result:2728- All ~500 C++/Swift compile steps of `Cmlx`/`MLX`/`MLXNN` **succeed** with29  CLT only.30- The build **fails** at31  `CompileMetalFile …/Source/Cmlx/mlx-generated/metal/steel/attn/kernels/steel_attention.metal`32  with `error: unable to spawn process 'metal' (No such file or directory)`.33- The CLT ships no `metal` binary (verified: nothing metal-related in34  `/Library/Developer/CommandLineTools/usr/bin`).3536**Conclusion: CLT-only builds are impossible today.** Only the `.metal`37kernel → `default.metallib` step needs the Metal toolchain.3839### 1.2 Why (verified in mlx-swift `Package.swift` at main)4041`Cmlx` uses a hybrid kernel strategy: many kernels are runtime-JIT'd (checked42-in generated `.cpp` sources with embedded kernel strings, compiled at runtime43via Metal's `newLibrary(source:)`), **but** a set of AOT `.metal` files44(arg_reduce, conv, gemv, layer_norm, steel attention/gemm, …) must be compiled45at build time into the resource bundle `mlx-swift_Cmlx.bundle/default.metallib`46(`SWIFTPM_BUNDLE="mlx-swift_Cmlx"`, `METAL_PATH="default.metallib"`).47No prebuilt-metallib or binary xcframework distribution exists (upstream work48in flight: mlx-swift PR #430 "Build SwiftPM default Metal library resource",49mlx#3597 `set_metallib_path`, mlx-swift#416 `GPU.setMetallibPath` — none50landed; re-check at each dependency bump).5152### 1.3 Getting the Metal compiler in the Xcode 26+ era5354- The Metal compiler is a separate ~700 MB downloadable component since55  Xcode 26: `xcodebuild -downloadComponent metalToolchain`56  (verify with `xcodebuild -showComponent metalToolchain`; CI export/import57  flags exist: `-exportPath` / `-importComponent … -importPath`).58- `xcodebuild` **refuses to run under CLT** ("requires Xcode" — reproduced59  locally). No standalone Metal Toolchain download exists on60  developer.apple.com (*none found publicly; UNVERIFIED whether one hides61  behind login*).62- **→ Full Xcode.app is required.** Per the charter (0.A.6), the rule adapts:63  **install Xcode strictly as a toolchain; drive everything via command-line64  `swift build`/`xcodebuild`; never open the IDE; never hand-author an65  `.xcodeproj`.** All automation lives in the `Makefile`.6667---6869## 2. One-Time Machine Setup7071```bash72# 1. Install Xcode.app (toolchain only — the IDE is never opened).73#    Mac App Store (mas), or download from developer.apple.com/download.74# 2. Point the toolchain at it:75sudo xcode-select -s /Applications/Xcode.app76sudo xcodebuild -license accept77# 3. Install the Metal toolchain component (Xcode 26+):78xcodebuild -downloadComponent metalToolchain79xcodebuild -showComponent metalToolchain   # verify "installed"80# 4. Sanity check:81xcrun -f metal    # must resolve inside Xcode/Metal toolchain, not fail82```8384Per-command alternative to the global `xcode-select` (keeps CLT default):85`DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer <cmd>` — the86Makefile uses this form so the machine's default toolchain is untouched.8788Other host requirements (already present on this Mac): `uv` (Python venv89bootstrap), `rsvg-convert` + `iconutil` (icon pipeline, Phase 5),90`notarytool` + `stapler` (ship with CLT, Phase 8).9192---9394## 3. Building the App9596### 3.1 Build commands9798Primary (officially documented for mlx-swift — README: "Although SwiftPM99(command line) cannot build the Metal shaders, **xcodebuild can** and it can100be used to do command line builds"):101102```bash103DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \104xcodebuild build \105  -scheme ZyquoMLX \106  -destination 'platform=OS X,arch=arm64' \107  -configuration Release \108  -derivedDataPath .build/xcode \109  -skipMacroValidation110```111112(SPM package schemes are auto-generated from `Package.swift` — no113`.xcodeproj` is ever created by hand.)114115**RESOLVED (tested 2026-07-30, Xcode 26.6 + Metal toolchain 17F109):**116117- Classic backend `swift build -c release`: compiles and links fine but118  **silently skips Metal** — no `default.metallib`, no `mlx-swift_Cmlx.bundle`119  → broken GPU path at runtime. Do not use.120- **New backend `swift build --build-system swiftbuild -c release`: fully121  works** — compiles all `.metal` kernels and emits122  `.build/out/Products/Release/mlx-swift_Cmlx.bundle/Contents/Resources/default.metallib`123  plus the other resource bundles, 0 warnings, ~90 s clean build.124  **This is the primary build command** (encoded in the Makefile);125  `xcodebuild` remains the documented fallback above. Products land in126  `.build/out/Products/{Release|Debug}/`.127128### 3.2 Assembling `Zyquo MLX.app` (Makefile)129130```131Zyquo MLX.app/132└── Contents/133    ├── Info.plist                  # CFBundleDisplayName "Zyquo MLX", com.zyquo.mlx,134    │                               # LSMinimumSystemVersion 14.0, arm64 priority,135    │                               # NSHighResolutionCapable, developer-tools category136    ├── MacOS/ZyquoMLX              # release binary137    └── Resources/138        ├── AppIcon.icns139        ├── mlx-swift_Cmlx.bundle/  # REQUIRED — contains default.metallib140        │                           # (missing bundle = GPU ops crash at runtime;141        │                           #  mlx-swift issue #345)142        └── <other SPM resource bundles>   # mlx-swift-lm / swift-transformers143                                           # bundles if produced by the build144```145146The Makefile copies **every** `*.bundle` from the build products directory147into `Contents/Resources/` — this is the classic mlx-swift packaging pitfall.148Existence proof that this recipe signs/notarizes/ships: sfomuseum/Docent149(discussed with mlx-swift maintainers in issue #345).150151### 3.3 Makefile targets (contract for Phase 1)152153| Target | Does |154|---|---|155| `make build` | debug build (xcodebuild or swift build per §3.1 outcome) |156| `make app` | release build + assemble `Zyquo MLX.app` + ad-hoc sign |157| `make dev` | `make app` + launch from terminal (proper activation) |158| `make icon` | SVG → PNGs (`rsvg-convert`) → `.iconset` → `iconutil -c icns` |159| `make release` | Phase 8: hardened-runtime Developer ID sign (nested-first) + notarize + staple |160| `make clean` | remove `.build/` + `dist/` |161162### 3.4 Python environment (runtime, not build-time)163164The Python side is **not** part of the app build — it is provisioned at first165run into `~/Library/Application Support/ZyquoMLX/py/` (outside the signed166bundle, per the notarization fallback already anticipated in the charter):167168```bash169uv python install 3.12170uv venv "$APP_SUPPORT/py/venv" --python 3.12171uv pip install --python "$APP_SUPPORT/py/venv" \172    mlx-lm==0.31.3 mlx-vlm==0.6.8 mlx-whisper==0.4.3 mlx-audio==0.4.6173```174175Pinned via a checked-in requirements lock (`uv pip compile`). Python 3.12176chosen deliberately: mlx wheels span cp310–cp314, but 3.12 is the safest177ecosystem-wide floor; wheels are `macosx_14_0_arm64`+ — consistent with the178app's Apple-Silicon-only gate. Repair = delete venv + re-provision (uv179supports `--offline` with a local wheel cache). Helper scripts live in180`PyBridge/scripts/` and are versioned with the app, never fetched remotely.181182---183184## 4. Signing & Notarization Posture (informs Phase 8; credentials from zyquo-term)185186- **Runtime Metal JIT is not process-JIT**: MLX compiles kernels at runtime187  through the Metal framework (`newLibrary(source:)`) — GPU shader188  compilation, not writable-executable process memory. No reports of MLX apps189  needing `com.apple.security.cs.allow-jit` or190  `allow-unsigned-executable-memory` (none in ml-explore trackers; Apple's191  LoRATrainingExample entitlements are sandbox + network.client + files192  read-only + iOS memory-limit only).193  **→ Start with Hardened Runtime and NO extra entitlements; add only what a194  failing notarization proves necessary, and document why here.**195- The `mlx-swift_Cmlx.bundle` contains no Mach-O code (metallib is data) —196  the classic failure is *omitting* the bundle, not signing it. Sign nested197  frameworks/dylibs first (if any), then the app, then notarize198  (`ditto -c -k --keepParent` → `notarytool submit --wait` → `stapler staple`).199- Python lives in Application Support (unsigned territory) → the bundled-200  Python notarization minefield (unsigned `.so` files) is avoided entirely.201- No App Sandbox (local ML workbench needing broad file/compute access),202  per the charter.203204---205206## 5. Open Items to Re-Verify During Phase 12072081. Does plain `swift build` produce `default.metallib` with the Metal209   toolchain installed (Swift 6.4 path)? → record result in §3.1.2102. Exact set of resource bundles emitted by mlx-swift-lm 3.31.4 /211   swift-transformers 1.3.x → finalize §3.2 copy list.2123. mlx-swift PR #430 / #416 status at dependency-bump time (a prebuilt213   metallib path would remove the Xcode requirement — re-test CLT-only then).214