SPB Git

spb/zyquo-mlx Public MIT

The local MLX foundry for your Mac — run, fine-tune, quantize, and ship models. Nothing leaves your machine.

Swift 93.4% Python 3.8% Makefile 2.2% Shell 0.5%
9.3 KB

# Zyquo MLX — Build Recipe (No Xcode IDE)

Phase 0 deliverable. The Metal-toolchain question was resolved empirically on this machine on 2026-07-30 and cross-checked against upstream sources. This document is the contract for the Phase 1 Makefile.


# 1. The Metal Toolchain Question — Resolved

# 1.1 Empirical test (this Mac)

Host: Apple M5 Max, 48 GB, macOS 27.0, Swift 6.4 (swiftlang-6.4.0.25.4), xcode-select -p/Library/Developer/CommandLineTools, no Xcode.app installed anywhere.

Test: fresh SPM package depending on mlx-swift (resolved 0.31.6), swift build. Result:

  • All ~500 C++/Swift compile steps of Cmlx/MLX/MLXNN succeed with CLT only.
  • The build fails at CompileMetalFile …/Source/Cmlx/mlx-generated/metal/steel/attn/kernels/steel_attention.metal with error: unable to spawn process 'metal' (No such file or directory).
  • The CLT ships no metal binary (verified: nothing metal-related in /Library/Developer/CommandLineTools/usr/bin).

Conclusion: CLT-only builds are impossible today. Only the .metal kernel → default.metallib step needs the Metal toolchain.

# 1.2 Why (verified in mlx-swift Package.swift at main)

Cmlx uses a hybrid kernel strategy: many kernels are runtime-JIT'd (checked -in generated .cpp sources with embedded kernel strings, compiled at runtime via Metal's newLibrary(source:)), but a set of AOT .metal files (arg_reduce, conv, gemv, layer_norm, steel attention/gemm, …) must be compiled at build time into the resource bundle mlx-swift_Cmlx.bundle/default.metallib (SWIFTPM_BUNDLE="mlx-swift_Cmlx", METAL_PATH="default.metallib"). No prebuilt-metallib or binary xcframework distribution exists (upstream work in flight: mlx-swift PR #430 "Build SwiftPM default Metal library resource", mlx#3597 set_metallib_path, mlx-swift#416 GPU.setMetallibPath — none landed; re-check at each dependency bump).

# 1.3 Getting the Metal compiler in the Xcode 26+ era

  • The Metal compiler is a separate ~700 MB downloadable component since Xcode 26: xcodebuild -downloadComponent metalToolchain (verify with xcodebuild -showComponent metalToolchain; CI export/import flags exist: -exportPath / -importComponent … -importPath).
  • xcodebuild refuses to run under CLT ("requires Xcode" — reproduced locally). No standalone Metal Toolchain download exists on developer.apple.com (none found publicly; UNVERIFIED whether one hides behind login).
  • → Full Xcode.app is required. Per the charter (0.A.6), the rule adapts: install Xcode strictly as a toolchain; drive everything via command-line swift build/xcodebuild; never open the IDE; never hand-author an .xcodeproj. All automation lives in the Makefile.

# 2. One-Time Machine Setup

bash
# 1. Install Xcode.app (toolchain only — the IDE is never opened).
#    Mac App Store (mas), or download from developer.apple.com/download.
# 2. Point the toolchain at it:
sudo xcode-select -s /Applications/Xcode.app
sudo xcodebuild -license accept
# 3. Install the Metal toolchain component (Xcode 26+):
xcodebuild -downloadComponent metalToolchain
xcodebuild -showComponent metalToolchain   # verify "installed"
# 4. Sanity check:
xcrun -f metal    # must resolve inside Xcode/Metal toolchain, not fail

Per-command alternative to the global xcode-select (keeps CLT default): DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer <cmd> — the Makefile uses this form so the machine's default toolchain is untouched.

Other host requirements (already present on this Mac): uv (Python venv bootstrap), rsvg-convert + iconutil (icon pipeline, Phase 5), notarytool + stapler (ship with CLT, Phase 8).


# 3. Building the App

# 3.1 Build commands

Primary (officially documented for mlx-swift — README: "Although SwiftPM (command line) cannot build the Metal shaders, xcodebuild can and it can be used to do command line builds"):

bash
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
xcodebuild build \
  -scheme ZyquoMLX \
  -destination 'platform=OS X,arch=arm64' \
  -configuration Release \
  -derivedDataPath .build/xcode \
  -skipMacroValidation

(SPM package schemes are auto-generated from Package.swift — no .xcodeproj is ever created by hand.)

RESOLVED (tested 2026-07-30, Xcode 26.6 + Metal toolchain 17F109):

  • Classic backend swift build -c release: compiles and links fine but silently skips Metal — no default.metallib, no mlx-swift_Cmlx.bundle → broken GPU path at runtime. Do not use.
  • New backend swift build --build-system swiftbuild -c release: fully works — compiles all .metal kernels and emits .build/out/Products/Release/mlx-swift_Cmlx.bundle/Contents/Resources/default.metallib plus the other resource bundles, 0 warnings, ~90 s clean build. This is the primary build command (encoded in the Makefile); xcodebuild remains the documented fallback above. Products land in .build/out/Products/{Release|Debug}/.

# 3.2 Assembling Zyquo MLX.app (Makefile)

text
Zyquo MLX.app/
└── Contents/
    ├── Info.plist                  # CFBundleDisplayName "Zyquo MLX", com.zyquo.mlx,
    │                               # LSMinimumSystemVersion 14.0, arm64 priority,
    │                               # NSHighResolutionCapable, developer-tools category
    ├── MacOS/ZyquoMLX              # release binary
    └── Resources/
        ├── AppIcon.icns
        ├── mlx-swift_Cmlx.bundle/  # REQUIRED — contains default.metallib
        │                           # (missing bundle = GPU ops crash at runtime;
        │                           #  mlx-swift issue #345)
        └── <other SPM resource bundles>   # mlx-swift-lm / swift-transformers
                                           # bundles if produced by the build

The Makefile copies every *.bundle from the build products directory into Contents/Resources/ — this is the classic mlx-swift packaging pitfall. Existence proof that this recipe signs/notarizes/ships: sfomuseum/Docent (discussed with mlx-swift maintainers in issue #345).

# 3.3 Makefile targets (contract for Phase 1)

Target Does
make build debug build (xcodebuild or swift build per §3.1 outcome)
make app release build + assemble Zyquo MLX.app + ad-hoc sign
make dev make app + launch from terminal (proper activation)
make icon SVG → PNGs (rsvg-convert) → .iconseticonutil -c icns
make release Phase 8: hardened-runtime Developer ID sign (nested-first) + notarize + staple
make clean remove .build/ + dist/

# 3.4 Python environment (runtime, not build-time)

The Python side is not part of the app build — it is provisioned at first run into ~/Library/Application Support/ZyquoMLX/py/ (outside the signed bundle, per the notarization fallback already anticipated in the charter):

bash
uv python install 3.12
uv venv "$APP_SUPPORT/py/venv" --python 3.12
uv pip install --python "$APP_SUPPORT/py/venv" \
    mlx-lm==0.31.3 mlx-vlm==0.6.8 mlx-whisper==0.4.3 mlx-audio==0.4.6

Pinned via a checked-in requirements lock (uv pip compile). Python 3.12 chosen deliberately: mlx wheels span cp310–cp314, but 3.12 is the safest ecosystem-wide floor; wheels are macosx_14_0_arm64+ — consistent with the app's Apple-Silicon-only gate. Repair = delete venv + re-provision (uv supports --offline with a local wheel cache). Helper scripts live in PyBridge/scripts/ and are versioned with the app, never fetched remotely.


# 4. Signing & Notarization Posture (informs Phase 8; credentials from zyquo-term)

  • Runtime Metal JIT is not process-JIT: MLX compiles kernels at runtime through the Metal framework (newLibrary(source:)) — GPU shader compilation, not writable-executable process memory. No reports of MLX apps needing com.apple.security.cs.allow-jit or allow-unsigned-executable-memory (none in ml-explore trackers; Apple's LoRATrainingExample entitlements are sandbox + network.client + files read-only + iOS memory-limit only). → Start with Hardened Runtime and NO extra entitlements; add only what a failing notarization proves necessary, and document why here.
  • The mlx-swift_Cmlx.bundle contains no Mach-O code (metallib is data) — the classic failure is omitting the bundle, not signing it. Sign nested frameworks/dylibs first (if any), then the app, then notarize (ditto -c -k --keepParentnotarytool submit --waitstapler staple).
  • Python lives in Application Support (unsigned territory) → the bundled- Python notarization minefield (unsigned .so files) is avoided entirely.
  • No App Sandbox (local ML workbench needing broad file/compute access), per the charter.

# 5. Open Items to Re-Verify During Phase 1

  1. Does plain swift build produce default.metallib with the Metal toolchain installed (Swift 6.4 path)? → record result in §3.1.
  2. Exact set of resource bundles emitted by mlx-swift-lm 3.31.4 / swift-transformers 1.3.x → finalize §3.2 copy list.
  3. mlx-swift PR #430 / #416 status at dependency-bump time (a prebuilt metallib path would remove the Xcode requirement — re-test CLT-only then).