SPB Git

Repository Public

technology-evaluation.md 29.0 KB

# Technology Evaluation

This document records the technical choices behind ORIX DJ 1.0: what was considered for each subsystem, what was chosen, why, what the choice costs, and what remains a risk. It is written after the fact from the source code, so "chosen" always means "this is what the code does", with a pointer to the file.

Numbers come from two measured reports, engine-benchmark.txt (release build) and real-music-analysis.tsv, and from the test run of 2026-11-21. Alternatives were evaluated on paper (API surface, licence, fit with the constraints below); none of the rejected alternatives was built or benchmarked, and this document does not claim measurements for them.

# Environment audit

Audited on the build machine:

Item Value
Operating system macOS 27.0 (build 26A5425a)
Machine Apple M5 Max, 18 cores, 48 GB
Xcode 27.2
Swift 6.4.2 (package and application compiled in Swift 6 language mode, strict concurrency complete)
SDK macOS 27.2
Deployment target macOS 15.0
Architecture arm64 only
Audio devices present MacBook Pro speakers (built-in, 2 channels, 48 kHz), AirPods Max (Bluetooth), a virtual Teams device
MIDI hardware present None
Anthropic API key None available
Signing Developer ID identity "Developer ID Application: Simon-Pierre Boucher (3YM54G49SN)" and a notarytool keychain profile exist on the machine

Consequences of this environment, which recur throughout the documentation:

  • No multi-channel audio interface was attached. Routing the cue bus to a second output pair was verified only by offline rendering into four channels.
  • No MIDI controller was attached. MIDI was exercised only through CoreMIDI virtual endpoints.
  • The Anthropic client was exercised only against a local stub (an in-process URLProtocol that answers in the API's wire format), never the live service.
  • Nobody has listened critically to the audio path. Every statement about audio quality below is a measurement on a synthetic signal, not a listening result.

# Constraints that shaped the decisions

  1. One render path with sample-accurate control. The automation language schedules actions on exact samples, so the engine must be able to apply a parameter change in the middle of a buffer.
  2. Testable without hardware. The same render code must run offline so that audio behaviour can be asserted in unit tests.
  3. No third-party code in the product. This removes licence obligations and supply-chain risk, at the price of writing and maintaining the DSP in-house.
  4. Bounded, deterministic automation. A script must not be able to stall the application, reach the file system or the network, or behave differently in the simulator and live.

# Audio input and output

Options. (a) An AVAudioEngine graph: player nodes, mixer nodes, effect nodes, with the system scheduling and connecting them. (b) A single AUHAL output unit (kAudioUnitSubType_HALOutput) whose render callback pulls an in-house render core.

Chosen: (b), AUHAL with our own render core (AudioOutput.swift, RenderCore.swift).

Reasons:

  • Sample-accurate events. RenderCore renders in chunks of at most 64 frames and shortens a chunk so that it ends exactly at the next timed event. With a node graph, parameter changes are applied by the nodes on their own schedule, and making several nodes change on the same sample is not something the graph API offers directly.
  • Offline determinism. DJEngine.renderOffline calls the same RenderCore.render that the device calls. The tests and the benchmark tool therefore exercise the shipping render path, not a simulation of it.
  • One place for real-time discipline. All render-thread code is in a few files that can be reviewed for allocation and locking.
  • Direct control of device, sample rate, buffer size and channel mapping (master pair, cue pair, split cue).

Costs and implications:

  • Latency. The engine adds no buffering of its own between the render core and the device. The latency shown in the application is the device-reported latency (device latency, safety offset and stream latency) plus one buffer. No live latency measurement with an audio interface was taken. The master limiter delays the signal by its look-ahead (5 ms, LimiterEffect), which is not included in the displayed figure; see mixer. The time stretcher reads ahead in the decoded track rather than delaying the output; see deck engine.

  • CPU. Measured offline on the build machine at 48 kHz: four decks with key lock, 14 effects and the limiter cost a mean of 61.2 µs per 256-frame buffer against a budget of 5333 µs (1.15 % load); two decks with EQ and filter cost 10.8 µs (0.20 %). The offline maxima (up to 396.8 µs) include scheduler preemption because the benchmark does not run on a real-time thread. Full table in performance.

  • Memory. Everything the render thread reads is in memory. Tracks are decoded completely into planar 32-bit float, stereo, at the file's own sample rate (TrackBuffer). That is 2 channels x 4 bytes x sample rate x 60:

    Sample rate Bytes per minute Five-minute track 150-minute cap
    44.1 kHz 21,168,000 (about 20.2 MiB) about 101 MiB about 3.0 GiB
    48 kHz 23,040,000 (about 22.0 MiB) about 110 MiB about 3.2 GiB
    96 kHz 46,080,000 (about 43.9 MiB) about 220 MiB about 6.4 GiB

    TrackBuffer.maxSeconds is 150.0 * 60; a longer file is refused with TrackLoadError.tooLong. Four ordinary five-minute tracks at 44.1 kHz occupy about 404 MiB. Mono files are duplicated to two channels. A file with more than two channels contributes only its first two.

  • Everything else is ours to build. Device changes, sample-rate changes and error recovery are handled in AudioOutput.swift and DJEngine.swift instead of by the system.

Risks: device-change recovery and Bluetooth devices have had only informal use during development; aggregate devices and interfaces with more than two channels have not been tried. There is no audio input at all.

Alternative kept in mind: AVAudioEngine remains the simpler choice for an application that does not need sample-accurate scripted control, and its manual rendering mode would also allow offline tests.

# Time stretching (key lock and pitch shift)

Options.

Option Licence Assessment
In-house WSOLA (waveform-similarity overlap-add) Own code Simple, cheap, runs inside the deck voice with no allocation; known weaknesses on transients and dense polyphony
AUNewTimePitch / AVAudioUnitTimePitch System No code to write, but an Audio Unit in the path with its own latency and buffering, harder to drive sample-accurately and to run deterministically offline
Rubber Band GPL or commercial licence Mature phase-vocoder family; licence incompatible with constraint 3 unless purchased
SoundTouch LGPL Mature WSOLA-style library; LGPL obligations and a C++ dependency
Signalsmith Stretch MIT Permissive licence, modern design; still a third-party C++ dependency

Chosen: in-house WSOLA (TimeStretcher in DeckVoice.swift).

How it works, briefly: Hann-windowed frames of 2048 samples (4096 above 60 kHz engine rate) with 50 % overlap; when the natural continuation of the previous frame is not within a quarter frame of the ideal analysis position, a normalised cross-correlation search over that tolerance picks the best-matching position. A Catmull-Rom resampler after the stretcher applies the pitch factor. Details in deck engine.

Measured (benchmark report, 48 kHz; the report's heading reads "440 Hz sine and 120 BPM clicks", but the figures below are computed on the sine only, as measureStretch in Benchmarks.swift shows):

Rate Measured pitch Error Residual below the tone Playhead speed
0.84 440.002 Hz +0.01 cents 40.5 dB 0.8396
0.92 440.014 Hz +0.06 cents 29.0 dB 0.9213
0.96 440.015 Hz +0.06 cents 46.5 dB 0.9612
1.00 440.000 Hz +0.00 cents 99.5 dB 1.0000
1.04 439.985 Hz -0.06 cents 160.2 dB 1.0388
1.08 439.989 Hz -0.04 cents 25.6 dB 1.0804
1.16 440.000 Hz +0.00 cents 37.9 dB 1.1584

Reading these numbers honestly:

  • Pitch is held within 0.06 cents on a steady sine at every tested rate.
  • The residual (everything that is not the tone) varies widely with rate, from 25.6 dB to 160.2 dB below the tone. A residual only 25 to 29 dB below the tone, as at 0.92 and 1.08, is not negligible. The very high figure at 1.04 should be read as a favourable alignment of this particular test tone at that rate, not as a general result; the cause was not investigated.
  • A sine and clicks are the easiest material for WSOLA. The known artefacts of the technique are doubled or smeared transients, and roughness on dense polyphonic material where no single alignment suits all voices. These have not been evaluated by listening on real music. Two offline automated mixes of real tracks with key lock were rendered (see performance); they analyse at the expected tempo, and nobody has listened to them yet.

Cost: two decks with key lock render in a mean of 12.7 µs per 256-frame buffer, against 10.8 µs without.

Risk and fallback: if listening shows the stretcher is not acceptable on typical material, the least invasive replacement inside the constraints is a better in-house algorithm (for example transient-aware WSOLA or a phase vocoder with phase locking). A permissively licensed library such as Signalsmith Stretch would be the pragmatic alternative and would mean relaxing constraint 3.

# Resampling

Options. Linear interpolation; cubic (Catmull-Rom) interpolation; windowed-sinc polyphase resampling; the system converter (AudioConverter).

Chosen: four-point Catmull-Rom cubic interpolation (DeckVoice.cubic), used both for varispeed playback and after the time stretcher, and to convert between the file's sample rate and the engine's.

It is cheap, has no state, and works at any continuously changing ratio, which is what pitch faders, sync nudges and brakes require.

Stated limitation: there is no anti-aliasing filter. When playback is faster than the source (pitch up, or a file at a higher sample rate than the engine), content above the new Nyquist frequency folds back into the audible band. At typical DJ pitch ranges of a few percent the folded energy comes from the top of the spectrum, where most music carries little, but this has not been measured or judged by ear. Cubic interpolation also has a gentle high-frequency roll-off and imperfect image rejection compared with a sinc resampler. A band-limited resampler is the obvious improvement and is not implemented.

# Beat detection

Options.

Option Assessment
Onset strength by spectral flux, tempo by autocorrelation and comb salience, one constant-tempo grid Classical, deterministic, fast, no model file; assumes the tempo is constant
Dynamic-programming beat tracking (per-beat positions that follow tempo drift) Follows live or drifting tempo; produces a list of beats rather than a grid, which then needs a variable-grid data model and tooling
Neural beat and downbeat trackers Best published accuracy on difficult material; needs a trained model, an inference runtime, and either third-party weights (licence) or a training effort

Chosen: spectral flux plus autocorrelation and comb analysis, with a constant-tempo grid (Onset.swift, Tempo.swift, BeatGrid.swift).

Reasons for a constant-tempo grid: sync, quantise, loops in beats and the automation language's beat-time scheduling all need a conversion between seconds and beats that is cheap and exact. A grid defined by one BPM, one phase anchor and a downbeat index gives that with a single formula. It matches electronically produced music, which is the main target. The price is stated plainly: tracks with tempo drift, live drummers or tempo changes cannot be gridded correctly; the analyser reports a tempo-stability figure so that such tracks can at least be recognised.

Measured:

  • Synthetic material (test suite, 40 tracks in four styles over ten tempi, default range 70 to 180 BPM): 36 of 40 tempo-correct within 1 %, the other four are octave errors (three double-time, one half-time), none unrelated.
  • Real music (60 commercial tracks, range 78 to 156 BPM): for the 20 tracks with a commonly published tempo, all 20 measured tempi are within 1 BPM of it (18 within 0.5 BPM, two exactly 1 BPM away), with no octave error. The reference tempi were recalled from public listings and not independently verified; the table is in analysis.
  • Range. DJSession.analysisOptions sets bpmRange = 78 ... 156. With the analyser's default 70 to 180 range, several real tracks at 84 to 90 BPM were read at double tempo (for example 168, 177 and 181). A range narrower than one octave at the top removes that ambiguity by construction, at the price of folding genuinely fast or slow music (for example 170 BPM drum and bass) into the range.
  • Speed: a five-minute 44.1 kHz stereo track is fully analysed (tempo, grid, key, loudness, waveform) in 0.294 s in a release build; the 60 real tracks (12,449 s of audio) were analysed in 32.0 s.

Risk: octave errors are inherent to this family of methods; the range setting is a convention, not a solution.

# Key detection

Options. Chroma features matched against key profiles (Krumhansl-Kessler or later profile sets); learned classifiers.

Chosen: a global chromagram from spectral peaks with tuning estimation, correlated with Krumhansl-Kessler probe-tone profiles extended with overtone contributions; output as a key name and a Camelot code (Key.swift).

Same reasoning as for beats: deterministic, no model file, no third-party weights. On synthetic material in the test suite 71 of 72 keys are exact and the remaining one is the relative key. On real music the agreement is lower: of the 11 tracks with a commonly published key (recalled from public listings, not independently verified), 6 match exactly, 2 are the relative major or minor and 3 are a fifth away. Across the 60 tracks the key confidence is often low (12 results below 0.10). The comparison and its caveats are in analysis. A single global key also cannot describe tracks that modulate.

# Waveform rendering

Options. (a) SwiftUI Canvas or Core Graphics paths redrawn as the track scrolls. (b) A Metal view with per-frame geometry. (c) A Metal fragment shader over a precomputed summary texture.

Chosen: (c) (WaveformRenderer.swift). Analysis produces a three-band summary plus a full-band peak, one byte each, at 150 columns per second (Waveform.swift). The renderer uploads it once per track as a four-channel texture (4096 texels wide, wrapped over rows) and draws each frame as one full-screen pass; the bands, beat grid, cue points, loop region and playhead are computed in the fragment shader from uniforms. There is no per-frame geometry and the raw audio never reaches the GPU.

Cost: the summary of a five-minute track is 45,000 columns x 4 bytes = 180 kB. The per-frame CPU work is filling one uniform structure. GPU time and frame pacing were not measured; the claim here is structural (one draw call, one small texture), not a benchmark.

Why not Canvas or Core Graphics: a scrolling waveform changes every pixel every frame, which means rebuilding and rasterising paths on the CPU on the main thread at display rate for up to four decks. The full-track overview, which is static apart from markers, is drawn with a SwiftUI Canvas. If Metal is unavailable the scrolling waveform stays empty; there is no fallback renderer. Details in waveform.

# Effects

Options. In-house DSP units; hosting Audio Units (v2 or v3).

Chosen: in-house DSP (ORIXDSP): 18 effect kinds built on shared primitives, all subclasses of EffectUnit, which provides parameter storage, click-free enable and bypass, and wet/dry mixing.

Reasons: beat-synchronised effects need the engine's beat position on every block, every parameter must be addressable from the script schema with a known range and unit, and the units must be allocation-free on the render thread and renderable offline for tests. A single catalogue table (EffectCatalog.swift) generates the rack interface, the script schema, the reference documentation and the model context.

Audio Unit hosting is not implemented. Hosting third-party plug-ins brings plug-in windows, state persistence, latency compensation, sandboxing questions and code on the render thread that the project does not control. It is the most visible functional gap relative to established DJ software.

Cost per effect, measured: between 1.96 µs (bitcrusher) and 36.31 µs (reverb) per 512-frame block at 48 kHz; table in DSP. Quality has been checked only against the objective criteria written in the unit tests (see testing), not by listening.

# Persistence

Options. SwiftData or Core Data; raw SQLite through its C API.

Chosen: raw SQLite behind a small wrapper (SQLiteDatabase.swift) and a store (LibraryStore.swift).

Reasons:

  • The library lives in the core package, which has no SwiftUI and no main-actor assumptions. LibraryStore is used from background queues (import, analysis) and from the interface; an explicit connection with its own locking is simpler to reason about there than model contexts.
  • Explicit schema and migrations: PRAGMA user_version with an append-only list of migration steps, tested by MigrationTests.
  • Smart crates compile to parameterised SQL predicates (SmartCrateSQL.swift); user text is only ever bound, never interpolated.
  • Control over durability and recovery: WAL journal mode, synchronous = NORMAL, foreign keys on, and a quick_check at open with a recovery path for a corrupt file (CorruptionTests).
  • SQLite ships with macOS, so constraint 3 holds.

Cost: hand-written SQL and row mapping, and no automatic CloudKit sync (which is not a goal). Projects are separate .orixdjproject packages. See persistence.

# Automation language

Options.

Option Assessment
JavaScriptCore A complete general-purpose language with a system runtime. Unbounded loops, recursion and allocation by default; time and determinism are not part of the language; every safety property would have to be enforced around it
Lua (embedded) Small and fast, but third-party C code (constraint 3) with the same unboundedness
Own language and interpreter Must be designed, implemented and documented; can be exactly as powerful as needed and no more

Chosen: an own language, ORIX Script (ORIXScript): lexer, recursive-descent parser, a compiler that performs name resolution, type and unit checking, range and capability validation, loop-bound and recursion checks, and lowering to a plan; a cooperative runtime in beat time; a simulator and debugger that reuse the runtime.

Reasons:

  • Determinism. The runtime's only clock is beat time supplied by its host. The same plan run against the live engine or the simulated rig produces the same sequence of actions.
  • Bounded execution. Limits are part of the language (CompileLimits, RuntimeLimits): a script that could run away is rejected at compile time or stopped by a budget at run time, without affecting audio because the runtime never executes on the render thread.
  • No host access. The language has no way to name a file, a URL, a process or an arbitrary host object. Its whole world is the schema of decks, mixer, effects, events and triggers.
  • Units in the type system. 4bars, 1/4beat, -12dB, 200Hz are checked at compile time, which removes a class of mistakes a general-purpose language would only show at run time.
  • Safe target for generated code. Model output can be compiled, budget-checked and simulated before a person decides to run it.

Cost: the language is one of the largest bodies of code in the package (about 4,600 lines), and users must learn it. The mitigation is the bundled documentation, 24 presets written in the language, a visual block builder that generates it, and tests that compile and simulate every documentation example. See automation and the security report.

Measured: in the benchmark's offline run, 3750 scheduler ticks had a mean cost of 2.5 µs (maximum 343.9 µs), and 687 timed commands were all applied on their target sample (none late, none dropped).

# Anthropic integration

Options. An official SDK (none exists for Swift); a third-party community client (constraint 3); raw HTTPS against the Messages API.

Chosen: raw HTTPS with URLSession (AnthropicClient.swift), streaming with server-sent events.

What the client actually sends, read from the code:

Item Value
Base URL https://api.anthropic.com
Endpoints GET /v1/models (query limit=100, paginated with after_id, at most 10 pages), used to list models and to validate a key; POST /v1/messages
Headers on every request x-api-key: <key>, anthropic-version: 2026-10-21, content-type: application/json
Optional header anthropic-beta: server-side-fallback-2026-07-01, only when the refusal fallback is enabled and the selected model is in the client's fallback list
Body fields model, max_tokens, stream: true, system (one text block with cache_control of type ephemeral), messages (role and plain-text content)
Optional body fields output_config.effort when the user picked an effort level; fallbacks: "default" together with the beta header
Not sent Sampling parameters, tools, thinking, metadata, attachments
Defaults (AnthropicSettings) model claude-opus-5-5, max_tokens 16000, at most 60 requests per hour (enforced locally), confirm before sending on, refusal fallback on
Stream events parsed message_start (model, input and cache token counts), content_block_delta with text_delta, message_delta (stop reason, refusal category, output tokens), error; every other event is ignored
Session Ephemeral configuration (no cookies, cache or credential storage), 120 s request timeout, 900 s resource timeout
Retries At most two, with 1.5 s then 3 s delay, only for overloaded, network and 5xx errors; one additional retry without the fallback parameter if the API rejects it
Outcome checks stop_reason is examined before the text is used: refusal and max_tokens are errors; an empty stop reason is an invalid response

The key is stored in the Keychain as a generic password (service ai.spboucher.orix.dj.anthropic, account api-key, accessible when unlocked, this device only). It is placed only in the request header; the interface sees a masked form.

Reasons for raw HTTPS: the surface used is two endpoints and four stream event types, small enough to own, and it keeps the product free of third-party code.

Risks: a hand-written client must follow API changes by hand; the version string, the beta header name and the model identifiers in the fallback list are constants in the source. No request was ever sent to the real API: no key was available, and the client was tested against an in-process stub transport that speaks the same wire format. Behaviour against the live service (including the fallback parameter and the cache control block) is unverified. See AI integration.

# MIDI

Options. The legacy packet-list API (MIDIInputPortCreate, MIDISend); the event-list API introduced with Universal MIDI Packet support.

Chosen: the CoreMIDI event-list API (MIDIManager.swift): MIDIClientCreateWithBlock, MIDIInputPortCreateWithProtocol with protocol ._1_0, and MIDISendEventList for output. With the MIDI 1.0 protocol requested, CoreMIDI delivers Universal MIDI Packet words in which each channel voice message is one 32-bit word, so the receive path is a fixed-size decode; a separate streaming byte parser (MIDIParser, running status, SysEx and real-time bytes skipped) remains for byte-oriented sources. Mapping is pure logic (MIDIMappingEngine) with no CoreMIDI dependency, so it is unit-tested directly.

One platform finding recorded in the source: on macOS 27, hot-plug notifications were not delivered unless the client was created on a thread with a running run loop, so the manager owns a private run-loop thread.

Limits: MIDI 1.0 channel voice messages only; no MIDI 2.0 protocol features, no HID controllers, no bundled controller mappings. Only CoreMIDI virtual endpoints were used for testing; no physical controller has been connected, so device-specific behaviour (high-resolution jog wheels, LED feedback conventions, 14-bit controls) is unverified.

# Supported audio formats

Decoding uses AVAudioFile (TrackBuffer.load) and metadata uses AVFoundation (MediaImporter). The accepted extensions are a fixed set, MediaImporter.supportedExtensions: mp3, wav, aif, aiff, flac, m4a, aac, caf.

MediaImporterFormatTests generates fixtures and requires that WAV, AIFF, CAF, AAC in M4A, ALAC in M4A and FLAC open and decode on the test machine, and that a hand-built silent MP3 opens. Files whose asset reports protected content are rejected with a specific error. Formats outside the list (for example Ogg Vorbis, Opus, WMA) are not accepted, whatever the system might be able to decode. Because decoding is delegated to the system, format support follows the macOS version; it was verified on macOS 27.0 only.

Recording is written as 32-bit float CAF and exported to 16 or 24-bit WAV or AIFF, AAC or ALAC in M4A, or FLAC, using the system encoders. See recording.

# Licensing

The product contains no third-party source code and links no third-party library. Package.swift declares no package dependencies; the linked system components are Accelerate, AudioToolbox, CoreAudio, AVFoundation, CoreMIDI and sqlite3. Build-time tools (XcodeGen, Xcode) are not shipped.

The key-profile numbers are the published Krumhansl-Kessler probe-tone ratings, and the reverb follows the well-known Freeverb topology; both are implemented from the published descriptions.

# Hardware compatibility and what could not be tested

Area Status
Apple silicon, macOS 27.0 Built, run and tested
macOS 15 and 26 Deployment target is 15.0; not run on those versions
Intel Macs Not supported (arm64 only)
Built-in output, 48 kHz Used during development
Bluetooth output Present and usable; latency is whatever the device reports; not suitable for cueing and not characterised
Multi-channel interfaces, separate cue output Not tested on hardware; verified by offline four-channel rendering only
Sample rates other than 48 kHz on a device Engine supports 44.1, 48, 88.2 and 96 kHz candidates; only 48 kHz was used on a device
MIDI controllers Not tested on hardware; CoreMIDI virtual endpoints only
Anthropic API Not tested live; local stub only
Long sessions, thermal behaviour, memory growth Not measured
Listening quality (time stretch, EQ, effects, limiter, resampler) Not evaluated

# Summary of decisions

Subsystem Decision Main reason Main cost or risk
Audio I/O AUHAL + own render core Sample-accurate events, offline testability More code to own; hardware coverage is thin
Track storage Whole track in memory as float32 Render thread never touches the disk About 20 MiB per minute at 44.1 kHz; 150-minute cap
Time stretching Own WSOLA No third-party code, cheap Artefacts on transients and polyphony not evaluated by ear
Resampling Catmull-Rom cubic Stateless, any ratio No anti-alias filter when speeding up
Beat detection Spectral flux + autocorrelation, constant grid Deterministic, fast, exact beat-time maths Octave errors; no variable tempo
Key detection Chroma + Krumhansl-Kessler Deterministic, no model Partial agreement on real music
Waveforms Metal shader over a summary texture One draw call per deck GPU cost not measured
Effects Own DSP Beat sync, schema integration, testable No plug-in hosting
Persistence Raw SQLite Explicit schema, threading, recovery Hand-written SQL
Automation Own language Determinism, bounds, no host access Size of the implementation; learning curve
AI Raw HTTPS Messages API, user key in Keychain No SDK for Swift, small surface Never run against the live API
MIDI CoreMIDI event-list API Current API, UMP-ready Never run with hardware
Files main
Menu

Theme