Repository Public
1# Audio engine23This document describes the render side of ORIX DJ: `RenderCore` (the audio graph), the queues that connect it to the rest of the program, and `AudioOutput` (the Core Audio device layer). The control side (`DJEngine`) is covered in [deck-engine.md](deck-engine.md); the signal path inside a render block is detailed in [mixer.md](mixer.md) and [dsp.md](dsp.md). For the place of the engine in the whole application see [overview.md](overview.md).45Sources:67- [`RenderCore.swift`](../../Packages/ORIXDJCore/Sources/ORIXDJEngine/RenderCore.swift)8- [`AudioOutput.swift`](../../Packages/ORIXDJCore/Sources/ORIXDJEngine/AudioOutput.swift)9- [`DJEngine.swift`](../../Packages/ORIXDJCore/Sources/ORIXDJEngine/DJEngine.swift)10- [`Primitives.swift`](../../Packages/ORIXDJCore/Sources/ORIXDSP/Primitives.swift) (`SPSCQueue`, `LinearRamp`)1112## Threads1314| Thread | Created by | What it does with the engine |15|---|---|---|16| Core Audio I/O thread | the AUHAL output unit | Calls `AudioOutput.renderCallback`, which calls `RenderCore.render`. The only thread that touches voices, strips, racks and mix buffers. |17| Scheduler thread `ai.spboucher.orix.dj.automation` | `AutomationHost.startScheduler()` | Wakes every 4 ms under the Mach time-constraint policy, calls `DJEngine.service()` (notices, master clock, sync) and advances automations. See [automation.md](automation.md). |18| Main thread | AppKit / SwiftUI | Reads snapshots on a 30 Hz timer (`AppModel.refreshFast`) and turns user input into control events. |19| Recorder queue `ai.spboucher.orix.dj.recorder` | `Recorder` | Drains the recording ring every 40 ms. See [recording.md](recording.md). |20| Work and analysis queues, CoreMIDI thread | `DJSession`, `MIDIManager` | Decode tracks, analyse, deliver MIDI; they reach the engine only through `DJEngine`. |2122All control-side entry points of `DJEngine` take one `NSRecursiveLock`. That lock is what makes the event queue single-producer even though several threads issue commands. The render thread never takes it.2324## Control events2526Every change to the audio graph is a `ControlEvent`, a `BitwiseCopyable` struct:2728| Field | Meaning |29|---|---|30| `time: Int64` | Engine sample time at which the event applies; `0` means as soon as possible. |31| `kind: UInt16` | An `EventKind`: `deckSetTrack`, `deckPlay`, `deckPause`, `deckSeek`, `deckRate`, `deckKeylock`, `deckPitch`, `deckReverse`, `deckLoop`, `deckLoopOff`, `deckBrake`, `param`, `clock`, `fxSet`, `fxSwap`, `fxEnable`, `fxMix`, `fxParam`, `record`, `panic`. |32| `deck: Int16` | Deck index, or rack index for effect events (`-1` is the master rack). |33| `index: Int32` | `MixParam` raw value, or slot and parameter index packed as `slot | (param << 8)`. |34| `a`, `b: Double` | Value and ramp time in seconds, or loop start and end. |35| `pointer` | Opaque pointer to a `TrackBuffer` or an `EffectUnit` for `deckSetTrack` and `fxSet`. |36| `tracked: Bool` | Set for automation events whose timing is measured. |3738Events travel through `RenderCore.events`, an `SPSCQueue<ControlEvent>(minimumCapacity: 4096)`. The queue rounds its storage up to a power of two with one slot kept empty, so the usable capacity is 8191 events. `push` never blocks: when the queue is full it returns `false`, and `DJEngine.push` counts the loss in `droppedEvents` (exposed as `droppedControlEvents`). A dropped event is lost; nothing retries it.3940### Intake and the pending list4142`render` starts with `intake()`, which pops events from the queue:4344- an event whose `time` is less than or equal to the current `sampleTime` is applied immediately (this covers `time == 0`);45- any other event is insertion-sorted into `pending`, a preallocated array of 512 events (`pendingCapacity`).4647Intake runs once per render call and stops when the pending list is full. Two consequences follow from the code: an untimed event takes effect at the start of the next device buffer, so its jitter is up to one buffer; and if 512 future events are already pending, events queued behind them wait in the queue until a slot frees up, even if they are due now. No test exercises that second situation.4849### Sub-block splitting5051The render loop processes audio in chunks of at most 64 frames (`RenderCore.chunk`):5253```swift54while done < total {55 applyDue()56 var n = min(Self.chunk, total - done)57 if pendingCount > 0 { n = max(1, min(n, Int(pending[0].time - sampleTime))) }58 renderChunk(done, n)59 sampleTime += Int64(n); done += n60 ...61}62```6364A chunk is shortened so that it ends exactly on the timestamp of the next pending event, and `applyDue()` applies every event whose time has been reached before the next chunk starts. A timed event therefore takes effect on its exact sample, independently of the device buffer size, provided it reached the queue before the render call that contains its timestamp. If it arrives later it is applied at the start of the next call, and for `tracked` events the lateness is recorded in `lateEvents`, `maxLateSamples` and `totalLateSamples`.6566What is verified: `scriptActionsLandOnExactSamples` in `EngineAudioTests.swift` requires scheduled deck starts to land within 2 samples of a click grid and requires `lateEvents == 0`; the measured values are in [performance.md](../development/performance.md).6768Parameter changes are not steps. `applyParam` converts the ramp time with `rampSamples`, which enforces a minimum of 4 ms (`max(Int(0.004 * sampleRate), ...)`), and stores the target in a `LinearRamp`. Gains are then interpolated linearly inside each chunk with `vDSP_vrampmul2`; filter coefficients are recomputed once per chunk while a ramp is running.6970```mermaid71flowchart TD72 C["Control threads: UI, MIDI, scheduler"] -->|"set, play, seek, insertEffect"| E["DJEngine (one lock, clamps values)"]73 E -->|"push ControlEvent"| Q["events: SPSC queue, 8191 slots"]74 Q -->|"intake at the start of each render call"| I{"event time reached"}75 I -->|yes| A["apply now"]76 I -->|no| P["pending list, sorted by time, 512 slots"]77 P -->|"applyDue before each chunk"| A78 A --> K["renderChunk: at most 64 frames, shortened to end on the next pending event"]79 K -->|"more frames in this call"| P80 K --> O["Device channels: master and cue"]81 K -->|"trackEnded, brakeFinished, releaseTrack, releaseEffect"| N["notices: SPSC queue, 2047 slots"]82 K -->|"end of call"| S["Snapshot cells (seqlock): engine state, 4 deck states"]83 N -->|"service on the scheduler thread"| E84 S -->|"read, retry if a write is in progress"| C85```8687## State published by the render thread8889### Snapshot seqlock9091`SnapshotCell<T>` is a single-writer, multi-reader sequence lock. The writer increments an `Atomic<UInt64>` before and after writing the storage; a reader copies the storage and retries if the sequence was odd or changed during the copy. The writer never waits. Readers allocate an array per read, which is acceptable because they run on control threads.9293Two cells exist: `deckStates` (four `DeckRenderState`: position, rate, playing, loaded, loop state, post-fader peak and RMS) and `engineState` (one `EngineRenderState`: `sampleTime`, `hostTime`, master peaks and RMS, cue peak, `limiterReductionDB`, `clippedBlocks`, render timing, `overloads`, event and lateness counters, recording counters). Both are written once at the end of every `render` call.9495The engine times itself: `render` reads `mach_absolute_time()` at entry and exit and publishes `lastRenderNanos`, `maxRenderNanos`, `totalRenderNanos` and `renderCalls`. A call that took longer than the duration of the buffer it produced increments `overloads`. These counters feed Settings → Audio → Engine measurements ([`AudioSettingsTab.swift`](../../App/Sources/Features/Settings/AudioSettingsTab.swift)) and the benchmark tool.9697`resetCounters()` writes the same state from a control thread. The code comment calls this a benign race accepted for diagnostics; it is not synchronised.9899### Notices100101The render thread reports back through `RenderCore.notices`, an `SPSCQueue<RenderNotice>(minimumCapacity: 1024)` (usable capacity 2047):102103| `NoticeKind` | Sent when | Handled in `DJEngine.service()` |104|---|---|---|105| `trackEnded` | a voice ran off the end (or the start, in reverse) | marks the deck paused, emits `trackEnd` and `pause` |106| `brakeFinished` | a brake reached zero speed | marks the deck paused, emits `pause` |107| `releaseTrack` | a `deckSetTrack` replaced a track | removes the old `TrackBuffer` from `retained` |108| `releaseEffect` | an `fxSet` replaced an effect unit | removes the old `EffectUnit` from `retained` |109110Object lifetime is handled this way: `DJEngine` keeps every `TrackBuffer` and `EffectUnit` it has handed to the render thread in the dictionary `retained`, keyed by pointer. The render thread only stores `Unmanaged` references and sends the pointer back when it stops using an object, so deallocation always happens on a control thread. The return value of `notices.push` is ignored: if the notice queue were full, a release notice would be lost and the object would stay in `retained` until the core is rebuilt. That is a leak, not a use-after-free.111112Output routing is not an event: `masterLeft`, `masterRight`, `cueLeft` and `cueRight` are `Atomic<Int>` values read at the end of each render call.113114## AudioOutput115116`AudioOutput` owns one AUHAL unit (`kAudioUnitType_Output` / `kAudioUnitSubType_HALOutput`). There is no `AVAudioEngine` and no second unit; the cue output is a second channel pair of the same device.117118`DJEngine.start()` picks the device whose UID matches `EngineConfiguration.deviceUID`, else the system default, else the first output device. `AudioOutput.start(core:deviceID:bufferFrames:)` then:1191201. Sets the device's nominal sample rate to the core's rate if they differ by more than 1 Hz, polling up to 40 times at 25 ms intervals for the change to take effect. Before that, `DJEngine.start()` has already replaced a rate the device does not offer by 48 kHz, or by the first rate the device offers, and rebuilt the core if needed. The rates considered are 44.1, 48, 88.2 and 96 kHz.1212. Clamps the requested buffer size to the device's `kAudioDevicePropertyBufferFrameSizeRange` and to `RenderCore.maxFrames` (4096), and sets `kAudioDevicePropertyBufferFrameSize`. The default is 256 frames (`EngineConfiguration.bufferFrames`); the settings screen offers 64, 128, 256, 512, 1024 and 2048 within the device's range.1223. Selects the device with `kAudioOutputUnitProperty_CurrentDevice`.1234. Sets the stream format: linear PCM, 32-bit float, native endian, non-interleaved, with as many channels as the device has outputs, capped at 64.1245. Sets `kAudioUnitProperty_MaximumFramesPerSlice` to 4096, installs the render callback, initialises and starts the unit.125126The callback maps the buffer list to an array of channel pointers obtained with `withUnsafeTemporaryAllocation` (64 pointers, on the stack) and calls `core.render(frameCount:channels:channelCount:)`. `render` clears every channel, then copies the master and cue buses to the configured channels (see [mixer.md](mixer.md)). If the device asked for more than 4096 frames, the frames beyond 4096 would stay silent.127128### Reported latency129130`AudioDevices.info` computes `latencyFrames` as `kAudioDevicePropertyLatency` + `kAudioDevicePropertySafetyOffset` (both output scope) + `kAudioStreamPropertyLatency` of the first output stream. The buffer is not included in that field. The application displays "device latency plus one buffer"; where and with which caveats is described in [performance.md](../development/performance.md).131132### Device changes133134`AudioDevices.observeChanges` listens to `kAudioHardwarePropertyDevices` and `kAudioHardwarePropertyDefaultOutputDevice`. The application's handler (`AppModel.devicesChanged`) restarts audio when the device in use is no longer present or the engine is not running. `DJEngine.reconfigure` stops the unit, applies the new configuration and starts again; when the sample rate changes, `rebuildCore` creates a new `RenderCore`, unloads the decks and replays every parameter and effect rack into it, and the application reloads the tracks.135136Not verified: device loss and hot-plug were not exercised with real hardware (see the environment notes in [testing.md](../development/testing.md)). The `EngineEvent` cases `devicesChanged` and `deviceLost` are declared but nothing in the package emits them.137138## Real-time rules and how the code follows them139140| Rule | How the code follows it |141|---|---|142| No allocation while rendering | Mix buffers (`deckL/R`, `mixL/R`, `cueL/R`, 4096 frames each), strips, rack slots and the pending list are allocated in `RenderCore.init`. `DeckVoice`, `TimeStretcher` and every `EffectUnit` allocate in their initialisers, which run on control threads. The callback's pointer array is a stack allocation. |143| No locks | The render path uses only the two SPSC queues, the seqlock writer, the recording ring and atomics. |144| No file or disk access | Tracks are fully decoded into memory before they reach the render thread ([deck-engine.md](deck-engine.md)); recording goes through a ring to a writer thread ([recording.md](recording.md)). |145| No deallocation | Replaced tracks and effects are handed back through notices and released by `service()`. |146| Bounded work | Fixed deck count (4), fixed rack size (8 slots, 5 racks), chunk size 64, block size capped at 4096. |147| Bad values cannot poison state | Events with non-finite values are ignored (`e.a.isFinite` checks); effects clamp their inputs with `sane` ([dsp.md](dsp.md)). |148149### Caveats150151These points were checked in the code and are stated as they are.152153- **Swift reference counting on the render thread.** The render path is Swift code operating on class instances. Rack slots are accessed through `Unmanaged._withUnsafeGuaranteedRef`, which avoids retain and release. Other accesses are ordinary strong references: `Unmanaged.takeUnretainedValue()` on the `RenderCore` in the callback, `let voice = voices[d]` on an array of `DeckVoice`, the `limiter` property, and inside `EffectUnit.setParameter` the copy of an `EffectParamSpec` (a struct holding strings). The compiler may emit atomic retain and release operations for these. They are not locks and do not allocate, but they are not free, and the code does not prove their absence.154- **Dynamic dispatch.** `EffectUnit` is an `open class`; `processWet`, `parameterChanged`, `observeBypassed` and `willEnable` are virtual calls.155- **Buffers cleared on the render thread.** The reverb, delay and echo units mark themselves stale while bypassed and call `reset()` in the first block after they are enabled again. `reset()` zeroes their delay memory on the render thread: for the delay and the echo that is two lines sized for 4 s of audio each. The cost of that first block has not been measured separately.156- **No automated allocation detector.** No test installs a malloc hook or runs the render path under an allocation or lock checker. The claim "allocation-free after init" rests on code review and on the structure described above, not on an instrumented run.157- **Not run under a real-time sanitizer or on a loaded machine with an audio interface.** See [performance.md](../development/performance.md) for what was and was not measured.158159## Offline rendering160161`DJEngine.renderOffline(frames:channels:)` calls `core.render` in blocks of `configuration.bufferFrames` (clamped to 16…4096) from the calling thread and calls `service()` after each block. `AutomationHost.renderOffline` additionally ticks the scheduler before each block. All engine tests, the benchmark tool and the offline mixes use this path, so they execute the same `render` function as the device callback but not on a real-time thread. The application's silent mode (`AppModel.startSilentRendering`, used for UI tests and screenshots) also uses it, paced by `usleep`.162