SPB Git

Repository Public

audio-engine.md 15.5 KB

# Audio engine

This document describes the render side of ORIX DJ: RenderCore (the audio graph), the queues that connect it to the rest of the program, and AudioOutput (the Core Audio device layer). The control side (DJEngine) is covered in deck-engine.md; the signal path inside a render block is detailed in mixer.md and dsp.md. For the place of the engine in the whole application see overview.md.

Sources:

# Threads

Thread Created by What it does with the engine
Core Audio I/O thread the AUHAL output unit Calls AudioOutput.renderCallback, which calls RenderCore.render. The only thread that touches voices, strips, racks and mix buffers.
Scheduler thread ai.spboucher.orix.dj.automation AutomationHost.startScheduler() Wakes every 4 ms under the Mach time-constraint policy, calls DJEngine.service() (notices, master clock, sync) and advances automations. See automation.md.
Main thread AppKit / SwiftUI Reads snapshots on a 30 Hz timer (AppModel.refreshFast) and turns user input into control events.
Recorder queue ai.spboucher.orix.dj.recorder Recorder Drains the recording ring every 40 ms. See recording.md.
Work and analysis queues, CoreMIDI thread DJSession, MIDIManager Decode tracks, analyse, deliver MIDI; they reach the engine only through DJEngine.

All control-side entry points of DJEngine take one NSRecursiveLock. That lock is what makes the event queue single-producer even though several threads issue commands. The render thread never takes it.

# Control events

Every change to the audio graph is a ControlEvent, a BitwiseCopyable struct:

Field Meaning
time: Int64 Engine sample time at which the event applies; 0 means as soon as possible.
kind: UInt16 An EventKind: deckSetTrack, deckPlay, deckPause, deckSeek, deckRate, deckKeylock, deckPitch, deckReverse, deckLoop, deckLoopOff, deckBrake, param, clock, fxSet, fxSwap, fxEnable, fxMix, fxParam, record, panic.
deck: Int16 Deck index, or rack index for effect events (-1 is the master rack).
index: Int32 MixParam raw value, or slot and parameter index packed as `slot
a, b: Double Value and ramp time in seconds, or loop start and end.
pointer Opaque pointer to a TrackBuffer or an EffectUnit for deckSetTrack and fxSet.
tracked: Bool Set for automation events whose timing is measured.

Events travel through RenderCore.events, an SPSCQueue<ControlEvent>(minimumCapacity: 4096). The queue rounds its storage up to a power of two with one slot kept empty, so the usable capacity is 8191 events. push never blocks: when the queue is full it returns false, and DJEngine.push counts the loss in droppedEvents (exposed as droppedControlEvents). A dropped event is lost; nothing retries it.

# Intake and the pending list

render starts with intake(), which pops events from the queue:

  • an event whose time is less than or equal to the current sampleTime is applied immediately (this covers time == 0);
  • any other event is insertion-sorted into pending, a preallocated array of 512 events (pendingCapacity).

Intake runs once per render call and stops when the pending list is full. Two consequences follow from the code: an untimed event takes effect at the start of the next device buffer, so its jitter is up to one buffer; and if 512 future events are already pending, events queued behind them wait in the queue until a slot frees up, even if they are due now. No test exercises that second situation.

# Sub-block splitting

The render loop processes audio in chunks of at most 64 frames (RenderCore.chunk):

swift
while done < total {
    applyDue()
    var n = min(Self.chunk, total - done)
    if pendingCount > 0 { n = max(1, min(n, Int(pending[0].time - sampleTime))) }
    renderChunk(done, n)
    sampleTime += Int64(n); done += n
    ...
}

A chunk is shortened so that it ends exactly on the timestamp of the next pending event, and applyDue() applies every event whose time has been reached before the next chunk starts. A timed event therefore takes effect on its exact sample, independently of the device buffer size, provided it reached the queue before the render call that contains its timestamp. If it arrives later it is applied at the start of the next call, and for tracked events the lateness is recorded in lateEvents, maxLateSamples and totalLateSamples.

What is verified: scriptActionsLandOnExactSamples in EngineAudioTests.swift requires scheduled deck starts to land within 2 samples of a click grid and requires lateEvents == 0; the measured values are in performance.md.

Parameter changes are not steps. applyParam converts the ramp time with rampSamples, which enforces a minimum of 4 ms (max(Int(0.004 * sampleRate), ...)), and stores the target in a LinearRamp. Gains are then interpolated linearly inside each chunk with vDSP_vrampmul2; filter coefficients are recomputed once per chunk while a ramp is running.

flowchart TD
    C["Control threads: UI, MIDI, scheduler"] -->|"set, play, seek, insertEffect"| E["DJEngine (one lock, clamps values)"]
    E -->|"push ControlEvent"| Q["events: SPSC queue, 8191 slots"]
    Q -->|"intake at the start of each render call"| I{"event time reached"}
    I -->|yes| A["apply now"]
    I -->|no| P["pending list, sorted by time, 512 slots"]
    P -->|"applyDue before each chunk"| A
    A --> K["renderChunk: at most 64 frames, shortened to end on the next pending event"]
    K -->|"more frames in this call"| P
    K --> O["Device channels: master and cue"]
    K -->|"trackEnded, brakeFinished, releaseTrack, releaseEffect"| N["notices: SPSC queue, 2047 slots"]
    K -->|"end of call"| S["Snapshot cells (seqlock): engine state, 4 deck states"]
    N -->|"service on the scheduler thread"| E
    S -->|"read, retry if a write is in progress"| C

# State published by the render thread

# Snapshot seqlock

SnapshotCell<T> is a single-writer, multi-reader sequence lock. The writer increments an Atomic<UInt64> before and after writing the storage; a reader copies the storage and retries if the sequence was odd or changed during the copy. The writer never waits. Readers allocate an array per read, which is acceptable because they run on control threads.

Two cells exist: deckStates (four DeckRenderState: position, rate, playing, loaded, loop state, post-fader peak and RMS) and engineState (one EngineRenderState: sampleTime, hostTime, master peaks and RMS, cue peak, limiterReductionDB, clippedBlocks, render timing, overloads, event and lateness counters, recording counters). Both are written once at the end of every render call.

The engine times itself: render reads mach_absolute_time() at entry and exit and publishes lastRenderNanos, maxRenderNanos, totalRenderNanos and renderCalls. A call that took longer than the duration of the buffer it produced increments overloads. These counters feed Settings → Audio → Engine measurements (AudioSettingsTab.swift) and the benchmark tool.

resetCounters() writes the same state from a control thread. The code comment calls this a benign race accepted for diagnostics; it is not synchronised.

# Notices

The render thread reports back through RenderCore.notices, an SPSCQueue<RenderNotice>(minimumCapacity: 1024) (usable capacity 2047):

NoticeKind Sent when Handled in DJEngine.service()
trackEnded a voice ran off the end (or the start, in reverse) marks the deck paused, emits trackEnd and pause
brakeFinished a brake reached zero speed marks the deck paused, emits pause
releaseTrack a deckSetTrack replaced a track removes the old TrackBuffer from retained
releaseEffect an fxSet replaced an effect unit removes the old EffectUnit from retained

Object lifetime is handled this way: DJEngine keeps every TrackBuffer and EffectUnit it has handed to the render thread in the dictionary retained, keyed by pointer. The render thread only stores Unmanaged references and sends the pointer back when it stops using an object, so deallocation always happens on a control thread. The return value of notices.push is ignored: if the notice queue were full, a release notice would be lost and the object would stay in retained until the core is rebuilt. That is a leak, not a use-after-free.

Output routing is not an event: masterLeft, masterRight, cueLeft and cueRight are Atomic<Int> values read at the end of each render call.

# AudioOutput

AudioOutput owns one AUHAL unit (kAudioUnitType_Output / kAudioUnitSubType_HALOutput). There is no AVAudioEngine and no second unit; the cue output is a second channel pair of the same device.

DJEngine.start() picks the device whose UID matches EngineConfiguration.deviceUID, else the system default, else the first output device. AudioOutput.start(core:deviceID:bufferFrames:) then:

  1. Sets the device's nominal sample rate to the core's rate if they differ by more than 1 Hz, polling up to 40 times at 25 ms intervals for the change to take effect. Before that, DJEngine.start() has already replaced a rate the device does not offer by 48 kHz, or by the first rate the device offers, and rebuilt the core if needed. The rates considered are 44.1, 48, 88.2 and 96 kHz.
  2. Clamps the requested buffer size to the device's kAudioDevicePropertyBufferFrameSizeRange and to RenderCore.maxFrames (4096), and sets kAudioDevicePropertyBufferFrameSize. The default is 256 frames (EngineConfiguration.bufferFrames); the settings screen offers 64, 128, 256, 512, 1024 and 2048 within the device's range.
  3. Selects the device with kAudioOutputUnitProperty_CurrentDevice.
  4. Sets the stream format: linear PCM, 32-bit float, native endian, non-interleaved, with as many channels as the device has outputs, capped at 64.
  5. Sets kAudioUnitProperty_MaximumFramesPerSlice to 4096, installs the render callback, initialises and starts the unit.

The callback maps the buffer list to an array of channel pointers obtained with withUnsafeTemporaryAllocation (64 pointers, on the stack) and calls core.render(frameCount:channels:channelCount:). render clears every channel, then copies the master and cue buses to the configured channels (see mixer.md). If the device asked for more than 4096 frames, the frames beyond 4096 would stay silent.

# Reported latency

AudioDevices.info computes latencyFrames as kAudioDevicePropertyLatency + kAudioDevicePropertySafetyOffset (both output scope) + kAudioStreamPropertyLatency of the first output stream. The buffer is not included in that field. The application displays "device latency plus one buffer"; where and with which caveats is described in performance.md.

# Device changes

AudioDevices.observeChanges listens to kAudioHardwarePropertyDevices and kAudioHardwarePropertyDefaultOutputDevice. The application's handler (AppModel.devicesChanged) restarts audio when the device in use is no longer present or the engine is not running. DJEngine.reconfigure stops the unit, applies the new configuration and starts again; when the sample rate changes, rebuildCore creates a new RenderCore, unloads the decks and replays every parameter and effect rack into it, and the application reloads the tracks.

Not verified: device loss and hot-plug were not exercised with real hardware (see the environment notes in testing.md). The EngineEvent cases devicesChanged and deviceLost are declared but nothing in the package emits them.

# Real-time rules and how the code follows them

Rule How the code follows it
No allocation while rendering Mix buffers (deckL/R, mixL/R, cueL/R, 4096 frames each), strips, rack slots and the pending list are allocated in RenderCore.init. DeckVoice, TimeStretcher and every EffectUnit allocate in their initialisers, which run on control threads. The callback's pointer array is a stack allocation.
No locks The render path uses only the two SPSC queues, the seqlock writer, the recording ring and atomics.
No file or disk access Tracks are fully decoded into memory before they reach the render thread (deck-engine.md); recording goes through a ring to a writer thread (recording.md).
No deallocation Replaced tracks and effects are handed back through notices and released by service().
Bounded work Fixed deck count (4), fixed rack size (8 slots, 5 racks), chunk size 64, block size capped at 4096.
Bad values cannot poison state Events with non-finite values are ignored (e.a.isFinite checks); effects clamp their inputs with sane (dsp.md).

# Caveats

These points were checked in the code and are stated as they are.

  • Swift reference counting on the render thread. The render path is Swift code operating on class instances. Rack slots are accessed through Unmanaged._withUnsafeGuaranteedRef, which avoids retain and release. Other accesses are ordinary strong references: Unmanaged.takeUnretainedValue() on the RenderCore in the callback, let voice = voices[d] on an array of DeckVoice, the limiter property, and inside EffectUnit.setParameter the copy of an EffectParamSpec (a struct holding strings). The compiler may emit atomic retain and release operations for these. They are not locks and do not allocate, but they are not free, and the code does not prove their absence.
  • Dynamic dispatch. EffectUnit is an open class; processWet, parameterChanged, observeBypassed and willEnable are virtual calls.
  • Buffers cleared on the render thread. The reverb, delay and echo units mark themselves stale while bypassed and call reset() in the first block after they are enabled again. reset() zeroes their delay memory on the render thread: for the delay and the echo that is two lines sized for 4 s of audio each. The cost of that first block has not been measured separately.
  • No automated allocation detector. No test installs a malloc hook or runs the render path under an allocation or lock checker. The claim "allocation-free after init" rests on code review and on the structure described above, not on an instrumented run.
  • Not run under a real-time sanitizer or on a loaded machine with an audio interface. See performance.md for what was and was not measured.

# Offline rendering

DJEngine.renderOffline(frames:channels:) calls core.render in blocks of configuration.bufferFrames (clamped to 16…4096) from the calling thread and calls service() after each block. AutomationHost.renderOffline additionally ticks the scheduler before each block. All engine tests, the benchmark tool and the offline mixes use this path, so they execute the same render function as the device callback but not on a real-time thread. The application's silent mode (AppModel.startSilentRendering, used for UI tests and screenshots) also uses it, paced by usleep.

Files main
Menu

Theme