Waveform display
ORIX DJ never draws raw audio samples. The analyser reduces every track to a small three-band summary once, the summary is stored with the track, and both waveform views draw from it: the scrolling deck waveform with a single Metal fragment-shader pass, the whole-track overview with a SwiftUI Canvas.
Sources:
Packages/ORIXDJCore/Sources/ORIXAnalysis/Waveform.swift(WaveformSummary,WaveformBuilder)App/Sources/Features/Waveforms/WaveformRenderer.swift(Metal renderer and shader source)App/Sources/Features/Waveforms/WaveformViews.swift(ScrollingWaveform,MetalWaveform,OverviewWaveform,OverviewCache)App/Sources/App/AppSettings.swift(WaveformPalette),App/Sources/DesignSystem/Tokens.swift(deck and hot-cue colours,ORIX.Palette.waveformWell)
Related documents: analysis.md for the rest of the analysis pass, persistence.md for how the summary is stored, deck-engine.md for the deck state the renderer reads, ../design/design-system.md for the colour system and ../development/performance.md for measured costs.
Data flow
flowchart TD
A["Decoded audio (TrackBuffer, planar Float32)"] --> B["WaveformBuilder.build"]
B --> C["WaveformSummary: low, mid, high, peak at 150 columns per second"]
C --> D["AnalysisResult, JSON then zlib, table track_analysis"]
C --> E["LoadedTrack.waveform on the deck"]
D --> E
E --> F["WaveformRenderer.upload: RGBA8 texture, 4096 columns per row"]
F --> G["waveFragment: one full-screen triangle per frame"]
H["DeckStatus: position, grid, cue, hot cues, loop"] --> G
E --> I["WaveformSummary.overview(columns:), max-pooled to the view width"]
I --> J["OverviewWaveform: three filled paths in a SwiftUI Canvas"]
H --> J
Summary format
WaveformSummary is a Codable, Sendable struct with four byte arrays of equal length and a rate:
| Field | Content | Scaling |
|---|---|---|
columnsPerSecond |
Columns per second of audio. 150 for a fresh analysis (WaveformBuilder.columnsPerSecond), lower after overview(columns:). |
Double |
low |
Peak magnitude of the band below 200 Hz in the column | 255 * sqrt(amplitude), rounded, clamped to 255 |
mid |
Peak magnitude of the 200 Hz to 2500 Hz band | same |
high |
Peak magnitude of the band above 2500 Hz | same |
peak |
Full-band sample peak over both channels | linear, 255 * abs(amplitude), 255 = 0 dBFS |
How the columns are made (WaveformBuilder.build):
- A column covers
sampleRate / 150samples: 294 samples at 44.1 kHz, 320 at 48 kHz. The column count isceil(frames * 150 / sampleRate). peakis computed per channel withvDSP_maxmgvand the larger of left and right is kept.- The band values are computed on the mono mix (
0.5 * (left + right), built once byTrackAnalyzer). The mix is filtered by 8th-order Butterworth filters, each realised as four biquad sections (WaveformBuilder.sections, Q values0.5098,0.6013,0.9000,2.5629; 48 dB per octave, -3 dB at the crossover) and run throughvDSP_biquad:- low: low-pass at
lowCrossover = 200Hz; - mid: high-pass at 200 Hz followed by low-pass at
highCrossover = 2500Hz; - high: high-pass at 2500 Hz.
The crossover frequency is clamped to
0.45 * sampleRate.
- low: low-pass at
- Each band value is the maximum magnitude of the filtered signal within the column, stored square-root scaled so that quiet material keeps visible height in 8 bits.
- The filters are causal IIR filters applied once, forward. Their group delay is not compensated, so band envelopes lag the
peakarray slightly, most at the low crossover. The delay has not been measured.
Size, from the constants: the four arrays take 4 bytes per column, 600 bytes per second of audio, 180 000 bytes for a five-minute track before compression.
overview(columns:) returns a max-pooled copy for the whole-track view: output column c is the maximum of source columns c*n/columns ..< (c+1)*n/columns; columnsPerSecond is scaled accordingly. It returns self when columns >= count and an empty summary when columns <= 0.
Storage
The summary is not stored on its own. It is a member of AnalysisResult, which DJSession.encode serialises with JSONEncoder and compresses with zlib (NSData.compressed(using: .zlib)); the blob goes to the track_analysis table of the library database (see persistence.md). Because the arrays are [UInt8], the JSON holds them as arrays of numbers; the compressed size was not measured. When a track is loaded, DJSession.storedAnalysis(for:) decodes the blob if track.analysisVersion == AnalysisResult.currentVersion (currently 1) and the summary is handed to the deck as LoadedTrack.waveform. A track without a current analysis is analysed during the load.
What the tests verify
WaveformTests in Packages/ORIXDJCore/Tests/ORIXAnalysisTests/AnalyzerTests.swift:
| Test | Assertion |
|---|---|
bandSeparation |
A 60 Hz sine gives low > 170, mid < 0.1 * low, high < 0.02 * low; a 1 kHz sine gives mid > 170 with low and high below 0.1 * mid; a 6 kHz sine gives high > 170, low < 0.02 * high, mid < 0.1 * high. For amplitude 0.5 the band value is within 4 of 180 (255 * sqrt(0.5)) and the linear peak within 2 of 127.5. The first and last 15 columns (100 ms) are skipped because of filter transients. |
columnCountAndOverview |
10 s of audio gives exactly 1500 columns at columnsPerSecond == 150; overview(columns: 100) has 100 columns at 10 columns per second, preserves the global maximum, and its first column equals the maximum of the first 15 source columns; clicks every 0.5 s appear every 75 columns with silence (peak == 0) between them. |
silenceGivesNoGrid (RobustnessTests) |
20 s of silence gives 3000 columns, all zero. |
Scrolling waveform (Metal)
Structure
ScrollingWaveform (SwiftUI) hosts MetalWaveform, an NSViewRepresentable around an MTKView. Its coordinator is a WaveformRenderer, one per deck view, which is the view's MTKViewDelegate. The Metal device (MTLCreateSystemDefaultDevice()), the render pipeline and the command queue are static and shared by all decks. The shader is a Metal Shading Language string compiled at first use with device.makeLibrary(source:options:); there is no .metal file in the app bundle.
The view is configured with colorPixelFormat = .bgra8Unorm, enableSetNeedsDisplay = false and isPaused = false, so MetalKit drives draw(in:) continuously at preferredFramesPerSecond.
Texture
WaveformRenderer.upload copies the summary into one rgba8Unorm texture, without mipmaps, usage .shaderRead:
| Channel | Content |
|---|---|
| R | low |
| G | mid |
| B | high |
| A | peak |
The texture is textureWidth = 4096 texels wide and ceil(columns / 4096) rows high; column i lives at texel (i % 4096, i / 4096). A five-minute track (45 000 columns) needs 11 rows. The upload happens when the deck's LoadedTrack changes or its waveform column count changes, not per frame. When no track is loaded a 1x1 zero texture is bound.
The alpha channel is uploaded but the current fragment shader reads only .rgb: the linear peak array is not drawn by the scrolling view.
Per-frame work
Each frame the renderer reads engine.deckStatus(deck), fills a WaveformUniforms block made only of float4 members (so the Swift and Metal layouts agree) and draws one triangle that covers the viewport (waveVertex generates it from vertex_id; there are no vertex buffers and no per-frame geometry). Everything visible is computed per pixel in waveFragment:
| Uniform | Content |
|---|---|
view |
drawable width and height in pixels, playhead position in seconds, seconds per pixel |
data |
columns per second, column count, texture width, track duration |
grid |
BPM, first beat in seconds, downbeat index, cue point in seconds (-1 when no track) |
loop |
loop start, loop end, loop active flag, gain (always 1 at present) |
hotA, hotB |
eight hot-cue times in seconds, -1 when unset |
low, mid, high |
band colours from the selected palette |
deck |
deck colour, used for the loop region and loop edges |
background |
well colour, default (0.02, 0.027, 0.055) |
misc |
backing scale factor, playing flag (not used by the shader) |
The fragment shader, in drawing order:
- Maps the pixel to track time:
t = playhead + (x - width / 2) * secondsPerPixel. The playhead is fixed at the horizontal centre and the audio scrolls under it. Vertical position is folded toy = abs(2 * py / height - 1), so the waveform is mirrored about the centre line. - Lightens the background by
0.012outside0...duration, so the start and end of the track are visible. - Tints an active loop region with the deck colour (16 %).
- Beat grid: for
bpm > 0, and only when beats are more than7 * scalepixels apart, draws a line at every beat of the constant-tempo grid. Lines whose index modulo 4 matchesdownbeatIndexare drawn at 42 % white, other beats at 14 %. - Waveform: reads up to 8 columns spanning the pixel (
taps = clamp(ceil(secondsPerPixel * columnsPerSecond), 1, 8)) withtexture.read(integer texel fetch, no filtering) and keeps the per-band maximum. Band heights arelow,0.86 * mid,0.72 * highof the half height, times0.94; they are composited in the order low, mid, high with a one-pixel anti-aliased edge. Audio left of the playhead is dimmed to 62 %. - Centre line at 10 % white.
- Loop edges in the deck colour; the cue point as an orange line (
1.0, 0.62, 0.04), full strength in the outer 45 % of the height and 55 % elsewhere. - Hot cues: a thin line in one of eight fixed colours plus a small flag at the top (
9 * scalepixels), so that hot cues differ from the cue point by shape as well as colour. - Playhead: a white vertical line at the centre.
Zoom and scrubbing
- Zoom is
AppSettings.waveformZoom, the number of seconds visible across the view: default6, clamped to1...32, stored inUserDefaultsunderui.waveformZoom. It is one setting for all decks. It changes with a pinch (MagnifyGesture) or the two magnifier buttons (factor 1.5). - Dragging horizontally scrubs: the translation is converted to seconds with the current zoom and sent to
engine.seek(deck:seconds:), so the real playhead moves. - There is no free scrolling away from the playhead; the view is always centred on the play position.
Frame pacing
At the end of every draw(in:) the renderer sets view.preferredFramesPerSecond to 60 while the deck is playing and 15 otherwise. The deck position comes from DJEngine.deckStatus, read on the main thread each frame; the renderer does not interpolate between engine updates. Frame time and GPU cost of this pass have not been measured (see ../development/performance.md).
Overview waveform
OverviewWaveform shows the whole track in a SwiftUI Canvas; it does not use Metal.
OverviewCachebuilds three closedPathoutlines (low, mid, high) fromwaveform.overview(columns: Int(viewWidth)), mirrored about the centre, with height factors0.94,0.8and0.66. The paths are rebuilt only when the track, the view size or the column count changes.- Each redraw fills the well (
ORIX.Palette.waveformWell,0x05070Ein both appearances), the three paths in the palette colours, a 45 % black veil over the played part, the loop region in the deck colour (35 % when active, 15 % otherwise), the cue point (0xFF9F0A), hot cues as half-height marks inORIX.hotCueColors, and a 2-point white playhead. - A click or drag seeks to the proportional position. For assistive technologies the view is one adjustable element whose increment and decrement actions seek by one fiftieth of the duration; its value is the played percentage.
- The overview does not draw the beat grid.
Palettes
WaveformPalette (App/Sources/App/AppSettings.swift) defines the band colours as sRGB hex. The choice is stored in UserDefaults under ui.palette and applies to both the scrolling and the overview waveform.
| Case | Title in the interface | Low | Mid | High |
|---|---|---|---|---|
spectrum (default) |
Spectrum (red / green / blue) | 0xFF4B3A |
0x3BDC6E |
0x4D8DFF |
ember |
Ember (amber / white / cyan) | 0xFF8A1F |
0xF2F4FB |
0x22D3EE |
colorBlindSafe |
Colour-blind safe (blue / orange / yellow) | 0x0072B2 |
0xE69F00 |
0xF0E442 |
mono |
Monochrome | 0x6E7592 |
0xA3AAC2 |
0xF2F4FB |
The colorBlindSafe values are the blue, orange and yellow of the Okabe-Ito palette. The default spectrum palette separates low and mid by red against green, which is the pair most affected by the common colour-vision deficiencies; colorBlindSafe and mono avoid that pair. In all palettes the bands also differ by height (1.0, 0.86, 0.72 of the scale) and by stacking order, so colour is not the only cue. None of the palettes has been checked with a colour-vision simulator or by affected users (see ../reports/accessibility.md).
Other colours are not part of the palette setting:
- Deck colour:
ORIX.deck(index)inTokens.swift, converted to sRGB and passed as thedeckuniform. - Hot cues: the overview uses
ORIX.hotCueColors; the Metal shader has its own constant array of eight colours written as floats. The two lists were written to match but are maintained separately. - Cue point: orange in both views.
When Metal is unavailable
There is no software fallback for the scrolling waveform. If MTLCreateSystemDefaultDevice() returns nil, or the shader fails to compile (the error is logged with NSLog), WaveformRenderer.pipeline is nil and draw(in:) returns before encoding anything: the scrolling view stays empty. The SwiftUI overlays (zoom buttons, the "Drop a track here" and "No waveform" messages) and scrubbing still work, and the overview waveform is unaffected because it is drawn with Canvas. This path has not been exercised: every Mac that can run the deployment target (macOS 15, Apple silicon) has a Metal device.
Limitations
- Fixed resolution of 150 columns per second (6.7 ms). Zooming in to 1 s across a wide Retina view shows several pixels per column; the shader does not interpolate between columns.
- At most 8 columns are sampled per pixel. When one pixel spans more than 8 columns (seconds per pixel above about 53 ms, for example 32 s shown in a drawable narrower than 600 pixels) some columns are skipped and short peaks can be missed.
- 8 bits per value with square-root scaling for the bands; heights are not a linear amplitude scale.
- The band split is fixed (200 Hz and 2500 Hz) and is not the mixer's EQ; band envelopes carry uncompensated filter delay.
- The bands are computed from the mono mix; stereo content that cancels in the mix is under-represented in the bands (the unused
peakarray is computed per channel). - The beat grid is the constant-tempo grid of analysis.md; the display cannot show tempo changes inside a track.
- Time and position are passed to the shader as 32-bit floats. For very long tracks at maximum zoom the precision of the playhead time approaches the size of a pixel; this has not been tested near the 150-minute limit of
TrackBuffer.maxSeconds. - The
gainuniform is constant: there is no user control of waveform height. - No listening or visual comparison against other software has been done; the correctness claims above are those of the unit tests listed.