Visualization
Three plots — waveform, spectrum, spectrogram — in a live form (while recording) and a file form (in the workstation). All are drawn from measured data; with no audio flowing they show a flat line or an empty plot.
Rendering technology: why not Metal
The brief asked for Metal "when justified by performance". It was evaluated and not used.
| Work per frame | Measured cost (Mac, Apple M5 Max, release build) |
|---|---|
| Spectrum frame: 4096-point FFT, smoothing, peak hold, bands | 15.1 µs |
| Rasterise a 512 × 512 spectrogram tile through a palette table | 0.65 ms |
| Waveform summary for 60 s of audio | 14 ms, once |
At 60 frames per second a frame is 16.7 ms. The heaviest of these, rasterising a tile, is done once per tile and cached as a CGImage; scrolling and zooming then draw images. With that budget a GPU pipeline would add shaders, texture management and a second code path without a measured benefit, so the app uses SwiftUI Canvas for vector plots and Core Graphics images for spectrogram tiles.
Caveat: these numbers come from a Mac. An older iPhone is several times slower, and nothing has been profiled on one. If device profiling shows dropped frames, the natural next step is a Metal view that uploads the dB tiles as textures and applies the palette in a fragment shader — the tile store already keeps numeric dB values separate from their colouring for exactly that reason.
Live views
The analysis thread writes into LiveVisuals; views read it inside a TimelineView. Two details matter:
- Rates are decoupled. Analysis follows the audio; drawing follows the display (60 Hz for the waveform, 30 Hz for spectrum, spectrogram and meters; 4–5 Hz with Reduce Motion).
- Data is read as view input, not inside the drawing closure. SwiftUI redraws a
Canvaswhen its inputs change. An earlier version fetched the buckets inside the closure, the inputs looked constant, and the waveform froze while audio kept flowing. The fetch now happens in theTimelineViewbody.
The live waveform shows the latest buckets right-aligned at 3 points per bar at true scale, with ±0.5 and ±1.0 FS guides.
File waveform
Mermaid source
flowchart LR S[PCM samples] --> B[512-sample bucket: min, max, RMS] S --> F[4096-point Hann FFT ending at the bucket] --> Bands[7 band powers] B --> P[Pyramid of levels] Bands --> P P --> Sel[Pick the coarsest level with one bucket per bar] Sel --> R[Re-bin to bars] --> C[Canvas]
WaveformSummary.buckets(for:points:) returns the coarsest level that still has at least one bucket per bar; WaveformRendering.rebin merges to exactly the number of bars. Gestures are shared by the waveform and the spectrogram through TimelineState: pinch to zoom about the pinch point, drag to pan (or to select, in selection mode), tap to move the playhead. Zoom is limited to 50 ms across the view.
Quiet recordings are magnified vertically when their peak is below 0.7 FS, up to 16×, and the factor is drawn on the plot.
| Style | Encoding |
|---|---|
| Frequency colours | one colour per bar: the band colours mixed by each band's share of RMS amplitude |
| Band shares | the bar divided bottom to top into seven segments in proportion to the same shares |
| Amplitude | minimum to maximum in the accent colour |
| RMS and peak envelope | RMS bar over a dimmer peak bar |
Colour mixing uses amplitude shares rather than power shares: with power, a strong bass component would hide everything else. The underlying band powers are unchanged, and analysis reports use power.
Spectrum plot
A dB grid every 20 dB, decade ticks on the logarithmic axis, a strip under the axis showing the seven bands, the spectrum filled with a gradient in which each horizontal position takes its band's colour, a peak-hold trace, a dashed line at the dominant frequency and a draggable cursor with a frequency and level readout. The vertical range is −120…0 dBFS for amplitude and power and −150…−20 dBFS/Hz for PSD.
Spectrogram
Mermaid source
flowchart LR Z[Visible range and width] --> L[Level L: hop = fft/4 × 2^L] L --> K[Tile keys, 256 columns each] K --> R[Read the mono range] --> T[STFT, hop ≤ one FFT length] T --> M[Max-pool to the level's column width] --> Cache[(dB tiles, 28 kept)] Cache --> LUT[Palette table] --> I[CGImage] --> Canvas
SpectrogramTileStore is an actor. A level's columns are fftSize/4 × 2^L samples wide. The transform never hops further than one FFT length, and coarser levels take the maximum over consecutive columns, so no audio is skipped at any zoom. Tiles hold numeric dB values; changing the palette, axis or range re-rasterises cached tiles without recomputing a transform. The store keeps 28 tiles; the view keeps images for what is on screen.
Tapping reads the value under the finger from the tile data and shows time, frequency and dBFS. A legend under the plot shows the palette with the dB values at its ends.
| Palette | Notes |
|---|---|
| ORIX Spectrum | ink → violet → blue → cyan → green → lime → amber → warm white, lightness rising |
| Scientific | viridis anchors |
| Thermal | inferno anchors |
| Monochrome | black to white |
| High Contrast | deliberate hard steps |
| Colorblind-friendly | cividis anchors |
Image export
The waveform is rendered at 3200 × 1200 pixels with its title, duration and band legend. The spectrogram is computed for the whole file at up to 3200 columns and 1024 rows and saved as PNG, with the palette and dB range in the file name.