SPB Git

Repository Public License

testing.md 7.0 KB

# Testing

Three suites, run by hand (there is no continuous integration).

Suite Command Count Runs on Result, 2026-11-20
Core Scripts/test.sh core 83 tests, 14 suites macOS 27 all passed (about 55 s; one test accumulates an hour of audio)
App Scripts/test.sh app 21 tests, 4 suites iOS 26.5 simulator, iPhone 17 Pro all passed
UI Scripts/test.sh ui 9 tests iOS 26.5 simulator, iPhone 17 Pro all passed (about 3.5 min)

Core and app tests use Swift Testing; UI tests use XCUITest.

# Core: reference signals

Every DSP result is compared with a signal whose answer is known. Signals come from SignalGenerator: sines, multi-tones, seeded Gaussian white noise, pink noise, linear chirps, impulses, silence, hard-clipped waves.

Suite What it proves
Window, FFT window gains equal their coefficients; a bin-centred sine reads its amplitude for six windows and four sizes; power and PSD scaling; DC and Nyquist; scalloping loss; leakage; resolution of two tones four bins apart; finite output for silence and full scale
Spectrum analyser dominant frequency within 0.5 Hz; band power equals sine power in the right band only; peak-hold decay rate; smoothing halves power in one step; spectral peaks; the log axis keeps narrow peaks
Spectrogram column timing of a tone burst; chirp ridge follows instantaneous frequency; an impulse appears only in the columns that contain it; streaming equals one-shot; raster rows and palette mapping; palette lightness
Time domain sine and noise statistics; clipping counts; silence location; DC; chunking invariance; an hour of audio without drift
Spectral descriptors of tones, white and pink noise; band energy of a multi-tone; flux
Loudness K-weighting coefficients; −3.01 LKFS reference at three sample rates; synthesised EBU Tech 3341 cases 1–5 and 3342 cases 1–2; true peak between samples
Voice activity, waveform bursts over noise are segmented; steady noise is not speech; bucket statistics; colour band equals measured band; pyramid and serialisation
Capture CAF files readable by Core Audio in three sample formats; an unclosed master with a torn write is valid; ring buffer order across threads; overflow accounting
Edit, encoders, effects every edit operation and undo/redo; rendered audio equals the edit list sample for sample; fades, normalise, channel modes; merge and rate mismatch; cancellation leaves no file; lossless round trips; AAC at every sample rate; resampling; filter responses; compressor curve; denoiser transparency and effect
Text SRT, WebVTT, CSV, JSON; time codes; search; segmentation; word and character error rates; citation verification

Report with tolerances: docs/reports/dsp-validation.md.

# App: the real pipeline in the simulator

The capture tests set AudioCaptureEngine.debugInputFile, which replaces the microphone with a file read in real time. Everything after the tap is the production path.

Test Asserts
Record, pause, resume, stop nothing accepted while paused; live spectrum reads the 1.5 kHz test tone within 3 Hz and its level within 0.5 dB; the analysis thread produced every expected bucket; no drops; frames written equal frames accepted; the finalised WAV is 24-bit stereo and analyses to the same tone
Analysis keeps up at 22.05 kHz buckets and spectrogram columns match the audio captured
Every preset the finalised file has the codec, bit depth, sample rate, channel count and lossless flag the preset declares
Interrupted session an unclosed master with a torn byte and a sidecar is found, never offered while it is the active session, recovered with its markers, and cleaned up
Empty and orphan masters an empty master is removed; a master without a sidecar is still recoverable
Import AAC, MP3, video audio track (copied, video unchanged byte for byte), video without audio rejected, random and truncated files rejected with no folder left, 8–96 kHz mono and stereo
Library import, duplicate, trash, restore, purge
Workflow analyse → remove silence → normalise → convert → analyse → report produces a new Apple Lossless recording at −1 dBFS and a PDF, and leaves the original byte-identical; a step with nothing to work on fails visibly
Exports six audio formats over a selection; seven transcript formats; four report formats; waveform image size; spectrogram ridge at the tone's frequency
View logic timeline zoom, pan and selection bounds; re-binning keeps extremes; a 20 ms click survives max-pooling at a coarse level; no preset calls lossy audio lossless; MP3 is not an export format

# UI flows

Test Flow
First launch welcome screen, then the recorder in the Ready state
Record consent reminder → recording → marker → pause (timer frozen) → resume → stop → the new recording opens, shows AAC-LC and Lossy, keeps its marker, plays
Library three imports (audio, MP3, video) appear; search filters; delete; found in Recently Deleted
Workstation spectrum, spectrogram, analysis (ran automatically), transcript and notes panels; export menu lists WAV and Apple Lossless and no MP3
Editor drag to select → delete → undo available → save as WAV → new recording opens as Linear PCM, Lossless
Actions run "Archive as lossless" on a recording; one new recording and one exported file
Settings settings and privacy screens
French the app launched in fr-CA shows « Prêt » and « Démarrer l’enregistrement »

# What these suites cannot show

Stated plainly, because the simulator hides all of it:

Not tested Why
Recording with the screen locked or the app in the background needs a physical device
Live Activity and Dynamic Island need a physical device
Real microphones, built-in stereo, Bluetooth, wired and USB routes the simulator has no such hardware
Phone calls, alarms and other interruptions cannot be produced in the simulator
Microphone permission denied not automated
Low storage not automated
Battery, thermal behaviour, memory pressure need a device and Instruments
Speech recognition inside the app the simulator only offers the legacy recogniser behind a system prompt; the engine was measured on macOS instead
Foundation Models inside the app generation fails in the iOS 26.5 simulator; verified on macOS instead
VoiceOver, Switch Control, large text on a device reviewed in code and simulator only

The device test plan is in docs/architecture/background-audio.md.

# Fixtures

Tests/Fixtures: speech-en.m4a (7 s of synthesised speech, AAC, 22.05 kHz mono), tone.mp3, clip.mp4 (video with an AAC track), clip-noaudio.mp4. All were generated for this project with macOS say and ffmpeg; none contains a real person's voice.

Files main
Menu

Theme