SPB Git

Repository Public License

README.md 45.3 KB

ORIX Audio icon

ORIX Audio

Hear Everything. Understand More.

A native audio recorder, analysis laboratory and meeting notebook for iPhone and iPad.
Every waveform, spectrum and number on screen is computed from the audio itself. Nothing leaves the device.

iOS 26+ iPadOS adaptive Swift 6 SwiftUI DSP with Accelerate and vDSP Visualization with Canvas and Core Graphics
On-device speech Foundation Models where supported Privacy first Core offline No account required
English and French-Canadian 83 core, 21 app and 9 UI tests TestFlight: 1.0.0 (1) in internal testing

Status, 2026-11-20. The app is implemented and passes its test suites in the iOS 26.5 simulator and on macOS 27. No physical iPhone or iPad was available, so everything that depends on real hardware — locked-screen recording, the Live Activity, real microphones, route changes, phone-call interruptions, battery use, on-device speech and Foundation Models on iOS — is implemented against Apple's documented APIs but not yet validated on a device. See Release Status and Known Limitations.

Recording screen with live waveform, spectrum and meters Spectrogram of a recording Analysis dashboard Meeting notes generated on-device
RecordSpectrogramAnalysisMeeting notes

# Highlights

  • A recorder built around not losing audio. The capture master is an append-only file that is valid after every write; an interrupted session is offered for recovery at the next launch.
  • Frequency-coloured waveforms that are measurements. Each bar is coloured from an FFT of that moment, split into seven bands.
  • A spectrum analyzer with honest units. Amplitude, power and power spectral density are three different scalings, each labelled.
  • A zoomable spectrogram for long files, computed in tiles so a multi-hour recording is never loaded whole.
  • A measurement laboratory: peak, RMS, crest factor, clipping, ITU-R BS.1770-4 loudness, loudness range, true peak, spectral descriptors, speech-activity estimates.
  • On-device transcription in the languages the system reports, with search, editing and subtitle export.
  • Meeting notes grounded in the transcript: every point links to the line it came from; points that can't be traced are shown apart.
  • A non-destructive editor with undo, an enhancement chain, and export to WAV, AIFF, CAF, Apple Lossless and AAC.
  • No account, no analytics, no uploads. English and French-Canadian.

# Contents

  1. Overview · 2. ORIX Philosophy · 3. Application Preview · 4. Recording · 5. Background Recording · 6. Multicolor Waveforms · 7. Spectrum Analyzer · 8. Spectrogram · 9. Audio Analysis · 10. Transcription · 11. Foundation Models Intelligence · 12. Meeting Studio · 13. Audio and Video Import · 14. Audio Editing · 15. Playback · 16. Local Library · 17. ORIX Actions · 18. Export · 19. Audio Architecture · 20. DSP Methodology · 21. Technology Stack · 22. Project Structure · 23. Installation · 24. Build Instructions · 25. Testing · 26. Performance · 27. Privacy · 28. Accessibility · 29. Localization · 30. Roadmap · 31. Release Status · 32. Known Limitations · 33. ORIX Ecosystem · 34. License

# 1. Overview

ORIX Audio is one app for the whole life of a recording: capture it, look at it, measure it, transcribe it, summarise it, edit it and export it.

Platforms iOS and iPadOS 26.0 or later, one universal binary
Language Swift 6 (strict concurrency), SwiftUI
Size of the code base about 11 900 lines of Swift, tests included
Third-party dependencies none
Network use by the app none (iOS may download a speech model when the user asks for it)
Bundle identifier ai.spboucher.orix.audio

The repository has two halves. Core/ is a Swift package, ORIXAudioCore, that holds everything with no user interface: signal processing, analysis, the capture file writer, the edit model, encoders, exporters, the transcription and notes engines. It builds and tests on macOS, which is where the reference-signal validation runs. ORIXAudio/ is the SwiftUI app that drives it.

# 2. ORIX Philosophy

ORIX apps are small, exact tools. Three rules shape this one:

  1. Nothing decorative pretends to be data. There are no simulated waveforms and no animated placeholders. When no audio is flowing, the plot is a flat line and says so.
  2. A number carries its unit and its limits. Levels are dBFS, relative to digital full scale — never presented as sound pressure. Estimates are labelled as estimates.
  3. A feature that can't be honoured isn't offered. There is no MP3 export because iOS has no MP3 encoder; no automatic speaker labels because Apple provides no on-device diarization; no cloud fallback for transcription because the audio would leave the device.

# 3. Application Preview

All images are captures of the running app in the iOS 26.5 simulator, produced by Scripts/screenshots.sh. The simulator has no usable microphone, so the recorder was fed from an audio file through a debug-only input that goes through the real capture pipeline. The transcript and notes shown were produced by the same engines running on a Mac and loaded for display, because the simulator cannot run Foundation Models. Details in Release Status.

Library Spectrum analyzer at the playhead Transcript synced to playback Audio editor
LibrarySpectrumTranscriptEditor
ORIX Actions workflows French-Canadian interface with French notes Light appearance
ActionsFrançais (Canada)Light appearance
iPad workspace: waveform, spectrogram, spectrum and analysis side by side iPad recording screen
iPad workspaceiPad recording

# 4. Recording

Start, pause, resume, stop, add markers (renameable while recording), title and participants. The screen shows the elapsed time to a tenth of a second, the real capture format, the input in use, the bytes written, RMS and peak meters with peak hold and a clip lamp, and momentary loudness.

Presets. The sample rate and the number of channels always follow what the active input delivers; the app shows the real values once recording starts and never assumes 96 kHz or stereo.

Preset Saved file Codec Channels Sample rate Lossy?
Voice .m4a AAC-LC, 64 kbps mono 24 kHz (resampled when the input is higher) yes
Meeting .m4a AAC-LC, 96 kbps mono input rate yes
Lecture .m4a AAC-LC, 80 kbps mono input rate yes
Interview .m4a AAC-LC, 96 kbps per channel stereo if the input provides it input rate yes
High Quality .wav Linear PCM, 24-bit stereo if the input provides it input rate no
Lossless .m4a Apple Lossless, 24-bit stereo if the input provides it input rate no

Apple's AAC encoder accepts only certain bit rates for a given sample rate and channel count, and nothing above 48 kHz. The writer asks the encoder for its valid rates and takes the closest one, and resamples higher-rate input to 48 kHz. (A fixed 96 kbps request used to fail at 22.05 kHz mono; that case is now a regression test.)

How a session is captured. Whatever the preset, the session is first written as uncompressed PCM to a capture master (16-bit for the AAC presets, 24-bit otherwise). The library file is encoded from the master when you stop, re-opened, and its duration checked before the master is deleted. See Audio Architecture and docs/architecture/recording.md.

Safety nets. Recording refuses to start with less than 128 MB free and stops itself, saving what it has, below 64 MB. If the storage can't keep up, the dropped sample count is shown instead of being hidden.

# 5. Background Recording

The app declares the audio background mode and records with the .playAndRecord session category, which is Apple's documented way to keep capturing when the screen locks or the app moves to the background. A Live Activity shows the recording on the Lock Screen and in the Dynamic Island, with pause/resume and add-marker buttons implemented as LiveActivityIntents.

Background recording lifecycle

  • Interruptions (a call, an alarm, another app taking the microphone): capture pauses, a marker is added at that position, and recording resumes when iOS reports that it may, or on your tap.
  • Route changes (headphones, Bluetooth, USB): the engine is re-tapped; if the new input has a different sample rate its audio is resampled so the master keeps one rate.
  • Files stay writable under lock because they use the "complete until first user authentication" protection class. The stricter "complete" class would fail every write on a locked device.

What it does not do: it does not record covertly (the system microphone indicator and the Live Activity are always present), and it does not survive a force-quit, a crash or a shutdown as a running recording. In those cases the audio already flushed to disk — at most two seconds behind — remains a valid file and is offered for recovery.

Not verified on hardware. Locked-screen capture, the Live Activity, interruptions and route changes need a physical iPhone and none was connected. In the simulator, the recovery path was exercised for real: the app was terminated mid-recording and the 33-second capture was found and recovered at the next launch. See docs/architecture/background-audio.md.

# 6. Multicolor Waveforms

A waveform sample has an amplitude, not a colour. To colour it meaningfully the app measures the spectrum around each moment.

Waveform rendering pipeline

For every bucket of 512 samples the processor stores the exact minimum, maximum and RMS, and the power in seven bands taken from a 4096-point Hann FFT whose window ends at the bucket's last sample.

Band Range Colour
Sub-bass 20–60 Hz #7C3AED
Bass 60–250 Hz #3B82F6
Low-mid 250–500 Hz #06B6D4
Mid 500 Hz–2 kHz #22C55E
Upper-mid 2–4 kHz #EAB308
Presence 4–6 kHz #F97316
Brilliance 6–20 kHz #F43F5E

The band edges live in FrequencyBandLayout and are configurable in code. Four drawing styles use the same data:

Style What it shows
Frequency colours bar tinted by the amplitude-weighted mix of the band colours
Band shares bar split into stacked segments, one per band — readable without colour vision
Amplitude plain minimum / maximum
RMS and peak envelope RMS drawn over the peak envelope

The 4096-point window gives 11.7 Hz bins at 48 kHz, enough to tell sub-bass from bass. The price is that colour responds over about 85 ms while the bar heights are exact per bucket. Quiet files are magnified vertically for readability, and the factor is printed on the plot ("display ×2.4"); it never changes a measurement.

The file view supports pinch zoom, pan, tap to seek, drag to select a range, markers, a playback cursor and image export. A summary is a pyramid of levels, each half the resolution of the one below, cached at 40 bytes per bucket. Not included: a per-channel stereo waveform (the display is a mono mix).

# 7. Spectrum Analyzer

Live while recording, and at the playhead for files.

  • FFT sizes 1024 to 16 384; Hann, Hamming, Blackman, Blackman–Harris, flat top and rectangular windows.
  • Linear or logarithmic frequency axis, 20 Hz–20 kHz.
  • Exponential smoothing (applied to power, the statistically meaningful average) and peak hold with selectable decay.
  • A draggable cursor reading frequency and level; dominant frequency by parabolic interpolation.
  • Seven band meters under the plot.

The three scalings are different physical quantities and are never mixed. With X[k] the DFT of the windowed frame, N the FFT size, CG the window's coherent gain, NPG its noise power gain and fs the sample rate:

Scaling Formula (one-sided, 0 < k < N/2) Reads Unit
Amplitude `2· X[k] / (N·CG)`
Power `2· X[k] ² / (N²·CG²)`
Power spectral density `2· X[k] ² / (fs·N·NPG)`

The DC and Nyquist bins are not doubled. Band power divides the summed bins by the window's equivalent noise bandwidth so that a band holding one sine reads that sine's power.

Window Coherent gain ENBW (bins)
Rectangular 1.000 1.000
Hann 0.500 1.500
Hamming 0.540 1.363
Blackman 0.420 1.727
Blackman–Harris 0.359 2.004
Flat top 0.216 3.770

# 8. Spectrogram

A short-time Fourier transform shown as colour: time across, frequency up, amplitude in dBFS as colour, with a legend that states the range.

Spectrogram tile processing

  • Frequency resolution from FFT 512 to 8192; time resolution follows the zoom.
  • Linear or logarithmic frequency axis, adjustable top frequency and dB range.
  • Six palettes: ORIX Spectrum, Scientific, Thermal, Monochrome, High Contrast, Colorblind-friendly. Five of them rise monotonically in lightness (this is tested); High Contrast uses hard steps on purpose.
  • Tap to read time, frequency and level at a point; the time axis is shared with the waveform and the playback cursor.
  • High-resolution PNG export of the whole file.

Long files are handled in tiles of 256 columns computed on demand at the resolution the zoom needs. Coarser levels take the maximum over consecutive columns rather than skipping audio, so a 20 ms click is still visible when a two-hour file fills the screen (also tested). Tiles are rasterised on the CPU through a 256-entry palette table.

# 9. Audio Analysis

One pass over the file, in chunks, with memory use independent of length.

Group Measurements
Time domain sample peak, RMS, crest factor, DC offset, zero-crossing rate, clipping (runs of ≥ 3 samples at ≥ 0.999 FS), background level (10th percentile of 400 ms RMS), dynamic spread, silences (≥ 0.5 s below −50 dBFS)
Loudness integrated, maximum momentary (400 ms) and short-term (3 s) loudness, loudness range, true peak, a short-term series for plotting
Frequency domain centroid, bandwidth, rolloff (85 %), flatness, flux, dominant frequency, band energy, spectral peaks, long-term average spectrum
Speech activity (estimates) active time and share, pauses, mean and longest pause, speaking rate when a transcript exists
Findings clipping, low level, little contrast between quiet and loud parts, long silence, potential distortion, unusual frequency concentration, DC offset, limited bandwidth

Peak and clipping are measured per channel so a clipped channel is not hidden by the mono mix. Speech activity is a heuristic on energy in the 300–3400 Hz band and is labelled "Estimate" wherever it appears. Findings are indications drawn from the numbers beneath them, not forensic or medical conclusions. Method and validation: docs/reports/dsp-validation.md.

# 10. Transcription

On-device transcription paths

  • Primary path: SpeechAnalyzer with SpeechTranscriber, on-device by design, with per-run audio time ranges.
  • Fallback: SFSpeechRecognizer with requiresOnDeviceRecognition = true, used only where it reports on-device support.
  • Otherwise: the app says the language can't be transcribed on this device. There is no server path.

The language list comes from the system at run time; nothing beyond it is offered. When a speech model isn't installed, the app asks before downloading it — that step needs a connection once and transfers a model, not audio.

Recordings, imports and the audio extracted from videos can all be transcribed, with progress and cancellation. The transcript follows playback, can be searched (case- and accent-insensitive), edited line by line, labelled with speakers by hand, copied, and exported as text, Markdown, SRT, WebVTT, CSV, JSON or PDF. Live text while recording is optional and display-only; capture never depends on it.

Measured accuracy — SpeechTranscriber on macOS 27, speech synthesised with macOS voices:

Locale Voice Reference words Substitutions WER CER
en-US Samantha 163 3 1.84 % 0.94 %
en-US Daniel 163 3 1.84 % 0.94 %
fr-CA Amélie 182 1 0.55 % 0.09 %
fr-CA Thomas 182 0 0.00 % 0.00 %

Read these as an upper bound. Synthetic speech is clean, close and unhurried; a real room with several voices will do worse. The numbers were measured on a Mac, not on an iPhone. Full report: docs/reports/transcription-benchmark.md.

# 11. Foundation Models Intelligence

Speech recognition produces the transcript. Apple's on-device language model then reads that text — it never hears audio.

Map, verify, reduce

  1. Map. The transcript is numbered line by line and cut into chunks of at most 4200 characters. For each chunk the model fills a typed structure (title, summary, key points, action items, decisions, topics) and must cite the line number behind every point.
  2. Verify. A citation holds only if the line exists inside the chunk the model was shown and the point shares vocabulary with that line or a neighbour. A task owner is kept only if that name appears in the transcript.
  3. Reduce. With more than one chunk, an overall title and summary are written from the section summaries.

Points that fail verification are not hidden and not trusted: they appear in a separate "not traced to a transcript line" group. Tapping a verified point plays the recording from its source. Everything is labelled as AI-generated and can be edited or deleted. Questions about a recording work the same way, and an answer with no valid citation is reported as not answered by the transcript.

The app checks SystemLanguageModel.default.availability and explains the reason when it is unavailable. Recording, analysis and transcription don't depend on it.

Verified on macOS 27 with the benchmark transcripts: 7 of 7 generated points grounded in English, 8 of 8 in French; "Who will prepare the release notes?" answered with a citation; "What is the capital of France?" and "Quel temps fera-t-il demain?" correctly refused. Outputs are saved in Benchmarks/results/. Generation fails inside the iOS 26.5 simulator and has never been run on an iPhone.

# 12. Meeting Studio

The Meeting, Lecture and Interview presets set the recording kind, which shapes the notes that follow. A meeting can carry a title, participants typed by you, markers, a transcript, a summary, action items with owners, decisions, topics and chapters, a search across all of it, and exports of the audio, transcript and notes.

Speakers. There is no automatic speaker separation: Apple offers no on-device diarization API and none was built. You can label lines "Speaker 1" to "Speaker 6" by hand. The app never infers who is speaking from a voice.

Consent. Before the first recording the app reminds you that recording people without their knowledge can be illegal and asks you to confirm. The reminder can be re-enabled in Settings.

# 13. Audio and Video Import

From Files (audio or video, several at once), from Photos (video), or by opening a file in ORIX Audio from another app.

text
Import → inspect metadata → extract or decode audio → waveform → spectrum → spectrogram → transcribe → report → export
  • Audio is copied into the library unchanged. Anything Core Audio can open is accepted. The tests read back WAV, AIFF, CAF, Apple Lossless, AAC and MP3 files, and import WAV at 8, 22.05, 44.1 and 96 kHz in mono and stereo.
  • Video has its first audio track extracted. AAC and Apple Lossless tracks are copied bit for bit into an .m4a; any other codec is decoded and stored as Apple Lossless. The video file is never modified (tested byte for byte).
  • Metadata shown: duration, sample rate, channels, codec, bit depth, bit rate, file size, source file name.
  • Failures are clean: a video with no audio track, a truncated file or random bytes produce a message and leave nothing behind in the library.

Not every codec a container can hold is decodable on iOS; when AVFoundation can't read a track, the import says so.

# 14. Audio Editing

The editor works on an edit decision list. The original file is never touched; saving renders a new recording.

Operation Notes
Trim / crop to selection, delete selection, silence selection
Split at playhead, insert silence
Merge (append another recording) both must have the same sample rate
Fade in / fade out raised-cosine ramps
Gain whole recording or selection
Normalize peak measured by a render pass, then set to −1 dBFS
Remove silences from measured silences, keeping 0.15 s of padding
Channels original, mono mix, left only, right only
Enhance see below
Undo / redo 100 steps, stored with the document

Enhancement chain — all streaming, so file length doesn't matter:

  • High-pass and low-pass, 24 dB/octave Butterworth.
  • Three-band equalizer: 150 Hz shelf, 1.5 kHz peak, 6 kHz shelf.
  • Feed-forward compressor with a 6 dB soft knee.
  • Noise reduction by spectral subtraction, using the 200 quietest non-silent frames of the recording as the noise profile. It works on steady noise such as hiss, hum or a fan; it cannot separate voices or remove music.
  • Voice enhance: a fixed chain (80 Hz high-pass, presence lift, 3:1 compression), not a learned model.

Not offered: de-essing and source separation — there is no validated implementation of either.

# 15. Playback

Play, pause, seek, scrub on the waveform, skip 15 seconds, jump between markers, loop a selection, and six speeds from 0.5× to 2× with pitch preserved. The cursor follows on the waveform and spectrogram and the transcript highlights the current line. Playback continues in the background with Lock Screen controls, pauses on an interruption, and pauses rather than switching to the speaker when headphones are unplugged.

# 16. Local Library

All, Recent, Favourites, folders and Recently Deleted (kept 30 days). Search covers titles, notes, participants and transcript text. Sort by date, title, duration or size; filter by transcript, kind, imported or lossless. Rename, duplicate, move, favourite, delete, restore, and select several recordings for batch actions. Each row shows a waveform thumbnail, date, duration, size, codec and status chips.

Metadata lives in SwiftData; audio and its sidecars live in one folder per recording, written atomically. There is no iCloud sync. Layout: docs/architecture/storage.md.

# 17. ORIX Actions

Reusable workflows: an ordered list of steps with editable parameters, run on one recording or many, with a per-step log, progress, cancellation and a list of the files produced.

Step What runs
Analyze the full analysis pass
Transcribe on-device transcription in the chosen language
Summarize notes from the transcript (skipped with an explanation when the model is unavailable)
Remove silence cuts measured silences longer than a threshold
Enhance noise reduction and the voice chain or a high-pass
Normalize to a target peak
Convert to any export format
Export transcript / Export analysis report a shareable file

Audio steps are collected and rendered once into a new recording; the original is left byte-for-byte intact (tested). A step that can't run is reported as failed or skipped, never faked. A workflow can run automatically after each recording or each import. Four starter workflows are provided and are ordinary, editable workflows: Meeting notes, Clean up voice, Archive as lossless, Subtitles.

# 18. Export

Kind Formats
Audio WAV (PCM 16- or 24-bit), AIFF (PCM 16-bit), CAF (PCM 24-bit), M4A Apple Lossless (24-bit), M4A AAC — whole recording or the selection; or the original file
Transcript text, Markdown, SRT, WebVTT, CSV, JSON, PDF
Notes Markdown, PDF, JSON
Analysis report PDF, Markdown, CSV, JSON — every value with its unit
Images waveform PNG, spectrogram PNG

Why no MP3: iOS ships an MP3 decoder but no encoder, so MP3 files can be imported and not written. A test asserts MP3 is absent from the export list.

# 19. Audio Architecture

Audio capture pipeline

Component Where Role
AudioSessionManager app session category, inputs, stereo pattern, route description
AudioCaptureEngine app input tap, two ring buffers, writer and analysis threads
CAFWriter core append-only PCM master that is valid after every write
RecorderController app lifecycle, interruptions, Live Activity, finalization
RecordingFinalizer, RecordingRecovery app encode and verify; find interrupted sessions
WaveformProcessor, SpectrumAnalyzer, SpectrogramProcessor core DSP for visuals
AudioAnalyzer and the accumulators core whole-file analysis
PlaybackEngine app player node, time-pitch, looping, Now Playing
MediaImporter app copy audio, extract video audio
TranscriptionEngine, LiveTranscriber core on-device speech
MeetingIntelligence core grounded notes and questions
EditDocument, EditRenderer, EffectChain core non-destructive editing
AudioFileWriter, ExportService core / app encoders and shareable files
ActionRunner app workflows

Threads. The input tap only copies samples into two wait-free single-producer/single-consumer rings. A writer thread drains one into the master and calls fsync every two seconds. An analysis thread drains the other into the DSP that feeds the meters and plots. A stalled interface or a slow FFT cannot delay a write, and a slow disk cannot block the tap.

Why the master is recoverable. It is a Core Audio Format file whose audio chunk declares size −1, which the CAF specification defines as "extends to the end of the file". No header ever needs patching, so the bytes on disk are a complete, readable file at any instant, even with a torn final write.

Real-time analysis thread

More: overview · audio engine · recording · background audio · storage.

# 20. DSP Methodology

Signal processing uses Accelerate/vDSP for the transforms and vector arithmetic; filters and loudness run in double precision.

  • FFT. vDSP_fft_zrip, whose real forward transform returns twice the mathematical DFT; the squared magnitudes are divided by four before any scaling. Windows are generated in periodic (DFT-even) form.
  • Dominant frequency. Strongest bin above 20 Hz, refined by parabolic interpolation on log magnitude; nothing is reported below −90 dBFS.
  • Loudness. ITU-R BS.1770-4 for mono and stereo: K-weighting from closed-form coefficients that reproduce the standard's 48 kHz table to 1e-8 and give the equivalent response at other rates; 400 ms blocks with 75 % overlap; absolute gate at −70 LUFS and relative gate 10 LU below. Loudness range follows EBU Tech 3342. True peak uses a 4× oversampling 48-tap Kaiser-windowed sinc.
  • Noise reduction. 1024-point Hann analysis and synthesis at 75 % overlap, over-subtraction 1–3×, a gain floor of −6 to −24 dB and temporal smoothing of the gains.

Validation. Every measurement is checked against a signal whose answer is known in advance: sines, multi-tones, white and pink noise, chirps, impulses, silence and clipped waves. A few of the 83 checks:

Check Expected Tolerance
Bin-centred sine, all six windows, four FFT sizes reads its amplitude 0.001
Flat-top window, off-bin sine 0 dBFS 0.02 dB
Hann scalloping loss 1.42 dB worst case 1.3–1.5 dB
White-noise PSD, integrated the variance 3 %
Dominant frequency, FFT 8192 61.7 Hz … 9876.5 Hz 0.5 Hz
Chirp ridge in the spectrogram instantaneous frequency 2 bins
Full-scale 997 Hz sine, at 44.1, 48 and 96 kHz −3.01 LKFS 0.05
Synthesised EBU Tech 3341 cases 1–5 −23.0 / −33.0 LUFS 0.1 LU
Synthesised EBU Tech 3342 cases 1–2 10 LU, 5 LU 1 LU
Inter-sample peak, fs/4 sine at 45° 0 dBTP 0.3 dB
Butterworth filters at cutoff −3.01 dB 0.15 dB

The loudness tests use signals synthesised to the EBU specifications, not the official EBU files, so the accurate description is implements BS.1770-4, verified against synthesised reference signals — not a claim of certification. Details: docs/architecture/dsp.md and docs/reports/dsp-validation.md.

# 21. Technology Stack

Area Used
Language and UI Swift 6 with strict concurrency, SwiftUI, Observation
Capture and playback AVFoundation, AVAudioEngine, AVAudioSession, AVAudioConverter
DSP Accelerate / vDSP
Speech Speech (SpeechAnalyzer, SpeechTranscriber, SFSpeechRecognizer on-device only)
Language model FoundationModels (LanguageModelSession, guided generation)
Drawing SwiftUI Canvas, Core Graphics images
Persistence SwiftData for metadata, FileManager for audio and sidecars, iOS file protection
System integration ActivityKit and WidgetKit (Live Activity), App Intents, MediaPlayer, LocalAuthentication, PhotosUI
Project XcodeGen (project.yml), Swift Package Manager

Metal was evaluated and not used. Rasterising a 512 × 512 spectrogram tile on the CPU through a palette table was measured at 0.65 ms on the development Mac, far inside a frame, so Core Graphics tiles drawn in a SwiftUI Canvas were kept. A GPU path would add complexity without a measured need; that could change after profiling on older iPhones. The reasoning is in docs/architecture/visualization.md.

# 22. Project Structure

text
ORIXAudio/
├── project.yml                  XcodeGen definition (app, Live Activity extension, unit and UI tests)
├── Core/                        Swift package ORIXAudioCore — no UI, builds on iOS and macOS
│   ├── Sources/ORIXAudioCore/
│   │   ├── DSP/                 FFT, windows, spectrum, spectrogram, waveform, filters, effects, palettes
│   │   ├── Analysis/            time-domain, spectral, loudness, speech activity, findings, analyzer
│   │   ├── Capture/             ring buffer, CAF master writer
│   │   ├── Editing/             edit document, renderer
│   │   ├── Export/              audio file I/O, text exporters
│   │   ├── Speech/              transcription engine, transcript model, error rates
│   │   └── Intelligence/        grounded notes and questions
│   ├── Sources/orix-bench/      command-line benchmarks (macOS)
│   └── Tests/                   83 reference-signal and engine tests
├── ORIXAudio/                   the app
│   ├── App/                     entry point, root navigation, debug hooks, core wording
│   ├── DesignSystem/            tokens and components
│   ├── Models/                  SwiftData models, presets, workflow steps
│   ├── Services/                session, capture, recorder, finalizer, playback, import, export, actions
│   ├── Features/                Recorder, Library, Detail, Editor, Actions, Settings, Visuals
│   └── Resources/               asset catalog, string catalogs, privacy manifest
├── ORIXAudioWidgets/            Live Activity
├── Shared/                      Live Activity attributes and intents
├── Tests/  UITests/             app tests (21) and UI tests (8), with fixtures
├── Benchmarks/                  reference passages and measured results
├── Localization/                fr-CA translations
├── Scripts/                     build, test, simulator, screenshots, localization, icon, badges
└── docs/                        architecture, design, development, release, reports, screenshots, assets

# 23. Installation

There is no public build yet. To run it from source you need a Mac with Xcode 27 (Swift 6.4, iOS 27 SDK), an iOS 26 or later simulator or device, and XcodeGen (brew install xcodegen). Full notes: docs/development/setup.md.

# 24. Build Instructions

bash
xcodegen generate                       # creates ORIXAudio.xcodeproj from project.yml
open ORIXAudio.xcodeproj                # then run the ORIXAudio scheme

Scripts/build.sh                        # or: command-line Debug build for the simulator
SIM="ORIXAudio iPad" Scripts/build.sh   # choose another simulator by name

The Xcode project is generated and not committed. Signing uses team 3YM54G49SN with automatic signing; simulator builds need no signing.

# 25. Testing

bash
Scripts/test.sh core     # 83 tests, Swift package, runs on macOS in about a minute
Scripts/test.sh app      # 21 tests in the simulator: capture pipeline, recovery, import, library, actions, exports
Scripts/test.sh ui       # 9 XCUITest flows in the simulator (incl. the accessibility audit)
Scripts/bench-transcription.sh           # word and character error rates, English and French
Scripts/l10n.py --check                  # fails if any string lacks a French translation
Suite Tests Where it ran Result on 2026-11-20
Core (ORIXAudioCoreTests) 83 in 14 suites macOS 27 all passed
App (ORIXAudioTests) 21 in 4 suites iOS 26.5 simulator, iPhone 17 Pro all passed
UI (ORIXAudioUITests) 8 iOS 26.5 simulator, iPhone 17 Pro all passed

The app tests drive the real capture engine through a debug-only file input and assert, among other things, that every accepted frame is written, that nothing is captured while paused, that each preset produces the format it declares, that an interrupted session with a torn write is recovered, and that a workflow leaves the original file intact. There is no continuous integration; the suites are run by hand. What the simulator cannot test is listed in docs/development/testing.md.

# 26. Performance

Measured with orix-bench in a release build on a Mac with an Apple M5 Max, single thread. These are not iPhone numbers; they show the cost of the algorithms, not battery or thermal behaviour on a phone.

Operation Result
FFT 4096 (window, transform, scaling) 2.5 µs
Live spectrum frame (4096 points, smoothing, peak hold, bands) 15.1 µs
Waveform summary with band colours, 60 s of audio 14 ms
Spectrogram 2048/512, 60 s of audio 15 ms
Rasterise a 512 × 512 spectrogram tile 0.65 ms
BS.1770 loudness and true peak, 60 s mono 95 ms
Full analysis, 10 min stereo 48 kHz 2.74 s
Export 10 min stereo to Apple Lossless 1.28 s
Export 10 min stereo to AAC 2.11 s
Noise reduction, voice enhance and export, 10 min stereo 3.67 s
Transcription, about 60 s of speech 0.6–0.7 s

Not measured: CPU, battery and memory while recording on an iPhone. Full table and method: docs/reports/performance.md.

# 27. Privacy

  • No account, no sign-in, no analytics, no advertising, no tracking.
  • The app contains no networking code of its own.
  • Recordings, transcripts, notes and reports are stored only on the device, under iOS file protection.
  • Transcription never falls back to a server; summaries use the on-device model or are unavailable.
  • Microphone access is requested when you first record; Photos is reached through the system picker, which needs no library permission; Face ID or passcode lock is optional.
  • Transcript text is never logged.
  • The privacy manifest declares no collected data and no tracking.

One thing to know: when the legacy recogniser is the only on-device path for a language, iOS shows its standard speech-recognition prompt, whose stock wording mentions sending audio to Apple. ORIX Audio forces on-device recognition on that path, and its own text in the prompt says so. Details: docs/reports/privacy.md.

# 28. Accessibility

VoiceOver labels and values on every control and plot (the waveform is adjustable to scrub; the spectrum announces its dominant frequency), Dynamic Type text styles throughout, Reduce Motion honoured by the live views, and no information carried by hue alone: bands are named beside their colours, the "band shares" waveform and three spectrogram palettes work without colour vision, and status chips pair colour with a symbol and a word.

This was reviewed in code and in the simulator, not audited with assistive technology on a device. Findings and gaps: docs/reports/accessibility.md.

# 29. Localization

English and French (Canada), more than 600 strings, none untranslated (Scripts/l10n.py --check fails otherwise). Strings are extracted from the compiler's output rather than typed by hand, merged with Localization/fr-CA.json by Scripts/l10n.py, and checked by a UI test that launches the app in French. Dates, durations, byte counts and numbers use the system formatters. The permission texts are localised too.

# 30. Roadmap

Ordered by what would most change confidence in the product:

  1. Validate on physical iPhones and iPads: locked-screen recording over hours, interruptions, route changes, the Live Activity, battery and thermal behaviour.
  2. Measure transcription on real recordings of real rooms, in both languages.
  3. Per-channel stereo waveform.
  4. Profile the visualisations on older devices and revisit a Metal path if the numbers call for it.
  5. Bake playback speed into exports; allow merging files with different sample rates.
  6. Speaker separation, if Apple ships an on-device API or a validated model becomes available.

# 31. Release Status

As of 2026-11-20:

Item Status
Implementation complete; builds and runs in the iOS 26.5 simulator on iPhone and iPad
Automated tests 83 core, 21 app, 9 UI — all passing
DSP validation done against synthesised reference signals
Transcription accuracy measured on macOS with synthetic speech only
Foundation Models verified on macOS; fails to generate in the simulator; untested on iOS hardware
Physical-device validation pending — no device was available
Repository published at https://git.spboucher.ai/orix-audio (the host has no private mode; publication was the owner's choice)
App Store Connect app record created (ID 6837544792); IPA validated with no errors
TestFlight build 1.0.0 (1) uploaded and processed; available to internal testers only. Not submitted for external testing or App Review. Details: RELEASE-REPORT.md, docs/release/apple-developer-verification.md

By the project's own quality gates, ORIX Audio is not release-ready until background recording has been tested on a physical iPhone. Checklist: docs/release/checklist.md.

# 32. Known Limitations

  • Nothing has run on a physical device. Locked-screen and background recording, the Live Activity and Dynamic Island, real microphone capture, built-in stereo capture, Bluetooth and wired route changes, phone-call interruptions and battery use are untested on hardware.
  • Speech and language models on iOS are untested on hardware. Transcription was measured on macOS; Foundation Models was verified on macOS and cannot generate in the iOS 26.5 simulator.
  • No speaker diarization. Manual labels only.
  • The waveform is a mono mix. Peak and clipping are measured per channel, but there is no per-channel waveform.
  • No MP3 export, no de-essing, no source separation.
  • Playback speed is not applied to exports.
  • Merging needs equal sample rates.
  • No Metal renderer; visualisations are drawn with Canvas and Core Graphics.
  • Speech-activity figures are heuristics and findings are indications.
  • Live text needs an installed speech model and is display-only; the saved transcript always comes from the file.
  • No iCloud sync. Recordings are included in device backups like other app data.
  • A recording does not continue through a force-quit, crash or shutdown. Audio up to the last two-second checkpoint stays in a valid file and is offered for recovery.
  • Loudness covers mono and stereo and was verified against synthesised signals, not the official EBU test files.
  • The system speech-recognition prompt on the legacy path uses Apple's stock wording about sending audio to Apple, although on-device recognition is enforced.
  • Accuracy and performance figures come from a Mac and from synthetic speech.

# 33. ORIX Ecosystem

ORIX is a family of exact Apple utilities. ORIX Scan and ORIX Calc are sibling projects. The three share the same neutral colours, spacing and radii, and one icon rule — a ring on a full-bleed tile — with each app bringing its own accent pair and inner glyph. ORIX Audio's pair is cyan #22D3EE with violet #8B5CF6, and its ring holds seven bars in the band colours.

The shared values are copied, not linked: ORIX Audio has no runtime or build dependency on the other apps. See design system, brand guidelines and icon design.

# 34. License

Proprietary. Copyright © 2026 Simon-Pierre Boucher. All rights reserved. See LICENSE.

ORIX Audio

Files main
Menu

Theme