Repository Public License
1<!--2ORIX Audio3File: docs/architecture/transcription.md4Author: Simon-Pierre Boucher5Contact: contact@spboucher.ai6-->78# Transcription and Meeting Intelligence910Two engines, kept apart on purpose: speech recognition turns audio into text; the language model reads text. The model is never given audio.1112## Speech recognition1314<p align="center"><img src="../assets/diagrams/transcription.svg" alt="On-device transcription paths" width="760"></p>1516<details><summary>Mermaid source</summary>1718```mermaid19flowchart TD20 F[Audio file] --> P{Path for this locale}21 P -->|SpeechTranscriber supports it, model installed| ST[SpeechAnalyzer + SpeechTranscriber]22 P -->|supported, model missing| D{User agreed to download?}23 D -->|yes| ST24 D -->|no| E1[Explain: model not installed]25 P -->|only the legacy recogniser, on-device capable| L[SFSpeechRecognizer, requiresOnDeviceRecognition]26 P -->|neither| E2[Explain: cannot be transcribed on this device]27 ST --> R[Timed runs] --> S[Segments] --> J[(transcript.json)]28 L --> S29```30</details>3132`TranscriptionEngine.path(for:)` decides at run time:3334| Path | Condition | Engine |35|---|---|---|36| `speechTranscriber` | `SpeechTranscriber.isAvailable`, locale supported, model installed | `SpeechAnalyzer` with a `SpeechTranscriber` module asking for audio time ranges |37| `speechTranscriberNeedsDownload` | supported, model not installed | the same, after `AssetInventory` installs the model with the user's consent |38| `legacyOnDevice` | `SFSpeechRecognizer.supportsOnDeviceRecognition` | `SFSpeechURLRecognitionRequest` with `requiresOnDeviceRecognition = true` |39| `unavailable` | none of the above | an error that says so |4041There is no branch that sends audio to a server. The supported-language list is whatever the two APIs report on the device; nothing else is offered.4243**Model downloads.** The first transcription in a language may need its speech model. The app asks first; the download is performed by iOS, needs a connection once, and transfers a model to the device. On the development Mac the first download took several minutes with no intermediate progress reported.4445**Timing.** Each recogniser result is an `AttributedString` whose runs carry audio time ranges. Runs are grouped into segments at sentence punctuation or after 14 seconds; word timings are kept for each segment.4647**Permission.** `SpeechAnalyzer` works without the Speech Recognition permission; the legacy recogniser requires it. The app therefore requests the permission only when the legacy path is used. iOS's prompt for it contains stock wording about sending audio to Apple; ORIX Audio's own line in that prompt states that it uses on-device recognition only.4849**Live text.** `LiveTranscriber` feeds captured audio to a `SpeechTranscriber` with volatile results enabled, through a bounded stream that drops buffers when the recogniser falls behind. It starts only if the model is already installed, is display-only, and its failure never affects capture. The saved transcript always comes from the finished file.5051### Transcript model5253Segments (start, end, text, optional speaker, word timings, edited flag) in `transcript.json`. Editing a line marks it edited and discards its word timings, since they no longer match. Speakers are assigned by hand; there is no diarization.5455### Accuracy5657Measured on macOS 27 with synthetic speech: 1.84 % word error rate in English and 0.00–0.55 % in French. Synthetic speech is an upper bound. See the [benchmark report](../reports/transcription-benchmark.md).5859## Meeting intelligence6061<p align="center"><img src="../assets/diagrams/foundation-models.svg" alt="Map, verify, reduce" width="760"></p>6263<details><summary>Mermaid source</summary>6465```mermaid66flowchart LR67 T[Transcript, lines numbered n] --> C[Chunks ≤ 4200 characters]68 C --> M[LanguageModelSession, guided generation]69 M --> V{Cited line exists in the chunk and shares words?}70 V -->|yes| G[Grounded item linked to its segment]71 V -->|no| U[Shown as not traced]72 G --> R[Overview from section summaries] --> N[(notes.json)]73 U --> N74```75</details>7677`MeetingIntelligence.notes(for:style:)`:78791. Checks `SystemLanguageModel.default.availability` and maps the reason to a plain explanation (device not eligible, Apple Intelligence off, model not ready).802. Splits the transcript into chunks of at most 4200 characters. If the model reports a context overflow, the chunk is halved and retried.813. For each chunk, asks for a `@Generable` structure: title, summary, up to five key points, up to six action items with an owner, up to five decisions, up to four topics. Every point carries the bracketed line number it rests on. The instructions name the transcript's language explicitly — an earlier version said "the same language as the transcript" and French input sometimes came back in English.824. **Verifies** each citation: the line must be inside the chunk shown, and the point must share at least one content word (three letters or more, stop words removed, accents folded) with that line or an adjacent one. An owner is kept only if the name occurs in the transcript.835. With several chunks, asks for one overall title and summary from the section summaries, and keeps each section as a chapter with its start time.846. Strips stray citation marks such as "(7)" from prose.8586Items that fail step 4 are kept but flagged, and the interface lists them under "Not traced to a transcript line". Questions retrieve up to ten segments by shared vocabulary, ask for an answer with cited lines, and treat an answer without a valid citation as unanswerable.8788Verification is lexical. It catches invented items and wrong line numbers; it cannot tell whether a paraphrase is faithful. That is why every output is labelled AI-generated, editable, and one tap from its source audio.8990### Where it has run9192| Environment | Result |93|---|---|94| macOS 27, Apple M5 Max | works: English 7/7 points grounded, French 8/8; out-of-transcript questions refused; about 7–8 s per one-minute transcript |95| iOS 26.5 simulator | the model reports itself available, then generation fails with a generic error; the app shows a message |96| Physical iPhone or iPad | never run |97