Repository Public
1# AI integration: Claude Studio and the Anthropic client23ORIX DJ can ask an Anthropic model to write, explain, fix or improve an ORIX Script automation, design an effect, plan a performance or draft a MIDI mapping. The feature is optional, uses the user's own API key, and is the only part of the application that opens a network connection. This document describes exactly what is sent, how the answer is handled, how the key is stored, and what has and has not been tested.45One fact first, because it qualifies everything below: **no request has ever been sent to the real Anthropic API from this code.** No API key was available during development. The client was exercised only against local stub transports (a `URLProtocol` in the tests and a debug-only one in the application). Wire-format details are implemented from the API documentation and verified against those stubs, not against the service.67Related documents: [automation](automation.md) (compiler, runtime, simulator), [overview](overview.md), [persistence](persistence.md), [security report](../reports/security.md), [testing](../development/testing.md), and the user-facing [security chapter](../orix-script/security.md).89## Components1011| Component | File | Responsibility |12| --- | --- | --- |13| `AnthropicClient` | `Packages/ORIXDJCore/Sources/ORIXDJEngine/Anthropic/AnthropicClient.swift` | Raw HTTPS client for two endpoints (Models, Messages), streaming, typed errors, bounded retries, hourly request budget. The file header notes that there is no official Swift SDK. |14| `APIKeyStore` | same file | Keychain storage of the key and the masked display form. |15| `AnthropicSettings` | same file | Model, effort, maximum output tokens, requests per hour, confirm-before-sending, refusal fallback. Stored in preferences; the key is not part of it. |16| `ScriptAssistant`, `AssistantContext`, `AssistantProposal` | `Anthropic/ScriptAssistant.swift` | Builds the request, extracts the script or mapping from the answer, validates it locally, computes the list of risks and the diff. |17| `ClaudeStudioModel` | `App/Sources/Features/ClaudeStudio/ClaudeStudioModel.swift` | Conversation state, sharing switches, streaming, cancellation, validation off the main thread. |18| `ClaudeStudioView`, `ClaudeContextPanel`, confirmation sheet | `ClaudeStudio/ClaudeStudioView.swift` | Task picker, prompt, context preview, "Send this to Anthropic?" sheet. |19| `ClaudeProposalCard` | `ClaudeStudio/ClaudeProposalCard.swift` | Shows a validated proposal and the user actions on it. |20| `ClaudeAccess` | `ClaudeStudio/ClaudeAccess.swift` | The only application code that reads the key; exposes the masked form to views. |21| `AISettingsTab` | `App/Sources/Features/Settings/AISettingsTab.swift` | Key entry, validation, model picker, limits. |22| `ClaudeStub` | `ClaudeStudio/ClaudeStub.swift` | Debug-only local transport. |2324## Script generation workflow2526```mermaid27sequenceDiagram28 actor User29 participant Studio as ClaudeStudio30 participant Assistant as ScriptAssistant31 participant Client as AnthropicClient32 participant API as Anthropic API33 User->>Studio: choose task, type prompt, set sharing switches34 Studio->>Assistant: request(task, prompt, context, history)35 Assistant-->>Studio: AnthropicRequest (system prompt and messages)36 Studio->>User: confirmation sheet with the exact message text37 User->>Studio: Send38 Studio->>Client: send(request, settings, apiKey from Keychain)39 Client->>API: POST /v1/messages with stream true40 API-->>Client: server-sent events41 Client-->>Studio: text deltas as they arrive42 Client-->>Studio: AnthropicResult after the stop reason is checked43 Studio->>Assistant: evaluate(answer, task)44 Assistant->>Assistant: extract code block, compile, simulate, list risks45 Assistant-->>Studio: AssistantProposal46 Studio->>User: proposal card (script, validation, simulation, risks, diff)47 User->>Studio: Save, Edit, Test, Copy or Run after a second confirmation48```4950The confirmation sheet appears when `AnthropicSettings.confirmBeforeSending` is true, which is the default. With it switched off in Settings, `send` goes straight to the client; the context preview in the side panel still shows the shared text continuously.5152## What is sent5354### Endpoints and headers5556| Item | Value in the code |57| --- | --- |58| Base URL | `https://api.anthropic.com` (default argument of `AnthropicClient.init`) |59| Messages | `POST /v1/messages` |60| Models | `GET /v1/models?limit=100`, then `&after_id=<last_id>` while `has_more` is true, at most 10 pages |61| `x-api-key` | the user's key |62| `anthropic-version` | `2026-10-21` (`AnthropicClient.apiVersion`) |63| `content-type` | `application/json` |64| `anthropic-beta` | `server-side-fallback-2026-07-01`, only on Messages requests that include `fallbacks` |6566No other header is set by the application. The session is `URLSessionConfiguration.ephemeral` (the code comment: "No cookies, no cache, no credential storage: nothing about a request is kept on disk"), with `timeoutIntervalForRequest = 120` seconds, `timeoutIntervalForResource = 900` seconds and `waitsForConnectivity = false`.6768### Request body (`AnthropicClient.body`)6970| Field | Value |71| --- | --- |72| `model` | `settings.model` |73| `max_tokens` | `settings.maxOutputTokens` (default 16000; the Settings screen clamps it to 256…128000) |74| `stream` | `true`, always |75| `system` | one text block containing the system prompt, with `cache_control: {"type": "ephemeral"}` |76| `messages` | array of `{"role", "content"}` with plain string content: earlier turns of the conversation, then the new user message |77| `output_config` | `{"effort": <value>}`, only when an effort is chosen in Settings (`low`, `medium`, `high`, `xhigh`, `max`; "Model default" omits the field) |78| `fallbacks` | `"default"`, only when the refusal fallback is on and the model is in `fallbackModels` |7980Not sent: `temperature`, `top_p`, `top_k`, `thinking`, `budget_tokens`, tools, images, files, metadata. `AnthropicTests.streamsTextAndSendsTheDocumentedRequest` asserts the URL, the three headers, `model`, `stream`, `fallbacks`, `output_config.effort`, the `cache_control` on the system block, the absence of the five sampling and thinking fields, and that the key does not appear in the body.8182### Models8384The application does not ship a list of models to choose from. The picker in Settings is built only from the list returned by the Models endpoint for the stored key (`AppSettings.verifiedModels`); a stored model that is not in that list is shown as "not available" with a warning. Identifiers that do appear in the code:8586- `AnthropicSettings.defaultModel = "claude-opus-5-5"`: the initial setting, and the model selected after validation when the stored one is not offered and this one is.87- `AnthropicClient.fallbackModels = ["claude-fable-5-2", "claude-fable-5-1", "claude-opus-5-5", "claude-opus-5", "claude-sonnet-5-5"]`: the models for which the `fallbacks` parameter and its beta header are sent.8889### System prompt (`ScriptAssistant.systemPrompt`)9091A constant string, identical on every request so that it can be served from the prompt cache:92931. a paragraph stating the role and that scripts are compiled, type-checked and simulated before the user decides, so "a path or statement that is not in the reference is a compile error the user will see";942. `ReferenceGenerator.languageBrief`: structure, types and literals, statements, every deck, mixer, master and effect path with its type and range, actions, events, builtins and limits, generated from the same `ScriptSchema` the compiler uses (see [automation](automation.md));953. one example automation ("Eight Bar Transition");964. the output contract: a complete script in one `orixdj` code block, a short bar-by-bar explanation, what to check before running live, restore what is changed, say so when the request cannot be expressed, do not rewrite when only asked to explain.9798The system prompt contains nothing about the user, the library or the machine.99100### User message (`ScriptAssistant.request`)101102```text103<instruction for the task>104105Request: <the user's prompt, when not empty>106107Context from the application:108<AssistantContext.rendered()>109```110111There are seven tasks (`AssistantTask`): `generate`, `explain`, `fix`, `improve`, `effect`, `midi`, `plan`, each with one fixed instruction sentence or paragraph. For `midi`, the list of mappable control targets (`ControlTargets.all`, identifier and kind) is appended to the context, and the instruction asks for a JSON object in a `json` code block.112113`AssistantContext.rendered()` produces the context text. Each part is controlled by a switch in the Claude Studio side panel:114115| Part | Content | Switch and default in `ClaudeStudioModel` |116| --- | --- | --- |117| Language version, deck letters | `ORIX Script version: 1.0`, `Decks available: A, B, C, D` | always |118| Deck states | master clock tempo; per deck: empty, loaded and paused, or playing; tempo; Camelot key | `includeDecks`, on |119| Track titles | title of each loaded track | `includeTitles`, off; the code comment calls titles "private library data" |120| Active effects | names of effect units that are switched on, per rack | `includeEffects`, on |121| MIDI controller | names of online MIDI sources, `midi` task only | `includeController`, on |122| Current script | source of the chosen editor document in an `orixdj` block | `includeScript`, off initially; set automatically to on for `explain`, `fix`, `improve` and off for the other tasks when the task changes |123| Compiler messages, simulator output | diagnostics with line, column and code; a short simulation summary | `includeDiagnostics`, on, effective only with the script |124125The diagnostics come from `ScriptCheck.run`, which compiles and simulates the shared script locally 250 ms after it changes.126127History: every earlier turn of the conversation that did not end in an error is resent as a user message (the exact text sent then) and an assistant message (the full answer). "New Conversation" clears it.128129Not part of any request: audio, file paths, file contents other than the shared script, library contents, crates, playlists, play history, MIDI mappings, settings.130131### What the user sees before sending132133- The side panel (`ClaudeContextPanel`) shows, under "Exact text that will be sent", the output of `studio.context(model).rendered()`, refreshed every second, with the statement "Audio is never uploaded. Your request, the ORIX Script language reference and the text above are all that is sent."134- The confirmation sheet shows the complete user message exactly as sent (`pending.request.messages.last?.content`), the number of earlier messages that are resent, the length of the language reference with a disclosure that shows its full text, the destination ("To api.anthropic.com over HTTPS, with your API key") and the model identifier.135- After sending, each turn in the conversation keeps `sentText`, the exact user message.136137## Streaming138139The response is read with `URLSession.bytes(for:)` and iterated line by line. Only lines starting with `data:` are considered; each payload is parsed as JSON and dispatched on its `type`:140141| Event type | What is read |142| --- | --- |143| `message_start` | `message.model` (the model that actually answers), `usage.input_tokens`, `usage.cache_read_input_tokens`, `usage.cache_creation_input_tokens` |144| `content_block_delta` | only when `delta.type` is `text_delta`: `delta.text` is appended to the result and passed to `onText`. Other delta types, including thinking deltas, are ignored |145| `message_delta` | `delta.stop_reason`, `delta.stop_details.category`, `usage.output_tokens` |146| `error` | `overloaded_error` becomes `.overloaded`; anything else becomes `.server(500, message)` |147| anything else (`content_block_start`, `content_block_stop`, `message_stop`, `ping`, …) | ignored |148149The `request-id` response header is kept in `AnthropicResult.requestID`. Text deltas arrive on a network thread; `ClaudeStudioModel` forwards them through an `AsyncStream` to the main actor so that they are appended in order.150151After the stream ends the stop reason is checked before the text is used (code comment: "Always look at the stop reason before trusting the text"): `refusal` throws `.refusal(category:)`, `max_tokens` throws `.truncated`, an empty stop reason throws `.invalidResponse`.152153## Errors, retries, cancellation, budget154155`AnthropicClient.error(status:body:headers:model:)` maps HTTP status codes to `AnthropicError`. The message is taken from the `error.message` field of the response body when present (the first 65536 bytes are read).156157| Condition | `AnthropicError` | Retried |158| --- | --- | --- |159| Empty key | `.noKey` | no; nothing is sent |160| Hourly budget reached | `.requestLimit(n)` | no; nothing is sent |161| HTTP 400, 413, 422 | `.badRequest(message)` | once without `fallbacks` and the beta header, only when the message mentions `fallback` or `anthropic-beta` |162| HTTP 401 | `.unauthorized` | no |163| HTTP 402 | `.billing` | no |164| HTTP 403 | `.forbidden(message)` | no |165| HTTP 404 | `.modelNotFound(model)` | no |166| HTTP 429 | `.rateLimited(retryAfter:)` from the `retry-after` header | no |167| HTTP 529, or `overloaded_error` in the stream | `.overloaded` | yes |168| Other status | `.server(code, message)` | yes when the code is 500 or above |169| Transport failure | `.network(description)` | yes |170| Stop reason `refusal` | `.refusal(category:)` | no |171| Stop reason `max_tokens` | `.truncated` | no |172| Unparseable response, missing stop reason | `.invalidResponse` | no |173| Task cancelled or `URLError.cancelled` | `.cancelled` | no |174175Retries are bounded: at most two retries after the first attempt, with pauses of 1.5 s and 3 s (`pow(2, attempt) * 0.75` seconds). `AnthropicTests.errorsAreTypedAndRetriesAreBounded` checks the mapping of 400, 401, 402, 403, 404, 429 and 529, counts exactly three requests for a persistent 529, and checks the refusal, truncation, fallback-rejection, missing-key and budget cases. The `models` call has no retry.176177The hourly budget (`maxRequestsPerHour`, default 60, 1…600 in Settings) is counted in memory per `AnthropicClient` instance, once per `send` call regardless of retries, and starts from zero at each launch (the Settings row says "Counted since ORIX DJ started").178179Cancellation: the Stop button cancels the task that awaits `send`. The turn is recorded with the text received so far and the error "Cancelled."; a failed or cancelled request puts the prompt back in the field so that it can be sent again.180181Refusals: when `refusalFallback` is on (default) and the model is in `fallbackModels`, the request opts into the server-side fallback, and the result's `model` field, taken from `message_start`, names the model that wrote the answer. Otherwise a refusal is shown as an error ("Claude declined this request", with the category when given). An answer with no code block is not an error: the proposal simply has no script and only the explanation is shown.182183Each error has a user-readable description (`errorDescription`) that says what to do next, for example "Anthropic rejected the API key. Check it in Settings → AI → Anthropic."184185## Validation of the answer186187```mermaid188flowchart TD189 A["Answer text"] --> T{"Task is midi?"}190 T -->|"yes"| J["First json or untagged code block"]191 J --> JP["Decode MIDIMappingProfile"]192 JP -->|"fails"| JE["Issue: the mapping could not be read"]193 JP -->|"ok"| JF["Drop entries with unknown targets, run profile.validate"]194 JF --> MC["MIDI proposal card with issues; Save Mapping asks for confirmation"]195 T -->|"no"| B["First orixdj, orix, orixscript or untagged block containing the word automation"]196 B -->|"none"| N["Explanation only, no script"]197 B -->|"found"| C["ScriptCompiler.compile with default capabilities and limits"]198 C -->|"errors"| X["Not valid: show diagnostics; no simulation"]199 C -->|"program"| S["Simulator.run in the simulated rig"]200 S --> R["Compute risks from requirements, simulation and source"]201 R --> V{"Compiled and simulation did not fail?"}202 V -->|"no"| X2["Not valid: Run is disabled"]203 V -->|"yes"| OK["Validated: Run is enabled, behind a confirmation"]204```205206`ScriptAssistant.evaluate(answer:task:simulation:)` is the whole validation. It treats the answer as untrusted text:2072081. The first fenced block tagged `orixdj`, `orix` or `orixscript` (or untagged) that contains the word `automation` is extracted. Everything else is the explanation.2092. The script is compiled with `ScriptCompiler.compile`, the same entry point and the same `CompileLimits` as a script typed in the editor.2103. If it compiles, `Simulator.run` executes the first automation in a `SimulatedRig` with the default `SimulationConfig` (deck A playing, deck B loaded, 128 BPM, at most 256 beats, default `RuntimeLimits`).2114. `risks(...)` computes a list locally from the compiled requirements, the simulated trace and the source text, not from the model's description: waits for events; longer than 256 beats; uses seconds; which decks it stops; loads or ejects tracks; which effects are still on at the end; simulation timed out; clamped values; up to three simulator warnings; a failed simulated run; changes `mixer.master.volume`; up to three compiler warnings.2125. `AssistantProposal.isValid` is `compile.succeeded && simulation.error == nil`.213214For Fix and Improve, the card also shows a line diff (`ScriptAssistant.diff`, longest common subsequence) between the script that was shared and the proposed one.215216### Model output is never run automatically217218Verified by reading every call site:219220- `ClaudeStudioModel.perform` ends by storing the proposal in the turn. Its file header states: "A proposal is never run from here: running is a user action." The model has no reference to `AutomationHost`.221- `ScriptAssistant.evaluate` calls only the compiler and the simulator. The simulator's target is a `SimulatedRig`; it has no path to `DJEngine`.222- The proposal card offers Save (to the script library, origin `.ai`), Edit (open in the editor), Test (simulate again), Copy, and Run…. Run is disabled unless the proposal is valid, and opens an alert, "Run this automation live?", whose text names the decks the script addresses, says which of them are playing now, quotes the first risk and mentions the emergency stop. Only the alert's "Run Live" button calls `AppModel.run(source:origin: "claude")`, which compiles the text again and starts it on the automation host like any other script.223- A MIDI mapping proposal is saved only after a confirmation dialog ("Save and Activate", "Save Without Activating", "Cancel").224225One launch option exists for screenshots: when the application is started with `ORIXClaudePrompt`, Claude Studio fills the prompt and calls `send`; with `ORIXClaudeAutoConfirm` it also confirms the sheet, but only when the debug stub transport is active. Neither option runs a proposal.226227`AnthropicTests.proposalsAreValidatedLocally` covers: a valid script (requirements, static length 32 beats, the "stops deck A" and "fx.echo switched on" risks); invented syntax (`deck.A.scratch(4beats)` and `system("rm -rf")` give two compile errors and no simulation); a script that compiles but fails in the simulator (`deck.D.play()` on an empty deck is not valid); an answer without a script; a MIDI proposal with an unknown target (`deck.A.launchMissiles` is removed and reported); the title switch of the context; and the diff.228229## API key handling230231| Aspect | Implementation |232| --- | --- |233| Storage | macOS Keychain, `kSecClassGenericPassword` |234| Service | `ai.spboucher.orix.dj.anthropic` (`APIKeyStore.defaultService`); isolated instances started with their own data directory (tests, screenshots) use the same name with the suffix `.test` |235| Account | `api-key` |236| Accessibility | `kSecAttrAccessibleWhenUnlockedThisDeviceOnly` |237| Label | `ORIX DJ – Anthropic API key` |238| Save | whitespace trimmed; the existing item is deleted, then the new one added; saving an empty string only deletes |239| Display | `APIKeyStore.masked`: the first 7 characters, an ellipsis and the last 4 (`sk-ant-…abcd`); a key of 12 characters or fewer is shown as bullets |240| Entry | a `SecureField`; the field is cleared as soon as Save is pressed |241| Validation | a call to the Models endpoint; success stores the returned list, `401` clears it |242| Removal | confirmation dialog, then `SecItemDelete`; the verified model list is cleared |243244The key is read from the Keychain at the moment of a request (`ClaudeAccess.keyForRequest()`), passed as a parameter to `AnthropicClient`, and placed in the `x-api-key` header. It is not held in a stored property of any view or model object; views only ever receive `maskedKey`. It is not part of `AnthropicSettings` (which is what is written to preferences under `ai.anthropic`), not part of projects, and not part of the request body. No code in the client or in Claude Studio writes to a log; `Scripts/check.sh` fails the repository check if a line calls `print`, `NSLog`, `os_log` or `Logger` with `apiKey` or `api_key` on it, and scans the repository for strings shaped like a real key (`sk-ant-` followed by 24 or more characters, the test placeholder excepted). Error messages shown to the user are built from the HTTP status and the server's `error.message`, never from the request.245246`AnthropicTests.keychainStoresAndMasksTheKey` saves, replaces, loads and deletes a key under a unique service name in the real Keychain and checks the masked form. See the [security report](../reports/security.md) for the limits of these checks.247248## The local stub249250`ClaudeStub.swift` is compiled only in Debug builds (`#if DEBUG`). When the application is launched with the default `ORIXClaudeStub` set to true, `ClaudeAccess` replaces the client with one whose `URLSession` uses `StubURLProtocol`:251252- `GET …/v1/models` returns three canned models (`claude-fable-5-2`, `claude-opus-5-5`, `claude-sonnet-5-5`);253- any other request is answered as a Messages stream in the server-sent-event format (`message_start`, `content_block_start`, `content_block_delta` chunks of about 14 characters every 18 ms, `content_block_stop`, `message_delta` with `end_turn`, `message_stop`);254- the answer depends on the first words of the user message: a canned script for most tasks, a canned mapping for MIDI (which deliberately contains one unknown target, `deck.A.sparkle`), canned prose for Explain, and for Fix and Improve a modified copy of the script that was shared, so that the diff view has something real to show.255256In stub mode the header carries the placeholder `stub-transport-no-key` instead of a key, no Keychain item is read, Settings shows a "Debug stub transport" row, and Claude Studio shows a badge. The request never leaves the process. The stub exists so that the request, streaming, validation and presentation path can be exercised and captured in screenshots without a key; it is not present in a Release build.257258## What was tested and what was not259260Tested (suite `AnthropicTests` in `Tests/ORIXDJEngineTests/SessionTests.swift`, five tests, all passing in the run used for this document; the transport is a `URLProtocol` stub, so no packet leaves the machine):261262- the request: URL, headers, body fields, absence of rejected parameters, prompt caching marker, key only in the header;263- streaming: text deltas delivered in order, thinking deltas ignored, token counts and request id read;264- error mapping, bounded retries, refusal and truncation handling, retry without the fallback parameter, missing key and request budget refused before any request;265- pagination of the model list and the `401` case;266- Keychain save, replace, load, delete and masking;267- local validation of proposals, the context switches and the diff.268269Not tested, and not verified in any other way:270271- any exchange with the real Anthropic API. It is therefore unverified that the service accepts the `anthropic-version` value, the `fallbacks` parameter with its beta header, the `output_config.effort` values offered in Settings, or the default model identifier, and that the real event stream is parsed correctly in all cases (for example an `error` event after partial text, or a fallback answer);272- prompt caching actually taking effect (the system prompt must exceed the model's minimum cacheable length);273- the quality of generated scripts. No model has ever produced a script for this application; the system prompt has not been evaluated;274- behaviour on slow or interrupted networks, proxies, and TLS failures;275- the Claude Studio views themselves beyond what the stub transport and screenshots exercise; there is no automated test of the confirmation sheet or of the Run confirmation.276277Known behaviours worth stating:278279- When a streamed request fails part-way with a retryable error, the retry starts a new stream and its deltas are appended after the text already shown; the displayed text is replaced by the correct final text only when the request completes.280- A `429` is reported to the user with the retry delay and is not retried automatically.281- The hourly request count is not persisted, so restarting the application resets it.282