SPB Git

Repository Public

custom-voices.md 19.9 KB 547 lines markdown

1<!-- source: https://docs.x.ai/developers/model-capabilities/audio/custom-voices (from https://docs.x.ai/llms-full.txt, retrieved 2026-09-19T05:28:55Z) -->2#### Model Capabilities34# Custom Voices56Clone a voice from a short reference clip and use it anywhere a built-in voice works. Upload an audio sample and immediately start using it in our TTS and Speech to Speech APIs.78> [!WARNING]9>10> Custom Voices is currently only available in the **United States**, with the exception of **Illinois**.1112## How to Use Custom Voices1314After creating a voice in the [console](https://console.x.ai/team/default/voice/voice-library?campaign=voice-docs-custom-voices\&utm_source=docs\&utm_medium=referral\&utm_campaign=developers-model-capabilities-audio-custom-voices\&utm_content=voice-library), click the three-dot menu on the voice card and select **Copy Voice ID**. If you created a custom voice via the API (Enterprise only), the `voice_id` is returned in the response.1516Custom voices are interchangeable with built-in voices across all voice APIs. Pass your `voice_id` to any of:1718* `POST /v1/tts`19* `wss://api.x.ai/v1/tts`20* `wss://api.x.ai/v1/realtime`2122Built-in voices remain available through `GET /v1/tts/voices`. Custom voices are returned by `GET /v1/custom-voices` only — they will not appear in the built-in voice list. Your custom voices are scoped to your team and are never available to other users.2324## Recording Your Reference Audio2526Create a custom voice by cloning a reference clip up to 120 seconds long. For best results:2728* **Record in a quiet setting**, ideally with a high-quality microphone.29* **Read naturally.** If it sounds like you're reading a script, the resulting voice will match this behavior.30* **Longer is better.** Clips under 30 seconds may lack detail. Aim for 90–120 seconds for the best results.31* **Speak expressively.** The resulting voice will match the expressiveness of your recording.3233### What to record3435The model picks up not just the timbre but the **delivery patterns** of the reference clip. For best results, match the recording to the content you intend to generate:3637* **Customer support** — Record realistic support exchanges including greetings, holds, troubleshooting steps, and sign-offs.38* **Audiobook narration** — Read a few paragraphs of prose with the pacing and inflection intended for the final output.39* **Conversational assistant** — Record natural, unscripted speech such as explaining a topic to a friend.40* **News or documentary** — Read a short article in a natural broadcast voice.4142A recording that reflects your intended use case will produce better results than a polished but unrelated sample.4344### Recording setup4546* **Microphone.** A studio condenser or quality USB microphone is recommended. Phone earbuds are usable but introduce noticeable noise.47* **Pop filter.** Recommended. Plosive sounds (`p`, `b`) are reproduced as audible thumps without one.48* **Room treatment.** Record in a small, soft-furnished room. Hard-walled rooms produce echo and reverb that will be reproduced in the resulting voice.49* **Single speaker.** The recording should contain only one voice with no background music or sound effects.50* **Background noise.** Silence the room. Turn off HVAC, fans, and notifications. Background noise will be cloned along with the voice.5152## Create a Custom Voice5354Get started in the console — create up to 30 custom voices for free and use them immediately across all voice APIs.5556[Clone Voice in Console](https://console.x.ai/team/default/voice/voice-library?campaign=voice-docs-custom-voices\&utm_source=docs\&utm_medium=referral\&utm_campaign=developers-model-capabilities-audio-custom-voices\&utm_content=voice-library)5758### API Quick Start5960> [!WARNING]61>62> The `POST /v1/custom-voices` endpoint is gated to teams on an Enterprise plan. **** to enable API access.6364Create a custom voice from a reference audio file, then synthesize speech with it:6566```bash67# 1. Create the voice from a reference clip (max 120s).68CREATE_RESPONSE=$(curl -s -X POST https://api.x.ai/v1/custom-voices \69  -H "Authorization: Bearer $XAI_API_KEY" \70  -F "name=Friendly Narrator" \71  -F "language=en" \72  -F "gender=female" \73  -F "tone=warm" \74  -F "use_case=narration" \75  -F "file=@reference.wav;type=audio/wav")7677echo "$CREATE_RESPONSE"78# {"voice_id":"abc123xy","name":"Friendly Narrator",...}7980# Extract the voice_id from the response (requires jq).81VOICE_ID=$(echo "$CREATE_RESPONSE" | jq -r '.voice_id')8283# 2. Use the new voice for TTS.84curl -X POST https://api.x.ai/v1/tts \85  -H "Authorization: Bearer $XAI_API_KEY" \86  -H "Content-Type: application/json" \87  -d "{88    \"text\": \"Hello! This audio was synthesized using my custom voice.\",89    \"voice_id\": \"$VOICE_ID\",90    \"language\": \"en\"91  }" \92  --output hello.mp393```9495```python customLanguage="pythonWithoutSDK"96import os97import requests9899# 1. Create the voice.100with open("reference.wav", "rb") as f:101    create = requests.post(102        "https://api.x.ai/v1/custom-voices",103        headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},104        files={"file": ("reference.wav", f, "audio/wav")},105        data={106            "name": "Friendly Narrator",107            "language": "en",108            "gender": "female",109            "tone": "warm",110            "use_case": "narration",111        },112    )113create.raise_for_status()114voice_id = create.json()["voice_id"]115116# 2. Synthesize speech with it.117speech = requests.post(118    "https://api.x.ai/v1/tts",119    headers={120        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",121        "Content-Type": "application/json",122    },123    json={124        "text": "Hello! This audio was synthesized using my custom voice.",125        "voice_id": voice_id,126        "language": "en",127    },128)129speech.raise_for_status()130with open("hello.mp3", "wb") as f:131    f.write(speech.content)132```133134```javascript customLanguage="javascriptWithoutSDK"135import fs from "fs";136137// 1. Create the voice.138const form = new FormData();139form.append("file", new Blob([fs.readFileSync("reference.wav")]), "reference.wav");140form.append("name", "Friendly Narrator");141form.append("language", "en");142form.append("gender", "female");143form.append("tone", "warm");144form.append("use_case", "narration");145146const createResp = await fetch("https://api.x.ai/v1/custom-voices", {147  method: "POST",148  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },149  body: form,150});151if (!createResp.ok) throw new Error(`Create error ${createResp.status}`);152const { voice_id } = await createResp.json();153154// 2. Synthesize speech with it.155const speech = await fetch("https://api.x.ai/v1/tts", {156  method: "POST",157  headers: {158    Authorization: `Bearer ${process.env.XAI_API_KEY}`,159    "Content-Type": "application/json",160  },161  body: JSON.stringify({162    text: "Hello! This audio was synthesized using my custom voice.",163    voice_id,164    language: "en",165  }),166});167const buffer = Buffer.from(await speech.arrayBuffer());168fs.writeFileSync("hello.mp3", buffer);169```170171```swift172import Foundation173174let apiKey = ProcessInfo.processInfo.environment["XAI_API_KEY"]!175176// 1. Create the voice.177let boundary = UUID().uuidString178var body = Data()179180func appendField(_ name: String, _ value: String) {181    body.append("--\(boundary)\r\n".data(using: .utf8)!)182    body.append("Content-Disposition: form-data; name=\"\(name)\"\r\n\r\n".data(using: .utf8)!)183    body.append("\(value)\r\n".data(using: .utf8)!)184}185186appendField("name", "Friendly Narrator")187appendField("language", "en")188appendField("gender", "female")189appendField("tone", "warm")190appendField("use_case", "narration")191192let audioData = try Data(contentsOf: URL(fileURLWithPath: "reference.wav"))193body.append("--\(boundary)\r\n".data(using: .utf8)!)194body.append("Content-Disposition: form-data; name=\"file\"; filename=\"reference.wav\"\r\n".data(using: .utf8)!)195body.append("Content-Type: audio/wav\r\n\r\n".data(using: .utf8)!)196body.append(audioData)197body.append("\r\n--\(boundary)--\r\n".data(using: .utf8)!)198199var request = URLRequest(url: URL(string: "https://api.x.ai/v1/custom-voices")!)200request.httpMethod = "POST"201request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")202request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")203request.httpBody = body204205let (data, _) = try await URLSession.shared.upload(for: request, from: body)206let json = try JSONSerialization.jsonObject(with: data) as! [String: Any]207let voiceId = json["voice_id"] as! String208209// 2. Synthesize speech with it.210var ttsRequest = URLRequest(url: URL(string: "https://api.x.ai/v1/tts")!)211ttsRequest.httpMethod = "POST"212ttsRequest.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")213ttsRequest.setValue("application/json", forHTTPHeaderField: "Content-Type")214ttsRequest.httpBody = try JSONSerialization.data(withJSONObject: [215    "text": "Hello! This audio was synthesized using my custom voice.",216    "voice_id": voiceId,217    "language": "en",218])219220let (audioBytes, _) = try await URLSession.shared.data(for: ttsRequest)221try audioBytes.write(to: URL(fileURLWithPath: "hello.mp3"))222```223224## Endpoints225226All endpoints sit under `https://api.x.ai/v1/custom-voices` and authenticate with a Bearer API key.227228### Create a custom voice229230`POST /v1/custom-voices` with `multipart/form-data`. Only `file` is required.231232| Field | Type | Required | Description |233|-------|------|----------|-------------|234| `file` | binary | yes | Reference audio. Max 120 s. |235| `name` | string | | Display name. |236| `description` | string | | Free-text description. |237| `gender` | string | | `male`, `female`, or `neutral`. |238| `accent` | string | | Free text (e.g. `British`, `American`). |239| `age` | string | | `young`, `middle-aged`, or `old`. |240| `language` | string | | ISO 639 (`en`) or BCP-47-style (`en-US`, `zh-CN`). Region must be uppercase. |241| `use_case` | string | | `conversational`, `narration`, `characters`, `educational`, `advertisement`, `social_media`, `entertainment`. |242| `tone` | string | | `warm`, `casual`, `professional`, `friendly`, `authoritative`, `expressive`, `calm`. |243244The following formats and settings are recommended for the uploaded reference file:245246| Setting | Recommendation |247|---------|----------------|248| **Codec** | `.wav` (uncompressed PCM) is recommended. MP3, FLAC, OGG, Opus, M4A, AAC, MKV, and MP4 are also accepted, but lossy formats may introduce compression artifacts that are reproduced in the resulting voice. |249| **Sample rate** | **24 kHz** recommended. Higher rates (44.1 kHz, 48 kHz) are downsampled server-side. Lower rates result in reduced fidelity. |250| **Bit depth** | **16-bit PCM** is sufficient. 24-bit is also supported. |251| **Channels** | **Mono** recommended. Stereo files are downmixed automatically, but recording in mono avoids potential phase artifacts. |252253#### Length254255* **No minimum, 120s maximum.** Clips of any length up to 120 seconds are accepted; longer clips are rejected with `400`.256* **90+ seconds recommended.** Longer clips capture more prosody and intonation variety, producing a more natural and expressive voice.257258A successful create returns `201` with the new voice object:259260```json261{262  "voice_id": "nlbqfwie",263  "name": "Friendly Narrator",264  "description": "Warm, conversational tone for narration.",265  "gender": "female",266  "accent": "American",267  "age": "young",268  "language": "en",269  "use_case": "narration",270  "tone": "warm",271  "created_at": "2026-04-26T18:56:34.872993+00:00"272}273```274275`voice_id` is an 8-character lowercase alphanumeric identifier.276277### List custom voices278279`GET /v1/custom-voices` returns all voices owned by your team, paginated.280281| Query parameter | Default | Description |282|-----------------|---------|-------------|283| `limit` | `100` | Page size, 1-1000. |284| `pagination_token` | | Token from the previous response. Omit on the first page. |285286```bash287curl -s "https://api.x.ai/v1/custom-voices?limit=50" \288  -H "Authorization: Bearer $XAI_API_KEY"289```290291```python customLanguage="pythonWithoutSDK"292import os293import requests294295response = requests.get(296    "https://api.x.ai/v1/custom-voices",297    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},298    params={"limit": 50},299)300for voice in response.json()["voices"]:301    print(f"{voice['voice_id']:10s}  {voice.get('name')}")302```303304```javascript customLanguage="javascriptWithoutSDK"305const response = await fetch(306  "https://api.x.ai/v1/custom-voices?limit=50",307  { headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` } },308);309const { voices } = await response.json();310voices.forEach((v) => console.log(`${v.voice_id}  ${v.name}`));311```312313Response:314315```json316{317  "voices": [318    {319      "voice_id": "nlbqfwie",320      "name": "Friendly Narrator",321      "description": "Warm, conversational tone for narration.",322      "gender": "female",323      "accent": "American",324      "age": "young",325      "language": "en",326      "use_case": "narration",327      "tone": "warm",328      "created_at": "2026-04-26T18:56:34.872993+00:00"329    }330  ],331  "pagination_token": null332}333```334335### Get a custom voice336337`GET /v1/custom-voices/{voice_id}` returns the metadata for a single voice. Returns `404` for unknown ids or for voices owned by another team.338339Response body matches the voice object format shown in [Create](#create-a-custom-voice).340341### Update metadata342343`PATCH /v1/custom-voices/{voice_id}` with a JSON body. All fields are optional and follow these rules:344345* **Field omitted** — no change.346* **Field set to `null`** — clears the value.347* **Field set to a non-empty string** — updates the value.348* **Field set to `""`** — rejected with `400`.349350This endpoint never changes the underlying audio. To re-record, delete the voice and create a new one.351352```bash353curl -X PATCH "https://api.x.ai/v1/custom-voices/nlbqfwie" \354  -H "Authorization: Bearer $XAI_API_KEY" \355  -H "Content-Type: application/json" \356  -d '{"description": "Updated after a tuning pass.", "tone": "calm"}'357```358359```python customLanguage="pythonWithoutSDK"360import os361import requests362363response = requests.patch(364    "https://api.x.ai/v1/custom-voices/nlbqfwie",365    headers={366        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",367        "Content-Type": "application/json",368    },369    json={"description": "Updated after a tuning pass.", "tone": "calm"},370)371print(response.json())372```373374```javascript customLanguage="javascriptWithoutSDK"375const response = await fetch(376  "https://api.x.ai/v1/custom-voices/nlbqfwie",377  {378    method: "PATCH",379    headers: {380      Authorization: `Bearer ${process.env.XAI_API_KEY}`,381      "Content-Type": "application/json",382    },383    body: JSON.stringify({384      description: "Updated after a tuning pass.",385      tone: "calm",386    }),387  },388);389if (!response.ok) throw new Error(`Update error ${response.status}: ${await response.text()}`);390console.log(await response.json());391```392393Returns the full updated voice object:394395```json396{397  "voice_id": "nlbqfwie",398  "name": "Friendly Narrator",399  "description": "Updated after a tuning pass.",400  "gender": "female",401  "accent": "American",402  "age": "young",403  "language": "en",404  "use_case": "narration",405  "tone": "calm",406  "created_at": "2026-04-26T18:56:34.872993+00:00"407}408```409410### Download the reference audio411412`GET /v1/custom-voices/{voice_id}/audio` streams the original reference file with the appropriate `Content-Type` header (e.g. `audio/wav`, `audio/mpeg`).413414### Delete a custom voice415416`DELETE /v1/custom-voices/{voice_id}` removes the voice and its underlying audio.417418```bash419curl -X DELETE "https://api.x.ai/v1/custom-voices/nlbqfwie" \420  -H "Authorization: Bearer $XAI_API_KEY"421```422423```python customLanguage="pythonWithoutSDK"424import os425import requests426427requests.delete(428    "https://api.x.ai/v1/custom-voices/nlbqfwie",429    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},430)431```432433```javascript customLanguage="javascriptWithoutSDK"434await fetch("https://api.x.ai/v1/custom-voices/nlbqfwie", {435  method: "DELETE",436  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },437});438```439440The response is `{"deleted": true}`. After deletion, subsequent requests for the same `voice_id` return `404` and any TTS / Speech to Speech calls referencing it will fail with an unknown-voice error.441442## Using a Custom Voice443444Once created, a custom `voice_id` works wherever a built-in `voice_id` works.445446### REST TTS447448```bash449curl -X POST https://api.x.ai/v1/tts \450  -H "Authorization: Bearer $XAI_API_KEY" \451  -H "Content-Type: application/json" \452  -d '{453    "text": "Welcome back. How can I help today?",454    "voice_id": "nlbqfwie",455    "language": "en"456  }' \457  --output welcome.mp3458```459460### Streaming TTS WebSocket461462Pass the custom voice as the `voice` query parameter when opening the connection. See [Text to Speech - Streaming](/developers/model-capabilities/audio/text-to-speech#streaming-tts-websocket) for the full event protocol.463464**Prerequisite:** Install the WebSocket client library — `pip install websockets` (Python) or `npm install ws` (Node.js).465466```python customLanguage="pythonWithoutSDK"467import asyncio468import base64469import json470import os471import websockets472473async def stream_with_custom_voice(voice_id: str):474    uri = f"wss://api.x.ai/v1/tts?language=en&voice={voice_id}&codec=mp3"475    async with websockets.connect(476        uri,477        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},478    ) as ws:479        await ws.send(json.dumps({"type": "text.delta", "delta": "Streaming with my custom voice."}))480        await ws.send(json.dumps({"type": "text.done"}))481        audio = bytearray()482        async for msg in ws:483            event = json.loads(msg)484            if event["type"] == "audio.delta":485                audio.extend(base64.b64decode(event["delta"]))486            elif event["type"] == "audio.done":487                break488        with open("stream.mp3", "wb") as f:489            f.write(audio)490491asyncio.run(stream_with_custom_voice("nlbqfwie"))492```493494### Speech to Speech API495496Set `voice` in the `session.update` message. See the [Speech to Speech API docs](/developers/model-capabilities/audio/speech-to-speech) for the full session lifecycle.497498```python customLanguage="pythonWithoutSDK"499import asyncio500import json501import os502import websockets503504async def realtime_with_custom_voice(voice_id: str):505    async with websockets.connect(506        "wss://api.x.ai/v1/realtime",507        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},508    ) as ws:509        await ws.send(json.dumps({510            "type": "session.update",511            "session": {512                "voice": voice_id,513                "instructions": "You are a helpful assistant.",514                "turn_detection": {"type": "server_vad"},515            },516        }))517        # ... continue with the standard realtime event loop ...518519asyncio.run(realtime_with_custom_voice("nlbqfwie"))520```521522## Limits523524| | Value |525|---|---:|526| Reference audio max duration | 120 seconds |527| Custom voices per team | **30** |528| Voice ID length | 8 characters, lowercase alphanumeric |529530### Need more than 30 voices?531532The default limit is 30 custom voices per team. If you need more, contact us to discuss higher limits.533534[Request more custom voices](https://x.ai/contact-sales)535536## Error Handling537538| Status | Meaning | Action |539|--------|---------|--------|540| `201` | Voice created | Save `voice_id` and start using it. |541| `200` | Successful read / update / delete | - |542| `400` | Bad request | Check: audio under 120 s; label values are within the allowed enums; PATCH does not contain empty strings. Also returned when the team's 30-voice limit is reached — delete an existing voice or [request more](https://x.ai/contact-sales). |543| `401` | Unauthorized | API key is missing or invalid. |544| `403` | Custom voices not enabled for this team, or `POST /v1/custom-voices` was called without an Enterprise contract | Create voices in the [console playground](https://console.x.ai/team/default/voice/voice-library?campaign=voice-docs-custom-voices\&utm_source=docs\&utm_medium=referral\&utm_campaign=developers-model-capabilities-audio-custom-voices\&utm_content=voice-library), or [contact sales](https://x.ai/contact-sales) to enable the create API. |545| `404` | Voice not found | The id does not exist or is owned by another team. |546| `500` | Server error | Retry with exponential backoff. |547
Files main
Menu

Theme