SPB Git

Repository Public

maximizing-cache-hits.md 4.8 KB 179 lines markdown

1<!-- source: https://docs.x.ai/developers/advanced-api-usage/prompt-caching/maximizing-cache-hits (from https://docs.x.ai/llms-full.txt, retrieved 2026-09-19T05:28:55Z) -->2#### Prompt Caching34# Maximizing Cache Hits56## Set `x-grok-conv-id` (Chat Completions API)78The `x-grok-conv-id` HTTP header routes requests with the same conversation ID to the same server. Since cache entries are stored per-server, this maximizes your cache hit rate.910```bash customLanguage="bash"11curl https://api.x.ai/v1/chat/completions \12  -H "Content-Type: application/json" \13  -H "Authorization: Bearer $XAI_API_KEY" \14  -H "x-grok-conv-id: conv_abc123" \15  -d '{16    "model": "grok-4.6",17    "messages": [18      {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},19      {"role": "user", "content": "What is prompt caching?"}20    ]21  }'22```2324```python customLanguage="pythonOpenAISDK"25from openai import OpenAI2627client = OpenAI(28    api_key="YOUR_XAI_API_KEY",29    base_url="https://api.x.ai/v1",30)3132response = client.chat.completions.create(33    model="grok-4.6",34    messages=[35        {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},36        {"role": "user", "content": "What is prompt caching?"},37    ],38    extra_headers={39        "x-grok-conv-id": "conv_abc123",40    },41)4243print(response.choices[0].message.content)44print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")45```4647```javascript customLanguage="javascriptOpenAISDK"48import OpenAI from 'openai';4950const client = new OpenAI({51  apiKey: 'YOUR_XAI_API_KEY',52  baseURL: 'https://api.x.ai/v1',53});5455const response = await client.chat.completions.create(56  {57    model: 'grok-4.6',58    messages: [59      {60        role: 'system',61        content:62          'You are Grok, a helpful and truthful AI assistant built by xAI.',63      },64      { role: 'user', content: 'What is prompt caching?' },65    ],66  },67  {68    headers: {69      'x-grok-conv-id': 'conv_abc123',70    },71  },72);7374console.log(response.choices[0].message.content);75console.log(76  `Cached tokens: ${response.usage.prompt_tokens_details.cached_tokens}`,77);78```7980## Set `prompt_cache_key` (Responses API)8182For the Responses API, use the `prompt_cache_key` field directly in the request body. It functions identically to setting `x-grok-conv-id` — it routes requests to the same server for cache reuse.8384```bash customLanguage="bash"85curl https://api.x.ai/v1/responses \86  -H "Content-Type: application/json" \87  -H "Authorization: Bearer $XAI_API_KEY" \88  -d '{89    "model": "grok-4.6",90    "input": "What is prompt caching?",91    "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"92  }'93```9495```python customLanguage="pythonOpenAISDK"96from openai import OpenAI9798client = OpenAI(99    api_key="YOUR_XAI_API_KEY",100    base_url="https://api.x.ai/v1",101)102103response = client.responses.create(104    model="grok-4.6",105    input="What is prompt caching?",106    extra_body={107        "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366",108    },109)110111print(response.output_text)112print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")113```114115```javascript customLanguage="javascriptOpenAISDK"116import OpenAI from 'openai';117118const client = new OpenAI({119  apiKey: 'YOUR_XAI_API_KEY',120  baseURL: 'https://api.x.ai/v1',121});122123const response = await client.responses.create({124  model: 'grok-4.6',125  input: 'What is prompt caching?',126  // @ts-expect-error -- xAI-specific field127  prompt_cache_key: 'b79ad29b-b3f9-463c-bca6-041d5058d366',128});129130console.log(response.output_text);131console.log(132  `Cached tokens: ${response.usage.input_tokens_details.cached_tokens}`,133);134```135136```javascript customLanguage="javascriptAISDK"137import { xai } from '@ai-sdk/xai';138import { generateText } from 'ai';139140const { text, usage } = await generateText({141  model: xai.responses('grok-4.6'),142  prompt: 'What is prompt caching?',143  providerOptions: {144    xai: {145      promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',146    },147  },148});149150console.log(text);151console.log(`Total tokens: ${usage.totalTokens}`);152```153154## Set `x-grok-conv-id` metadata (gRPC API)155156For the gRPC API using the xAI SDK, pass `x-grok-conv-id` as gRPC metadata to enable sticky routing for cache reuse.157158```python customLanguage="pythonXAI"159from xai_sdk import Client160from xai_sdk.chat import system, user161162client = Client(163    api_key="YOUR_API_KEY",164    metadata=(("x-grok-conv-id", "conv_abc123"),),165)166167chat = client.chat.create(model="grok-4.6")168chat.append(system("You are Grok, a helpful and truthful AI assistant built by xAI."))169chat.append(user("What is prompt caching?"))170171response = chat.sample()172print(f"Response: {response.content}")173print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")174```175176## Next177178* [What Breaks Caching](/developers/advanced-api-usage/prompt-caching/multi-turn)179
Files main
Menu

Theme