Repository Public
1# Flex inference23> [!NOTE]4> **Note** : This version of the page covers the new [Interactions API](https://ai.google.dev/gemini-api/docs/interactions/interactions-overview), which is currently in Beta. 5> For stable production deployments, we recommend you continue to use the `generateContent` API. You can use the toggle on this page to switch between the versions.67> [!WARNING]8> **Preview:** The Gemini Flex API is in [Preview](https://cloud.google.com/products#product-launch-stages).910The Gemini Flex API is an inference tier that offers a 50% cost reduction11compared to standard rates, in exchange for variable latency and best-effort12availability. It's designed for latency-tolerant workloads that require13synchronous processing but don't need the real-time performance of the standard14API.1516## How to use Flex1718To use the Flex tier, specify the `service_tier` as `flex` in your request. By default, requests use the standard tier if this field is omitted.1920### Python2122 from google import genai2324 client = genai.Client()2526 try:27 interaction = client.interactions.create(28 model="gemini-3.5-flash",29 input="Analyze this dataset for trends...",30 service_tier='flex'31 )32 print(interaction.output_text)33 except Exception as e:34 print(f"Flex request failed: {e}")3536### JavaScript3738 import { GoogleGenAI } from '@google/genai';3940 const client = new GoogleGenAI({});4142 async function main() {43 try {44 const interaction = await client.interactions.create({45 model: 'gemini-3.5-flash',46 input: 'Analyze this dataset for trends...',47 service_tier: 'flex'48 });49 console.log(interaction.output_text);50 } catch (e) {51 console.log(`Flex request failed: ${e}`);52 }53 }54 await main();5556### REST5758 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \59 -H "Content-Type: application/json" \60 -H "x-goog-api-key: $GEMINI_API_KEY" \61 -H "Api-Revision: 2026-05-20" \62 -d '{63 "model": "gemini-3.5-flash",64 "input": "Analyze this dataset for trends...",65 "service_tier": "flex"66 }'6768## How Flex inference works6970Gemini Flex inference bridges the gap between the standard API and the 24-hour71turnaround of the [Batch API](https://ai.google.dev/gemini-api/docs/batch-api). It utilizes off-peak,72"sheddable" compute capacity to provide a cost-effective solution for background73tasks and sequential workflows.7475| Feature | Flex | Priority | Standard | Batch |76|---|---|---|---|---|77| **Pricing** | 50% discount | 75-100% more than Standard | Full price | 50% discount |78| **Latency** | Minutes (1--15 min target) | Low (Seconds) | Seconds to minutes | Up to 24 hours |79| **Reliability** | Best-effort (Sheddable) | High (Non-sheddable) | High / Medium-high | High (for throughput) |80| **Interface** | Synchronous | Synchronous | Synchronous | Asynchronous |8182### Key benefits8384- **Cost efficiency**: Substantial savings for non-production evals, background agents, and data enrichment.85- **Low friction**: Simply add a single parameter to your existing requests.86- **Synchronous workflows**: Ideal for sequential API chains where the next request depends on the output of the previous one, making it more flexible than Batch for agentic workflows.8788### Use cases8990- **Offline evaluations**: Running "LLM-as-a-judge" regression tests or leaderboards.91- **Background agents**: Sequential tasks like CRM updates, profile building, or content moderation where minutes of delay are acceptable.92- **Budget-constrained research**: Academic experiments that require high token volume on a limited budget.9394### Rate limits9596Flex inference traffic counts towards your general [rate limits](https://aistudio.google.com/rate-limit); it doesn't97offer extended rate limits like the [Batch API](https://ai.google.dev/gemini-api/docs/batch-api).9899### Sheddable capacity100101Flex traffic is treated with lower priority. If there is a spike in102standard traffic, Flex requests may be preempted or evicted to ensure capacity103for high-priority users. If you're looking for high-priority inference, check104[Priority inference](https://ai.google.dev/gemini-api/docs/interactions/priority-inference)105106### Error codes107108When Flex capacity is unavailable or the system is congested, the API will109return standard error codes:110111- **503 Service Unavailable**: The system is currently at capacity.112- **429 Too Many Requests**: Rate limits or resource exhaustion.113114### Client responsibility115116- **No server-side fallback**: To prevent unexpected charges, the system won't automatically upgrade a Flex request to the Standard tier if Flex capacity is full.117- **Retries**: You must implement your own client-side retry logic with exponential backoff.118- **Timeouts**: Because Flex requests may sit in a queue, we recommend increasing client-side timeouts to 10 minutes or more to avoid premature connection closure.119120## Adjust timeout windows121122You can configure per-request timeouts for the REST API and client libraries.123Always ensure your client-side timeout covers the intended server patience124window (e.g., 600s+ for Flex wait queues). The SDKs expect timeout values in125milliseconds.126127### Per-request timeouts128129### Python130131 from google import genai132133 client = genai.Client(http_options={"timeout": 900000})134135 try:136 interaction = client.interactions.create(137 model="gemini-3.5-flash",138 input="why is the sky blue?",139 service_tier="flex",140 )141 except Exception as e:142 print(f"Flex request failed: {e}")143144### JavaScript145146 import { GoogleGenAI } from '@google/genai';147148 const client = new GoogleGenAI({});149150 async function main() {151 try {152 const interaction = await client.interactions.create({153 model: "gemini-3.5-flash",154 input: "why is the sky blue?",155 service_tier: "flex",156 }, {timeout: 900000});157 } catch (e) {158 console.log(`Flex request failed: ${e}`);159 }160 }161162 await main();163164## Implement retries165166Because Flex is sheddable and fails with 503 errors, here is an example of167optionally implementing retry logic to continue with failed requests:168169### Python170171 import time172 from google import genai173174 client = genai.Client()175176 def call_with_retry(max_retries=3, base_delay=5):177 for attempt in range(max_retries):178 try:179 return client.interactions.create(180 model="gemini-3.5-flash",181 input="Analyze this batch statement.",182 service_tier="flex",183 )184 except Exception as e:185 if attempt < max_retries - 1:186 delay = base_delay * (2 ** attempt) # Exponential Backoff187 print(f"Flex busy, retrying in {delay}s...")188 time.sleep(delay)189 else:190 print("Flex exhausted, falling back to Standard...")191 return client.interactions.create(192 model="gemini-3.5-flash",193 input="Analyze this batch statement."194 )195196 interaction = call_with_retry()197 print(interaction.output_text)198199### JavaScript200201 import { GoogleGenAI } from '@google/genai';202203 const ai = new GoogleGenAI({});204205 async function sleep(ms) {206 return new Promise(resolve => setTimeout(resolve, ms));207 }208209 async function callWithRetry(maxRetries = 3, baseDelay = 5) {210 for (let attempt = 0; attempt < maxRetries; attempt++) {211 try {212 console.log(`Attempt ${attempt + 1}: Calling Flex tier...`);213 const interaction = await ai.interactions.create({214 model: "gemini-3.5-flash",215 input: "Analyze this batch statement.",216 service_tier: 'flex',217 });218 return interaction;219 } catch (e) {220 if (attempt < maxRetries - 1) {221 const delay = baseDelay * (2 ** attempt);222 console.log(`Flex busy, retrying in ${delay}s...`);223 await sleep(delay * 1000);224 } else {225 console.log("Flex exhausted, falling back to Standard...");226 return await ai.interactions.create({227 model: "gemini-3.5-flash",228 input: "Analyze this batch statement.",229 });230 }231 }232 }233 }234235 async function main() {236 const interaction = await callWithRetry();237 console.log(interaction.output_text);238 }239240 await main();241242## Pricing243244Flex inference is priced at 50% of the [standard API](https://ai.google.dev/gemini-api/docs/pricing)245and billed per token.246247## Supported models248249The following models support Flex inference:250251| Model | Flex inference |252|---|---|253| [Gemini 3.5 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-3.5-flash) | ✔️ |254| [Gemini 3.1 Flash-Lite](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-lite) | ✔️ |255| [Gemini 3.1 Pro Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-pro-preview) | ✔️ |256| [Gemini 3 Flash Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3-flash-preview) | ✔️ |257| [Gemini 2.5 Pro](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-pro) | ✔️ |258| [Gemini 2.5 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash) | ✔️ |259| [Gemini 2.5 Flash-Lite](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash-lite) | ✔️ |260261## What's next262263- [Priority inference](https://ai.google.dev/gemini-api/docs/interactions/priority-inference) for ultra-low latency.264- [Tokens](https://ai.google.dev/gemini-api/docs/interactions/tokens): Understand tokens.