Repository Public
1> [!WARNING]2> **Preview:** The Gemini Flex API is in [Preview](https://cloud.google.com/products#product-launch-stages).34The Gemini Flex API is an inference tier that offers a 50% cost reduction5compared to standard rates, in exchange for variable latency and best-effort6availability. It's designed for latency-tolerant workloads that require7synchronous processing but don't need the real-time performance of the standard8API.910## How to use Flex1112To use the Flex tier, specify the `service_tier` as `flex` in your request. By default, requests use the standard tier if this field is omitted.1314### Python1516 from google import genai1718 client = genai.Client()1920 interaction = client.interactions.create(21 model="gemini-3.8-flash",22 input="Analyze this dataset for trends...",23 service_tier='flex'24 )25 print(interaction.output_text)2627### JavaScript2829 import { GoogleGenAI } from '@google/genai';3031 const client = new GoogleGenAI({});3233 async function main() {34 const interaction = await client.interactions.create({35 model: 'gemini-3.8-flash',36 input: 'Analyze this dataset for trends...',37 service_tier: 'flex'38 });39 console.log(interaction.output_text);40 }41 await main();4243### Java4445 import com.google.genai.Client;46 import com.google.genai.gaos.models.interactions.CreateModelInteraction;47 import com.google.genai.gaos.models.interactions.Interaction;48 import com.google.genai.gaos.models.interactions.InteractionsInput;49 import com.google.genai.gaos.models.interactions.Model;50 import com.google.genai.gaos.models.interactions.ServiceTier;51 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;5253 Client client = new Client();5455 CreateModelInteraction params =56 CreateModelInteraction.builder()57 .model(Model.of("gemini-3.8-flash"))58 .input(InteractionsInput.of("Analyze this dataset for trends..."))59 .serviceTier(ServiceTier.FLEX)60 .build();6162 Interaction interaction =63 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();6465 System.out.println(interaction.outputText().orElse(""));6667### REST6869 curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \70 -H "Content-Type: application/json" \71 -H "x-goog-api-key: $GEMINI_API_KEY" \72 -d '{73 "model": "gemini-3.8-flash",74 "input": "Analyze this dataset for trends...",75 "service_tier": "flex"76 }'7778## How Flex inference works7980Gemini Flex inference bridges the gap between the standard API and the 24-hour81turnaround of the [Batch API](https://ai.google.dev/gemini-api/docs/batch-api). It utilizes off-peak,82"sheddable" compute capacity to provide a cost-effective solution for background83tasks and sequential workflows.8485| Feature | Flex | Priority | Standard | Batch |86|---|---|---|---|---|87| **Pricing** | 50% discount | 75-100% more than Standard | Full price | 50% discount |88| **Latency** | Minutes (1--15 min target) | Low (Seconds) | Seconds to minutes | Up to 24 hours |89| **Reliability** | Best-effort (Sheddable) | High (Non-sheddable) | High / Medium-high | High (for throughput) |90| **Interface** | Synchronous | Synchronous | Synchronous | Asynchronous |9192### Key benefits9394- **Cost efficiency**: Substantial savings for non-production evals, background agents, and data enrichment.95- **Low friction**: Simply add a single parameter to your existing requests.96- **Synchronous workflows**: Ideal for sequential API chains where the next request depends on the output of the previous one, making it more flexible than Batch for agentic workflows.9798### Use cases99100- **Offline evaluations**: Running "LLM-as-a-judge" regression tests or leaderboards.101- **Background agents**: Sequential tasks like CRM updates, profile building, or content moderation where minutes of delay are acceptable.102- **Budget-constrained research**: Academic experiments that require high token volume on a limited budget.103104### Rate limits105106Flex inference traffic counts towards your general [rate limits](https://aistudio.google.com/rate-limit); it doesn't107offer extended rate limits like the [Batch API](https://ai.google.dev/gemini-api/docs/batch-api).108109### Sheddable capacity110111Flex traffic is treated with lower priority. If there is a spike in112standard traffic, Flex requests may be preempted or evicted to ensure capacity113for high-priority users. If you're looking for high-priority inference, check114[Priority inference](https://ai.google.dev/gemini-api/docs/priority-inference)115116### Error codes117118When Flex capacity is unavailable or the system is congested, the API will119return standard error codes:120121- **503 Service Unavailable**: The system is currently at capacity.122- **429 Too Many Requests**: Rate limits or resource exhaustion.123124### Client responsibility125126- **No server-side fallback**: To prevent unexpected charges, the system won't automatically upgrade a Flex request to the Standard tier if Flex capacity is full.127- **Retries**: You must implement your own client-side retry logic with exponential backoff.128- **Timeouts**: Because Flex requests may sit in a queue, we recommend increasing client-side timeouts to 10 minutes or more to avoid premature connection closure.129130## Adjust timeout windows131132You can configure per-request timeouts for the REST API and client libraries.133Always ensure your client-side timeout covers the intended server patience134window (e.g., 600s+ for Flex wait queues). The SDKs expect timeout values in135milliseconds.136137### Per-request timeouts138139### Python140141 from google import genai142143 client = genai.Client(http_options={"timeout": 900000})144145 interaction = client.interactions.create(146 model="gemini-3.8-flash",147 input="why is the sky blue?",148 service_tier="flex",149 )150151### JavaScript152153 import { GoogleGenAI } from '@google/genai';154155 const client = new GoogleGenAI({});156157 async function main() {158 const interaction = await client.interactions.create({159 model: "gemini-3.8-flash",160 input: "why is the sky blue?",161 service_tier: "flex",162 }, {timeout: 900000});163 }164165 await main();166167### Java168169 import com.google.genai.Client;170 import com.google.genai.gaos.models.interactions.CreateModelInteraction;171 import com.google.genai.gaos.models.interactions.Interaction;172 import com.google.genai.gaos.models.interactions.InteractionsInput;173 import com.google.genai.gaos.models.interactions.Model;174 import com.google.genai.gaos.models.interactions.ServiceTier;175 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;176 import com.google.genai.types.HttpOptions;177178 Client client =179 Client.builder()180 .httpOptions(HttpOptions.builder().timeout(900000).build())181 .build();182183 CreateModelInteraction params =184 CreateModelInteraction.builder()185 .model(Model.of("gemini-3.8-flash"))186 .input(InteractionsInput.of("why is the sky blue?"))187 .serviceTier(ServiceTier.FLEX)188 .build();189190 Interaction interaction =191 client.interactions.create(CreateInteractionRequestBody.of(params)).interaction().get();192193## Implement retries194195Because Flex is sheddable and fails with 503 errors, here is an example of196optionally implementing retry logic to continue with failed requests:197198### Python199200 import time201 from google import genai202203 client = genai.Client()204205 def call_with_retry(max_retries=3, base_delay=5):206 for attempt in range(max_retries):207 try:208 return client.interactions.create(209 model="gemini-3.8-flash",210 input="Analyze this batch statement.",211 service_tier="flex",212 )213 except Exception as e:214 if attempt < max_retries - 1:215 delay = base_delay * (2 ** attempt) # Exponential Backoff216 print(f"Flex busy, retrying in {delay}s...")217 time.sleep(delay)218 else:219 print("Flex exhausted, falling back to Standard...")220 return client.interactions.create(221 model="gemini-3.8-flash",222 input="Analyze this batch statement."223 )224225 interaction = call_with_retry()226 print(interaction.output_text)227228### JavaScript229230 import { GoogleGenAI } from '@google/genai';231232 const ai = new GoogleGenAI({});233234 async function sleep(ms) {235 return new Promise(resolve => setTimeout(resolve, ms));236 }237238 async function callWithRetry(maxRetries = 3, baseDelay = 5) {239 for (let attempt = 0; attempt < maxRetries; attempt++) {240 try {241 console.log(`Attempt ${attempt + 1}: Calling Flex tier...`);242 const interaction = await ai.interactions.create({243 model: "gemini-3.8-flash",244 input: "Analyze this batch statement.",245 service_tier: 'flex',246 });247 return interaction;248 } catch (e) {249 if (attempt < maxRetries - 1) {250 const delay = baseDelay * (2 ** attempt);251 console.log(`Flex busy, retrying in ${delay}s...`);252 await sleep(delay * 1000);253 } else {254 console.log("Flex exhausted, falling back to Standard...");255 return await ai.interactions.create({256 model: "gemini-3.8-flash",257 input: "Analyze this batch statement.",258 });259 }260 }261 }262 }263264 async function main() {265 const interaction = await callWithRetry();266 console.log(interaction.output_text);267 }268269 await main();270271### Java272273 import com.google.genai.Client;274 import com.google.genai.gaos.models.interactions.CreateModelInteraction;275 import com.google.genai.gaos.models.interactions.Interaction;276 import com.google.genai.gaos.models.interactions.InteractionsInput;277 import com.google.genai.gaos.models.interactions.Model;278 import com.google.genai.gaos.models.interactions.ServiceTier;279 import com.google.genai.gaos.models.operations.CreateInteractionRequestBody;280281 Client client = new Client();282283 int maxRetries = 3;284 int baseDelay = 5;285 Interaction interaction = null;286287 for (int attempt = 0; attempt < maxRetries; attempt++) {288 try {289 CreateModelInteraction flexParams =290 CreateModelInteraction.builder()291 .model(Model.of("gemini-3.8-flash"))292 .input(InteractionsInput.of("Analyze this batch statement."))293 .serviceTier(ServiceTier.FLEX)294 .build();295 interaction =296 client.interactions.create(CreateInteractionRequestBody.of(flexParams)).interaction().get();297 break;298 } catch (Exception e) {299 if (attempt < maxRetries - 1) {300 int delay = baseDelay * (1 << attempt); // Exponential Backoff301 System.out.println("Flex busy, retrying in " + delay + "s...");302 Thread.sleep(delay * 1000L);303 } else {304 System.out.println("Flex exhausted, falling back to Standard...");305 CreateModelInteraction standardParams =306 CreateModelInteraction.builder()307 .model(Model.of("gemini-3.8-flash"))308 .input(InteractionsInput.of("Analyze this batch statement."))309 .build();310 interaction =311 client312 .interactions313 .create(CreateInteractionRequestBody.of(standardParams))314 .interaction()315 .get();316 }317 }318 }319320 if (interaction != null) {321 System.out.println(interaction.outputText().orElse(""));322 }323324## Pricing325326Flex inference is priced at 50% of the [standard API](https://ai.google.dev/gemini-api/docs/pricing)327and billed per token.328329## Supported models330331The following models support Flex inference:332333| Model | Flex inference |334|---|---|335| [Gemini 3.8 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash) | ✔️ |336| [Gemini 3.7 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-3.7-flash) | ✔️ |337| [Gemini 3.6 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-3.6-flash) | ✔️ |338| [Gemini 3.5 Flash-Lite](https://ai.google.dev/gemini-api/docs/models/gemini-3.5-flash-lite) | ✔️ |339| [Gemini 3.5 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-3.5-flash) | ✔️ |340| [Gemini 3.1 Flash-Lite](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-lite) | ✔️ |341| [Gemini 3.1 Pro Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3.1-pro-preview) | ✔️ |342| [Gemini 3 Flash Preview](https://ai.google.dev/gemini-api/docs/models/gemini-3-flash-preview) | ✔️ |343| [Gemini 2.5 Pro](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-pro) | ✔️ |344| [Gemini 2.5 Flash](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash) | ✔️ |345| [Gemini 2.5 Flash-Lite](https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flash-lite) | ✔️ |346347## What's next348349- [Priority inference](https://ai.google.dev/gemini-api/docs/priority-inference) for ultra-low latency.350- [Tokens](https://ai.google.dev/gemini-api/docs/tokens): Understand tokens.