Repository Public
1# Batch Mode23The Gemini API Batch Mode is designed to process large volumes of requests4asynchronously at [50% of the standard cost](/gemini-api/docs/pricing).5The target turnaround time is 24 hours, but in majority of cases, it is much6quicker.78Use Batch Mode for large-scale, non-urgent tasks such as data9pre-processing or running evaluations where an immediate response is not10required.11| **Note:** You can access Batch Mode using our Python libraries or the REST API. Support for other languages (including JavaScript/TypeScript) is coming soon.1213Creating a batch job14--------------------1516You have two ways to submit your requests in Batch Mode:1718- **[Inline Requests](#inline-requests):** A list of [`GenerateContentRequest`](/api/batch-mode#GenerateContentRequest) objects directly included in your batch creation request. This is suitable for smaller batches that keep the total request size under 20MB. The **output** returned from the model is a list of `inlineResponse` objects.19- **[Input File](#input-file):** A [JSON Lines (JSONL)](https://jsonlines.org/) file where each line contains a complete [`GenerateContentRequest`](/api/batch-mode#GenerateContentRequest) object. This method is recommended for larger requests. The **output** returned from the model is a JSONL file where each line is either a `GenerateContentResponse` or a status object.2021### Inline requests2223For a small number of requests, you can directly embed the24[`GenerateContentRequest`](/api/batch-mode#GenerateContentRequest) objects25within your [`BatchGenerateContentRequest`](/api/batch-mode#request-body). The26following example calls the27[`BatchGenerateContent`](/api/batch-mode#google.ai.generativelanguage.v1beta.BatchService.BatchGenerateContent)28method with inline requests: 2930### Python313233 from google import genai34 from google.genai import types3536 client = genai.Client()3738 # A list of dictionaries, where each is a GenerateContentRequest39 inline_requests = [40 {41 'contents': [{42 'parts': [{'text': 'Tell me a one-sentence joke.'}],43 'role': 'user'44 }]45 },46 {47 'contents': [{48 'parts': [{'text': 'Why is the sky blue?'}],49 'role': 'user'50 }]51 }52 ]5354 inline_batch_job = client.batches.create(55 model="models/gemini-2.5-flash",56 src=inline_requests,57 config={58 'display_name': "inlined-requests-job-1",59 },60 )6162 print(f"Created batch job: {inline_batch_job.name}")6364### REST6566 curl https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:batchGenerateContent \67 -H "x-goog-api-key: $GEMINI_API_KEY" \68 -X POST \69 -H "Content-Type:application/json" \70 -d '{71 "batch": {72 "display_name": "my-batch-requests",73 "input_config": {74 "requests": {75 "requests": [76 {77 "request": {"contents": [{"parts": [{"text": "Describe the process of photosynthesis."}]}]},78 "metadata": {79 "key": "request-1"80 }81 },82 {83 "request": {"contents": [{"parts": [{"text": "Describe the process of photosynthesis."}]}]},84 "metadata": {85 "key": "request-2"86 }87 }88 ]89 }90 }91 }92 }'9394### Input file9596For larger sets of requests, prepare a JSON Lines (JSONL) file. Each line in97this file must be a JSON object containing a user-defined key and a request98object, where the request is a valid99[`GenerateContentRequest`](/api/batch-mode#GenerateContentRequest) object. The100user-defined key is used in the response to indicate which output is the result101of which request. For example, the request with the key defined as `request-1`102will have its response annotated with the same key name.103104This file is uploaded using the [File API](/gemini-api/docs/files). The maximum105allowed file size for an input file is 2GB.106107The following is an example of a JSONL file. You can save it in a file named108`my-batch-requests.json`: 109110 {"key": "request-1", "request": {"contents": [{"parts": [{"text": "Describe the process of photosynthesis."}]}], "generation_config": {"temperature": 0.7}}}111 {"key": "request-2", "request": {"contents": [{"parts": [{"text": "What are the main ingredients in a Margherita pizza?"}]}]}}112113Similarly to inline requests, you can specify other parameters like system114instructions, tools or other configurations in each request JSON.115116You can upload this file using the [File API](/gemini-api/docs/files) as117shown in the following example. If118you are working with multimodal input, you can reference other uploaded files119within your JSONL file. 120121### Python122123124 from google import genai125 from google.genai import types126127 client = genai.Client()128129 # Create a sample JSONL file130 with open("my-batch-requests.jsonl", "w") as f:131 requests = [132 {"key": "request-1", "request": {"contents": [{"parts": [{"text": "Describe the process of photosynthesis."}]}]}},133 {"key": "request-2", "request": {"contents": [{"parts": [{"text": "What are the main ingredients in a Margherita pizza?"}]}]}}134 ]135 for req in requests:136 f.write(json.dumps(req) + "\n")137138 # Upload the file to the File API139 uploaded_file = client.files.upload(140 file='my-batch-requests.jsonl',141 config=types.UploadFileConfig(display_name='my-batch-requests', mime_type='jsonl')142 )143144 print(f"Uploaded file: {uploaded_file.name}")145146### REST147148 tmp_batch_input_file=batch_input.tmp149 echo -e '{"contents": [{"parts": [{"text": "Describe the process of photosynthesis."}]}], "generationConfig": {"temperature": 0.7}}\n{"contents": [{"parts": [{"text": "What are the main ingredients in a Margherita pizza?"}]}]}' > batch_input.tmp150 MIME_TYPE=$(file -b --mime-type "${tmp_batch_input_file}")151 NUM_BYTES=$(wc -c < "${tmp_batch_input_file}")152 DISPLAY_NAME=BatchInput153154 tmp_header_file=upload-header.tmp155156 # Initial resumable request defining metadata.157 # The upload url is in the response headers dump them to a file.158 curl "https://generativelanguage.googleapis.com/upload/v1beta/files \159 -D "${tmp_header_file}" \160 -H "x-goog-api-key: $GEMINI_API_KEY" \161 -H "X-Goog-Upload-Protocol: resumable" \162 -H "X-Goog-Upload-Command: start" \163 -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \164 -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \165 -H "Content-Type: application/jsonl" \166 -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null167168 upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")169 rm "${tmp_header_file}"170171 # Upload the actual bytes.172 curl "${upload_url}" \173 -H "Content-Length: ${NUM_BYTES}" \174 -H "X-Goog-Upload-Offset: 0" \175 -H "X-Goog-Upload-Command: upload, finalize" \176 --data-binary "@${tmp_batch_input_file}" 2> /dev/null > file_info.json177178 file_uri=$(jq ".file.uri" file_info.json)179180The following example calls the181[`BatchGenerateContent`](/api/batch-mode#google.ai.generativelanguage.v1beta.BatchService.BatchGenerateContent)182method with the input file uploaded using File API: 183184### Python185186187 # Assumes `uploaded_file` is the file object from the previous step188 file_batch_job = client.batches.create(189 model="gemini-2.5-flash",190 src=uploaded_file.name,191 config={192 'display_name': "file-upload-job-1",193 },194 )195196 print(f"Created batch job: {file_batch_job.name}")197198### REST199200 BATCH_INPUT_FILE='files/123456' # File ID201 curl https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:batchGenerateContent \202 -X POST \203 -H "x-goog-api-key: $GEMINI_API_KEY" \204 -H "Content-Type:application/json" \205 -d "{206 'batch': {207 'display_name': 'my-batch-requests',208 'input_config': {209 'requests': {210 'file_name': ${BATCH_INPUT_FILE}211 }212 }213 }214 }"215216When you create a batch job, you will get a job name returned. Use this name217for [monitoring](#batch-job-status) the job status as well as218[retrieving the results](#retrieve-batch-results) once the job completes.219220The following is an example output that contains a job name: 221222223 Created batch job from file: batches/123456789224225### Request configuration226227You can include any request configurations you would use in a standard non-batch228request. For example, you could specify the temperature, system instructions or229even pass in other modalities. The following example shows an example inline230request that contains a system instruction for one of the requests: 231232 inline_requests_list = [233 {'contents': [{'parts': [{'text': 'Write a short poem about a cloud.'}]}]},234 {'contents': [{'parts': [{'text': 'Write a short poem about a cat.'}]}], 'system_instructions': {'parts': [{'text': 'You are a cat. Your name is Neko.'}]}}235 ]236237Similarly can specify tools to use for a request. The following example238shows a request that enables the [Google Search tool](/gemini-api/docs/google-search): 239240 inline_requests_list = [241 {'contents': [{'parts': [{'text': 'Who won the euro 1998?'}]}]},242 {'contents': [{'parts': [{'text': 'Who won the euro 2025?'}]}], 'tools': [{'google_search ': {}}]}243 ]244245You can specify [structured output](/gemini-api/docs/structured-output) as well.246The following example shows how to specify for your batch requests. 247248 from google import genai249 from pydantic import BaseModel, TypeAdapter250251 class Recipe(BaseModel):252 recipe_name: str253 ingredients: list[str]254255 client = genai.Client()256257 # A list of dictionaries, where each is a GenerateContentRequest258 inline_requests = [259 {260 'contents': [{261 'parts': [{'text': 'List a few popular cookie recipes, and include the amounts of ingredients.'}],262 'role': 'user'263 }],264 'config': {265 'response_mime_type': 'application/json',266 'response_schema': list[Recipe]267 }268 },269 {270 'contents': [{271 'parts': [{'text': 'List a few popular gluten free cookie recipes, and include the amounts of ingredients.'}],272 'role': 'user'273 }],274 'config': {275 'response_mime_type': 'application/json',276 'response_schema': list[Recipe]277 }278 }279 ]280281 inline_batch_job = client.batches.create(282 model="models/gemini-2.5-flash",283 src=inline_requests,284 config={285 'display_name': "structured-output-job-1"286 },287 )288289 # wait for the job to finish290 job_name = inline_batch_job.name291 print(f"Polling status for job: {job_name}")292293 while True:294 batch_job_inline = client.batches.get(name=job_name)295 if batch_job_inline.state.name in ('JOB_STATE_SUCCEEDED', 'JOB_STATE_FAILED', 'JOB_STATE_CANCELLED', 'JOB_STATE_EXPIRED'):296 break297 print(f"Job not finished. Current state: {batch_job_inline.state.name}. Waiting 30 seconds...")298 time.sleep(30)299300 print(f"Job finished with state: {batch_job_inline.state.name}")301302 # print the response303 for i, inline_response in enumerate(batch_job_inline.dest.inlined_responses):304 print(f"\n--- Response {i+1} ---")305306 # Check for a successful response307 if inline_response.response:308 # The .text property is a shortcut to the generated text.309 print(inline_response.response.text)310311Monitoring job status312---------------------313314Use the operation name obtained when creating the batch job to poll its status.315The state field of the batch job will indicate its current status. A batch job316can be in one of the following states:317318- `JOB_STATE_PENDING`: The job has been created and is waiting to be processed by the service.319- `JOB_STATE_RUNNING`: The job is in progress.320- `JOB_STATE_SUCCEEDED`: The job completed successfully. You can now retrieve the results.321- `JOB_STATE_FAILED`: The job failed. Check the error details for more information.322- `JOB_STATE_CANCELLED`: The job was cancelled by the user.323- `JOB_STATE_EXPIRED`: The job has expired because it was running or pending for more than 48 hours. The job will not have any results to retrieve. You can try submitting the job again or splitting up the requests into smaller batches.324325You can poll the job status periodically to check for completion. 326327### Python328329330 # Use the name of the job you want to check331 # e.g., inline_batch_job.name from the previous step332 job_name = "YOUR_BATCH_JOB_NAME" # (e.g. 'batches/your-batch-id')333 batch_job = client.batches.get(name=job_name)334335 completed_states = set([336 'JOB_STATE_SUCCEEDED',337 'JOB_STATE_FAILED',338 'JOB_STATE_CANCELLED',339 'JOB_STATE_EXPIRED',340 ])341342 print(f"Polling status for job: {job_name}")343 batch_job = client.batches.get(name=job_name) # Initial get344 while batch_job.state.name not in completed_states:345 print(f"Current state: {batch_job.state.name}")346 time.sleep(30) # Wait for 30 seconds before polling again347 batch_job = client.batches.get(name=job_name)348349 print(f"Job finished with state: {batch_job.state.name}")350 if batch_job.state.name == 'JOB_STATE_FAILED':351 print(f"Error: {batch_job.error}")352353Retrieving results354------------------355356Once the job status indicates your batch job has succeeded, the results are357available in the `response` field. 358359### Python360361 import json362363 # Use the name of the job you want to check364 # e.g., inline_batch_job.name from the previous step365 job_name = "YOUR_BATCH_JOB_NAME"366 batch_job = client.batches.get(name=job_name)367368 if batch_job.state.name == 'JOB_STATE_SUCCEEDED':369370 # If batch job was created with a file371 if batch_job.dest and batch_job.dest.file_name:372 # Results are in a file373 result_file_name = batch_job.dest.file_name374 print(f"Results are in file: {result_file_name}")375376 print("Downloading result file content...")377 file_content = client.files.download(file=result_file_name)378 # Process file_content (bytes) as needed379 print(file_content.decode('utf-8'))380381 # If batch job was created with inline request382 elif batch_job.dest and batch_job.dest.inlined_responses:383 # Results are inline384 print("Results are inline:")385 for i, inline_response in enumerate(batch_job.dest.inlined_responses):386 print(f"Response {i+1}:")387 if inline_response.response:388 # Accessing response, structure may vary.389 try:390 print(inline_response.response.text)391 except AttributeError:392 print(inline_response.response) # Fallback393 elif inline_response.error:394 print(f"Error: {inline_response.error}")395 else:396 print("No results found (neither file nor inline).")397 else:398 print(f"Job did not succeed. Final state: {batch_job.state.name}")399 if batch_job.error:400 print(f"Error: {batch_job.error}")401402### REST403404 BATCH_NAME="batches/123456" # Your batch job name405406 curl https://generativelanguage.googleapis.com/v1beta/$BATCH_NAME \407 -H "x-goog-api-key: $GEMINI_API_KEY" \408 -H "Content-Type:application/json" 2> /dev/null > batch_status.json409410 if jq -r '.done' batch_status.json | grep -q "false"; then411 echo "Batch has not finished processing"412 fi413414 batch_state=$(jq -r '.metadata.state' batch_status.json)415 if [[ $batch_state = "JOB_STATE_SUCCEEDED" ]]; then416 if [[ $(jq '.response | has("inlinedResponses")' batch_status.json) = "true" ]]; then417 jq -r '.response.inlinedResponses' batch_status.json418 exit419 fi420 responses_file_name=$(jq -r '.response.responsesFile' batch_status.json)421 curl https://generativelanguage.googleapis.com/download/v1beta/$responses_file_name:download?alt=media \422 -H "x-goog-api-key: $GEMINI_API_KEY" 2> /dev/null423 elif [[ $batch_state = "JOB_STATE_FAILED" ]]; then424 jq '.error' batch_status.json425 elif [[ $batch_state == "JOB_STATE_CANCELLED" ]]; then426 echo "Batch was cancelled by the user"427 elif [[ $batch_state == "JOB_STATE_EXPIRED" ]]; then428 echo "Batch expired after 48 hours"429 fi430431Cancelling a batch job432----------------------433434You can cancel an ongoing batch job using its name. When a job is435canceled, it stops processing new requests. 436437### Python438439 # Cancel a batch job440 client.batches.cancel(name=batch_job_to_cancel.name)441442### REST443444 BATCH_NAME="batches/123456" # Your batch job name445446 # Cancel the batch447 curl https://generativelanguage.googleapis.com/v1beta/$BATCH_NAME:cancel \448 -H "x-goog-api-key: $GEMINI_API_KEY" \449450 # Confirm that the status of the batch after cancellation is JOB_STATE_CANCELLED451 curl https://generativelanguage.googleapis.com/v1beta/$BATCH_NAME \452 -H "x-goog-api-key: $GEMINI_API_KEY" \453 -H "Content-Type:application/json" 2> /dev/null | jq -r '.metadata.state'454455Deleting a batch job456--------------------457458You can delete an existing batch job using its name. When a job is459deleted, it stops processing new requests and is removed from the list of460batch jobs. 461462### Python463464 # Delete a batch job465 client.batches.delete(name=batch_job_to_delete.name)466467### REST468469 BATCH_NAME="batches/123456" # Your batch job name470471 # Delete the batch job472 curl https://generativelanguage.googleapis.com/v1beta/$BATCH_NAME:delete \473 -H "x-goog-api-key: $GEMINI_API_KEY"474475Technical details476-----------------477478- **Supported models:** Batch Mode supports a range of Gemini models. Refer to the [Models page](/gemini-api/docs/models) for each model's support of Batch Mode. The supported modalities for Batch Mode are the same as what's supported on the interactive (or non-batch mode) API.479- **Pricing:** Batch Mode usage is priced at 50% of the standard interactive API cost for the equivalent model. See the [pricing page](/gemini-api/docs/pricing) for details. Refer to the [rate limits page](/gemini-api/docs/rate-limits#batch-mode) for details on rate limits for this feature.480- **Service Level Objective (SLO):** Batch jobs are designed to complete within a 24-hour turnaround time. Many jobs may complete much faster depending on their size and current system load.481- **Caching:** [Context caching](/gemini-api/docs/caching) is enabled for batch requests. If a request in your batch results in a cache hit, the cached tokens are priced the same as for non-batch mode traffic.482483Best practices484--------------485486- **Use input files for large requests:** For a large number of requests, always use the file input method for better manageability and to avoid hitting request size limits for the [`BatchGenerateContent`](/api/batch-mode#google.ai.generativelanguage.v1beta.BatchService.BatchGenerateContent) call itself. Note that there's a the 2GB file size limit per input file.487- **Error handling:** Check the `batchStats` for `failedRequestCount` after a job completes. If using file output, parse each line to check if it's a `GenerateContentResponse` or a status object indicating an error for that specific request. See the [troubleshooting488 guide](/gemini-api/docs/troubleshooting#error-codes) for a complete set of error codes.489- **Submit jobs once:** The creation of a batch job is not idempotent. If you send the same creation request twice, two separate batch jobs will be created.490- **Break up very large batches:** While the target turnaround time is 24 hours, actual processing time can vary based on system load and job size. For large jobs, consider breaking them into smaller batches if intermediate results are needed sooner.491492What's next493-----------494495Check out the [batch mode notebook](https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Batch_mode.ipynb)496for more examples.