Repository Public
1The Gemini Live API allows for real-time, bidirectional interaction with Gemini models, supporting audio, video, and text inputs and native audio outputs. This guide explains how to integrate with the API using the Google GenAI SDK on your server.2[Try the Live API in Google AI Studio](https://aistudio.google.com/live) [Clone the example app from GitHub](https://github.com/google-gemini/gemini-live-api-examples/tree/main/gemini-live-genai-python-sdk) [Use coding agent skills](https://ai.google.dev/gemini-api/docs/coding-agents)34## Overview56The Gemini Live API uses WebSockets for real-time communication. The `google-genai` SDK provides a high-level asynchronous interface for managing these connections.78Key concepts:910- **Session**: A persistent connection to the model.11- **Config**: Setting up modalities (audio/text), voice, and system instructions.12- **Real-time Input**: Sending audio and video frames as blobs.1314## Connecting to the Live API1516Start a Live API session with an API key:1718### Python1920 import asyncio21 from google import genai2223 client = genai.Client(api_key="YOUR_API_KEY")2425 model = "gemini-3.8-live"26 config = {"response_modalities": ["AUDIO"]}2728 async def main():29 async with client.aio.live.connect(model=model, config=config) as session:30 print("Session started")31 # Send content...3233 if __name__ == "__main__":34 asyncio.run(main())3536### JavaScript3738 import { GoogleGenAI, Modality } from '@google/genai';3940 const ai = new GoogleGenAI({ apiKey: "YOUR_API_KEY"});41 const model = 'gemini-3.8-live';42 const config = { responseModalities: [Modality.AUDIO] };4344 async function main() {4546 const session = await ai.live.connect({47 model: model,48 callbacks: {49 onopen: function () {50 console.debug('Opened');51 },52 onmessage: function (message) {53 console.debug(message);54 },55 onerror: function (e) {56 console.debug('Error:', e.message);57 },58 onclose: function (e) {59 console.debug('Close:', e.reason);60 },61 },62 config: config,63 });6465 console.debug("Session started");66 // Send content...6768 session.close();69 }7071 main();7273## Sending text7475Text can be sent using `send_realtime_input` (Python) or `sendRealtimeInput` (JavaScript).7677### Python7879 await session.send_realtime_input(text="Hello, how are you?")8081### JavaScript8283 session.sendRealtimeInput({84 text: 'Hello, how are you?'85 });8687## Sending audio8889Audio needs to be sent as raw PCM data (raw 16-bit PCM audio, 16kHz, little-endian).9091### Python9293 # Assuming 'chunk' is your raw PCM audio bytes94 await session.send_realtime_input(95 audio=types.Blob(96 data=chunk,97 mime_type="audio/pcm;rate=16000"98 )99 )100101### JavaScript102103 // Assuming 'chunk' is a Buffer of raw PCM audio104 session.sendRealtimeInput({105 audio: {106 data: chunk.toString('base64'),107 mimeType: 'audio/pcm;rate=16000'108 }109 });110111For an example of how to get the audio from the client device (e.g. the browser)112see the end-to-end example on [GitHub](https://github.com/google-gemini/gemini-live-api-examples/blob/main/gemini-live-genai-python-sdk/frontend/media-handler.js#L31-L70).113114## Sending video115116Video frames are sent as individual images (e.g., JPEG or PNG) at a specific frame rate (max 1 frame per second).117118### Python119120 # Assuming 'frame' is your JPEG-encoded image bytes121 await session.send_realtime_input(122 video=types.Blob(123 data=frame,124 mime_type="image/jpeg"125 )126 )127128### JavaScript129130 // Assuming 'frame' is a Buffer of JPEG-encoded image data131 session.sendRealtimeInput({132 video: {133 data: frame.toString('base64'),134 mimeType: 'image/jpeg'135 }136 });137138For an example of how to get the video from the client device (e.g. the browser)139see the end-to-end example on [GitHub](https://github.com/google-gemini/gemini-live-api-examples/blob/main/gemini-live-genai-python-sdk/frontend/media-handler.js#L84-L120).140141## Receiving audio142143The model's audio responses are received as chunks of data.144145### Python146147 async for response in session.receive():148 if response.server_content and response.server_content.model_turn:149 for part in response.server_content.model_turn.parts:150 if part.inline_data:151 audio_data = part.inline_data.data152 # Process or play the audio data153154### JavaScript155156 // Inside the onmessage callback157 const content = response.serverContent;158 if (content?.modelTurn?.parts) {159 for (const part of content.modelTurn.parts) {160 if (part.inlineData) {161 const audioData = part.inlineData.data;162 // Process or play audioData (base64 encoded string)163 }164 }165 }166167See the example app on GitHub to learn how to [receive the audio on your server](https://github.com/google-gemini/gemini-live-api-examples/blob/main/gemini-live-genai-python-sdk/gemini_live.py#L86-L98) and [play it in the browser](https://github.com/google-gemini/gemini-live-api-examples/blob/main/gemini-live-genai-python-sdk/frontend/media-handler.js#L145-L174).168169## Receiving text170171Transcriptions for both user input and model output are available in the server content.172173### Python174175 async for response in session.receive():176 content = response.server_content177 if content:178 if content.input_transcription:179 print(f"User: {content.input_transcription.text}")180 if content.output_transcription:181 print(f"Gemini: {content.output_transcription.text}")182183### JavaScript184185 // Inside the onmessage callback186 const content = response.serverContent;187 if (content?.inputTranscription) {188 console.log('User:', content.inputTranscription.text);189 }190 if (content?.outputTranscription) {191 console.log('Gemini:', content.outputTranscription.text);192 }193194## Handling tool calls195196The API supports tool calling (function calling). When the model requests a tool call, you must execute the function and send the response back.197198### Python199200 async for response in session.receive():201 if response.tool_call:202 function_responses = []203 for fc in response.tool_call.function_calls:204 # 1. Execute the function locally205 result = my_tool_function(**fc.args)206207 # 2. Prepare the response208 function_responses.append(types.FunctionResponse(209 name=fc.name,210 id=fc.id,211 response={"result": result}212 ))213214 # 3. Send the tool response back to the session215 await session.send_tool_response(function_responses=function_responses)216217### JavaScript218219 // Inside the onmessage callback220 if (response.toolCall) {221 const functionResponses = [];222 for (const fc of response.toolCall.functionCalls) {223 const result = myToolFunction(fc.args);224 functionResponses.push({225 name: fc.name,226 id: fc.id,227 response: { result }228 });229 }230 session.sendToolResponse({ functionResponses });231 }232233## What's next234235- Read the full Live API [Capabilities](https://ai.google.dev/gemini-api/docs/live-guide) guide for key capabilities and configurations; including Voice Activity Detection and native audio features.236- Read the [Tool use](https://ai.google.dev/gemini-api/docs/live-tools) guide to learn how to integrate Live API with tools and function calling.237- Read the [Session management](https://ai.google.dev/gemini-api/docs/live-session) guide for managing long running conversations.238- Read the [Ephemeral tokens](https://ai.google.dev/gemini-api/docs/ephemeral-tokens) guide for secure authentication in [client-to-server](https://ai.google.dev/gemini-api/docs/live-api#implementation-approach) applications.239- For more information about the underlying WebSockets API, see the [WebSockets API reference](https://ai.google.dev/api/live).