gemini-live-api-dev

Name: gemini-live-api-dev
Author: google-gemini

5.9kinstalls389forks

How to install gemini-live-api-dev?

google-gemini/gemini-skills/gemini-live-api-dev

$npx -y skills add google-gemini/gemini-skills --skill gemini-live-api-dev

Installs into the current project.

Summary

Use this skill when building real-time, bidirectional streaming applications with the Gemini Live API. Covers WebSocket-based audio/video/text streaming, voice activity detection (VAD), native audio features, function calling, session management, ephemeral tokens for client-side

Files · 1

View on GitHub

SKILL.md

1	# Gemini Live API Development Skill
2
3	## Overview
4
5	The Live API enables low-latency, real-time voice and video interactions with Gemini over WebSockets. It processes continuous streams of audio, video, or text to deliver immediate, human-like spoken responses.
6
7	Key capabilities:
8	- Bidirectional audio streaming — real-time mic-to-speaker conversations
9	- Video streaming — send camera/screen frames alongside audio
10	- Text input/output — send and receive text within a live session
11	- Audio transcriptions — get text transcripts of both input and output audio
12	- Voice Activity Detection (VAD) — automatic interruption handling
13	- Native audio — thinking (with configurable `thinkingLevel`)
14	- Function calling — synchronous tool use
15	- Google Search grounding — ground responses in real-time search results
16	- Session management — context compression, session resumption, GoAway signals
17	- Ephemeral tokens — secure client-side authentication
18
19	> [!NOTE]
20	> The Live API currently only supports WebSockets. For WebRTC support or simplified integration, use a [partner integration](#partner-integrations).
21
22	## Models
23
24	- `gemini-3.1-flash-live-preview` — Optimized for low-latency, real-time dialogue. Native audio output, thinking (via `thinkingLevel`). 128k context window. This is the recommended model for all Live API use cases.
25	- `gemini-3.5-live-translate-preview` — Real-time streaming translation model.
26
27	> [!WARNING]
28	> The following Live API models are deprecated and will be shut down. Migrate to `gemini-3.1-flash-live-preview`.
29	> - `gemini-2.5-flash-native-audio-preview-12-2025` — Migrate to `gemini-3.1-flash-live-preview`.
30	> - `gemini-live-2.5-flash-preview` — Released June 17, 2025. Shutdown: December 9, 2025.
31	> - `gemini-2.0-flash-live-001` — Released April 9, 2025. Shutdown: December 9, 2025.
32
33	## SDKs
34
35	- Python: `google-genai` — `pip install google-genai`
36	- JavaScript/TypeScript: `@google/genai` — `npm install @google/genai`
37
38	> [!WARNING]
39	> Legacy SDKs `google-generativeai` (Python) and `@google/generative-ai` (JS) are deprecated. Use the new SDKs above.
40
41	## Partner Integrations
42
43	To streamline real-time audio/video app development, use a third-party integration supporting the Gemini Live API over WebRTC or WebSockets:
44
45	- [LiveKit](https://docs.livekit.io/agents/models/realtime/plugins/gemini/) — Use the Gemini Live API with LiveKit Agents.
46	- [Pipecat by Daily](https://docs.pipecat.ai/guides/features/gemini-live) — Create a real-time AI chatbot using Gemini Live and Pipecat.
47	- [Fishjam by Software Mansion](https://docs.fishjam.io/tutorials/gemini-live-integration) — Create live video and audio streaming applications with Fishjam.
48	- [Vision Agents by Stream](https://visionagents.ai/integrations/gemini) — Build real-time voice and video AI applications with Vision Agents.
49	- [Voximplant](https://voximplant.com/products/gemini-client) — Connect inbound and outbound calls to Live API with Voximplant.
50	- [Firebase AI SDK](https://firebase.google.com/docs/ai-logic/live-api?api=dev) — Get started with the Gemini Live API using Firebase AI Logic.
51
52	## Audio Formats
53
54	- Input: Raw PCM, little-endian, 16-bit, mono. 16kHz native (will resample others). MIME type: `audio/pcm;rate=16000`
55	- Output: Raw PCM, little-endian, 16-bit, mono. 24kHz sample rate.
56
57	> [!IMPORTANT]
58	> Use `send_realtime_input` / `sendRealtimeInput` for all real-time user input (audio, video, and text). `send_client_content` / `sendClientContent` is only supported for seeding initial context history (requires setting `initial_history_in_client_content` in `history_config`). Do not use it to send new user messages during the conversation.
59
60	> [!WARNING]
61	> Do not use `media` in `sendRealtimeInput`. Use the specific keys: `audio` for audio data, `video` for images/video frames, and `text` for text input.
62
63	---
64
65	## Quick Start
66
67	### Authentication
68
69	#### Python
70
71	```python
72	from google import genai
73
74	client = genai.Client(api_key="YOUR_API_KEY")
75	```
76
77	#### JavaScript
78
79	```js
80	import { GoogleGenAI } from '@google/genai';
81
82	const ai = new GoogleGenAI({ apiKey: 'YOUR_API_KEY' });
83	```
84
85	### Connecting to the Live API
86
87	#### Python
88	```python
89	from google.genai import types
90
91	config = types.LiveConnectConfig(
92	response_modalities=[types.Modality.AUDIO],
93	system_instruction=types.Content(
94	parts=[types.Part(text="You are a helpful assistant.")]
95	)
96	)
97
98	async with client.aio.live.connect(model="gemini-3.1-flash-live-preview", config=co

People who liked this also viewed

6 picks

Type

gemini-live-api-dev

5.9kinstalls389forks

View on GitHub

How to install gemini-live-api-dev?

google-gemini/gemini-skills/gemini-live-api-dev

$npx -y skills add google-gemini/gemini-skills --skill gemini-live-api-dev

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Gemini Live API Development Skill
2
3	## Overview
4
5	The Live API enables low-latency, real-time voice and video interactions with Gemini over WebSockets. It processes continuous streams of audio, video, or text to deliver immediate, human-like spoken responses.
6
7	Key capabilities:
8	- Bidirectional audio streaming — real-time mic-to-speaker conversations
9	- Video streaming — send camera/screen frames alongside audio
10	- Text input/output — send and receive text within a live session
11	- Audio transcriptions — get text transcripts of both input and output audio
12	- Voice Activity Detection (VAD) — automatic interruption handling
13	- Native audio — thinking (with configurable `thinkingLevel`)
14	- Function calling — synchronous tool use
15	- Google Search grounding — ground responses in real-time search results
16	- Session management — context compression, session resumption, GoAway signals
17	- Ephemeral tokens — secure client-side authentication
18
19	> [!NOTE]
20	> The Live API currently only supports WebSockets. For WebRTC support or simplified integration, use a [partner integration](#partner-integrations).
21
22	## Models
23
24	- `gemini-3.1-flash-live-preview` — Optimized for low-latency, real-time dialogue. Native audio output, thinking (via `thinkingLevel`). 128k context window. This is the recommended model for all Live API use cases.
25	- `gemini-3.5-live-translate-preview` — Real-time streaming translation model.
26
27	> [!WARNING]
28	> The following Live API models are deprecated and will be shut down. Migrate to `gemini-3.1-flash-live-preview`.
29	> - `gemini-2.5-flash-native-audio-preview-12-2025` — Migrate to `gemini-3.1-flash-live-preview`.
30	> - `gemini-live-2.5-flash-preview` — Released June 17, 2025. Shutdown: December 9, 2025.
31	> - `gemini-2.0-flash-live-001` — Released April 9, 2025. Shutdown: December 9, 2025.
32
33	## SDKs
34
35	- Python: `google-genai` — `pip install google-genai`
36	- JavaScript/TypeScript: `@google/genai` — `npm install @google/genai`
37
38	> [!WARNING]
39	> Legacy SDKs `google-generativeai` (Python) and `@google/generative-ai` (JS) are deprecated. Use the new SDKs above.
40
41	## Partner Integrations
42
43	To streamline real-time audio/video app development, use a third-party integration supporting the Gemini Live API over WebRTC or WebSockets:
44
45	- [LiveKit](https://docs.livekit.io/agents/models/realtime/plugins/gemini/) — Use the Gemini Live API with LiveKit Agents.
46	- [Pipecat by Daily](https://docs.pipecat.ai/guides/features/gemini-live) — Create a real-time AI chatbot using Gemini Live and Pipecat.
47	- [Fishjam by Software Mansion](https://docs.fishjam.io/tutorials/gemini-live-integration) — Create live video and audio streaming applications with Fishjam.
48	- [Vision Agents by Stream](https://visionagents.ai/integrations/gemini) — Build real-time voice and video AI applications with Vision Agents.
49	- [Voximplant](https://voximplant.com/products/gemini-client) — Connect inbound and outbound calls to Live API with Voximplant.
50	- [Firebase AI SDK](https://firebase.google.com/docs/ai-logic/live-api?api=dev) — Get started with the Gemini Live API using Firebase AI Logic.
51
52	## Audio Formats
53
54	- Input: Raw PCM, little-endian, 16-bit, mono. 16kHz native (will resample others). MIME type: `audio/pcm;rate=16000`
55	- Output: Raw PCM, little-endian, 16-bit, mono. 24kHz sample rate.
56
57	> [!IMPORTANT]
58	> Use `send_realtime_input` / `sendRealtimeInput` for all real-time user input (audio, video, and text). `send_client_content` / `sendClientContent` is only supported for seeding initial context history (requires setting `initial_history_in_client_content` in `history_config`). Do not use it to send new user messages during the conversation.
59
60	> [!WARNING]
61	> Do not use `media` in `sendRealtimeInput`. Use the specific keys: `audio` for audio data, `video` for images/video frames, and `text` for text input.
62
63	---
64
65	## Quick Start
66
67	### Authentication
68
69	#### Python
70
71	```python
72	from google import genai
73
74	client = genai.Client(api_key="YOUR_API_KEY")
75	```
76
77	#### JavaScript
78
79	```js
80	import { GoogleGenAI } from '@google/genai';
81
82	const ai = new GoogleGenAI({ apiKey: 'YOUR_API_KEY' });
83	```
84
85	### Connecting to the Live API
86
87	#### Python
88	```python
89	from google.genai import types
90
91	config = types.LiveConnectConfig(
92	response_modalities=[types.Modality.AUDIO],
93	system_instruction=types.Content(
94	parts=[types.Part(text="You are a helpful assistant.")]
95	)
96	)
97
98	async with client.aio.live.connect(model="gemini-3.1-flash-live-preview", config=co

People who liked this also viewed

6 picks

Type