Skip to main content

Gemini Live

GeminiLiveAdapter bridges a bidirectional audio stream to Google’s Gemini Live native-audio API. It speaks the same connect / send_audio / receive_events / close surface as OpenAIRealtimeAdapter, so you can swap engines without touching the call handler. Use it as an alternative to OpenAI Realtime when you want native-audio Gemini voices, longer context, or lower per-minute pricing.

Install

Native-audio Gemini Live is a v1alpha-only API. Install the optional extra:
Set GEMINI_API_KEY in your environment.

Constructor

Engine usage

Pass the GeminiLive engine marker to phone.agent(...). Patter builds the adapter server-side at call time and transcodes the carrier’s mulaw 8 kHz to and from Gemini’s PCM16 for you:
The TypeScript marker additionally exposes affectiveDialog, proactiveAudio, vad, thinking, thinkingBudget, and apiVersion. The Python adapter has no constructor argument for those, so the Python marker omits them rather than accepting a silent no-op. The Python adapter pins v1alpha, which is where native-audio models are served.

Adapter usage

phone.agent(engine=...) takes the GeminiLive marker, not an adapter instance — passing GeminiLiveAdapter(...) raises a TypeError. Drive the adapter directly only when you are wiring your own audio transport:
Tools work the same way as on OpenAI Realtime — pass tools=[Tool(...)] on phone.agent(...) and Patter forwards function calls to Gemini’s function_declarations shape.

Models

The defaults change as Google promotes native audio to GA. The GeminiLiveModel enum tracks the currently shipped identifiers.

Voices

Puck, Charon, Kore, Fenrir, Aoede — Gemini’s built-in PrebuiltVoiceConfig set. Pass any one as voice=.

When to use Gemini Live vs alternatives

Notes

  • Barge-in is implicit: Gemini Live runs server-side VAD and interrupts on user speech. cancel_response() / cancelResponse() is a no-op for compatibility.
  • The adapter itself resamples nothing — pass PCM16 mono at input_sample_rate Hz. On the engine=GeminiLive(...) path Patter inserts a carrier-codec shim that decodes the carrier’s mulaw 8 kHz up to input_sample_rate and mulaw-encodes output_sample_rate PCM back down.
  • Cost is not metered: the adapter does not surface Gemini’s token usage, so Gemini Live calls report $0 AI cost. Telephony cost is unaffected. Same gap in the TypeScript SDK.
  • Gemini Live fixes the prompt and tool list at connect time. Multi-agent handoff_to therefore logs a warning and keeps the original agent instead of swapping it.
  • google-genai (Python) and @google/genai (Node) are imported lazily, so default installs of getpatter do not pay the load cost.

What’s Next

Engines

All engines side by side.

OpenAI Realtime

The default engine.

Agents

System prompts, tools, first messages.

Tools

Function calling inside a realtime session.