# Google GenAI SDK — Audio API Reference

SDK version: 1.68.0 (as of 2026-03-26)

## Three Audio Capabilities

### 1. Text-to-Speech via `generate_content`

Same request/response pattern as image generation. Set `response_modalities: ["AUDIO"]`.

```python
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Read this aloud: Hello world",
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Kore")
            )
        ),
    ),
)
# Audio in response.candidates[0].content.parts[0].inline_data
# .data = raw audio bytes, .mime_type = e.g. "audio/pcm" or "audio/mp3"
```

**SpeechConfig options:**
- `voice_config.prebuilt_voice_config.voice_name` — e.g. "Peri", "Charon", "Kore", "Fenrir"
- `voice_config.replicated_voice_config` — custom voice clone from sample audio (`.voice_sample_audio` bytes + `.mime_type`)
- `multi_speaker_voice_config.speaker_voice_configs` — exactly 2 speakers for dialogue
- `language_code` — ISO 639-1 (e.g. "en", "es", "ja")

### 2. Live API (WebSocket streaming)

Bidirectional real-time audio. Async only.

```python
config = types.LiveConnectConfig(
    response_modalities=["AUDIO"],
    speech_config=types.SpeechConfig(...),
    input_audio_transcription=types.AudioTranscriptionConfig(),
    output_audio_transcription=types.AudioTranscriptionConfig(),
    realtime_input_config=types.RealtimeInputConfig(
        automatic_activity_detection=types.AutomaticActivityDetection(
            start_of_speech_sensitivity="HIGH",
            end_of_speech_sensitivity="HIGH",
            silence_duration_ms=500,
        ),
        activity_handling="START_OF_ACTIVITY_INTERRUPTS",
    ),
)

async with client.aio.live.connect(model="gemini-2.0-flash-live-001", config=config) as session:
    await session.send_realtime_input(
        audio=types.Blob(data=pcm_bytes, mime_type="audio/pcm;rate=16000")
    )
    async for msg in session.receive():
        if msg.server_content and msg.server_content.model_turn:
            for part in msg.server_content.model_turn.parts:
                if part.inline_data:
                    # part.inline_data.data = audio chunk bytes
                    pass
```

**Key types:**
- `LiveConnectConfig` — session config (modalities, speech, VAD, transcription)
- `RealtimeInputConfig` — VAD sensitivity, interruption handling, turn coverage
- `AudioTranscriptionConfig` — language_codes list (BCP-47), auto-detects if unset
- `VoiceActivityType` — ACTIVITY_START / ACTIVITY_END signals

### 3. Live Music API (experimental)

Generates music from text prompts via WebSocket.

```python
async with client.aio.live_music.connect(model="lyria-realtime-exp") as session:
    await session.set_music_generation_config(types.LiveMusicGenerationConfig(
        temperature=1.0,
        bpm=120,
        scale="C_MAJOR_A_MINOR",
        brightness=0.5,
        density=0.5,
        guidance=3.0,
        music_generation_mode="QUALITY",
    ))
    await session.set_weighted_prompts([
        types.WeightedPrompt(text="upbeat jazz piano", weight=1.0)
    ])
    await session.play()
    async for msg in session.receive():
        # msg contains AudioChunk with audio data
        pass
```

**MusicGenerationConfig controls:**
- `temperature` [0.0, 3.0], `top_k` [1, 1000], `seed`
- `guidance` [0.0, 6.0] — prompt adherence strength
- `bpm` [60, 200], `density` [0.0, 1.0], `brightness` [0.0, 1.0]
- `scale` — 12 musical keys (C_MAJOR_A_MINOR, D_MAJOR_B_MINOR, etc.)
- `mute_bass`, `mute_drums`, `only_bass_and_drums`
- `music_generation_mode` — QUALITY / DIVERSITY / VOCALIZATION

## Prebuilt Voice Names

Known from SDK source: Peri, Charon, Kore, Fenrir (there may be more — needs live API testing to enumerate).

## Audio Formats

- Input: `audio/pcm;rate=16000` (Live API), `audio/wav`, `audio/mp3`
- Output: varies by endpoint — typically `audio/pcm` from Live, format TBD from generate_content

## Key Source Files (SDK 1.68.0)

- `google/genai/live.py` — AsyncLive, AsyncSession
- `google/genai/live_music.py` — AsyncLiveMusic, AsyncMusicSession
- `google/genai/types.py` — All type definitions (SpeechConfig at ~line 5024, LiveConnectConfig at ~line 18786)
