# genmedia audio — v0.3.0 Design Spec

## Summary

Add `genmedia audio` subcommand for text-to-speech generation using Google's Gemini TTS models. Fits the existing request/response architecture — no streaming required.

## API Surface (validated 2026-03-26)

- **Models:** `gemini-2.5-flash-preview-tts` (default), `gemini-2.5-pro-preview-tts`
- **SDK method:** `client.models.generate_content()` with `response_modalities=["AUDIO"]`
- **Output format:** 24kHz 16-bit mono PCM (`audio/L16;codec=pcm;rate=24000`), wrapped in WAV for usability
- **30 prebuilt voices:** achernar, achird, algenib, algieba, alnilam, aoede, autonoe, callirrhoe, charon, despina, enceladus, erinome, fenrir, gacrux, iapetus, kore, laomedeia, leda, orus, puck, pulcherrima, rasalgethi, sadachbia, sadaltager, schedar, sulafat, umbriel, vindemiatrix, zephyr, zubenelgenubi
- **Multi-speaker:** `MultiSpeakerVoiceConfig` supports exactly 2 speakers — deferred to v0.3.1

## CLI Design

```
genmedia audio "Read this aloud with feeling: Hello world"
genmedia audio --voice kore "The lighthouse keeper stood alone."
genmedia audio --voice charon --model gemini-2.5-pro-preview-tts "..."
genmedia audio --list-voices
echo "text from stdin" | genmedia audio --output speech.wav
genmedia audio "..." --output - | ffplay -autoexit -  # pipe to player
```

### Options

| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `PROMPT` | argument | required (or stdin) | Text to speak, or instructions + text |
| `--voice` / `-v` | choice | `kore` | Prebuilt voice name |
| `--model` / `-m` | string | `gemini-2.5-flash-preview-tts` | Model ID |
| `--language` / `-l` | string | none (auto) | ISO 639-1 language code |
| `--output` / `-o` | path | auto-named WAV | Output file (use `-` for stdout) |
| `--output-dir` / `-d` | path | `/tmp/genmedia` | Output directory |
| `--count` / `-n` | int | 1 | Number of variations |
| `--pretty` | flag | false | Human-friendly output |
| `--dry-run` | flag | false | Show request without calling API |
| `--list-models` | flag | false | List TTS models |
| `--list-voices` | flag | false | List available voice names |

### What's NOT in v0.3.0

- Multi-speaker dialogue (`--speakers`)
- Voice cloning from sample audio (`--voice-sample`)
- Live API / streaming mode
- Music generation

## Backend: TtsBackend

New backend class following existing pattern:

- `build_request()` — constructs GenerateContentConfig with SpeechConfig
- `validate()` — voice name validation, prompt required
- `generate()` — calls `generate_content`, extracts PCM audio from inline_data, wraps in WAV bytes

## Output

PCM from the API gets wrapped in a WAV header before writing. Output format is always `.wav` (no format flag — WAV is the universal interchange format for audio).

The `write_media_files()` function already handles this — just needs WAV added to MIME_TO_EXT mapping.

## Testing

- Unit tests: mock generate_content response with fake PCM bytes, verify WAV wrapping, voice validation, dry-run output, stdin prompt
- Integration test: generate with live API, verify WAV file > 0 bytes and parseable

## Open Questions

1. Should `--voice` accept partial matches (e.g., `fen` -> `fenrir`)? Probably not — explicit is better.
2. Should we validate voice names client-side or let the API reject? API rejection gives us the full list in the error, which is nice. But client-side validation gives faster feedback. Lean toward client-side with a hardcoded list that the API error can supplement.
3. The prompt for TTS is interesting — you can say "Read this aloud" or just provide raw text. Should we have a `--raw` flag that wraps text in "Read this aloud:" automatically? Probably YAGNI.

## Future: Audio/Video Composition (v0.4.0 candidate)

### The idea

A `genmedia compose` (or `genmedia mix`) command that merges a video file with an audio file into a single output. This enables workflows like:

```bash
genmedia audio --voice kore "The lake had been still for a thousand years." -o narration.wav
genmedia video "a slow pan across a calm lake at dawn" -o scene.mp4
genmedia compose scene.mp4 narration.wav -o final.mp4
```

### The duration mismatch problem

Veo generates fixed-duration video (4/6/8 seconds). TTS duration is unpredictable — it depends on text length, voice speed, and pacing. So audio and video will almost never be the same length.

**Approaches:**
1. **Audio-first workflow:** Generate narration, measure duration, pick closest Veo duration (round up), generate video. Best results but requires the user to run commands in the right order.
2. **Trim to shortest:** ffmpeg `--shortest` flag cuts output to whichever track ends first. Simple, but you lose content.
3. **Pad audio with silence:** If audio is shorter than video, pad with silence so the video plays fully. If audio is longer, this doesn't help.
4. **Document the constraint:** For v0.4.0, just make it composable and let users manage duration. The tool handles the merge; the user handles the creative decision about what fits.

**Recommendation:** Option 4 for v0.4.0. Use ffmpeg under the hood with `--shortest` as default behavior and a `--pad` flag to pad the shorter track with silence instead. Document the Veo duration constraint clearly.

### Dependencies

This would require `ffmpeg` as a system dependency (not pip-installable). The compose command should check for ffmpeg on PATH and give a clear error if missing. This is the only genmedia subcommand that would need an external binary.

### Not decided yet

- Whether this ships as part of genmedia or whether audio becomes its own standalone tool (in which case composition lives elsewhere)
- Whether we should support replacing Veo's built-in audio track with custom audio (strip existing + add new)
