# Voice Agent Management

## Creating a Voice Agent

Voice agents MUST be linked to a text agent. There is only one creation method:

1. Check existence first: **`get_voice_agent_by_text_agent(text_agent_id)`** — avoid duplicates.
2. **`quick_create_voice_agent(text_agent_id, template_id)`** — Creates a voice agent pre-configured from a template, linked to the text agent. Automatically resolves `tenant_id` from the text agent.
3. **`update_voice_agent(voice_agent_id, instruction=...)`** — Customize the instruction to match your use case.
4. **`publish_voice_agent(voice_agent_id)`** — Push draft config live.

There is NO other creation method. Do NOT use `create_agent` or any text-agent tool to create voice agents.

## Configuring and Optimizing Instructions

Voice agent instructions follow the same principles as text agent instructions. Use `update_voice_agent(voice_agent_id, instruction=...)` to set them.

**Structure instructions with markdown headers**: Role & Identity, Core Flow, Guardrails, Communication Style — just like text agents.

**Key differences from text agents**:
- Voice agents need concise responses — users are listening, not reading.
- Include explicit turn-taking guidance (e.g., "Ask one question at a time, wait for the answer").
- Add pronunciation hints for brand names or technical terms if the TTS mispronounces them.
- Specify escalation triggers (e.g., "If the user says 'speak to a human', transfer immediately").

**Publishing flow** (same as text agents):
1. `update_voice_agent(voice_agent_id, instruction=...)` — creates/updates draft config
2. `publish_voice_agent(voice_agent_id)` — makes it live
3. Verify by calling `get_voice_agent(voice_agent_id)` — confirm `published_agent_config` has the new instruction

## Customizing the Voice (Voice Clone)

To use a custom voice instead of the default:
1. **`upload_user_voice(audio_url=<url>)`** — Uploads the user's audio. Tell user: "Audio uploaded successfully." Do NOT mention storage details.
2. **`submit_voice_clone(tenant_id, voice_name, gcs_url)`** — Starts the cloning process. Tell user: "Voice cloning started, please wait."
3. **`get_minimax_voice(id)`** — Poll until status=`completed`. The tool auto-confirms when ready. Do NOT tell user about internal statuses.
4. When completed, share the `demo_audio` URL so the user can listen.
5. **CRITICAL — MUST NOT SKIP**: Take `agent_config_voice_id` from the `get_minimax_voice` response and call:
   **`update_voice_agent(voice_agent_id, agent_config={"voice_id": "<agent_config_voice_id>"})`**
   Without this step the cloned voice will NOT take effect on the agent.
6. **`publish_voice_agent(voice_agent_id)`** — Push the new voice live.

### User-facing communication

| Internal step | What to tell user |
|---------------|-------------------|
| upload_user_voice | "Audio uploaded successfully" |
| submit_voice_clone | "Voice cloning started, this may take a moment" |
| get_minimax_voice (polling) | "Still processing..." or "Almost ready..." |
| status=completed | "Your voice clone is ready! Here's a preview: {demo_audio}" |
| update_voice_agent + publish | "Your custom voice has been applied to the agent" |

**NEVER expose to user**: GCS URLs, internal status names (pending/processing/tentative), voice_id, clone_model, minimax IDs.

### Internal notes (for tool logic, not user-facing)

- Clone lifecycle: `pending` → `processing` → `tentative` → `completed` (or `failed`)
- `tentative` means clone finished but awaiting confirmation — `get_minimax_voice` auto-confirms it to `completed`
- `failed` — retry with `submit_voice_clone(id=<existing_id>, ...)`
- Always use `upload_user_voice` first — external URLs (e.g. from Wati) require auth and cannot be passed to `submit_voice_clone` directly

To browse existing voices: `search_voice_library(keyword=...)` or `search_collected_voices(tenant_id=...)`.

**NOTE**: If `submit_voice_clone` returns 503, Minimax clone is not configured for this deployment.

## ASR (Speech-to-Text) for Voice Messages

When a user sends a voice message (audio) instead of text, the webhook automatically:
1. Downloads the audio file (with Wati token authentication)
2. Uploads to ASR for transcription
3. Passes the transcribed text + original audio URL to the agent

The agent receives voice messages in this format:
```
[Voice message | audio_url: <url> | format: audio.opus]
<transcribed text>
```

If the user's intent involves using their audio for voice cloning, use the `audio_url` with `upload_user_voice`. Otherwise, just process the transcribed text normally.

## Starting Calls

### Inbound Calls (WhatsApp)
- **`new_voice_call(call_id, sdp)`** — Accept an incoming WhatsApp call. Requires tenant validation and usage gating. Call is processed asynchronously.
- Uses the **published** agent config.

### Web Calls (Browser)
- **`web_new_voice_call(agent_id, call_id, sdp)`** — Start a browser-based call via WebRTC.
- Use **`get_webrtc_config()`** first to get ICE servers (STUN/TURN) for frontend setup.
- Uses the **published** agent config.

### Test Calls (Development)
- **`test_new_voice_call(agent_id, call_id, sdp)`** — Same as web call but skips tenant validation.
- Uses the **draft** agent config — best for iterating on config before publishing.

### Outbound Calls (WhatsApp)
- **`initiate_outbound_call(waid)`** — Call a WhatsApp user by their WAID.
- Uses the **draft** agent config (test mode). Requires a valid WhatsApp user ID.
- Optional: set `voice_language`, `accent`, `agent_id`, `channel_phone_number`.

### Ending Calls
- **`terminate_voice_call(call_id)`** — End a production call.
- **`test_terminate_voice_call(call_id)`** — End a test call (skips Wati API).

## Critical Gotchas

### Only Use quick_create_voice_agent
`quick_create_voice_agent(text_agent_id, template_id)` is the only creation method. It handles tenant setup internally. Always call `get_voice_agent_by_text_agent(text_agent_id)` before creation to avoid duplicates.

### voice_id Must Reference a Completed Clone
`agent_config.voice_id` must reference a voice with `status=completed`. The `get_minimax_voice` tool auto-confirms tentative clones, so just poll until completed. Setting an incomplete or invalid voice_id will cause failures.

### Never Expose Internal Details to Users
Do NOT show users: voice_id, GCS URLs, clone_model, minimax config, internal status names. Only share: upload success, cloning progress, preview audio link, and final confirmation.

### Draft Config Required Before Publishing
`publish_voice_agent` will fail if `agent_config` is empty. Always call `update_voice_agent` to set the draft config before publishing.

### Test Calls Use Draft Config, Production Calls Use Published Config
`test_new_voice_call` and `initiate_outbound_call` use draft config — changes are reflected immediately. `new_voice_call` and `web_new_voice_call` use published config — must call `publish_voice_agent` first.

### Text Agent Linkage
Use `get_voice_agent_by_text_agent(text_agent_id)` to find the voice counterpart of a text agent. This is the primary lookup when working across text and voice systems.

## agent_config Fields Reference

```json
{
  "tone": "professional, helpful, and concise",
  "speed": 1,
  "voice": "VoiceDisplayName",
  "persona": "Professional Voice Assistant",
  "language": "en",
  "services": ["lead_qualification", "meeting_booking", "q_and_a"],
  "voice_id": "minimax-voice-uuid",
  "expertise": ["customer_service", "sales", "scheduling"],
  "default_accent": "us",
  "model_provider": "minimax",
  "max_call_duration": 600,
  "prompt_config": {
    "greeting_template": "Hello! How can I help you today?",
    "realtime_template": "## Role & Objective\n...",
    "system_instructions": "You are a helpful assistant...",
    "auto_language_switching": true
  },
  "outbound_prompt_config": {
    "greeting_template": "Hello! This is {{.CompanyName}}...",
    "realtime_template": "## Role & Objective\n...",
    "system_instructions": "..."
  },
  "minimax_config": {
    "model": "speech-02-turbo",
    "speed": 1,
    "format": "pcm",
    "volume": 1,
    "bitrate": 128000,
    "channels": 1,
    "voice_id": "astra-voice-xxx",
    "sample_rate": 32000
  },
  "rag_config": {
    "enabled": true,
    "token": "...",
    "base_url": "...",
    "timeout": 30,
    "max_retries": 3
  },
  "silence_config": {},
  "integrated_actions": [],
  "outbound_integrated_actions": []
}
```

### Top-level fields

| Field | Type | Description |
|-------|------|-------------|
| `voice` | string | Display name of the selected voice |
| `voice_id` | string | Minimax voice record's `id` (NOT its `voice_id` field). Must reference a clone with `status=completed` |
| `speed` | number | Speech speed multiplier (default 1) |
| `tone` | string | Tone descriptor (e.g. "professional, helpful") |
| `persona` | string | Agent persona label |
| `language` | string | Primary language code. Default: `en`. See supported values below. Do NOT use "multilingual" — for multi-language support, set `auto_language_switching: true` in `prompt_config` instead |
| `default_accent` | string | Accent variant (e.g. `us`, `uk`, `in`) |
| `model_provider` | string | LLM backend: `openai`, `gemini`, or `minimax` |
| `services` | []string | Enabled service types |
| `expertise` | []string | Domain expertise tags |
| `max_call_duration` | int | Hard limit in seconds; call auto-terminates after this |

### prompt_config / outbound_prompt_config

Separate configurations for inbound vs outbound calls. Both share the same structure:

| Field | Description |
|-------|-------------|
| `realtime_template` | System prompt / instruction for voice calls (equivalent to text agent's instruction) |
| `greeting_template` | Opening message when the call starts. Supports `{{.CompanyName}}` etc. |
| `system_instructions` | Additional system-level instructions appended to the prompt |
| `auto_language_switching` | If true, agent auto-detects and switches to the caller's language. Set this when user wants "multilingual" support |

### Language Configuration

**`language` field** must be one of the supported language codes (default: `en`):

`en`, `zh`, `yue`, `es`, `fr`, `de`, `ja`, `ko`, `pt`, `it`, `ru`, `ar`, `hi`, `th`, `vi`, `id`, `ms`, `nl`, `pl`, `tr`, `sv`, `no`, `da`, `fi`, `el`, `he`, `uk`, `cs`, `ro`, `hu`, `bg`, `fa`, `sk`, `hr`, `tl`, `sl`, `ca`, `nn`, `ta`, `af`

**Rules:**
- Default to `en` unless the user specifies a different primary language
- **Never** set `language` to `"multilingual"` — it is not a valid value
- When the user says "multilingual" or "multi-language", set `language` to their primary language (or `en` by default) AND set `prompt_config.auto_language_switching: true`
- Example: user says "I want multilingual support, mainly Chinese" → `language: "zh"`, `auto_language_switching: true`
- Example: user says "support multiple languages" → `language: "en"`, `auto_language_switching: true`

### minimax_config

Low-level TTS engine parameters (only relevant when `model_provider=minimax`):

| Field | Description |
|-------|-------------|
| `model` | TTS model (e.g. `speech-02-turbo`) |
| `voice_id` | Internal Minimax voice ID (auto-derived from top-level `voice_id`) |
| `speed`, `volume` | Playback tuning |
| `format`, `bitrate`, `channels`, `sample_rate` | Audio encoding parameters |

### rag_config

Controls knowledge-base retrieval for the voice agent:

| Field | Description |
|-------|-------------|
| `enabled` | Whether RAG is active |
| `token`, `base_url` | Credentials and endpoint for the knowledge service |
| `timeout`, `max_retries` | Request tuning |

### Other fields

- `silence_config` — Silence detection thresholds (affects when the agent speaks during pauses)
- `integrated_actions` / `outbound_integrated_actions` — Action definitions triggered during inbound/outbound calls

### Updating agent_config

`update_voice_agent` performs a safe GET → merge → PUT for all `agent_config` changes. Examples:

- **Bind a cloned voice**: `update_voice_agent(voice_agent_id, agent_config={voice_id: "uuid", voice: "MyVoice"})`
- **Change language**: `update_voice_agent(voice_agent_id, agent_config={language: "zh", default_accent: "cn"})`
- **Update instruction**: `update_voice_agent(voice_agent_id, instruction="new prompt text")`
- **Update greeting**: `update_voice_agent(voice_agent_id, greeting="Hello! ...")`
- **Combo**: `update_voice_agent(voice_agent_id, instruction="...", agent_config={language: "en", tone: "friendly"})`

The `instruction` and `greeting` shortcut params map to `prompt_config.realtime_template` and `prompt_config.greeting_template` respectively. The `agent_config` param merges at the top level. All three can be combined in a single call.
