# API Reference

RoohAI exposes a REST API and two WebSocket endpoints via FastAPI.

**Base URL:** `http://localhost:8000`

---

## REST Endpoints

### Health Check

```
GET /api/health
```

Returns server status and active model names.

**Response:**

```json
{
  "status": "ok",
  "models": {
    "stt": "deepgram",
    "tts": "cartesia",
    "llm": "bedrock-claude"
  }
}
```

---

### List Models

```
GET /api/models
```

Returns all registered models and which are currently active.

**Response:**

```json
{
  "stt": ["whisper", "wav2vec2", "deepgram"],
  "tts": ["speecht5", "bark", "piper", "cartesia"],
  "llm": ["bedrock", "local"],
  "active_stt": "deepgram",
  "active_tts": "cartesia",
  "active_llm": "bedrock-claude"
}
```

---

### Swap Model

```
POST /api/models/swap
Content-Type: application/json
```

Hot-swap a model at runtime without restarting the server.

**Request:**

```json
{
  "model_type": "tts",
  "model_name": "piper"
}
```

**Response:**

```json
{
  "status": "ok",
  "model_type": "tts",
  "model_name": "piper",
  "active_models": {
    "stt": "deepgram",
    "tts": "piper",
    "llm": "bedrock-claude"
  }
}
```

---

### Model Catalog

```
GET /api/catalog
```

Returns metadata for all registered models, including display names, descriptions, dependencies, and configuration fields.

**Response:**

```json
{
  "stt": [
    {
      "name": "whisper",
      "display_name": "OpenAI Whisper",
      "description": "Open-source speech recognition model...",
      "kind": "local",
      "pip_dependencies": ["transformers>=4.46.0,<5.0.0", "torch", "sentencepiece"],
      "estimated_download_mb": 150,
      "config_fields": [...]
    }
  ],
  "tts": [...],
  "llm": [...],
  "vad": [...]
}
```

---

### List Agents

```
GET /api/agents
```

**Response:**

```json
{
  "agents": [
    {
      "name": "my-agent",
      "status": "active",
      "pipeline": {"stt": "deepgram", "llm": "bedrock-claude", "tts": "cartesia", "vad": "silero"},
      "models": {...}
    }
  ]
}
```

---

### Create Agent

```
POST /api/agents
Content-Type: application/json
```

**Request:**

```json
{
  "name": "my-agent",
  "system_prompt": "You are a helpful voice assistant. Be concise.",
  "llm_streaming": true,
  "pipeline": {
    "stt": "deepgram",
    "llm": "bedrock-claude",
    "tts": "piper",
    "vad": "silero"
  },
  "models": {
    "deepgram": {"class": "deepgram", "type": "stt", "model": "nova-3"},
    "bedrock-claude": {"class": "bedrock-claude", "type": "llm"},
    "piper": {"class": "piper", "type": "tts"},
    "silero": {"class": "silero", "type": "vad"}
  },
  "secrets": {
    "deepgram": {"api_key": "dg-xxxx"}
  }
}
```

The `system_prompt` field is **required**. It defines the LLM's persona and behavior for the agent.

The `llm_streaming` field is optional (default: `true`). When `true`, the LLM streams tokens as they arrive and TTS begins on the first completed sentence. When `false`, the server waits for the full LLM response before starting TTS, sending a single `llm_response` message.

**Response:**

```json
{
  "status": "ok",
  "agent": "my-agent"
}
```

---

### Delete Agent

```
DELETE /api/agents/{name}
```

**Response:**

```json
{
  "status": "ok"
}
```

---

### Activate Agent

```
POST /api/agents/{name}/activate
```

Loads the agent's models into the pipeline. Deactivates the previously active agent.

**Response:**

```json
{
  "status": "ok",
  "agent": "my-agent"
}
```

---

### WebRTC Offer

```
POST /api/webrtc/offer
Content-Type: application/json
```

Accept an SDP offer from the browser's RTCPeerConnection and return an SDP answer. This is the signaling endpoint that establishes the WebRTC session.

**Request:**

```json
{
  "sdp": "v=0\r\no=- 46117317...",
  "type": "offer"
}
```

**Response:**

```json
{
  "session_id": "a1b2c3d4-...",
  "sdp": "v=0\r\no=- 38912455...",
  "type": "answer"
}
```

The `session_id` uniquely identifies this WebRTC connection and is passed through the pipeline to the LLM model and hooks.

The server creates an `AudioProcessorTrack` that:
- Receives the client's audio track (mic input via Opus)
- Runs it through VAD → STT → LLM → TTS (with `session_id` flowing to the LLM)
- Sends TTS audio back as a remote audio track
- Exchanges status messages via an RTCDataChannel named `"status"`

---

### WebRTC ICE

```
POST /api/webrtc/ice
```

ICE candidate exchange endpoint for trickle ICE. For localhost connections, ICE candidates are gathered automatically during the offer/answer exchange. This endpoint exists for future use with STUN/TURN servers.

**Response:**

```json
{
  "status": "ok"
}
```

---

## WebSocket Endpoints

### Voice Streaming

```
ws://localhost:8000/ws/voice-stream
```

The primary endpoint for real-time voice conversation. Accepts audio chunks and returns transcriptions, LLM responses, and TTS audio.

**Connection flow:**

1. Client connects
2. Server sends `{"type": "session", "session_id": "uuid"}` — unique connection identifier
3. Server sends `{"type": "status", "state": "listening"}`
4. Client streams `audio_chunk` messages
5. Server responds with transcription, LLM, and audio messages

The `session_id` is a UUID generated per WebSocket connection. It is passed through the entire pipeline to the LLM model and hooks, enabling per-session state (conversation history, agent memory, etc.).

**Client -> Server messages:**

```json
// Audio data (PCM16, 16kHz, base64-encoded)
{"type": "audio_chunk", "audio": "base64-encoded-pcm16"}

// Notify server that all queued audio finished playing
{"type": "playback_done"}

// Pause audio capture (connection stays alive, server ignores audio chunks)
{"type": "pause_audio"}

// Resume audio capture
{"type": "resume_audio"}

// Keep-alive
{"type": "ping"}
```

**Server -> Client messages:**

```json
// Session identifier (sent immediately on connect)
{"type": "session", "session_id": "a1b2c3d4-e5f6-..."}

// Pipeline state changes
{"type": "status", "state": "listening"}
{"type": "status", "state": "listening", "vad": "speech_start"}
{"type": "status", "state": "listening", "vad": "speech_end"}
{"type": "status", "state": "transcribing"}
{"type": "status", "state": "thinking"}
{"type": "status", "state": "speaking"}
{"type": "status", "state": "paused"}

// Live transcription (replaced on each update)
{"type": "interim_transcription", "text": "Tell me about..."}

// Final transcription (permanent)
{"type": "transcription", "text": "Tell me about books."}

// AI response — streaming (token-by-token as they arrive from the LLM)
{"type": "llm_response_chunk", "text": "Books"}
{"type": "llm_response_chunk", "text": " are"}
{"type": "llm_response_chunk", "text": " collections"}

// AI response — stream complete (contains full accumulated text)
{"type": "llm_response_done", "text": "Books are collections of..."}

// AI response — non-streaming fallback (full response at once, for LLMs without streaming)
{"type": "llm_response", "text": "Books are collections of..."}

// TTS audio (one or more chunks — starts arriving before LLM stream is complete)
{"type": "audio", "audio": "base64-encoded-wav"}

// Barge-in: stop playback immediately
{"type": "interrupt"}

// Pipeline timing metrics (sent after each completed turn)
{"type": "metrics", "stt_ms": 230.5, "llm_ms": 1200.3, "tts_ms": 450.1, "total_ms": 1880.9}

// Error
{"type": "error", "message": "Something went wrong"}

// Keep-alive response
{"type": "pong"}
```

**Barge-in protocol:**

1. While AI audio is playing, client continues sending `audio_chunk`
2. Server VAD detects user speech -> sends `{"type": "interrupt"}`
3. Client stops playback, clears queue
4. Server cancels in-progress TTS, starts processing new utterance
5. Client sends `{"type": "playback_done"}` when playback stops

---

### Agent Preparation

```
ws://localhost:8000/ws/agent-prepare?name={agent-name}
```

Used by the wizard UI to prepare an agent with live progress updates (model downloads, validation).

**Server -> Client messages:**

```json
{"status": "progress", "step": "Installing deepgram-sdk", "percent": 25}
{"status": "progress", "step": "Validating API key", "percent": 50}
{"status": "progress", "step": "Loading model", "percent": 75}
{"status": "complete", "message": "Agent ready"}
{"status": "error", "message": "Invalid API key"}
```

---

### WebRTC Data Channel

When using WebRTC transport, the client creates an `RTCDataChannel` named `"status"` for bidirectional JSON messages. The message format is identical to the WebSocket message types above, with one addition:

**Server → Client (WebRTC only):**

```json
// All TTS frames have been enqueued — audio is playing via the remote track
{"type": "playback_complete"}
```

After receiving `playback_complete`, the client sends `playback_done` once the audio finishes playing (or immediately on barge-in interrupt).

**Client → Server (WebRTC only):**

```json
// Notify server that playback has finished
{"type": "playback_done"}

// Pause/resume audio capture (connection stays alive)
{"type": "pause_audio"}
{"type": "resume_audio"}

// Keep-alive
{"type": "ping"}
```

> **Note:** With WebRTC, audio flows as a native media track (Opus codec), not as base64 JSON messages. The data channel carries only status/control messages.

---

## Frontend

The web UI is served as static files:

| Path | Description |
|------|-------------|
| `GET /` | Main application (index.html) |
| `GET /static/*` | Static assets (JS, CSS, AudioWorklet) |
