<p align="center">
  <h1 align="center">RoohAI</h1>
  <p align="center">
    Open-source voice AI framework for building real-time voice agents.<br/>
    Swap STT, TTS, and LLM models with a single line of config.
  </p>
</p>

<p align="center">
  <a href="https://pypi.org/project/roohai/"><img src="https://img.shields.io/pypi/v/roohai?color=blue" alt="PyPI"></a>
  <a href="https://pypi.org/project/roohai/"><img src="https://img.shields.io/pypi/pyversions/roohai" alt="Python"></a>
  <a href="https://github.com/Fraser27/roohai-framework/blob/main/LICENSE"><img src="https://img.shields.io/github/license/Fraser27/roohai-framework" alt="License"></a>
</p>

<p align="center"><em>Teri awaaz sun kar, meri rooh ko sukoon milta hai.</em></p>
<p align="center"><a href="https://frasersequeira.medium.com/building-voice-agents-with-rooh-b4ece2abbb14"><h3>Why Rooh</h3></a></p>



## Features

- **Real-time voice** — WebRTC and WebSocket transports with sub-second latency
- **Swappable models** — Mix and match STT, TTS, LLM, and VAD providers via YAML config
- **Hot-swap at runtime** — Change models without restarting the server
- **Agent wizard UI** — Browser-based GUI to create agents, pick models, and start talking
- **LLM streaming** — Token-by-token responses with sentence-boundary TTS overlap
- **Barge-in** — Interrupt the AI mid-sentence by speaking
- **Hooks & extensibility** — Plug in custom LLM logic, tool use (Strands SDK), and observability
- **Built-in frontend** — Dark-themed vanilla HTML/CSS/JS UI, no build step required

## Quick Start

```bash
pip install roohai
```

Then start the server:

```bash
roohai
# Open http://localhost:8000
```

Use the web UI to create an agent, select your models, and start a conversation.

All STT, TTS, and LLM providers are included by default. For NVIDIA models, install the extra:

```bash
pip install "roohai[nvidia]"
```

## Supported Models

### Speech-to-Text

| Name | Provider | Notes |
|------|----------|-------|
| `deepgram` | Deepgram Nova | Cloud API, streaming support |
| `nvidia-parakeet` | NVIDIA | Local, high accuracy |
| `whisper-tiny` | HuggingFace | Local, fast, English-focused. **Default** |
| `whisper-base` | HuggingFace | Better accuracy, still lightweight |
| `whisper-small` | HuggingFace | Best local accuracy |


### Text-to-Speech

| Name | Provider | Notes |
|------|----------|-------|
| `cartesia` | Cartesia Sonic | Cloud API, natural voices |
| `deepgram` | Deepgram Aura | Cloud API, natural voices  |
| `piper` | Piper TTS | Local ONNX, multiple voices. **Default** |
| `speecht5` | HuggingFace | Local, lightweight |
| `bark` | HuggingFace | Local, expressive |


### LLM

| Name | Provider | Notes |
|------|----------|-------|
| `bedrock` | AWS Bedrock | Claude Haiku/Sonnet/Opus. **Default** |
| `openai` | OpenAI | GPT-4o, GPT-4o-mini via Strands SDK |
| `anthropic` | Anthropic | Claude models via Strands SDK |
| `gemini` | Google | Gemini Flash/Pro via Strands SDK |
| `ollama` | Ollama | Any local model (Llama 3, Mistral, etc.) |
| `local` | HuggingFace | Any local causal LM (direct, no Strands) |

### VAD

| Name | Provider |
|------|----------|
| `silero` | Silero VAD |

## Configuration

### Environment Variables

| Variable | Required for |
|----------|-------------|
| `BEDROCK_API_KEY` or `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` | Bedrock LLM |
| `AWS_DEFAULT_REGION` | Bedrock (default: `us-east-1`) |
| `OPENAI_API_KEY` | OpenAI LLM |
| `ANTHROPIC_API_KEY` | Anthropic LLM |
| `GOOGLE_API_KEY` | Gemini LLM |
| `DEEPGRAM_API_KEY` | Deepgram STT/TTS |
| `CARTESIA_API_KEY` | Cartesia TTS |

API keys can also be set through the agent wizard UI — they're stored in `~/.roohai/secrets.yaml` with `0600` permissions.

### Agent Config

Agents are defined as YAML files in `~/.roohai/agents/`. Each agent specifies its models, system prompt, and transport:

```yaml
name: my-agent
system_prompt: "You are a helpful voice assistant."
llm_streaming: true
pipeline:
  stt: whisper-base
  tts: piper
  llm: bedrock-claude
  vad: silero
transport: websocket
```

## Usage

### CLI

```bash
roohai                           # Start with defaults
roohai --port 3000               # Custom port
roohai --reload                  # Auto-reload for development
roohai --log-level debug         # Verbose logging
```

### Python API

#### Builder Pattern

```python
from roohai import Rooh

pipeline = (
    Rooh.builder()
    .stt("whisper-tiny")
    .tts("piper")
    .llm("bedrock", model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
    .vad("silero")
    .system_prompt("You are a helpful assistant.")
    .build()
)
pipeline.load()

# Transcribe
text = await pipeline.transcribe(audio_bytes)

# Chat
response = await pipeline.chat("Hello, how are you?")

# Stream
async for chunk in pipeline.chat_stream("Tell me a story"):
    print(chunk, end="")

# Full pipeline: audio in -> text + audio out
transcription, response, audio = await pipeline.process_audio(audio_bytes)
```

#### From Config

```python
from roohai import Rooh

pipeline = Rooh.from_config({
    "pipeline": {"stt": "whisper-tiny", "tts": "piper", "llm": "bedrock-claude", "vad": "silero"},
    "system_prompt": "You are a helpful assistant.",
})
pipeline.load()
```

### REST API

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/health` | Health check with active model info |
| `POST` | `/api/transcribe` | Audio file -> transcription |
| `POST` | `/api/chat` | Text -> LLM response |
| `POST` | `/api/synthesize` | Text -> WAV audio |
| `POST` | `/api/voice-chat` | Audio in -> text + audio out |
| `POST` | `/api/webrtc/offer` | WebRTC SDP offer/answer |
| `GET` | `/api/models` | List available and active models |
| `POST` | `/api/models/swap` | Hot-swap a model at runtime |

## Examples

The [`examples/`](examples/) directory contains complete working apps:

- **[quickstart](examples/quickstart/)** — Minimal voice agent
- **[barge-in-hook](examples/barge-in-hook/)** — Custom barge-in handling
- **[session-memory-agent](examples/session-memory-agent/)** — Per-session conversation memory with Strands SDK
- **[voice-weather-agent](examples/voice-weather-agent/)** — Voice agent with tool use (weather API)
- **[skill-interview-agent](examples/skill-interview-agent/)** — Structured interview agent

## Extending RoohAI

### Custom Models

Create a class extending `STTModel`, `TTSModel`, or `LLMModel`:

```python
from roohai import STTModel, registry

class MySTT(STTModel):
    def load(self): ...
    def unload(self): ...
    @property
    def is_loaded(self) -> bool: ...
    def transcribe(self, audio, sample_rate) -> str: ...

registry.register_stt("my-stt", MySTT)
```

### LLM Hooks

Override LLM behavior with hooks for tool use, RAG, or custom logic:

```python
pipeline.set_llm_hooks(
    hook=my_batch_handler,
    stream_hook=my_streaming_handler,
)
```

See the [Strands SDK integration](examples/session-memory-agent/) for a full example with tool use and conversation memory.

## Documentation

Full docs are available at `http://localhost:8000/guide` when the server is running, including architecture details, advanced configuration, and the complete model catalog.

## Contributing

Contributions are welcome. Please open an issue first to discuss what you'd like to change.

```bash
git clone https://github.com/Fraser27/roohai-framework.git
cd roohai-framework
pip install -e ".[all]"
pytest
```

## License

Apache 2.0 — see [LICENSE](LICENSE).
