# SalTTS

**Real-time, controllable, adaptive neural TTS system for AI VTubers**

SalTTS is a production-ready text-to-speech system designed for real-time AI VTuber applications with <300ms latency.

## Features

- **Dual Generator Architecture**: Neural (Flow/Diffusion/GAN) + Parametric (WORLD vocoder)
- **Real-time Streaming**: <300ms end-to-end latency with KV caching
- **Adaptive Mixing**: RL-based (Soft Actor-Critic) with HNSW memory
- **Multi-modal Fusion**: 3-channel NMDB integration
- **High Quality**: Target MOS 4.0+, character consistency 0.85+

## Architecture

```
NMDB (3 channels) → Multi-Modal Fusion → Prosody Generation →
Dual Generators (A+B) → RL Adaptive Mixing → Post-Processing → Audio
```

## Installation

### Using Poetry (Recommended)

```bash
poetry install
```

### Using pip

```bash
pip install -e .
```

## Quick Start

```python
from saltts.inference import SalTTSEngine

# Initialize engine
engine = SalTTSEngine(config_dir="config")

# Generate speech
audio = engine.synthesize(
    text="Hello, this is SalTTS!",
    emotion="neutral",
    speaker_id=0
)

# Save audio
engine.save_audio(audio, "output.wav")
```

## Configuration

Configuration files are located in `config/`:
- `model.yaml` - Model architectures
- `runtime.yaml` - Runtime settings
- `training.yaml` - Training hyperparameters
- `nmdb_channels.yaml` - NMDB configuration
- `deployment.yaml` - Deployment settings

## Training

```bash
# Stage 1: Train generators
python scripts/train_stage1.py

# Stage 2: Train transformer
python scripts/train_stage2.py

# Stage 3: Train integration
python scripts/train_stage3.py

# Stage 4: RL training
python scripts/train_stage4_rl.py
```

## Performance

- End-to-end latency: <300ms (target <200ms)
- Real-Time Factor: <0.5
- MOS score: 4.0+
- Character consistency: 0.85+

## License

MIT License - see LICENSE file for details
