# Surivoice — MVP Architectural Plan

A local-first, open-source CLI tool that takes a video/audio file, extracts audio, performs speech-to-text transcription with speaker diarization, and outputs a structured Markdown transcript.

---

## 1. High-Level Architecture

```mermaid
graph TD
    CLI["CLI Entry Point<br/>(typer)"] --> Pipeline["Pipeline Orchestrator"]
    Pipeline --> AudioExtractor["Audio Extractor"]
    Pipeline --> Transcriber["Transcriber"]
    Pipeline --> Diarizer["Speaker Diarizer"]
    Pipeline --> Merger["Segment Merger"]
    Pipeline --> Formatter["Output Formatter"]

    AudioExtractor -->|WAV 16kHz mono| Transcriber
    AudioExtractor -->|WAV 16kHz mono| Diarizer
    Transcriber -->|TranscriptionSegment list| Merger
    Diarizer -->|DiarizationSegment list| Merger
    Merger -->|MergedSegment list| Formatter
    Formatter -->|Markdown string| CLI
```

The architecture follows a **linear pipeline** pattern. Each stage is a self-contained module with a single responsibility, communicating through well-defined data structures. The pipeline orchestrator is the only component that knows about all stages — individual stages have zero knowledge of each other.

---

## 2. Module Division & Responsibilities

The project is organized into a single Python package `surivoice` with the following internal modules:

```
surivoice/
├── __init__.py
├── __main__.py            # python -m surivoice support
├── cli.py                 # CLI argument parsing, entry point
├── pipeline.py            # Pipeline orchestrator
├── audio/
│   ├── __init__.py
│   └── extractor.py       # Audio extraction from video/audio files
├── transcription/
│   ├── __init__.py
│   └── transcriber.py     # Speech-to-text via faster-whisper
├── diarization/
│   ├── __init__.py
│   └── diarizer.py        # Speaker diarization via pyannote.audio
├── merge/
│   ├── __init__.py
│   └── merger.py           # Timestamp-based segment merging
├── output/
│   ├── __init__.py
│   └── formatter.py       # Markdown output generation
├── models.py              # Shared data structures (Pydantic models)
├── config.py              # Configuration / defaults
├── errors.py              # Custom exception hierarchy
└── progress.py            # Progress reporting utilities
```

### Module Responsibilities

| Module | Responsibility |
|--------|---------------|
| `cli` | Parse CLI arguments, validate input file exists, invoke the pipeline, write output |
| `pipeline` | Orchestrate the full processing pipeline, manage temp files, report progress |
| `audio.extractor` | Extract audio from any video/audio format → WAV 16kHz mono PCM |
| `transcription.transcriber` | Run faster-whisper, return timestamped transcription segments |
| `diarization.diarizer` | Run pyannote.audio, return timestamped speaker segments |
| `merge.merger` | Align transcription segments with speaker labels using timestamp overlap |
| `output.formatter` | Convert merged segments into structured Markdown |
| `models` | Define all shared data structures as immutable Pydantic models |
| `config` | Centralized defaults (model size, device, language, etc.) |
| `errors` | Custom exception hierarchy for clear error messages |
| `progress` | Wrap `rich` progress bars for pipeline stage reporting |

---

## 3. Recommended Libraries & Justifications

| Dependency | Purpose | Justification |
|-----------|---------|---------------|
| **`faster-whisper`** | Speech-to-text | 2–4× faster than `openai-whisper` with identical accuracy. Uses CTranslate2 for optimized inference. ~3 GB VRAM for large-v2 (int8) vs ~11 GB for original. |
| **`pyannote.audio`** | Speaker diarization | Industry-standard open-source diarization. Pre-trained pipeline `pyannote/speaker-diarization-3.1` achieves state-of-the-art DER. Runs locally. |
| **`ffmpeg` (subprocess)** | Audio extraction | Direct `subprocess` call to `ffmpeg` avoids a Python wrapper dependency. FFmpeg is a hard requirement anyway (pyannote needs it). One-line command: `ffmpeg -i input -ar 16000 -ac 1 -f wav output.wav`. No need for `pydub` or `ffmpeg-python` as wrappers add complexity without benefit for this single use case. |
| **`typer`** | CLI framework | Built on top of Click, but uses **Python type hints** natively for argument parsing — perfect fit with Pydantic and strict typing philosophy. Auto-generates help, shell completion. Same extensibility as Click (subcommands, plugins) with less boilerplate. |
| **`pydantic`** | Data validation & typing | Strict runtime validation, immutable models via `frozen=True`, serialization to dict/JSON for free. Enforces type safety across the entire pipeline. |
| **`rich`** | Terminal output | Beautiful progress bars, spinners, formatted error messages. Massively improves UX for long-running operations. |
| **`torch`** | ML runtime | Required by both `faster-whisper` and `pyannote.audio`. Not a direct dependency — pulled transitively. |

### System Dependency

| Dependency | Required | Notes |
|-----------|----------|-------|
| **FFmpeg** | Yes | Must be installed and on `PATH`. Checked at CLI startup with a clear error message if missing. |

> [!IMPORTANT]
> **Hugging Face Token**: `pyannote.audio` requires accepting model license terms on Hugging Face and providing an auth token. The CLI must support `--hf-token` flag and/or `HF_TOKEN` environment variable. This is a mandatory step for first-time users and must be clearly documented.

---

## 4. Data Flow

```mermaid
sequenceDiagram
    participant User
    participant CLI
    participant Pipeline
    participant AudioExtractor
    participant Transcriber
    participant Diarizer
    participant Merger
    participant Formatter

    User->>CLI: surivoice -i meeting.mp4 -o transcript.md
    CLI->>Pipeline: run(input_path, output_path, config)

    Pipeline->>AudioExtractor: extract(input_path) → Path
    Note right of AudioExtractor: video/audio → WAV 16kHz mono

    par Parallel execution (future optimization)
        Pipeline->>Transcriber: transcribe(wav_path) → list[TranscriptionSegment]
        Pipeline->>Diarizer: diarize(wav_path) → list[DiarizationSegment]
    end

    Pipeline->>Merger: merge(transcriptions, diarizations) → list[MergedSegment]
    Pipeline->>Formatter: format(merged_segments) → str
    Pipeline-->>CLI: markdown_content
    CLI->>User: Write transcript.md
```

### Step-by-step

1. **Input validation** — CLI checks the file exists and is a supported format
2. **Audio extraction** — FFmpeg converts input to a temporary WAV file (16kHz, mono, PCM s16le)
3. **Transcription** — `faster-whisper` processes the WAV and returns word/segment-level timestamps
4. **Diarization** — `pyannote.audio` processes the same WAV and returns speaker-labeled time ranges
5. **Merging** — Transcription segments are assigned speaker labels based on maximum timestamp overlap
6. **Formatting** — Merged segments are rendered as structured Markdown
7. **Output** — The Markdown string is written to the output file (or stdout if `-o -`)

---

## 5. Data Structures

All data structures are **frozen Pydantic models** to enforce immutability and runtime type validation throughout the pipeline.

```python
from pydantic import BaseModel


class TranscriptionSegment(BaseModel, frozen=True):
    """A segment of transcribed text with timestamps."""
    start: float       # seconds
    end: float         # seconds
    text: str


class DiarizationSegment(BaseModel, frozen=True):
    """A speaker-labeled time range."""
    start: float       # seconds
    end: float         # seconds
    speaker: str       # e.g., "SPEAKER_00", "SPEAKER_01"


class MergedSegment(BaseModel, frozen=True):
    """A transcription segment attributed to a speaker."""
    start: float       # seconds
    end: float         # seconds
    speaker: str
    text: str


class TranscriptionResult(BaseModel, frozen=True):
    """Complete result of the pipeline."""
    segments: tuple[MergedSegment, ...]
    detected_language: str | None
    duration_seconds: float
    speakers_count: int
```

### Merging Algorithm

The merger uses a **maximum overlap** strategy:

1. For each `TranscriptionSegment`, compute the temporal overlap with every `DiarizationSegment`
2. Assign the speaker label from the `DiarizationSegment` with the greatest overlap
3. Consecutive segments with the same speaker are coalesced into a single `MergedSegment`

This is an O(T × D) algorithm where T = transcription segments and D = diarization segments. For typical meetings (< 2 hours), this is well within acceptable performance bounds. If needed in the future, an interval-tree approach can reduce this to O(T log D).

---

## 6. Error Handling Strategy

### Exception Hierarchy

```python
class SurivoiceError(Exception):
    """Base exception for all Surivoice errors."""

class InputFileError(SurivoiceError):
    """File not found, unreadable, or unsupported format."""

class FFmpegError(SurivoiceError):
    """FFmpeg not installed or extraction failed."""

class TranscriptionError(SurivoiceError):
    """Whisper model load or inference failure."""

class DiarizationError(SurivoiceError):
    """Pyannote model load or inference failure."""

class MergeError(SurivoiceError):
    """Segment alignment failure (e.g., empty inputs)."""

class OutputError(SurivoiceError):
    """Unable to write output file."""
```

### Strategy

| Principle | Implementation |
|-----------|---------------|
| **Fail fast with clear messages** | Validate input file and FFmpeg availability before starting heavy processing |
| **No silent failures** | Every caught exception is logged with context; empty results raise warnings |
| **Structured logging** | Use `logging` module with `rich.logging.RichHandler` for formatted output |
| **Graceful degradation** | If diarization fails but transcription succeeds, output transcript without speaker labels (with a warning) |
| **Temp file cleanup** | Use `tempfile.TemporaryDirectory` as a context manager — guaranteed cleanup even on crash |
| **Exit codes** | 0 = success, 1 = user error (bad input), 2 = system error (FFmpeg missing), 3 = processing error |

---

## 7. Performance Considerations

| Concern | Strategy |
|---------|----------|
| **Model loading time** | Models are loaded once per invocation. Display a spinner during load. Document expected first-run download times. |
| **GPU vs CPU** | Auto-detect CUDA availability via `torch.cuda.is_available()`. Default to GPU if available, fall back to CPU. Expose `--device cpu\|cuda` flag. |
| **Model size selection** | Default to `medium` model for balance of speed/accuracy. Expose `--model tiny\|base\|small\|medium\|large-v2\|large-v3` flag. |
| **int8 quantization** | Use `compute_type="int8"` by default on GPU for ~2× speed and ~3× memory reduction with minimal accuracy loss. |
| **Temp file I/O** | WAV files stored in system temp dir. For very large files (> 2 hours), this is the main disk bottleneck. Document this. |
| **Parallelism (future)** | Transcription and diarization are independent — they can run concurrently. *Not in MVP* due to GPU memory contention, but the architecture makes this trivial to add. |
| **Memory** | `faster-whisper` streams audio in chunks, keeping memory usage bounded regardless of file length. |

### Expected Performance (rough benchmarks)

| File Length | Model | Device | Estimated Time |
|------------|-------|--------|---------------|
| 10 min | medium | GPU (RTX 3060) | ~1–2 min |
| 10 min | medium | CPU (8-core) | ~5–8 min |
| 60 min | medium | GPU (RTX 3060) | ~6–10 min |
| 60 min | large-v3 | GPU (RTX 3080) | ~10–15 min |

---

## 8. Scalability Considerations

Surivoice is a **single-file-at-a-time CLI tool** — "scalability" here means:

| Dimension | Approach |
|-----------|----------|
| **Long files (> 2 hours)** | `faster-whisper` handles streaming internally. Diarization may require more RAM. Document recommended hardware. |
| **Batch processing** | Not in MVP, but the pipeline's `run()` function takes a single file — a future `batch` command can call it in a loop or with multiprocessing. |
| **Multiple languages** | `faster-whisper` auto-detects language by default. Expose `--language` flag to override. |
| **Multiple output formats** | Architecture supports adding new `Formatter` subclasses (JSON, SRT, VTT) without touching the pipeline. |
| **Plugin system (future)** | Typer's (Click-based) group/subcommand pattern naturally supports plugin discovery via entry points. |

---

## 9. Risks & Mitigation

| Risk | Severity | Mitigation |
|------|----------|------------|
| **Hugging Face token requirement** | High | Clear onboarding docs. CLI checks for `HF_TOKEN` early and provides actionable error message with a link. Consider bundling a community model that doesn't require auth. |
| **CUDA/PyTorch version conflicts** | High | Pin compatible versions in `pyproject.toml`. Provide separate install extras: `pip install surivoice[cpu]` vs `pip install surivoice[gpu]`. Document common issues. |
| **FFmpeg not installed** | Medium | Check at startup with `shutil.which("ffmpeg")`. Show install instructions per OS. |
| **Large model download on first run** | Medium | Show download progress. Document model sizes. Allow `--model tiny` for quick testing. |
| **Diarization inaccuracy on noisy audio** | Medium | Document known limitations. Expose `--min-speakers` / `--max-speakers` flags to help pyannote. |
| **Memory exhaustion on long files** | Medium | Document recommended RAM per model size. Warn if system RAM is below threshold. |
| **Windows compatibility** | Low | `pyannote.audio` officially supports Linux/macOS only. Document this. Don't block Windows, but don't test for it in MVP. |

---

## 10. Future Extensibility Strategy

The architecture is designed for these future extensions **without refactoring**:

```mermaid
graph LR
    subgraph MVP
        CLI_MVP["CLI (typer)"]
        Pipeline_MVP["Pipeline"]
        Modules["Audio / Transcribe / Diarize / Merge / Format"]
    end

    subgraph Future
        JSON["JSON Formatter"]
        SRT["SRT Formatter"]
        Summary["LLM Summary"]
        Rename["Speaker Rename"]
        Interactive["Interactive Mode"]
        Batch["Batch Processing"]
        API["REST API"]
        Plugins["Plugin System"]
    end

    Modules --> JSON
    Modules --> SRT
    Pipeline_MVP --> Summary
    Pipeline_MVP --> Rename
    CLI_MVP --> Interactive
    CLI_MVP --> Batch
    Pipeline_MVP --> API
    CLI_MVP --> Plugins
```

### Extension Points

| Extension | How It Plugs In |
|-----------|----------------|
| `--json` output | New `JsonFormatter` class implementing a `Formatter` protocol. Zero changes to pipeline. |
| `--srt` / `--vtt` output | Same — new formatter classes. |
| `--summary` | Post-processing step after the merger. Takes `list[MergedSegment]`, calls a local LLM. Added as an optional pipeline stage. |
| `--rename-speakers` | Post-processing step that maps `SPEAKER_XX` → user-provided names. Can be interactive. |
| `--interactive` | TUI using `rich` or `textual`. Wraps the same pipeline. |
| Batch CLI | `surivoice batch ./meetings/` — loops over files calling the same pipeline. |
| Plugin system | Typer/Click entry points: third-party packages can register new commands or formatters. |

### Formatter Protocol

```python
from typing import Protocol

class Formatter(Protocol):
    def format(self, result: TranscriptionResult) -> str: ...
```

All formatters implement this protocol. The pipeline is parameterized by `Formatter`, not a concrete class.

---

## 11. Packaging Strategy

### Project Structure

```
Surivoice/
├── pyproject.toml
├── README.md
├── LICENSE                  # MIT
├── CHANGELOG.md
├── src/
│   └── surivoice/
│       ├── __init__.py
│       ├── __main__.py
│       ├── cli.py
│       ├── ...
│       └── models.py
└── tests/
    ├── conftest.py
    ├── test_merger.py
    ├── test_formatter.py
    └── fixtures/
        └── short_sample.wav
```

### `pyproject.toml` Key Sections

```toml
[project]
name = "surivoice"
version = "0.1.0"
description = "Local-first CLI for speech transcription with speaker diarization"
requires-python = ">=3.10"
license = { text = "MIT" }

dependencies = [
    "typer>=0.9",
    "pydantic>=2.0",
    "rich>=13.0",
    "faster-whisper>=1.0",
    "pyannote.audio>=3.1",
]

[project.optional-dependencies]
cpu = ["torch>=2.0"]
gpu = ["torch>=2.0"]  # User installs CUDA-enabled torch separately
dev = [
    "pytest>=8.0",
    "pytest-cov",
    "ruff",
    "mypy",
]

[project.scripts]
surivoice = "surivoice.cli:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/surivoice"]

[tool.ruff]
target-version = "py310"
line-length = 100

[tool.mypy]
strict = true
```

### Key Decisions

| Decision | Rationale |
|----------|-----------|
| **`src/` layout** | Prevents accidental imports from the project root. Standard for PyPI packages. |
| **Hatchling build backend** | Modern, fast, minimal config. Better than setuptools for new projects. |
| **`[project.scripts]` entry point** | Registers `surivoice` as a CLI command on install. |
| **`__main__.py`** | Enables `python -m surivoice` invocation for development/debugging. |
| **`requires-python = ">=3.10"`** | Uses `X | Y` union syntax, `match/case` (if needed), and modern typing features. |
| **`ruff` + `mypy --strict`** | Enforces code quality from day one. Contributors get instant feedback. |

---

## 12. Ambiguities & Decisions

| Ambiguity | Decision | Alternative |
|-----------|----------|-------------|
| **Whisper model default** | Use `medium` — best speed/accuracy tradeoff for MVP | `small` for faster CI, `large-v3` for production quality |
| **Diarization model** | Use `pyannote/speaker-diarization-3.1` (requires HF token) | `pyannote/speaker-diarization-community-1` is a free alternative with slightly lower accuracy |
| **Output to stdout vs file** | Require `-o` flag, support `-o -` for stdout | Default to stdout (Unix philosophy) — but Markdown is nicer as a file |
| **Language detection** | Auto-detect by default, expose `--language` override | Force user to specify language — worse UX |
| **Parallel transcription + diarization** | Sequential in MVP (simpler, avoids GPU memory contention) | Parallel with `asyncio` — add in v0.2 |
| **Formatter extensibility** | Use `Protocol` (structural typing) | Use ABC — more explicit but heavier |

---

## MVP Execution Plan

Each step is a self-contained unit of work that produces a testable result.

| Step | Deliverable | Depends On |
|------|------------|------------|
| **1** | Project scaffolding + CLI skeleton | — |
| **2** | Audio extraction module | Step 1 |
| **3** | Transcription module | Step 2 |
| **4** | Diarization module | Step 2 |
| **5** | Segment merger | Steps 3, 4 |
| **6** | Markdown formatter + output | Step 5 |
| **7** | Pipeline orchestrator (wire everything) | Steps 2–6 |
| **8** | Polish: error messages, progress bars, README | Step 7 |

---

### Step 1 — Project Scaffolding & CLI Skeleton (NEXT)

Goal: A working `surivoice --help` command, installable via `pip install -e .`, with all tooling configured.

#### Files to Create

| File | Purpose |
|------|---------|
| `pyproject.toml` | Package metadata, dependencies, entry point, ruff/mypy config |
| `src/surivoice/__init__.py` | Package init with `__version__` |
| `src/surivoice/__main__.py` | `python -m surivoice` support |
| `src/surivoice/cli.py` | Typer app with `transcribe` command (stub) |
| `src/surivoice/models.py` | Pydantic models for all data structures |
| `src/surivoice/config.py` | `PipelineConfig` Pydantic model with defaults |
| `src/surivoice/errors.py` | Custom exception hierarchy |
| `tests/conftest.py` | Pytest fixtures |
| `tests/test_cli.py` | Test `--help` works, invalid input shows error |
| `README.md` | Project description, install instructions |
| `LICENSE` | MIT license |

#### CLI Interface (Step 1 scope)

```python
import typer

app = typer.Typer(
    name="surivoice",
    help="Local-first speech transcription with speaker diarization.",
)

@app.command()
def transcribe(
    input_file: Path = typer.Option(..., "--input", "-i", help="Input video/audio file"),
    output_file: Path = typer.Option(..., "--output", "-o", help="Output markdown file"),
    model: str = typer.Option("medium", help="Whisper model size"),
    device: str = typer.Option("auto", help="Device: auto, cpu, or cuda"),
    language: str | None = typer.Option(None, help="Language code (auto-detect if omitted)"),
    hf_token: str | None = typer.Option(None, envvar="HF_TOKEN", help="Hugging Face token"),
) -> None:
    """Transcribe an audio/video file with speaker diarization."""
    ...
```

#### Verification for Step 1

```bash
# Install in dev mode
pip install -e ".[dev]"

# CLI works
surivoice --help
surivoice transcribe --help

# Typing passes
mypy src/

# Linting passes
ruff check src/ tests/

# Tests pass
pytest tests/
```
