# deliberate

Multi-LLM ensemble orchestrator for code generation and review.

## Overview

**deliberate** is a CLI tool that orchestrates multiple LLM agents to collaboratively plan, execute, and review coding tasks. It uses ensemble methods - independent generation, peer voting, and synthesis - to produce higher-quality outputs than any single agent.

**Core insight**: LLMs are better at evaluating code than generating it. By having multiple agents generate candidates and then cross-evaluate, we exploit this asymmetry.

## Features

- **Multi-agent planning**: Multiple LLMs propose plans, optionally debate, and vote on the best approach
- **Isolated execution**: Each agent works in its own git worktree to prevent conflicts
- **Peer review**: Multiple agents score and vote on execution results
- **Budget enforcement**: Hard limits on tokens, cost, and time
- **CI integration**: Non-interactive mode with structured output for automation
- **OpenTelemetry tracing**: Built-in observability support
- **Testable**: Fake adapters enable full workflow testing at $0

## Installation

```bash
# Clone the repository
git clone https://github.com/hardbyte/deliberate.git
cd deliberate

# Install with uv (recommended)
uv sync

# Or install with pip
pip install -e .

# With dev dependencies
uv sync --dev
# or
pip install -e ".[dev]"

# With tracing support
uv sync --extra tracing
# or
pip install -e ".[tracing]"
```

## Quick Start

1. Initialize a configuration file:

```bash
deliberate init
```

2. Edit `.deliberate.yaml` to configure your agents

3. Set up environment variables (copy `.env.example` to `.env` and add your API keys):

```bash
cp .env.example .env
# Edit .env and add your API keys
```

4. Run a task:

```bash
# Run with inline task description
deliberate run "Add a function to calculate fibonacci numbers"

# Or load task from a file
echo "Add a function to calculate fibonacci numbers" > task.txt
deliberate run "@task.txt"
```

## Configuration

### Configuration File Locations

Deliberate searches for configuration files in the following order:

1. **Explicit path**: `deliberate run --config /path/to/config.yaml`
2. **Current directory**: `./.deliberate.yaml` or `./deliberate.yaml`
3. **User config directory**: OS-specific user configuration directory
   - Linux/macOS: `~/.config/deliberate/config.yaml`
   - Windows: `%APPDATA%\deliberate\config.yaml`

To create a user-level configuration:

```bash
# Create config in user config directory
deliberate init --user

# Or create in current directory (default)
deliberate init
```

### Configuration Format

Edit `.deliberate.yaml` to configure agents and workflow:

```yaml
agents:
  claude:
    type: cli
    command: ["claude", "--print", "-p"]
    capabilities: [planner, executor, reviewer]

  gemini:
    type: cli
    command: ["gemini", "-y", "--output-format", "json"]
    # API key loaded from GEMINI_API_KEY environment variable
    capabilities: [planner, reviewer]

workflow:
  planning:
    enabled: true
    agents: [claude, gemini]

  execution:
    enabled: true
    agents: [claude]

  review:
    enabled: true
    agents: [gemini, codex]
```

### Environment Variables

Deliberate uses [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/) to load environment variables:

1. **System environment**: `export GEMINI_API_KEY=your-key`
2. **.env file**: Create a `.env` file (see `.env.example`)
3. **Config file inline**: Add under `agents.<name>.env` (not recommended for secrets)

Priority: config inline > .env file > system environment

## CLI Commands

```bash
# Run a task through the deliberate
deliberate run "your task description"

# Load task from a file
deliberate run "@task.txt"
echo "Add error handling to the API" > task.txt
deliberate run "@task.txt"

# Run with specific agents
deliberate run "task" --agents claude,gemini

# Skip planning or review phases
deliberate run "task" --skip-planning
deliberate run "task" --skip-review

# Output as JSON
deliberate run "task" --json

# Enable tracing
deliberate run "task" --trace
deliberate run "task" --otlp-endpoint http://localhost:4317

# Validate configuration
deliberate validate

# Create new config file
deliberate init                # Create .deliberate.yaml in current directory
deliberate init --user         # Create config.yaml in user config directory
```

## Workflow Phases

### 1. Planning Phase

Multiple agents propose plans for the task. They can optionally debate their approaches, and a judge (another LLM or voting) selects the best plan.

### 2. Execution Phase

The selected plan is executed by one or more agents in isolated git worktrees. Each agent produces a diff and summary of changes.

### 3. Review Phase

Multiple agents review the execution results, scoring on criteria like correctness, code quality, completeness, and risk. Votes are aggregated using Borda count or approval voting.

## Voting Methods

- **Borda count**: Points based on ranking position
- **Approval voting**: Count approvals above a threshold
- **Weighted Borda**: Borda with weighted reviewer importance

## Testing

```bash
# Run all tests
uv run pytest

# Run unit tests only
uv run pytest tests/unit/

# Run integration tests
uv run pytest tests/integration/

# Run with coverage
uv run pytest --cov=deliberate

# Skip live LLM tests (default)
uv run pytest -m "not live_llm"

# Run live tests (costs money!)
RUN_LIVE_LLM_TESTS=1 uv run pytest -m live_llm
```


## Architecture

```
deliberate/
├── src/deliberate/
│   ├── cli.py           # Typer CLI entrypoint
│   ├── config.py        # Pydantic configuration
│   ├── orchestrator.py  # Main workflow coordinator
│   ├── types.py         # Core type definitions
│   ├── adapters/        # LLM backend adapters
│   ├── phases/          # Planning, execution, review
│   ├── voting/          # Vote aggregation algorithms
│   ├── git/             # Worktree management
│   ├── budget/          # Cost tracking
│   ├── tracing/         # OpenTelemetry integration
│   └── prompts/         # Prompt templates
└── tests/
    ├── unit/            # Fast, no-cost unit tests
    ├── integration/     # Workflow tests with fakes
    └── live/            # Optional real API tests
```

## License

MIT
