# zwarm

Multi-agent CLI orchestration research platform. Coordinate multiple coding agents (Codex, Claude Code) with delegation, conversation, trajectory alignment, and automatic context management.

## Key Features

- **Multi-adapter support**: Codex MCP, Claude Code adapters with unified interface
- **Sync & async modes**: Conversational (iterative refinement) or fire-and-forget
- **Token tracking**: Per-session token usage tracked and persisted for cost analysis
- **Context compaction**: Automatic LRU-style pruning when approaching context limits
- **Trajectory watchers**: Composable guardrails (progress, budget, scope, pattern, delegation)
- **State persistence**: Resume sessions, track history, replay events
- **Weave integration**: Full tracing and observability

## Installation

```bash
# From the workspace (recommended during development)
cd /path/to/labs
uv sync

# Or install directly
uv pip install -e ./zwarm
```

**Requirements:**
- Python 3.13+
- `codex` CLI installed (for Codex adapter)
- `claude` CLI installed (for Claude Code adapter)

## Quick Start

```bash
# 1. Initialize zwarm in your project
zwarm init

# 2. Test an executor directly
zwarm exec --task "What is 2+2?"

# 3. Run the orchestrator with a task
zwarm orchestrate --task "Create a hello world Python function"

# 4. Check state after running
zwarm status

# 5. View event history
zwarm history
```

### Task Input Options

```bash
# Direct task
zwarm orchestrate --task "Build a REST API"

# From file
zwarm orchestrate --task-file task.md

# From stdin
echo "Fix the bug in auth.py" | zwarm orchestrate
```

## Configuration

zwarm looks for configuration in this order:
1. `--config` flag (YAML file)
2. `.zwarm/config.toml` (created by `zwarm init`)
3. `config.toml` in working directory (legacy, for backwards compat)
4. Default settings

### Minimal .zwarm/config.toml

```toml
[weave]
enabled = true
project = "your-wandb-entity/zwarm"

[executor]
adapter = "codex_mcp"  # or "claude_code"
```

### Environment Variables

```bash
# Weave tracing (optional but recommended)
export WEAVE_PROJECT="your-entity/zwarm"

# Executor authentication (required - set based on which adapter you use)
export OPENAI_API_KEY="sk-..."        # Required for codex_mcp adapter
export ANTHROPIC_API_KEY="sk-ant-..." # Required for claude_code adapter
```

**Important:** The orchestrator agent runs with your credentials, but the executor adapters (Codex, Claude Code) need their own authentication. If executors fail with auth errors, check that the appropriate API key is set in your environment.

You can also put these in a `.env` file in your project root - zwarm will load it automatically.

### Full Configuration Reference

```yaml
# config.yaml
orchestrator:
  lm: gpt-5-mini              # Model for the orchestrator itself
  max_steps: 100              # Maximum orchestrator steps
  compaction:                 # Context window management
    enabled: true
    max_tokens: 100000        # Trigger compaction above this
    threshold_pct: 0.85       # Compact at 85% of max
    target_pct: 0.7           # Target 70% after compaction
    keep_first_n: 2           # Always keep system + task
    keep_last_n: 10           # Always keep recent context

executor:
  adapter: codex_mcp          # Default adapter: codex_mcp | claude_code
  model: null                 # Model override (null = use adapter default)
                              # codex_mcp default: gpt-5.1-codex-mini
                              # claude_code default: claude-sonnet-4-5-20250514
  sandbox: workspace-write    # Codex sandbox mode

weave:
  enabled: true
  project: your-entity/zwarm

state_dir: .zwarm             # State directory for sessions/events

watchers:
  enabled: true
  message_role: user            # Role for nudge messages: user | assistant | system
  watchers:
    - name: progress
    - name: budget
      config:
        max_steps: 50
        max_sessions: 10
    - name: delegation_reminder
      config:
        threshold: 10           # Nudge after N consecutive non-delegation calls
        lookback: 30            # How many messages to check
    - name: scope
      config:
        keywords: []
```

## Adapters

zwarm supports multiple CLI coding agents through adapters. Each adapter wraps a different coding CLI and handles the mechanics of starting sessions, sending messages, and capturing responses.

### Codex MCP (default)

Uses Codex via MCP server for true conversational sessions. This is the recommended adapter for iterative work where you need back-and-forth refinement.

```bash
# Sync mode (conversational)
zwarm exec --adapter codex_mcp --task "Add a login function"

# The orchestrator can have back-and-forth conversations
# using delegate() and converse() tools
```

| Setting | Value |
|---------|-------|
| Default model | `gpt-5.1-codex-mini` |
| Requires | `codex` CLI installed |
| Auth | `OPENAI_API_KEY` environment variable |

### Claude Code

Uses Claude Code CLI for execution. Good alternative when you want Claude's capabilities.

```bash
zwarm exec --adapter claude_code --task "Fix the type errors"
```

| Setting | Value |
|---------|-------|
| Default model | `claude-sonnet-4-5-20250514` |
| Requires | `claude` CLI installed and authenticated |
| Auth | `ANTHROPIC_API_KEY` or `claude` CLI auth |

### Model Selection

Models are selected with this precedence (highest to lowest):

1. **Per-delegation override**: `delegate(task="...", model="o3")`
2. **Config file**: `executor.model` in config.toml or zwarm.yaml
3. **Adapter default**: Each adapter has a sensible default

```yaml
# config.toml - override the default model
[executor]
adapter = "codex_mcp"
model = "gpt-5.1-codex-max"  # Use the more capable model
```

```bash
# Or override per-execution
zwarm exec --model gpt-5.1-codex-max --task "Complex refactoring"
```

## Watchers (Trajectory Alignment)

Watchers are composable guardrails that monitor agent behavior and can intervene when things go wrong.

### Available Watchers

| Watcher | Description |
|---------|-------------|
| `progress` | Detects stuck/spinning agents |
| `budget` | Monitors step/session limits (counts only active sessions) |
| `scope` | Detects scope creep from original task |
| `pattern` | Custom regex pattern matching |
| `quality` | Code quality checks |
| `delegation` | Ensures orchestrator delegates instead of writing code directly |
| `delegation_reminder` | Nudges after many consecutive non-delegation tool calls (default: 10) |

### Enabling Watchers

```yaml
# config.yaml
watchers:
  enabled: true
  message_role: user              # How nudges appear: user | assistant | system
  watchers:
    - name: progress
      config:
        max_same_calls: 3         # Flag after 3 identical tool calls
    - name: budget
      config:
        max_steps: 50
        max_sessions: 10
    - name: delegation_reminder
      config:
        threshold: 10             # Nudge after 10 non-delegation calls
    - name: scope
      config:
        avoid_keywords:
          - "refactor everything"
          - "rewrite"
```

The `message_role` setting controls how watcher nudges are injected:
- `user` (default): Appears as a user message - strong nudge, agent must respond
- `assistant`: Appears as a previous assistant thought - softer, agent can continue
- `system`: Appears as system instruction - authoritative guidance

### Watcher Actions

Watchers can return different actions:
- `continue` - Keep going
- `warn` - Log warning but continue
- `pause` - Pause for human review
- `stop` - Stop the orchestrator

## Weave Integration

zwarm integrates with [Weave](https://wandb.ai/site/weave) for tracing and observability.

### Enabling Weave

```bash
# Via environment variable
export WEAVE_PROJECT="your-entity/zwarm"

# Or via config.toml
[weave]
enabled = true
project = "your-entity/zwarm"
```

### What Gets Traced

- Orchestrator `step()` calls with tool inputs/outputs
- Individual adapter calls (`_call_codex`, `_call_claude`)
- Delegation tools (`delegate`, `converse`, `end_session`)
- All tool executions

View traces at: `https://wandb.ai/your-entity/zwarm/weave`

## CLI Reference

### init

Initialize zwarm in a project directory.

```bash
zwarm init [OPTIONS]

Options:
  -w, --working-dir PATH    Working directory [default: .]
  -y, --yes                 Accept defaults, no prompts
  --with-project            Also create zwarm.yaml project config
```

**What it creates:**

1. `config.toml` - User settings (Weave project, adapter preferences, watchers)
2. `.zwarm/` - State directory for sessions and events
3. `zwarm.yaml` (optional) - Project-specific task configuration

**Examples:**

```bash
# Interactive setup with prompts
zwarm init

# Non-interactive with defaults
zwarm init --yes

# Create project config too
zwarm init --with-project

# Initialize in a different directory
zwarm init --working-dir /path/to/project
```

### orchestrate

Start an orchestrator session to delegate tasks.

```bash
zwarm orchestrate [OPTIONS]

Options:
  -t, --task TEXT           Task description
  -f, --task-file PATH      Read task from file
  -c, --config PATH         Config file (YAML)
  --adapter TEXT            Executor adapter override
  --resume                  Resume from previous state
  --set KEY=VALUE           Override config values
```

### exec

Run a single executor directly (for testing). This bypasses the orchestrator entirely and hits the adapter (Codex/Claude) immediately with your task - useful for verifying adapters work before running full orchestration.

```bash
zwarm exec [OPTIONS]

Options:
  -t, --task TEXT           Task to execute
  --adapter TEXT            Adapter to use [default: codex_mcp]
  --model TEXT              Model override
  --mode [sync|async]       Execution mode [default: sync]
```

**Note:** Unlike `orchestrate`, this does NOT use watchers, compaction, state persistence, or multi-step planning. It's a single direct call to the executor.

### status

Show current orchestrator state.

```bash
zwarm status [OPTIONS]

Options:
  --sessions                Show session details
  --tasks                   Show task details
  --json                    Output as JSON
```

### history

Show event history.

```bash
zwarm history [OPTIONS]

Options:
  -n, --limit INTEGER       Number of events [default: 20]
  --session TEXT            Filter by session ID
  --json                    Output as JSON
```

### configs

Manage configuration files.

```bash
zwarm configs list          # List available configs
zwarm configs show NAME     # Show config contents
```

### clean

Clean up zwarm state (useful for starting fresh).

```bash
zwarm clean [OPTIONS]

Options:
  --all                     Remove everything (events, sessions, state)
  --events                  Remove only events
  --sessions                Remove only sessions
  -y, --yes                 Skip confirmation prompt
```

**Examples:**

```bash
# Clean everything and start fresh
zwarm clean --all --yes

# Clean only events log
zwarm clean --events
```

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                     Orchestrator                         │
│  (Plans, delegates, supervises - does NOT write code)   │
├─────────────────────────────────────────────────────────┤
│                    Delegation Tools                      │
│   delegate() | converse() | check_session() | bash()    │
└───────────────┬─────────────────────┬───────────────────┘
                │                     │
        ┌───────▼───────┐     ┌───────▼───────┐
        │  Codex MCP    │     │  Claude Code  │
        │   Adapter     │     │    Adapter    │
        └───────┬───────┘     └───────┬───────┘
                │                     │
        ┌───────▼───────┐     ┌───────▼───────┐
        │    codex      │     │    claude     │
        │  mcp-server   │     │     CLI       │
        └───────────────┘     └───────────────┘
```

### Key Concepts

- **Orchestrator**: Plans and delegates but never writes code directly
- **Executors**: CLI agents (Codex, Claude) that do the actual coding
- **Sessions**: Conversations with executors (sync or async)
- **Watchers**: Trajectory aligners that monitor and intervene

### State Management

All state and config is stored in flat files under `.zwarm/`:

```
.zwarm/
├── config.toml             # Runtime settings (weave, adapter, watchers)
├── state.json              # Current state
├── events.jsonl            # Append-only event log
├── sessions/
│   └── <session-id>/
│       ├── messages.json   # Conversation history
│       └── metadata.json   # Session info
└── orchestrator/
    └── messages.json       # Orchestrator history (for resume)
```

## Development

### Running Tests

```bash
# Run all zwarm tests (68 tests)
uv run pytest src/zwarm/ -v

# Run specific test modules
uv run pytest src/zwarm/core/test_compact.py -v      # Context compaction
uv run pytest src/zwarm/watchers/test_watchers.py -v # Watchers
uv run pytest src/zwarm/adapters/test_codex_mcp.py -v # Codex adapter

# Run integration tests (requires codex CLI)
uv run pytest -m integration
```

### Project Structure

```
zwarm/
├── src/zwarm/
│   ├── adapters/           # Executor adapters
│   │   ├── base.py         # ExecutorAdapter protocol
│   │   ├── codex_mcp.py    # Codex MCP adapter (with token tracking)
│   │   └── claude_code.py  # Claude Code adapter (with token tracking)
│   ├── cli/
│   │   └── main.py         # Typer CLI
│   ├── core/
│   │   ├── compact.py      # Context window compaction (LRU pruning)
│   │   ├── config.py       # Configuration loading
│   │   ├── environment.py  # OrchestratorEnv (progress display)
│   │   ├── models.py       # ConversationSession, Message, Event, etc.
│   │   └── state.py        # Flat-file state management
│   ├── tools/
│   │   └── delegation.py   # delegate, converse, check_session, etc.
│   ├── watchers/
│   │   ├── base.py         # Watcher protocol
│   │   ├── builtin.py      # Built-in watchers (progress, budget, scope, etc.)
│   │   ├── registry.py     # Watcher registration
│   │   └── manager.py      # WatcherManager
│   ├── prompts/
│   │   └── orchestrator.py # Orchestrator system prompt
│   └── orchestrator.py     # Main Orchestrator class
├── configs/                # Example configurations
├── README.md
└── pyproject.toml
```

## Research Context

zwarm is a research platform exploring:

1. **Agent reliability** - Can orchestrators reliably delegate and verify work?
2. **Agent meta-capability** - Can agents effectively use other agents?
3. **Long-running agents** - Can agents run for days, not hours?

See [ZWARM_PLAN.md](ZWARM_PLAN.md) for detailed design documentation.

## License

Research project - see repository license.
