Metadata-Version: 2.4
Name: loom-agents
Version: 0.3.1
Summary: Multi-agent project orchestration system
Author: Jason Bauman
License-Expression: MIT
Requires-Python: >=3.12
Requires-Dist: anthropic>=0.49.0
Requires-Dist: asyncpg>=0.30.0
Requires-Dist: click>=8.1.0
Requires-Dist: fastmcp>=2.0.0
Requires-Dist: jinja2>=3.1.0
Requires-Dist: pathspec>=0.12.0
Requires-Dist: pydantic>=2.5.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: redis[hiredis]>=5.0.0
Requires-Dist: rich>=13.0.0
Provides-Extra: gcp
Requires-Dist: google-cloud-secret-manager>=2.20.0; extra == 'gcp'
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.20.0; extra == 'otel'
Requires-Dist: opentelemetry-exporter-otlp>=1.20.0; extra == 'otel'
Requires-Dist: opentelemetry-sdk>=1.20.0; extra == 'otel'
Description-Content-Type: text/markdown

# Loom

Multi-agent project orchestration system. Postgres is the source of truth, Redis is the cache and event bus, and a [FastMCP](https://github.com/jlowin/fastmcp) server gives Claude Code (or any MCP client) full access to the task graph.

```
pipx install loom-agents
loom init
loom up
# Open Claude Code — Loom tools auto-register via .mcp.json
```

## How It Works

Loom breaks large goals into a dependency graph of tasks, then lets multiple AI agents (or humans) claim, execute, and complete them concurrently. An orchestrator loop monitors the system: sweeping expired claims, retrying failures with exponential backoff, and escalating issues that exceed retry limits.

```
┌─────────────┐     ┌──────────┐     ┌───────────┐
│  Claude Code │────▶│ MCP Server│────▶│ Postgres  │
│  (agent)     │◀────│ (FastMCP) │◀────│ (tasks)   │
└─────────────┘     └──────────┘     └───────────┘
                         │
                    ┌────▼────┐
                    │  Redis   │
                    │ (cache + │
                    │  events) │
                    └─────────┘
```

**Write path:** MCP tool → Postgres (ACID) → Redis cache sync → Redis event publish → dependency resolution.

**Read path:** MCP tool → Redis (fast) → Postgres fallback on cache miss.

## Installation

### From PyPI

```bash
pipx install loom-agents
# or
pip install loom-agents
```

### From source

```bash
git clone https://github.com/MennoAf/agent_loom.git
cd agent_loom
uv sync
```

### Prerequisites

- Python 3.12+
- Docker Desktop (for Postgres + Redis containers)

## Quick Start

```bash
# 1. Set your API key (one-time global setup)
mkdir -p ~/.loom
cat > ~/.loom/config.yaml << 'EOF'
skills:
  api_key: "sk-ant-..."   # Your Anthropic API key
EOF

# 2. Initialize a project
loom init --name my-project

# 3. Start Postgres + Redis
loom up

# 4. Open Claude Code in the same directory
#    Loom MCP tools auto-register via .mcp.json

# 5. Decompose a goal into tasks
loom decompose "Build a REST API with user authentication"

# 6. Check project status
loom status
```

Once initialized, Claude Code agents can use MCP tools like `loom_ready`, `loom_claim`, `loom_done` to work through the task graph autonomously.

### API Key Setup

Loom needs an Anthropic API key for AI-powered skills (decompose, enrichment, escalation). You only need to set this **once** — it applies to all projects:

**Option A: Global config (recommended)** — set once, works everywhere:
```yaml
# ~/.loom/config.yaml
skills:
  api_key: "sk-ant-..."
```

**Option B: Environment variable** — if you already have `ANTHROPIC_API_KEY` in your shell, Loom picks it up automatically. Or set `LOOM_SKILLS_API_KEY` explicitly:
```bash
export ANTHROPIC_API_KEY="sk-ant-..."   # Loom falls back to this
# or
export LOOM_SKILLS_API_KEY="sk-ant-..."  # Takes priority if both are set
```

Priority order: `LOOM_SKILLS_API_KEY` env var > `ANTHROPIC_API_KEY` env var > `~/.loom/config.yaml` > project `.loom/config.yaml`.

## CLI Reference

| Command | Description |
|---------|-------------|
| `loom init [--name NAME]` | Scaffold a new project (config, docker-compose, .mcp.json, AGENTS.md) |
| `loom up` | Start Postgres + Redis, run migrations, create project in DB |
| `loom down` | Stop containers |
| `loom status [TASK_ID]` | Project overview or single task detail |
| `loom create TITLE [--priority p0\|p1\|p2] [--depends-on ID]` | Create a new task |
| `loom claim TASK_ID [--agent NAME]` | Claim a pending task |
| `loom done TASK_ID [--output JSON] [--branch-name BR] [--skip-verify]` | Mark a claimed task as done |
| `loom fail TASK_ID --reason TEXT` | Mark a task as failed |
| `loom heartbeat TASK_ID [--progress TEXT] [--percent N]` | Extend claim TTL with optional progress |
| `loom idea TITLE [--context JSON]` | Capture an idea for the backlog |
| `loom ideas [--drop ID]` | List all ideas, or drop one |
| `loom reset TASK_ID [--clear-output]` | Reset a stuck or failed task back to pending |
| `loom scan [--execute]` | Process completion markers from `.loom/completions/` |
| `loom recover [--execute]` | Classify and recover orphaned tasks (dry-run by default) |
| `loom decompose GOAL [--from FILE] [--epic-id ID]` | Decompose a goal or epic into a task graph using AI |
| `loom graph [--format json\|mermaid\|summary]` | Show the dependency graph |
| `loom workflow list` | List available workflows |
| `loom workflow run NAME [--input key=value]` | Run a workflow |
| `loom workflow resume RUN_ID` | Resume a paused workflow |
| `loom workflow status [RUN_ID]` | Workflow run status or list all runs |
| `loom orchestrate [--once\|--daemon]` | Run the orchestrator (single sweep or continuous) |
| `loom dead-letter [--retry TASK_ID]` | List or retry permanently failed tasks |
| `loom skill list` | List available skills |
| `loom skill run NAME [--input key=value]` | Run a skill with given inputs |
| `loom config show` | Show merged configuration |
| `loom config set KEY VALUE` | Set a project config value (dot notation, e.g. `skills.model`) |
| `loom logs [-n LINES] [-f] [AGENT]` | Show recent log entries, optionally filtered by agent |
| `loom digest [--vault PATH]` | Write daily project digest to Obsidian vault |
| `loom deploy [--project GCP_ID] [--region REGION]` | Deploy to GCP Cloud Run |
| `loom project list` | List all projects |
| `loom project create NAME [-d DESC]` | Create a new project |
| `loom project switch PROJECT_ID` | Switch active project |
| `loom project archive PROJECT_ID` | Archive a project |
| `loom agent create NAME [--role ROLE]` | Create an agent with API key |
| `loom agent list` | List all active agents |
| `loom agent deactivate AGENT_ID` | Deactivate an agent |
| `loom dashboard [--port PORT]` | Open the web dashboard in your browser |

### Decompose Options

The `loom decompose` command turns a goal into a full task graph. Key flags:

| Flag | Description |
|------|-------------|
| `--from FILE` | Read goal from a file instead of the command line |
| `-y, --yes` | Skip confirmation, write directly to DB |
| `--depth N` | Max hierarchy depth (default: 3) |
| `--no-enrich` | Skip context/criteria/complexity enrichment LLM calls |
| `--enrich-concurrency N` | Max parallel enrichment calls (default: 5 from config) |
| `--scan-root PATH` | Scan directory for existing code to inject into the prompt |
| `--no-validate` | Skip codebase validation of generated tasks |
| `--no-merge` | Skip merging of trivially small tasks |
| `--min-complexity N` | Threshold for trivial-task merging (default: 2) |
| `--optimize-parallel` | Run parallelism optimization pass on the task graph |
| `--epic-id ID` | Decompose an existing epic into subtasks (reads title/context as goal) |
| `--parent-epic-id ID` | Link all generated tasks/epics to an existing parent epic |
| `--force-leaf` | Convert childless epics to pending tasks (auto-enabled with `--epic-id`) |

**Codebase-aware mode:** Pass `--scan-root .` to scan your project for existing code. The scanner extracts file paths, classes, functions, and imports from Python source files and injects the summary into the LLM prompt so it avoids generating tasks for code that already exists. After decomposition, the validator marks tasks as pre-completed when they match existing artifacts with high confidence, and repairs dependency edges accordingly.

**Parallel enrichment:** By default, enrichment runs up to 5 tasks concurrently (configurable via `--enrich-concurrency` or the `skills.enrichment_concurrency` config). Within each task, context enrichment runs first, then done-criteria and complexity estimation run in parallel. This reduces enrichment time from ~20 minutes to ~2-3 minutes for a 30-task graph.

## MCP Tools

These tools are available to any MCP client (Claude Code, etc.) when the Loom server is running:

| Tool | Description |
|------|-------------|
| `loom_ready` | List tasks with no open blockers, sorted by priority |
| `loom_claim` | Atomically claim a task (uses `SELECT FOR UPDATE SKIP LOCKED`) |
| `loom_done` | Mark task complete with output, trigger dependency resolution |
| `loom_fail` | Mark task failed, trigger retry or dead letter |
| `loom_escalate` | Flag a task as blocked with escalation message |
| `loom_create` | Create a new task with title, priority, dependencies |
| `loom_status` | Task detail or project overview (includes ready count) |
| `loom_message` | Send a message to another agent |
| `loom_decompose` | Decompose a goal or epic into a task graph using AI |
| `loom_graph` | Return the full dependency graph (JSON or Mermaid) |
| `loom_update` | Update mutable task fields (title, context, priority, deps, status) |
| `loom_heartbeat` | Extend claim TTL with optional progress/percent tracking |
| `loom_reset` | Reset a stuck or failed task back to pending |
| `loom_batch_done` | Mark multiple tasks done at once, with optional dead-letter |
| `loom_verify_paths` | Pre-flight check that file paths in tasks exist on disk |
| `loom_round_summary` | Post-round summary: run tests, snapshot project status |
| `loom_orchestrate` | Run orchestrator tick or check status |
| `loom_workflow` | Run, resume, or check workflow status |
| `loom_dead_letter` | List permanently failed tasks |
| `loom_projects` | List all projects with task count summaries |
| `loom_create_project` | Create a new project |
| `loom_switch_project` | Switch the active project context |
| `loom_archive_project` | Archive a project (soft-delete) |
| `loom_idea` | Capture an idea for the backlog (excluded from orchestration) |
| `loom_recover` | Classify orphaned tasks and optionally recover them |

## Architecture

### Data Model

Tasks form a directed acyclic graph (DAG) with dependencies. Each task has:
- **ID:** `loom-{8 hex chars}` (e.g., `loom-a1b2c3d4`)
- **Status:** `idea` | `pending` → `claimed` → `done` | `failed` | `blocked` | `epic`
- **Priority:** `p0` (critical), `p1` (high), `p2` (normal)
- **Dependencies:** other task IDs that must complete first

### Orchestrator Loop

When running as a daemon (`loom orchestrate --daemon`), the orchestrator periodically:
1. **Sweeps expired claims** — releases tasks held past their TTL
2. **Detects stale agents** — flags claimed tasks with no recent heartbeat
3. **Retries failed tasks** — exponential backoff with jitter, up to `max_retries`
4. **Handles escalations** — runs the `debug_failure` skill with per-task dedup guards and rate limiting (`max_escalations_per_tick`)
5. **Dispatches merges** — detects merge tasks in the ready queue and runs them with a Redis distributed lock
6. **Checks completion** — detects when all tasks are done

### Merge Agent

When an agent completes work in a git worktree and calls `loom_done` with a `branch_name`, Loom automatically creates a p0-priority merge task. The orchestrator detects these merge tasks in the ready queue, acquires a Redis distributed lock to serialize merges, and dispatches the merge executor as a background asyncio task. The merge agent merges the branch into main, runs the test suite, pushes to origin, and cleans up the worktree -- or rolls back and fails the task if anything goes wrong (conflict, test failure, git error). See [docs/merge-agent.md](docs/merge-agent.md) for the full sequence diagram, conflict resolution logic, Redis lock mechanism, and troubleshooting guide.

### Orchestrator Resilience

If the orchestrator session dies (context compaction, timeout, crash), in-progress work is not lost:

1. **Completion markers:** `loom done` and `loom fail` CLI commands write JSON markers to `.loom/completions/` as a filesystem fallback. These survive regardless of DB/Redis state.
2. **CLI self-report:** Subagents spawned via the Task tool don't have MCP access, but they can self-report via CLI commands (`loom done`, `loom fail`, `loom heartbeat`) with full parity to MCP tools.
3. **Recovery protocol:** `loom recover` (CLI) or `loom_recover` (MCP) classifies all claimed tasks:
   - **Marker tasks** — have a completion marker → auto-complete
   - **Stale tasks** — no heartbeat for 3x heartbeat interval → re-queue
   - **Expired tasks** — claim TTL passed → re-queue
   - **Active tasks** — recent heartbeat → leave alone

```bash
# Dry-run: see what would be recovered
loom recover

# Apply the recovery plan
loom recover --execute

# Process only filesystem markers
loom scan --execute
```

The `generate_agent_prompt()` helper in `loom.cli` bakes self-reporting instructions into subagent prompts automatically.

### Module Contracts

| Module | Responsibility |
|--------|---------------|
| `graph/store.py` | Only writer to Postgres for task data |
| `graph/cache.py` | Only reader from Redis; falls back to `store.py` |
| `bus/channels.py` | All Redis key patterns (no string literals elsewhere) |
| `mcp/tools.py` | Thin coordinators only (each tool ≤15 lines) |
| `db/migrations/` | Append-only — never modify existing migration files |

## Configuration

Loom loads configuration in three layers (later layers override earlier ones):

1. **Global:** `~/.loom/config.yaml`
2. **Project:** `.loom/config.yaml`
3. **Environment variables**

### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `LOOM_DATABASE_URL` | Postgres connection string | `postgresql://loom:loom_local@localhost:5432/loom` |
| `LOOM_DATABASE_POOL_MIN` | Minimum pool connections | `2` |
| `LOOM_DATABASE_POOL_MAX` | Maximum pool connections | `10` |
| `LOOM_REDIS_URL` | Redis connection string | `redis://localhost:6379` |
| `LOOM_MCP_PORT` | MCP server port | `8765` |
| `LOOM_HEALTH_PORT` | Health check endpoint port | `8765` |
| `LOOM_LOG_LEVEL` | Logging level | `INFO` |
| `LOOM_PROJECT_ID` | Project UUID | (from config) |
| `LOOM_SKILLS_API_KEY` | Anthropic API key for AI skills | (falls back to `ANTHROPIC_API_KEY`) |
| `LOOM_SKILLS_MODEL` | Model for AI skills | `claude-sonnet-4-6` |
| `LOOM_ENRICHMENT_CONCURRENCY` | Max parallel tasks during decompose enrichment | `5` |
| `LOOM_CLAIM_TTL` | Claim timeout in seconds | `600` |
| `LOOM_MAX_RETRIES` | Max retry attempts before dead letter | `3` |
| `LOOM_RETRY_BASE_DELAY` | Base retry delay in seconds | `60` |
| `LOOM_RETRY_MAX_DELAY` | Max retry delay in seconds | `3600` |
| `LOOM_ENABLE_ESCALATION` | Enable AI-generated fix tasks on failure | `true` |
| `LOOM_SLACK_WEBHOOK_URL` | Slack webhook for escalation notifications | |
| `LOOM_OBSIDIAN_VAULT_PATH` | Path to Obsidian vault for daily digests | |
| `LOOM_GCP_PROJECT` | GCP project ID (enables Secret Manager) | |
| `LOOM_ENV` | Set to `production` for JSON logging | |
| `LOOM_AUTH_ENABLED` | Enable agent authentication & RBAC (`true`/`false`) | `false` |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | OpenTelemetry collector endpoint | |

### Example config.yaml

```yaml
loom:
  project_name: my-project
  project_id: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

database:
  url: postgresql://loom:loom_local@localhost:5432/loom
  pool_min_size: 2
  pool_max_size: 10

redis:
  url: redis://localhost:6379

skills:
  api_key: ""              # Anthropic API key (or use ANTHROPIC_API_KEY env var)
  model: claude-sonnet-4-6
  max_tokens: 8192
  enrichment_concurrency: 5

orchestration:
  claim_ttl_seconds: 600
  max_retries: 3
  sweep_interval_seconds: 60
  enable_escalation: true
  escalation_dedup_ttl_seconds: 600
  max_escalations_per_tick: 5

integrations:
  slack_webhook_url: ""
  obsidian_vault_path: ""

auth:
  enabled: false

dashboard:
  enabled: true

logging:
  level: INFO
```

## Features

### Idea Backlog

When the task queue is empty, ideas capture "what should we work on next?" without cluttering the active task graph. Ideas flow through a simple funnel:

```
idea → decompose → tasks → done
```

Ideas are tasks with `status: "idea"` — fully excluded from orchestration, the ready queue, sweeper, and claiming. They exist purely as a backlog.

```bash
# Capture an idea (one-liner)
loom idea "Add webhook support for external integrations"
loom idea "What if we gave Warp a phone number" --context '{"notes": "for SMS alerts"}'

# List the backlog
loom ideas

# Drop one you don't want
loom ideas --drop loom-a1b2c3d4

# Promote an idea by decomposing it — automatically converts idea → epic
loom decompose --epic-id loom-a1b2c3d4
```

MCP tools work the same way: `loom_idea(title, context)` to capture, `loom_status(filter="idea")` to list, and `loom_decompose(epic_id="loom-xxx")` auto-promotes ideas to epics before decomposing. When `loom_ready` returns an empty list and ideas exist, it returns a hint pointing agents to the backlog.

### Skills

Loom includes 10 built-in AI skills that agents can invoke:

| Skill | Purpose |
|-------|---------|
| `decompose_project` | Break a goal into a hierarchical task graph |
| `write_task_context` | Generate rich context for a task |
| `define_done` | Create testable acceptance criteria |
| `estimate_complexity` | Estimate task complexity and effort |
| `debug_failure` | Diagnose failures and propose fixes |
| `review_output` | Review output against acceptance criteria |
| `generate_test_plan` | Generate a test plan for a task |
| `write_spec` | Write a technical specification |
| `identify_parallelism` | Find groups of tasks that can run in parallel |
| `summarize_graph` | Generate a project status narrative |

Custom skills can be added as `.md` files in `.loom/skills/`.

### Workflows

Workflows chain skills and actions into multi-step pipelines:

| Workflow | Steps |
|----------|-------|
| `ship_feature` | spec → decompose → confirm → write tasks |
| `debug_and_fix` | diagnose → confirm → create fix tasks |
| `audit_codebase` | decompose → find parallelism → confirm → write tasks → summarize |
| `deploy_to_prod` | generate test plan → confirm → write deploy tasks |

Custom workflows can be added as `.yaml` files in `.loom/workflows/`.

### Multi-project Support

Loom supports multiple projects per instance. Tasks are isolated per project — each project has its own task graph, ready queue, and cache namespace.

```bash
# Create and manage projects
loom project create "my-api" -d "REST API backend"
loom project create "my-frontend" -d "React frontend"
loom project list
loom project switch <PROJECT_ID>
loom project archive <PROJECT_ID>
```

MCP tools also support project management: `loom_projects`, `loom_create_project`, `loom_switch_project`, `loom_archive_project`.

### Web Dashboard

Loom includes a built-in web dashboard served on the same port as the health check endpoint (default 8765). No additional dependencies or build steps required.

```bash
loom dashboard    # Opens http://localhost:8765/dashboard in your browser
```

Features:
- **Task Board:** Kanban-style columns (Pending, Claimed, Done, Failed, Blocked) with priority badges
- **Dependency Graph:** Mermaid.js-powered DAG with status-colored nodes
- **Project Overview:** All projects with progress bars and task counts
- **Real-time Updates:** Server-Sent Events (SSE) push task state changes to the browser
- **Dark Mode:** Follows system preference via `prefers-color-scheme`

The dashboard uses HTMX + Jinja2 server-rendered HTML — no JavaScript build step, no React, no npm.

### Agent Auth & RBAC

Optional role-based access control for MCP tools. Disabled by default for backward compatibility.

```bash
# Enable auth
export LOOM_AUTH_ENABLED=true

# Create agents with roles
loom agent create my-worker --role worker
# → prints API key (shown once, store securely)

loom agent create my-lead --role lead
loom agent list
loom agent deactivate <AGENT_ID>
```

**Roles and permissions:**

| Role | Allowed Tools |
|------|---------------|
| `readonly` | `loom_ready`, `loom_status`, `loom_graph`, `loom_projects` |
| `worker` | readonly + `loom_claim`, `loom_done`, `loom_fail`, `loom_heartbeat`, `loom_message` |
| `lead` | worker + `loom_create`, `loom_update`, `loom_escalate`, `loom_decompose`, `loom_workflow`, `loom_orchestrate` |
| `admin` | all tools |

Agents can be scoped to specific projects — a worker assigned to project A cannot access tasks in project B.

### Observability

#### Metrics

Prometheus-compatible metrics are served at `/metrics` on the health check port:

```bash
curl http://localhost:8765/metrics
```

Available metrics: `tasks_created_total`, `tasks_completed_total`, `tasks_failed_total`, `claims_active`, `cache_hits_total`, `cache_misses_total`, `circuit_breaker_state`, `mcp_tool_duration_seconds`, `db_query_duration_seconds`.

#### Tracing

OpenTelemetry tracing is supported as an optional dependency:

```bash
pip install loom-agents[otel]
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
```

When configured, MCP tool calls and database queries emit spans with tool name, task ID, and project ID attributes. When not configured, tracing decorators are transparent noops with zero overhead.

#### Redis Circuit Breaker

The Redis cache layer includes a circuit breaker that automatically falls back to Postgres when Redis is experiencing issues. The circuit opens after 5 consecutive failures, enters half-open state after 30 seconds, and closes again on a successful probe.

## Deployment

### Local (default)

```bash
loom init
loom up        # Starts Postgres + Redis in Docker
# MCP server runs via Claude Code's .mcp.json auto-registration
```

### Docker Compose (full stack)

```bash
loom init
docker compose -f docker-compose.loom.yml --profile mcp up
```

The `--profile mcp` flag starts the Loom MCP server container alongside Postgres and Redis.

### GCP Cloud Run

```bash
# Build and deploy
loom deploy --project my-gcp-project --region us-central1

# Or manually
docker build -t gcr.io/my-project/loom-mcp .
docker push gcr.io/my-project/loom-mcp
gcloud run deploy loom-mcp --image gcr.io/my-project/loom-mcp --region us-central1
```

Cloud Run features:
- **Cloud SQL Auth Proxy:** Set `CLOUD_SQL_CONNECTION_NAME` for Unix socket connections
- **Memorystore TLS:** Set `LOOM_REDIS_TLS=true` for encrypted Redis
- **Secret Manager:** Set `LOOM_GCP_PROJECT` to auto-load secrets (`loom-database-url`, `loom-redis-url`, `loom-skills-api-key`)
- **Health checks:** `/health` endpoint on port 8765 returns `{"status": "healthy"}` with Postgres and Redis status
- **Structured logging:** JSON output when `K_SERVICE` (Cloud Run) or `LOOM_ENV=production` is set
- **Graceful shutdown:** SIGTERM handler drains connections cleanly

### CI/CD

GitHub Actions workflows are included:
- **`.github/workflows/ci.yml`** — Tests on every PR and push to main
- **`.github/workflows/publish.yml`** — Publishes to PyPI on `v*` tags (trusted publishing via OIDC)
- **`cloudbuild.yaml`** — GCP Cloud Build: build, push, deploy to Cloud Run

## Testing

Tests use [testcontainers](https://testcontainers-python.readthedocs.io/) to spin up real Postgres 16 and Redis 7 instances. Docker Desktop must be running.

```bash
# Run all tests
uv run pytest tests/ -v

# Run a single test
uv run pytest tests/integration/test_e2e.py::test_full_agent_workflow -v
```

## Contributing

```bash
git clone https://github.com/MennoAf/agent_loom.git
cd agent_loom
uv sync
uv run pytest tests/ -v
```

## License

MIT
