# Loom

Multi-agent project orchestration. Break a goal into a dependency graph of tasks, then let multiple AI agents work through them in parallel.

Postgres is the source of truth, Redis is the cache and event bus, a [FastMCP](https://github.com/jlowin/fastmcp) server gives Claude Code (or any MCP client) full access to the task graph.

```bash
# Install from the repo (requires access)
pipx install git+ssh://git@github.com/MennoAf/agent_loom.git

loom init
loom up
# Open Claude Code — Loom tools auto-register via .mcp.json
```

## How It Works

```
decompose → claim → work → done → unblock dependents
```

Loom decomposes large goals into a DAG of tasks, then lets agents (or humans) claim, execute, and complete them concurrently. An orchestrator monitors the system: sweeping expired claims, retrying failures, escalating blockers, and dispatching merges.

```
┌─────────────┐     ┌──────────┐     ┌───────────┐
│  Claude Code │────▶│ MCP Server│────▶│ Postgres  │
│  (agent)     │◀────│ (FastMCP) │◀────│ (tasks)   │
└─────────────┘     └──────────┘     └───────────┘
                         │
                    ┌────▼────┐
                    │  Redis   │
                    │ (cache + │
                    │  events) │
                    └─────────┘
```

**Write path:** MCP tool → Postgres (ACID) → Redis cache sync → event publish → dependency resolution.

**Read path:** MCP tool → Redis (fast) → Postgres fallback on cache miss.

## Quick Start

```bash
# 1. Install (from the repo — requires access)
pipx install git+ssh://git@github.com/MennoAf/agent_loom.git

# Or install from a local clone:
#   git clone git@github.com:MennoAf/agent_loom.git && pipx install ./agent_loom

# 2. Initialize a project
cd my-project
loom init --name my-project

# 3. Start services
loom up

# 4. Verify everything works
loom doctor

# 5. Open Claude Code in the same directory
#    Loom MCP tools auto-register via .mcp.json

# 6. Decompose a goal into tasks (auto-scans your codebase)
loom decompose "Build a REST API with user authentication"

# 7. Check project status
loom status
```

### API Key Setup

Loom needs an LLM API key for AI-powered skills (decompose, enrichment, escalation). Set it once — it applies to all projects.

**Anthropic (default):**
```bash
export ANTHROPIC_API_KEY="sk-ant-..."
```

**OpenAI / OpenAI-compatible (Gemini, Ollama, vLLM, LiteLLM):**
```bash
export LOOM_SKILLS_PROVIDER=openai
export OPENAI_API_KEY="sk-..."

# For local models or custom endpoints:
export LOOM_SKILLS_BASE_URL=http://localhost:11434/v1   # Ollama example
```

Or set it in `~/.loom/config.yaml` for global config:
```yaml
skills:
  provider: anthropic         # or "openai"
  api_key: "sk-ant-..."
  base_url: ""                # custom endpoint for OpenAI-compatible APIs
```

Priority order: `LOOM_SKILLS_API_KEY` > provider-specific env var (`ANTHROPIC_API_KEY` or `OPENAI_API_KEY`) > `~/.loom/config.yaml` > project `.loom/config.yaml`.

### Multi-LLM Provider Support

Loom supports any LLM accessible via the Anthropic or OpenAI API format:

| Provider | Config |
|----------|--------|
| **Anthropic** (default) | `ANTHROPIC_API_KEY` — Claude models |
| **OpenAI** | `LOOM_SKILLS_PROVIDER=openai` + `OPENAI_API_KEY` |
| **Gemini** | `LOOM_SKILLS_PROVIDER=openai` + `OPENAI_API_KEY` + `LOOM_SKILLS_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/` |
| **Ollama** | `LOOM_SKILLS_PROVIDER=openai` + `LOOM_SKILLS_BASE_URL=http://localhost:11434/v1` |
| **vLLM / LiteLLM** | `LOOM_SKILLS_PROVIDER=openai` + `LOOM_SKILLS_BASE_URL=http://localhost:8000/v1` |

Install the OpenAI SDK when using non-Anthropic providers:
```bash
pipx inject loom-agents openai
```

Individual skills can override the global provider via YAML frontmatter:
```yaml
# .loom/skills/my-skill.md
---
name: my-skill
provider: openai
model: gpt-4o
---
```

## CLI Reference

| Command | Description |
|---------|-------------|
| `loom init [--name NAME]` | Scaffold a new project (config, docker-compose, .mcp.json, AGENTS.md) |
| `loom up` | Start Postgres + Redis, run migrations, create project in DB |
| `loom down` | Stop containers |
| `loom doctor` | Check system health (Docker, Postgres, Redis, config, project) |
| `loom status [TASK_ID]` | Project overview or single task detail |
| `loom create TITLE [--suggest-deps] [--depends-on ID]` | Create a task (with optional dependency suggestions) |
| `loom claim TASK_ID [--agent NAME]` | Claim a pending task |
| `loom done TASK_ID [--output JSON] [--branch-name BR]` | Mark a claimed task as done |
| `loom fail TASK_ID --reason TEXT` | Mark a task as failed |
| `loom heartbeat TASK_ID [--progress TEXT] [--percent N]` | Extend claim TTL with optional progress |
| `loom reset TASK_ID [--clear-output]` | Reset a stuck or failed task back to pending |
| `loom idea TITLE [--context JSON]` | Capture an idea for the backlog |
| `loom ideas [--drop ID]` | List all ideas, or drop one |
| `loom decompose GOAL [options]` | Decompose a goal into a task graph using AI |
| `loom graph [--format json\|mermaid\|summary]` | Show the dependency graph |
| `loom scan [--execute]` | Process completion markers from `.loom/completions/` |
| `loom recover [--execute]` | Classify and recover orphaned tasks (dry-run by default) |
| `loom orchestrate [--once\|--daemon]` | Run the orchestrator (single sweep or continuous) |
| `loom dead-letter [--retry TASK_ID]` | List or retry permanently failed tasks |
| `loom workflow list\|run\|resume\|status` | Manage workflows |
| `loom project list\|create\|switch\|archive` | Manage projects |
| `loom agent create\|list\|deactivate` | Manage agent auth (RBAC) |
| `loom skill list\|run` | List or run skills |
| `loom config show\|set` | View or update configuration |
| `loom dashboard [--port PORT]` | Open the web dashboard |
| `loom digest [--vault PATH]` | Write daily digest to Obsidian vault |
| `loom deploy [--project GCP_ID]` | Deploy to GCP Cloud Run |

### Decompose

`loom decompose` turns a goal into a full task graph. It auto-scans your codebase by default to avoid generating tasks for code that already exists.

| Flag | Description |
|------|-------------|
| `--from FILE` | Read goal from a file instead of the command line |
| `-y, --yes` | Skip confirmation, write directly to DB |
| `--depth N` | Max hierarchy depth (default: 3) |
| `--no-enrich` | Skip context/criteria/complexity enrichment |
| `--enrich-concurrency N` | Max parallel enrichment calls (default: 5) |
| `--no-scan` | Disable automatic codebase scanning |
| `--scan-root PATH` | Scan a specific directory (default: current directory) |
| `--no-validate` | Skip codebase validation of generated tasks |
| `--no-merge` | Skip merging of trivially small tasks |
| `--min-complexity N` | Threshold for trivial-task merging (default: 2) |
| `--optimize-parallel` | Run parallelism optimization pass |
| `--epic-id ID` | Decompose an existing epic into subtasks |
| `--force-leaf` | Convert childless epics to pending tasks |

The decompose pipeline:
1. **LLM generates** a hierarchical task graph from your goal
2. **Auto-fix paths** — stale file references corrected against real file tree
3. **Codebase validation** — marks tasks as pre-completed when code already exists
4. **Trivial merge** — folds tiny tasks into parents
5. **File-affinity merge** — groups tasks sharing files to prevent merge conflicts
6. **Enrichment** — adds context, acceptance criteria, and complexity scores (parallel, ~2-3 min for 30 tasks)

### Doctor

`loom doctor` runs 9 checks to verify your setup:

```
$ loom doctor
Loom Doctor

  ✓ Docker Desktop running
  ✓ Loom initialized (.loom/config.yaml)
  ✓ docker-compose.loom.yml exists
  ✓ Postgres connectable
  ✓ Redis connectable
  ✓ Migrations up to date
  ! API key not configured (optional — needed for decompose/skills)
  ✓ .mcp.json has loom entry
  ✓ Project exists in database

Some checks failed. See suggestions above.
```

### Dependency Suggestions

When creating tasks, Loom can suggest related existing tasks as potential dependencies:

```bash
loom create "Add user authentication" --suggest-deps
# Created task loom-a1b2c3d4: Add user authentication
#
# Suggested dependencies:
#   loom-e5f6g7h8: Set up database models (score: 0.45 — title overlap: 0.40, file overlap: 0.50)
#   loom-i9j0k1l2: Create user API endpoints (score: 0.35 — title overlap: 0.30, file overlap: 0.40)
#
# Use --depends-on to add dependencies, e.g.:
#   loom create "Add user authentication" --depends-on loom-e5f6g7h8
```

Also available via MCP: `loom_create(title="...", suggest_deps=True)`.

## MCP Tools

Available to any MCP client when the Loom server is running:

| Tool | Description |
|------|-------------|
| `loom_ready` | List tasks with no open blockers, sorted by priority |
| `loom_claim` | Atomically claim a task (`SELECT FOR UPDATE SKIP LOCKED`) |
| `loom_done` | Mark complete with output, trigger dependency resolution |
| `loom_fail` | Mark failed, trigger retry or dead letter |
| `loom_escalate` | Flag a task as blocked with escalation message |
| `loom_create` | Create a task (with optional `suggest_deps` for dependency suggestions) |
| `loom_status` | Task detail, filtered list, or project overview |
| `loom_update` | Update task fields (title, context, priority, deps, status) |
| `loom_message` | Send a message to another agent |
| `loom_decompose` | Decompose a goal or epic into a task graph |
| `loom_graph` | Full dependency graph (JSON, Mermaid, or summary) |
| `loom_heartbeat` | Extend claim TTL with progress tracking |
| `loom_reset` | Reset a stuck/failed task back to pending |
| `loom_batch_done` | Mark multiple tasks done/dead-lettered at once |
| `loom_batch_claim` | Atomically claim multiple tasks at once |
| `loom_verify_paths` | Pre-flight check that file paths in tasks exist |
| `loom_round_summary` | Post-round summary: run tests, snapshot status |
| `loom_orchestrate` | Plan execution waves based on dependency graph and file overlap |
| `loom_orchestrate_tick` | Run a single orchestrator sweep (expire claims, retry, escalate) |
| `loom_workflow` | Run, resume, or check workflow status |
| `loom_dead_letter` | List permanently failed tasks |
| `loom_projects` | List all projects |
| `loom_create_project` | Create a new project |
| `loom_switch_project` | Switch active project |
| `loom_archive_project` | Archive a project |
| `loom_idea` | Capture an idea for the backlog |
| `loom_issue` | File an issue (optionally targeting another project) |
| `loom_issues` | List open issues for the current project |
| `loom_recover` | Classify and recover orphaned tasks |

## Architecture

### Data Model

Tasks form a directed acyclic graph (DAG) with dependencies:

- **ID:** `loom-{8 hex chars}` (e.g., `loom-a1b2c3d4`)
- **Status:** `idea` → `pending` → `claimed` → `done` | `failed` | `blocked` | `epic`
- **Priority:** `p0` (critical), `p1` (high), `p2` (normal)
- **Dependencies:** task IDs that must complete first

### Orchestrator

When running as a daemon (`loom orchestrate --daemon`), the orchestrator periodically:

1. **Sweeps expired claims** — releases tasks held past their TTL
2. **Detects stale agents** — flags claimed tasks with no recent heartbeat
3. **Retries failed tasks** — exponential backoff with jitter, up to `max_retries`
4. **Handles escalations** — runs `debug_failure` with per-task dedup guards and rate limiting
5. **Dispatches merges** — detects merge tasks and runs them with a Redis distributed lock
6. **Checks completion** — detects when all tasks are done

### Merge Agent

When an agent calls `loom_done` with a `branch_name`, Loom creates a p0 merge task. The orchestrator acquires a Redis lock to serialize merges and dispatches the merge executor: merge branch → run tests → push → cleanup (or rollback on failure).

### Resilience

If the orchestrator dies, work is not lost:

1. **Completion markers** — `loom done` / `loom fail` write JSON to `.loom/completions/` as a filesystem fallback
2. **CLI self-report** — subagents without MCP access report via CLI (`loom done`, `loom fail`, `loom heartbeat`)
3. **Recovery** — `loom recover` classifies claimed tasks (marker → complete, stale → re-queue, expired → re-queue, active → leave)

```bash
loom recover              # dry-run
loom recover --execute    # apply recovery plan
loom scan --execute       # process filesystem markers only
```

### Module Contracts

| Module | Responsibility |
|--------|---------------|
| `graph/store.py` | Only writer to Postgres for task data |
| `graph/cache.py` | Only reader from Redis; falls back to `store.py` |
| `bus/channels.py` | All Redis key patterns (no string literals elsewhere) |
| `mcp/tools.py` | Thin coordinators (each tool ≤15 lines) |
| `skills/providers.py` | LLM client abstraction (Anthropic + OpenAI protocol) |
| `db/migrations/` | Append-only — never modify existing files |

## Features

### Idea Backlog

Ideas capture "what should we work on next?" without cluttering the active task graph:

```bash
loom idea "Add webhook support"
loom ideas                             # list backlog
loom ideas --drop loom-a1b2c3d4        # remove one
loom decompose --epic-id loom-a1b2c3d4 # promote → decompose
```

### Skills

10 built-in AI skills (works with any configured provider):

| Skill | Purpose |
|-------|---------|
| `decompose_project` | Break a goal into a hierarchical task graph |
| `write_task_context` | Generate rich context for a task |
| `define_done` | Create testable acceptance criteria |
| `estimate_complexity` | Estimate task complexity and effort |
| `debug_failure` | Diagnose failures and propose fixes |
| `review_output` | Review output against acceptance criteria |
| `generate_test_plan` | Generate a test plan for a task |
| `write_spec` | Write a technical specification |
| `identify_parallelism` | Find tasks that can run in parallel |
| `summarize_graph` | Generate a project status narrative |

Custom skills: add `.md` files to `.loom/skills/` with YAML frontmatter. Skills can specify their own `provider` and `model` to use a different LLM than the global default.

### Workflows

Multi-step pipelines chaining skills and actions:

| Workflow | Steps |
|----------|-------|
| `ship_feature` | spec → decompose → confirm → write tasks |
| `debug_and_fix` | diagnose → confirm → create fix tasks |
| `audit_codebase` | decompose → find parallelism → confirm → write tasks → summarize |
| `deploy_to_prod` | generate test plan → confirm → write deploy tasks |

Custom workflows: add `.yaml` files to `.loom/workflows/`.

### Multi-Project Support

Tasks are isolated per project — each has its own task graph, ready queue, and cache namespace.

```bash
loom project create "my-api" -d "REST API backend"
loom project list
loom project switch <PROJECT_ID>
loom project archive <PROJECT_ID>
```

### Web Dashboard

Built-in HTMX + Jinja2 dashboard — no npm, no build step:

```bash
loom dashboard    # Opens http://localhost:8765/dashboard
```

- Kanban task board with priority badges
- Mermaid.js dependency graph with status-colored nodes
- Project overview with progress bars
- Real-time SSE updates
- Dark mode (follows system preference)

### Agent Auth & RBAC

Optional role-based access control (disabled by default):

```bash
export LOOM_AUTH_ENABLED=true
loom agent create my-worker --role worker   # prints API key
```

| Role | Access |
|------|--------|
| `readonly` | `loom_ready`, `loom_status`, `loom_graph`, `loom_projects` |
| `worker` | + `loom_claim`, `loom_done`, `loom_fail`, `loom_heartbeat`, `loom_message` |
| `lead` | + `loom_create`, `loom_update`, `loom_escalate`, `loom_decompose`, `loom_workflow` |
| `admin` | all tools |

### Observability

**Metrics:** Prometheus-compatible at `/metrics` — tasks created/completed/failed, cache hits/misses, circuit breaker state, tool duration.

**Tracing:** Optional OpenTelemetry support:
```bash
pip install loom-agents[otel]
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
```

**Circuit breaker:** Redis cache auto-falls back to Postgres after 5 consecutive failures, recovers after 30s.

## Configuration

Three layers (later overrides earlier): `~/.loom/config.yaml` → `.loom/config.yaml` → environment variables.

### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `LOOM_SKILLS_PROVIDER` | LLM provider (`anthropic` or `openai`) | `anthropic` |
| `LOOM_SKILLS_BASE_URL` | Custom API endpoint (for Ollama, vLLM, etc.) | |
| `LOOM_SKILLS_API_KEY` | API key for LLM skills | (falls back to provider-specific key) |
| `LOOM_SKILLS_MODEL` | Model for AI skills | `claude-sonnet-4-6` |
| `LOOM_DATABASE_URL` | Postgres connection string | `postgresql://loom:loom_local@localhost:5432/loom` |
| `LOOM_REDIS_URL` | Redis connection string | `redis://localhost:6379` |
| `LOOM_MCP_PORT` | MCP server port | `8765` |
| `LOOM_LOG_LEVEL` | Logging level | `INFO` |
| `LOOM_PROJECT_ID` | Project UUID | (from config) |
| `LOOM_ENRICHMENT_CONCURRENCY` | Max parallel enrichment tasks | `5` |
| `LOOM_CLAIM_TTL` | Claim timeout in seconds | `1800` |
| `LOOM_MAX_RETRIES` | Max retry attempts before dead letter | `3` |
| `LOOM_ENABLE_ESCALATION` | Enable AI-generated fix tasks on failure | `true` |
| `LOOM_SLACK_WEBHOOK_URL` | Slack webhook for notifications | |
| `LOOM_OBSIDIAN_VAULT_PATH` | Obsidian vault for daily digests | |
| `LOOM_AUTH_ENABLED` | Enable agent RBAC | `false` |
| `LOOM_GCP_PROJECT` | GCP project (enables Secret Manager) | |

### Example config.yaml

```yaml
loom:
  project_name: my-project
  project_id: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

database:
  url: postgresql://loom:loom_local@localhost:5432/loom

redis:
  url: redis://localhost:6379

skills:
  provider: anthropic         # or "openai"
  api_key: ""                 # or use env var
  base_url: ""                # for OpenAI-compatible endpoints
  model: claude-sonnet-4-6
  max_tokens: 8192
  enrichment_concurrency: 5

orchestration:
  claim_ttl_seconds: 1800
  max_retries: 3
  enable_escalation: true

integrations:
  slack_webhook_url: ""
  obsidian_vault_path: ""
```

## Deployment

### Local (default)

```bash
loom init && loom up
# MCP server runs via Claude Code's .mcp.json auto-registration
```

### Docker Compose (full stack)

```bash
docker compose -f docker-compose.loom.yml --profile mcp up
```

### GCP Cloud Run

```bash
loom deploy --project my-gcp-project --region us-central1
```

Supports Cloud SQL Auth Proxy, Memorystore TLS, Secret Manager, structured logging, and graceful shutdown. See the deploy command help for details.

## Testing

1100+ tests using [testcontainers](https://testcontainers-python.readthedocs.io/) (real Postgres 16 + Redis 7). Docker Desktop required.

```bash
uv run pytest tests/ -v                    # all tests
uv run pytest tests/ -v -m "not live"      # skip tests needing real API keys
```

## Development

```bash
git clone git@github.com:MennoAf/agent_loom.git
cd agent_loom
uv sync
uv run pytest tests/ -v
```

## License

MIT
