# Migration Guide: Multi-Repo Support

**Version**: 0.2.0+
**Date**: February 2026

## Overview

Starting with version 0.2.0, Clew supports indexing multiple repositories in a shared graph. This requires migrating the database schema to include `repo_id` on all nodes.

## What Changed

### Schema Changes
- **Repository nodes**: Now use `id` as primary key instead of `url`
- **File/CodeBlock nodes**: Added `repo_id` property for filtering
- **Constraints**: Composite uniqueness on `(repo_id, path)` instead of just `path`

### Identity Stability
- Repository IDs are now deterministic (SHA-256 hash of URL)
- Re-ingesting the same repo updates existing data (no duplicates)

## Migration Paths

### Option 1: Fresh Start (Recommended for Alpha Users)

**Best for**: Small datasets, test environments

```bash
# 1. Stop services
docker-compose down

# 2. Delete Neo4j data
docker volume rm clew-engine_neo4j-data

# 3. Start services (schema auto-created)
docker-compose up -d

# 4. Re-ingest repositories
docker-compose run --rm ingestion python -m src.ingest <repo-url>
```

**Pros**: Clean, guaranteed to work
**Cons**: Loses existing data

---

### Option 2: In-Place Migration (Preserve Data)

**Best for**: Production deployments with existing data

#### Step 1: Backup Data

```bash
# Backup Neo4j database
docker exec clew-neo4j neo4j-admin database dump neo4j --to-path=/backups
docker cp clew-neo4j:/backups ./neo4j-backup/
```

#### Step 2: Apply Migration

The `GraphStore` class will automatically create new constraints on startup. You need to migrate existing data to have `repo_id` fields.

**Run Migration Script**:

```bash
# Install dependencies
pip install neo4j

# Set environment variables
export NEO4J_URI=bolt://localhost:7687
export NEO4J_USER=neo4j
export NEO4J_PASSWORD=password

# Run migration
python scripts/migrate_repo_id.py
```

**Or manually via Cypher**:

```cypher
// 1. Add repo_id to Repository nodes (deterministic hash)
MATCH (r:Repository)
WHERE r.id IS NULL AND r.url IS NOT NULL
WITH r, substring(toString(randomUUID()), 0, 16) as id
SET r.id = id
RETURN r.url, r.id;

// 2. Propagate repo_id to File nodes
MATCH (r:Repository)-[:CONTAINS]->(f:File)
WHERE f.repo_id IS NULL
SET f.repo_id = r.id
RETURN count(f) as migrated_files;

// 3. Propagate repo_id to CodeBlock nodes
MATCH (f:File)-[:DEFINES]->(c:CodeBlock)
WHERE c.repo_id IS NULL
SET c.repo_id = f.repo_id
RETURN count(c) as migrated_codeblocks;

// 4. Propagate to Module and Function nodes
MATCH (f:File)-[:IMPORTS]->(m:Module)
WHERE m.repo_id IS NULL
SET m.repo_id = f.repo_id;

MATCH (f:File)-[:CALLS]->(fn:Function)
WHERE fn.repo_id IS NULL
SET fn.repo_id = f.repo_id;
```

#### Step 3: Verify Migration

```bash
# Check that all nodes have repo_id
docker exec clew-neo4j cypher-shell -u neo4j -p password \
  "MATCH (f:File) WHERE f.repo_id IS NULL RETURN count(f);"

# Should return 0
```

#### Step 4: Restart Services

```bash
# Restart to apply new schema
docker-compose restart

# Check logs for "Neo4j schema migrations completed successfully"
docker-compose logs api | grep schema
```

---

## Verification Checklist

After migration, verify everything works:

- [ ] **Constraints Exist**
  ```bash
  docker exec clew-neo4j cypher-shell -u neo4j -p password "SHOW CONSTRAINTS;"
  ```
  Expected: `file_repo_path_unique`, `codeblock_unique`

- [ ] **Indexes Exist**
  ```bash
  docker exec clew-neo4j cypher-shell -u neo4j -p password "SHOW INDEXES;"
  ```
  Expected: `file_repo_id`, `codeblock_repo_id`, `module_repo_id`, `function_repo_id`

- [ ] **All Nodes Have repo_id**
  ```cypher
  MATCH (n) WHERE n:File OR n:CodeBlock OR n:Module OR n:Function
  WHERE n.repo_id IS NULL
  RETURN count(n) as missing_repo_id;
  ```
  Expected: `0`

- [ ] **Search Works**
  ```bash
  curl http://localhost:8000/api/v1/search -d '{"query": "test", "limit": 5}'
  ```

- [ ] **Re-ingestion Doesn't Create Duplicates**
  ```bash
  # Count files before
  docker exec clew-neo4j cypher-shell -u neo4j -p password \
    "MATCH (f:File) RETURN count(f);"

  # Re-ingest same repo
  docker-compose run --rm ingestion python -m src.ingest <same-repo-url>

  # Count files after (should be same)
  docker exec clew-neo4j cypher-shell -u neo4j -p password \
    "MATCH (f:File) RETURN count(f);"
  ```

---

## Rollback

If migration fails:

```bash
# 1. Stop services
docker-compose down

# 2. Restore backup
docker cp ./neo4j-backup/ clew-neo4j:/backups
docker exec clew-neo4j neo4j-admin database load neo4j --from-path=/backups

# 3. Downgrade to previous version
git checkout v0.1.0  # or your previous version
docker-compose build
docker-compose up -d
```

---

## Troubleshooting

### "Constraint already exists" Error

**Cause**: Trying to create duplicate constraint

**Solution**: This is expected and safe. The `IF NOT EXISTS` clause prevents errors.

### "Neo4j connection failed"

**Cause**: Neo4j not ready when migration runs

**Solution**: Wait for Neo4j to fully start (check `docker-compose logs neo4j`)

### Duplicate Nodes After Migration

**Cause**: Constraints not enforced before re-ingestion

**Solution**:
1. Verify constraints exist: `SHOW CONSTRAINTS;`
2. Delete duplicate nodes manually:
   ```cypher
   MATCH (f:File {path: $path, repo_id: $repo_id})
   WITH f.path, f.repo_id, collect(f) as nodes
   WHERE size(nodes) > 1
   FOREACH (n IN nodes[1..] | DELETE n)
   ```

### Performance Degradation

**Cause**: Indexes not created

**Solution**: Verify indexes exist with `SHOW INDEXES;`. If missing, restart services.

---

## Support

If you encounter issues:

1. Check [GitHub Issues](https://github.com/matchdav/clew-engine/issues)
2. Review logs: `docker-compose logs`
3. Join [Discord](https://discord.gg/clew) for community support

---

## Breaking Changes

### API Changes
- ✅ `/search` endpoint: Added optional `repo_id` parameter
- ✅ `/traverse` endpoint: Added optional `repo_id` parameter
- ✅ MCP tools: Added `repo_id` parameter to `search_codebase` and `explore_module`

### Backward Compatibility
- Old ingestion code will fail (missing `repo_id` parameter)
- Old search queries work (defaults to searching all repos)

---

**Last Updated**: February 10, 2026
