# Rebuild Metadata Implementation

## Summary

Added `mcp-vector-search reset rebuild-metadata` command to reconstruct `index_metadata.json` from the existing chunks.lance database.

## Implementation Details

### 1. New CLI Command

**Location**: `src/mcp_vector_search/cli/commands/reset.py`

**Command**: `mcp-vector-search reset rebuild-metadata`

**Options**:
- `--project-root/-p`: Specify project root directory (default: current directory)
- `--force/-f`: Overwrite existing metadata file if present

### 2. What It Does

The command:
1. Opens the existing `chunks.lance` database
2. Queries all unique file paths from the chunks table
3. For each file path:
   - Checks if the file exists on the filesystem
   - Gets the current modification time (mtime)
   - Adds it to the metadata dictionary
4. Writes the rebuilt `index_metadata.json` with:
   - Current index version (`__version__`)
   - Current timestamp (`indexed_at`)
   - File modification times (`file_mtimes`)

### 3. Usage Examples

**Basic usage** (current directory):
```bash
mcp-vector-search reset rebuild-metadata
```

**With project root**:
```bash
mcp-vector-search reset rebuild-metadata --project-root /path/to/project
```

**Force overwrite existing metadata**:
```bash
mcp-vector-search reset rebuild-metadata --force
```

### 4. Automatic Detection

Added automatic detection in `status` command:
- If `chunks.lance` exists BUT `index_metadata.json` is missing
- Displays warning: "Missing index_metadata.json but chunks database exists"
- Suggests running: `mcp-vector-search reset rebuild-metadata`

**Location**: `src/mcp_vector_search/cli/commands/status.py` (lines ~213-223)

### 5. Use Cases

**When to use this command**:
- `index_metadata.json` is missing or corrupted
- After restoring a chunks database backup
- Incremental indexing thinks everything needs reindexing
- Metadata file is empty/invalid JSON

**What it fixes**:
- Prevents full reindexing when only incremental update needed
- Restores change detection for files
- Enables proper incremental indexing workflow

### 6. Output Example

```
Rebuilding metadata from chunks database...

  Reading file paths from chunks database...
  Found 1,234 unique files in database

  Checking filesystem for modification times...
  ✓ 1,200 files found on disk
  ⚠ 34 files missing or inaccessible

  Writing index_metadata.json...

╭─ Rebuild Complete ────────────────────────────────────╮
│ ✅ Metadata rebuilt successfully!                     │
│                                                       │
│ Summary:                                              │
│ • Files in database: 1,234                            │
│ • Files found on disk: 1,200                          │
│ • Files missing: 34                                   │
│ • Metadata file: .mcp-vector-search/index_metadata.json │
│                                                       │
│ The metadata file now reflects the current state of  │
│ your chunks database. Incremental indexing will work │
│ correctly on the next run.                            │
╰───────────────────────────────────────────────────────╯
```

### 7. Error Handling

The command handles:
- Project not initialized → Error message
- No chunks database → Error with helpful message
- Metadata file exists without `--force` → Warning
- Files missing from filesystem → Warning in output
- Database connection errors → Graceful error handling

### 8. Technical Implementation

**Key functions**:
- `rebuild_metadata()`: Main CLI command handler
- `_rebuild_metadata_async()`: Async implementation for database operations

**Dependencies**:
- `ChunksBackend`: To read file paths from chunks.lance
- `IndexMetadata`: To save metadata file with correct format
- Pandas: To process unique file paths efficiently

**Performance**:
- Uses LanceDB scanner to read only `file_path` column (efficient)
- Processes unique files only (no duplicates)
- Batch operation (single database read)

### 9. Integration

The command is integrated into the reset command group:
- `mcp-vector-search reset index` → Reset entire index
- `mcp-vector-search reset health` → Check index health
- `mcp-vector-search reset rebuild-metadata` → Rebuild metadata (NEW)
- `mcp-vector-search reset all` → Complete reset

### 10. Files Modified

1. **src/mcp_vector_search/cli/commands/reset.py**
   - Added `rebuild_metadata()` command
   - Added `_rebuild_metadata_async()` helper
   - Updated help text in `reset_main()` callback

2. **src/mcp_vector_search/cli/commands/status.py**
   - Added automatic detection for missing metadata
   - Shows warning and suggests rebuild command

### 11. Testing Notes

**Manual testing checklist**:
- [x] Command compiles without syntax errors
- [x] Command appears in `mcp-vector-search reset --help`
- [x] Help text is clear and comprehensive
- [ ] Run on a project with missing metadata file
- [ ] Verify metadata file is correctly rebuilt
- [ ] Verify incremental indexing works after rebuild
- [ ] Test `--force` flag behavior
- [ ] Test error cases (no database, no project, etc.)

**Automated testing** (not implemented yet):
- Unit tests for `_rebuild_metadata_async()`
- Integration tests for full workflow
- Edge case testing (empty database, all files missing, etc.)

## Benefits

1. **Prevents unnecessary reindexing**: Restores change detection
2. **Self-healing**: Automatically detects and suggests fix
3. **Fast recovery**: No need to reindex entire codebase
4. **Data preservation**: Uses existing chunks database
5. **User-friendly**: Clear error messages and progress output

## Future Enhancements

Potential improvements:
1. **Automatic rebuild**: Offer to rebuild automatically when detected
2. **Dry-run mode**: Preview what would be rebuilt without writing
3. **Validation**: Compare metadata against database for consistency
4. **Progress bar**: Show progress for large codebases
5. **Statistics**: Compare old vs new metadata (if old exists)
