# Testing Guide: Skip Delete Optimization

## Quick Verification

### 1. Syntax Check
```bash
python -m py_compile src/mcp_vector_search/core/indexer.py
# Expected: No output (success)
```

### 2. Run Verification Script
```bash
python test_skip_delete_optimization.py
# Expected: Shows all tests passing with detailed change summary
```

## Integration Testing

### Test 1: Force Reindex with Fresh Database
**Objective**: Verify delete operations are skipped when building fresh database

```bash
# Clean existing database
rm -rf .mcp-vector-search/lance.new .mcp-vector-search/code_search.lance.new

# Run force reindex with timing
mcp-vector-search index -f --limit 100

# Expected behavior:
# - No "Deleting chunks for file" log messages
# - Faster indexing compared to before optimization
# - Database builds successfully to .new location
```

### Test 2: Incremental Reindex (Backward Compatibility)
**Objective**: Verify delete operations still work for incremental updates

```bash
# Do initial index
mcp-vector-search index

# Modify a file
touch some_file.py

# Re-index (incremental)
mcp-vector-search index

# Expected behavior:
# - Delete operations still execute (skip_delete=False by default)
# - Old chunks removed before adding new ones
# - No functional change from previous behavior
```

### Test 3: Watcher Behavior
**Objective**: Verify watcher still deletes old chunks on file changes

```bash
# Start watcher in one terminal
mcp-vector-search watch

# Modify a file in another terminal
echo "# comment" >> some_file.py

# Expected behavior:
# - Watcher detects change
# - Calls index_file() with skip_delete=False (default)
# - Old chunks deleted before re-indexing
```

## Performance Benchmarking

### Before Optimization (Baseline)
```bash
# Checkout previous commit
git stash
git checkout HEAD~1

# Time force reindex
time mcp-vector-search index -f --limit 100

# Record timing
# Example: real 2m30s, user 1m45s, sys 0m15s
```

### After Optimization
```bash
# Return to optimized version
git checkout -
git stash pop

# Time force reindex
time mcp-vector-search index -f --limit 100

# Record timing
# Expected: Faster than baseline (reduced per-file overhead)
# Example: real 2m15s, user 1m40s, sys 0m12s
```

### Metrics to Track
- **Total indexing time**: Should decrease
- **Per-file processing time**: Should be more consistent
- **Database query count**: Should be lower (no unnecessary deletes)

## Unit Testing Considerations

### Existing Tests Should Pass
```bash
# Run existing tests to ensure no regressions
pytest tests/unit/core/test_indexer.py -v
pytest tests/integration/test_indexing_workflow.py -v
```

### New Test Cases (Future Work)
Consider adding tests for:
1. `_atomic_rebuild_active` flag lifecycle
2. `skip_delete` parameter propagation
3. Conditional delete behavior
4. Flag reset on success/failure

## Edge Cases to Verify

### 1. Atomic Rebuild Failure
```python
# Simulate failure in _atomic_rebuild_databases()
# Expected: _atomic_rebuild_active reset to False
```

### 2. Finalization Rollback
```python
# Simulate failure in _finalize_atomic_rebuild()
# Expected: _atomic_rebuild_active reset to False after rollback
```

### 3. Mixed Incremental and Force Reindex
```bash
# Run incremental index
mcp-vector-search index

# Then force reindex
mcp-vector-search index -f

# Expected: Both work correctly with appropriate skip_delete values
```

## Validation Checklist

- [ ] Syntax check passes
- [ ] Verification script shows all changes
- [ ] Force reindex completes successfully
- [ ] Incremental reindex still works
- [ ] Watcher detects and reindexes changes
- [ ] Existing unit tests pass
- [ ] Performance improvement measurable
- [ ] No functional regressions
- [ ] Flag properly reset on success
- [ ] Flag properly reset on failure

## Success Criteria

✅ **Optimization is successful if**:
1. Force reindex completes without errors
2. Delete operations skipped during force reindex (check logs)
3. Incremental updates still delete old chunks
4. Measurable performance improvement (reduced overhead)
5. All existing tests pass
6. No functional changes to incremental indexing

## Rollback Plan

If issues are found:
```bash
git diff src/mcp_vector_search/core/indexer.py > optimization.patch
git checkout src/mcp_vector_search/core/indexer.py
# Revert to previous behavior
```

To re-apply:
```bash
git apply optimization.patch
```

## Additional Notes

### Debug Logging
To verify skip_delete behavior, add temporary debug logging:
```python
logger.debug(f"skip_delete={skip_delete}, atomic_rebuild_active={self._atomic_rebuild_active}")
```

### Monitoring in Production
After deployment, monitor:
- Force reindex duration (should decrease)
- Database query counts (should be lower)
- Error rates (should remain same)
- Incremental index behavior (should be unchanged)
