# Progress Bar Implementation for KG Build

## Summary

Added progress bars to the Knowledge Graph build pipeline to provide visibility during slow operations (30-60+ seconds).

## Changes Made

### 1. CLI Integration (`src/mcp_vector_search/cli/commands/kg.py`)

**Modified:** Subprocess build function `_do_build_sync()`

- Created `ProgressTracker` instance with console output
- Passed `progress_tracker` to `build_from_chunks_sync()`

```python
from ...core.progress import ProgressTracker

progress_tracker = ProgressTracker(console, verbose=False)
builder = KGBuilder(kg, project_root)
build_stats = builder.build_from_chunks_sync(
    chunks,
    show_progress=True,
    skip_documents=skip_documents,
    progress_tracker=progress_tracker,
)
```

### 2. KG Builder Progress Tracking (`src/mcp_vector_search/core/kg_builder.py`)

**Modified:** `build_from_chunks_sync()` method

#### Phase 1: Scanning Chunks (Lines 382-420)

Added progress bars to chunk extraction loops:

1. **Code Chunks Loop**: Shows progress every 500 chunks
   - Uses `progress_bar_with_eta()` for ETA estimation
   - Displays: `Scanning code chunks... ━━━━━━ 67% 54,234/81,238 [01:23 remaining]`

2. **Text Chunks Loop**: Shows progress every 100 chunks
   - Uses `progress_bar_with_eta()` for ETA estimation
   - Displays: `Scanning text chunks... ━━━━━━ 45% 2,277/5,060 [00:05 remaining]`

**Key Implementation Details:**
- Tracks start time for accurate ETA calculation
- Updates every N chunks to avoid overwhelming output
- Shows final progress bar update on last chunk

#### Phase 3: Building Relationships (Lines 563-630)

Added progress bars to relationship validation and insertion:

1. **Validation Loop**: Shows progress every 1,000 relationships
   - Validates source/target entity IDs exist in Kuzu
   - Displays: `Validating calls... ━━━━━━ 67% 17,000/25,691 [00:08 remaining]`

2. **Insertion Progress**: Shows status for each relationship type
   - Before insertion: `→ Inserting 2,500 calls relationships...`
   - After insertion: `✓ 2,500 calls inserted`

**Key Implementation Details:**
- Counts total relationships across all types for accurate progress
- Tracks processed count across all relationship types
- Shows per-type progress during validation
- Provides insertion status messages

## Progress Bar Features

The `ProgressTracker` class (from `src/mcp_vector_search/core/progress.py`) provides:

1. **Phase-based tracking**: Clear separation of build phases
2. **Inline progress bars**: Updates in-place without spawning threads (Kuzu-safe)
3. **ETA estimation**: Shows estimated time remaining
4. **Percentage and counts**: `67% 54,234/81,238`
5. **Thread-safe**: Uses simple `sys.stderr.write()` without Rich background threads

## Example Output

```
Building Knowledge Graph
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Phase 1/4: Scanning chunks
  Scanning code chunks... ━━━━━━━━━━━━━━━━ 100% 4,532/4,532 [00:12 remaining]
  Scanning text chunks... ━━━━━━━━━━━━━━━━ 100% 57,812/57,812 [01:45 remaining]
  ✓ Processed 62,344 chunks
  ✓ Found 4,532 code entities
  ✓ Found 57,812 doc sections
  ✓ Found 26 tags
  (completed in 2.1s)

Phase 2/4: Inserting entities
  ✓ 4,532 code entities
  ✓ 57,812 doc sections
  ✓ 26 tags
  (completed in 1.5s)

Phase 3/4: Building relationships
  Validating calls... ━━━━━━━━━━━━━━━━ 100% 2,500/2,500 [00:00 remaining]
  → Inserting 2,500 calls relationships...
  ✓ 2,500 calls inserted
  Validating imports... ━━━━━━━━━━━━━━━━ 100% 3,200/3,200 [00:00 remaining]
  → Inserting 3,200 imports relationships...
  ✓ 3,200 imports inserted
  (completed in 8.7s)

Phase 4/4: Extracting git metadata
  ✓ 5 persons
  ✓ 1 project
  ✓ 4,532 authored relationships
  ✓ 4,532 part_of relationships
  (completed in 0.8s)

✓ Knowledge graph built successfully! Total: 62,370 entities, 12,764 relationships
  Time: 13.1s
```

## Testing

To test the progress bars:

```bash
# Build KG with progress bars
mcp-vector-search kg build --force

# Expected: Progress bars show during Phase 1 (scanning) and Phase 3 (relationships)
# Each progress bar updates in-place with ETA estimates
```

## Key Design Decisions

1. **No Rich Progress Bars**: Used `ProgressTracker.progress_bar_with_eta()` instead of Rich's `Progress` class to avoid background threads that conflict with Kuzu's Rust bindings.

2. **Update Frequency**:
   - Code chunks: Every 500 (typical: 4-5k chunks)
   - Text chunks: Every 100 (typical: 50-60k chunks)
   - Relationships: Every 1,000 (typical: 10-30k relationships)

   These frequencies balance responsiveness vs. overhead.

3. **ETA Calculation**: Uses start time and current progress to estimate remaining time, providing users with actionable feedback.

4. **Thread Safety**: All progress tracking uses simple `sys.stderr.write()` without background rendering threads, ensuring compatibility with Kuzu's single-threaded requirements.

## Performance Impact

- **Minimal overhead**: Progress updates occur every N items (500/100/1000)
- **No additional computation**: Uses existing loop counters
- **No threading**: Safe for Kuzu's Rust bindings
- **Expected overhead**: <0.1% of total build time

## Future Enhancements

Possible improvements:

1. Add progress bars to Phase 4 (git metadata extraction) if it becomes slow
2. Add configurable update frequency via CLI flag (e.g., `--progress-interval 1000`)
3. Add option to disable progress bars (e.g., `--no-progress`)
4. Add progress bars to Phase 2 (entity insertion) if batches become large
