# Memory Cap Fix Deployment Guide

**Version:** 2.5.22
**Critical Fix:** Memory cap enforcement now works correctly
**Priority:** HIGH - Deploy immediately to prevent OOM crashes

---

## Quick Deploy Steps

### 1. Pull Latest Code
```bash
cd /path/to/mcp-vector-search
git pull origin main
git checkout v2.5.22  # Or latest tag
```

### 2. Install/Update Package
```bash
# If using pip
pip install --upgrade .

# If using uv (recommended)
uv pip install --upgrade .

# Or install from PyPI (once published)
pip install --upgrade mcp-vector-search
```

### 3. Verify Version
```bash
python -c "import mcp_vector_search; print(f'Version: {mcp_vector_search.__version__}')"
# Expected output: Version: 2.5.22
```

### 4. Set Memory Cap (if not already set)
```bash
# Add to your environment or systemd service file
export MCP_VECTOR_SEARCH_MAX_MEMORY_GB=25

# For systemd service, add to /etc/systemd/system/mcp-indexer.service:
[Service]
Environment="MCP_VECTOR_SEARCH_MAX_MEMORY_GB=25"
```

### 5. Restart Service
```bash
# If using systemd
sudo systemctl restart mcp-indexer
sudo systemctl status mcp-indexer

# If running manually
pkill -f mcp-vector-search
mcp-vector-search index /path/to/project
```

### 6. Monitor for 30 Minutes
```bash
# Watch memory usage
watch -n 2 'ps aux | grep mcp-vector-search | grep -v grep | awk "{printf \"Memory: %.2f GB\\n\", \$6/1024/1024}"'

# Check logs for memory warnings
tail -f /path/to/logs/indexer.log | grep -i memory
```

---

## AWS EC2 Deployment

### Option A: Direct SSH Deployment
```bash
# SSH into EC2 instance
ssh ubuntu@your-ec2-instance

# Pull latest code
cd /path/to/mcp-vector-search
git pull origin main

# Reinstall
source venv/bin/activate
pip install --upgrade .

# Verify version
python -c "import mcp_vector_search; print(mcp_vector_search.__version__)"

# Restart service
sudo systemctl restart mcp-indexer
```

### Option B: Docker Deployment
```bash
# Build new image
docker build -t mcp-vector-search:2.5.22 .

# Stop old container
docker stop mcp-indexer

# Start new container with memory cap
docker run -d \
  --name mcp-indexer \
  --restart unless-stopped \
  -e MCP_VECTOR_SEARCH_MAX_MEMORY_GB=25 \
  -v /path/to/data:/data \
  mcp-vector-search:2.5.22 \
  index /data/project
```

### Option C: AWS ECS/Fargate Deployment
```bash
# Update task definition with new image
aws ecs register-task-definition \
  --family mcp-indexer \
  --container-definitions '[{
    "name": "indexer",
    "image": "your-repo/mcp-vector-search:2.5.22",
    "environment": [{
      "name": "MCP_VECTOR_SEARCH_MAX_MEMORY_GB",
      "value": "25"
    }]
  }]'

# Update service
aws ecs update-service \
  --cluster your-cluster \
  --service mcp-indexer \
  --force-new-deployment
```

---

## Verification Checklist

- [ ] Version is 2.5.22 (`python -c "import mcp_vector_search; print(mcp_vector_search.__version__)"`)
- [ ] Environment variable is set (`echo $MCP_VECTOR_SEARCH_MAX_MEMORY_GB`)
- [ ] Service restarted successfully
- [ ] Memory cap appears in logs: `"Memory monitor initialized: 25.0GB cap"`
- [ ] Memory stays under cap for 30 minutes
- [ ] See batch size adjustments in logs (if under pressure)
- [ ] No OOM crashes
- [ ] Indexing completes successfully

---

## Expected Log Messages

### Successful Deployment
```
[INFO] Memory monitor initialized: 25.0GB cap (warn: 80%, critical: 90%)
[INFO] Starting indexing with memory cap enforcement
[INFO] Phase 1: Parsing and chunking files...
[INFO] Phase 2: Embedding pending chunks...
```

### Under Memory Pressure (Normal)
```
[WARN] Memory usage high: 18.50GB / 25.0GB (74.0%)
[INFO] Adjusted embedding batch size: 1000 → 750 (memory usage: 70.5%)
```

### Critical Memory Pressure (Should Wait, Not Crash)
```
[WARN] Memory limit exceeded after loading batch (88.2% of 25.0GB), waiting for memory to free up...
[INFO] Memory usage dropped to 75.3%, resuming processing
```

### Severe Memory Pressure (Should Reduce Batch, Not Crash)
```
[ERROR] Memory limit exceeded before embedding (91.5% of 25.0GB). Reducing batch size and retrying...
[INFO] Reduced batch size to 250 due to memory pressure
```

---

## Rollback Plan

If deployment fails or issues occur:

### 1. Rollback to Previous Version
```bash
git checkout v2.5.21  # Or your previous stable version
pip install --upgrade .
sudo systemctl restart mcp-indexer
```

### 2. Temporarily Increase Memory Cap
```bash
# Quick fix: increase cap temporarily
export MCP_VECTOR_SEARCH_MAX_MEMORY_GB=50
sudo systemctl restart mcp-indexer
```

### 3. Report Issue
Collect this information:
```bash
# Version
python -c "import mcp_vector_search; print(mcp_vector_search.__version__)"

# Environment
echo $MCP_VECTOR_SEARCH_MAX_MEMORY_GB

# Memory usage
ps aux | grep mcp-vector-search | grep -v grep

# Recent logs
tail -100 /path/to/logs/indexer.log
```

---

## Performance Notes

### Expected Changes
- **Memory usage:** Capped at configured limit (was uncontrolled)
- **Indexing speed:** 10-20% slower under memory pressure (acceptable tradeoff)
- **Stability:** No more OOM crashes (critical improvement)

### Not Expected
- **No indexing progress:** Should always make forward progress (slower is OK)
- **Constant memory warnings:** Should only warn when approaching limit
- **Complete indexing failure:** Should complete eventually, even if slower

---

## Monitoring Setup

### CloudWatch Alarms (AWS)
```bash
# Memory usage alarm
aws cloudwatch put-metric-alarm \
  --alarm-name mcp-indexer-memory-high \
  --alarm-description "Memory usage above 80% for 10 minutes" \
  --metric-name MemoryUtilization \
  --namespace AWS/ECS \
  --statistic Average \
  --period 300 \
  --threshold 80 \
  --comparison-operator GreaterThanThreshold \
  --evaluation-periods 2
```

### Custom Metrics
Add to your monitoring dashboard:
- `process_memory_usage_gb` - Current memory in GB
- `memory_cap_enforcement_events` - Number of times backpressure applied
- `batch_size_reductions` - Number of batch size reductions
- `memory_warnings` - Number of memory warnings logged

---

## FAQ

**Q: Will this slow down indexing?**
A: Yes, by 10-20% under memory pressure. This is the tradeoff for preventing OOM crashes.

**Q: What if indexing seems stuck?**
A: Check logs for "Memory limit exceeded" messages. If memory cap is too low, temporarily increase it.

**Q: How do I know the fix is working?**
A: Memory usage should NEVER exceed your configured cap. Check with `ps aux` or monitoring dashboard.

**Q: Can I disable the memory cap?**
A: No, it's always enabled. Set a high cap (e.g., 100GB) if you have plenty of memory.

**Q: What memory cap should I use?**
A: Use 70-80% of your system memory. For 32GB system, use 25GB. For 64GB system, use 50GB.

---

## Support

Issues or questions?
- Email: bob@matsuoka.com
- GitHub Issues: https://github.com/your-repo/mcp-vector-search/issues
