# AI Migration Implementation Summary

## Features Implemented

### 1. **Core AI Migration Module** (`onepasswd/ai_migrate.py`)

- **Structured Output with Pydantic**: Uses OpenAI's structured output feature to enforce consistent JSON schema
- **Async/Concurrent Processing**: Processes multiple batches concurrently with configurable limits
- **Rate Limiting**: Built-in semaphore and exponential backoff for API rate limits
- **Real-time Progress**: Callback system for live progress updates
- **Privacy-First**: Never sends passwords to AI - only metadata (keywords, title, URL, etc.)

### 2. **CLI Commands**

#### `onepasswd ai-migrate`
Generate AI-powered review file with intelligent metadata filling.

**Options:**
- `--context TEXT` - User context for better AI inference
- `--model TEXT` - OpenAI model (default: gpt-4o-mini)
- `--api-key TEXT` - OpenAI API key (or use OPENAI_API_KEY env var)
- `--batch-size INT` - Entries per API call (default: 20)
- `--max-concurrent INT` - Max concurrent requests (default: 3)
- `--output PATH` - Output review file path

**Features:**
- Real-time progress display as batches complete
- Shows sample results for each batch
- Displays confidence levels (high/medium/low)
- Handles rate limits gracefully with retry logic

#### `onepasswd ai-migrate-apply`
Apply approved changes from review file.

**Options:**
- `--dry-run` - Preview changes without applying
- Creates automatic backup before applying
- Shows summary of applied/skipped/failed entries

### 3. **Key Technical Features**

#### Structured Output (Pydantic Models)
```python
class PasswordEntryMetadata(BaseModel):
    entry_index: int
    keywords: List[str]
    title: Optional[str]
    url: Optional[str]
    username: Optional[str]
    category: Optional[str]
    description: Optional[str]
    tags: List[str]
    confidence: str  # high/medium/low
    notes: Optional[str]
```

Benefits:
- Guaranteed valid JSON structure
- Type safety
- No parsing errors
- Clear schema definition

#### Async Processing with Rate Limiting
```python
async def process_entries_async(
    db,
    max_concurrent: int = 3,  # Control concurrency
    progress_callback: Optional[callable] = None
)
```

Features:
- Semaphore-based concurrency control
- Exponential backoff for rate limits
- Real-time progress callbacks
- Concurrent batch processing

#### Rate Limit Handling
- Automatic retry with exponential backoff
- Max 3 retries per batch
- Graceful degradation on persistent failures
- Logs warnings for rate limit events

### 4. **User Workflow**

```bash
# Step 1: Generate review file with AI inference
onepasswd ai-migrate --context "Personal accounts" --max-concurrent 2

# Output shows real-time progress:
# [Batch 1/5] ✓ Completed - 20 entries processed
#   Sample: [github, work]
#     → GitHub Work Account
#     → https://github.com
#     → Confidence: high

# Step 2: Review the generated JSON file
# - Edit 'proposed' fields as needed
# - Set 'approved: false' to skip entries
# - Add your own notes

# Step 3: Preview changes (optional)
onepasswd ai-migrate-apply review.json --dry-run

# Step 4: Apply changes
onepasswd ai-migrate-apply review.json
```

### 5. **AI Inference Rules**

The AI follows these guidelines:
1. **Conservative** - Only fills fields with confidence
2. **URL normalization** - Always uses HTTPS
3. **Title capitalization** - Proper case for titles
4. **Known services** - Uses standard URLs for popular services
5. **Username inference** - Only if email/username patterns detected
6. **Category selection** - Concise 1-2 word categories
7. **Tag relevance** - Lowercase, relevant tags only
8. **Confidence scoring** - Rates own confidence level

### 6. **Error Handling**

- **Rate Limits**: Exponential backoff with 3 retries
- **API Errors**: Graceful failure with logging
- **Missing Dependencies**: Clear error messages with installation instructions
- **Invalid API Keys**: Validation before processing
- **Malformed Responses**: Structured output prevents parsing errors

### 7. **Performance Considerations**

- **Batch Size**: 20 entries per API call (configurable)
- **Concurrency**: 3 concurrent requests (configurable, default safe for rate limits)
- **Cost**: ~$0.01 per 100 entries with gpt-4o-mini
- **Speed**: Concurrent processing significantly faster than sequential

Example: 100 entries
- Sequential: ~60 seconds (5 batches × 12s each)
- Concurrent (3x): ~24 seconds (5 batches in 2 rounds)

### 8. **Documentation**

- **Main README**: Updated with AI migration feature
- **AI Migration Guide**: Comprehensive documentation in `docs/AI_MIGRATION.md`
- **Inline Help**: `onepasswd ai-migrate --help`
- **Examples**: Multiple usage examples throughout

### 9. **Testing**

Test file created: `test_ai_migrate.py`

Tests cover:
- Metadata extraction (no password leakage)
- Missing field identification
- Prompt generation
- Review file generation
- OpenAI availability check

## Usage Examples

### Basic Usage
```bash
export OPENAI_API_KEY='sk-...'
onepasswd ai-migrate
```

### With Context
```bash
onepasswd ai-migrate --context "Work accounts for ACME Corp"
```

### Rate Limited Environment
```bash
onepasswd ai-migrate --max-concurrent 1 --batch-size 10
```

### Different Model
```bash
onepasswd ai-migrate --model gpt-4o
```

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────┐
│                    User (CLI)                            │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│              ai-migrate command                          │
│  - Parse options                                         │
│  - Authenticate user                                     │
│  - Setup progress callback                               │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│         AIPasswordMigrator.process_entries_async()       │
│  - Collect entries with missing fields                   │
│  - Create batches                                        │
│  - Setup semaphore (max_concurrent)                      │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│              Concurrent Batch Processing                 │
│  ┌────────┐  ┌────────┐  ┌────────┐                     │
│  │Batch 1 │  │Batch 2 │  │Batch 3 │  (max_concurrent=3) │
│  └────┬───┘  └────┬───┘  └────┬───┘                     │
│       │           │           │                          │
└───────┼───────────┼───────────┼──────────────────────────┘
        │           │           │
        ▼           ▼           ▼
┌─────────────────────────────────────────────────────────┐
│              OpenAI API (Structured Output)              │
│  - Pydantic schema validation                            │
│  - Temperature: 0.3                                      │
│  - Model: gpt-4o-mini (default)                          │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼ (Per batch completion)
┌─────────────────────────────────────────────────────────┐
│              Progress Callback                           │
│  - Update UI with batch result                           │
│  - Show sample entry                                     │
│  - Display confidence                                    │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼ (All batches done)
┌─────────────────────────────────────────────────────────┐
│           Generate Review File (JSON)                    │
│  - Write all results                                     │
│  - Add instructions                                      │
│  - Set approved: true (default)                          │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│            User Reviews & Edits File                     │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│          ai-migrate-apply command                        │
│  - Read review file                                      │
│  - Create backup                                         │
│  - Apply approved changes                                │
│  - Report results                                        │
└─────────────────────────────────────────────────────────┘
```

## Security & Privacy

✅ **Safe:**
- Passwords never sent to OpenAI
- Only metadata sent (keywords, title, URL, etc.)
- Master password never leaves local machine
- All passwords remain encrypted locally

⚠️ **User Responsibility:**
- Review context carefully (no sensitive info)
- Check generated review file before applying
- Secure API key storage

## Future Enhancements

Potential improvements:
- [ ] Local LLM support (Ollama, Llama.cpp)
- [ ] Custom inference rules/templates
- [ ] URL normalization and validation
- [ ] Duplicate detection
- [ ] Interactive TUI for batch review
- [ ] Confidence threshold filtering
- [ ] Category suggestions based on user's existing categories
- [ ] Multi-language support for prompts
