# Contributing to Mini VideoRAG

Thank you for contributing! This guide covers the essentials for adding features and submitting PRs.

---

## Quick Start

```bash
# Setup
git clone https://github.com/bdallard/mini-videorag.git
cd mini-videorag

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

# Copy and configure environment
cp .env.example .env
cp config/video_rag_config.example.yml config/video_rag_config.yml

# Verify
make test-ci
```

---

## Architecture Principles

1. **Framework-Agnostic Core**: Business logic independent of Temporal/FastAPI
2. **Lazy Loading**: ML dependencies loaded only when needed
3. **Factory Pattern**: All processors via `ProcessorFactory` (never direct imports)
4. **Type-Safe Config**: Pydantic models (`WorkflowStepConfig`, `VideoRAGSessionConfig`)

### Key Rules

- **Processors**: Instantiate via `ProcessorFactory.create()` only
- **Imports**: Lazy imports for processors (inside functions, not module-level)
- **Errors**: Use custom exceptions from `mini_videorag.core.exceptions`
- **Tests**: Centralized parametrized tests + processor-specific business logic

---

## Adding a New Processor

### 1. Create Module Structure

```bash
mkdir -p mini_videorag/processors/scene_detection
touch mini_videorag/processors/scene_detection/{__init__.py,base.py,your_impl.py,formatter.py}
```

### 2. Implement Base Class (`base.py`)

```python
from abc import ABC
from mini_videorag.core.processor import BaseVideoProcessor

class BaseSceneDetectionProcessor(BaseVideoProcessor, ABC):
    @property
    def name(self) -> str:
        return "scene_detection"

    @property
    def requires_audio(self) -> bool:
        return False

    @property
    def requires_video(self) -> bool:
        return True
```

### 3. Implement Processor (`your_impl.py`)

```python
from mini_videorag.core.processor import ProcessorResult
from mini_videorag.core.exceptions import VideoLoadError
from .base import BaseSceneDetectionProcessor

class YourSceneDetectionProcessor(BaseSceneDetectionProcessor):
    def __init__(self, threshold: float = 0.5, device: str = None):
        self.threshold = threshold
        # Device auto-detection (standard pattern)
        if device is None:
            import torch
            self.device = "mps" if torch.backends.mps.is_available() else \
                         "cuda:0" if torch.cuda.is_available() else "cpu"
        else:
            self.device = device

    @property
    def provider(self) -> str:
        return "your_provider"

    def process(self, video_path: str, video_id: str, **kwargs) -> ProcessorResult:
        try:
            self.validate_input(video_path)
            # Your implementation
            return ProcessorResult(
                processor_name=self.name,
                video_id=video_id,
                success=True,
                data={"scene_count": 5, "timestamps": [0.0, 5.2, 10.5]},
            )
        except VideoLoadError as e:
            return ProcessorResult(
                processor_name=self.name,
                video_id=video_id,
                success=False,
                error=f"VideoLoadError: {e}",
            )
```

### 4. Create Formatter (`formatter.py`)

```python
from mini_videorag.core.formatters import register_formatter, FormattedOutput
from mini_videorag.core.processor import ProcessorResult

@register_formatter("scene_detection")
def format_scene_detection(result: ProcessorResult) -> FormattedOutput:
    if not result.success:
        return FormattedOutput(
            summary_line="SCENE_DETECTION: failed",
            content_text="",
            detail_line=f"error={result.error}",
            is_empty=True,
        )

    count = result.data.get("scene_count", 0)
    return FormattedOutput(
        summary_line=f"SCENE_DETECTION: {count} scenes",
        content_text=str(result.data.get("timestamps", [])),
        detail_line=f"success | scene_count={count}",
        is_empty=count == 0,
        rag_metadata={"visual_scene_count": count},
        vlm_context=f"SCENE_DETECTION: {count} scenes detected",
    )
```

### 5. Register in Factory

Edit `mini_videorag/processors/factory.py`:

```python
class ProcessorFactory:
    _PROVIDER_PATHS: Dict[str, Dict[str, str]] = {
        # ... existing ...
        "scene_detection": {
            "your_provider": "mini_videorag.processors.scene_detection:YourSceneDetectionProcessor",
        },
    }

    DEFAULTS: Dict[str, str] = {
        # ... existing ...
        "scene_detection": "your_provider",
    }
```

### 6. Update Imports

Edit `mini_videorag/processors/__init__.py`:

```python
from .scene_detection.base import BaseSceneDetectionProcessor

__all__ = [
    # ... existing ...
    "BaseSceneDetectionProcessor",
]
```

### 7. Add to Config

Edit `config/video_rag_config.yml`:

```yaml
processors:
  scene_detection:
    provider: your_provider
    threshold: 0.5

workflow:
  scene_detection:
    depends_on: []
    enabled: true
```

### 8. Register Formatter

Edit `mini_videorag/core/formatters.py`:

```python
@functools.lru_cache(maxsize=1)
def load_all_formatters():
    formatter_modules = [
        # ... existing ...
        "mini_videorag.processors.scene_detection.formatter",
    ]
    # ... rest of function ...
```

### 9. Add Tests

**A. Add to centralized tests**

`tests/test_processor_factory.py`:
```python
@pytest.mark.parametrize(
    "processor_type,expected_provider,expected_name,requires_audio,requires_video",
    [
        # ... existing ...
        ("scene_detection", "your_provider", "scene_detection", False, True),
    ],
)
```

`tests/test_formatters_common.py`:
```python
@pytest.mark.parametrize(
    "processor_name,formatter_func",
    [
        # ... existing ...
        ("scene_detection", format_scene_detection),
    ],
)
```

**B. Create processor-specific tests**

`tests/processors/scene_detection/test_your_impl.py`:
```python
import pytest

class TestYourSceneDetectionProcessor:
    @pytest.mark.unit
    def test_invalid_video_path(self):
        # Lazy import (CRITICAL - avoids loading ML dependencies)
        from mini_videorag.processors.scene_detection.your_impl import (
            YourSceneDetectionProcessor,
        )

        processor = YourSceneDetectionProcessor()
        result = processor.process(
            video_path="/nonexistent/video.mp4",
            video_id="test"
        )
        assert result.success is False
        assert result.error is not None

    @pytest.mark.unit
    def test_custom_threshold(self):
        from mini_videorag.processors.scene_detection.your_impl import (
            YourSceneDetectionProcessor,
        )

        processor = YourSceneDetectionProcessor(threshold=0.7)
        assert processor.threshold == 0.7
```

---

## Creating Custom Processor Plugins

You can extend Mini VideoRAG with custom processors without modifying the core codebase.

### Option 1: Runtime Registration (Programmatic)

Register a custom processor at runtime:

```python
from mini_videorag.processors import ProcessorFactory
from mini_videorag.core.processor import BaseVideoProcessor

# Create your custom processor
class MyCustomProcessor(BaseVideoProcessor):
    @property
    def name(self) -> str:
        return "my_processor"

    @property
    def provider(self) -> str:
        return "my_provider"

    def process(self, video_path, video_id, **kwargs):
        # Your implementation
        return ProcessorResult(...)

# Register it
factory = ProcessorFactory()
factory.register_processor(
    "my_processor",
    "my_provider",
    "my_package.processors:MyCustomProcessor",
    set_as_default=True
)

# Use it
processor = factory.create("my_processor")
```

### Option 2: Entry Points (Installable Package)

Create a separate package with entry points:

**1. Package structure:**
```
videorag-my-plugin/
├── pyproject.toml
└── videorag_my_plugin/
    └── processors.py
```

**2. Implement processor (`processors.py`):**
```python
from mini_videorag.core.processor import BaseVideoProcessor, ProcessorResult

class MyCustomProcessor(BaseVideoProcessor):
    @property
    def name(self) -> str:
        return "my_processor"

    @property
    def provider(self) -> str:
        return "my_provider"

    def process(self, video_path, video_id, **kwargs):
        # Your implementation
        return ProcessorResult(
            processor_name=self.name,
            video_id=video_id,
            success=True,
            data={"result": "data"}
        )
```

**3. Register entry point (`pyproject.toml`):**
```toml
[project]
name = "videorag-my-plugin"
version = "0.1.0"
dependencies = ["mini-videorag"]

[project.entry-points."mini_videorag.processors"]
my_processor.my_provider = "videorag_my_plugin.processors:MyCustomProcessor"
```

**4. Install and use:**
```bash
pip install videorag-my-plugin

# Auto-discover plugins
factory = ProcessorFactory(auto_discover=True)
processor = factory.create("my_processor")

# Or manually discover
factory = ProcessorFactory()
factory.discover_plugins()
processor = factory.create("my_processor")
```

### Option 3: YAML Configuration

Register processors via configuration file with zero code changes:

**1. Create your processor package:**
```
my-processors/
├── pyproject.toml
└── my_processors/
    └── scene_detector.py
```

**2. Implement your processor (`scene_detector.py`):**
```python
from mini_videorag.core.processor import BaseVideoProcessor, ProcessorResult

class OpenCVSceneDetector(BaseVideoProcessor):
    @property
    def name(self) -> str:
        return "scene_detection"

    @property
    def provider(self) -> str:
        return "opencv"

    def process(self, video_path, video_id, **kwargs):
        # Your implementation
        return ProcessorResult(...)
```

**3. Add to your `config/video_rag_config.yml`:**
```yaml
custom_processors:
  - processor_type: scene_detection
    provider: opencv
    class_path: "my_processors.scene_detector:OpenCVSceneDetector"
    default: true  # Set as default provider

  - processor_type: sentiment
    provider: transformers
    class_path: "my_processors.sentiment:TransformersSentiment"
    default: false

# Use in workflow
workflow:
  scene_detection:
    depends_on: ["frame_extraction"]
    enabled: true
```

**4. Use it:**
```python
from mini_videorag.utils.config_loader import load_video_rag_config, create_pipeline_from_config

# Custom processors automatically registered from config
config = load_video_rag_config("config/video_rag_config.yml")
pipeline = create_pipeline_from_config(config)

# Process video
results = pipeline.run_all("video.mp4", "video_123")
```

**Benefits:**
- ✅ No code changes needed - pure configuration
- ✅ Easy to enable/disable processors
- ✅ Version control friendly
- ✅ Perfect for deployment scenarios
- ✅ Can be combined with workflow dependencies

---

## Code Standards

### Imports

```python
# ✅ CORRECT - Use factory
from mini_videorag.processors import ProcessorFactory
factory = ProcessorFactory()
ocr = factory.create("ocr")

# ✅ CORRECT - Lazy import in tests
def test_something():
    from mini_videorag.processors.ocr.easy_ocr import EasyOCRProcessor
    processor = EasyOCRProcessor()

# ❌ WRONG - Direct import at module level
from mini_videorag.processors.ocr.easy_ocr import EasyOCRProcessor
```

### Error Handling

Use custom exceptions:
```python
from mini_videorag.core.exceptions import VideoLoadError, ModelInitError, ProcessingError

try:
    # processing
except VideoLoadError as e:
    logger.error(f"Failed: {e}", exc_info=True)
    return ProcessorResult(..., success=False, error=f"VideoLoadError: {e}")
```

### Device Management

Standard pattern for all ML processors:
```python
def __init__(self, device: Optional[str] = None):
    if device is None:
        import torch
        if torch.backends.mps.is_available():
            self.device = "mps"
        elif torch.cuda.is_available():
            self.device = "cuda:0"
        else:
            self.device = "cpu"
    else:
        self.device = device
```

---

## Testing

### Run Tests

```bash
# Fast (skip slow tests and model downloads - RECOMMENDED)
make test-ci

# Full (includes model downloads - SLOW)
make test

# Specific file
pytest tests/processors/scene_detection/ -v

# By marker
pytest -m "unit"
pytest -m "not slow and not requires_download"
```

### Test Markers

```python
@pytest.mark.unit                # Fast, no external dependencies
@pytest.mark.integration         # Real dependencies
@pytest.mark.slow                # >10 seconds
@pytest.mark.requires_download   # Downloads models/files
```

### Testing Checklist

- [ ] Add processor to `tests/test_processor_factory.py` parametrized test
- [ ] Add formatter to `tests/test_formatters_common.py` parametrized tests
- [ ] Create `tests/processors/[name]/test_[impl].py` with business logic tests
- [ ] All imports are lazy (inside test functions)
- [ ] Tests pass: `make test-ci`

**See `tests/TESTING_GUIDE.md` for comprehensive documentation.**

---

## Pull Request Process

### Before Submitting

```bash
make test-ci      # Tests pass
make lint         # Code quality
make format       # Auto-format
```

### PR Template

```markdown
## Description
Brief description

## Type
- [ ] Feature
- [ ] Bug fix
- [ ] Refactor
- [ ] Docs
- [ ] Tests

## Testing
- [ ] Tests pass (`make test-ci`)
- [ ] Added tests for new functionality

## Checklist
- [ ] Code follows style guidelines
- [ ] Updated documentation
- [ ] No breaking changes
```

---

## Development Commands

```bash
# Testing
make test-ci             # Fast CI mode (recommended)
make test                # Full suite
make test-all            # All Python versions (3.10, 3.11, 3.12)

# Code Quality
make lint                # Pylint + mypy + format-check
make format              # Auto-format with black
make type-check          # Type checking only

# Tox
tox -e py310             # Test on Python 3.10
tox -e lint              # Linting
tox -e all-checks        # All checks
```

---

## Common Patterns

### Lazy Model Initialization

```python
def __init__(self):
    self.model = None  # Lazy-loaded

def _init_model(self):
    if self.model is None:
        try:
            self.model = load_model()
            self.model.to(self.device)
        except Exception as e:
            raise ModelInitError(f"Failed: {e}") from e

def process(self, ...):
    self._init_model()  # Loads on first call
```

Thank you for contributing 🫶
