# Testing Guide

## Running Tests

### All Tests
```bash
cd backend
pytest
```

### Unit Tests Only
```bash
pytest tests/unit/ -v
```

### Integration Tests
```bash
pytest tests/integration/ -v
```

### Performance Benchmarks
```bash
pytest tests/performance/ -v --benchmark-only
```

### With Coverage
```bash
pytest --cov=app --cov-report=html
# Open htmlcov/index.html to view coverage report
```

## Test Categories

### Unit Tests (`tests/unit/`)
- **test_agents.py**: AI agent functionality
- **test_storage.py**: File storage, metadata, lineage
- **test_transformations.py**: Data transformations
- **test_quality.py**: Profiling and quality scanning

### Integration Tests (`tests/integration/`)
- **test_e2e.py**: End-to-end pipeline flows

### Performance Tests (`tests/performance/`)
- **test_benchmarks.py**: Speed and memory benchmarks

## Test Fixtures

Common fixtures are in `tests/conftest.py`:
- `test_settings`: Test configuration
- `sample_dataframe`: Sample data for testing
- `sample_csv_file`: Temporary CSV file
- `metadata_store`: DuckDB metadata store
- `file_storage`: File storage handler

## Writing Tests

### Unit Test Example
```python
def test_drop_duplicates(sample_dataframe):
    """Test dropping duplicate rows."""
    from app.transformations.cleaning import drop_duplicates_pandas
    
    result = drop_duplicates_pandas(sample_dataframe)
    
    assert len(result) < len(sample_dataframe)
```

### Async Test Example
```python
@pytest.mark.asyncio
async def test_pipeline_execution():
    """Test async pipeline execution."""
    orchestrator = AgentOrchestrator()
    result = await orchestrator.execute_pipeline(input_data)
    
    assert result.success
```

### Benchmark Example
```python
def test_performance(benchmark):
    """Benchmark operation performance."""
    result = benchmark(expensive_function, arg1, arg2)
    assert result is not None
```

## Coverage Goals

- **Target**: 70%+ overall coverage
- **Unit Tests**: 80%+ coverage for core modules
- **Integration Tests**: Cover critical user flows
- **Performance Tests**: Establish baseline metrics

## Continuous Integration

Tests run automatically on:
- Every commit (unit tests)
- Pull requests (all tests)
- Nightly builds (including benchmarks)

## Troubleshooting

### Tests Fail Locally
```bash
# Clear cache
pytest --cache-clear

# Run single test
pytest tests/unit/test_storage.py::TestFileStorage::test_read_csv -v
```

### Coverage Report Not Generated
```bash
# Install coverage dependencies
pip install pytest-cov

# Generate report
pytest --cov=app --cov-report=html
```

### Slow Tests
```bash
# Run fast tests only
pytest -m "not slow"

# Profile slow tests
pytest --durations=10
```
