# Story 5.4: Plugin Testing Guide and Fixtures (Remaining Work)

## Status: Complete

## Priority: Medium

## Estimated Effort: Small (1-2 days)

## Dependencies: Story 5.2 (Filter plugin type)

## Epic: Developer Experience

---

## Context

The `fapilog.testing` module was implemented in **Story 4.27** and provides core testing utilities. However, several items from the original scope remain unimplemented:

### Already Implemented (Story 4.27)

| Component                     | Location                | Status      |
| ----------------------------- | ----------------------- | ----------- |
| `MockSink`                    | `testing/mocks.py`      | ✅ Complete |
| `MockEnricher`                | `testing/mocks.py`      | ✅ Complete |
| `MockRedactor`                | `testing/mocks.py`      | ✅ Complete |
| `MockProcessor`               | `testing/mocks.py`      | ✅ Complete |
| `validate_sink()`             | `testing/validators.py` | ✅ Complete |
| `validate_enricher()`         | `testing/validators.py` | ✅ Complete |
| `validate_redactor()`         | `testing/validators.py` | ✅ Complete |
| `validate_processor()`        | `testing/validators.py` | ✅ Complete |
| `validate_plugin_lifecycle()` | `testing/validators.py` | ✅ Complete |
| `create_log_event()`          | `testing/factories.py`  | ✅ Complete |
| `create_batch_events()`       | `testing/factories.py`  | ✅ Complete |

### Remaining Work (This Story)

| Component                 | Status     | Notes                               |
| ------------------------- | ---------- | ----------------------------------- |
| `MockFilter`              | ❌ Missing | Story 5.2 added filters but no mock |
| `validate_filter()`       | ❌ Missing | No filter validator                 |
| pytest fixtures           | ❌ Missing | No `@pytest.fixture` decorators     |
| `benchmarks.py`           | ❌ Missing | No performance testing utilities    |
| `docs/plugins/testing.md` | ❌ Missing | No dedicated testing guide          |

---

## Problem Statement

1. **No MockFilter**: Story 5.2 added the Filter plugin type, but `fapilog.testing` doesn't include a `MockFilter` for testing filter chains.
2. **No pytest fixtures**: Users must manually instantiate mocks; fixtures would simplify test setup.
3. **No benchmarks**: No utilities for measuring plugin performance overhead.
4. **No documentation**: No dedicated guide for testing plugins.

---

## Acceptance Criteria

### AC1: MockFilter and validate_filter()

- [ ] Add `MockFilter` to `fapilog.testing.mocks`
- [ ] Add `MockFilterConfig` with configurable drop behavior
- [ ] Add `validate_filter()` to `fapilog.testing.validators`
- [ ] Export from `fapilog.testing.__init__`

### AC2: pytest Fixtures Module

- [ ] Create `src/fapilog/testing/fixtures.py`
- [ ] Add `@pytest.fixture` for `mock_sink`, `mock_enricher`, `mock_redactor`, `mock_processor`, `mock_filter`
- [ ] Add `started_mock_sink` async fixture with cleanup
- [ ] Add `assert_valid_*` helper functions
- [ ] Document fixture usage in docstrings

### AC3: Performance Benchmarks

- [ ] Create `src/fapilog/testing/benchmarks.py`
- [ ] Add `BenchmarkResult` dataclass
- [ ] Add `benchmark_async()` for generic async benchmarking
- [ ] Add `benchmark_sink()`, `benchmark_enricher()`, `benchmark_filter()` helpers
- [ ] Export from `fapilog.testing.__init__`

### AC4: Documentation

- [ ] Create `docs/plugins/testing.md`
- [ ] Document all mocks with examples
- [ ] Document validators with examples
- [ ] Document pytest fixtures usage
- [ ] Document benchmark utilities
- [ ] Add CI/CD integration guidance

---

## Technical Design

### 1. MockFilter

```python
# src/fapilog/testing/mocks.py (addition)

@dataclass
class MockFilterConfig:
    """Configuration for mock filter."""
    drop_levels: list[str] = field(default_factory=list)  # Levels to drop
    drop_rate: float = 0.0  # Probability of dropping (0-1)
    fail_on_call: int | None = None  # Fail on Nth call
    latency_seconds: float = 0.0


class MockFilter:
    """Mock filter for testing filter chains."""

    name = "mock"

    def __init__(self, config: MockFilterConfig | None = None) -> None:
        self._config = config or MockFilterConfig()
        self.filtered_events: list[dict[str, Any]] = []
        self.dropped_events: list[dict[str, Any]] = []
        self.call_count: int = 0
        self.start_called: bool = False
        self.stop_called: bool = False

    async def start(self) -> None:
        self.start_called = True

    async def stop(self) -> None:
        self.stop_called = True

    async def filter(self, event: dict[str, Any]) -> dict[str, Any] | None:
        self.call_count += 1

        if self._config.fail_on_call is not None:
            if self.call_count == self._config.fail_on_call:
                raise RuntimeError("Mock filter failure")

        if self._config.latency_seconds > 0:
            await asyncio.sleep(self._config.latency_seconds)

        level = str(event.get("level", "")).upper()
        if level in self._config.drop_levels:
            self.dropped_events.append(event.copy())
            return None

        if self._config.drop_rate > 0:
            import random
            if random.random() < self._config.drop_rate:
                self.dropped_events.append(event.copy())
                return None

        self.filtered_events.append(event.copy())
        return event

    async def health_check(self) -> bool:
        return True

    def reset(self) -> None:
        self.filtered_events.clear()
        self.dropped_events.clear()
        self.call_count = 0
        self.start_called = False
        self.stop_called = False
```

### 2. validate_filter()

```python
# src/fapilog/testing/validators.py (addition)

def validate_filter(plugin: Any) -> ValidationResult:
    """Validate a filter plugin conforms to BaseFilter protocol."""
    errors: list[str] = []

    # Check name attribute
    if not hasattr(plugin, "name"):
        errors.append("Missing 'name' attribute")
    elif not isinstance(plugin.name, str):
        errors.append("'name' must be a string")

    # Check required methods
    for method in ("start", "stop", "filter", "health_check"):
        if not hasattr(plugin, method):
            errors.append(f"Missing '{method}' method")
        elif not callable(getattr(plugin, method)):
            errors.append(f"'{method}' must be callable")

    # Check filter signature
    if hasattr(plugin, "filter"):
        sig = inspect.signature(plugin.filter)
        params = list(sig.parameters.keys())
        if "event" not in params and len(params) < 2:
            errors.append("filter() must accept 'event' parameter")

    return ValidationResult(valid=len(errors) == 0, errors=errors)
```

### 3. pytest Fixtures

```python
# src/fapilog/testing/fixtures.py

from __future__ import annotations

from typing import AsyncIterator

import pytest

from .mocks import (
    MockSink,
    MockEnricher,
    MockRedactor,
    MockProcessor,
    MockFilter,
)


@pytest.fixture
def mock_sink() -> MockSink:
    """Provide a fresh MockSink for testing."""
    return MockSink()


@pytest.fixture
def mock_enricher() -> MockEnricher:
    """Provide a fresh MockEnricher for testing."""
    return MockEnricher()


@pytest.fixture
def mock_redactor() -> MockRedactor:
    """Provide a fresh MockRedactor for testing."""
    return MockRedactor()


@pytest.fixture
def mock_processor() -> MockProcessor:
    """Provide a fresh MockProcessor for testing."""
    return MockProcessor()


@pytest.fixture
def mock_filter() -> MockFilter:
    """Provide a fresh MockFilter for testing."""
    return MockFilter()


@pytest.fixture
async def started_mock_sink() -> AsyncIterator[MockSink]:
    """Provide a started MockSink with automatic cleanup."""
    sink = MockSink()
    await sink.start()
    yield sink
    await sink.stop()
```

### 4. Benchmarks Module

```python
# src/fapilog/testing/benchmarks.py

from __future__ import annotations

import time
from dataclasses import dataclass
from typing import Any, Awaitable, Callable


@dataclass
class BenchmarkResult:
    """Results from a plugin benchmark."""

    name: str
    iterations: int
    total_seconds: float
    ops_per_second: float
    avg_latency_ms: float
    min_latency_ms: float
    max_latency_ms: float

    def __str__(self) -> str:
        return (
            f"{self.name}: {self.ops_per_second:.0f} ops/s, "
            f"avg={self.avg_latency_ms:.3f}ms"
        )


async def benchmark_async(
    name: str,
    fn: Callable[..., Awaitable[Any]],
    *args: Any,
    iterations: int = 1000,
    warmup: int = 100,
    **kwargs: Any,
) -> BenchmarkResult:
    """Benchmark an async function with warmup."""
    # Warmup
    for _ in range(warmup):
        await fn(*args, **kwargs)

    # Measure
    latencies: list[float] = []
    start_total = time.perf_counter()

    for _ in range(iterations):
        start = time.perf_counter()
        await fn(*args, **kwargs)
        latencies.append((time.perf_counter() - start) * 1000)

    total_seconds = time.perf_counter() - start_total

    return BenchmarkResult(
        name=name,
        iterations=iterations,
        total_seconds=total_seconds,
        ops_per_second=iterations / total_seconds,
        avg_latency_ms=sum(latencies) / len(latencies),
        min_latency_ms=min(latencies),
        max_latency_ms=max(latencies),
    )


async def benchmark_sink(sink: Any, iterations: int = 1000) -> BenchmarkResult:
    """Benchmark a sink's write performance."""
    entry = {"level": "INFO", "message": "benchmark"}
    await sink.start()
    try:
        return await benchmark_async(
            f"sink:{getattr(sink, 'name', 'unknown')}",
            sink.write,
            entry,
            iterations=iterations,
        )
    finally:
        await sink.stop()


async def benchmark_filter(filter_plugin: Any, iterations: int = 1000) -> BenchmarkResult:
    """Benchmark a filter's filter performance."""
    event = {"level": "INFO", "message": "benchmark"}
    await filter_plugin.start()
    try:
        return await benchmark_async(
            f"filter:{getattr(filter_plugin, 'name', 'unknown')}",
            filter_plugin.filter,
            event,
            iterations=iterations,
        )
    finally:
        await filter_plugin.stop()
```

---

## Test Plan

### Unit Tests

1. **test_mock_filter.py**

   - Test MockFilter passes events
   - Test MockFilter drops by level
   - Test MockFilter drops by rate
   - Test error injection works

2. **test_validate_filter.py**

   - Test validate_filter passes for valid
   - Test validate_filter fails for invalid
   - Test filter signature validation

3. **test_fixtures.py**

   - Test fixtures return fresh instances
   - Test started_mock_sink lifecycle

4. **test_benchmarks.py**
   - Test benchmark_async returns results
   - Test warmup is excluded
   - Test benchmark_sink/filter work

---

## Documentation Structure

```
docs/plugins/testing.md
├── Quick Start
├── Mock Plugins
│   ├── MockSink
│   ├── MockEnricher
│   ├── MockRedactor
│   ├── MockProcessor
│   └── MockFilter
├── Validators
│   ├── validate_sink()
│   ├── validate_enricher()
│   ├── validate_filter()
│   └── validate_plugin_lifecycle()
├── pytest Fixtures
├── Benchmarks
└── CI/CD Integration
```

---

## Rollout Plan

1. Add `MockFilter` and `MockFilterConfig` to `mocks.py`
2. Add `validate_filter()` to `validators.py`
3. Create `fixtures.py` with pytest fixtures
4. Create `benchmarks.py` with performance utilities
5. Update `__init__.py` exports
6. Create `docs/plugins/testing.md`
7. Add unit tests
8. Release

---

## Related Stories

- Story 4.27: Plugin testing utilities (foundation - completed)
- Story 5.2: Filter plugin type (prerequisite - completed)
