# Story 7.8: Add Mutation Testing CI Integration

**Status:** POSTPONED - Not looking great from an effort to reward perspective
**Priority:** P2
**Depends on:** None (can proceed after any P0 story)
**Effort:** M (4-8 hours)

---

## Problem Statement

Line coverage tells us which code is executed by tests, but not whether tests would catch bugs in that code. A test that executes a line but doesn't assert on its behavior provides false confidence.

**Mutation testing** addresses this by:

1. Making small changes (mutations) to the source code
2. Running the test suite against each mutation
3. Reporting which mutations were "killed" (tests failed) vs "survived" (tests passed)

Surviving mutations indicate weak tests — the code changed but tests didn't notice.

**Example:**

```python
# Original code
def is_valid(count: int) -> bool:
    return count >= 0

# Mutation: change >= to >
def is_valid(count: int) -> bool:
    return count > 0  # Survives if tests only check count=5, not count=0
```

If the mutation survives, we know `count == 0` isn't tested.

---

## Goals

1. **Integrate mutation testing** into the CI pipeline
2. **Establish baseline mutation score** for critical modules
3. **Identify weak tests** through surviving mutations
4. **Provide actionable feedback** on test quality

---

## Acceptance Criteria

- [ ] Mutation testing tool (mutmut or cosmic-ray) configured
- [ ] CI job runs mutation testing on critical modules
- [ ] Baseline mutation score established for core modules
- [ ] Report generated showing surviving mutations
- [ ] Documentation for interpreting results and fixing weak tests
- [ ] Mutation testing runs in reasonable time (<10 minutes)

---

## Technical Approach

### 1. Tool Selection

**mutmut** (Recommended):

- Pure Python, no external dependencies
- Good performance with caching
- Simple configuration
- Active maintenance

**Alternative: cosmic-ray**

- More features but slower
- Requires more configuration

### 2. Configuration

**pyproject.toml:**

```toml
[project.optional-dependencies]
dev = [
    # ... existing deps ...
    "mutmut>=2.4.0",
]

[tool.mutmut]
# Target specific modules for mutation testing
paths_to_mutate = [
    "src/fapilog/core/circuit_breaker.py",
    "src/fapilog/core/serialization.py",
    "src/fapilog/plugins/redactors/field_mask.py",
    "src/fapilog/plugins/filters/sampling.py",
]

# Exclude test files and generated code
paths_to_exclude = [
    "tests/",
    "docs/",
    "examples/",
]

# Use pytest as test runner
runner = "pytest -x --tb=no -q"

# Skip mutations that are likely false positives
dict_synonyms = ["Struct", "NamedStruct"]
```

**setup.cfg (alternative):**

```ini
[mutmut]
paths_to_mutate=src/fapilog/core/
tests_dir=tests/
runner=pytest -x --tb=no -q
```

### 3. CI Integration

**GitHub Actions: `.github/workflows/mutation.yml`**

```yaml
name: Mutation Testing

on:
  # Run weekly to avoid slowing every PR
  schedule:
    - cron: "0 3 * * 0" # Sunday 3 AM UTC
  # Allow manual trigger
  workflow_dispatch:
  # Run on PRs that touch critical code
  pull_request:
    paths:
      - "src/fapilog/core/**"
      - "src/fapilog/plugins/redactors/**"

jobs:
  mutation:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          pip install -e ".[dev]"

      - name: Restore mutation cache
        uses: actions/cache@v4
        with:
          path: .mutmut-cache
          key: mutmut-${{ hashFiles('src/fapilog/core/**', 'tests/**') }}
          restore-keys: mutmut-

      - name: Run mutation testing
        run: |
          mutmut run --paths-to-mutate=src/fapilog/core/circuit_breaker.py || true

      - name: Generate report
        run: |
          mutmut results
          mutmut html
        if: always()

      - name: Upload mutation report
        uses: actions/upload-artifact@v4
        with:
          name: mutation-report
          path: html/
        if: always()

      - name: Check mutation score
        run: |
          # Get mutation score and fail if below threshold
          SCORE=$(mutmut results | grep -oP '\d+(?=% killed)')
          echo "Mutation score: ${SCORE}%"
          if [ "$SCORE" -lt 70 ]; then
            echo "::warning::Mutation score ${SCORE}% is below 70% threshold"
          fi
```

### 4. Targeted Modules

Start with high-value, well-tested modules:

| Module               | Priority | Reason                                        |
| -------------------- | -------- | --------------------------------------------- |
| `circuit_breaker.py` | High     | State machine logic, critical for reliability |
| `field_mask.py`      | High     | Security-critical redaction                   |
| `sampling.py`        | Medium   | Probability logic, easy to get wrong          |
| `serialization.py`   | Medium   | Data transformation                           |

### 5. Interpreting Results

**Killed mutation**: Test caught the change (good)

```
Mutant 1: KILLED
  src/fapilog/core/circuit_breaker.py:91
  - if self._failure_count >= self._config.failure_threshold:
  + if self._failure_count > self._config.failure_threshold:
```

**Survived mutation**: Test didn't catch the change (needs investigation)

```
Mutant 2: SURVIVED
  src/fapilog/core/circuit_breaker.py:72
  - return self._half_open_calls < self._config.half_open_max_calls
  + return self._half_open_calls <= self._config.half_open_max_calls
```

**Action for survived mutation**: Add test for boundary condition.

### 6. Script for Local Use

**scripts/run_mutation.py:**

```python
#!/usr/bin/env python3
"""
Run mutation testing on critical modules.

Usage:
    python scripts/run_mutation.py [module]
    python scripts/run_mutation.py circuit_breaker
    python scripts/run_mutation.py --all
"""

import subprocess
import sys
from pathlib import Path

CRITICAL_MODULES = [
    "src/fapilog/core/circuit_breaker.py",
    "src/fapilog/plugins/redactors/field_mask.py",
]


def run_mutmut(path: str) -> int:
    """Run mutmut on a specific path."""
    print(f"\n{'='*60}")
    print(f"Running mutation testing on: {path}")
    print('='*60)

    result = subprocess.run(
        ["mutmut", "run", f"--paths-to-mutate={path}"],
        capture_output=False,
    )

    subprocess.run(["mutmut", "results"])

    return result.returncode


def main() -> int:
    if len(sys.argv) < 2 or sys.argv[1] == "--all":
        for module in CRITICAL_MODULES:
            run_mutmut(module)
        return 0

    module_name = sys.argv[1]

    # Find matching module
    for path in CRITICAL_MODULES:
        if module_name in path:
            return run_mutmut(path)

    # Try as direct path
    if Path(module_name).exists():
        return run_mutmut(module_name)

    print(f"Unknown module: {module_name}")
    print(f"Available: {CRITICAL_MODULES}")
    return 1


if __name__ == "__main__":
    sys.exit(main())
```

---

## Files Changed

| File                             | Action                           |
| -------------------------------- | -------------------------------- |
| `pyproject.toml`                 | Add mutmut dependency and config |
| `.github/workflows/mutation.yml` | New CI workflow                  |
| `scripts/run_mutation.py`        | New script for local use         |

**Lines added:** ~150

---

## Risk Assessment

**Risk Level: Low**

- Does not modify production code
- CI job is optional (schedule/manual trigger)
- Can be disabled quickly if issues arise
- No impact on existing test suite

---

## Rollback Plan

1. Delete `.github/workflows/mutation.yml`
2. Remove mutmut from `pyproject.toml`
3. Delete `scripts/run_mutation.py`

---

## Definition of Done

- [ ] mutmut added to dev dependencies
- [ ] Configuration added to pyproject.toml
- [ ] CI workflow created (scheduled + manual)
- [ ] Baseline mutation score recorded for critical modules
- [ ] Local script for running mutation tests
- [ ] Documentation for interpreting results
- [ ] At least one module achieves 70%+ mutation score

---

## Baseline Targets

| Module               | Target Score | Notes                      |
| -------------------- | ------------ | -------------------------- |
| `circuit_breaker.py` | 70%          | State transitions critical |
| `field_mask.py`      | 80%          | Security-critical          |
| `sampling.py`        | 70%          | Probability boundaries     |

---

## Related Stories

- **7.4**: Test Assertion Strength Lint (mutation testing finds similar issues)
- **7.6**: Property-Based Tests (complements mutation testing)

---

## Future Considerations

- Integrate mutation score into PR checks (block if score drops)
- Expand to more modules as test quality improves
- Consider mutation testing for plugin author test templates
- Evaluate incremental mutation testing (only test changed code)
