# Story 7.4: Add Test Assertion Strength Lint

**Status:** Complete
**Priority:** P0
**Depends on:** None
**Effort:** 1 day

---

## Problem Statement

The Test Value Audit found ~230+ instances of "always-true" assertions that provide no defect-prevention value:

- `>= 0` patterns: ~100 occurrences across 14 files
- `>= 1` patterns: ~130 occurrences across 34 files

```python
assert result.dropped >= 0    # Always true (counts can't be negative)
assert result.processed >= 0  # Always true
assert result.submitted >= 1  # Almost always true if anything was submitted
assert response is not None   # Often true by default
```

These assertions:
- Pass regardless of whether behavior is correct
- Create false confidence in test coverage
- Waste reviewer time (they look like real tests)

Without automated enforcement, developers will continue adding weak assertions to hit coverage.

---

## Goals

1. **Detect weak assertion patterns** in test files
2. **Fail CI** when weak assertions are added
3. **Provide clear guidance** on how to fix
4. **Be configurable** to allow legitimate uses (with justification)

---

## Acceptance Criteria

- [ ] Script detects `>= 0` assertions on numeric results
- [ ] Script detects `>= 1` assertions where exact count is knowable
- [ ] Script detects `is not None` assertions without behavioral follow-up
- [ ] Script runs in CI and fails on violations
- [ ] Existing violations documented and tracked for cleanup
- [ ] Escape hatch available for legitimate edge cases (with comment)

---

## Technical Approach

### Option A: Custom Ruff/Flake8 Rule (Recommended)

Create a custom linter rule that integrates with existing tooling.

**File: `scripts/lint_test_assertions.py`**

```python
#!/usr/bin/env python3
"""
Lint test files for weak assertion patterns.

Detects:
- assert x >= 0 (always true for unsigned values)
- assert x >= 1 (often true without testing)
- assert x is not None (without behavioral assertion)

Usage:
    python scripts/lint_test_assertions.py tests/
    python scripts/lint_test_assertions.py tests/unit/test_core_logger.py
"""

import ast
import sys
from pathlib import Path
from dataclasses import dataclass


@dataclass
class Violation:
    file: Path
    line: int
    code: str
    message: str
    suggestion: str


class WeakAssertionVisitor(ast.NodeVisitor):
    """AST visitor to detect weak assertion patterns."""

    def __init__(self, filepath: Path, source_lines: list[str]):
        self.filepath = filepath
        self.source_lines = source_lines
        self.violations: list[Violation] = []
        self._current_function: str | None = None

    def visit_FunctionDef(self, node: ast.FunctionDef) -> None:
        self._current_function = node.name
        self.generic_visit(node)
        self._current_function = None

    def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> None:
        self._current_function = node.name
        self.generic_visit(node)
        self._current_function = None

    def visit_Assert(self, node: ast.Assert) -> None:
        self._check_comparison(node)
        self._check_is_not_none(node)
        self.generic_visit(node)
    
    def _check_comparison(self, node: ast.Assert) -> None:
        """Check for >= 0 or >= 1 patterns."""
        test = node.test
        
        if not isinstance(test, ast.Compare):
            return
        
        # Check for x >= 0 or x >= 1
        if len(test.ops) == 1 and isinstance(test.ops[0], ast.GtE):
            if len(test.comparators) == 1:
                right = test.comparators[0]
                
                # Check for >= 0
                if isinstance(right, ast.Constant) and right.value == 0:
                    self._add_violation(
                        node,
                        "WA001",
                        "Assertion `>= 0` is always true for non-negative values",
                        "Use `== expected_value` with a computed expected value",
                    )
                
                # Check for >= 1
                elif isinstance(right, ast.Constant) and right.value == 1:
                    self._add_violation(
                        node,
                        "WA002",
                        "Assertion `>= 1` may be too weak if exact count is known",
                        "Consider `== expected_count` if the count is deterministic",
                    )
    
    def _check_is_not_none(self, node: ast.Assert) -> None:
        """Check for bare `is not None` assertions."""
        test = node.test
        
        if isinstance(test, ast.Compare):
            if len(test.ops) == 1 and isinstance(test.ops[0], ast.IsNot):
                if len(test.comparators) == 1:
                    right = test.comparators[0]
                    if isinstance(right, ast.Constant) and right.value is None:
                        # Check if there's a behavioral assertion after
                        # This is a heuristic - we flag standalone `is not None`
                        self._add_violation(
                            node,
                            "WA003",
                            "Assertion `is not None` should be followed by behavioral check",
                            "Add assertion on the actual value (e.g., assert x.field == 'expected')",
                        )
    
    def _has_noqa(self, lineno: int, code: str) -> bool:
        """Check if line has a noqa comment suppressing this code."""
        if lineno < 1 or lineno > len(self.source_lines):
            return False
        line = self.source_lines[lineno - 1]
        # Check for # noqa: WA001 or # noqa (blanket)
        if "# noqa" in line:
            if f"# noqa: {code}" in line or "# noqa:" not in line:
                return True
        return False

    def _add_violation(
        self, node: ast.Assert, code: str, message: str, suggestion: str
    ) -> None:
        # Check for suppression comment
        if self._has_noqa(node.lineno, code):
            return

        self.violations.append(
            Violation(
                file=self.filepath,
                line=node.lineno,
                code=code,
                message=message,
                suggestion=suggestion,
            )
        )


def lint_file(filepath: Path) -> list[Violation]:
    """Lint a single file for weak assertions."""
    try:
        source = filepath.read_text()
        tree = ast.parse(source)
        source_lines = source.splitlines()
    except SyntaxError:
        return []

    visitor = WeakAssertionVisitor(filepath, source_lines)
    visitor.visit(tree)
    return visitor.violations


def lint_directory(directory: Path) -> list[Violation]:
    """Lint all test files in a directory."""
    violations = []
    
    for filepath in directory.rglob("test_*.py"):
        violations.extend(lint_file(filepath))
    
    return violations


def main() -> int:
    if len(sys.argv) < 2:
        print("Usage: python lint_test_assertions.py <path>")
        return 1
    
    path = Path(sys.argv[1])
    
    if path.is_file():
        violations = lint_file(path)
    elif path.is_dir():
        violations = lint_directory(path)
    else:
        print(f"Error: {path} is not a file or directory")
        return 1
    
    if not violations:
        print("No weak assertions found ✓")
        return 0
    
    print(f"Found {len(violations)} weak assertion(s):\n")
    
    for v in violations:
        print(f"{v.file}:{v.line}: [{v.code}] {v.message}")
        print(f"  Suggestion: {v.suggestion}")
        print()
    
    return 1


if __name__ == "__main__":
    sys.exit(main())
```

### Integration with CI

**Update `tox.ini`:**

```ini
[testenv]
commands =
    ruff check .
    mypy --config-file pyproject.toml --explicit-package-bases src/
    python scripts/lint_test_assertions.py tests/  # Add this line
    pytest tests/ --cov=src/fapilog --cov-report=term-missing --cov-fail-under=90
```

**Or add to pre-commit (append to existing local hooks section):**

```yaml
# .pre-commit-config.yaml - add to existing "repo: local" section
      - id: lint-test-assertions
        name: Lint weak test assertions
        entry: python scripts/lint_test_assertions.py tests/
        language: system
        pass_filenames: false
        always_run: true
        stages: [pre-commit]
        description: "Detect weak assertion patterns (>= 0, >= 1, is not None)"
```

### Suppression Mechanism

Allow legitimate uses with a comment:

```python
# This is legitimately checking that the system doesn't go negative
assert result.error_count >= 0  # noqa: WA001 - verifying no underflow
```

The linter should respect `# noqa: WA00X` comments.

### Baseline File

For existing violations, create a baseline to track cleanup progress:

**File: `.weak-assertion-baseline.txt`**

```text
# Weak assertion baseline - track cleanup progress
# Generated: 2026-01-07
# Total: ~230 violations (WA001: ~100, WA002: ~130)

tests/unit/test_core_logger_priority1_coverage.py:69  # WA001
tests/unit/test_core_logger_priority1_coverage.py:70  # WA001
tests/unit/test_core_logger_priority2_coverage.py:45  # WA002
# ... etc
```

The lint script should:
1. Report new violations not in baseline
2. Report when baseline violations are fixed (for cleanup tracking)

---

## Violation Codes

| Code | Pattern | Severity | Example |
|------|---------|----------|---------|
| WA001 | `assert x >= 0` | Error | `assert result.dropped >= 0` |
| WA002 | `assert x >= 1` | Warning | `assert result.processed >= 1` |
| WA003 | `assert x is not None` (standalone) | Warning | `assert response is not None` |

---

## Files Changed

| File | Action |
|------|--------|
| `scripts/lint_test_assertions.py` | New file |
| `tox.ini` | Add lint command |
| `.pre-commit-config.yaml` | Add hook |
| `.weak-assertion-baseline.txt` | New file (174 violations baselined) |
| `tests/unit/test_lint_test_assertions.py` | New file (17 unit tests) |

---

## Rollout Plan

### Phase 1: Baseline Mode (Day 1)

1. Create the lint script
2. Run against entire test suite
3. Generate baseline file with all existing violations
4. Configure CI to run lint but only fail on *new* violations

### Phase 2: Cleanup Mode (Ongoing)

1. As Story 7.1 progresses, violations are fixed
2. Baseline file is updated to remove fixed violations
3. CI prevents new violations from being added

### Phase 3: Strict Mode (After Story 7.1)

1. Remove baseline file
2. CI fails on any violation
3. All assertions are strong

---

## Risk Assessment

**Risk Level: Low**

- Does not modify production code
- Does not modify existing tests (initially)
- Baseline mode prevents blocking existing PRs
- Can be disabled quickly if issues arise

---

## Rollback Plan

1. Remove `python scripts/lint_test_assertions.py tests/` line from `tox.ini`
2. Remove lint hook from `.pre-commit-config.yaml` (if added)
3. Optionally delete `scripts/lint_test_assertions.py` and `.weak-assertion-baseline.txt`

No other files affected; existing tests continue to pass.

---

## Definition of Done

- [x] Lint script detects WA001, WA002, WA003 patterns
- [x] Lint script supports `# noqa:` suppression
- [x] Lint runs in CI (tox or pre-commit)
- [x] Baseline file generated for existing violations (174 violations)
- [x] New violations block PRs
- [x] Documentation added for developers (script docstrings + --help)
- [x] Script has unit tests (17 tests)


