# Story 4.50: Regex ReDoS Protection for RegexMaskRedactor

**Status:** Complete
**Priority:** Medium
**Depends on:** None

---

## Context / Background

The `RegexMaskRedactor` accepts user-supplied regex patterns for masking field paths. These patterns are compiled with Python's `re.compile()` and executed via `fullmatch()` at runtime.

**Problem:** Python's `re` module uses a backtracking engine susceptible to catastrophic backtracking (ReDoS). Pathological patterns like `(a+)+$` can cause exponential execution time, hanging the logging pipeline and potentially the entire application.

**Current state:** `src/fapilog/plugins/redactors/regex_mask.py:51-55` compiles patterns directly. The existing guardrails (`max_depth=16`, `max_keys_scanned=1000`) only limit traversal, not regex execution time.

**Example vulnerable pattern:**

```python
config = RegexMaskConfig(patterns=[r"(a+)+\.secret"])
# Input path: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaX.secret"
# Result: Regex engine hangs for minutes/hours
```

**Why not runtime timeouts?** The `fullmatch()` call happens per-key traversed (up to 1000 per event) multiplied by pattern count. Adding signal or thread-based timeouts per match would add unacceptable overhead. Config-time validation is the correct approach.

---

## Scope (In / Out)

### In Scope

- Add regex complexity validation at config time
- Provide clear error messages for rejected patterns
- Document safe pattern guidelines
- Optional `allow_unsafe_patterns` escape hatch for advanced users

### Out of Scope

- Runtime timeout protection (too expensive per-match)
- Replacing `re` with `re2` library (adds C dependency)
- Pattern auto-correction or suggestions

---

## Acceptance Criteria

### AC1: Reject Pathological Patterns at Config Time

**Description:** Patterns with known ReDoS-vulnerable constructs are rejected during `RegexMaskRedactor` initialization with a clear error message.

**Validation:**

```python
from fapilog.plugins.redactors.regex_mask import RegexMaskRedactor

# These should record errors during init
dangerous_patterns = [
    r"(a+)+",           # Nested quantifiers
    r"(a|a)+",          # Overlapping alternation with quantifier
    r"(.*a){10,}",      # Excessive repetition with wildcard
]

for pattern in dangerous_patterns:
    redactor = RegexMaskRedactor(config={"patterns": [pattern]})
    assert redactor._pattern_errors, f"Should reject: {pattern}"
```

### AC2: Accept Safe Patterns

**Description:** Common safe patterns used for field path matching are accepted without error.

**Validation:**

```python
# These should be accepted
safe_patterns = [
    r"user\.email",              # Literal path
    r"user\..*\.secret",         # Simple wildcard
    r"request\.headers\.[^.]+",  # Character class (no nesting)
    r"(password|secret|token)",  # Simple alternation without quantifier
]

for pattern in safe_patterns:
    redactor = RegexMaskRedactor(config={"patterns": [pattern]})
    assert not redactor._pattern_errors, f"Should accept: {pattern}"
```

### AC3: Health Check Reports Pattern Issues

**Description:** `health_check()` returns `False` if any patterns were rejected as potentially dangerous.

**Validation:**

```python
redactor = RegexMaskRedactor(config={"patterns": [r"(a+)+"]})
assert await redactor.health_check() is False
```

### AC4: Escape Hatch for Advanced Users

**Description:** Users can bypass validation with `allow_unsafe_patterns=True` if they understand the risks.

**Validation:**

```python
# Normally rejected pattern is allowed with escape hatch
redactor = RegexMaskRedactor(config={
    "patterns": [r"(a+)+"],
    "allow_unsafe_patterns": True
})
assert not redactor._pattern_errors
```

---

## Implementation Notes

### File Changes

```text
src/fapilog/plugins/redactors/regex_mask.py (MODIFIED - add validation)
tests/unit/test_regex_redactor.py (MODIFIED - add ReDoS tests)
docs/user-guide/redactors.md (MODIFIED - document safe patterns)
```

### Pattern Validation Heuristics

Detect common ReDoS patterns:

1. Nested quantifiers: `(a+)+`, `(a*)*`, `(a+)*`
2. Overlapping alternation with quantifiers: `(a|aa)+`
3. Excessive bounded repetition: `(.+){10,}`

```python
import re

REDOS_DETECTORS = [
    re.compile(r"\([^)]*[+*][^)]*\)[+*]"),  # Nested quantifiers
    re.compile(r"\([^)]*\|[^)]*\)[+*]"),     # Alternation + quantifier
    re.compile(r"\.\*[^)]*\)\{[0-9]+,"),     # Wildcard in bounded repetition
]

def is_potentially_dangerous(pattern: str) -> str | None:
    """Return reason if dangerous, None if safe."""
    for i, detector in enumerate(REDOS_DETECTORS):
        if detector.search(pattern):
            reasons = [
                "nested quantifiers detected",
                "alternation with quantifier detected",
                "wildcard in bounded repetition detected",
            ]
            return reasons[i]
    return None
```

---

## Tasks

### Phase 1: Pattern Validation

- [ ] Add `is_potentially_dangerous()` helper function
- [ ] Add `allow_unsafe_patterns` config field (default: False)
- [ ] Integrate validation into `__init__` pattern compilation
- [ ] Add rejected patterns to `_pattern_errors` with reason
- [ ] Update `health_check()` to check for dangerous pattern errors

### Phase 2: Documentation

- [ ] Document safe regex patterns in configuration guide
- [ ] Add examples of patterns to avoid
- [ ] Document the `allow_unsafe_patterns` escape hatch
- [ ] Update CHANGELOG

---

## Tests

### Unit Tests

- `tests/unit/test_regex_redactor.py`
  - `test_rejects_nested_quantifier_pattern`
  - `test_rejects_overlapping_alternation_pattern`
  - `test_rejects_wildcard_bounded_repetition`
  - `test_accepts_safe_literal_pattern`
  - `test_accepts_safe_wildcard_pattern`
  - `test_accepts_safe_character_class_pattern`
  - `test_health_check_fails_on_dangerous_pattern`
  - `test_allow_unsafe_patterns_bypasses_validation`

---

## Definition of Done

### Code Complete

- [x] All acceptance criteria implemented
- [x] Code follows project patterns
- [x] No new linting errors

### Quality Assurance

- [x] Unit tests written and passing
- [x] `ruff check` passes
- [x] `mypy` passes
- [x] No regression in existing tests

### Documentation

- [x] Code has docstrings where needed
- [x] Configuration docs updated
- [x] CHANGELOG updated

---

## Risks / Rollback

### Risks

1. **Risk:** False positives reject legitimate patterns
   - **Mitigation:** Conservative detection (obvious ReDoS only); `allow_unsafe_patterns` escape hatch

2. **Risk:** Heuristics miss some dangerous patterns
   - **Mitigation:** Document that validation is best-effort; recommend simple patterns in docs

### Rollback Plan

If issues occur:

1. Revert changes to `regex_mask.py`
2. Pattern validation is additive; removal is safe

---

## Related Stories

- **Related:** Story 4.47 - Redaction defaults alignment
- **Related:** Story 4.48 - Fallback redaction list recursion

---

## Code Review

**Date:** 2026-01-22
**Reviewer:** Claude
**Verdict:** OK to PR

### Summary

Reviewed ReDoS protection implementation for `RegexMaskRedactor`. Pattern validation at config time prevents catastrophic backtracking, with escape hatch for advanced users.

### AC Verification

| Criterion | Evidence |
|-----------|----------|
| AC1: Reject pathological patterns | `regex_mask.py:77-82` - validates patterns via `_is_potentially_dangerous()` |
| AC2: Accept safe patterns | `regex_mask.py:29-34` - detectors only match specific dangerous constructs |
| AC3: Health check reports issues | `regex_mask.py:187-194` - returns `False` if `_pattern_errors` non-empty |
| AC4: Escape hatch | `regex_mask.py:57-60,77` - `allow_unsafe_patterns` field skips validation |

### Quality Gates

- [x] ruff check passed
- [x] ruff format passed
- [x] mypy passed
- [x] diff-cover 100% on changed lines
- [x] No weak assertions
- [x] No new dead code

---

## Change Log

| Date       | Change        | Author |
|------------|---------------|--------|
| 2026-01-22 | Code review complete | Claude |
| 2025-01-22 | Initial draft | Claude |
