# Story 4.46: Fallback Sink PII Protection

**Status:** Complete
**Priority:** High
**Depends on:** None

---

## Context / Background

An external audit identified a security concern with the stderr fallback behavior:

1. **Fallback writes unredacted payloads**: When a primary sink fails, `src/fapilog/plugins/sinks/fallback.py:42` writes the full payload to stderr:
   ```python
   def _write_to_stderr(payload: Any, *, serialized: bool) -> None:
       text = _format_payload(payload, serialized=serialized)
       sys.stderr.write(text)
       sys.stderr.flush()
   ```

2. **Default redactors is empty**: `src/fapilog/core/settings.py:688` defaults `redactors` to an empty list:
   ```python
   redactors: list[str] = Field(
       default_factory=list,
       description=("Redactor plugins to use (by name); empty to disable"),
   )
   ```

This creates a scenario where sensitive data (passwords, tokens, PII) can leak to stderr when:
- A sink fails (network error, disk full, etc.)
- The user hasn't configured redactors (common in development)
- The fallback mechanism triggers

Stderr often ends up in container logs, system journals, or monitoring systems that may not have the same access controls as the primary log destination.

---

## Scope (In / Out)

### In Scope

- Add diagnostic warning when fallback triggers with no redactors configured
- Add `fallback_redact_mode` setting: `"inherit"` (use pipeline redactors), `"minimal"` (built-in sensitive field list), `"none"` (current behavior)
- Default `fallback_redact_mode` to `"minimal"` for safe defaults
- Add built-in minimal redaction for common sensitive fields (password, secret, token, api_key, authorization)
- Document the security implications of fallback behavior

### Out of Scope

- Making redactors non-empty by default (breaking change, separate discussion)
- Encrypting stderr output (impractical)
- Fallback to encrypted file instead of stderr (enterprise feature)

---

## Acceptance Criteria

### AC1: Warning When Fallback Triggers Without Redaction

**Description:** When fallback writes to stderr and no redaction is configured, emit a diagnostic warning.

**Validation:**
```python
# When fallback triggers with redactors=[] and fallback_redact_mode="none":
# Diagnostic: "WARN [sink] fallback triggered without redaction configured"
```

### AC2: Minimal Redaction Mode

**Description:** `fallback_redact_mode="minimal"` applies built-in field masking to fallback output.

**Validation:**
```python
# Input payload with sensitive data:
payload = {"user": "alice", "password": "secret123", "api_key": "sk-xxx"}

# Output to stderr with fallback_redact_mode="minimal":
# {"user": "alice", "password": "***", "api_key": "***"}
```

### AC3: Inherit Mode Uses Pipeline Redactors

**Description:** `fallback_redact_mode="inherit"` applies the same redactors configured in the pipeline.

**Validation:**
```python
settings = Settings(
    core={"redactors": ["field-mask"]},
    redactor_config={"field_mask": {"fields_to_mask": ["ssn"]}},
)
# Fallback output should also mask "ssn" field
```

### AC4: None Mode Preserves Current Behavior

**Description:** `fallback_redact_mode="none"` writes unredacted payload (opt-in to current behavior).

**Validation:**
```python
settings = Settings(core={"fallback_redact_mode": "none"})
# Fallback writes full payload to stderr (current behavior)
# Diagnostic warning still emitted
```

### AC5: Default is Minimal

**Description:** Without explicit configuration, fallback uses minimal built-in redaction.

**Validation:**
```python
settings = Settings()
assert settings.core.fallback_redact_mode == "minimal"
```

---

## Implementation Notes

### File Changes

```
src/fapilog/plugins/sinks/fallback.py (MODIFIED - add redaction)
src/fapilog/core/settings.py (MODIFIED - add fallback_redact_mode)
src/fapilog/core/defaults.py (MODIFIED - add SENSITIVE_FIELD_DEFAULTS)
tests/unit/sinks/test_fallback.py (MODIFIED - new test cases)
docs/user-guide/security.md (MODIFIED - document fallback behavior)
```

### Minimal Sensitive Fields List

```python
# src/fapilog/core/defaults.py
FALLBACK_SENSITIVE_FIELDS = frozenset({
    "password",
    "passwd",
    "secret",
    "token",
    "api_key",
    "apikey",
    "api_secret",
    "apisecret",
    "authorization",
    "auth",
    "credential",
    "credentials",
    "private_key",
    "privatekey",
    "access_token",
    "refresh_token",
})
```

### Fallback Redaction Logic

```python
def _minimal_redact(payload: dict[str, Any]) -> dict[str, Any]:
    """Apply minimal redaction for fallback safety."""
    result = {}
    for key, value in payload.items():
        if key.lower() in FALLBACK_SENSITIVE_FIELDS:
            result[key] = "***"
        elif isinstance(value, dict):
            result[key] = _minimal_redact(value)
        else:
            result[key] = value
    return result
```

---

## Tasks

### Phase 1: Core Implementation

- [ ] Add `fallback_redact_mode` to `CoreSettings`
- [ ] Add `FALLBACK_SENSITIVE_FIELDS` constant
- [ ] Implement `_minimal_redact()` function
- [ ] Update `_write_to_stderr()` to apply redaction based on mode
- [ ] Add diagnostic warning for unredacted fallback

### Phase 2: Integration

- [ ] Pass redactor context to fallback handler
- [ ] Implement "inherit" mode using pipeline redactors
- [ ] Handle edge cases (redactor failures, nested objects)

### Phase 3: Testing & Documentation

- [ ] Add unit tests for all redaction modes
- [ ] Add integration test for fallback with redaction
- [ ] Document fallback security implications
- [ ] Update CHANGELOG

---

## Tests

### Unit Tests

- `tests/unit/sinks/test_fallback.py`
  - `test_minimal_redact_masks_sensitive_fields`
  - `test_minimal_redact_handles_nested_dicts`
  - `test_fallback_default_mode_is_minimal`
  - `test_fallback_none_mode_no_redaction`
  - `test_fallback_inherit_mode_uses_pipeline`
  - `test_diagnostic_warning_on_unredacted_fallback`

### Integration Tests

- `tests/integration/test_fallback_redaction.py`
  - End-to-end test with sink failure triggering redacted fallback

---

## Definition of Done

### Code Complete

- [ ] `fallback_redact_mode` setting implemented
- [ ] Minimal redaction applied by default
- [ ] All three modes working correctly
- [ ] Diagnostic warning on unredacted fallback

### Quality Assurance

- [ ] Unit tests written and passing
- [ ] `ruff check` passes
- [ ] `mypy` passes
- [ ] No regression in existing tests

### Documentation

- [ ] Security implications documented
- [ ] Configuration options documented
- [ ] CHANGELOG updated

---

## Risks / Rollback

### Risks

1. **Risk:** Performance impact from redaction on every fallback
   - **Mitigation:** Minimal redaction is simple dict traversal; fallback is rare path

2. **Risk:** False negatives (sensitive fields not in default list)
   - **Mitigation:** Document that minimal is a safety net, not comprehensive; recommend proper redactor config

3. **Risk:** Breaking change for users parsing stderr output
   - **Mitigation:** Add `fallback_redact_mode="none"` for opt-out

### Rollback Plan

If issues occur:
1. Users can set `fallback_redact_mode="none"` to restore current behavior
2. Minimal redaction is additive; doesn't affect primary pipeline

---

## Related Stories

- **Related:** Story 4.45 - Webhook secure defaults (security theme)
- **Related:** Story 3.5 - Plugin allowlist secure default (security defaults theme)

---

## Code Review

**Date:** 2026-01-18
**Reviewer:** Claude
**Verdict:** OK to PR

### Summary

Reviewed implementation of fallback PII protection with three redaction modes (minimal, inherit, none). All acceptance criteria verified against committed code.

### AC Verification

| Criterion | Evidence |
|-----------|----------|
| AC1: Warning on unredacted fallback | `fallback.py:109-117` - emits diagnostic warning when `redact_mode == "none"` |
| AC2: Minimal redaction mode | `fallback.py:14-31` - `minimal_redact()` masks FALLBACK_SENSITIVE_FIELDS |
| AC3: Inherit mode uses pipeline | `fallback.py:85-86` - passes through (pipeline already redacted) |
| AC4: None mode preserves behavior | `fallback.py:82-86` - no redaction applied |
| AC5: Default is minimal | `settings.py:777` - `default="minimal"` |

### Quality Gates

- [x] ruff check passed
- [x] ruff format passed
- [x] mypy passed
- [x] diff-cover >= 90% (93%)
- [x] No weak assertions
- [x] No dead code

---

## Change Log

| Date | Change | Author |
|------|--------|--------|
| 2026-01-18 | Initial draft from audit findings | Claude |
| 2026-01-18 | Implementation complete, code review passed | Claude |
