# Story 4.37: Plugin Error Handling Documentation

**Status:** Draft  
**Priority:** High  
**Depends on:** None  
**Effort:** 0.5 days

---

## Problem Statement

Plugin authors need clear guidance on how to handle errors. The current documentation mentions "errors should be contained" but doesn't explain:

1. **What "contained" means**: Should plugins swallow exceptions? Log them? Return defaults?
2. **When to raise vs. contain**: Are there cases where raising is appropriate?
3. **How to emit diagnostics**: The `diagnostics.warn()` API isn't documented for plugin authors
4. **Error isolation guarantees**: What does fapilog do if a plugin raises?
5. **Recovery patterns**: How should plugins implement graceful degradation?

### Current Documentation Gap

From `BaseSink` docstring:
> "Resilient: exceptions MUST be contained; never allow sink errors to crash the core pipeline."

But there's no guide explaining:
- How to contain errors properly
- What diagnostics to emit
- How to implement health checks that reflect error state
- Best practices for retry vs. fail-fast

---

## Goals

1. Create comprehensive error handling guide for plugin authors
2. Document the diagnostics API (`fapilog.core.diagnostics`)
3. Provide code examples for common error scenarios
4. Document fapilog's isolation behavior
5. Add best practices section

---

## Implementation Plan

### Phase 1: Create Documentation

**File: `docs/plugins/error-handling.md`**

```markdown
# Plugin Error Handling

This guide explains how plugins should handle errors to maintain system reliability.

## Core Principle: Contain Errors

Fapilog plugins operate in a pipeline. A failing plugin should not crash the entire
logging system. The rule is simple:

> **Plugins MUST NOT raise exceptions into the core pipeline.**

## What Does "Contain Errors" Mean?

### For Sinks

Sinks receive log entries and write them somewhere. If writing fails:

```python
class MySink:
    name = "my-sink"
    
    async def write(self, entry: dict) -> None:
        try:
            await self._client.send(entry)
        except ConnectionError as exc:
            # 1. Emit diagnostic (rate-limited automatically)
            from fapilog.core.diagnostics import warn
            warn(
                "my-sink",
                "failed to send log",
                error=str(exc),
                entry_level=entry.get("level"),
            )
            # 2. Update internal state for health check
            self._last_error = str(exc)
            self._consecutive_failures += 1
            # 3. Do NOT re-raise
```

### For Enrichers

Enrichers add fields to events. If enrichment fails, return empty dict:

```python
class MyEnricher:
    name = "my-enricher"
    
    async def enrich(self, event: dict) -> dict:
        try:
            user_info = await self._lookup_user(event.get("user_id"))
            return {"user_name": user_info.name, "user_email": user_info.email}
        except Exception as exc:
            from fapilog.core.diagnostics import warn
            warn(
                "my-enricher",
                "user lookup failed",
                error=str(exc),
                user_id=event.get("user_id"),
            )
            # Return empty dict - enrichment skipped but event continues
            return {}
```

### For Redactors

Redactors must be extra careful - a failing redactor could leak sensitive data:

```python
class MyRedactor:
    name = "my-redactor"
    
    async def redact(self, event: dict) -> dict:
        try:
            return self._apply_redaction(event)
        except Exception as exc:
            from fapilog.core.diagnostics import warn
            warn(
                "my-redactor",
                "redaction failed - blocking event",
                error=str(exc),
            )
            # Option 1: Return heavily redacted fallback
            return {"level": event.get("level"), "message": "[REDACTION_ERROR]"}
            
            # Option 2: Return original (risky if contains sensitive data)
            # return event
```

## Using the Diagnostics API

The `fapilog.core.diagnostics` module provides rate-limited warning emission:

```python
from fapilog.core.diagnostics import warn

# Basic usage
warn("component-name", "what happened")

# With structured context
warn(
    "my-sink",
    "connection failed",
    error="timeout",
    host="logs.example.com",
    attempt=3,
)

# Rate limiting (automatic, but you can hint)
warn(
    "my-sink",
    "repeated failure",
    _rate_limit_key="connection-error",  # Groups similar warnings
)
```

### Diagnostics Best Practices

1. **Use descriptive component names**: `"my-sink"` not `"sink"`
2. **Include actionable context**: error type, relevant IDs, counts
3. **Don't include sensitive data**: No passwords, tokens, PII
4. **Use rate limit keys for hot paths**: Prevents log flooding

## Fapilog's Isolation Behavior

Even if you don't contain errors perfectly, fapilog provides isolation:

### Enricher Isolation

```python
# In fapilog's enricher runner (enrich_parallel)
for enricher in enrichers:
    try:
        additions = await enricher.enrich(event)
        event.update(additions)
    except Exception:
        # Exception logged, enricher skipped, pipeline continues
        pass
```

### Sink Isolation (Fanout)

```python
# In fapilog's fanout writer
for sink in sinks:
    try:
        await sink.write(entry)
    except Exception:
        # Exception logged, sink skipped, other sinks still called
        pass
```

**However**: Relying on fapilog's isolation is a fallback. Well-behaved plugins
should contain their own errors for:

- Better diagnostics (you know what went wrong)
- Health check accuracy
- Performance (exception handling is expensive)

## Health Checks and Error State

Your health check should reflect error state:

```python
class MySink:
    name = "my-sink"
    
    def __init__(self):
        self._consecutive_failures = 0
        self._last_success_time = None
    
    async def write(self, entry: dict) -> None:
        try:
            await self._send(entry)
            self._consecutive_failures = 0
            self._last_success_time = time.time()
        except Exception:
            self._consecutive_failures += 1
    
    async def health_check(self) -> bool:
        # Unhealthy after 5 consecutive failures
        if self._consecutive_failures >= 5:
            return False
        
        # Unhealthy if no success in last 60 seconds
        if self._last_success_time:
            if time.time() - self._last_success_time > 60:
                return False
        
        return True
```

## When to Raise Exceptions

Raise exceptions only in:

1. **`__init__`**: Invalid configuration should fail fast
2. **`start()`**: Resource initialization failures (optional - can also return and mark unhealthy)

```python
class MySink:
    def __init__(self, endpoint: str):
        if not endpoint.startswith("https://"):
            raise ValueError("endpoint must use HTTPS")  # OK to raise
    
    async def start(self) -> None:
        try:
            self._client = await connect(self._endpoint)
        except ConnectionError as exc:
            # Option 1: Raise (sink won't be used)
            raise
            
            # Option 2: Mark unhealthy but don't raise
            self._connection_error = str(exc)
```

## Retry Patterns

For transient failures, consider retry with backoff:

```python
from fapilog.core.retry import AsyncRetrier, RetryConfig

class MySink:
    def __init__(self):
        self._retrier = AsyncRetrier(RetryConfig(
            max_attempts=3,
            base_delay=1.0,
            max_delay=10.0,
        ))
    
    async def write(self, entry: dict) -> None:
        try:
            await self._retrier.retry(lambda: self._send(entry))
        except Exception:
            # All retries failed, contain the error
            pass
```

## Summary

| Scenario | Action |
|----------|--------|
| Write/enrich/redact fails | Contain, emit diagnostic, continue |
| Config invalid in `__init__` | Raise exception |
| Resource init fails in `start()` | Raise or mark unhealthy |
| Transient network error | Retry with backoff, then contain |
| Repeated failures | Update health check to return False |
```

### Phase 2: Add to Plugin Documentation Index

**File: `docs/plugins/index.md`**

Add to toctree:
```markdown
error-handling
```

Add to overview:
```markdown
### [Error Handling](error-handling.md)

How to handle errors gracefully in your plugins.
```

---

## Acceptance Criteria

- [ ] `docs/plugins/error-handling.md` created with comprehensive guide
- [ ] Diagnostics API documented with examples
- [ ] Fapilog's isolation behavior explained
- [ ] Health check integration covered
- [ ] Retry patterns documented
- [ ] Added to plugins index
- [ ] Documentation builds without warnings

---

## Files Changed

| File | Change |
|------|--------|
| `docs/plugins/error-handling.md` | New file |
| `docs/plugins/index.md` | Add to toctree and overview |

