# Story 7.2: Add Serialization Failure Recovery Tests

**Status:** Complete
**Priority:** P0
**Depends on:** None
**Effort:** 1 day

---

## Problem Statement

The Test Value Audit identified a critical gap: **serialization failure recovery is untested**.

When `orjson.dumps()` fails (e.g., due to a non-serializable object in the log event), the test suite does not verify:

1. What happens to events *before* the failure in the same batch
2. What happens to events *after* the failure
3. Whether metrics correctly reflect partial success
4. Whether the pipeline remains healthy for subsequent batches
5. What error information is captured

Current tests for serialization focus on:
- Happy path (serialize valid objects)
- Raising errors for invalid objects

But they don't test **recovery behavior** — the actual production scenario where one bad event shouldn't corrupt an entire batch or crash the pipeline.

---

## Goals

1. **Verify partial batch handling** — Events before a failure should still reach sinks
2. **Verify pipeline continuity** — Subsequent batches should process normally
3. **Verify metrics accuracy** — Counts should reflect what actually succeeded/failed
4. **Verify error capture** — Failed serialization should be logged with context
5. **Verify no data loss** — Good events are never silently dropped

---

## Acceptance Criteria

- [x] Test verifies events before serialization failure reach the sink
- [x] Test verifies events after serialization failure in same batch are handled
- [x] Test verifies subsequent batches process normally after a failure
- [x] Test verifies metrics count partial successes correctly
- [x] Test verifies serialization error is captured with event context
- [x] All tests are integration-style (real pipeline, minimal mocking)

---

## Technical Approach

### Test 1: Partial Batch Success

Inject a non-serializable object mid-batch and verify events before it succeeded.

```python
import asyncio
from typing import Any

import pytest

from fapilog import get_logger


@pytest.mark.asyncio
async def test_serialization_failure_mid_batch_preserves_prior_events():
    """Events before a serialization failure should still reach the sink."""

    class NonSerializable:
        """Object that cannot be JSON serialized."""
        pass

    collected: list[dict[str, Any]] = []

    async def collecting_sink(event: dict[str, Any]) -> None:
        collected.append(event)

    logger = get_logger(name="test-serialization")
    logger._sink_write = collecting_sink  # type: ignore[attr-defined]

    # First 3 events are valid
    logger.info("event-1")
    logger.info("event-2")
    logger.info("event-3")

    # This event contains non-serializable data
    logger.info("bad-event", payload=NonSerializable())

    # These events are valid
    logger.info("event-5")
    logger.info("event-6")

    result = await logger.stop_and_drain()

    # Verify: valid events before the bad one should have succeeded
    messages = [e["message"] for e in collected]
    assert "event-1" in messages
    assert "event-2" in messages
    assert "event-3" in messages

    # Verify: the bad event should not be in collected
    assert "bad-event" not in messages

    # Verify: events after should also succeed (pipeline recovered)
    assert "event-5" in messages
    assert "event-6" in messages

    # Verify: metrics reflect reality
    assert result.submitted == 6
    assert result.processed == 5  # Exactly the good ones
    assert len(collected) == 5
```

### Test 2: Pipeline Continuity After Failure

Verify the pipeline remains healthy after a serialization failure.

```python
@pytest.mark.asyncio
async def test_pipeline_continues_after_serialization_failure():
    """Pipeline should remain healthy after a serialization failure."""

    class NonSerializable:
        pass

    collected: list[dict[str, Any]] = []

    async def collecting_sink(event: dict[str, Any]) -> None:
        collected.append(event)

    logger = get_logger(name="test-continuity")
    logger._sink_write = collecting_sink  # type: ignore[attr-defined]

    # Batch 1: Contains a bad event
    logger.info("batch1-event1")
    logger.info("batch1-bad", payload=NonSerializable())
    logger.info("batch1-event3")

    # Wait for batch to flush
    await asyncio.sleep(0.1)

    # Batch 2: All valid events (should process normally)
    logger.info("batch2-event1")
    logger.info("batch2-event2")
    logger.info("batch2-event3")

    result = await logger.stop_and_drain()

    messages = [e["message"] for e in collected]

    # Batch 1 valid events should be present
    assert "batch1-event1" in messages
    assert "batch1-event3" in messages
    assert "batch1-bad" not in messages

    # Batch 2 events should all be present (pipeline recovered)
    assert "batch2-event1" in messages
    assert "batch2-event2" in messages
    assert "batch2-event3" in messages

    # Verify exact counts
    assert len(collected) == 5  # 2 from batch1 + 3 from batch2
```

### Test 3: Metrics Accuracy

Verify metrics correctly reflect partial batch processing.

```python
@pytest.mark.asyncio
async def test_metrics_reflect_serialization_failures():
    """Metrics should accurately count successes and failures."""

    class NonSerializable:
        pass

    collected: list[dict[str, Any]] = []

    async def collecting_sink(event: dict[str, Any]) -> None:
        collected.append(event)

    logger = get_logger(name="test-metrics")
    logger._sink_write = collecting_sink  # type: ignore[attr-defined]

    # Submit 5 valid, 1 invalid, 4 valid = 10 total
    for i in range(5):
        logger.info(f"valid-{i}")

    logger.info("invalid", payload=NonSerializable())

    for i in range(4):
        logger.info(f"valid-post-{i}")

    result = await logger.stop_and_drain()

    # Submitted should be 10
    assert result.submitted == 10

    # Processed should be 9 (the valid ones)
    assert result.processed == 9

    # Exactly 1 event should have failed serialization
    assert result.submitted - result.processed == 1

    # Collected events should match processed count
    assert len(collected) == 9
```

### Test 4: Error Context Capture

Verify serialization errors include enough context to debug.

```python
from unittest.mock import patch


@pytest.mark.asyncio
async def test_serialization_error_captures_context():
    """Serialization failures should be logged with event context."""

    class NonSerializable:
        pass

    diagnostics: list[dict[str, Any]] = []

    def capture_diagnostic(component: str, message: str, **fields: Any) -> None:
        diagnostics.append({"component": component, "message": message, **fields})

    with patch("fapilog.core.diagnostics.warn", side_effect=capture_diagnostic):
        logger = get_logger(name="test-context")

        async def noop_sink(event: dict[str, Any]) -> None:
            pass

        logger._sink_write = noop_sink  # type: ignore[attr-defined]

        logger.info("bad-event", user_id="u-123", payload=NonSerializable())

        await logger.stop_and_drain()

    # Find the serialization error diagnostic
    serialization_errors = [
        d for d in diagnostics
        if "serializ" in d.get("message", "").lower()
        or d.get("component") == "serialization"
    ]

    assert len(serialization_errors) == 1

    error = serialization_errors[0]
    # Should include context about which event failed
    assert "bad-event" in str(error) or "user_id" in str(error)
```

### Test 5: Nested Non-Serializable Objects

Verify deeply nested non-serializable objects are handled.

```python
@pytest.mark.asyncio
async def test_deeply_nested_non_serializable_handled():
    """Deeply nested non-serializable objects should be caught."""

    class NonSerializable:
        pass

    collected: list[dict[str, Any]] = []

    async def collecting_sink(event: dict[str, Any]) -> None:
        collected.append(event)

    logger = get_logger(name="test-nested")
    logger._sink_write = collecting_sink  # type: ignore[attr-defined]

    # Valid event
    logger.info("valid-before")

    # Deeply nested bad object
    logger.info(
        "nested-bad",
        data={
            "level1": {
                "level2": {
                    "level3": NonSerializable()
                }
            }
        }
    )

    # Valid event
    logger.info("valid-after")

    result = await logger.stop_and_drain()

    messages = [e["message"] for e in collected]
    assert "valid-before" in messages
    assert "valid-after" in messages
    assert "nested-bad" not in messages

    # Verify exact counts
    assert len(collected) == 2
    assert result.processed == 2
```

---

## Files Changed

| File | Action |
|------|--------|
| `tests/integration/test_serialization_recovery.py` | New file |

**Lines added:** ~250

**Note:** The test file should include shared imports at the top:

```python
from __future__ import annotations

import asyncio
from typing import Any
from unittest.mock import patch

import pytest

from fapilog import get_logger
```

---

## Risk Assessment

**Risk Level: Low**

- Tests only, no production code changes
- May reveal existing bugs (which is the point)
- No coverage impact (additive)

---

## Rollback Plan

Delete `tests/integration/test_serialization_recovery.py`; no other files affected.

---

## Definition of Done

- [x] 5 integration tests for serialization failure recovery
- [x] Tests verify partial batch success
- [x] Tests verify pipeline continuity
- [x] Tests verify metrics accuracy
- [x] Tests verify error context capture
- [x] All tests pass
- [x] Tests added to CI


