# Story 7.10: Add Worker/Logger Lifecycle Coverage Tests

**Status:** Complete
**Priority:** P1
**Depends on:** None
**Effort:** 2-3 days

---

## Problem Statement

The Code Coverage Report identified `worker.py` (80.15%) and `logger.py` (84.27%) as having significant coverage gaps, particularly in lifecycle and error-handling paths.

These are core components where untested paths represent real production risk:

1. **Shutdown/drain ordering** - The sequence of stopping workers, draining queues, and stopping plugins is critical for data integrity
2. **Cancellation paths** - `CancelledError` handling in worker loops
3. **Sink write failures mid-batch** - What happens when a sink fails partway through a batch
4. **Thread-loop mode** - The dedicated background thread path for sync contexts
5. **Backpressure edge cases** - Same-thread vs cross-thread enqueue failure paths

Current tests exercise the happy path but don't verify recovery behavior in these scenarios.

---

## Goals

1. **Verify shutdown ordering** - Plugins stop in correct order after workers drain
2. **Verify cancellation handling** - Worker loop handles `CancelledError` gracefully
3. **Verify partial batch failures** - Events before sink failure are counted correctly
4. **Verify thread-loop mode** - Background thread mode works correctly for sync callers
5. **Verify backpressure diagnostics** - Drop events emit appropriate warnings

---

## Acceptance Criteria

### AC1: Worker Shutdown Ordering

- [x] Test verifies `stop_plugins()` calls plugins in reverse registration order
- [x] Test verifies plugin stop errors are contained (don't block other plugins)
- [x] Test verifies diagnostics are emitted when plugin stop fails

### AC2: Worker Cancellation Handling

- [x] Test verifies `CancelledError` exits worker loop cleanly
- [x] Test verifies queue is not corrupted after cancellation
- [x] Test verifies metrics reflect events processed before cancellation

### AC3: Sink Write Failure Mid-Batch

- [x] Test verifies batch-level exception increments `dropped` counter
- [x] Test verifies `_record_sink_error()` captures sink name
- [x] Test verifies `_emit_sink_flush_error()` emits diagnostic with error context
- [x] Test verifies subsequent batches process normally after sink failure

### AC4: Serialization Fallback Paths

- [x] Test verifies strict envelope mode drops unserializable events
- [x] Test verifies best-effort mode falls back to `serialize_mapping_to_json_bytes`
- [x] Test verifies diagnostics include mode (strict/best-effort) and error type

### AC5: Backpressure and Enqueue Edge Cases

- [x] Test verifies same-thread enqueue drops with diagnostic when queue full
- [x] Test verifies cross-thread enqueue timeout path
- [x] Test verifies `drop_on_full=False` waits indefinitely (with timeout in test)
- [x] Test verifies `_queue_high_watermark` updates correctly

### AC6: Thread-Loop Mode

- [x] Test verifies sync logger creates background thread when no event loop
- [x] Test verifies thread cleanup on drain
- [x] Test verifies timeout warning when thread doesn't join

---

## Technical Approach

### Test Files

Create new test file: `tests/integration/test_worker_lifecycle.py`

### Test 1: Plugin Stop Ordering

```python
@pytest.mark.asyncio
async def test_stop_plugins_reverse_order():
    """Plugins should stop in reverse registration order."""
    stop_order: list[str] = []

    class TrackedPlugin:
        def __init__(self, name: str):
            self.name = name

        async def stop(self):
            stop_order.append(self.name)

    plugins = [TrackedPlugin(f"plugin-{i}") for i in range(3)]

    await stop_plugins([], [], [], plugins)  # enrichers list

    # Should stop in reverse: plugin-2, plugin-1, plugin-0
    assert stop_order == ["plugin-2", "plugin-1", "plugin-0"]
```

### Test 2: Plugin Stop Error Containment

```python
@pytest.mark.asyncio
async def test_stop_plugins_contains_errors():
    """Plugin stop errors should not block other plugins."""
    stopped: list[str] = []

    class FailingPlugin:
        name = "failing"
        async def stop(self):
            raise RuntimeError("Stop failed")

    class GoodPlugin:
        def __init__(self, name: str):
            self.name = name
        async def stop(self):
            stopped.append(self.name)

    plugins = [GoodPlugin("good-1"), FailingPlugin(), GoodPlugin("good-2")]

    # Should not raise, should stop all plugins that can be stopped
    await stop_plugins([], [], [], plugins)

    assert "good-1" in stopped
    assert "good-2" in stopped
```

### Test 3: Sink Write Failure Metrics

```python
@pytest.mark.asyncio
async def test_sink_failure_increments_dropped():
    """Sink write failure should increment dropped counter."""

    async def failing_sink(event: dict) -> None:
        raise IOError("Sink unavailable")

    logger = get_logger(name="test-sink-fail")
    logger._sink_write = failing_sink

    logger.info("event-1")
    logger.info("event-2")

    result = await logger.stop_and_drain()

    # Events should be counted as dropped when sink fails
    assert result.dropped > 0
```

### Test 4: Strict Envelope Mode

```python
@pytest.mark.asyncio
async def test_strict_envelope_mode_drops_unserializable():
    """Strict mode should drop events that fail envelope serialization."""

    class NonSerializable:
        pass

    collected: list[dict] = []

    async def collecting_sink(event: dict) -> None:
        collected.append(event)

    # Configure strict envelope mode
    with patch.dict(os.environ, {"FAPILOG_CORE__STRICT_ENVELOPE_MODE": "true"}):
        logger = get_logger(name="test-strict")
        logger._sink_write = collecting_sink
        logger._serialize_in_flush = True

        logger.info("valid-event")
        logger.info("bad-event", payload=NonSerializable())
        logger.info("another-valid")

        result = await logger.stop_and_drain()

    # Only valid events should reach sink
    messages = [e["message"] for e in collected]
    assert "valid-event" in messages
    assert "another-valid" in messages
    assert "bad-event" not in messages
```

### Test 5: Backpressure Same-Thread Drop

```python
@pytest.mark.asyncio
async def test_same_thread_backpressure_drop():
    """Same-thread enqueue should drop with diagnostic when queue full."""
    diagnostics: list[dict] = []

    def capture_warn(component: str, message: str, **fields):
        diagnostics.append({"component": component, "message": message, **fields})

    # Create logger with tiny queue
    logger = SyncLoggerFacade(
        name="test-bp",
        queue_capacity=2,
        batch_max_size=100,
        batch_timeout_seconds=10.0,
        backpressure_wait_ms=0,
        drop_on_full=True,
        sink_write=lambda e: asyncio.sleep(10),  # Slow sink
    )
    logger.start()

    with patch("fapilog.core.diagnostics.warn", side_effect=capture_warn):
        # Fill queue beyond capacity from same thread
        for i in range(10):
            logger._enqueue("INFO", f"event-{i}")

    # Should have backpressure drop diagnostics
    bp_diagnostics = [d for d in diagnostics if d["component"] == "backpressure"]
    assert len(bp_diagnostics) > 0
```

---

## Files Changed

| File | Action |
|------|--------|
| `tests/integration/test_worker_lifecycle.py` | New file |

**Lines added:** ~400-500

---

## Risk Assessment

**Risk Level: Low**

- Tests only, no production code changes
- May reveal existing bugs (intended)
- Coverage improvement without behavioral changes

---

## Rollback Plan

Delete `tests/integration/test_worker_lifecycle.py`; no other files affected.

---

## Definition of Done

- [x] All acceptance criteria tests implemented
- [x] Tests verify shutdown ordering
- [x] Tests verify cancellation handling
- [x] Tests verify sink failure recovery
- [x] Tests verify serialization fallback paths
- [x] Tests verify backpressure edge cases
- [x] Tests verify thread-loop mode
- [x] All tests pass
- [x] Coverage of `worker.py` improved toward 90%
- [x] Coverage of `logger.py` improved toward 90%
