# Story 4.38: Refactor Logger Flush Logic (DRY Violation)

**Status:** Draft  
**Priority:** Medium  
**Depends on:** None  
**Effort:** 1 day

---

## Problem Statement

The `SyncLoggerFacade` and `AsyncLoggerFacade` classes in `logger.py` contain nearly identical implementations of:

1. `_flush_batch()` - ~130 lines, duplicated
2. `_worker_main()` - ~80 lines, duplicated
3. `_async_enqueue()` - ~50 lines, duplicated
4. Various helper methods

### Duplication Analysis

| Method | SyncLoggerFacade Lines | AsyncLoggerFacade Lines | Shared Logic |
|--------|----------------------|------------------------|--------------|
| `_flush_batch` | 661-799 | 1476-1594 | ~95% |
| `_worker_main` | 599-659 | 1396-1474 | ~90% |
| `_async_enqueue` | 833-885 | 1628-1680 | ~98% |

### Impact

1. **Maintenance burden**: Bug fixes must be applied twice
2. **Drift risk**: Easy for implementations to diverge unintentionally
3. **Code bloat**: `logger.py` is ~1680 lines when it could be ~1000

---

## Goals

1. Extract shared logic into reusable components
2. Reduce duplication to < 10%
3. Maintain identical behavior (no functional changes)
4. Improve testability of core logic

---

## Design Options

### Option A: Mixin Class

Extract shared methods into a mixin:

```python
class LoggerFlushMixin:
    """Shared flush and worker logic for logger facades."""
    
    async def _flush_batch(self, batch: list[dict]) -> None:
        # Shared implementation
        ...
    
    async def _worker_main(self) -> None:
        # Shared implementation
        ...


class SyncLoggerFacade(LoggerFlushMixin):
    # Sync-specific methods only
    ...


class AsyncLoggerFacade(LoggerFlushMixin):
    # Async-specific methods only
    ...
```

**Pros:**
- Simple refactoring
- Minimal structural change

**Cons:**
- Mixins can be confusing
- Type checking gets tricky

### Option B: Composition with Worker Class

Extract worker logic into separate class:

```python
class LoggerWorker:
    """Shared background worker for processing log batches."""
    
    def __init__(
        self,
        queue: NonBlockingRingQueue,
        sink_write: Callable,
        enrichers: list,
        redactors: list,
        ...
    ):
        ...
    
    async def run(self) -> None:
        """Main worker loop."""
        ...
    
    async def flush_batch(self, batch: list[dict]) -> None:
        """Process and write a batch."""
        ...


class SyncLoggerFacade:
    def __init__(self, ...):
        self._worker = LoggerWorker(...)
    
    async def _worker_main(self) -> None:
        await self._worker.run()


class AsyncLoggerFacade:
    def __init__(self, ...):
        self._worker = LoggerWorker(...)
    
    async def _worker_main(self) -> None:
        await self._worker.run()
```

**Pros:**
- Clear separation of concerns
- Easier to test worker in isolation
- Better composition

**Cons:**
- More structural change
- Need to pass dependencies

### Option C: Base Class

Create abstract base class:

```python
class BaseLoggerFacade(ABC):
    """Base class with shared implementation."""
    
    async def _flush_batch(self, batch: list[dict]) -> None:
        # Shared implementation
        ...
    
    @abstractmethod
    def info(self, message: str, **metadata) -> None:
        ...


class SyncLoggerFacade(BaseLoggerFacade):
    def info(self, message: str, **metadata) -> None:
        self._enqueue("INFO", message, **metadata)


class AsyncLoggerFacade(BaseLoggerFacade):
    async def info(self, message: str, **metadata) -> None:
        await self._enqueue("INFO", message, **metadata)
```

**Pros:**
- Natural inheritance
- Type system understands relationship

**Cons:**
- Sync vs async methods don't fit well in same base
- May need Protocol instead of ABC

---

## Recommendation

**Option B (Composition)** - Extract `LoggerWorker` class that handles:
- Worker loop (`_worker_main`)
- Batch processing (`_flush_batch`)
- Queue management (`_async_enqueue`)

The facades remain thin wrappers that provide sync/async public API.

---

## Implementation Plan

### Phase 1: Create LoggerWorker Class

**File: `src/fapilog/core/worker.py`** (new file)

```python
"""
Background worker for processing log batches.

Extracted from SyncLoggerFacade/AsyncLoggerFacade to share implementation.
"""

from __future__ import annotations

import asyncio
import time
from typing import Any, Callable, Awaitable

from ..metrics.metrics import MetricsCollector
from ..plugins.enrichers import BaseEnricher, enrich_parallel
from ..plugins.redactors import BaseRedactor, redact_in_order
from .concurrency import NonBlockingRingQueue
from .serialization import SerializedView, serialize_envelope, serialize_mapping_to_json_bytes


class LoggerWorker:
    """Background worker that processes log batches.
    
    Responsibilities:
    - Pull events from queue in batches
    - Apply enrichers and redactors
    - Serialize and write to sink(s)
    - Handle errors with containment
    """
    
    def __init__(
        self,
        *,
        queue: NonBlockingRingQueue,
        batch_max_size: int,
        batch_timeout_seconds: float,
        sink_write: Callable[[dict], Awaitable[None]],
        sink_write_serialized: Callable[[SerializedView], Awaitable[None]] | None,
        enrichers: list[BaseEnricher],
        redactors: list[BaseRedactor],
        metrics: MetricsCollector | None,
        serialize_in_flush: bool,
        stop_flag_getter: Callable[[], bool],
        drained_event: asyncio.Event | None,
        flush_event: asyncio.Event | None,
        flush_done_event: asyncio.Event | None,
    ):
        self._queue = queue
        self._batch_max_size = batch_max_size
        self._batch_timeout_seconds = batch_timeout_seconds
        self._sink_write = sink_write
        self._sink_write_serialized = sink_write_serialized
        self._enrichers = enrichers
        self._redactors = redactors
        self._metrics = metrics
        self._serialize_in_flush = serialize_in_flush
        self._stop_flag_getter = stop_flag_getter
        self._drained_event = drained_event
        self._flush_event = flush_event
        self._flush_done_event = flush_done_event
        self._processed = 0
        self._dropped = 0
    
    @property
    def processed(self) -> int:
        return self._processed
    
    @property
    def dropped(self) -> int:
        return self._dropped
    
    async def run(self, *, in_thread_mode: bool = False) -> None:
        """Main worker loop."""
        batch: list[dict] = []
        next_flush_deadline: float | None = None
        
        try:
            while True:
                if self._stop_flag_getter():
                    # Drain remaining
                    while True:
                        ok, item = self._queue.try_dequeue()
                        if not ok or item is None:
                            break
                        batch.append(item)
                    await self._flush_batch(batch)
                    
                    if in_thread_mode:
                        asyncio.get_running_loop().stop()
                    if self._drained_event:
                        self._drained_event.set()
                    return
                
                # Check flush request
                if self._flush_event and self._flush_event.is_set():
                    while True:
                        ok, item = self._queue.try_dequeue()
                        if not ok or item is None:
                            break
                        batch.append(item)
                    if batch:
                        await self._flush_batch(batch)
                        next_flush_deadline = None
                    self._flush_event.clear()
                    if self._flush_done_event:
                        self._flush_done_event.set()
                    continue
                
                # Pull from queue
                ok, item = self._queue.try_dequeue()
                if ok and item is not None:
                    batch.append(item)
                    if len(batch) >= self._batch_max_size:
                        await self._flush_batch(batch)
                        next_flush_deadline = None
                        continue
                    if next_flush_deadline is None:
                        next_flush_deadline = time.perf_counter() + self._batch_timeout_seconds
                    continue
                
                # Check deadline
                now = time.perf_counter()
                if next_flush_deadline is not None and now >= next_flush_deadline:
                    await self._flush_batch(batch)
                    next_flush_deadline = None
                    continue
                
                await asyncio.sleep(0.001)
        except asyncio.CancelledError:
            return
        except Exception as exc:
            self._emit_diagnostic("worker_main error", exc)
    
    async def _flush_batch(self, batch: list[dict]) -> None:
        """Process and write a batch of events."""
        if not batch:
            return
        
        start = time.perf_counter()
        try:
            for entry in batch:
                entry = await self._enrich(entry)
                entry = await self._redact(entry)
                await self._write(entry)
                self._processed += 1
        except Exception as exc:
            self._dropped += len(batch)
            self._emit_diagnostic("flush error", exc)
        finally:
            await self._record_metrics(len(batch), time.perf_counter() - start)
            batch.clear()
    
    async def _enrich(self, entry: dict) -> dict:
        if not self._enrichers:
            return entry
        try:
            return await enrich_parallel(entry, self._enrichers, metrics=self._metrics)
        except Exception:
            return entry
    
    async def _redact(self, entry: dict) -> dict:
        if not self._redactors:
            return entry
        try:
            return await redact_in_order(entry, self._redactors, metrics=self._metrics)
        except Exception:
            return entry
    
    async def _write(self, entry: dict) -> None:
        if self._serialize_in_flush and self._sink_write_serialized:
            view = await self._try_serialize(entry)
            if view:
                try:
                    await self._sink_write_serialized(view)
                    return
                except Exception:
                    pass
        await self._sink_write(entry)
    
    async def _try_serialize(self, entry: dict) -> SerializedView | None:
        try:
            return serialize_envelope(entry)
        except Exception:
            try:
                return serialize_mapping_to_json_bytes(entry)
            except Exception:
                return None
    
    async def _record_metrics(self, batch_size: int, latency: float) -> None:
        if self._metrics:
            try:
                await self._metrics.record_flush(batch_size=batch_size, latency_seconds=latency)
            except Exception:
                pass
    
    def _emit_diagnostic(self, message: str, exc: Exception) -> None:
        try:
            from .diagnostics import warn
            warn("worker", message, error_type=type(exc).__name__, error=str(exc))
        except Exception:
            pass
```

### Phase 2: Refactor Facades to Use Worker

**File: `src/fapilog/core/logger.py`**

```python
from .worker import LoggerWorker


class SyncLoggerFacade:
    def __init__(self, ...):
        # ... existing init ...
        self._worker: LoggerWorker | None = None
    
    def _create_worker(self) -> LoggerWorker:
        return LoggerWorker(
            queue=self._queue,
            batch_max_size=self._batch_max_size,
            # ... pass all dependencies ...
        )
    
    async def _worker_main(self) -> None:
        if self._worker is None:
            self._worker = self._create_worker()
        await self._worker.run(in_thread_mode=self._worker_thread is not None)
    
    # Remove duplicated _flush_batch, keep facade thin
```

### Phase 3: Update Tests

Ensure all existing tests pass. Add new tests for `LoggerWorker` in isolation.

---

## Acceptance Criteria

- [ ] `LoggerWorker` class extracted to `core/worker.py`
- [ ] `SyncLoggerFacade` uses `LoggerWorker`
- [ ] `AsyncLoggerFacade` uses `LoggerWorker`
- [ ] Duplicated code reduced by > 200 lines
- [ ] All existing tests pass
- [ ] New tests for `LoggerWorker`
- [ ] No functional changes

---

## Files Changed

| File | Change |
|------|--------|
| `src/fapilog/core/worker.py` | New file with `LoggerWorker` |
| `src/fapilog/core/logger.py` | Refactor to use `LoggerWorker` |
| `tests/unit/test_logger_worker.py` | New test file |

