# Story 5.11: Plugin Hot-Reload Capability

## Status: Draft

## Priority: Low

## Estimated Effort: Large (5-7 days)

## Dependencies: Story 5.10 (plugin dependencies)

## Epic: Plugin System Advancement

---

## Context

Long-running applications (daemons, servers) may need to update plugin configuration without restart:

- Change log destination (add/remove sinks)
- Update redaction rules for compliance
- Enable/disable debug enrichers
- Update sampling rates

Currently, plugin configuration is static at startup. Changing requires restart.

---

## Problem Statement

1. Cannot add/remove plugins without application restart
2. Cannot update plugin configuration at runtime
3. No way to reload plugins after dependency updates
4. Configuration changes require deployment

---

## Acceptance Criteria

### AC1: Runtime Plugin Management

- [ ] `logger.add_sink(sink)` - Add sink at runtime
- [ ] `logger.remove_sink(name)` - Remove sink by name
- [ ] `logger.reload_plugin(name)` - Reload plugin with new config
- [ ] Similar methods for enrichers, redactors, processors

### AC2: Configuration Watching

- [ ] Optional file watcher for config changes
- [ ] Reload triggered on config file modification
- [ ] Debounced reload to handle rapid changes

### AC3: Safe Reload Process

- [ ] New plugin started before old stopped
- [ ] Graceful drain of old plugin
- [ ] Rollback on new plugin start failure
- [ ] No log loss during reload

### AC4: Health During Reload

- [ ] Health check reflects reload in progress
- [ ] Metrics track reload count and duration
- [ ] Diagnostics emitted on reload events

### AC5: Constraints

- [ ] Cannot remove last sink (safety)
- [ ] Cannot break dependency chain
- [ ] Rate limit on reload frequency

---

## Technical Design

### 1. Runtime Plugin Management API

```python
# src/fapilog/plugins/manager.py
"""Runtime plugin management."""

from __future__ import annotations

import asyncio
from dataclasses import dataclass
from typing import Any


@dataclass
class ReloadResult:
    """Result of a plugin reload operation."""

    success: bool
    plugin_name: str
    plugin_type: str
    old_version: str | None = None
    new_version: str | None = None
    error: str | None = None
    duration_seconds: float = 0.0


class PluginManager:
    """Manages runtime plugin lifecycle."""

    def __init__(
        self,
        *,
        sinks: list[Any],
        enrichers: list[Any],
        redactors: list[Any],
        processors: list[Any],
        filters: list[Any] = None,
    ) -> None:
        self._sinks = list(sinks)
        self._enrichers = list(enrichers)
        self._redactors = list(redactors)
        self._processors = list(processors)
        self._filters = list(filters or [])
        self._lock = asyncio.Lock()
        self._reload_in_progress = False

    @property
    def sinks(self) -> list[Any]:
        return list(self._sinks)

    @property
    def enrichers(self) -> list[Any]:
        return list(self._enrichers)

    @property
    def redactors(self) -> list[Any]:
        return list(self._redactors)

    async def add_sink(self, sink: Any) -> bool:
        """Add a sink at runtime.

        Returns True if successful.
        """
        async with self._lock:
            try:
                # Start the sink
                if hasattr(sink, "start"):
                    await sink.start()

                self._sinks.append(sink)

                diagnostics.debug(
                    "plugin-manager",
                    "sink added",
                    sink_name=getattr(sink, "name", "unknown"),
                )
                return True

            except Exception as exc:
                diagnostics.warn(
                    "plugin-manager",
                    "failed to add sink",
                    error=str(exc),
                )
                return False

    async def remove_sink(self, name: str) -> bool:
        """Remove a sink by name.

        Returns True if successful.
        Fails if it would remove the last sink.
        """
        async with self._lock:
            # Find sink by name
            sink_to_remove = None
            for sink in self._sinks:
                if getattr(sink, "name", "") == name:
                    sink_to_remove = sink
                    break

            if sink_to_remove is None:
                return False

            # Safety: don't remove last sink
            if len(self._sinks) <= 1:
                diagnostics.warn(
                    "plugin-manager",
                    "cannot remove last sink",
                    sink_name=name,
                )
                return False

            try:
                # Stop the sink gracefully
                if hasattr(sink_to_remove, "stop"):
                    await sink_to_remove.stop()

                self._sinks.remove(sink_to_remove)

                diagnostics.debug(
                    "plugin-manager",
                    "sink removed",
                    sink_name=name,
                )
                return True

            except Exception as exc:
                diagnostics.warn(
                    "plugin-manager",
                    "error removing sink",
                    sink_name=name,
                    error=str(exc),
                )
                return False

    async def reload_sink(
        self,
        name: str,
        new_config: dict[str, Any],
    ) -> ReloadResult:
        """Reload a sink with new configuration.

        Uses blue-green pattern:
        1. Create new sink with new config
        2. Start new sink
        3. Swap in new sink
        4. Drain and stop old sink
        5. Rollback if new sink fails
        """
        import time
        start_time = time.perf_counter()

        async with self._lock:
            self._reload_in_progress = True

            try:
                # Find existing sink
                old_sink = None
                old_index = -1
                for i, sink in enumerate(self._sinks):
                    if getattr(sink, "name", "") == name:
                        old_sink = sink
                        old_index = i
                        break

                if old_sink is None:
                    return ReloadResult(
                        success=False,
                        plugin_name=name,
                        plugin_type="sink",
                        error="Sink not found",
                    )

                # Get old version
                old_version = _get_plugin_version(old_sink)

                # Create new sink
                try:
                    new_sink = await _create_sink(name, new_config)
                except Exception as exc:
                    return ReloadResult(
                        success=False,
                        plugin_name=name,
                        plugin_type="sink",
                        old_version=old_version,
                        error=f"Failed to create new sink: {exc}",
                    )

                # Start new sink
                try:
                    if hasattr(new_sink, "start"):
                        await new_sink.start()
                except Exception as exc:
                    return ReloadResult(
                        success=False,
                        plugin_name=name,
                        plugin_type="sink",
                        old_version=old_version,
                        error=f"Failed to start new sink: {exc}",
                    )

                # Swap in new sink
                self._sinks[old_index] = new_sink

                # Stop old sink (gracefully)
                try:
                    if hasattr(old_sink, "flush"):
                        await old_sink.flush()
                    if hasattr(old_sink, "stop"):
                        await old_sink.stop()
                except Exception:
                    pass  # Best effort

                duration = time.perf_counter() - start_time
                new_version = _get_plugin_version(new_sink)

                return ReloadResult(
                    success=True,
                    plugin_name=name,
                    plugin_type="sink",
                    old_version=old_version,
                    new_version=new_version,
                    duration_seconds=duration,
                )

            finally:
                self._reload_in_progress = False

    @property
    def is_reloading(self) -> bool:
        """Check if a reload is in progress."""
        return self._reload_in_progress


def _get_plugin_version(plugin: Any) -> str | None:
    """Get version from plugin or its module."""
    try:
        import importlib
        mod = importlib.import_module(type(plugin).__module__)
        metadata = getattr(mod, "PLUGIN_METADATA", {})
        return metadata.get("version")
    except Exception:
        return None
```

### 2. Logger Integration

```python
# Update logger facades

class SyncLoggerFacade:

    def __init__(self, ..., plugin_manager: PluginManager = None):
        self._plugin_manager = plugin_manager

    def add_sink(self, sink: Any) -> bool:
        """Add a sink at runtime."""
        if self._plugin_manager is None:
            return False
        return self._schedule_async_result(
            self._plugin_manager.add_sink(sink)
        )

    def remove_sink(self, name: str) -> bool:
        """Remove a sink by name."""
        if self._plugin_manager is None:
            return False
        return self._schedule_async_result(
            self._plugin_manager.remove_sink(name)
        )

    async def reload_sink(
        self,
        name: str,
        config: dict[str, Any],
    ) -> ReloadResult:
        """Reload a sink with new configuration."""
        if self._plugin_manager is None:
            return ReloadResult(
                success=False,
                plugin_name=name,
                plugin_type="sink",
                error="Plugin manager not available",
            )
        return await self._plugin_manager.reload_sink(name, config)
```

### 3. Configuration Watcher

```python
# src/fapilog/plugins/config_watcher.py
"""Watch configuration files for changes."""

from __future__ import annotations

import asyncio
import hashlib
from pathlib import Path
from typing import Any, Callable


class ConfigWatcher:
    """Watch config file and trigger reload on changes."""

    def __init__(
        self,
        config_path: Path,
        on_change: Callable[[dict[str, Any]], None],
        *,
        debounce_seconds: float = 2.0,
        min_reload_interval: float = 10.0,
    ) -> None:
        self._path = config_path
        self._on_change = on_change
        self._debounce = debounce_seconds
        self._min_interval = min_reload_interval

        self._last_hash: str | None = None
        self._last_reload = 0.0
        self._running = False
        self._task: asyncio.Task | None = None

    async def start(self) -> None:
        """Start watching for changes."""
        self._running = True
        self._last_hash = self._compute_hash()
        self._task = asyncio.create_task(self._watch_loop())

    async def stop(self) -> None:
        """Stop watching."""
        self._running = False
        if self._task:
            self._task.cancel()
            try:
                await self._task
            except asyncio.CancelledError:
                pass

    async def _watch_loop(self) -> None:
        """Main watch loop."""
        while self._running:
            await asyncio.sleep(self._debounce)

            try:
                current_hash = self._compute_hash()

                if current_hash != self._last_hash:
                    # Config changed
                    import time
                    now = time.monotonic()

                    if now - self._last_reload >= self._min_interval:
                        self._last_hash = current_hash
                        self._last_reload = now

                        config = self._load_config()
                        self._on_change(config)

            except Exception:
                pass

    def _compute_hash(self) -> str:
        """Compute hash of config file."""
        try:
            content = self._path.read_bytes()
            return hashlib.md5(content).hexdigest()
        except Exception:
            return ""

    def _load_config(self) -> dict[str, Any]:
        """Load config from file."""
        import json
        import yaml

        content = self._path.read_text()

        if self._path.suffix in (".yaml", ".yml"):
            return yaml.safe_load(content)
        elif self._path.suffix == ".json":
            return json.loads(content)
        else:
            raise ValueError(f"Unsupported config format: {self._path.suffix}")
```

---

## Test Plan

### Unit Tests

1. **test_plugin_manager.py**

   - Test add/remove sink
   - Test reload with success
   - Test reload with failure/rollback
   - Test cannot remove last sink

2. **test_config_watcher.py**
   - Test change detection
   - Test debouncing
   - Test minimum interval

### Integration Tests

1. **test_live_reload.py**
   - Test full reload cycle
   - Test no log loss during reload
   - Test health during reload

---

## Documentation Updates

1. Create `docs/user-guide/hot-reload.md`
2. Update `docs/plugins/configuration.md`
3. Add examples for runtime plugin management

---

## Security Considerations

- Rate limit reloads to prevent DoS
- Validate new config before loading
- Log all reload operations for audit

---

## Related Stories

- Story 5.10: Plugin dependencies (respect during reload)

