# Story 4.19: Simple Plugin Configuration API

**Status:** Draft  
**Priority:** High  
**Depends on:** 4.14 (fapilog-tamper package bootstrap - pattern reference)  
**Epic:** Plugin System Simplification

---

## Summary

Design and implement a minimal plugin configuration API that replaces hardcoded plugin instantiation with a configuration-driven approach. The API enables users to select, configure, and extend plugins via settings and Python entry points.

---

## Problem Statement

### Current State

Plugins are **hardcoded** in `get_logger()` (~200 lines duplicated for sync/async):

```python
# Enrichers - always instantiated
default_enrichers = [RuntimeInfoEnricher(), ContextVarsEnricher()]

# Integrity - entry-point based (WORKING PATTERN)
if cfg.integrity_plugin:
    plugin = load_integrity_plugin(cfg.integrity_plugin)
    enricher = plugin.get_enricher(cfg.integrity_config)
    default_enrichers.append(enricher)

# Redactors - partially configurable via redactors_order
if cfg.enable_redactors and cfg.redactors_order:
    for name in cfg.redactors_order:
        if name == "field-mask":
            redactors.append(FieldMaskRedactor(...))
        # ... more hardcoded if/else
```

### Pain Points

1. **No extension mechanism** - Users cannot add custom plugins without forking
2. **No disable mechanism** - Cannot turn off built-in plugins (e.g., RuntimeInfoEnricher)
3. **Duplicated logic** - Sync and async `get_logger()` have identical plugin setup
4. **Inconsistent patterns** - Integrity uses entry points; others use hardcoded imports
5. **Hidden configuration** - Plugin config scattered across CoreSettings fields

---

## Goals

| Goal                  | Success Metric                             |
| --------------------- | ------------------------------------------ |
| Configuration-driven  | All plugins selectable via settings        |
| Entry-point discovery | Third-party plugins work via `pip install` |
| Minimal code          | Core loader < 100 lines                    |
| Zero breaking changes | Existing `get_logger()` behavior unchanged |
| Testable              | Works in tests without external packages   |

## Non-Goals

- Plugin marketplace or remote discovery
- Dynamic hot-reload of plugins
- Complex dependency resolution between plugins
- Plugin health monitoring dashboard
- Automatic plugin updates

---

## Design

### 1. Entry Point Groups

Standardize on these entry point groups (consistent with existing `fapilog.integrity`):

```
fapilog.sinks       → Sink plugins
fapilog.enrichers   → Enricher plugins
fapilog.redactors   → Redactor plugins
fapilog.processors  → Processor plugins
fapilog.integrity   → Integrity plugins (existing)
```

### 2. Plugin Loader Module

New file: `src/fapilog/plugins/loader.py`

```python
"""
Simple plugin loader using Python entry points.

Loads plugins by name from:
1. Built-in registry (shipped with fapilog)
2. Entry points (third-party packages)
"""

from __future__ import annotations

import importlib.metadata
from typing import Any, TypeVar

from .sinks import BaseSink
from .enrichers import BaseEnricher
from .redactors import BaseRedactor
from .processors import BaseProcessor

T = TypeVar("T")

# Built-in plugin registry (name → factory function)
BUILTIN_SINKS: dict[str, type] = {}
BUILTIN_ENRICHERS: dict[str, type] = {}
BUILTIN_REDACTORS: dict[str, type] = {}
BUILTIN_PROCESSORS: dict[str, type] = {}

def register_builtin(group: str, name: str, cls: type) -> None:
    """Register a built-in plugin class."""
    registry = {
        "fapilog.sinks": BUILTIN_SINKS,
        "fapilog.enrichers": BUILTIN_ENRICHERS,
        "fapilog.redactors": BUILTIN_REDACTORS,
        "fapilog.processors": BUILTIN_PROCESSORS,
    }.get(group)
    if registry is not None:
        registry[name] = cls


def load_plugin(
    group: str,
    name: str,
    config: dict[str, Any] | None = None,
) -> Any:
    """Load a plugin by group and name.

    Args:
        group: Entry point group (e.g., "fapilog.sinks")
        name: Plugin name (e.g., "stdout_json")
        config: Optional configuration dict passed to constructor

    Returns:
        Instantiated plugin

    Raises:
        PluginNotFoundError: Plugin not found in built-ins or entry points
        PluginLoadError: Plugin found but failed to load/instantiate
    """
    config = config or {}

    # 1. Check built-in registry first
    registry = {
        "fapilog.sinks": BUILTIN_SINKS,
        "fapilog.enrichers": BUILTIN_ENRICHERS,
        "fapilog.redactors": BUILTIN_REDACTORS,
        "fapilog.processors": BUILTIN_PROCESSORS,
    }.get(group, {})

    if name in registry:
        cls = registry[name]
        return cls(**config) if config else cls()

    # 2. Try entry points
    try:
        eps = importlib.metadata.entry_points(group=group)
        for ep in eps:
            if ep.name == name:
                cls = ep.load()
                return cls(**config) if config else cls()
    except Exception as e:
        raise PluginLoadError(f"Failed to load plugin '{name}' from {group}: {e}") from e

    raise PluginNotFoundError(f"Plugin '{name}' not found in group '{group}'")


def list_available_plugins(group: str) -> list[str]:
    """List all available plugins for a group (built-in + entry points)."""
    registry = {
        "fapilog.sinks": BUILTIN_SINKS,
        "fapilog.enrichers": BUILTIN_ENRICHERS,
        "fapilog.redactors": BUILTIN_REDACTORS,
        "fapilog.processors": BUILTIN_PROCESSORS,
    }.get(group, {})

    names = set(registry.keys())

    try:
        eps = importlib.metadata.entry_points(group=group)
        for ep in eps:
            names.add(ep.name)
    except Exception:
        pass

    return sorted(names)


class PluginNotFoundError(Exception):
    """Plugin not found in built-ins or entry points."""
    pass


class PluginLoadError(Exception):
    """Plugin found but failed to load."""
    pass
```

### 3. Built-in Registration

Each built-in plugin module registers itself:

```python
# src/fapilog/plugins/sinks/stdout_json.py
from ..loader import register_builtin

class StdoutJsonSink:
    name = "stdout_json"
    # ... implementation ...

# Register at module load
register_builtin("fapilog.sinks", "stdout_json", StdoutJsonSink)
```

Alternatively, use a central registration in `__init__.py`:

```python
# src/fapilog/plugins/sinks/__init__.py
from .stdout_json import StdoutJsonSink
from .rotating_file import RotatingFileSink
from .http_client import HttpSink
from ..loader import register_builtin

register_builtin("fapilog.sinks", "stdout_json", StdoutJsonSink)
register_builtin("fapilog.sinks", "rotating_file", RotatingFileSink)
register_builtin("fapilog.sinks", "http", HttpSink)
```

### 4. Settings Schema

Update `CoreSettings` to use plugin names instead of booleans:

```python
class CoreSettings(BaseModel):
    """Core logging configuration."""

    # Current (deprecated but preserved for backward compat)
    enable_enrichers: bool = True
    enable_redactors: bool = True
    redactors_order: list[str] = ["field-mask", "regex-mask", "url-credentials"]

    # New: explicit plugin selection
    sinks: list[str] = Field(
        default=["stdout_json"],
        description="Sink plugins to use (by name)",
    )
    enrichers: list[str] = Field(
        default=["runtime_info", "context_vars"],
        description="Enricher plugins to use (by name)",
    )
    redactors: list[str] = Field(
        default=["field-mask", "regex-mask", "url-credentials"],
        description="Redactor plugins to use (by name)",
    )
    processors: list[str] = Field(
        default=[],
        description="Processor plugins to use (by name)",
    )
```

Environment variable configuration:

```bash
# Select plugins
FAPILOG_CORE__SINKS='["stdout_json", "rotating_file"]'
FAPILOG_CORE__ENRICHERS='["runtime_info", "k8s_metadata"]'
FAPILOG_CORE__REDACTORS='["field-mask"]'

# Disable all enrichers
FAPILOG_CORE__ENRICHERS='[]'
```

### 5. Per-Plugin Configuration

Add a nested configuration section:

```python
class SinkConfigs(BaseModel):
    """Per-sink configuration."""
    stdout_json: dict[str, Any] = Field(default_factory=dict)
    rotating_file: RotatingFileSinkSettings = Field(default_factory=RotatingFileSinkSettings)
    http: HttpSinkSettings = Field(default_factory=HttpSinkSettings)
    # Third-party plugins use the generic dict pattern

class Settings(BaseSettings):
    core: CoreSettings
    sink_config: SinkConfigs = Field(default_factory=SinkConfigs)
    enricher_config: dict[str, dict[str, Any]] = Field(default_factory=dict)
    redactor_config: dict[str, dict[str, Any]] = Field(default_factory=dict)
```

Environment variable configuration:

```bash
# Configure rotating_file sink
FAPILOG_SINK_CONFIG__ROTATING_FILE__PATH="/var/log/app.log"
FAPILOG_SINK_CONFIG__ROTATING_FILE__MAX_SIZE_MB=100

# Configure a third-party plugin
FAPILOG_SINK_CONFIG__SPLUNK__URL="https://splunk.example.com"
FAPILOG_SINK_CONFIG__SPLUNK__TOKEN="xxx"
```

### 6. Refactored get_logger()

Simplify `get_logger()` to use the loader:

```python
def get_logger(name: str | None = None, *, settings: Settings | None = None) -> SyncLoggerFacade:
    cfg = (settings or Settings()).core

    # Load plugins by name
    enrichers = _load_plugins("fapilog.enrichers", cfg.enrichers, settings)
    redactors = _load_plugins("fapilog.redactors", cfg.redactors, settings)
    sinks = _load_plugins("fapilog.sinks", cfg.sinks, settings)

    # Integrity plugin (existing pattern)
    if cfg.integrity_plugin:
        integrity = load_integrity_plugin(cfg.integrity_plugin)
        if enricher := integrity.get_enricher(cfg.integrity_config):
            enrichers.append(enricher)
        # ... sink wrapper

    # Create logger with loaded plugins
    return SyncLoggerFacade(
        enrichers=enrichers,
        redactors=redactors,
        # ... rest of config
    )


def _load_plugins(group: str, names: list[str], settings: Settings) -> list[Any]:
    """Load plugins by name with their configurations."""
    plugins = []
    config_section = {
        "fapilog.sinks": settings.sink_config,
        "fapilog.enrichers": settings.enricher_config,
        "fapilog.redactors": settings.redactor_config,
    }.get(group, {})

    for name in names:
        config = getattr(config_section, name, None) or config_section.get(name, {})
        try:
            plugin = load_plugin(group, name, config)
            plugins.append(plugin)
        except PluginNotFoundError:
            warn("plugins", f"Plugin '{name}' not found, skipping")
        except PluginLoadError as e:
            warn("plugins", f"Failed to load '{name}': {e}")

    return plugins
```

### 7. Third-Party Plugin Example

A third-party package (`fapilog-splunk`) would:

**pyproject.toml:**

```toml
[project]
name = "fapilog-splunk"
dependencies = ["fapilog>=0.4.0", "httpx"]

[project.entry-points."fapilog.sinks"]
splunk = "fapilog_splunk:SplunkSink"
```

**fapilog_splunk/**init**.py:**

```python
from fapilog.plugins.sinks import BaseSink

class SplunkSink(BaseSink):
    name = "splunk"

    def __init__(self, url: str, token: str, index: str = "main"):
        self.url = url
        self.token = token
        self.index = index

    async def write(self, entry: dict) -> None:
        # Send to Splunk HEC
        ...
```

**User configuration:**

```bash
pip install fapilog-splunk

# Configure via environment
FAPILOG_CORE__SINKS='["stdout_json", "splunk"]'
FAPILOG_SINK_CONFIG__SPLUNK__URL="https://splunk.example.com:8088"
FAPILOG_SINK_CONFIG__SPLUNK__TOKEN="xxx-xxx-xxx"
```

---

## Migration Path

### Phase 1: Add Loader (Non-Breaking)

1. Add `loader.py` with `load_plugin()`, `register_builtin()`
2. Register all built-in plugins
3. Add new settings fields (`core.sinks`, `core.enrichers`, etc.)
4. Keep existing behavior as default

### Phase 2: Refactor get_logger() (Non-Breaking)

1. Use loader internally
2. Preserve backward compatibility with old settings
3. Deprecation warnings for old patterns

### Phase 3: Deprecate Old Settings (Future)

1. Mark `enable_redactors`, `redactors_order` as deprecated
2. Document migration to new settings
3. Remove in next major version

---

## Acceptance Criteria

### Must Have

- [ ] `load_plugin(group, name, config)` loads built-in and entry-point plugins
- [ ] `list_available_plugins(group)` returns available plugin names
- [ ] All built-in plugins registered in `BUILTIN_*` registries
- [ ] `core.sinks`, `core.enrichers`, `core.redactors` settings work
- [ ] Per-plugin config via `sink_config`, `enricher_config`, `redactor_config`
- [ ] Existing `get_logger()` behavior unchanged with default settings
- [ ] Third-party plugins discoverable via entry points
- [ ] Error handling: missing plugins logged, don't crash
- [ ] 90%+ test coverage

### Should Have

- [ ] Deprecation warnings for old settings patterns
- [ ] CLI command: `fapilog plugins list` (future)
- [ ] Documentation with examples

### Won't Have

- [ ] Plugin marketplace
- [ ] Hot reload
- [ ] Version constraints checking
- [ ] Plugin dependency resolution

---

## Test Plan

### Unit Tests

```python
def test_load_builtin_plugin():
    """Built-in plugins load by name."""
    sink = load_plugin("fapilog.sinks", "stdout_json")
    assert isinstance(sink, StdoutJsonSink)

def test_load_builtin_with_config():
    """Built-in plugins accept configuration."""
    sink = load_plugin("fapilog.sinks", "rotating_file", {"path": "/tmp/test.log"})
    assert sink.path == "/tmp/test.log"

def test_load_plugin_not_found():
    """Unknown plugins raise PluginNotFoundError."""
    with pytest.raises(PluginNotFoundError):
        load_plugin("fapilog.sinks", "nonexistent")

def test_list_available_plugins():
    """List includes built-ins."""
    plugins = list_available_plugins("fapilog.sinks")
    assert "stdout_json" in plugins
    assert "rotating_file" in plugins

def test_entry_point_plugin(monkeypatch):
    """Entry point plugins are discovered."""
    # Mock entry_points to return a fake plugin
    ...
```

### Integration Tests

```python
def test_get_logger_with_custom_sinks():
    """get_logger respects core.sinks setting."""
    settings = Settings(core=CoreSettings(sinks=["rotating_file"]))
    logger = get_logger(settings=settings)
    # Verify rotating_file sink is used

def test_get_logger_empty_enrichers():
    """get_logger works with no enrichers."""
    settings = Settings(core=CoreSettings(enrichers=[]))
    logger = get_logger(settings=settings)
    # Verify no enrichers attached

def test_third_party_plugin_integration():
    """Third-party plugin works via entry point."""
    # Requires test fixture package with entry point
    ...
```

---

## Files Changed

| File                                         | Change                                         |
| -------------------------------------------- | ---------------------------------------------- |
| `src/fapilog/plugins/loader.py`              | **NEW** - Core loader (~80 lines)              |
| `src/fapilog/plugins/__init__.py`            | Export `load_plugin`, `list_available_plugins` |
| `src/fapilog/plugins/sinks/__init__.py`      | Register built-ins                             |
| `src/fapilog/plugins/enrichers/__init__.py`  | Register built-ins                             |
| `src/fapilog/plugins/redactors/__init__.py`  | Register built-ins                             |
| `src/fapilog/plugins/processors/__init__.py` | Register built-ins                             |
| `src/fapilog/core/settings.py`               | Add new plugin settings fields                 |
| `src/fapilog/__init__.py`                    | Refactor `get_logger()` to use loader          |
| `tests/unit/test_plugin_loader.py`           | **NEW** - Loader tests                         |
| `docs/plugins/configuration.md`              | **NEW** - Configuration guide                  |

---

## Estimated Effort

| Task                     | Hours  |
| ------------------------ | ------ |
| loader.py implementation | 2      |
| Built-in registration    | 1      |
| Settings updates         | 1      |
| get_logger() refactor    | 3      |
| Unit tests               | 2      |
| Integration tests        | 2      |
| Documentation            | 2      |
| **Total**                | **13** |

---

## Decisions

| Question                                   | Decision                                                                                                                                                                                  |
| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Naming convention**                      | Support both hyphens and underscores. Normalize to underscores internally. Alias built-in names (e.g., `runtime-info` → `runtime_info`).                                                  |
| **Lazy vs eager loading**                  | Eager at `get_logger()` for fail-fast behavior.                                                                                                                                           |
| **Plugin ordering**                        | Yes, respect list order from settings. Enrichers run in order, redactors apply in order.                                                                                                  |
| **Config validation**                      | Defer to plugin constructor (keep loader simple).                                                                                                                                         |
| **Integrity wrapping with multiple sinks** | Wrap each sink individually (per-sink wrapping). This preserves clear tamper semantics per output stream. Only wrap the multiplexer if per-sink wrapping complicates tamper verification. |

### Name Normalization Implementation

```python
def _normalize_plugin_name(name: str) -> str:
    """Normalize plugin name: hyphens → underscores, lowercase."""
    return name.lower().replace("-", "_")

# Built-in aliases for common variations
BUILTIN_ALIASES = {
    "runtime-info": "runtime_info",
    "context-vars": "context_vars",
    "field-mask": "field_mask",
    "regex-mask": "regex_mask",
    "url-credentials": "url_credentials",
    "stdout-json": "stdout_json",
    "rotating-file": "rotating_file",
}

def load_plugin(group: str, name: str, config: dict | None = None) -> Any:
    normalized = _normalize_plugin_name(name)
    # Check alias first
    canonical = BUILTIN_ALIASES.get(normalized, normalized)
    # ... rest of loading logic
```

### Integrity Wrapping Decision Rationale

**Per-sink wrapping (default):**

```
Enricher → Entry → SealedSink(FileA) → FileA.jsonl + manifest
                 → SealedSink(FileB) → FileB.jsonl + manifest
```

- Each sink gets its own hash chain and manifest
- Verification is independent per sink
- Clear audit trail per output destination

**Multiplexer wrapping (alternative, not default):**

```
Enricher → Entry → SealedMultiplexer → FileA.jsonl
                                     → FileB.jsonl
                                     → single manifest
```

- Single chain across all outputs
- Simpler but couples sink integrity
- May complicate "which sink was tampered?" forensics
