# Story 4.23: Complete Plugin Metadata for All Built-in Plugins

**Status:** Complete  
**Priority:** High  
**Depends on:** None  
**Effort:** 0.5-1 day  
**PR:** [#169](https://github.com/chris-haste/fapilog/pull/169)

---

## Problem Statement

The plugin audit revealed inconsistent `PLUGIN_METADATA` across built-in plugins:

### Issues Found:

1. **`ZeroCopyProcessor` has no `PLUGIN_METADATA`** at all
2. **`UrlCredentialsRedactor` has incomplete metadata** - missing `compatibility` key
3. **Some plugins have inconsistent naming** between `name` attribute and `PLUGIN_METADATA["name"]`
4. **No automated validation** of metadata completeness at build/test time

This inconsistency affects:

- Plugin discovery and catalog generation
- Compatibility validation
- Documentation auto-generation
- Third-party plugin authors who use built-ins as reference

---

## Goals

1. **Add complete `PLUGIN_METADATA`** to `ZeroCopyProcessor`
2. **Fix `UrlCredentialsRedactor`** metadata to include `compatibility`
3. **Audit and fix all built-in plugins** for metadata consistency
4. **Add automated test** to verify all plugins have complete metadata
5. **Document required vs optional metadata fields**

---

## Audit Results

### Current Plugin Metadata Status

| Plugin                   | Has PLUGIN_METADATA | Has `compatibility` | Consistent `name` |
| ------------------------ | ------------------- | ------------------- | ----------------- |
| `StdoutJsonSink`         | ✅                  | ❌ Missing          | ✅                |
| `RotatingFileSink`       | ✅                  | ❌ Missing          | ⚠️ Version 0.1.0  |
| `HttpSink`               | ❌ Missing          | ❌                  | N/A               |
| `WebhookSink`            | ❌ Missing          | ❌                  | N/A               |
| `RuntimeInfoEnricher`    | ✅                  | ✅                  | ✅                |
| `ContextVarsEnricher`    | ✅                  | ✅                  | ✅                |
| `FieldMaskRedactor`      | ✅                  | ✅                  | ✅                |
| `RegexMaskRedactor`      | ✅                  | ✅                  | ✅                |
| `UrlCredentialsRedactor` | ✅                  | ❌ Missing          | ✅                |
| `ZeroCopyProcessor`      | ❌ Missing          | ❌                  | N/A               |

### Required Metadata Fields

Per `PluginMetadata` model in `plugins/metadata.py`:

```python
# Required fields
name: str
version: str
description: str
author: str
plugin_type: str
entry_point: str
compatibility: PluginCompatibility  # Contains min_fapilog_version

# Optional fields (with defaults)
author_email: str | None = None
homepage: str | None = None
repository: str | None = None
license: str = "MIT"
dependencies: list[str] = []
api_version: str = "1.0"
config_schema: dict | None = None
default_config: dict | None = None
tags: list[str] = []
marketplace_id: str | None = None
```

---

## Implementation Plan

### Phase 1: Fix ZeroCopyProcessor

**File: `src/fapilog/plugins/processors/zero_copy.py`**

Add at end of file:

```python
# Plugin metadata for discovery
PLUGIN_METADATA = {
    "name": "zero-copy",
    "version": "1.0.0",
    "plugin_type": "processor",
    "entry_point": "fapilog.plugins.processors.zero_copy:ZeroCopyProcessor",
    "description": "Zero-copy pass-through processor for performance benchmarking.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "3.0.0"},
    "api_version": "1.0",
}
```

### Phase 2: Fix UrlCredentialsRedactor

**File: `src/fapilog/plugins/redactors/url_credentials.py`**

Update metadata:

```python
PLUGIN_METADATA = {
    "name": "url-credentials",
    "version": "1.0.0",
    "author": "Fapilog Core",
    "plugin_type": "redactor",
    "entry_point": "fapilog.plugins.redactors.url_credentials:UrlCredentialsRedactor",
    "description": "Strips user:pass@ credentials from URL-like strings.",
    "compatibility": {"min_fapilog_version": "3.0.0"},  # ADD THIS
    "api_version": "1.0",
}
```

### Phase 3: Add Missing Sink Metadata

**File: `src/fapilog/plugins/sinks/http_client.py`**

Add at end of file:

```python
PLUGIN_METADATA = {
    "name": "http",
    "version": "1.0.0",
    "plugin_type": "sink",
    "entry_point": "fapilog.plugins.sinks.http_client:HttpSink",
    "description": "Async HTTP sink that POSTs JSON to a configured endpoint.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "3.0.0"},
    "api_version": "1.0",
    "config_schema": {
        "type": "object",
        "properties": {
            "endpoint": {"type": "string", "description": "HTTP endpoint URL"},
            "headers": {"type": "object", "description": "Default headers"},
            "retry_max_attempts": {"type": "integer"},
            "timeout_seconds": {"type": "number"},
        },
        "required": ["endpoint"],
    },
}
```

**File: `src/fapilog/plugins/sinks/webhook.py`**

Add at end of file:

```python
PLUGIN_METADATA = {
    "name": "webhook",
    "version": "1.0.0",
    "plugin_type": "sink",
    "entry_point": "fapilog.plugins.sinks.webhook:WebhookSink",
    "description": "Webhook sink that POSTs JSON with optional signing.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "3.0.0"},
    "api_version": "1.0",
    "config_schema": {
        "type": "object",
        "properties": {
            "endpoint": {"type": "string", "description": "Webhook URL"},
            "secret": {"type": "string", "description": "Shared secret for signing"},
            "headers": {"type": "object", "description": "Additional headers"},
            "timeout_seconds": {"type": "number"},
        },
        "required": ["endpoint"],
    },
}
```

**File: `src/fapilog/plugins/sinks/stdout_json.py`**

Update to include compatibility:

```python
PLUGIN_METADATA = {
    "name": "stdout-json",
    "version": "1.0.0",
    "plugin_type": "sink",
    "entry_point": "fapilog.plugins.sinks.stdout_json:StdoutJsonSink",
    "description": "Async stdout JSONL sink",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "3.0.0"},  # ADD THIS
    "api_version": "1.0",
}
```

**File: `src/fapilog/plugins/sinks/rotating_file.py`**

Update version and add compatibility:

```python
PLUGIN_METADATA = {
    "name": "rotating-file",
    "version": "1.0.0",  # UPDATE from 0.1.0
    "plugin_type": "sink",
    "entry_point": "fapilog.plugins.sinks.rotating_file:RotatingFileSink",
    "description": "Async rotating file sink with size/time rotation and retention",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "3.0.0"},  # ADD THIS
    "api_version": "1.0",
    "config_schema": {
        "type": "object",
        "properties": {
            "directory": {"type": "string", "description": "Log directory path"},
            "filename_prefix": {"type": "string"},
            "mode": {"type": "string", "enum": ["json", "text"]},
            "max_bytes": {"type": "integer"},
            "interval_seconds": {"type": "integer"},
            "max_files": {"type": "integer"},
            "compress_rotated": {"type": "boolean"},
        },
        "required": ["directory"],
    },
}
```

### Phase 4: Add Automated Validation Test

**File: `tests/unit/test_plugin_metadata_completeness.py`**

```python
"""Verify all built-in plugins have complete PLUGIN_METADATA."""

from __future__ import annotations

import importlib
from pathlib import Path
from typing import Any

import pytest

# All built-in plugin modules that should have PLUGIN_METADATA
BUILTIN_PLUGIN_MODULES = [
    # Sinks
    "fapilog.plugins.sinks.stdout_json",
    "fapilog.plugins.sinks.rotating_file",
    "fapilog.plugins.sinks.http_client",
    "fapilog.plugins.sinks.webhook",
    # Enrichers
    "fapilog.plugins.enrichers.runtime_info",
    "fapilog.plugins.enrichers.context_vars",
    # Redactors
    "fapilog.plugins.redactors.field_mask",
    "fapilog.plugins.redactors.regex_mask",
    "fapilog.plugins.redactors.url_credentials",
    # Processors
    "fapilog.plugins.processors.zero_copy",
]

REQUIRED_FIELDS = {
    "name",
    "version",
    "plugin_type",
    "entry_point",
    "description",
    "author",
    "compatibility",
    "api_version",
}


@pytest.mark.parametrize("module_path", BUILTIN_PLUGIN_MODULES)
def test_plugin_has_metadata(module_path: str) -> None:
    """Each plugin module must export PLUGIN_METADATA."""
    module = importlib.import_module(module_path)
    assert hasattr(module, "PLUGIN_METADATA"), (
        f"{module_path} missing PLUGIN_METADATA"
    )


@pytest.mark.parametrize("module_path", BUILTIN_PLUGIN_MODULES)
def test_plugin_metadata_has_required_fields(module_path: str) -> None:
    """Each PLUGIN_METADATA must have all required fields."""
    module = importlib.import_module(module_path)
    metadata: dict[str, Any] = getattr(module, "PLUGIN_METADATA", {})

    missing = REQUIRED_FIELDS - set(metadata.keys())
    assert not missing, (
        f"{module_path} PLUGIN_METADATA missing fields: {missing}"
    )


@pytest.mark.parametrize("module_path", BUILTIN_PLUGIN_MODULES)
def test_plugin_metadata_compatibility_valid(module_path: str) -> None:
    """compatibility must have min_fapilog_version."""
    module = importlib.import_module(module_path)
    metadata: dict[str, Any] = getattr(module, "PLUGIN_METADATA", {})

    compat = metadata.get("compatibility", {})
    assert "min_fapilog_version" in compat, (
        f"{module_path} compatibility missing min_fapilog_version"
    )


@pytest.mark.parametrize("module_path", BUILTIN_PLUGIN_MODULES)
def test_plugin_metadata_api_version_format(module_path: str) -> None:
    """api_version must be in 'X.Y' format."""
    from fapilog.plugins.versioning import parse_api_version

    module = importlib.import_module(module_path)
    metadata: dict[str, Any] = getattr(module, "PLUGIN_METADATA", {})

    api_version = metadata.get("api_version", "1.0")
    # Should not raise
    major, minor = parse_api_version(api_version)
    assert major >= 1, f"{module_path} api_version major must be >= 1"


@pytest.mark.parametrize("module_path", BUILTIN_PLUGIN_MODULES)
def test_plugin_type_is_valid(module_path: str) -> None:
    """plugin_type must be one of the valid types."""
    module = importlib.import_module(module_path)
    metadata: dict[str, Any] = getattr(module, "PLUGIN_METADATA", {})

    valid_types = {"sink", "processor", "enricher", "redactor", "alerting"}
    plugin_type = metadata.get("plugin_type")
    assert plugin_type in valid_types, (
        f"{module_path} has invalid plugin_type: {plugin_type}"
    )
```

### Phase 5: Update Plugin Guide Generation

Ensure `scripts/generate_env_matrix.py` (or similar) reads all PLUGIN_METADATA to generate `docs/plugin-guide.md`.

---

## Acceptance Criteria

### Must Have

- [x] All 10 built-in plugins have `PLUGIN_METADATA` export
- [x] All metadata includes `compatibility.min_fapilog_version`
- [x] All metadata includes valid `api_version` in "X.Y" format
- [x] Automated test validates metadata completeness (60 tests)
- [ ] `docs/plugin-guide.md` regenerated with complete plugin list
- [x] All existing tests pass (1026 tests)

### Should Have

- [x] CI runs metadata completeness check (via test suite)
- [ ] Documentation on required vs optional metadata fields

### Won't Have (This Story)

- JSON Schema validation of config_schema fields
- Automated metadata extraction for third-party plugins

---

## Test Plan

1. **Unit Tests**

   - `test_plugin_has_metadata()` for each module
   - `test_plugin_metadata_has_required_fields()` for each module
   - `test_plugin_metadata_compatibility_valid()` for each module
   - `test_plugin_metadata_api_version_format()` for each module
   - `test_plugin_type_is_valid()` for each module

2. **Integration Tests**
   - Plugin loader discovers all plugins via PLUGIN_METADATA
   - Plugin catalog generation works correctly

---

## Estimated Effort

| Task                                | Effort       |
| ----------------------------------- | ------------ |
| Fix ZeroCopyProcessor metadata      | 10 min       |
| Fix UrlCredentialsRedactor metadata | 5 min        |
| Add HttpSink metadata               | 15 min       |
| Add WebhookSink metadata            | 15 min       |
| Update StdoutJsonSink metadata      | 5 min        |
| Update RotatingFileSink metadata    | 10 min       |
| Create completeness test            | 30 min       |
| Update plugin guide                 | 15 min       |
| **Total**                           | **~2 hours** |

---

## Related Stories

- **4.21** - Health check protocol (adds health_check to plugins)
- **Future** - Plugin marketplace registration
