# Story 5.21: Kubernetes Pod Metadata Enricher

## Status: Complete

## Priority: Medium

## Estimated Effort: Small (1 day)

## Dependencies: Story 5.19 (parse_plugin_config pattern)

## Epic: Built-in Enrichers

---

## Context

The majority of Python applications in production run on Kubernetes. When troubleshooting issues in a K8s environment, operators need to correlate logs with specific pods, nodes, namespaces, and deployments.

Currently, users must manually inject this metadata into every log call or configure external log processors (like Fluent Bit) to add it. A built-in Kubernetes enricher would provide this out of the box.

---

## Problem Statement

1. **Manual metadata injection** is error-prone and inconsistent
2. **No standard K8s fields** in log entries for filtering in log aggregators
3. **Difficult correlation** between logs and K8s resources
4. **Deployment name not available** - pod name is `my-app-7d4b8c9f6-abc12` but users want `my-app`

---

## Scope Decision

This story focuses on the **sweet spot**: high-value fields available via simple environment variables. Advanced features (labels, annotations) are deferred to keep the initial implementation simple and the deployment manifest minimal.

| Feature                    | This Story | Future Enhancement          |
| -------------------------- | ---------- | --------------------------- |
| Pod name, namespace, node  | ✅         |                             |
| Deployment name extraction | ✅         |                             |
| Container name, cluster    | ✅         |                             |
| Pod labels/annotations     |            | ✅ (requires volume mounts) |
| Service account, pod UID   |            | ✅ (rarely needed)          |

---

## Acceptance Criteria

### AC1: Core Enricher Implementation

- [ ] Create `KubernetesEnricher` class implementing `BaseEnricher`
- [ ] Use `parse_plugin_config` pattern from story 5.19
- [ ] Read metadata from Kubernetes Downward API environment variables
- [ ] Cache all metadata at startup (pod metadata is immutable)
- [ ] Return empty dict if not running in K8s (safe no-op)

### AC2: Standard Kubernetes Fields

- [ ] `k8s_pod`: Pod name from `POD_NAME` env var
- [ ] `k8s_namespace`: Namespace from `POD_NAMESPACE` env var
- [ ] `k8s_node`: Node name from `NODE_NAME` env var
- [ ] `k8s_container`: Container name from `CONTAINER_NAME` env var
- [ ] `k8s_cluster`: Cluster name from `CLUSTER_NAME` env var (optional)

### AC3: Deployment Name Extraction

- [ ] `k8s_deployment`: Extract deployment name from pod name
- [ ] Handle Deployment pattern: `my-app-7d4b8c9f6-abc12` → `my-app`
- [ ] Handle StatefulSet pattern: `redis-0` → `redis`
- [ ] Handle DaemonSet pattern: `fluent-bit-abc12` → `fluent-bit`

### AC4: Configuration

- [ ] Configurable field prefix (default: `k8s_`)
- [ ] `skip_if_not_k8s`: Return empty dict if `POD_NAME` not set (default: true)
- [ ] Pydantic config with `extra="forbid"` for typo detection

### AC5: Performance

- [ ] Zero I/O during enrichment (all data cached at startup)
- [ ] < 0.1ms overhead per log entry

### AC6: Documentation

- [ ] Document enricher in `docs/plugins/enrichers.md`
- [ ] Provide minimal Kubernetes manifest snippet for Downward API env vars

---

## Technical Design

### 1. Kubernetes Downward API Background

Kubernetes exposes pod metadata via environment variables using the Downward API:

```yaml
# Minimal deployment snippet for fapilog KubernetesEnricher
env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: POD_NAMESPACE
    valueFrom:
      fieldRef:
        fieldPath: metadata.namespace
  - name: NODE_NAME
    valueFrom:
      fieldRef:
        fieldPath: spec.nodeName
  - name: CONTAINER_NAME
    value: "my-container" # Static value, K8s doesn't auto-populate this
  - name: CLUSTER_NAME
    value: "production" # Optional, set by your deployment tooling
```

This is **all that's required**—no volume mounts, no K8s API calls, no RBAC permissions.

### 2. Enricher Implementation

```python
# src/fapilog/plugins/enrichers/kubernetes.py

from __future__ import annotations

import os
import re
from typing import Any

from pydantic import BaseModel, ConfigDict

from fapilog.plugins.utils import parse_plugin_config


class KubernetesEnricherConfig(BaseModel):
    """Configuration for Kubernetes enricher."""

    model_config = ConfigDict(frozen=True, extra="forbid")

    # Field prefix for all K8s fields
    prefix: str = "k8s_"

    # Skip enrichment if not in K8s (no POD_NAME env var)
    skip_if_not_k8s: bool = True

    # Environment variable names (override if non-standard)
    pod_name_env: str = "POD_NAME"
    pod_namespace_env: str = "POD_NAMESPACE"
    node_name_env: str = "NODE_NAME"
    container_name_env: str = "CONTAINER_NAME"
    cluster_name_env: str = "CLUSTER_NAME"


# Regex patterns for deployment name extraction
_DEPLOYMENT_PATTERN = re.compile(r"^(.+)-[a-f0-9]{8,10}-[a-z0-9]{5}$")  # Deployment
_STATEFULSET_PATTERN = re.compile(r"^(.+)-\d+$")  # StatefulSet
_DAEMONSET_PATTERN = re.compile(r"^(.+)-[a-z0-9]{5}$")  # DaemonSet


def _extract_deployment_name(pod_name: str) -> str | None:
    """Extract deployment/statefulset/daemonset name from pod name.

    Examples:
        my-app-7d4b8c9f6-abc12 → my-app (Deployment with ReplicaSet hash)
        my-app-abc12           → my-app (DaemonSet)
        redis-0                → redis  (StatefulSet)
    """
    # Try Deployment pattern first (most common)
    if match := _DEPLOYMENT_PATTERN.match(pod_name):
        return match.group(1)
    # StatefulSet pattern
    if match := _STATEFULSET_PATTERN.match(pod_name):
        return match.group(1)
    # DaemonSet pattern (must come after Deployment, as it's less specific)
    if match := _DAEMONSET_PATTERN.match(pod_name):
        return match.group(1)
    return None


class KubernetesEnricher:
    """Enrich log entries with Kubernetes pod metadata.

    Reads metadata from environment variables at startup (set via K8s Downward API),
    then returns cached fields on every enrich() call for zero overhead.

    Fields added (with default prefix "k8s_"):
    - k8s_pod: Pod name
    - k8s_namespace: Pod namespace
    - k8s_node: Node name
    - k8s_container: Container name
    - k8s_cluster: Cluster name (if set)
    - k8s_deployment: Deployment name (extracted from pod name)

    Requires K8s deployment to expose metadata via Downward API env vars.
    """

    name = "kubernetes"

    def __init__(
        self,
        config: KubernetesEnricherConfig | dict[str, Any] | None = None,
        **kwargs: Any,
    ) -> None:
        self._config = parse_plugin_config(KubernetesEnricherConfig, config, **kwargs)
        self._cached_fields: dict[str, Any] = {}  # Populated in start()

    async def start(self) -> None:
        """Load Kubernetes metadata from environment variables."""
        self._cached_fields = self._build_cached_fields()

    async def stop(self) -> None:
        """Cleanup (no-op)."""
        pass

    async def enrich(self, event: dict) -> dict:
        """Add Kubernetes metadata to the event.

        Returns cached fields dict (all metadata is immutable).
        """
        return dict(self._cached_fields)  # Returns empty dict if not K8s

    async def health_check(self) -> bool:
        """Always returns True (graceful no-op if not in K8s)."""
        return True

    def _build_cached_fields(self) -> dict[str, Any]:
        """Build the cached fields dict from environment variables."""
        cfg = self._config

        # Check if we're in K8s
        pod_name = os.getenv(cfg.pod_name_env)
        if not pod_name and cfg.skip_if_not_k8s:
            return {}

        fields: dict[str, Any] = {}
        prefix = cfg.prefix

        # Core fields from environment
        if pod_name:
            fields[f"{prefix}pod"] = pod_name

        if namespace := os.getenv(cfg.pod_namespace_env):
            fields[f"{prefix}namespace"] = namespace

        if node := os.getenv(cfg.node_name_env):
            fields[f"{prefix}node"] = node

        if container := os.getenv(cfg.container_name_env):
            fields[f"{prefix}container"] = container

        if cluster := os.getenv(cfg.cluster_name_env):
            fields[f"{prefix}cluster"] = cluster

        # Extract deployment name (the killer feature)
        if pod_name:
            if deployment := _extract_deployment_name(pod_name):
                fields[f"{prefix}deployment"] = deployment

        return fields


# Plugin metadata
PLUGIN_METADATA = {
    "name": "kubernetes",
    "version": "1.0.0",
    "plugin_type": "enricher",
    "entry_point": "fapilog.plugins.enrichers.kubernetes:KubernetesEnricher",
    "description": "Adds Kubernetes pod metadata to log entries.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "0.4.0"},
    "api_version": "1.0",
}
```

### 3. Registration

```python
# src/fapilog/plugins/enrichers/__init__.py

from .kubernetes import KubernetesEnricher, KubernetesEnricherConfig

register_builtin(
    "fapilog.enrichers",
    "kubernetes",
    KubernetesEnricher,
    aliases=["k8s"],
)

__all__ = [
    # ... existing exports ...
    "KubernetesEnricher",
    "KubernetesEnricherConfig",
]
```

---

## Test Plan

### Unit Tests

**test_kubernetes_enricher.py**

```python
import os
import pytest
from unittest.mock import patch
from fapilog.plugins.enrichers.kubernetes import (
    KubernetesEnricher,
    KubernetesEnricherConfig,
    _extract_deployment_name,
)


class TestDeploymentNameExtraction:
    """Test deployment name extraction from pod names."""

    def test_deployment_pod_name(self):
        """Standard Deployment with ReplicaSet hash."""
        assert _extract_deployment_name("my-app-7d4b8c9f6-abc12") == "my-app"
        assert _extract_deployment_name("api-server-6f9c5d8b7a-xyzpq") == "api-server"

    def test_statefulset_pod_name(self):
        """StatefulSet with ordinal suffix."""
        assert _extract_deployment_name("redis-0") == "redis"
        assert _extract_deployment_name("postgres-master-2") == "postgres-master"

    def test_daemonset_pod_name(self):
        """DaemonSet with 5-char hash."""
        assert _extract_deployment_name("fluent-bit-abc12") == "fluent-bit"

    def test_unrecognized_pattern(self):
        """Unrecognized patterns return None."""
        assert _extract_deployment_name("simple-pod") is None


@pytest.fixture
def k8s_env():
    """Set up K8s environment variables."""
    env = {
        "POD_NAME": "my-app-7d4b8c9f6-abc12",
        "POD_NAMESPACE": "production",
        "NODE_NAME": "node-1",
        "CONTAINER_NAME": "main",
        "CLUSTER_NAME": "prod-cluster",
    }
    with patch.dict(os.environ, env, clear=True):
        yield env


@pytest.mark.asyncio
async def test_basic_metadata(k8s_env):
    """Should add all K8s metadata fields."""
    enricher = KubernetesEnricher()
    await enricher.start()
    result = await enricher.enrich({})

    assert result["k8s_pod"] == "my-app-7d4b8c9f6-abc12"
    assert result["k8s_namespace"] == "production"
    assert result["k8s_node"] == "node-1"
    assert result["k8s_container"] == "main"
    assert result["k8s_cluster"] == "prod-cluster"
    assert result["k8s_deployment"] == "my-app"  # Extracted!


@pytest.mark.asyncio
async def test_not_in_kubernetes():
    """Should return empty dict when not in K8s."""
    with patch.dict(os.environ, {}, clear=True):
        enricher = KubernetesEnricher()
        await enricher.start()
        result = await enricher.enrich({})
        assert result == {}


@pytest.mark.asyncio
async def test_custom_prefix():
    """Should use custom field prefix."""
    with patch.dict(os.environ, {"POD_NAME": "test-pod"}, clear=True):
        enricher = KubernetesEnricher(prefix="kube_")
        await enricher.start()
        result = await enricher.enrich({})
        assert "kube_pod" in result
        assert "k8s_pod" not in result


@pytest.mark.asyncio
async def test_config_validation():
    """Should reject unknown config fields."""
    with pytest.raises(Exception):  # Pydantic ValidationError
        KubernetesEnricher(invalid_field="value")


@pytest.mark.asyncio
async def test_cached_results():
    """Enricher should return cached dict copies."""
    with patch.dict(os.environ, {"POD_NAME": "test"}, clear=True):
        enricher = KubernetesEnricher()
        await enricher.start()

        result1 = await enricher.enrich({})
        result2 = await enricher.enrich({})

        assert result1 == result2
        assert result1 is not result2  # Different dict instances


```

### Integration Tests

```python
@pytest.mark.asyncio
async def test_enricher_in_pipeline(k8s_env):
    """Test enricher works in full logging pipeline."""
    from fapilog import get_logger, Settings
    from fapilog.testing import MockSink

    sink = MockSink()
    settings = Settings()
    settings.core.enrichers = ["kubernetes"]

    logger = get_logger(settings=settings, sinks=[sink])
    logger.info("Test message")
    await logger.drain()

    entries = sink.get_entries()
    assert len(entries) >= 1
    assert entries[0].get("k8s_pod") == "my-app-7d4b8c9f6-abc12"
    assert entries[0].get("k8s_deployment") == "my-app"
```

---

## Documentation Updates

### docs/plugins/enrichers.md

Add section:

````markdown
## Kubernetes Enricher

Automatically adds Kubernetes pod metadata to log entries. The killer feature:
**deployment name extraction** (`my-app-7d4b8c9f6-abc12` → `my-app`).

### Quick Start

```python
from fapilog import get_logger, Settings

settings = Settings()
settings.core.enrichers = ["kubernetes"]
logger = get_logger(settings=settings)

logger.info("Processing request")
# Log includes: k8s_pod, k8s_namespace, k8s_node, k8s_deployment, etc.
```
````

### Kubernetes Deployment Configuration

Add these environment variables to your deployment manifest:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
        - name: main
          image: my-app:latest
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CONTAINER_NAME
              value: "main"
            - name: CLUSTER_NAME
              value: "prod-cluster" # Optional
```

That's it! No volume mounts, no K8s API calls, no RBAC permissions required.

### Fields Added

| Field            | Description                 | Example                  |
| ---------------- | --------------------------- | ------------------------ |
| `k8s_pod`        | Pod name                    | `my-app-7d4b8c9f6-abc12` |
| `k8s_namespace`  | Namespace                   | `production`             |
| `k8s_node`       | Node name                   | `node-pool-1-abc`        |
| `k8s_deployment` | Deployment name (extracted) | `my-app`                 |
| `k8s_container`  | Container name              | `main`                   |
| `k8s_cluster`    | Cluster name (if set)       | `prod-cluster`           |

### Configuration

```python
from fapilog.plugins.enrichers import KubernetesEnricher

enricher = KubernetesEnricher(
    prefix="k8s_",           # Field name prefix (default)
    skip_if_not_k8s=True,    # No-op if POD_NAME not set (default)
)
```

```

---

## Backwards Compatibility

- New enricher, no breaking changes
- Returns empty dict when not running in Kubernetes (safe no-op)
- No external dependencies

---

## Performance Considerations

- All metadata cached at startup (pod metadata is immutable)
- Zero I/O during `enrich()` - just returns cached dict copy
- No Kubernetes API calls (reads environment variables only)
- Overhead: < 0.1ms per log entry

---

## Future Enhancements (Out of Scope)

These are deferred to future stories if there's user demand:

| Feature | Why Deferred |
|---------|--------------|
| Pod labels/annotations | Requires volume mounts, more complex K8s manifest |
| Service account name | Rarely needed in logs |
| Pod UID | Pod name is usually sufficient for correlation |
| Node labels | Rarely available via Downward API |

---

## Related Stories

- Story 5.19: `parse_plugin_config` pattern (used by this enricher)
- Story 5.5: Cloud sink examples (K8s logs often go to cloud sinks)
- Future: Cloud metadata enricher (AWS/GCP/Azure instance metadata)

```
