# Story 5.8: OpenTelemetry Plugin Integration

## Status: Cancelled — Deferred to Enterprise

## Priority: ~~Medium~~ N/A

## Estimated Effort: ~~Medium (3-4 days)~~ N/A

## Dependencies: ~~Story 5.7~~ N/A (5.7 also cancelled)

## Epic: Enterprise

---

## Cancellation Rationale

**Decision Date:** January 2026

This story has been cancelled and deferred to a future enterprise package (`fapilog-otel` or similar) for the following reasons:

### 1. Core OTEL Support Already Exists

The `ContextVarsEnricher` in `src/fapilog/plugins/enrichers/context_vars.py` already:

- Extracts `trace_id` and `span_id` from OpenTelemetry context when available
- Works gracefully — if OTEL isn't installed, fields are simply omitted
- Uses defensive try/except to avoid import errors

```python
# Already in ContextVarsEnricher (lines 36-50)
try:
    from opentelemetry.trace import get_current_span
    span = get_current_span()
    ctx = span.get_span_context()
    if ctx and ctx.is_valid():
        data["trace_id"] = f"{ctx.trace_id:032x}"
        data["span_id"] = f"{ctx.span_id:016x}"
except Exception:
    pass
```

**This covers the 80% use case** — users with OTEL in their environment automatically get trace context in their logs.

### 2. Story 5.7 Dependency Cancelled

This story depended on Story 5.7 (Plugin Observability Metrics) which has also been cancelled. The foundation doesn't exist.

### 3. Enterprise-Specific Features

What this story proposed beyond existing functionality:

| Feature                | Assessment                                               |
| ---------------------- | -------------------------------------------------------- |
| Plugin operation spans | Wraps every plugin call in a span — performance overhead |
| OTLP metrics export    | Specialized — most users use Prometheus                  |
| Baggage propagation    | Niche enterprise use case                                |
| Complex OtelSettings   | Over-engineering for core library                        |

### 4. Heavy Dependencies

Would require adding to core:

- `opentelemetry-api>=1.20.0`
- `opentelemetry-sdk>=1.20.0`
- `opentelemetry-exporter-otlp>=1.20.0`

These should be opt-in via a separate package, not bundled in core.

### 5. Lean Core Philosophy

The core library's job is structured logging that works. Advanced distributed tracing integration is a specialized concern for enterprise deployments with dedicated observability teams.

---

## What Users Should Do Instead

### For Trace Context in Logs (Already Works)

Enable `ContextVarsEnricher` — it automatically extracts OTEL context:

```python
from fapilog import get_logger

logger = get_logger(enrichers=["context_vars"])
# Logs will include trace_id and span_id when OTEL context exists
```

### For Custom OTEL Integration

Users needing deeper integration can create their own enricher:

```python
class CustomOtelEnricher:
    name = "custom_otel"

    async def enrich(self, event: dict) -> dict:
        # Custom OTEL extraction logic
        return {"trace_id": ..., "baggage": ...}
```

---

## Future Enterprise Package Scope

If/when customer demand warrants, a `fapilog-otel` package could include:

- Plugin operation span creation
- OTLP metrics exporter integration
- Baggage field extraction
- Span attribute configuration
- W3C trace context header propagation

---

## Original Context (Archived)

> OpenTelemetry is the industry standard for distributed tracing and observability. Enterprise users expect their logging library to:
>
> - Propagate trace context (trace_id, span_id) through logs
> - Create spans for plugin operations for debugging
> - Export metrics in OTLP format
> - Integrate with existing OTEL infrastructure

---

## Original Problem Statement (Archived)

> Users with OpenTelemetry instrumentation cannot:
>
> 1. See fapilog plugin operations in their distributed traces
> 2. Correlate slow logs with specific plugin processing
> 3. Use OTLP for metrics export
> 4. Leverage OTEL baggage for log enrichment

---

## Original Acceptance Criteria (Archived)

> **Note:** These criteria are preserved for historical reference. This story has been cancelled.

### ~~AC1: Plugin Tracing~~

- ~~Create spans for each plugin operation~~ → **Enterprise feature, adds overhead**
- ~~Span attributes include plugin_name, plugin_type, event_level~~
- ~~Span status reflects operation success/failure~~
- ~~Spans linked to current trace context~~

### ~~AC2: Trace Context Propagation~~

- ~~Log events automatically include trace_id, span_id when available~~ → **Already works via ContextVarsEnricher**
- ~~Baggage items can be configured for inclusion in logs~~ → **Niche use case**
- ~~Trace context preserved across async boundaries~~

### ~~AC3: OTLP Metrics Export~~

- ~~Optional OTLP metrics exporter~~ → **Enterprise feature**
- ~~All fapilog metrics exportable via OTLP~~
- ~~Configurable endpoint and headers~~
- ~~Graceful degradation when OTEL not available~~

### ~~AC4: Configuration~~

- ~~`otel.enabled` - Master switch for OTEL integration~~
- ~~`otel.tracing.enabled` - Enable plugin span creation~~
- ~~`otel.tracing.sample_rate` - Span sampling rate~~
- ~~`otel.metrics.enabled` - Enable OTLP metrics~~
- ~~`otel.metrics.endpoint` - OTLP endpoint~~

### ~~AC5: Zero Dependencies by Default~~ ✅

- ~~OTEL integration optional~~ → **Already true — ContextVarsEnricher handles gracefully**
- ~~No import errors when opentelemetry not installed~~ → **Already works**
- ~~Graceful no-op when disabled~~ → **Already works**

---

## Original Technical Design (Archived)

> **Note:** This design is preserved for reference if an enterprise package is created in the future.

### 1. Optional Dependencies

```toml
# pyproject.toml
[project.optional-dependencies]
otel = [
    "opentelemetry-api>=1.20.0",
    "opentelemetry-sdk>=1.20.0",
    "opentelemetry-exporter-otlp>=1.20.0",
]
```

### 2. OTEL Configuration

```python
# src/fapilog/core/settings.py

class OtelTracingSettings(BaseModel):
    """OpenTelemetry tracing configuration."""

    enabled: bool = Field(
        default=False,
        description="Enable span creation for plugin operations",
    )
    sample_rate: float = Field(
        default=1.0,
        ge=0.0,
        le=1.0,
        description="Span sampling rate (1.0 = all spans)",
    )
    include_attributes: list[str] = Field(
        default_factory=lambda: ["level", "message"],
        description="Log attributes to include in span attributes",
    )


class OtelMetricsSettings(BaseModel):
    """OpenTelemetry metrics configuration."""

    enabled: bool = Field(
        default=False,
        description="Enable OTLP metrics export",
    )
    endpoint: str = Field(
        default="http://localhost:4317",
        description="OTLP endpoint URL",
    )
    headers: dict[str, str] = Field(
        default_factory=dict,
        description="Headers for OTLP requests",
    )
    export_interval_seconds: float = Field(
        default=60.0,
        description="Metrics export interval",
    )


class OtelSettings(BaseModel):
    """OpenTelemetry integration settings."""

    enabled: bool = Field(
        default=False,
        description="Master switch for OpenTelemetry integration",
    )
    tracing: OtelTracingSettings = Field(
        default_factory=OtelTracingSettings,
    )
    metrics: OtelMetricsSettings = Field(
        default_factory=OtelMetricsSettings,
    )
    baggage_fields: list[str] = Field(
        default_factory=list,
        description="Baggage keys to include in log enrichment",
    )
```

### 3. OTEL Tracing Module

```python
# src/fapilog/observability/otel_tracing.py
"""OpenTelemetry tracing integration for plugins."""

from __future__ import annotations

import functools
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator, Callable, TypeVar

# Lazy imports to avoid dependency issues
_tracer = None
_otel_available = None


def _check_otel() -> bool:
    """Check if OpenTelemetry is available."""
    global _otel_available
    if _otel_available is None:
        try:
            from opentelemetry import trace
            _otel_available = True
        except ImportError:
            _otel_available = False
    return _otel_available


def _get_tracer():
    """Get or create the fapilog tracer."""
    global _tracer
    if _tracer is None and _check_otel():
        from opentelemetry import trace
        _tracer = trace.get_tracer("fapilog")
    return _tracer


@asynccontextmanager
async def plugin_span(
    plugin_name: str,
    plugin_type: str,
    operation: str,
    *,
    attributes: dict[str, Any] | None = None,
) -> AsyncIterator[Any]:
    """Create a span for a plugin operation."""

    tracer = _get_tracer()
    if tracer is None:
        yield None
        return

    span_name = f"fapilog.{plugin_type}.{operation}"
    span_attributes = {
        "fapilog.plugin.name": plugin_name,
        "fapilog.plugin.type": plugin_type,
        "fapilog.operation": operation,
    }
    if attributes:
        for key, value in attributes.items():
            if isinstance(value, (str, int, float, bool)):
                span_attributes[f"fapilog.{key}"] = value

    from opentelemetry import trace
    from opentelemetry.trace import Status, StatusCode

    with tracer.start_as_current_span(
        span_name,
        attributes=span_attributes,
    ) as span:
        try:
            yield span
        except Exception as exc:
            span.set_status(Status(StatusCode.ERROR, str(exc)))
            span.record_exception(exc)
            raise


def get_current_trace_context() -> dict[str, str]:
    """Get current trace context for log enrichment."""
    if not _check_otel():
        return {}

    try:
        from opentelemetry import trace

        span = trace.get_current_span()
        ctx = span.get_span_context()

        if ctx and ctx.is_valid:
            return {
                "trace_id": f"{ctx.trace_id:032x}",
                "span_id": f"{ctx.span_id:016x}",
                "trace_flags": f"{ctx.trace_flags:02x}",
            }
    except Exception:
        pass

    return {}


def get_baggage_items(keys: list[str]) -> dict[str, str]:
    """Get baggage items for log enrichment."""
    if not _check_otel() or not keys:
        return {}

    try:
        from opentelemetry import baggage

        items = {}
        for key in keys:
            value = baggage.get_baggage(key)
            if value:
                items[key] = value
        return items
    except Exception:
        return {}
```

### 4. Integration with Plugin Operations

```python
# Update worker.py

from ..observability.otel_tracing import plugin_span

class LoggerWorker:

    async def _apply_enrichers(self, entry: dict[str, Any]) -> dict[str, Any]:
        enrichers = self._enrichers_getter()
        if not enrichers:
            return entry

        for enricher in enrichers:
            plugin_name = getattr(enricher, "name", type(enricher).__name__)

            async with plugin_span(
                plugin_name,
                "enricher",
                "enrich",
                attributes={"level": entry.get("level")},
            ):
                try:
                    result = await enricher.enrich(dict(entry))
                    entry.update(result)
                except Exception:
                    # ... existing error handling ...
                    pass

        return entry
```

### 5. OTEL Enricher

```python
# src/fapilog/plugins/enrichers/otel.py
"""OpenTelemetry context enricher."""

from __future__ import annotations

from typing import Any

from ...observability.otel_tracing import (
    get_baggage_items,
    get_current_trace_context,
)


class OtelContextEnricher:
    """Enriches events with OpenTelemetry context."""

    name = "otel_context"

    def __init__(
        self,
        *,
        baggage_fields: list[str] | None = None,
    ) -> None:
        self._baggage_fields = baggage_fields or []

    async def start(self) -> None:
        pass

    async def stop(self) -> None:
        pass

    async def enrich(self, event: dict[str, Any]) -> dict[str, Any]:
        data: dict[str, Any] = {}

        # Add trace context
        trace_ctx = get_current_trace_context()
        data.update(trace_ctx)

        # Add baggage items
        baggage = get_baggage_items(self._baggage_fields)
        if baggage:
            data["baggage"] = baggage

        return data

    async def health_check(self) -> bool:
        return True


PLUGIN_METADATA = {
    "name": "otel_context",
    "version": "1.0.0",
    "plugin_type": "enricher",
    "entry_point": "fapilog.plugins.enrichers.otel:OtelContextEnricher",
    "description": "Enriches logs with OpenTelemetry trace context and baggage.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "0.4.0"},
    "api_version": "1.0",
}
```

### 6. OTLP Metrics Export

```python
# src/fapilog/observability/otel_metrics.py
"""OpenTelemetry metrics export."""

from __future__ import annotations

from typing import Any


class OtlpMetricsExporter:
    """OTLP metrics exporter wrapper."""

    def __init__(
        self,
        *,
        endpoint: str = "http://localhost:4317",
        headers: dict[str, str] | None = None,
        export_interval_seconds: float = 60.0,
    ) -> None:
        self._endpoint = endpoint
        self._headers = headers or {}
        self._interval = export_interval_seconds
        self._provider = None

    def start(self) -> None:
        try:
            from opentelemetry import metrics
            from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import (
                OTLPMetricExporter,
            )
            from opentelemetry.sdk.metrics import MeterProvider
            from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader

            exporter = OTLPMetricExporter(
                endpoint=self._endpoint,
                headers=self._headers,
            )

            reader = PeriodicExportingMetricReader(
                exporter,
                export_interval_millis=int(self._interval * 1000),
            )

            self._provider = MeterProvider(metric_readers=[reader])
            metrics.set_meter_provider(self._provider)

        except ImportError:
            pass

    def stop(self) -> None:
        if self._provider:
            try:
                self._provider.shutdown()
            except Exception:
                pass
```

---

## Test Plan

### Unit Tests

1. **test_otel_tracing.py**

   - Test span creation with attributes
   - Test graceful handling when OTEL not installed
   - Test trace context extraction

2. **test_otel_enricher.py**

   - Test trace_id/span_id enrichment
   - Test baggage extraction
   - Test no-op when no context

3. **test_otel_metrics.py**
   - Test OTLP exporter configuration
   - Test graceful shutdown

### Integration Tests

1. **test_otel_integration.py** (requires OTEL collector)
   - Test full trace propagation
   - Test metrics export to collector
   - Verify span attributes

---

## Documentation Updates

1. Create `docs/integrations/opentelemetry.md`
2. Update `docs/core-concepts/context-binding.md` with OTEL
3. Add `docs/examples/otel-integration.md`

---

## Related Stories

- Story 5.7: Plugin observability metrics — **Also cancelled, deferred to enterprise**
- Story 5.2: Filter plugin type (completed)

---

## See Also

- `src/fapilog/plugins/enrichers/context_vars.py` — Existing OTEL trace context extraction
- `docs/stories/5.7.plugin-observability-metrics.md` — Related cancellation rationale
