# Supervisor Telemetry

OpenTelemetry-based observability via [t_obs](https://pypi.org/project/t-obs/).

## Architecture

```
supervise() → Root Span
    └── step spans (1.1, 1.2, 2.1, ...)
            └── OTLP Exporter → Backend
```

## Core Components

| Component | Purpose | Source |
|-----------|---------|--------|
| `init_supervision()` | Configure t_obs before agent imports | [entry.py](../entry.py) |
| `StepTracer` | Unified step span management | [traces/step_tracer.py](traces/step_tracer.py) |
| Resource attributes | Deployment env and run metadata on spans | [resources.py](resources.py) |

## Configuration

OTLP endpoint is resolved by t_obs via `ObsConfig` defaults. Override with the
`OTEL_EXPORTER_OTLP_ENDPOINT` environment variable when needed.

See [config/](config/) for configuration internals.

## How It Works

1. **Entry**: `init_supervision()` calls `t_obs.configure_observability()`, then `supervise()` creates root span
2. **Steps**: Each `@step`/`step_scope` creates child span via [StepTracer](traces/step_tracer.py)
3. **Exit**: Root span enriched with workflow data, closed, telemetry flushed to backend

## Initialization

For proper trace context in logs, initialize telemetry early:

```python
from thoughtful.supervisor import init_supervision
init_supervision()  # FIRST line of task.py
```

**Why?** Python's `logging.basicConfig(force=True)` can break trace context propagation if called before telemetry is configured. Early initialization prevents this.

See [entry.py](../entry.py) for implementation and [config/context.py](config/context.py) for state tracking.

## Public API

For custom instrumentation within bots:

| Function | Returns | Source |
|----------|---------|--------|
| `get_tracer(name)` | Namespaced OpenTelemetry tracer | [traces/__init__.py](traces/__init__.py) |
| `get_meter(name)` | Namespaced OpenTelemetry meter | [metrics/__init__.py](metrics/__init__.py) |
| `get_logger(name)` | Namespaced Python logger | [logs/__init__.py](logs/__init__.py) |

```python
from thoughtful.supervisor import get_tracer, get_meter, get_logger

tracer = get_tracer("my_module")
with tracer.start_as_current_span("custom_operation"):
    do_work()

meter = get_meter()
counter = meter.create_counter("custom_events")
counter.add(1)

logger = get_logger("my_module")
logger.info("Custom log with trace context")
```

**Note:** All telemetry signals (traces, metrics, logs) are exported to the OTLP endpoint configured by t_obs.

## Production Considerations

### Thread Safety

Global telemetry state is protected by `threading.Lock`. See [config/context.py](config/context.py).

### Shutdown Behavior

On `supervise()` exit:
1. Root span closed with final status
2. All providers flushed (traces, metrics, logs) with 5s timeout
3. StepTracer reset for clean state

### Record Tracking

Record status changes are automatically added to step spans as events via `record_status_updated()` in StepTracer.
