# Project Notes
- Always use .venv/bin/python (not system python) for running Python commands
- OpenTelemetry Python repos (core and contrib) are at `/Users/pabcolli/github/open-telemetry/_python/`

# otelmini - Codebase Overview

A minimal OpenTelemetry Python SDK with only `opentelemetry-api` as a dependency. Exports traces, metrics, and logs via JSON-OTLP over HTTP.

## Project Overview

| Field | Value |
|-------|-------|
| Version | 0.0.1 |
| Author | Pablo Collins (Splunk) |
| License | Apache-2.0 |
| Python | 3.8+ |
| Build System | hatchling |

## Project Structure

```
otelmini/
├── src/otelmini/              # Main implementation
│   ├── trace.py               # Tracing implementation
│   ├── metric.py              # Metrics implementation
│   ├── log.py                 # Logging implementation
│   ├── types.py               # Core data types (MiniSpan, Resource, InstrumentationScope)
│   ├── processor.py           # Batch processing and threading
│   ├── _lib.py                # Base exporters, retry logic, HTTP export
│   ├── encode.py              # JSON-OTLP encoding for traces, logs, metrics
│   ├── point.py               # Metrics data structures
│   ├── distro.py              # Auto-instrumentation entry point
│   └── auto/                  # Auto-instrumentation module
├── examples/                  # OtelTest integration tests (see below)
├── tests/                     # Unit tests (pytest)
└── pyproject.toml             # Build configuration
```

## Components Implemented

### 1. Traces (`trace.py`, `types.py`, `encode.py`)

**Core Classes:**
- **MiniTracerProvider** - Manages span processors, creates MiniTracer instances, reads OTEL_SERVICE_NAME
- **MiniTracer** - Creates spans via `start_span()` and `start_as_current_span()`
- **MiniSpan** - Full span implementation with attributes, events, status, and serialization

**Exporters:**
- **HttpSpanExporter** - HTTP POST to `localhost:4318/v1/traces` with JSON-OTLP
- **ConsoleSpanExporter** - Prints JSON-OTLP to stdout

### 2. Metrics (`metric.py`, `point.py`, `encode.py`)

**Core Classes:**
- **MeterProvider** - Holds metric readers and MetricProducer
- **Meter** - Creates instruments via `create_counter()`
- **Counter** - Monotonic increment-only instrument
- **MetricProducer** - Manages registered counters, produces MetricsData

**Data Structures:**
- NumberDataPoint, Sum, Gauge, Histogram, ExponentialHistogram
- Metric, ScopeMetrics, ResourceMetrics, MetricsData
- AggregationTemporality (CUMULATIVE, DELTA)

**Exporters:**
- **HttpMetricExporter** - HTTP POST to `localhost:4318/v1/metrics` with JSON-OTLP
- **ConsoleMetricExporter** - Prints JSON-OTLP to stdout

### 3. Logs (`log.py`)

**Core Classes:**
- **LoggerProvider** - Manages log record processors
- **Logger** - Converts Python logging.LogRecord → MiniLogRecord
- **MiniLogRecord** - Structured log with timestamp, severity, body, attributes, trace context
- **BatchLogRecordProcessor** - Batching with configurable size and interval

**Exporters:**
- **HttpLogExporter** - HTTP POST to `localhost:4318/v1/logs` with JSON-OTLP
- **ConsoleLogExporter** - Prints JSON-OTLP to stdout

**Bridge:**
- **OtelBridgeLoggingHandler** - Bridges stdlib `logging` → OpenTelemetry

## Infrastructure Components

### Processing Pipeline (`processor.py`)

- **BatchProcessor** - Threaded batching with configurable batch size
- **Batcher** - Thread-safe queue with RLock
- **Timer** - Background thread for periodic exports
- **Fork awareness** - Reinitializes on process fork

### Export Infrastructure (`_lib.py`)

- **Exporter[T]** - Generic interface returning ExportResult (SUCCESS/FAILURE)
- **Retrier** - Exponential backoff (2^attempt * base_seconds), configurable max_retries
- **_HttpExporter** - HTTP with retryable status codes (429, 502, 503, 504)

### JSON-OTLP Encoding (`encode.py`)

- `encode_trace_request()` - Encode spans to JSON-OTLP
- `encode_logs_request()` - Encode logs to JSON-OTLP
- `encode_metrics_request()` - Encode metrics to JSON-OTLP
- Value encoding: bool, str, int, float, bytes, arrays, maps

## Auto-Instrumentation

**Entry Point:** `otel = "otelmini.distro:auto_instrument"`

**Components:**
- `distro.py` - Modifies PYTHONPATH, runs subprocess with instrumentation
- `auto/sitecustomize.py` - Python import hook calling `set_up_tracing()` and `set_up_logging()`
- `AutoInstrumentationManager` - Environment/config handling, provider setup

**Defaults:**
- Batch size: 144 spans
- Export interval: 12 seconds

## Data Flow

```
Instrumentation (spans/metrics/logs)
         ↓
Processor[T] (on_start → on_end)
         ↓
Batcher (queue + background thread)
         ↓
Timer (periodic flush)
         ↓
Exporter (HTTP)
         ↓
Encode to JSON-OTLP
         ↓
Backend (localhost:4318)
```

## Implementation Status

### Fully Implemented
- Traces: MiniTracer, MiniSpan, HttpSpanExporter, ConsoleSpanExporter
- Logs: Logger, MiniLogRecord, HttpLogExporter, ConsoleLogExporter
- Metrics: Counter, Meter, HttpMetricExporter, ConsoleMetricExporter
- Auto-instrumentation for tracing and logging
- Retry logic and batching infrastructure
- JSON-OTLP encoding for traces, logs, and metrics

### Partially Implemented
- Metrics: Counter, UpDownCounter, Histogram, Gauge, ObservableCounter, ObservableGauge, ObservableUpDownCounter
- Sampling: AlwaysOnSampler, AlwaysOffSampler, TraceIdRatioBasedSampler, ParentBasedSampler
- Span links: Supported via `links` parameter

### Not Implemented
- ExponentialHistogram metric type
- Context propagation and baggage
- View/aggregation customization
- Periodic metric readers

## Dependencies

**Core:**
- opentelemetry-api

**Dev:**
- pytest, coverage, mypy, oteltest

## Usage Examples

**Manual Tracing:**
```python
import os
from opentelemetry import trace
from otelmini.processor import BatchProcessor
from otelmini.trace import HttpSpanExporter, MiniTracerProvider

os.environ["OTEL_SERVICE_NAME"] = "my-service"

tp = MiniTracerProvider(BatchProcessor(HttpSpanExporter()))
trace.set_tracer_provider(tp)
tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("operation"):
    # code here
    pass

tp.shutdown()
```

**Metrics:**
```python
from otelmini.metric import MeterProvider, ManualExportingMetricReader, HttpMetricExporter

reader = ManualExportingMetricReader(HttpMetricExporter())
provider = MeterProvider(metric_readers=(reader,))
meter = provider.get_meter("app")
counter = meter.create_counter("operations")
counter.add(42)
reader.force_flush()
```

**Auto-instrumentation:**
```bash
otel python my_app.py
```

## Design Patterns

1. **Provider Pattern** - TracerProvider, MeterProvider, LoggerProvider as central registries
2. **Processor Pattern** - on_start/on_end lifecycle with BatchProcessor
3. **Exporter Pattern** - Generic Exporter[T] with HTTP implementations
4. **Context Manager** - Spans support `with` statement for automatic cleanup
5. **Fork Awareness** - ForkAware base class prevents resource leaks in multiprocessing

## Architecture Summary

The architecture cleanly separates:

- **Collection** - Providers create instrumentation objects
- **Processing** - Processors handle batching and lifecycle
- **Encoding** - JSON-OTLP serialization for wire format
- **Export** - HTTP with retry logic and graceful degradation

## Testing

### Unit Tests (`tests/`)

Standard pytest tests for individual components:

```bash
.venv/bin/python -m pytest tests/
```

### OtelTest Integration Tests (`examples/`)

End-to-end tests using [oteltest](https://pypi.org/project/oteltest/) that verify actual telemetry export. Each `ott_*.py` file contains:
1. A runnable `__main__` block that generates telemetry
2. An `*OtelTest` class that oteltest uses to configure and verify the test

**How oteltest works:**
- Spins up an OTLP collector that receives telemetry
- Installs requirements and runs the Python script
- Calls `on_stop()` with received telemetry for assertions

**Running oteltests:**
```bash
# Run a single test
.venv/bin/oteltest examples/ott_metrics.py

# Run all tests in a directory
.venv/bin/oteltest examples/

# Run comparison tests
.venv/bin/oteltest examples/compare/ott_span_otelmini.py
.venv/bin/oteltest examples/compare/ott_span_otelpython.py
```

**Output:** JSON files are saved to `examples/json/` or `examples/compare/json/` with telemetry received by the collector.

**OtelTest class interface:**
```python
class MyOtelTest:
    def environment_variables(self) -> Mapping[str, str]: ...  # Env vars to set
    def requirements(self) -> Sequence[str]: ...               # Packages to install
    def wrapper_command(self) -> str: ...                      # "" or "otel" for auto-instrumentation
    def is_http(self) -> bool: ...                             # True for HTTP/JSON export
    def on_start(self) -> Optional[float]: ...                 # Return timeout seconds or None
    def on_stop(self, tel, stdout, stderr, returncode): ...    # Assert on received telemetry
```

**Available oteltests:**

| File | Tests | Key Assertions |
|------|-------|----------------|
| `ott_manual_trace_http.py` | Manual tracing with HTTP export | 12 spans received |
| `ott_manual_logs.py` | Manual logging setup | 12 logs received |
| `ott_auto_trace.py` | Auto-instrumentation for traces | Spans received |
| `ott_auto_logs.py` | Auto-instrumentation for logs | Logs received |
| `ott_metrics.py` | Counter via auto-instrumentation | Monotonic, cumulative, value=42 |
| `ott_histogram.py` | Histogram with bucket boundaries | count=6, sum=666, buckets |
| `ott_up_down_counter.py` | UpDownCounter (allows negative) | Non-monotonic, value=7 |
| `ott_observable_gauge.py` | ObservableGauge with callback | Gauge type, value=65.5 |
| `ott_implicit_shutdown.py` | Relies on atexit handler | 13 spans (tests implicit shutdown) |
| `ott_explicit_shutdown.py` | Calls tp.shutdown() explicitly | 13 spans |
| `compare/*.py` | Side-by-side otelmini vs otel-python | Validates compatible output |
