# Burt Logger - Project Overview

## Purpose

Burt Logger is a production-ready Python package that enables customers to automatically collect LLM (Large Language Model) request/response data for training dataset creation and model fine-tuning. It's designed specifically for off-policy distillation and supervised fine-tuning (SFT) workflows.

## Key Features

### 🚀 Non-Blocking & Asynchronous

-  Background thread with queue-based processing
-  Returns immediately from `log()` calls
-  Zero impact on application performance
-  Typical overhead: ~0.001ms per log call

### 📦 Intelligent Batching

-  Batch by size (default: 10 logs)
-  Batch by time (default: 5 seconds)
-  Configurable thresholds
-  Efficient network usage

### 🛡️ Production-Ready

-  Thread-safe operations
-  Automatic retry with exponential backoff
-  Graceful error handling
-  Configurable queue overflow behavior
-  Clean shutdown with log flushing

### 🔌 Provider Agnostic

-  Works with any LLM provider (OpenAI, Anthropic, etc.)
-  Flexible schema for request/response data
-  Optional metadata support

### 📊 Observable

-  Statistics tracking (queued, sent, failed)
-  Debug logging support
-  Easy monitoring integration

## Technical Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                     Customer Application                     │
│                                                              │
│  ┌────────────┐        ┌──────────────┐                     │
│  │ LLM Call   │───────▶│ logger.log() │                     │
│  └────────────┘        └──────┬───────┘                     │
│                               │ (non-blocking)              │
│                               ▼                              │
│                    ┌────────────────────┐                    │
│                    │  Thread-Safe Queue │                    │
│                    └─────────┬──────────┘                    │
│                              │                               │
│                              ▼                               │
│                    ┌────────────────────┐                    │
│                    │  Background Thread │                    │
│                    │    (Worker Loop)   │                    │
│                    └─────────┬──────────┘                    │
│                              │                               │
│                    ┌─────────▼──────────┐                    │
│                    │  Batching Logic    │                    │
│                    │  - By size         │                    │
│                    │  - By time         │                    │
│                    └─────────┬──────────┘                    │
│                              │                               │
│                              ▼                               │
│                    ┌────────────────────┐                    │
│                    │  HTTP POST Request │                    │
│                    │  (with retry)      │                    │
│                    └─────────┬──────────┘                    │
└──────────────────────────────┼───────────────────────────────┘
                               │
                               ▼
                    ┌────────────────────┐
                    │  Backend API       │
                    │  (Customer's)      │
                    └────────────────────┘
```

## Project Structure

```
burt-logger-python/
│
├── burt_logger/              # Main package code
│   ├── __init__.py           # Package exports (LLMLogger)
│   └── logger.py             # Core implementation
│
├── tests/                    # Test suite
│   ├── __init__.py
│   └── test_logger.py        # Comprehensive tests
│
├── examples/                 # Usage examples
│   ├── simple_example.py     # Basic usage without external deps
│   ├── openai_example.py     # OpenAI integration
│   └── anthropic_example.py  # Anthropic integration
│
├── docs/                     # Documentation
│   ├── README.md             # Main documentation
│   ├── QUICKSTART.md         # Quick start guide
│   ├── CONTRIBUTING.md       # Contributor guide
│   └── PROJECT_OVERVIEW.md   # This file
│
├── Configuration Files
│   ├── setup.py              # Package setup (legacy)
│   ├── pyproject.toml        # Modern package configuration
│   ├── pytest.ini            # Pytest configuration
│   ├── MANIFEST.in           # Package manifest
│   ├── Makefile              # Development commands
│   └── .gitignore            # Git ignore patterns
│
├── Dependencies
│   ├── requirements.txt      # Runtime dependencies
│   └── requirements-dev.txt  # Development dependencies
│
├── Scripts
│   └── verify_install.py     # Installation verification
│
└── Metadata
    ├── LICENSE               # MIT License
    └── .python-version       # Python version for pyenv

```

## Core Components

### 1. LLMLogger Class (`logger.py`)

The main class that customers interact with. Key methods:

-  `__init__()` - Initialize logger with configuration
-  `log()` - Queue a log entry (non-blocking)
-  `flush()` - Force flush remaining logs
-  `shutdown()` - Gracefully shutdown and cleanup
-  `get_stats()` - Get performance statistics

### 2. Worker Thread

Background thread that:

-  Monitors the queue
-  Batches logs based on size/time
-  Sends batches to backend
-  Handles retries and errors

### 3. Queue Management

Thread-safe queue (`queue.Queue`) that:

-  Stores log entries temporarily
-  Has configurable max size
-  Handles overflow gracefully

### 4. Retry Logic

Exponential backoff retry system:

-  Retries on 5xx errors and network issues
-  No retry on 4xx client errors
-  Configurable max retries and delays

## Configuration Options

| Parameter             | Default  | Purpose               |
| --------------------- | -------- | --------------------- |
| `endpoint`            | Required | Backend API URL       |
| `api_key`             | Required | Authentication key    |
| `batch_size`          | 10       | Logs per batch        |
| `flush_interval`      | 5.0s     | Time before flushing  |
| `max_queue_size`      | 10000    | Max queued logs       |
| `max_retries`         | 3        | Retry attempts        |
| `initial_retry_delay` | 1.0s     | Initial backoff delay |
| `max_retry_delay`     | 60.0s    | Max backoff delay     |
| `timeout`             | 10.0s    | HTTP timeout          |
| `debug`               | False    | Enable debug logs     |

## API Contract

### Request Format (to Backend)

```json
{
  "logs": [
    {
      "request": {
        "model": "gpt-4",
        "messages": [...],
        "temperature": 0.7
      },
      "response": {
        "completion": "...",
        "tokens": {...}
      },
      "metadata": {
        "user_id": "123",
        "session_id": "abc"
      },
      "timestamp": 1234567890.123
    }
  ],
  "timestamp": 1234567890.456
}
```

### Expected Response from Backend

-  Success: HTTP 200, 201, or 202
-  Server Error: HTTP 5xx (will retry)
-  Client Error: HTTP 4xx (won't retry)
-  Rate Limit: HTTP 429 (will retry)

## Development Workflow

### Setup

```bash
git clone https://github.com/trainburt/burt-logger-python.git
cd burt-logger-python
pip install -e ".[dev]"
```

### Common Tasks

```bash
make test           # Run tests
make test-cov       # Tests with coverage
make lint           # Run linters
make format         # Format code
make quality        # All checks
make verify         # Verify installation
```

### Before Committing

```bash
make quality        # Ensure all checks pass
```

## Testing Strategy

### Test Categories

1. **Basic Functionality** - Initialization, logging, stats
2. **Thread Safety** - Concurrent logging, race conditions
3. **Batching** - Size triggers, time triggers, manual flush
4. **Retry Logic** - Success, failures, exponential backoff
5. **Shutdown** - Graceful cleanup, log flushing
6. **HTTP** - Headers, payload structure, authentication

### Test Coverage

Target: >80% code coverage

Current coverage areas:

-  ✅ Core logging functionality
-  ✅ Thread-safe operations
-  ✅ Batching mechanisms
-  ✅ Retry and error handling
-  ✅ Shutdown and cleanup
-  ✅ HTTP request formatting

### Running Tests

```bash
# All tests
pytest

# With coverage
pytest --cov=burt_logger --cov-report=html

# Specific test class
pytest tests/test_logger.py::TestBatching

# With verbose output
pytest -v
```

## Deployment

### Building

```bash
# Build distribution packages
python -m build

# This creates:
# - dist/burt_logger-0.1.0-py3-none-any.whl
# - dist/burt-logger-0.1.0.tar.gz
```

### Publishing

```bash
# Test on TestPyPI first
python -m twine upload --repository testpypi dist/*

# Then publish to PyPI
python -m twine upload dist/*
```

### Installation for Customers

```bash
pip install burt-logger
```

## Customer Integration

### Minimal Integration (3 lines)

```python
from burt_logger import LLMLogger

logger = LLMLogger(endpoint="...", api_key="...")
logger.log(request={...}, response={...})
logger.shutdown()
```

### Production Integration

```python
import atexit
from burt_logger import LLMLogger

# Initialize once at application startup
logger = LLMLogger(
    endpoint="https://api.example.com/logs",
    api_key="sk-production-key",
    batch_size=50,
    flush_interval=10.0,
    debug=False,
)

# Register cleanup
atexit.register(logger.shutdown)

# Use throughout application
def call_llm(prompt):
    response = llm_provider.create(prompt)
    logger.log(request={...}, response={...})
    return response
```

## Performance Characteristics

### Overhead

-  `log()` call: ~0.001ms (just queue insertion)
-  Memory per log: ~1-5KB
-  Network: Batched (efficient)

### Scalability

-  Handles 1000+ logs/second easily
-  Queue size: configurable (default 10K logs = ~50MB)
-  Single background thread (minimal resource usage)

### Network Efficiency

-  1000 logs/sec with batch_size=10 → 100 requests/sec
-  Adjustable for your backend capacity

## Monitoring

### Built-in Statistics

```python
stats = logger.get_stats()
# {
#     'logs_queued': 1000,
#     'logs_sent': 950,
#     'logs_failed': 50,
#     'batches_sent': 95,
#     'batches_failed': 5
# }
```

### Recommended Monitoring

1. **Queue Size** - `log_queue.qsize()`
2. **Failed Logs** - `stats['logs_failed']`
3. **Success Rate** - `logs_sent / logs_queued`
4. **Worker Thread** - `_worker_thread.is_alive()`

## Future Enhancements

Potential improvements for future versions:

1. **Async/await support** - For async applications
2. **Custom serializers** - Support for different data formats
3. **Compression** - Gzip compression for large payloads
4. **Disk persistence** - Save failed logs to disk
5. **Multiple endpoints** - Round-robin or failover
6. **Metrics export** - Prometheus/StatsD integration
7. **Schema validation** - Validate request/response format
8. **Rate limiting** - Client-side rate limiting

## Security Considerations

1. **API Key Storage** - Never hardcode keys, use environment variables
2. **HTTPS Required** - Always use HTTPS endpoints
3. **Data Privacy** - Ensure compliance with data regulations
4. **Network Security** - Backend should validate requests

## Dependencies

### Runtime

-  `requests` >= 2.25.0 (only external dependency)

### Development

-  `pytest` >= 7.0.0
-  `pytest-cov` >= 4.0.0
-  `black` >= 22.0.0
-  `flake8` >= 5.0.0
-  `mypy` >= 1.0.0

## License

MIT License - See LICENSE file

## Support

-  **GitHub Issues**: Bug reports and feature requests
-  **Email**: support@burt.ai
-  **Documentation**: README.md, QUICKSTART.md

## Contributors

See CONTRIBUTING.md for contribution guidelines.

---

Last Updated: November 2024
Version: 0.1.0
