# UniverseOS
**High-Performance Traffic Shadowing & Experimentation Platform**

UniverseOS is a specialized infrastructure layer designed for safe, real-time experimentation with production traffic. While currently optimized for Large Language Models (LLMs), its core architecture provides a generic, high-concurrency shadowing gateway that enables "Parallel Universe" testing—allowing engineers to validate new models, prompts, or configurations against live production data with zero impact on end-user latency or reliability.

## Core Architecture

UniverseOS is built on a split-plane architecture designed for scale and resilience:

*   **Data Plane (C++ Gateway)**: A high-performance, non-blocking reverse proxy written in C++. It utilizes asynchronous I/O to handle high throughput with minimal overhead. The Gateway intercepts incoming requests and mirrors them to shadow backends asynchronously, ensuring the primary response path remains unaffected.
*   **Control Plane (Registry & Policy)**: A dynamic service discovery and routing engine. It allows for hot-swapping of shadow models and granular traffic routing policies (e.g., "shadow 10% of traffic to Model B") without restarting the gateway.
*   **Observability Plane**: A dedicated metrics ingestion pipeline that captures latency, throughput, and model-specific telemetry (e.g., token usage) for side-by-side performance comparison.

## Key Features

*   **Zero-Latency Shadowing**: The shadowing mechanism is completely decoupled from the primary request path. Shadow responses are processed out-of-band, guaranteeing no latency penalty for the end user.
*   **Language Agnostic Integration**: Designed as a network-level infrastructure component. While a Python SDK is provided for convenience, the system works with any client capable of making HTTP requests.
*   **Production Safety**: Failures in shadow models are isolated. If a shadow model crashes or times out, the primary request completes successfully, and the error is logged for analysis.
*   **Live A/B Testing & Evaluation**: Facilitates "dark launching" of new models. Compare the quality and cost of a new model against the production baseline using real-world inputs before exposing it to users.

## Quickstart

### 1. Deploy the Control Plane
The core infrastructure runs as a set of containerized services.

```bash
docker-compose up -d
```

### 2. Install the Python SDK
For Python applications, the SDK provides a seamless integration point.

```bash
pip install universeos
```

### 3. Integration
Wrap your existing API calls to enable automatic shadowing. The SDK handles the communication with the Gateway.

```python
from universeos import universe_shadow, init_universe

# Initialize connection to the sidecar/gateway
init_universe()

@universe_shadow
def generate_response(prompt):
    # Your existing production logic (e.g., OpenAI, Anthropic, or local LLM)
    return production_client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )
```

## Technical Specifications

*   **Gateway**: C++17, Asynchronous Socket I/O (poll/epoll)
*   **Protocol**: HTTP/1.1 (Streaming Support)
*   **Configuration**: Dynamic YAML-based policy loading
*   **SDK**: Python 3.7+ (Thread-safe, minimal dependencies)

## Building from Source

To build the high-performance Gateway and Control Plane services from source:

```bash
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j4
```
