Metadata-Version: 2.4
Name: isagellm-gateway
Version: 0.5.2.7
Summary: sageLLM Gateway - OpenAI/Anthropic compatible API Gateway (L6)
Author-email: IntelliStream Team <intellistream@github.com>
License: Apache-2.0
Project-URL: Documentation, https://intellistream.github.io/sagellm-docs/
Project-URL: Homepage, https://github.com/intellistream/sagellm-gateway
Project-URL: Issues, https://github.com/intellistream/sagellm-gateway/issues
Project-URL: Repository, https://github.com/intellistream/sagellm-gateway
Keywords: sagellm,gateway,llm,openai,anthropic,inference
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: ==3.11.*
Description-Content-Type: text/markdown
Requires-Dist: isagellm-protocol<0.6.0,>=0.5.2.8
Requires-Dist: isagellm-control-plane<0.6.0,>=0.5.2.8
Requires-Dist: numpy>=1.26.0
Requires-Dist: fastapi<1.0.0,>=0.115.0
Requires-Dist: uvicorn[standard]<1.0.0,>=0.34.0
Requires-Dist: pydantic<3.0.0,>=2.10.0
Requires-Dist: sse-starlette>=1.8.0
Requires-Dist: httpx<1.0.0,>=0.28.0
Requires-Dist: python-multipart>=0.0.5
Requires-Dist: opentelemetry-api<2.0.0,>=1.20.0
Requires-Dist: opentelemetry-sdk<2.0.0,>=1.20.0
Requires-Dist: opentelemetry-instrumentation-fastapi<1.0.0,>=0.41b0
Requires-Dist: opentelemetry-instrumentation-httpx<1.0.0,>=0.41b0
Requires-Dist: opentelemetry-exporter-otlp<2.0.0,>=1.20.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Requires-Dist: isage-pypi-publisher>=0.2.0; extra == "dev"

# sageLLM Gateway

## Protocol Compliance (Mandatory)

- MUST follow Protocol v0.1:
  https://github.com/intellistream/sagellm-docs/blob/main/docs/specs/protocol_v0.1.md
- Any globally shared definitions (fields, error codes, metrics, IDs, schemas) MUST be added to
  Protocol first.

[![CI](https://github.com/intellistream/sagellm-gateway/actions/workflows/ci.yml/badge.svg)](https://github.com/intellistream/sagellm-gateway/actions/workflows/ci.yml)
[![PyPI version](https://badge.fury.io/py/isagellm-gateway.svg)](https://badge.fury.io/py/isagellm-gateway)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)

**isagellm-gateway** - OpenAI/Anthropic Compatible API Gateway for sageLLM

## Overview

The `isagellm-gateway` package serves as the primary entry point for external API requests in the
sageLLM architecture. It provides an OpenAI-compatible interface and handles session management,
request routing, and integration with the sageLLM Control Plane.

**Key Features:**

- **OpenAI Compatibility**: Full support for `/v1/chat/completions` and `/v1/models`.
- **Session Management**: Built-in support for conversational sessions via `/sessions` endpoints.
- **Control Plane Integration**: Seamlessly connects with `isagellm-control-plane` for model
  management and routing.
- **Distributed Tracing**: Built-in OpenTelemetry support for observability across services
  (Jaeger/Zipkin compatible).
- **Protocol Compliance**: Strictly adheres to sageLLM Protocol v0.1.

## Dependencies

- **Python**: >= 3.10
- **isagellm-protocol**: >= 0.4.0.0
- **isagellm-control-plane**: Required for runtime operation.

## Installation

Install the gateway and the required control plane package:

```bash
pip install isagellm-gateway isagellm-control-plane
```

## Quick Start

### Start the Server

Start the gateway server on port 8080:

```bash
sagellm-gateway --port 8080
```

### Usage Examples

**1. Chat Completion (OpenAI API)**

```bash
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt2",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

**2. List Models**

```bash
curl http://localhost:8080/v1/models
```

**3. Health Check**

```bash
curl http://localhost:8080/health
```

## API Documentation

### Core Endpoints

- `POST /v1/chat/completions`: Generate chat completions (supports streaming).
- `POST /v1/embeddings`: Generate embeddings.
- `GET /v1/models`: List available models.
- `GET /health`: Health check and status.

### Session Management

- `POST /sessions`: Create a new session.
- `GET /sessions`: List all sessions.
- `GET /sessions/{session_id}`: Get session details.
- `DELETE /sessions/{session_id}`: Delete a session.

## Architecture

```
┌─────────────────────────────────────────┐
│         sageLLM Gateway                 │
│  • OpenAI-compatible API                │
│  • Session State Management             │
│  • Request Validation & Routing         │
└─────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────┐
│         Control Plane                   │
│  (managed by isagellm-control-plane)    │
└─────────────────────────────────────────┘
```

## Development Guide

### Prerequisites

- Python 3.10+
- Git

### Setup

1. **Clone the repository:**

   ```bash
   git clone git@github.com:intellistream/sagellm-gateway.git
   cd sagellm-gateway
   ```

1. **Initialize development environment:**

   ```bash
   # Installs dependencies and hooks
   ./quickstart.sh

   # Note: You also need the control plane for local testing
   pip install isagellm-control-plane
   ```

### Testing

Run the test suite with `pytest`:

```bash
pytest tests/ -v
```

### Linting

Check code quality with `ruff`:

```bash
ruff check .
```

## Versioning

Current Version: **0.4.0.11**

For version history, see [CHANGELOG.md](CHANGELOG.md).
