# LLMHosts API Reference

LLMHosts exposes two services: the **Python proxy** (port 4000) for OpenAI/Anthropic-compatible inference, and the **Next.js frontend** (port 3000) for the web dashboard and user management. Behind Traefik, both are served on `https://llmhosts.com`.

**Routing**: `/v1/*`, `/health`, `/metrics`, `/admin/*` → Python proxy. Everything else (`/`, `/api/*`) → Next.js frontend.

## Table of Contents

- [Authentication](#authentication)
- [Proxy Endpoints (OpenAI/Anthropic Compatible)](#proxy-endpoints)
- [Management Endpoints](#management-endpoints)
- [Frontend API Endpoints](#frontend-api-endpoints)
- [Custom Response Headers](#custom-response-headers)
- [Rate Limiting](#rate-limiting)
- [Error Handling](#error-handling)
- [OpenAPI Spec](#openapi-spec)

---

## Authentication

LLMHosts uses three authentication methods depending on the endpoint:

| Method | Used By | Format |
|--------|---------|--------|
| **API Key** | `/v1/*` proxy endpoints | `Authorization: Bearer <key>` |
| **Session Cookie** | `/api/*` user endpoints | HTTP-only cookie (set by Auth.js) |
| **None** | `/health`, `/metrics`, public endpoints | No auth required |

### API Key Authentication

When `LLMHOST_SECURITY_REQUIREAUTH=true` (default in production), all `/v1/*` proxy requests require an API key:

```bash
curl -H "Authorization: Bearer llmh_abc123..." \
  https://llmhosts.com/v1/chat/completions \
  -d '{"model": "llama2:7b", "messages": [{"role": "user", "content": "Hello"}]}'
```

API keys are created via the dashboard at `/dashboard/keys` or via the CLI: `llmhosts keys add`.

---

## Proxy Endpoints

These endpoints are OpenAI and Anthropic SDK-compatible. Point any client that supports the OpenAI API format to `https://llmhosts.com` as the base URL.

### POST /v1/chat/completions

OpenAI-compatible chat completion. Supports streaming.

**Auth**: API Key (when auth is enabled)

**Request**:
```json
{
  "model": "llama2:7b",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is LLMHosts?"}
  ],
  "stream": false,
  "temperature": 0.7,
  "max_tokens": 2048,
  "top_p": 1.0,
  "stop": ["\n\n"],
  "tools": []
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `model` | string | Yes | Model identifier (e.g., `llama2:7b`, `gpt-4o`, `claude-3-opus`) |
| `messages` | array | Yes | Chat messages with `role` and `content` |
| `stream` | boolean | No | Enable SSE streaming (default: `false`) |
| `temperature` | number | No | Sampling temperature 0-2 (default: `0.7`) |
| `max_tokens` | integer | No | Maximum tokens to generate |
| `top_p` | number | No | Nucleus sampling threshold |
| `stop` | string[] | No | Stop sequences |
| `tools` | array | No | Function calling tools (OpenAI format) |

**Response** (non-streaming):
```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1708560000,
  "model": "llama2:7b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "LLMHosts turns your home GPU into production AI infrastructure."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 12,
    "total_tokens": 37
  }
}
```

**Response** (streaming, `stream: true`): Server-Sent Events (SSE)
```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"LLM"},"index":0}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hosts"},"index":0}]}

data: [DONE]
```

**curl example**:
```bash
# Non-streaming
curl -X POST https://llmhosts.com/v1/chat/completions \
  -H "Authorization: Bearer llmh_abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama2:7b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Streaming
curl -X POST https://llmhosts.com/v1/chat/completions \
  -H "Authorization: Bearer llmh_abc123" \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "llama2:7b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'
```

**OpenAI Python SDK**:
```python
from openai import OpenAI

client = OpenAI(
    base_url="https://llmhosts.com/v1",
    api_key="llmh_abc123"
)

response = client.chat.completions.create(
    model="llama2:7b",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```

**Status codes**: `200` success, `400` invalid request, `401` unauthorized, `429` rate limited, `502` backend unavailable

---

### POST /v1/messages

Anthropic-compatible messages endpoint. Supports streaming.

**Auth**: API Key (when auth is enabled)

**Request**:
```json
{
  "model": "claude-3-opus",
  "messages": [
    {"role": "user", "content": "What is LLMHosts?"}
  ],
  "system": "You are a helpful assistant.",
  "max_tokens": 1024,
  "temperature": 1.0,
  "stop_sequences": [],
  "metadata": {}
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `model` | string | Yes | Model identifier |
| `messages` | array | Yes | Chat messages with `role` and `content` |
| `system` | string | No | System prompt |
| `max_tokens` | integer | Yes | Maximum tokens to generate |
| `temperature` | number | No | Sampling temperature (default: `1.0`) |
| `top_p` | number | No | Nucleus sampling |
| `top_k` | integer | No | Top-k sampling |
| `stop_sequences` | string[] | No | Stop sequences |
| `metadata` | object | No | Arbitrary metadata |

**Response**:
```json
{
  "id": "msg-abc123",
  "type": "message",
  "role": "assistant",
  "content": [
    {"type": "text", "text": "LLMHosts turns your home GPU into production AI infrastructure."}
  ],
  "model": "claude-3-opus",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 20,
    "output_tokens": 12
  }
}
```

**curl example**:
```bash
curl -X POST https://llmhosts.com/v1/messages \
  -H "Authorization: Bearer llmh_abc123" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-opus",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

**Anthropic Python SDK**:
```python
from anthropic import Anthropic

client = Anthropic(
    base_url="https://llmhosts.com/v1",
    api_key="llmh_abc123"
)

message = client.messages.create(
    model="claude-3-opus",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)
print(message.content[0].text)
```

---

### GET /v1/models

List all available models across all configured backends.

**Auth**: API Key (when auth is enabled)

**Response**:
```json
{
  "object": "list",
  "data": [
    {"id": "llama2:7b", "object": "model", "owned_by": "ollama"},
    {"id": "gpt-4o", "object": "model", "owned_by": "openai"},
    {"id": "claude-3-opus", "object": "model", "owned_by": "anthropic"}
  ]
}
```

**curl example**:
```bash
curl https://llmhosts.com/v1/models \
  -H "Authorization: Bearer llmh_abc123"
```

---

### GET /health

Health check endpoint. No authentication required.

**Response**:
```json
{
  "status": "ok",
  "version": "0.1.0",
  "timestamp": 1708560000
}
```

**curl example**:
```bash
curl https://llmhosts.com/health
```

---

### GET /metrics

Prometheus-format metrics for monitoring. No authentication required.

**Response**: Prometheus text exposition format
```
# HELP llmhost_requests_total Total proxy requests
# TYPE llmhost_requests_total counter
llmhost_requests_total{model="llama2:7b",backend="ollama"} 1523
...
```

**curl example**:
```bash
curl https://llmhosts.com/metrics
```

---

## Management Endpoints

Dashboard and management APIs on the Python backend (port 4000).

### GET /api/metrics

Aggregated metrics for the dashboard.

**Response**:
```json
{
  "uptime": 86400,
  "requests_total": 1523,
  "rpm": 12.5,
  "cache_hit_rate": 0.34,
  "avg_latency": 245.3,
  "savings_total": 15.20,
  "by_backend": {"ollama": 1200, "openai": 323},
  "by_model": {"llama2:7b": 900, "gpt-4o": 323}
}
```

### GET /api/savings

Cost savings breakdown.

**Query params**: `period` — `day`, `week`, `month`, or `all`

**Response**:
```json
{
  "total_requests": 1523,
  "local_requests": 1200,
  "cloud_requests": 323,
  "cache_hits": 518,
  "actual_cost": 4.82,
  "estimated_without_llmhost": 20.02,
  "savings_amount": 15.20,
  "savings_percent": 75.9,
  "honesty_score": 0.95,
  "methodology": "Conservative: only counts verified local inference and cache hits."
}
```

### GET /api/backends

List all configured backends and their status.

**Response**:
```json
{
  "backends": [
    {
      "id": "ollama-local",
      "type": "ollama",
      "url": "http://localhost:11434",
      "status": "healthy",
      "models": ["llama2:7b", "codellama:13b"],
      "latency_ms": 12
    }
  ]
}
```

### GET /api/cache/stats

Cache tier statistics.

**Response**:
```json
{
  "tiers_active": ["exact", "namespace", "vcache"],
  "combined_hit_rate": 0.34,
  "total_entries": 518,
  "total_size_bytes": 52428800,
  "tiers": [
    {"name": "exact", "entries": 200, "hit_rate": 0.15, "size_bytes": 10485760},
    {"name": "namespace", "entries": 150, "hit_rate": 0.12, "size_bytes": 15728640},
    {"name": "vcache", "entries": 168, "hit_rate": 0.07, "size_bytes": 26214400}
  ]
}
```

### POST /api/cache/invalidate

Invalidate cache entries.

**Request**:
```json
{
  "clear_all": false,
  "tier": "exact",
  "namespace": "openai",
  "model": "gpt-4o",
  "cache_keys": ["key1", "key2"]
}
```

All fields are optional. `clear_all: true` clears everything.

### GET /api/topology

Network topology of all discovered backends.

**Response**:
```json
{
  "nodes": [
    {
      "backend_id": "ollama-local",
      "backend_type": "ollama",
      "name": "Local Ollama",
      "url": "http://localhost:11434",
      "is_local": true,
      "status": "healthy",
      "latency_ms": 12,
      "models": ["llama2:7b"],
      "uptime_percent": 99.9
    }
  ]
}
```

### GET /api/requests

Paginated request log.

**Query params**: `limit` (1-500, default 50), `offset` (default 0)

**Response**:
```json
{
  "items": [
    {
      "id": "req-abc123",
      "timestamp": "2026-02-22T01:00:00Z",
      "model_requested": "gpt-4o",
      "model_used": "llama2:7b",
      "backend": "ollama",
      "cached": false,
      "latency_ms": 245,
      "cost": 0.0,
      "input_tokens": 25,
      "output_tokens": 50
    }
  ],
  "total": 1523,
  "offset": 0,
  "limit": 50
}
```

### WebSocket /api/ws/live

Real-time dashboard updates via WebSocket.

**Events**:
```json
{"type": "metrics", "data": {"rpm": 12.5, "cache_hit_rate": 0.34}}
{"type": "request", "data": {"id": "req-abc123", "model": "llama2:7b", "latency_ms": 245}}
{"type": "backend_status", "data": {"id": "ollama-local", "status": "healthy"}}
```

---

## Frontend API Endpoints

User-facing API routes on the Next.js frontend. All `/api/*` routes require session authentication unless noted.

### Authentication

#### POST /api/auth/register

Create a new account. Sends verification email.

**Auth**: None

**Request**:
```json
{
  "email": "user@example.com",
  "password": "minimum8chars",
  "name": "Jane Doe"
}
```

**Response** (`201`):
```json
{
  "message": "Account created successfully. Please check your email to verify your account.",
  "requiresVerification": true,
  "user": {
    "id": "clm...",
    "email": "user@example.com",
    "name": "Jane Doe",
    "plan": "free",
    "createdAt": "2026-02-22T01:00:00Z"
  }
}
```

**Status codes**: `201` created, `400` validation error or email taken, `429` rate limited

#### GET /api/auth/verify-email

Verify email address via token from verification email. Redirects on completion.

**Auth**: None
**Query params**: `token`, `email`

**Redirects to**: `/dashboard?verified=true` on success, `/auth/error?error=InvalidToken|TokenExpired|VerificationFailed` on failure

#### POST /api/auth/forgot-password

Request a password reset email. Always returns 200 to prevent email enumeration.

**Auth**: None

**Request**: `{"email": "user@example.com"}`
**Response**: `{"message": "If an account exists, a reset link has been sent."}`

#### POST /api/auth/reset-password

Reset password using token from reset email.

**Auth**: None

**Request**: `{"token": "abc123", "email": "user@example.com", "newPassword": "newpassword"}`
**Response**: `{"message": "Password reset successfully."}`

#### POST /api/auth/resend-verification

Resend the verification email.

**Auth**: None

**Request**: `{"email": "user@example.com"}`
**Response**: `{"message": "Verification email sent."}`

---

### User Management

#### GET /api/user

Get the current authenticated user's profile.

**Auth**: Session required

**Response**:
```json
{
  "user": {
    "id": "clm...",
    "email": "user@example.com",
    "name": "Jane Doe",
    "image": null,
    "plan": "free",
    "createdAt": "2026-02-22T01:00:00Z"
  }
}
```

#### PATCH /api/user

Update user profile.

**Auth**: Session required

**Request**: `{"name": "Jane Smith"}`
**Response**: Updated user object

---

### API Keys (Frontend)

#### GET /api/keys

List the current user's API keys.

**Auth**: Session required

**Response**:
```json
{
  "keys": [
    {
      "id": "clm...",
      "name": "My Key",
      "prefix": "llmh_abc1...",
      "lastUsed": "2026-02-22T01:00:00Z",
      "expiresAt": null,
      "createdAt": "2026-02-22T01:00:00Z"
    }
  ]
}
```

#### POST /api/keys

Create a new API key. The full key is shown only once in the response.

**Auth**: Session required

**Request**: `{"name": "Production Key"}`
**Response** (`201`):
```json
{
  "id": "clm...",
  "name": "Production Key",
  "key": "llmh_abc123def456...",
  "prefix": "llmh_abc1...",
  "createdAt": "2026-02-22T01:00:00Z"
}
```

**Limits**: Free plan: 2 keys. Pro plan: 10 keys. Returns `403` if limit reached.

#### DELETE /api/keys/{id}

Revoke an API key.

**Auth**: Session required

**Response**: `{"id": "clm...", "status": "revoked"}`

---

### Stripe Billing

#### POST /api/stripe/checkout

Create a Stripe Checkout session for plan upgrades.

**Auth**: Session required

**Request**: `{"priceId": "price_abc123"}`
**Response**: `{"url": "https://checkout.stripe.com/c/pay/cs_..."}`

Redirect the user to the returned URL.

#### GET /api/stripe/portal

Redirect to Stripe Customer Portal for subscription management.

**Auth**: Session required

**Response**: `302` redirect to Stripe portal

#### POST /api/stripe/webhook

Stripe webhook receiver. Verified via `stripe-signature` header.

**Auth**: Stripe signature (not session)

**Events handled**: `checkout.session.completed`, `customer.subscription.updated`, `customer.subscription.deleted`, `invoice.payment_failed`

---

### Contact

#### POST /api/contact

Submit the contact form. No authentication required.

**Request**:
```json
{
  "name": "Jane Doe",
  "email": "jane@example.com",
  "subject": "Question about pricing",
  "message": "I'd like to learn more about the team plan."
}
```

**Response**: `{"ok": true}`
**Status codes**: `200` sent, `400` validation error, `422` schema error

---

## Custom Response Headers

All responses from the Python proxy may include:

| Header | Description | Example |
|--------|-------------|---------|
| `X-Request-ID` | Unique request correlation ID for tracing | `req-a1b2c3d4` |
| `X-LLMHosts-Cost` | Request cost in USD | `0.00015` |
| `X-LLMHosts-Model` | Actual model used (may differ from requested) | `llama2:7b` |
| `X-LLMHosts-Privacy-Tier` | Privacy classification of the request | `public`, `personal`, `private` |
| `X-Accel-Buffering` | Disables proxy buffering for SSE streams | `no` |

---

## Rate Limiting

| Scope | Limit | Applies To |
|-------|-------|------------|
| Proxy (per API key) | Configurable via `rate_limit_rpm` | `/v1/*` |
| Registration | Per IP | `POST /api/auth/register` |
| Verification | Per IP | `GET /api/auth/verify-email` |
| Password reset | Per IP | `POST /api/auth/forgot-password` |

Rate-limited responses return `429 Too Many Requests` with a `Retry-After` header.

---

## Error Handling

All errors follow a consistent JSON format:

```json
{
  "error": "Human-readable error message"
}
```

Common status codes:

| Code | Meaning |
|------|---------|
| `400` | Bad request — invalid input |
| `401` | Unauthorized — missing or invalid API key/session |
| `403` | Forbidden — plan limit reached |
| `404` | Not found |
| `422` | Validation error — schema mismatch |
| `429` | Rate limited — retry after delay |
| `500` | Internal server error |
| `502` | Bad gateway — backend unavailable |

---

## OpenAPI Spec

A machine-readable OpenAPI 3.1.0 specification is available at:

```
docs/architecture/openapi.yaml
```

This can be imported into tools like Swagger UI, Postman, or used for client code generation.
