Metadata-Version: 2.4
Name: vllm-sr
Version: 0.1.0b1
Summary: vLLM Semantic Router - Intelligent routing for Mixture-of-Models
Author: vLLM-SR Team
License: Apache-2.0
Project-URL: Homepage, https://github.com/vllm-project/vllm-semantic-router
Project-URL: Documentation, https://github.com/vllm-project/vllm-semantic-router/blob/main/README.md
Project-URL: Repository, https://github.com/vllm-project/vllm-semantic-router
Project-URL: Issues, https://github.com/vllm-project/vllm-semantic-router/issues
Keywords: vllm,semantic-router,llm,routing,caching
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1.7
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: jinja2>=3.1.4
Requires-Dist: requests>=2.31.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: huggingface_hub[cli]>=0.20.0
Provides-Extra: dev
Requires-Dist: pytest>=8.4.1; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"

# vLLM Semantic Router

Intelligent Router for Mixture-of-Models (MoM).

GitHub: https://github.com/vllm-project/semantic-router

## Quick Start

### Installation

```bash
# Install from PyPI
pip install vllm-sr

# Or install from source (development)
cd src/vllm-sr
pip install -e .
```

### Usage

```bash
# Initialize vLLM Semantic Router Configuration
vllm-sr init

# Start the router (includes dashboard)
vllm-sr serve

# Open dashboard in browser
vllm-sr dashboard

# View logs
vllm-sr logs router
vllm-sr logs envoy
vllm-sr logs dashboard

# Check status
vllm-sr status

# Stop
vllm-sr stop
```

## Features

- **Router**: Intelligent request routing based on intent classification
- **Envoy Proxy**: High-performance proxy with ext_proc integration
- **Dashboard**: Web UI for monitoring and testing (http://localhost:8700)
- **Metrics**: Prometheus metrics endpoint (http://localhost:9190/metrics)

## Endpoints

After running `vllm-sr serve`, the following endpoints are available:

| Endpoint | Port | Description |
|----------|------|-------------|
| Dashboard | 8700 | Web UI for monitoring and Playground |
| API | 8888* | Chat completions API (configurable in config.yaml) |
| Metrics | 9190 | Prometheus metrics |
| gRPC | 50051 | Router gRPC (internal) |

*Default port, configurable via `listeners` in config.yaml

## License

Apache 2.0
