Metadata-Version: 2.4
Name: nordlys
Version: 0.7.5
Summary: Intelligent LLM Infrastructure with Smart Model Selection
Project-URL: Homepage, https://www.nordlyslabs.com
Project-URL: Repository, https://github.com/Nordlys-Labs/nordlys
Project-URL: Documentation, https://docs.nordlyslabs.com
Project-URL: Bug Tracker, https://github.com/Nordlys-Labs/nordlys/issues
Author-email: Botir Khaltaev <botirkhaltaev@llmadaptive.uk>
License-Expression: MIT
Keywords: ai,llm,machine-learning,model-selection,nordlys
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: <3.14,>=3.11
Requires-Dist: cachetools<6,>=5.0.0
Requires-Dist: hdbscan<=0.8.40,>=0.8.33
Requires-Dist: mpmath>=1.3.0
Requires-Dist: numba>=0.56.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: polars>=1.0.0
Requires-Dist: pydantic<3,>=2.11.5
Requires-Dist: scikit-learn<1.7.0,>=1.5.0
Requires-Dist: sympy>=1.13.3
Requires-Dist: tqdm>=4.66.0
Requires-Dist: umap-learn>=0.5.0
Provides-Extra: cpu
Requires-Dist: nordlys-core<1,>=0.3.1; extra == 'cpu'
Requires-Dist: sentence-transformers<6,>=5.2.0; extra == 'cpu'
Requires-Dist: torch>=2.2.0; extra == 'cpu'
Provides-Extra: cu12
Requires-Dist: cuml-cu12>=26.2.0; extra == 'cu12'
Requires-Dist: cupy-cuda12x<14.0.0,>=13.0.0; extra == 'cu12'
Requires-Dist: nordlys-core-cu12<1,>=0.3.1; (sys_platform == 'linux') and extra == 'cu12'
Requires-Dist: sentence-transformers<6,>=5.2.0; extra == 'cu12'
Requires-Dist: torch>=2.2.0; extra == 'cu12'
Provides-Extra: cu13
Requires-Dist: cuml-cu13>=26.2.0; extra == 'cu13'
Requires-Dist: cupy-cuda13x<14.0.0,>=13.0.0; extra == 'cu13'
Requires-Dist: nordlys-core-cu13<1,>=0.3.1; (sys_platform == 'linux') and extra == 'cu13'
Requires-Dist: sentence-transformers<6,>=5.2.0; extra == 'cu13'
Requires-Dist: torch>=2.2.0; extra == 'cu13'
Description-Content-Type: text/markdown

# Nordlys

Smart LLM model routing with a checkpoint-based runtime.

## Install

```bash
uv pip install -e .
```

## Quick Start

```python
from nordlys import Dataset, Trainer, Router, ModelConfig

# 1. Define models
models = [
    ModelConfig(id="openai/gpt-4", cost_input=30.0, cost_output=60.0),
    ModelConfig(id="openai/gpt-4o-mini", cost_input=0.15, cost_output=0.6),
]

# 2. Build training dataset with binary targets per model
dataset = Dataset.from_list([
    {
        "id": "1",
        "input": "Design a database schema for this app",
        "targets": {"openai/gpt-4": 1, "openai/gpt-4o-mini": 0},
    },
    {
        "id": "2",
        "input": "Summarize this short changelog",
        "targets": {"openai/gpt-4": 0, "openai/gpt-4o-mini": 1},
    },
])

# 3. Train checkpoint, then create runtime router
checkpoint = Trainer(models=models).fit(dataset)
router = Router(checkpoint=checkpoint)

result = router.route("Implement this parser")
print(result.model_id)
```

## How It Works

1. **Clusters** similar prompts together
2. **Learns** which model performs best per cluster
3. **Routes** new prompts to the optimal model

## Runtime API

- `router.route(prompt, models=None)` routes one prompt.
- `router.route_batch(prompts, models=None)` routes a list of prompts.
- Optional `models` filter restricts candidates to specific model IDs.

## Checkpoint I/O

```python
checkpoint.to_json_file("router.json")
loaded = Router(checkpoint="router.json")
```

## Links

- [Docs](https://docs.nordlyslabs.com)
- [Issues](https://github.com/Nordlys-Labs/nordlys/issues)

## Citation

This project is inspired by the Universal Router approach:

```bibtex
@article{universalrouter2025,
  title={Universal Router: Foundation Model Routing for Arbitrary Tasks},
  author={},
  journal={arXiv preprint arXiv:2502.08773},
  year={2025},
  url={https://arxiv.org/pdf/2502.08773}
}
```

**Paper**: [Universal Router: Foundation Model Routing for Arbitrary Tasks](https://arxiv.org/pdf/2502.08773)
