Metadata-Version: 2.3
Name: llm_adaptive_router
Version: 0.1.3
Summary: An adaptive router for LLM model selection
Project-URL: Homepage, https://github.com/emingenc/llm_adaptive_router
Project-URL: Source, https://github.com/emingenc/llm_adaptive_router
Project-URL: Issue Tracker, https://github.com/emingenc/llm_adaptive_router/issues
Author-email: Emin Genc <emingench@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: adaptive,llm,llm-adaptive-router,llm-router,router
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.6
Requires-Dist: langchain-community==0.3.1
Requires-Dist: langchain-core==0.3.8
Requires-Dist: langchain-openai==0.2.1
Requires-Dist: langchain-text-splitters==0.3.0
Requires-Dist: langchain==0.3.1
Description-Content-Type: text/markdown

# LLM Adaptive Router

LLM Adaptive Router is a Python package that enables dynamic model selection based on query content. It uses efficient vector search for initial categorization and LLM-based fine-grained selection for complex cases. The router can adapt and learn from feedback, making it suitable for a wide range of applications.

## Features

- Dynamic model selection based on query content
- Efficient vector search for initial categorization
- LLM-based fine-grained selection for complex cases
- Adaptive learning from feedback
- Flexible configuration of routes and models
- Easy integration with LangChain and various LLM providers

## Installation

You can install LLM Adaptive Router using pip:

```bash
pip3 install llm-adaptive-router
```

## Quick Start

Here's a basic example of how to use LLM Adaptive Router:

```python
from llm_adaptive_router import AdaptiveRouter, create_route_metadata
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI

# Initialize LLM models
gpt_3_5_turbo = ChatOpenAI("gpt-3.5-turbo", temperature=0)
codex = ChatOpenAI("codex", temperature=0)
gpt_4 = ChatOpenAI("gpt-4", temperature=0)

# Define routes
routes = {
    "general": create_route_metadata(
        model="gpt-3.5-turbo",
        invoker=gpt_3_5_turbo,
        capabilities=["general knowledge"],
        cost=0.002,
        example_sentences=["What is the capital of France?", "Explain photosynthesis."]
    ),
    "code": create_route_metadata(
        model="codex",
        invoker=codex,
        capabilities=["code generation", "debugging"],
        cost=0.005,
        example_sentences=["Write a Python function to sort a list.", "Debug this JavaScript code."]
    ),
    "math": create_route_metadata(
        model="gpt-4",
        invoker=gpt_4,
        capabilities=["advanced math", "problem solving"],
        cost=0.01,
        example_sentences=["Solve this differential equation.", "Prove the Pythagorean theorem."]
    )
}

# Initialize router
router = AdaptiveRouter(
    vectorstore=Chroma(embedding_function=OpenAIEmbeddings()),
    llm=ChatOpenAI("gpt-3.5-turbo", temperature=0),
    embeddings=OpenAIEmbeddings(),
    routes=routes
)

# Use the router
query = "Write a Python function to calculate the Fibonacci sequence"
selected_model_route = router.route(query)
selected_model_name = selected_model_route.model
invoker = selected_model_route.invoker
response = invoker.invoke(query)

print(f"Selected model: {selected_model_name}")
print(f"Response: {response}")
```

## Detailed Usage

### Creating Route Metadata

Use the `create_route_metadata` function to define routes:

```python
from llm_adaptive_router import create_route_metadata

route = create_route_metadata(
    model="model_name",
    invoker=model_function,
    capabilities=["capability1", "capability2"],
    cost=0.01,
    example_sentences=["Example query 1", "Example query 2"],
    additional_info={"key": "value"}
)
```

### Initializing the AdaptiveRouter

Create an instance of `AdaptiveRouter` with your configured routes:

```python
router = AdaptiveRouter(
    vectorstore=your_vectorstore,
    llm=your_llm,
    embeddings=your_embeddings,
    routes=your_routes
)
```

### Routing Queries

Use the `route` method to select the appropriate model for a query:

```python
selected_model_route = router.route("Your query here")
selected_model_name = selected_model_route.model
invoker = selected_model_route.invoker
response = invoker.invoke("Your query here")
```

### Adding Feedback

Improve the router's performance by providing feedback:

```python
router.add_feedback(query, selected_model, performance_score)
```

### Advanced Features

- Custom Vector Stores: LLM Adaptive Router supports various vector stores. You can use any vector store that implements the `VectorStore` interface from LangChain.
- Dynamic Route Updates: You can add or remove routes dynamically:

```python
router.add_route("new_route", new_route_metadata)
router.remove_route("old_route")
```

- Adjusting Router Behavior: Fine-tune the router's behavior:

```python
router.set_complexity_threshold(0.8)
router.set_update_frequency(200)
```

