Metadata-Version: 2.4
Name: llama-index-llms-cloudflare-ai-gateway
Version: 0.2.0
Summary: llama-index llms cloudflare ai gateway integration
Author-email: Chojan Shang <psiace@outlook.com>
Maintainer: psiace
License-Expression: MIT
Keywords: ai,cloudflare,gateway
Requires-Python: <4.0,>=3.10
Requires-Dist: httpx<1,>=0.24.0
Requires-Dist: llama-index-core<0.14,>=0.13.0
Description-Content-Type: text/markdown

# LlamaIndex LLM Integration for Cloudflare AI Gateway

This package provides integration between LlamaIndex and Cloudflare AI Gateway, allowing you to use multiple AI models from different providers with automatic fallback.

## Features

- **Multiple Providers**: Use OpenAI, Anthropic, Mistral, Groq, and more
- **Automatic Fallback**: If one provider fails, automatically tries the next
- **Streaming Support**: Both chat and completion streaming
- **Async Support**: Full async/await support
- **Caching**: Built-in request caching with Cloudflare AI Gateway

## Installation

```bash
pip install llama-index-llms-cloudflare-ai-gateway
```

## Quick Start

```python
from llama_index.llms.cloudflare_ai_gateway import CloudflareAIGateway
from llama_index.llms.openai import OpenAI
from llama_index.llms.anthropic import Anthropic
from llama_index.core.llms import ChatMessage

# Create LLM instances
openai_llm = OpenAI(model="gpt-4o-mini", api_key="your-openai-key")
anthropic_llm = Anthropic(
    model="claude-3-5-sonnet-20241022", api_key="your-anthropic-key"
)

# Create Cloudflare AI Gateway LLM
llm = CloudflareAIGateway(
    llms=[openai_llm, anthropic_llm],  # Try OpenAI first, then Anthropic
    account_id="your-cloudflare-account-id",
    gateway="your-ai-gateway-name",
    api_key="your-cloudflare-api-key",
)

# Use the LLM
messages = [ChatMessage(role="user", content="What is 2+2?")]
response = llm.chat(messages)
print(response.message.content)
```

## Supported Providers

- OpenAI
- Anthropic
- Mistral AI
- Groq
- DeepSeek
- Perplexity
- Replicate
- Grok
- Azure OpenAI

## Testing

```bash
# Set environment variables
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export CLOUDFLARE_ACCOUNT_ID="your-id"
export CLOUDFLARE_API_KEY="your-key"
export CLOUDFLARE_GATEWAY="your-gateway"

# Run tests
uv run pytest tests/
```
