# langchain-litellm

This package contains the [LangChain](https://github.com/langchain-ai/langchain) integration with [LiteLLM](https://github.com/BerriAI/litellm)

## Installation

```bash
pip install -qU langchain-litellm
```
# ChatLiteLLM

[LiteLLM](https://github.com/BerriAI/litellm) is a library that simplifies calling Anthropic, Azure, Huggingface, Replicate, etc. 

This notebook covers how to get started with using Langchain + the LiteLLM I/O library. 

## Overview
### Integration details

| Class | Package | Local | Serializable | JS support| Package downloads | Package latest |
| :---  | :--- | :---: | :---: |  :---: | :---: | :---: |
| [ChatLitellm](https://python.langchain.com/docs/integrations/chat/litellm/) | [langchain-litellm](https://python.langchain.com/api_reference/litellm/index.html)| ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-litellm?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-litellm?style=flat-square&label=%20) |

### Model features
| [Tool calling](https://python.langchain.com/docs/how_to/tool_calling/) | [Structured output](https://python.langchain.com/docs/how_to/structured_output/) | JSON mode | Image input | Audio input | Video input | [Token-level streaming](https://python.langchain.com/docs/integrations/chat/litellm/#chatlitellm-also-supports-async-and-streaming-functionality) | [Native async](https://python.langchain.com/docs/integrations/chat/litellm/#chatlitellm-also-supports-async-and-streaming-functionality) | [Token usage](https://python.langchain.com/docs/how_to/chat_token_usage_tracking/) | [Logprobs](https://python.langchain.com/docs/how_to/logprobs/) |
| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |
| ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | 

### Setup
To access ChatLiteLLM models you'll need to create an OpenAI, Anthropic, Azure, Replicate, OpenRouter, Hugging Face, Together AI or Cohere account, get an API key, and export it as an environment variable.
## Chat Models

`ChatLiteLLM` class exposes chat models from [LiteLLM](https://github.com/BerriAI/litellm).

```python
from langchain_litellm.chat_models import ChatLiteLLM
from langchain_core.messages import HumanMessage
chat = ChatLiteLLM(model="gpt-3.5-turbo")
messages = [
    HumanMessage(
        content="Translate this sentence from English to French. I love programming."
    )
]
response = chat(messages)
print(response)
```

## `ChatLiteLLM` also supports async and streaming functionality:
```python
from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler
await chat.agenerate([messages])
chat = ChatLiteLLM(
    streaming=True,
    verbose=True,
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
)
response = chat(messages)
print(response)
```