Metadata-Version: 2.3
Name: langchain-xinference
Version: 0.1.1
Summary: langchain xinference chain
License: MIT
Requires-Python: <4.0,>=3.9
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: aiohttp (>=3.8.0,<4.0.0)
Requires-Dist: langchain-core (>=0.3.15,<0.4.0)
Requires-Dist: pydantic (>=2.10.0,<3.0.0)
Requires-Dist: xinference-client (>=1.4.0,<2.0.0)
Project-URL: Repository, https://github.com/TheSongg/langchain-xinference
Description-Content-Type: text/markdown

# langchain-xinference

This package contains the LangChain integration with Xinference

## Installation

```bash
pip install -U langchain-xinference
```

## Dependence
Install Xinference by using pip as follows. (For more options, see [Installation page](https://inference.readthedocs.io/en/latest/getting_started/installation.html).)

```bash
pip install "xinference[all]"
```

To start a local instance of Xinference, run the following command:

```bash
$ xinference-local
```

## Chat Models

`ChatXinference` class exposes chat models from Xinference.

```python
from langchain_xinference.chat_models import ChatXinference
from langchain.prompts import PromptTemplate

llm = ChatXinference(
  server_url="http://0.0.0.0:9997",  # replace your xinference server url
  model_uid={model_uid}  # replace model_uid with the model UID return from launching the model
         )
prompt = PromptTemplate(input=["country"], template="Q: where can we visit in the capital of {country}? A:")
chain = prompt | llm

chain.invoke(input={"country": "France"})

ai_res = chain.stream(input={"country": "France"})
for chunk in ai_res:
    print(chunk.content)
```

## LLMs
`Xinference` class exposes LLMs from Xinference.

```python
from langchain_xinference.llms import Xinference
from langchain.prompts import PromptTemplate

llm = Xinference(
    server_url="http://0.0.0.0:9997",  # replace your xinference server url
    model_uid={model_uid}  # replace model_uid with the model UID return from launching the model
 )
prompt = PromptTemplate(input=["country"], template="Q: where can we visit in the capital of {country}? A:")
chain = prompt | llm
chain.invoke(input={"country": "France"})

ai_res = chain.stream(input={"country": "France"})
for chunk in ai_res:
    print(chunk)
```

