Metadata-Version: 2.1
Name: llama-index-postprocessor-xinference-rerank
Version: 0.1.0
Summary: llama-index postprocessor xinference rerank integration
License: MIT
Author: Your Name
Author-email: you@example.com
Requires-Python: >=3.8.1,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: llama-index-core (>=0.11.0,<0.12.0)
Description-Content-Type: text/markdown

# LlamaIndex Postprocessor Integration: Xinference Rerank

Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models.

You can find a list of built-in rerank models in Xinference from its document [Rerank Models](https://inference.readthedocs.io/en/latest/models/builtin/rerank/index.html)

To learn more about Xinference in general, visit https://inference.readthedocs.io/en/stable/models/model_abilities/rerank.html

## Installation

```shell
pip install llama-index-postprocessor-xinference-rerank
```

## Usage

**Parameters Description:**

- `model`: Model uid not model name, sometimes they may be the same (e.g., `bge-reranker-base`).
- `base_url`: base url of Xinference (e.g., `http://localhost:9997`).
- `top_n`: Top n nodes to return from reranker. (default 5).

**Nodes Rerank Example**

```python
from llama_index.postprocessor.xinference_rerank import XinferenceRerank

xi_model_uid = "xinference model uid"
xi_base_url = "xinference base url"

xi_rerank = XinferenceRerank(
    top_n=5,
    model=xi_model_uid,
    base_url=xi_base_url,
)


def test_rerank_nodes(nodes, query_str):
    response = xi_rerank.postprocess_nodes(nodes, query_str)
```

