Metadata-Version: 2.1
Name: fastrag
Version: 3.0.0
Summary: An Efficient Retrieval Augmentation and Generation Framework for Intel Hardwares.
Home-page: https://github.com/IntelLabs/fastRAG
Author: Intel Labs
License: Apache-2.0
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8, <4
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: haystack-ai ==2.1.2
Requires-Dist: transformers >=4.35.2
Requires-Dist: datasets
Requires-Dist: evaluate
Requires-Dist: pandas
Requires-Dist: nltk
Requires-Dist: tqdm
Requires-Dist: numba
Requires-Dist: openpyxl
Requires-Dist: numpy
Requires-Dist: protobuf ==3.20.2
Requires-Dist: ujson
Requires-Dist: accelerate
Requires-Dist: fastapi
Requires-Dist: uvicorn
Requires-Dist: Pillow ==10.1.0
Requires-Dist: chainlit ==1.0.506
Requires-Dist: sentence-transformers ==2.3.0
Requires-Dist: events
Provides-Extra: colbert
Requires-Dist: colbert-ai ; extra == 'colbert'
Provides-Extra: dev
Requires-Dist: black ; extra == 'dev'
Requires-Dist: isort ; extra == 'dev'
Requires-Dist: pytest ; extra == 'dev'
Requires-Dist: pre-commit ; extra == 'dev'
Provides-Extra: elastic
Requires-Dist: elasticsearch-haystack ; extra == 'elastic'
Provides-Extra: faiss-cpu
Requires-Dist: faiss-cpu ; extra == 'faiss-cpu'
Provides-Extra: faiss-gpu
Requires-Dist: faiss-gpu ; extra == 'faiss-gpu'
Provides-Extra: habana
Requires-Dist: optimum[habana] ; extra == 'habana'
Provides-Extra: intel
Requires-Dist: intel-extension-for-pytorch ; extra == 'intel'
Requires-Dist: optimum[ipex,neural-compressor] ; extra == 'intel'
Provides-Extra: llama_cpp
Requires-Dist: llama-cpp-python ; extra == 'llama_cpp'
Provides-Extra: llmlingua
Requires-Dist: llmlingua ; extra == 'llmlingua'
Provides-Extra: openvino
Requires-Dist: optimum[nncf,openvino] ; extra == 'openvino'
Requires-Dist: openvino ; extra == 'openvino'
Provides-Extra: qdrant
Requires-Dist: qdrant-haystack >=3.2.0 ; extra == 'qdrant'
Provides-Extra: quantize_reranker
Requires-Dist: opencv-python-headless ; extra == 'quantize_reranker'
Requires-Dist: intel-extension-for-transformers ==1.1 ; extra == 'quantize_reranker'
Requires-Dist: neural-compressor ==2.1.1 ; extra == 'quantize_reranker'
Requires-Dist: pytrec-eval ; extra == 'quantize_reranker'
Requires-Dist: transformers >=4.28.1 ; extra == 'quantize_reranker'
Requires-Dist: datasets >=1.8.0 ; extra == 'quantize_reranker'
Requires-Dist: sentencepiece !=0.1.92 ; extra == 'quantize_reranker'
Requires-Dist: protobuf ==3.20.2 ; extra == 'quantize_reranker'
Requires-Dist: torch ==1.13.1 ; extra == 'quantize_reranker'
Requires-Dist: onnx ==1.14.0 ; extra == 'quantize_reranker'
Requires-Dist: onnxruntime ==1.13.1 ; extra == 'quantize_reranker'
Requires-Dist: onnxruntime-extensions ==0.8.0 ; extra == 'quantize_reranker'
Requires-Dist: sentence-transformers ==2.2.2 ; extra == 'quantize_reranker'

<div align="center">
    <img src="assets/fastrag_header.png" width="300"/>

---

<h4 align="center">
    <p>Build and explore efficient retrieval-augmented generative models and applications</p>
</h4>

![PyPI - Version](https://img.shields.io/pypi/v/fastrag) ![PyPI - Downloads](https://img.shields.io/pypi/dm/fastrag)

:round_pushpin: <a href="#round_pushpin-installation">Installation</a> • :rocket: <a href="components.md">Components</a> • :books: <a href="examples.md">Examples</a> • :red_car: <a href="getting_started.md">Getting Started</a> • :pill: <a href="Demo.md">Demos</a> • :pencil2: <a href="scripts/README.md">Scripts</a> • :bar_chart: <a href="benchmarks/README.md">Benchmarks</a>

</div>

fast**RAG** is a research framework for ***efficient*** and ***optimized*** retrieval augmented generative pipelines,
incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers
with a comprehensive tool-set for advancing retrieval augmented generation.

Comments, suggestions, issues and pull-requests are welcomed! :heart:

> [!IMPORTANT]
> Now compatible with Haystack v2+. Please report any possible issues you find.

## :mega: Updates

- **2024-05**: fastRAG V3 is Haystack 2.0 compatible :fire:
- **2023-12**: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; [REPLUG](https://arxiv.org/abs/2301.12652) text generation.
- **2023-06**: ColBERT index modification: adding/removing documents; see [IndexUpdater](libs/colbert/colbert/index_updater.py).
- **2023-05**: [RAG with LLM and dynamic prompt synthesis example](examples/rag-prompt-hf.ipynb).
- **2023-04**: Qdrant `DocumentStore` support.

## Key Features

- **Optimized RAG**: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
- **Optimized for Intel Hardware**: Leverage [Intel extensions for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch), [🤗 Optimum Intel](https://github.com/huggingface/optimum-intel) and [🤗 Optimum-Habana](https://github.com/huggingface/optimum-habana) for *running as optimal as possible* on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
- **Customizable**: fastRAG is built using [Haystack](https://github.com/deepset-ai/haystack) and HuggingFace. All of fastRAG's components are 100% Haystack compatible.

## :rocket: Components

For a brief overview of the various unique components in fastRAG refer to the [Components Overview](components.md) page.

<div class="tg-wrap" align="center">
<table style="undefined;table-layout: fixed; width: 600px; text-align: center;">
<colgroup>
<!-- <col style="width: 229px"> -->
<!-- <col style="width: 238px"> -->
</colgroup>
<tbody>
  <tr>
    <td colspan="2"><strong><em>LLM Backends</em></td>
  </tr>
  <tr>
    <td><a href="components.md#fastrag-running-llms-with-habana-gaudi-(dl1)-and-gaudi-2">Intel Gaudi Accelerators</a></td>
    <td><em>Running LLMs on Gaudi 2</td>
  </tr>
  <tr>
    <td><a href="components.md#fastrag-running-llms-with-onnx-runtime">ONNX Runtime</a></td>
    <td><em>Running LLMs with optimized ONNX-runtime</td>
  </tr>
  <tr>
    <td><a href="components.md#fastrag-running-quantized-llms-using-openvino">OpenVINO</a></td>
    <td><em>Running quantized LLMs using OpenVINO</td>
  </tr>
  <tr>
    <td><a href="components.md#fastrag-running-rag-pipelines-with-llms-on-a-llama-cpp-backend">Llama-CPP</a></td>
    <td><em>Running RAG Pipelines with LLMs on a Llama CPP backend</td>
  </tr>
  <tr>
    <td colspan="2"><strong><em>Optimized Components</em></td>
  </tr>
  <tr>
    <td><a href="scripts/optimizations/embedders/README.md">Embedders</a></td>
    <td>Optimized int8 bi-encoders</td>
  </tr>
  <tr>
    <td><a href="scripts/optimizations/reranker_quantization/quantization.md">Rankers</a></td>
    <td>Optimized/sparse cross-encoders</td>
  </tr>
  <tr>
    <td colspan="2"><strong><em>RAG-efficient Components</em></td>
  </tr>
  <tr>
    <td><a href="components.md#ColBERT-v2-with-PLAID-Engine">ColBERT</a></td>
    <td>Token-based late interaction</td>
  </tr>
  <tr>
    <td><a href="components.md#Fusion-In-Decoder">Fusion-in-Decoder (FiD)</a></td>
    <td>Generative multi-document encoder-decoder</td>
  </tr>
  <tr>
    <td><a href="components.md#REPLUG">REPLUG</a></td>
    <td>Improved multi-document decoder</td>
  </tr>
  <tr>
    <td><a href="components.md#ColBERT-v2-with-PLAID-Engine">PLAID</a></td>
    <td>Incredibly efficient indexing engine</td>
  </tr>
</tbody>
</table></div>

## :round_pushpin: Installation

Preliminary requirements:

- **Python** 3.8 or higher.
- **PyTorch** 2.0 or higher.

To set up the software, install from `pip` or clone the project for the bleeding-edge updates. Run the following, preferably in a newly created virtual environment:

via `pip` pypi:

```bash
pip install fastrag
```

or from a local clone:

```bash
pip install .
```

There are several dependencies to consider, depending on your specific usage (also works with `pip install fastrag[*]` package):

```bash
# Additional engines/components
pip install .[intel]               # Intel optimized backend [Optimum-intel, IPEX]
pip install .[elastic]             # Support for ElasticSearch store
pip install .[qdrant]              # Support for Qdrant store
pip install .[colbert]             # Support for ColBERT+PLAID; requires FAISS
pip install .[faiss-cpu]           # CPU-based Faiss library
pip install .[faiss-gpu]           # GPU-based Faiss library

# Development tools
pip install .[dev]
```

## License

The code is licensed under the [Apache 2.0 License](LICENSE).

## Disclaimer

This is not an official Intel product.
