Metadata-Version: 2.1
Name: gai-lib
Version: 0.9
Summary: Gai/Gen is the Universal Multi-Modal Wrapper Library for LLM. The library is designed to provide a simplified and unified interface for seamless switching between multi-modal open source language models on a local machine and OpenAI APIs.
Author: kakkoii1337
Author-email: kakkoii1337@gmail.com
Classifier: Programming Language :: Python :: 3.10
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: ttt
Requires-Dist: accelerate==0.25.0; extra == "ttt"
Requires-Dist: anthropic==0.8.1; extra == "ttt"
Requires-Dist: bitsandbytes==0.41.3.post2; extra == "ttt"
Requires-Dist: exllama==0.1.0; extra == "ttt"
Requires-Dist: gradio==4.11.0; extra == "ttt"
Requires-Dist: httpx; extra == "ttt"
Requires-Dist: ipykernel==6.27.1; extra == "ttt"
Requires-Dist: llama_cpp_python==0.2.25; extra == "ttt"
Requires-Dist: openai==1.6.1; extra == "ttt"
Requires-Dist: pydantic; extra == "ttt"
Requires-Dist: python-dotenv==1.0.0; extra == "ttt"
Requires-Dist: scipy==1.11.4; extra == "ttt"
Requires-Dist: torch==2.1.2; extra == "ttt"
Requires-Dist: torchaudio==2.1.2; extra == "ttt"
Requires-Dist: torchvision==0.16.2; extra == "ttt"
Requires-Dist: transformers==4.36.2; extra == "ttt"
Requires-Dist: uvicorn==0.23.2; extra == "ttt"
Requires-Dist: fastapi; extra == "ttt"
Provides-Extra: itt
Requires-Dist: torch==2.0.1; extra == "itt"
Requires-Dist: torchvision==0.15.2; extra == "itt"
Requires-Dist: transformers==4.31.0; extra == "itt"
Requires-Dist: tokenizers<0.14,>=0.12.1; extra == "itt"
Requires-Dist: sentencepiece==0.1.99; extra == "itt"
Requires-Dist: shortuuid; extra == "itt"
Requires-Dist: accelerate==0.21.0; extra == "itt"
Requires-Dist: peft==0.4.0; extra == "itt"
Requires-Dist: bitsandbytes==0.41.0; extra == "itt"
Requires-Dist: pydantic<2,>=1; extra == "itt"
Requires-Dist: markdown2[all]; extra == "itt"
Requires-Dist: numpy; extra == "itt"
Requires-Dist: openai==1.6.1; extra == "itt"
Requires-Dist: python-dotenv; extra == "itt"
Requires-Dist: scikit-learn==1.2.2; extra == "itt"
Requires-Dist: gradio==3.35.2; extra == "itt"
Requires-Dist: gradio_client==0.2.9; extra == "itt"
Requires-Dist: requests; extra == "itt"
Requires-Dist: httpx==0.24.0; extra == "itt"
Requires-Dist: ipykernel==6.27.1; extra == "itt"
Requires-Dist: uvicorn; extra == "itt"
Requires-Dist: fastapi; extra == "itt"
Requires-Dist: einops==0.6.1; extra == "itt"
Requires-Dist: einops-exts==0.0.4; extra == "itt"
Requires-Dist: timm==0.6.13; extra == "itt"
Provides-Extra: stt
Requires-Dist: accelerate==0.25.0; extra == "stt"
Requires-Dist: ipykernel==6.27.1; extra == "stt"
Requires-Dist: openai==1.6.1; extra == "stt"
Requires-Dist: python-dotenv==1.0.0; extra == "stt"
Requires-Dist: torch==2.1.2; extra == "stt"
Requires-Dist: torchaudio==2.1.2; extra == "stt"
Requires-Dist: torchvision==0.16.2; extra == "stt"
Requires-Dist: transformers==4.36.2; extra == "stt"
Requires-Dist: uvicorn==0.23.2; extra == "stt"
Requires-Dist: pydub==0.25.1; extra == "stt"
Requires-Dist: python_multipart==0.0.6; extra == "stt"
Requires-Dist: fastapi; extra == "stt"
Provides-Extra: tts
Requires-Dist: torch==2.1.2; extra == "tts"
Requires-Dist: torchaudio==2.1.2; extra == "tts"
Requires-Dist: transformers==4.36.2; extra == "tts"
Requires-Dist: ipykernel==6.27.1; extra == "tts"
Requires-Dist: openai==1.6.1; extra == "tts"
Requires-Dist: python-dotenv==1.0.0; extra == "tts"
Requires-Dist: TTS==0.22.0; extra == "tts"
Requires-Dist: deepspeed==0.12.6; extra == "tts"
Requires-Dist: uvicorn==0.23.2; extra == "tts"
Requires-Dist: fastapi; extra == "tts"

# Gai/Gen: Universal LLM Wrapper

This is a Universal Multi-Modal Wrapper Library for LLM inferencing.

The library provides a simplified and unified interface for seamless switching between multi-modal open source language models on a local machine and OpenAI APIs.

This is intended for Developers who are targetting the use of multi-modal LLMs for both OpenAI API and local machine models.

As the main focus is on 7 billion parameters and below open source models, running on commodity hardware, the library is designed to be a singleton wrapper. Only one model is loaded and cached into memory at any one time.

The wrappers are organised under the `gen` folder according to 4 categories:

-   ttt: Text-to-Text
-   tts: Text-to-Speech
-   stt: Speech-to-Text
-   itt: Image-to-Text

---

## Setting Up

It is highly recommended that you install the following variants in separate virtual environments.

```bash
# Install library for text-to-text generation
pip install gai-aio[TTT]

# Install library for text-to-speech generation
pip install gai-aio[TTS]

# Install library for speech-to-text generation
pip install gai-aio[STT]

# Install library for image-to-text generation
pip install gai-aio[ITT]
```

or

```bash
git clone https://www.github.com/kakkoii1337/gai-aio
```

## Configuration

-   Create the default application directory `~/gai` and the default models directory `~/gai/models`. If you want to change this defaults, change `~/.gairc`

-   Copy `gai.json` from this repository into `~/gai`. This file contains the configurations for models and their respective loaders.

## API Key

-   All API keys should be stored in a `.env` file in the root directory of the project. For example,

    ```.env
    OPENAI_API_KEY=<--replace-with-your-api-key-->
    ANTHROPIC_API_KEY=<--replace-with-your-api-key-->
    ```

## Requirements

-   The instructions are tested mainly on:
    -   Windows 11 with WSL
    -   Ubuntu 20.04.2 LTS
    -   NVIDIA RTX 2060 GPU with 8GB VRAM
    -   CUDA Toolkit is required for the GPU accelerated models. Run `nvidia-smi` to check if CUDA is installed.
        If you need help, refer to this https://gist.github.com/kakkoii1337/8a8d4d0bc71fa9c099a683d1601f219e

## Credits

This library is made possible by the generosity and hardwork of the following open source projects. You are highly encouraged to check out the original source and documentations.

TTT

-   [TheBloke](https://huggingface.co/TheBloke) for all the quantized models in the demo
-   [turboderp](https://github.com/turboderp/exllama) for ExLlama
-   Meta Team for the [LLaMa2](https://ai.meta.com/llama/) Model
-   HuggingFace team for the [Transformers](https://huggingface.co/docs/transformers/llm_tutorial) library and open source models
-   [Mistral AI Team for [Mistral7B](https://mistral.ai/news/announcing-mistral-7b/) Model
-   Georgi Gerganov for [LLaMaCpp](https://github.com/ggerganov/llama.cpp)

ITT

-   Liu HaoTian for the [LLaVa](https://github.com/haotian-liu/LLaVA) Model and Library

TTS

-   [Coqui-AI](https://github.com/coqui-ai/TTS) for the xTTS Model

STT

-   [OpenAI](https://huggingface.co/openai/whisper-large-v3) for Open Sourcing Whisper v3

---

## Quick Start

**1. Setup OpenAI API Key.**

Save your OpenAI API key in a .env file in the root directory of your project.

```bash
OPENAI_API_KEY=<--replace-with-your-api-key-->
```

**2. Run GPT4.**

Run Text-to-Text generation using OpenAI by loading `gpt-4` wrapper.

```python
from gai.gen import Gaigen
gen = Gaigen.GetInstance().load('gpt-4')

response = gen.create(messages=[{'role':'USER','content':'Tell me a one paragraph short story.'},{'role':'ASSISTANT','content':''}])
print(response)
```

**3. Install Mistral7B.**

Open `~/gai/gai.json` and refer to `model_path`:

-   under `mistral-8k`, refer to `model-path` for the repo name.
-   Go to huggingface
-   Download the repo `Mistral-7B-Instruct-v0.1-GPTQ` into the `~/gai/models` folder.

**4. Run Mistral7B.**

Run Text-to-Text generation using Mistral7B by replacing `gpt-4` with `mistral-8k`.

```python
from gai.gen import Gaigen
gen = Gaigen.GetInstance().load('mistral-8k')

response = gen.create(messages=[{'role':'USER','content':'Tell me a one paragraph short story.'},{'role':'ASSISTANT','content':''}])
print(response)
```

## Examples

-   [Text-to-Text Generation](/docs/TTT.ipynb)
-   [Speech-to-Text Generation](/docs/STT.ipynb)
-   [Text-to-Speech Generation](/docs/TTS.ipynb)
-   [Image-to-Text Generation](/docs/ITT.ipynb)
