<h1 align="center">
  phidata
</h1>
<p align="center">
    <em>AI Toolkit for Engineers</em>
</p>

<p align="center">
<a href="https://python.org/pypi/phidata" target="_blank">
    <img src="https://img.shields.io/pypi/v/phidata?color=blue&label=version" alt="version">
</a>
<a href="https://github.com/phidatahq/phidata" target="_blank">
    <img src="https://img.shields.io/badge/python->=3.7-blue" alt="pythonversion">
</a>
<a href="https://github.com/phidatahq/phidata" target="_blank">
    <img src="https://pepy.tech/badge/phidata" alt="downloads">
</a>
<a href="https://github.com/phidatahq/phidata/actions/workflows/build.yml" target="_blank">
    <img src="https://github.com/phidatahq/phidata/actions/workflows/build.yml/badge.svg" alt="build-status">
</a>
</p>

---

<h2 align="center">
  🧰  Phidata is a everything-included toolkit for building products using LLMs
</h2>

<br />

It solves the problem of building LLM applications by providing:

### 💻 Software layer

- Components for **building** LLM apps: **RAG, Agents, Workflows**
- Components for **extending** LLM apps: **VectorDbs, Storage, Memory, Cache**
- Components for **monitoring** LLM apps: **Model Inputs/Outputs, Quality, Cost**
- Components for **improving** LLM apps: **Fine-tuning, RLHF**
- Components for **securing** LLM apps: **I/O Validation, Guardrails**

### 📱 Application layer

- Tools for serving LLM apps: **FastApi, Django, Streamlit**
- Tools for serving LLM components: **PgVector, Postgres, Redis**

### 🌉 Infrastructure layer

- Infrastructure for running LLM apps locally: **Docker**
- Infrastructure for running LLM apps in production: **AWS**
- Best practices like testing, formatting, CI/CD, security and secret management.

Our goal is to integrate the 3 layers of software development using 1 toolkit and build production-grade LLM Apps.

## 🚀 How it works

- **Create your LLM app** from a template using `phi ws create`
- **Run your app locally** using `phi ws up dev:docker`
- **Run your app on AWS** using `phi ws up prd:aws`

## 🎯 For more information:

- Read the <a href="https://docs.phidata.com" target="_blank">documentation</a>
- Read about <a href="https://docs.phidata.com/intro/basics" target="_blank">phidata basics</a>
- Chat with us on <a href="https://discord.gg/4MtYHHrgA8" target="_blank">Discord</a>
- Email us at <a href="mailto:help@phidata.com" target="_blank">help@phidata.com</a>

## 👩‍💻 Quickstart: Build a LLM App 🧑‍💻

Let's build a **LLM App** with GPT-4 using PgVector for Knowledge Base and Storage. We'll serve the app using Streamlit and FastApi, running locally on Docker.

> Install <a href="https://docs.docker.com/desktop/install/mac-install/" target="_blank">docker desktop</a> before moving ahead

### Setup

Open the `Terminal` and create a python virtual environment

```bash
python3 -m venv ~/.venvs/llmenv
source ~/.venvs/llmenv/bin/activate
```

Install phidata

```bash
pip install phidata
```

### Create your codebase

Create your codebase using the `llm-app` template that is pre-configured with FastApi, Streamlit and PgVector. Use this codebase as a starting point for your LLM product.

```bash
phi ws create -t llm-app -n llm-app
```

<img src="https://github.com/phidatahq/phidata/assets/22579644/dee7b884-028e-48a0-9074-4deb7a055b97" height=600  alt="create-llm-app"/>

This will create a folder named `llm-app` with the following structure:

```bash
llm-app
├── api               # directory for FastApi routes
├── app               # directory for Streamlit apps
├── db                # directory for database components
├── llm               # directory for LLM components
    ├── conversations       # LLM conversations
    ├── knowledge_base.py   # LLM knowledge base
    └── storage.py          # LLM storage
├── notebooks         # directory for Jupyter notebooks
├── Dockerfile        # Dockerfile for the application
├── pyproject.toml    # python project definition
├── requirements.txt  # python dependencies generated by pyproject.toml
├── scripts           # directory for helper scripts
├── utils             # directory for shared utilities
└── workspace
    ├── dev_resources.py  # Dev resources running locally
    ├── prd_resources.py  # Production resources running on AWS
    ├── jupyter           # Jupyter notebook resources
    ├── secrets           # directory for storing secrets
    └── settings.py       # Phidata workspace settings
```

### Set OpenAI Key

Set the `OPENAI_API_KEY` environment variable. You can get one <a href="https://platform.openai.com/account/api-keys" target="_blank">from OpenAI here</a>.

```bash
export OPENAI_API_KEY=sk-***
```

### Serve you LLM App using Streamlit

<a href="https://streamlit.io" target="_blank">Streamlit</a> allows us to build micro front-ends for our LLM App and is extremely useful for building basic applications in pure python. Start the `app` group using:

```bash
phi ws up --group app
```

<img src="https://github.com/phidatahq/phidata/assets/22579644/dee35e20-9fe5-4623-af5e-2e8a6ffac58c" height=600  alt="run-llm-app"/>

**Press Enter** to confirm and give a few minutes for the image to download (only the first time). Verify container status and view logs on the docker dashboard.

Open <a href="http://localhost:8501" target="_blank">localhost:8501</a> to view streamlit apps that you can customize and make your own.

### Chat with PDFs

- Click on **Chat with PDFs** in the sidebar
- Enter a username and wait for the knowledge base to load.
- Choose between `RAG` or `Autonomous` mode.
- Ask "How do I make chicken tikka salad?"
- The streamlit apps are defined in the `app` folder.
- The `Conversations` powering these apps are defined in the `llm/conversations` folder.
- The Streamlit application is defined in the `workspace/dev_resources.py` file.

<img width="1573" alt="chat-with-pdf" src="https://github.com/phidatahq/phidata/assets/22579644/8529aad8-f74c-464d-8bf8-2272a3281b25">

### Serve your LLM App using FastApi

Streamlit is great for building micro front-ends but any production application will be built using a front-end framework like `next.js` backed by a RestApi built with a framework like `FastApi`.

Your LLM App comes pre-configured with FastApi, start the `api` group using:

```bash
phi ws up --group api
```

**Press Enter** to confirm and give a few minutes for the image to download.

### View API Endpoints

- Open <a href="http://localhost:8000/docs" target="_blank">localhost:8000/docs</a> to view the API Endpoints.
- Test the `v1/pdf/conversation/chat` endpoint with `{"message": "how do I make chicken tikka salad"}`
- Checkout the `api/routes/pdf_routes.py` file for endpoints that you can integrate with your front-end or product.
- The FastApi application is defined in the `workspace/dev_resources.py` file.

### Delete local resources

Play around and stop the workspace using:

```bash
phi ws down
```

### Run your LLM App on AWS

Read how to <a href="https://docs.phidata.com/guides/llm-app#run-on-aws" target="_blank">run your LLM App on AWS here</a>.

### More information:

- Read the <a href="https://docs.phidata.com" target="_blank">documentation</a>
- Read about <a href="https://docs.phidata.com/intro/basics" target="_blank">phidata basics</a>
- Chat with us on <a href="https://discord.gg/4MtYHHrgA8" target="_blank">Discord</a>
- Email us at <a href="mailto:help@phidata.com" target="_blank">help@phidata.com</a>
