Metadata-Version: 2.1
Name: TinyAGI
Version: 0.0.1
Summary: A modular package for interacting with llama.cpp models via a server or CLI, supporting LoRA and JSON configurations.
Home-page: https://github.com/SullyGreene/TinyAGI
Author: SullyGreene
Author-email: your.email@example.com
Project-URL: Bug Tracker, https://github.com/SullyGreene/TinyAGI/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: llama-cpp-python (>=0.1.48)
Requires-Dist: flask

# TinyAGI

A modular Python package that provides both a Flask API server and a command-line interface (CLI) for interacting with `llama.cpp` models, including support for LoRA addons and configuration via JSON files.

## Features

- **Flask API Server**: Serve models via HTTP endpoints.
- **Command-Line Interface (CLI)**: Interact with models directly from the terminal.
- **LoRA Support**: Utilize LoRA adapters with your models.
- **JSON Configuration**: Manage settings and parameters via JSON files.
- **Modular Design**: Use components independently in your projects.

## Installation

Install from PyPI:

```bash
pip install TinyAGI
```

## Usage

### Configuration

Create a `config.json` file in your working directory:

```json
{
  "model_path": "path/to/your/model.bin",
  "lora_path": "path/to/your/lora_adapter.bin",
  "model_params": {
    "n_ctx": 2048,
    "n_gpu_layers": 20
  },
  "inference_params": {
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 512
  }
}
```

### Running the Server

Start the Flask API server:

```bash
tinyagi-server
```

The server will be accessible at `http://localhost:5000`.

### Using the CLI

#### Generate Text from a Prompt

```bash
tinyagi-cli generate --prompt "Why is the sky blue?"
```

#### Stream Output

```bash
tinyagi-cli generate --prompt "Tell me a story." --stream
```

#### Generate Embeddings

```bash
tinyagi-cli embed --input "Text to embed"
```

#### Reload the Model Configuration

```bash
tinyagi-cli reload --config new_config.json
```

## API Endpoints

- `POST /chat`: Chat with the model using messages.
- `POST /generate`: Generate text from a prompt.
- `POST /embed`: Generate embeddings.
- `POST /reload`: Reload the model configuration.
- `GET /config`: Get the current configuration.

### Example: Chat Endpoint

Request:

```bash
curl -X POST http://localhost:5000/chat \
  -H 'Content-Type: application/json' \
  -d '{
        "messages": [
            {"role": "user", "content": "Hello! Who are you?"}
        ]
    }'
```

Response:

```json
{
  "response": "Assistant: I am an AI language model developed to assist with your questions."
}
```

## Contributing

Contributions are welcome! Please open an issue or submit a pull request on GitHub.

## License

This project is licensed under the MIT License.

## Contact

For any inquiries or support, please contact [SullyGreene](https://github.com/SullyGreene).

