Metadata-Version: 2.1
Name: llama-trainer
Version: 0.2.1
Summary: Llama trainer utility
Home-page: https://github.com/Riccorl/llama-trainer
Author: Riccardo Orlando
Author-email: orlandorcc@gmail.com
License: Apache
Keywords: NLP deep learning transformer pytorch llama llms hf huggingface
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: torch<2.1,>=1.10
Requires-Dist: accelerate<0.23,>=0.21
Requires-Dist: peft<0.5,>=0.4
Requires-Dist: bitsandbytes==0.40.2
Requires-Dist: transformers<5.1,>=4.31
Requires-Dist: trl==0.4.7
Requires-Dist: scipy
Requires-Dist: black

# 🦙 Llama Trainer Utility

[![Upload to PyPi](https://github.com/Riccorl/llama-trainer/actions/workflows/python-publish-pypi.yml/badge.svg)](https://github.com/Riccorl/llama-trainer/actions/workflows/python-publish-pypi.yml)

A "just few lines of code" utility for fine-tuning (not only) Llama models.

To install:

```bash
pip install llama-trainer
```

### Training and Inference

#### Training

```python
from llama_trainer import LlamaTrainer
from datasets import load_dataset

dataset = load_dataset("timdettmers/openassistant-guanaco")

# define your instruction-based sample
def to_instruction_fn(sample):
    return sample["text"]

formatting_func = to_instruction_fn

output_dir = "llama-2-7b-hf-finetune"
llama_trainer = LlamaTrainer(
    model_name="meta-llama/Llama-2-7b-hf", 
    dataset=dataset, 
    formatting_func=formatting_func,
    output_dir=output_dir
)
llama_trainer.train()
```

#### Inference

```python
from llama_trainer import LlamaInfer
import transformers as tr


llama_infer = LlamaInfer(output_dir)

prompt = "### Human: Give me some output!### Assistant:"
print(llama_infer(prompt))
```
