# ![LLAMATOR](assets/LLAMATOR.svg)

Red Teaming python-framework for testing chatbots and LLM systems

[![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC_BY--NC--SA_4.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/llamator)](https://pypi.org/project/llamator)
[![PyPI](https://badge.fury.io/py/llamator.svg)](https://badge.fury.io/py/llamator)
[![Docker](https://img.shields.io/badge/dockerfile-gray.svg?logo=docker)](https://github.com/LLAMATOR-Core/llamator/blob/release/docker)
[![Downloads](https://pepy.tech/badge/llamator)](https://pepy.tech/project/llamator)
[![Downloads](https://pepy.tech/badge/llamator/month)](https://pepy.tech/project/llamator)
[![GitHub Repo stars](https://img.shields.io/github/stars/LLAMATOR-Core/llamator)](https://github.com/LLAMATOR-Core/llamator/stargazers)

## Install 🚀

```bash
pip install llamator==3.0.0
```

## Documentation 📚

Documentation Link: [https://LLAMATOR-Core.github.io/llamator](https://LLAMATOR-Core.github.io/llamator/)

## Guides ▶️

* [LLM Red Teaming with LLAMATOR](https://www.youtube.com/watch?v=JGTZMJsgZJU)
* [LLAMATOR: Quick Start Guide](https://www.youtube.com/watch?v=MMez57DsO1k)

## Examples 💡

* 📄 [RAG bot testing via REST API](https://github.com/LLAMATOR-Core/llamator/blob/release/examples/llamator-api.ipynb)
* 🧙‍♂️ [Gandalf web bot testing via Selenium](https://github.com/LLAMATOR-Core/llamator/blob/release/examples/llamator-selenium.ipynb)
* 💬 [Telegram bot testing via Telethon](https://github.com/LLAMATOR-Core/llamator/blob/release/examples/llamator-telegram.ipynb)
* 📱 [WhatsApp bot testing via Selenium](https://github.com/LLAMATOR-Core/llamator/blob/release/examples/llamator-whatsapp.ipynb)
* 🔗 [LangChain client testing with custom attack](https://github.com/LLAMATOR-Core/llamator/blob/release/examples/llamator-langchain-custom-attack.ipynb)
* 🐋 [Running LLAMATOR in Docker](https://github.com/LLAMATOR-Core/llamator/blob/release/docker)


## Supported Clients 🛠️

* 🌐 All LangChain clients
* 🧠 OpenAI-like API
* ⚙️ Custom Class (Telegram, WhatsApp, Selenium, etc.)

## Unique Features 🌟

* ️🗡 Support for custom attacks from the user
* 👜 Large selection of attacks on RAG / Agent / Prompt in English and Russian
* 🛡 Custom configuration of chat clients
* 📊 History of attack requests and responses in Excel and CSV format
* 📄 Test report document in DOCX format

## OWASP Classification 🔒

* 💉 [LLM01: Prompt Injection and Jailbreaks](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM01_PromptInjection.md)
* 🕵 [LLM07: System Prompt Leakage](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM07_SystemPromptLeakage.md)
* 🎭 [LLM09: Misinformation](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM09_Misinformation.md)

## Community 🌍

* 📣 [Telegram Channel — AI Security Lab](https://t.me/aisecuritylab)
* 💬 [Telegram Chat — LLAMATOR | AI Red Team Community](https://t.me/llamator)

## Supported by 🚀

* [AI Security Lab ITMO](https://ai.itmo.ru/aisecuritylab)
* [Raft Security](https://raftds.ru/)
* [AI Talent Hub](https://ai.itmo.ru/)

## License 📜

© Roman Neronov, Timur Nizamov, Nikita Ivanov

This project is licensed under the terms of the **Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International** license. See the [LICENSE](LICENSE) file for details.

[![Creative Commons License](https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
