Metadata-Version: 2.1
Name: nendo-plugin-textgen
Version: 0.1.1
Summary: A text generation plugin using local LLMs or other text generation methods. Builds on top of `transformers` by Hugging Face.
Home-page: https://okio.ai
License: MIT
Keywords: AI,generative,music,okio,nendo,text generation,LLM,transformers,music analysis,song analysis
Author: Aaron Abebe
Author-email: aaron@okio.ai
Requires-Python: >=3.8,<3.11
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Multimedia :: Sound/Audio
Requires-Dist: accelerate (>=0.24.1,<0.25.0)
Requires-Dist: nendo (>=0.2.0,<0.3.0)
Requires-Dist: optimum (>=1.15.0,<2.0.0)
Requires-Dist: pydantic (>=2.4.2,<3.0.0)
Requires-Dist: tiktoken (>=0.3.3,<0.4.0)
Requires-Dist: transformers (>=4.37.2,<5.0.0)
Requires-Dist: transformers_stream_generator (>=0.0.4,<0.0.5)
Project-URL: Repository, https://github.com/okio-ai/nendo_plugin_textgen
Description-Content-Type: text/markdown

# Nendo Plugin TextGen

<br>
<p align="left">
    <img src="https://okio.ai/docs/assets/nendo_core_logo.png" width="350" alt="nendo core">
</p>
<br>

<p align="left">
<a href="https://okio.ai" target="_blank">
    <img src="https://img.shields.io/website/https/okio.ai" alt="Website">
</a>
<a href="https://twitter.com/okio_ai" target="_blank">
    <img src="https://img.shields.io/twitter/url/https/twitter.com/okio_ai.svg?style=social&label=Follow%20%40okio_ai" alt="Twitter">
</a>
<a href="https://discord.gg/gaZMZKzScj" target="_blank">
    <img src="https://dcbadge.vercel.app/api/server/XpkUsjwXTp?compact=true&style=flat" alt="Discord">
</a>
</p>

---

A text generation plugin using local LLMs or other text generation methods. 
Builds on top of `transformers` by Hugging Face.

## Features 
- Generate text from a prompt using a local LLM
- Access all LLMs from the [Hugging Face Model Hub](https://huggingface.co/models)

## Requirements

Since we depend on `transformers`, please make sure that you fulfill their requirements.
You also need Pytorch installed on your system, please refer to the [pytorch installation instructions](https://pytorch.org/get-started/locally/).

## Installation

1. [Install Nendo](https://github.com/okio-ai/nendo#installation)
2. `pip install nendo-plugin-textgen`

If you have a CUDA GPU you can install the following library for an additional speedup: 

`pip install flash-attn --no-build-isolation`

Then set `ATTN_IMPLEMENTATION=flash_attention_2` in your environment variables.


## Usage

Take a look at a basic usage example below.
For more detailed information, please refer to the [documentation](https://okio.ai/docs/plugins).

```pycon
>>> from nendo import Nendo
>>> nd = Nendo(plugins=["nendo_plugin_textgen"])

>>> nd.plugins.textgen(prompt=["Tell me about your favorite song."])[0]
```


## Contributing

Visit our docs to learn all about how to contribute to Nendo: [Contributing](https://okio.ai/docs/contributing/)


## License

Nendo: MIT License

