Metadata-Version: 2.1
Name: llmlite
Version: 0.0.4
Summary: A library helps to chat with all kinds of LLMs consistently.
License: MIT
Author: kerthcet
Author-email: kerthcet@gmail.com
Requires-Python: >=3.11,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: accelerate (==0.22.0)
Requires-Dist: pydantic (>=2.3.0,<3.0.0)
Requires-Dist: sentencepiece (==0.1.99)
Requires-Dist: tokenizers (==0.13.3)
Requires-Dist: torch (==2.0.0)
Requires-Dist: transformers (==4.32.1)
Description-Content-Type: text/markdown

# llmlite

A library helps to communicate with all kinds of LLMs consistently.

## How to install

```cmd
pip install llmlite==0.0.4
```

## How to use

```python
from llmlite.apis import ChatLLM, ChatMessage

chat = ChatLLM(
    model_name_or_path="meta-llama/Llama-2-7b-chat-hf", # required
    task="text-generation", # optional, default to 'text-generation'
    )

result = chat.completion(
  messages=[
    ChatMessage(role="system", content="You're a honest assistant."),
    ChatMessage(role="user", content="There's a llama in my garden, what should I do?"),
  ],
  temperature=0.2, # optional, default to '0.2'
  max_length=2048, # optional, default to '2048'
  do_sample=True, # optional, default to False
  top_p=0.7, # optional, default to '0.7'
  top_k=3, # optional, default to '3'
)

#Output: Oh my goodness, a llama in your garden?! 😱 That's quite a surprise! 😅 As an honest assistant, I must inform you that llamas are not typically known for their gardening skills, so it's possible that the llama in your garden may have wandered there accidentally or is seeking shelter. 🐮 ...

```

## Integrations

| Model | State | Note |
| ---- | ---- | ---- |
| Llama-2 | Done ✅ | |
| ChatGLM2 | Done ✅ | |
| ChatGPT | WIP ⏳ | [issue#6](https://github.com/InftyAI/ChatLLM/issues/6) |
| Claude-2 | RoadMap 📋 | [issue#7](https://github.com/InftyAI/ChatLLM/issues/7)
| Falcon | RoadMap 📋 | [issue#8](https://github.com/InftyAI/ChatLLM/issues/8)
| StableLM | RoadMap 📋 | [issue#11](https://github.com/InftyAI/ChatLLM/issues/11) |
| ... | ... | ... |

## Contributions

🚀 All kinds of contributions are welcomed ! Please follow [Contributing](/CONTRIBUTING.md).

## Contributors

🎉 Thanks to all these contributors.

<a href="https://github.com/InftyAI/ChatLLM/graphs/contributors">
  <img src="https://contrib.rocks/image?repo=InftyAI/ChatLLM" />
</a>

