Metadata-Version: 2.1
Name: llm-nosrun
Version: 0.2
Summary: LLM plugin for models hosted by NOS Cloud
Author: Sudeep Pillai
License: Apache-2.0
Project-URL: Homepage, https://github.com/autonomi-ai/llm-nosrun
Project-URL: Changelog, https://github.com/autonomi-ai/llm-nosrun/releases
Project-URL: Issues, https://github.com/autonomi-ai/llm-nosrun/issues
Project-URL: CI, https://github.com/autonomi-ai/llm-nosrun/actions
Classifier: License :: OSI Approved :: Apache Software License
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: llm>=0.8
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-httpx; extra == "test"

# llm-nosrun

[![PyPI](https://img.shields.io/pypi/v/llm-nosrun.svg)](https://pypi.org/project/llm-nosrun/)
[![Changelog](https://img.shields.io/github/v/release/autonomi-ai/llm-nosrun?include_prereleases&label=changelog)](https://github.com/autonomi-ai/llm-nosrun/releases)
[![Tests](https://github.com/autonomi-ai/llm-nosrun/workflows/Test/badge.svg)](https://github.com/autonomi-ai/llm-nosrun/actions?query=workflow%3ATest)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/autonomi-ai/llm-nosrun/blob/main/LICENSE)

[LLM](https://llm.datasette.io/) plugin for models hosted by [NOS Cloud](https://app.nos.run/)

## Installation

First, [install the LLM command-line utility](https://llm.datasette.io/en/stable/setup.html).

Now install this plugin in the same environment as LLM.
```bash
llm install llm-nosrun
```
## Configuration

You will need an API key from NOS Cloud. You can [obtain one here](https://app.nos.run/).

You can set that as an environment variable called `LLM_NOSRUN_KEY`, or add it to the `llm` set of saved keys using:

```bash
llm keys set nosrun
```
```
Enter key: <paste key here>
```

## Usage

To list available models, run:
```bash
llm models list
```
You should see a list that looks something like this:
```
NOSRun: TinyLlama/TinyLlama-1.1B-Chat-v1.0
NOSRun: meta-llama/Llama-2-7b-chat-hf
NOSRun: meta-llama/Llama-2-13b-chat-hf
NOSRun: meta-llama/Llama-2-70b-chat-hf
NOSRun: HuggingFaceH4/zephyr-7b-beta
NOSRun: HuggingFaceH4/tiny-random-LlamaForCausalLM
NOSRun: NousResearch/Yarn-Mistral-7b-128k
NOSRun: mistralai/Mistral-7B-Instruct-v0.2
NOSRun: mistralai/Mixtral-8x7B-Instruct-v0.1
NOSRun: TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ
NOSRun: TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ
NOSRun: mlabonne/phixtral-2x2_8
NOSRun: mlabonne/phixtral-4x2_8

```
To run a prompt against a model, pass its full model ID to the `-m` option, like this:
```bash
llm -m TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
  'Five strident names for a pet walrus' \
  --system 'You love coming up with creative names for pets'
```
You can set a shorter alias for a model using the `llm aliases` command like so:
```bash
llm aliases set tinyllama TinyLlama/TinyLlama-1.1B-Chat-v1.0
```
Now you can prompt Llama 2 70B using:
```bash
cat llm_nosrun.py | \
  llm -m tinyllama -s 'explain this code'
```
## Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:
```bash
cd llm-nosrun
python3 -m venv venv
source venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
pip install -e '.[test]'
```
To run the tests:
```bash
pytest
```
