Metadata-Version: 2.4
Name: llmpm
Version: 1.0.0
Summary: LLM Package Manager — download, run, and share AI models from the command line
License: MIT
Project-URL: Homepage, https://github.com/llmpm-dev/llmpm
Project-URL: Repository, https://github.com/llmpm-dev/llmpm
Project-URL: Issues, https://github.com/llmpm-dev/llmpm/issues
Keywords: llm,ai,models,package-manager,huggingface,llama
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: click>=8.1
Requires-Dist: rich>=13.0
Requires-Dist: huggingface_hub>=0.20
Requires-Dist: requests>=2.28
Requires-Dist: tqdm>=4.65
Requires-Dist: questionary>=2.0
Requires-Dist: humanize>=4.0
Requires-Dist: llama-cpp-python>=0.2.0
Requires-Dist: diffusers>=0.27
Requires-Dist: transformers>=4.35
Requires-Dist: accelerate>=0.24
Requires-Dist: torch>=2.0
Provides-Extra: gguf
Requires-Dist: llama-cpp-python>=0.2.0; extra == "gguf"
Provides-Extra: transformers
Requires-Dist: transformers>=4.35; extra == "transformers"
Requires-Dist: torch>=2.0; extra == "transformers"
Requires-Dist: accelerate>=0.24; extra == "transformers"
Provides-Extra: diffusion
Requires-Dist: diffusers>=0.27; extra == "diffusion"
Requires-Dist: transformers>=4.35; extra == "diffusion"
Requires-Dist: torch>=2.0; extra == "diffusion"
Requires-Dist: accelerate>=0.24; extra == "diffusion"
Provides-Extra: vision
Requires-Dist: transformers>=4.35; extra == "vision"
Requires-Dist: torch>=2.0; extra == "vision"
Requires-Dist: accelerate>=0.24; extra == "vision"
Requires-Dist: Pillow>=10.0; extra == "vision"
Provides-Extra: audio
Requires-Dist: transformers>=4.35; extra == "audio"
Requires-Dist: torch>=2.0; extra == "audio"
Requires-Dist: accelerate>=0.24; extra == "audio"
Requires-Dist: numpy>=1.24; extra == "audio"
Provides-Extra: all
Requires-Dist: llama-cpp-python>=0.2.0; extra == "all"
Requires-Dist: transformers>=4.35; extra == "all"
Requires-Dist: torch>=2.0; extra == "all"
Requires-Dist: accelerate>=0.24; extra == "all"
Requires-Dist: diffusers>=0.27; extra == "all"
Requires-Dist: Pillow>=10.0; extra == "all"
Requires-Dist: numpy>=1.24; extra == "all"

# llmpm — LLM Package Manager

> Download, run, and share AI models from the command line.

`llmpm` is a package manager for AI models, inspired by npm.
Models are sourced from [HuggingFace Hub](https://huggingface.co).
Supports text generation (GGUF via **llama.cpp** and Transformer checkpoints), image generation (Diffusion), vision, speech-to-text (ASR), and text-to-speech (TTS).

---

## Installation

### via pip (recommended)

```sh
pip install llmpm
```

### via npm

```sh
npm install -g llmpm
```

> The npm package automatically installs the Python backend via pip.

---

## Quick start

```sh
# Install a GGUF model (interactive quantisation picker)
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF

# Run it
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF
```

---

## Commands

| Command                | Description                                   |
| ---------------------- | --------------------------------------------- |
| `llmpm install <repo>` | Download and install a model from HuggingFace |
| `llmpm run <repo>`     | Run an installed model (interactive chat)     |
| `llmpm serve <repo>`   | Serve a model as an OpenAI-compatible HTTP API |
| `llmpm push <repo>`    | Upload a model to HuggingFace Hub             |
| `llmpm list`           | Show all installed models                     |
| `llmpm info <repo>`    | Show details about a model                    |
| `llmpm uninstall <repo>` | Uninstall a model                           |

---

## `llmpm install`

```sh
# Interactively choose a GGUF quantisation
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF

# Install a specific quantisation
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --quant Q4_K_M

# Install a full Transformer model
llmpm install meta-llama/Llama-3.2-1B-Instruct

# Install a single specific file
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --file Llama-3.2-1B-Instruct-Q4_K_M.gguf

# Skip prompts (pick best default)
llmpm install bartowski/Llama-3.2-1B-Instruct-GGUF --no-interactive
```

Models are stored in `~/.llmpm/models/`.

---

## `llmpm run`

`llmpm run` auto-detects the model type and launches the appropriate interactive session. It supports text generation, image generation, vision, speech-to-text (ASR), and text-to-speech (TTS) models.

### Text generation (GGUF & Transformers)

```sh
# Interactive chat
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF

# Single-turn inference
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --prompt "Explain quantum computing"

# With a system prompt
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --system "You are a helpful pirate."

# Limit response length
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --max-tokens 512

# GGUF: tune context window and GPU layers
llmpm run bartowski/Llama-3.2-1B-Instruct-GGUF --ctx 8192 --gpu-layers 32

# Transformer model
llmpm run HuggingFaceTB/SmolLM2-1.7B-Instruct
```

### Image generation (Diffusion)

Generates an image from a text prompt and saves it as a PNG on your Desktop.

```sh
# Single prompt → saves llmpm_<timestamp>.png to ~/Desktop
llmpm run amused/amused-256 --prompt "a cyberpunk city at sunset"

# Interactive session (type a prompt, get an image each time)
llmpm run amused/amused-256
```

In interactive mode type your prompt and press Enter. The output path is printed after each generation. Type `/exit` to quit.

> Requires: `pip install diffusers torch accelerate`

### Vision (image-to-text)

Describe or answer questions about an image. Pass the image file path via `--prompt`.

```sh
# Single image description
llmpm run Salesforce/blip-image-captioning-base --prompt /path/to/photo.jpg

# Interactive session: type an image path at each prompt
llmpm run Salesforce/blip-image-captioning-base
```

> Requires: `pip install transformers torch Pillow`

### Speech-to-text / ASR

Transcribe an audio file. Pass the audio file path via `--prompt`.

```sh
# Transcribe a single file
llmpm run openai/whisper-base --prompt recording.wav

# Interactive: enter an audio file path at each prompt
llmpm run openai/whisper-base
```

Supported formats depend on your installed audio libraries (wav, flac, mp3, …).

> Requires: `pip install transformers torch`

### Text-to-speech / TTS

Convert text to speech. The output WAV file is saved to your Desktop.

```sh
# Single utterance → saves llmpm_<timestamp>.wav to ~/Desktop
llmpm run suno/bark-small --prompt "Hello, how are you today?"

# Interactive session
llmpm run suno/bark-small
```

> Requires: `pip install transformers torch`

### `llmpm run` options

| Option | Default | Description |
| --- | --- | --- |
| `--prompt` / `-p` | — | Single-turn prompt or input file path (non-interactive) |
| `--system` / `-s` | — | System prompt (text generation only) |
| `--max-tokens` | `128000` | Maximum tokens to generate per response |
| `--ctx` | `128000` | Context window size (GGUF only) |
| `--gpu-layers` | `-1` | GPU layers to offload, `-1` = all (GGUF only) |
| `--verbose` | off | Show model loading output |

### Interactive session commands

These commands work in any interactive session:

| Command          | Action                                        |
| ---------------- | --------------------------------------------- |
| `/exit`          | End the session                               |
| `/clear`         | Clear conversation history (text gen only)    |
| `/system <text>` | Update the system prompt (text gen only)      |

### Model type detection

`llmpm run` reads `config.json` / `model_index.json` from the installed model to determine the pipeline type before loading any weights. The detected type is printed at startup:

```
  Detected: Image Generation (Diffusion)
  Loading model…  ✓
```

If detection is ambiguous the model falls back to the text-generation backend.

---

## `llmpm serve`

Start a local HTTP server exposing the model as an OpenAI-compatible REST API.
A browser-based chat UI is also available at `/chat`.

```sh
# Serve on the default port (8080)
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF

# Custom port and host
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --port 9000 --host 0.0.0.0

# Set the default max tokens (clients may override per-request)
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --max-tokens 2048

# GGUF options
llmpm serve bartowski/Llama-3.2-1B-Instruct-GGUF --ctx 8192 --gpu-layers 32
```

### `llmpm serve` options

| Option | Default | Description |
| --- | --- | --- |
| `--port` / `-p` | `8080` | Port to listen on (auto-increments if busy) |
| `--host` / `-H` | `localhost` | Host/address to bind to |
| `--max-tokens` | `128000` | Default max tokens per response (overridable per-request) |
| `--ctx` | `128000` | Context window size (GGUF only) |
| `--gpu-layers` | `-1` | GPU layers to offload, `-1` = all (GGUF only) |

### Endpoints

| Method | Path | Models | Description |
| --- | --- | --- | --- |
| `GET` | `/chat` | all | Browser chat / image-gen UI |
| `GET` | `/health` | all | `{"status":"ok","model":"<id>"}` |
| `POST` | `/v1/chat/completions` | text-gen | OpenAI-compatible chat inference (SSE streaming supported) |
| `POST` | `/v1/images/generations` | diffusion | Text-to-image; pass `image` (base64) for image-to-image |
| `POST` | `/v1/audio/transcriptions` | ASR | Speech-to-text |
| `POST` | `/v1/audio/speech` | TTS | Text-to-speech |

### Example API calls

```sh
# Text generation
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 256}'

# Text-to-image
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a cat in a forest", "n": 1}'

# Image-to-image (include the source image as base64 in the same endpoint)
IMAGE_B64=$(base64 -i input.png)
curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d "{\"prompt\": \"turn it into a painting\", \"image\": \"$IMAGE_B64\"}"
```

Response shape (both text-to-image and image-to-image):

```json
{
  "created": 1234567890,
  "data": [{ "b64_json": "<base64-png>" }]
}
```

---

## `llmpm push`

```sh
# Push an already-installed model
llmpm push my-org/my-fine-tune

# Push a local directory
llmpm push my-org/my-fine-tune --path ./my-model-dir

# Push as private repository
llmpm push my-org/my-fine-tune --private

# Custom commit message
llmpm push my-org/my-fine-tune -m "Add Q4_K_M quantisation"
```

Requires a HuggingFace token (run `huggingface-cli login` or set `HF_TOKEN`).

---

## Backends

| Model type              | Pipeline          | Backend                        | Extra install                                    |
| ----------------------- | ----------------- | ------------------------------ | ------------------------------------------------ |
| `.gguf` files           | Text generation   | llama.cpp via llama-cpp-python | `pip install llmpm[gguf]`                        |
| `.safetensors` / `.bin` | Text generation   | HuggingFace Transformers       | `pip install llmpm[transformers]`                |
| Diffusion models        | Image generation  | HuggingFace Diffusers          | `pip install llmpm[diffusion]`                   |
| Vision models           | Image-to-text     | HuggingFace Transformers       | `pip install llmpm[vision]`                      |
| Whisper / ASR models    | Speech-to-text    | HuggingFace Transformers       | `pip install llmpm[audio]`                       |
| TTS models              | Text-to-speech    | HuggingFace Transformers       | `pip install llmpm[audio]`                       |

Install all backends at once:

```sh
pip install llmpm[all]
```

---

## Configuration

| Variable      | Default    | Description                            |
| ------------- | ---------- | -------------------------------------- |
| `LLMPM_HOME`  | `~/.llmpm` | Root directory for models and registry |
| `HF_TOKEN`    | —          | HuggingFace API token for gated models |
| `LLPM_PYTHON` | `python3`  | Python binary used by the npm shim     |

---

## License

MIT
