# vLLM Config Wizard: The Missing Tool for GPU-Powered LLM Deployment

*Stop guessing. Start deploying. Size your LLMs with confidence.*

---

## The Problem: Deploying LLMs is a Black Box

You've been there. You want to run a 70B parameter model on your shiny new A100, but:

- **Will it fit?** 🤔
- **How much VRAM will it use?** 📊
- **What's the optimal configuration?** ⚙️
- **Will it OOM at runtime?** 💥

Until now, answering these questions required:
- Manual calculations with spreadsheets
- Trial-and-error deployments
- Reading through vLLM documentation
- Praying your GPU doesn't run out of memory

**There had to be a better way.**

---

## Introducing vLLM Config Wizard

[![PyPI version](https://badge.fury.io/py/vllm-wizard.svg)](https://pypi.org/project/vllm-wizard/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

**vLLM Config Wizard** is an open-source CLI tool that takes the guesswork out of LLM deployment. Give it your model and hardware, and it generates:

✅ **VRAM feasibility analysis** with detailed breakdown  
✅ **Optimized vLLM configuration** ready to deploy  
✅ **Performance estimates** (throughput, latency)  
✅ **Docker & Kubernetes manifests**  

All **offline**. No model downloads. No API calls. Just instant answers.

---

## 🚀 Quick Start

### Installation

```bash
pip install vllm-wizard
```

That's it. No dependencies to configure. No HuggingFace tokens needed.

### Your First Plan

```bash
# Check if LLaMA-2-7B fits on an RTX 4090
vllm-wizard plan --model meta-llama/Llama-2-7b-hf --gpu "RTX 4090"
```

**[IMAGE: Screenshot of successful plan output showing "FITS" status]**

---

## ✨ Features That Make Deployment Painless

### 1. Offline Model Estimation

Unlike other tools, **vLLM Config Wizard never downloads models**. It uses a built-in database of 80+ known models and intelligent architecture estimation based on parameter count.

```bash
# Works instantly, no internet required
vllm-wizard plan --model meta-llama/Llama-2-70b --gpu "A100 80GB"
```

**[IMAGE: Screenshot showing offline operation with model lookup]**

### 2. Smart GPU Detection

The tool knows 80+ GPUs by heart:

| GPU | VRAM | Type |
|-----|------|------|
| RTX 4090 | 24 GB | Consumer |
| RTX A6000 | 48 GB | Professional |
| A100 80GB | 80 GB | Datacenter |
| H100 | 80 GB | Datacenter |
| L40S | 48 GB | Datacenter |

Don't see your GPU? Just provide VRAM manually:

```bash
vllm-wizard plan --model mistralai/Mistral-7B --gpu "Custom GPU" --vram-gb 16
```

### 3. VRAM Breakdown That Actually Makes Sense

```
VRAM Breakdown
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Total VRAM:              24.00 GiB
Target Allocation:       21.12 GiB (88%)

Model Weights:           13.48 GiB  64%
KV Cache:                 2.30 GiB  11%
Overhead:                 1.00 GiB   5%
─────────────────────────────────────────
Headroom:                 4.34 GiB  21%

Status: ✅ FITS  |  OOM Risk: LOW
```

**[IMAGE: Screenshot of detailed VRAM breakdown table]**

### 4. Automatic Recommendations

When things don't fit, the wizard suggests solutions:

```bash
# 70B model on single GPU? It'll recommend quantization
vllm-wizard plan --model meta-llama/Llama-2-70b --gpu "RTX 4090"
```

Output:
```
⚠️  Configuration does not fit in VRAM
💡 Recommendation: Use AWQ 4-bit quantization
💡 Recommendation: Use tensor parallelism with 4x A100
```

**[IMAGE: Screenshot showing recommendations for incompatible setup]**

### 5. Generate Deployment Artifacts

```bash
# Generate vLLM serve command, Docker compose, and profile
vllm-wizard generate \
  --output-dir ./deployment \
  --model meta-llama/Llama-2-7b-hf \
  --gpu "RTX 4090" \
  --emit command,compose,profile
```

Creates:
- `serve_command.sh` - Ready-to-run vLLM command
- `docker-compose.yaml` - Container deployment
- `profile.yaml` - Reusable configuration
- `plan.json` - Full analysis data

**[IMAGE: Screenshot of generated files]**

---

## 🎯 Real-World Examples

### Example 1: Single GPU Deployment

```bash
vllm-wizard plan \
  --model meta-llama/Llama-2-7b-hf \
  --gpu "RTX 4090" \
  --max-model-len 4096 \
  --concurrency 4
```

**[IMAGE: Screenshot showing single GPU plan output]**

### Example 2: Multi-GPU with Tensor Parallelism

```bash
vllm-wizard plan \
  --model meta-llama/Llama-2-70b \
  --gpu "A100 80GB" \
  --gpus 4 \
  --tensor-parallel-size 4 \
  --interconnect nvlink
```

**[IMAGE: Screenshot showing multi-GPU configuration]**

### Example 3: Quantized Model for Consumer GPU

```bash
vllm-wizard plan \
  --model meta-llama/Llama-2-70b \
  --gpu "RTX 4090" \
  --quantization awq \
  --max-model-len 2048
```

**[IMAGE: Screenshot showing quantized model fitting on consumer GPU]**

---

## 📊 Performance Estimates

Get approximate throughput and latency metrics:

```
Performance Estimates (approximate)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Decode:        108 - 201 tokens/s
Prefill:      2282 - 5324 tokens/s
TTFT:           96 - 224 ms

Assumptions:
• Heuristic estimate; real performance depends on vLLM version
• Based on reference RTX 4090 performance scaled for 7.0B parameters
• Context length scaling assumes typical attention patterns
```

**[IMAGE: Screenshot of performance estimates section]**

---

## 🛠️ Advanced Usage

### Profile-Based Configuration

Save and reuse configurations:

```yaml
# profile.yaml
model:
  id: "meta-llama/Llama-2-7b-hf"
  dtype: "auto"
  max_model_len: 4096

hardware:
  gpu_name: "RTX 4090"
  gpus: 1

workload:
  prompt_tokens: 512
  gen_tokens: 256
  concurrency: 4
```

```bash
# Use the profile
vllm-wizard plan --profile profile.yaml
```

### JSON Output for Automation

```bash
vllm-wizard plan \
  --model meta-llama/Llama-2-7b-hf \
  --gpu "RTX 4090" \
  --json > plan.json

# Extract specific values
jq '.feasibility.fits' plan.json
jq '.config.tensor_parallel_size' plan.json
```

**[IMAGE: Screenshot showing JSON output]**

---

## 🔧 How It Works

### The Memory Model

vLLM Config Wizard calculates:

1. **Weights Memory**: `parameters × bytes_per_param`
   - FP16/BF16: 2 bytes
   - INT8: 1 byte  
   - AWQ/GPTQ: ~0.55 bytes

2. **KV Cache Memory**: 
   ```
   2 × num_kv_heads × head_dim × num_layers × 
   context_len × concurrency × dtype_bytes
   ```

3. **Overhead**: Framework + communication buffers

4. **Headroom**: Safety margin for runtime allocations

### Architecture Estimation

For unknown models, the tool estimates architecture based on parameter count:

| Parameters | Layers | Hidden Size | Heads |
|------------|--------|-------------|-------|
| <3B | 28 | 2560 | 20 |
| 3-7B | 32 | 4096 | 32 |
| 13B | 40 | 5120 | 40 |
| 30-70B | 60-80 | 6144-8192 | 48-64 |
| 400B+ | 126 | 16384 | 128 |

---

## 🤝 Open Source & Community

**vLLM Config Wizard** is open source under the Apache 2.0 license.

- **GitHub**: https://github.com/vashkelis/vllm-wizard
- **PyPI**: https://pypi.org/project/vllm-wizard/
- **Issues & Feature Requests**: GitHub Issues

### Contributing

We welcome contributions! Areas where help is needed:

- 🎯 More GPU models in the lookup table
- 🎯 Additional model architectures
- 🎯 Web UI (FastAPI-based)
- 🎯 Performance benchmarking data
- 🎯 Documentation improvements

```bash
# Clone and setup for development
git clone https://github.com/vashkelis/vllm-wizard.git
cd vllm-wizard
pip install -e ".[dev]"
pytest
```

---

## 📈 What's Next?

We're actively working on:

- [ ] Web UI for browser-based planning
- [ ] Batch planning for multiple models
- [ ] Cost estimation for cloud GPUs
- [ ] Integration with popular model registries
- [ ] Performance calibration from real benchmarks

**[IMAGE: Concept mockup of web UI]**

---

## 🎉 Get Started Now

```bash
pip install vllm-wizard
vllm-wizard plan --model meta-llama/Llama-2-7b-hf --gpu "RTX 4090"
```

Stop guessing. Start deploying with confidence.

---

## 📚 Additional Resources

- **Full Documentation**: [GitHub README](https://github.com/vashkelis/vllm-wizard#readme)
- **vLLM Documentation**: https://docs.vllm.ai/
- **GPU Memory Guide**: Understanding VRAM for LLMs

---

*Built with ❤️ by the vLLM Config Wizard team. Licensed under Apache 2.0.*

*Have questions? Open an issue on GitHub or reach out on Twitter.*

---

## Image Placeholders

**[IMAGE 1]**: Hero image showing CLI in action with colorful output  
**[IMAGE 2]**: VRAM breakdown table with visual bars  
**[IMAGE 3]**: "FITS" vs "DOES NOT FIT" comparison  
**[IMAGE 4]**: Generated artifacts (docker-compose, serve command)  
**[IMAGE 5]**: Multi-GPU tensor parallelism example  
**[IMAGE 6]**: JSON output for automation  
**[IMAGE 7]**: Web UI concept mockup  
**[IMAGE 8]**: GitHub repo screenshot with stars/forks  

---

*Last updated: February 2025*
