# VPS Deployment Guide

Deploy LLMHosts on a cloud VPS to get a dedicated AI proxy with a public IP, automatic TLS, and optional GPU acceleration. This guide covers provider selection, initial server setup, deployment, and remote access.

## Provider Recommendations

### GPU VPS Providers

| Provider | GPU Options | Starting Price | Notes |
|----------|------------|----------------|-------|
| **Hetzner** | None (CPU only) | ~$4/mo (CX22) | Best value for CPU proxy + cloud API routing |
| **Lambda** | A10 (24GB), A100 (40/80GB), H100 | ~$0.50/hr (A10) | On-demand GPU, great for inference |
| **RunPod** | A40, A100, H100, consumer GPUs | ~$0.40/hr (RTX 4090) | Spot pricing available, good API |
| **Vast.ai** | Consumer + datacenter GPUs | ~$0.20/hr (RTX 3090) | Marketplace model, cheapest GPU |
| **OVH** | T4 (16GB), V100 (32GB) | ~$80/mo (T4) | Fixed monthly pricing, EU datacenters |
| **Vultr** | A100 (80GB) | ~$2.60/hr | Simple UI, good docs |

### CPU-Only Providers (proxy + cloud API routing)

If you only route to cloud APIs (OpenAI, Anthropic) and use LLMHosts for caching, routing, and cost optimization, you do not need a GPU:

| Provider | Specs | Starting Price |
|----------|-------|----------------|
| **Hetzner** | 2 vCPU, 4GB RAM | ~$4/mo |
| **DigitalOcean** | 2 vCPU, 4GB RAM | ~$24/mo |
| **Linode** | 2 vCPU, 4GB RAM | ~$24/mo |
| **Oracle Cloud** | 4 ARM cores, 24GB RAM | Free tier |

A 2 vCPU / 4 GB RAM instance handles LLMHosts proxy traffic comfortably. The proxy itself is lightweight -- the GPU requirement comes from local model inference, not from LLMHosts itself.

## Initial Server Setup

These steps assume a fresh Ubuntu 24.04 LTS server. Adapt for your distribution.

### 1. Connect and update

```bash
ssh root@YOUR_SERVER_IP

apt update && apt upgrade -y
```

### 2. Create a non-root user

```bash
adduser llmhost
usermod -aG sudo llmhost

# Copy SSH keys
mkdir -p /home/llmhost/.ssh
cp ~/.ssh/authorized_keys /home/llmhost/.ssh/
chown -R llmhost:llmhost /home/llmhost/.ssh
chmod 700 /home/llmhost/.ssh
chmod 600 /home/llmhost/.ssh/authorized_keys
```

### 3. Harden SSH

```bash
cat >> /etc/ssh/sshd_config << 'EOF'
PermitRootLogin no
PasswordAuthentication no
MaxAuthTries 3
EOF

systemctl restart sshd
```

Verify you can still log in as the new user before closing the root session:

```bash
ssh llmhost@YOUR_SERVER_IP
```

### 4. Configure the firewall

```bash
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP (for TLS challenge)
sudo ufw allow 443/tcp   # HTTPS
sudo ufw enable
```

Do **not** open port 4000. LLMHosts binds to `127.0.0.1:4000` by default and the reverse proxy handles external traffic on 443.

### 5. Install fail2ban

```bash
sudo apt install -y fail2ban
sudo systemctl enable fail2ban
```

## Install Docker

```bash
# Install Docker using the official convenience script
curl -fsSL https://get.docker.com | sh

# Add your user to the docker group
sudo usermod -aG docker llmhost
newgrp docker

# Verify
docker --version
docker compose version
```

### GPU: Install nvidia-container-toolkit

Only needed if your VPS has a GPU:

```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update && sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify
docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.0-base nvidia-smi
```

## Deploy LLMHosts

### 1. Clone the repository

```bash
su - llmhost
git clone https://github.com/LookNoHandsMom/LLMHosts.com.git
cd LLMHosts.com
```

### 2. Configure environment

```bash
cat > .env << 'EOF'
LLMHOST_SECURITY_REQUIREAUTH=true
LLMHOST_SERVER_HOST=0.0.0.0
LLMHOST_SERVER_LOGLEVEL=info
LLMHOST_OBSERVABILITY_LOGFORMAT=json
LLMHOST_CACHE_ENABLED=true
LLMHOST_CACHE_MAXSIZEMB=1024
EOF
```

### 3. Configure the Caddyfile

```bash
cat > Caddyfile << 'EOF'
llmhost.example.com {
    reverse_proxy llmhosts:4000
}
EOF
```

Replace `llmhost.example.com` with your actual domain. Point your domain's DNS A record to the VPS public IP before starting Caddy.

### 4. Start the stack

```bash
# CPU only
docker compose -f docker-compose.prod.yml up -d

# With GPU
docker compose -f docker-compose.prod.yml -f docker-compose.gpu.yml up -d
```

### 5. Verify

```bash
# Check container health
docker compose -f docker-compose.prod.yml ps

# Check the health endpoint
curl -s http://localhost:4000/health | python3 -m json.tool

# Check via public URL (after DNS propagates)
curl -s https://llmhost.example.com/health
```

### 6. Add API keys

Exec into the container to add your cloud API keys:

```bash
docker compose -f docker-compose.prod.yml exec llmhost llmhosts keys add openai sk-proj-...
docker compose -f docker-compose.prod.yml exec llmhost llmhosts keys add anthropic sk-ant-...
docker compose -f docker-compose.prod.yml exec llmhost llmhosts keys validate
```

### 7. Install local models (GPU only)

If running Ollama alongside LLMHosts:

```bash
# Option A: Ollama on the host
sudo apt install -y curl
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2
ollama pull qwen2.5:7b

# Set LLMHosts to find host Ollama
echo 'LLMHOST_OLLAMA_HOST=http://host.docker.internal:11434' >> .env
docker compose -f docker-compose.prod.yml up -d
```

## Domain + TLS Setup

### DNS configuration

Create an A record pointing your domain to the VPS IP:

| Type | Name | Value | TTL |
|------|------|-------|-----|
| A | `llmhosts` | `203.0.113.50` | 300 |

### Automatic TLS

Caddy obtains and renews TLS certificates automatically from Let's Encrypt. No manual certificate management is needed. The only requirements are:

1. DNS is pointing to the server.
2. Ports 80 and 443 are open.
3. The `Caddyfile` has the correct domain name.

Caddy stores certificates in the `caddy_data` Docker volume. They persist across container restarts.

### Verify TLS

```bash
curl -vI https://llmhost.example.com/health 2>&1 | grep -E "(SSL|subject|expire)"
```

## Remote Access via Tunnel

If you do not want to set up a domain and TLS, or if the VPS is behind a NAT, use LLMHosts's built-in tunnel support.

### Tailscale (recommended for private access)

Install Tailscale on the VPS:

```bash
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
```

Your VPS gets a Tailscale IP (e.g., `100.x.y.z`). Any device on your tailnet can reach LLMHosts at `http://100.x.y.z:4000`.

For public access, use Tailscale Funnel:

```bash
# Inside the LLMHosts container or on the host
llmhosts tunnel --provider tailscale --funnel
```

### Cloudflare Tunnel (free, public URL)

Install `cloudflared` on the VPS and start a quick tunnel:

```bash
# On the VPS host
curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloudflare-main.gpg
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared any main' | sudo tee /etc/apt/sources.list.d/cloudflared.list
sudo apt update && sudo apt install cloudflared

# Start the tunnel via LLMHosts
llmhosts tunnel --provider cloudflare
```

This gives you a temporary public `*.trycloudflare.com` URL. For a permanent URL, create a named Cloudflare Tunnel in the Cloudflare dashboard.

## Client Configuration

Once LLMHosts is running and accessible, configure your AI tools to use it:

```bash
# On your laptop / development machine
export OPENAI_BASE_URL=https://llmhost.example.com/v1
export OPENAI_API_KEY=your-llmhost-api-key

# Test it
curl https://llmhost.example.com/v1/models
```

This works with any tool that supports the OpenAI API format: Cursor, Continue, aider, LangChain, OpenAI Python SDK, etc.

## Cost Comparison

Monthly estimates for running LLMHosts on various providers (as of early 2026):

### CPU-Only (proxy + cloud API routing)

| Provider | Spec | Monthly Cost | Best For |
|----------|------|--------------|----------|
| Oracle Cloud | 4 ARM / 24GB | **$0** (free tier) | Testing, personal use |
| Hetzner CX22 | 2 vCPU / 4GB | **$4** | Light workloads |
| Hetzner CX32 | 4 vCPU / 8GB | **$7** | Medium workloads |
| DigitalOcean | 2 vCPU / 4GB | **$24** | US/EU presence |

### With GPU (local inference)

| Provider | GPU | VRAM | Monthly Cost | Best For |
|----------|-----|------|--------------|----------|
| Vast.ai | RTX 3090 | 24GB | **~$144** (spot) | Budget GPU inference |
| RunPod | RTX 4090 | 24GB | **~$288** (spot) | Good performance/price |
| Lambda | A10G | 24GB | **~$360** | Reliable on-demand |
| OVH | T4 | 16GB | **~$80** | Fixed pricing, EU |

### Break-even analysis

If you spend $100/month on OpenAI/Anthropic APIs and LLMHosts routes 60% of requests to local models, your effective savings are:

```
Monthly API spend:        $100
Local offload (60%):     -$60
VPS cost (Hetzner GPU):  +$30 (estimate)
Net savings:              $30/month
```

The savings increase with higher API spend. LLMHosts's cache layer provides additional savings by deduplicating repeated queries regardless of whether they hit local or cloud models.

## Maintenance

### View logs

```bash
docker compose -f docker-compose.prod.yml logs -f llmhost
```

### Update LLMHosts

```bash
cd ~/LLMHosts.com
git pull origin main
docker compose -f docker-compose.prod.yml build llmhost
docker compose -f docker-compose.prod.yml up -d llmhost
```

### System updates

```bash
sudo apt update && sudo apt upgrade -y
# Reboot if kernel was updated
sudo reboot
```

### Backup

Back up the LLMHosts data volume (config, keys, audit DB):

```bash
docker run --rm -v llmhosts_data:/data -v /home/llmhost/backups:/backup alpine \
  tar czf /backup/llmhost-$(date +%Y%m%d).tar.gz /data
```

### Monitor disk usage

```bash
docker system df
df -h
```

LLMHosts's SQLite audit DB and cache grow over time. Set `LLMHOST_CACHE_MAXSIZEMB` to cap cache size. The audit DB can be vacuumed if needed.
