# Tutorial: Single NVIDIA GPU with Ollama on Ubuntu

Deploy LLMHosts with a single NVIDIA GPU running Ollama for local inference on Ubuntu 22.04/24.04.

**Time**: ~15 minutes | **Prerequisites**: Ubuntu 22.04+, NVIDIA GPU (8GB+ VRAM), root/sudo access

## 1. Install NVIDIA Drivers

```bash
# Check if drivers are already installed
nvidia-smi

# If not installed:
sudo apt update
sudo apt install -y nvidia-driver-535
sudo reboot
```

After reboot, verify: `nvidia-smi` should show your GPU.

## 2. Install Docker

```bash
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
# Log out and back in, then verify:
docker run hello-world
```

## 3. Install NVIDIA Container Toolkit

This lets Docker containers access the GPU.

```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list > /dev/null

sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify GPU in Docker:
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi
```

## 4. Install Ollama

```bash
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model (llama2:7b fits in 8GB VRAM)
ollama pull llama2:7b

# Verify:
ollama list
curl http://localhost:11434/api/tags
```

## 5. Install LLMHosts

```bash
pip install llmhosts

# Verify:
llmhosts --version
llmhosts doctor
```

## 6. Configure

```bash
# First-run wizard
llmhosts setup

# Or manual config:
mkdir -p ~/.config/llmhost
cat > ~/.config/llmhost/config.toml << 'EOF'
[server]
host = "0.0.0.0"
port = 4000
log_level = "info"

[security]
require_auth = true

[cache]
enabled = true
max_size_mb = 1024

[ollama]
host = "http://localhost:11434"
EOF
```

## 7. Start the Server

```bash
llmhosts serve
```

## 8. Verify

In another terminal:

```bash
# Health check
curl http://localhost:4000/health
# Expected: {"status":"ok","version":"0.1.0",...}

# List models
curl http://localhost:4000/v1/models
# Expected: {"data":[{"id":"llama2:7b","owned_by":"ollama"},...]}

# Test completion
curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama2:7b",
    "messages": [{"role": "user", "content": "What is 2+2?"}]
  }'
```

## 9. Run as a Service (Optional)

Create a systemd service so LLMHosts starts on boot:

```bash
sudo tee /etc/systemd/system/llmhost.service << 'EOF'
[Unit]
Description=LLMHosts AI Proxy
After=network.target ollama.service

[Service]
Type=simple
User=your-username
ExecStart=/usr/local/bin/llmhosts serve
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now llmhost
sudo systemctl status llmhost
```

## Troubleshooting

| Issue | Fix |
|-------|-----|
| `nvidia-smi: command not found` | Install drivers: `sudo apt install nvidia-driver-535` |
| Ollama not using GPU | Check `OLLAMA_GPU_LAYERS=-1` and restart Ollama |
| Model too large for VRAM | Use quantized version: `ollama pull llama2:7b-q4_K_M` |
| Port 4000 already in use | Change port: `llmhosts serve --port 8080` |

See the full [Troubleshooting Guide](../guides/troubleshooting.md) for more.

## Next Steps

- [Add remote access with Tailscale](./docker-tailscale.md)
- [Add cloud API fallback](./cpu-only-byok.md)
- [Deploy with Docker Compose](../deployment/docker-production.md)
