# LLMHosts Troubleshooting Guide

Start every troubleshooting session with `llmhosts doctor` — it runs all diagnostic checks automatically and highlights failures.

```bash
llmhosts doctor
```

## Quick Reference

| Symptom | Section |
|---------|---------|
| `502 Bad Gateway` | [#1 Backend unreachable](#1-502-bad-gateway) |
| `401 Unauthorized` | [#2 Authentication failed](#2-401-unauthorized) |
| `429 Too Many Requests` | [#3 Rate limited](#3-429-too-many-requests) |
| `Model not found` | [#4 Model not found](#4-model-not-found) |
| High latency / slow responses | [#5 High latency](#5-high-latency) |
| Connection refused | [#6 Connection refused](#6-connection-refused) |
| TLS / certificate errors | [#7 TLS errors](#7-tls-certificate-errors) |
| Cache not working | [#8 Cache issues](#8-cache-not-working) |
| `pip install` fails | [#9 Installation errors](#9-pip-install-fails) |
| Import errors after install | [#10 Missing optional deps](#10-import-errors-after-install) |
| Config not loading | [#11 Config issues](#11-config-not-loading) |
| GPU not detected | [#12 GPU detection](#12-gpu-not-detected) |
| Tunnel not connecting | [#13 Tunnel issues](#13-tunnel-not-connecting) |
| Dashboard shows no data | [#14 Dashboard blank](#14-dashboard-shows-no-data) |
| Email verification not received | [#15 Email delivery](#15-email-verification-not-received) |
| Streaming cuts off | [#16 Streaming issues](#16-streaming-cuts-off) |
| Docker container won't start | [#17 Docker issues](#17-docker-container-wont-start) |
| Prisma / database errors | [#18 Database errors](#18-prisma-database-errors) |

---

## Installation

### 9. `pip install` fails

**Symptom**: `pip install llmhosts` fails with build errors or dependency conflicts.

**Diagnostic**:
```bash
python3 --version   # Requires 3.10+
pip --version
llmhosts doctor      # If partially installed
```

**Common causes and fixes**:

| Error | Cause | Fix |
|-------|-------|-----|
| `Python 3.9 not supported` | Python too old | Install Python 3.10+ |
| `externally-managed-environment` | System Python on Ubuntu 24+ | Use `pipx install llmhost` or create a venv: `python3 -m venv .venv && source .venv/bin/activate` |
| `No matching distribution` | Wrong Python version or platform | Check `python3 --version` is 3.10+ |
| Build errors with `onnxruntime` | Missing system libs | Use `pip install llmhosts` (core only, no ONNX). Add `[smart]` only if you need the ML router. |

**Install tiers**:
```bash
pip install llmhosts            # Core (~50MB) — CLI, proxy, cache
pip install "llmhosts[smart]"   # + ONNX/FAISS router (~150MB)
pip install "llmhosts[full]"    # + PyTorch for Tier 3 router (~2GB)
```

### 10. Import errors after install

**Symptom**: `ModuleNotFoundError: No module named 'faiss'` or similar on startup.

**Diagnostic**:
```bash
llmhosts doctor
pip show llmhost | grep Requires
```

**Fix**: These are optional dependencies. Either install the right tier or ignore the warning — LLMHosts falls back gracefully:
```bash
# For FAISS/ONNX router features
pip install "llmhosts[smart]"

# For full Tier 3 router
pip install "llmhosts[full]"
```

The core `llmhosts` package works without these. The router falls back to rule-based (Tier 1) when ML dependencies are missing.

---

## Configuration

### 11. Config not loading

**Symptom**: Settings in `config.toml` are ignored, or env vars don't take effect.

**Diagnostic**:
```bash
llmhosts doctor
llmhosts serve --config /path/to/config.toml  # Explicit path
```

**Config priority** (highest wins):
1. CLI flags (`--port 8080`)
2. Environment variables (`LLMHOST_SERVER_PORT=8080`)
3. `--config` flag path
4. `~/.config/llmhost/config.toml`
5. Built-in defaults

**Common causes**:

| Issue | Fix |
|-------|-----|
| Env var wrong prefix | All env vars must start with `LLMHOSTS_` (e.g., `LLMHOSTS_SERVER_PORT`) |
| TOML syntax error | Validate with `python3 -c "import tomllib; tomllib.load(open('config.toml', 'rb'))"` |
| Docker env not passed | Add to `docker-compose.yml` `environment:` section or `.env` file |
| Wrong config path | Check `llmhosts doctor` output for "Config file" location |

### 12. GPU not detected

**Symptom**: `llmhosts doctor` shows no GPU, or Ollama runs in CPU mode.

**Diagnostic**:
```bash
nvidia-smi                    # Should show GPU info
llmhosts doctor                # Check GPU detection section
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi  # Test Docker GPU access
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| `nvidia-smi: command not found` | Install NVIDIA drivers: `sudo apt install nvidia-driver-535` |
| Docker can't see GPU | Install nvidia-container-toolkit: `sudo apt install nvidia-container-toolkit && sudo systemctl restart docker` |
| GPU detected but Ollama uses CPU | Set `OLLAMA_GPU_LAYERS=-1` or check Ollama logs: `docker logs ollama` |
| Wrong GPU selected | Set `CUDA_VISIBLE_DEVICES=0` to select specific GPU |

---

## Connectivity

### 1. 502 Bad Gateway

**Symptom**: Requests to `/v1/chat/completions` return `502 Bad Gateway`.

**Diagnostic**:
```bash
llmhosts doctor
curl http://localhost:4000/health           # Python proxy health
curl http://localhost:11434/api/tags        # Ollama health
docker compose logs llmhost                 # Proxy logs
docker compose logs ollama                  # Backend logs
```

**Decision tree**:
```
502 Bad Gateway
├── Is llmhost running? → Check: curl localhost:4000/health
│   └── No → docker compose up -d llmhost
├── Is Ollama running? → Check: curl localhost:11434/api/tags
│   └── No → docker compose up -d ollama
├── Is Ollama reachable from llmhost? → Check proxy logs for connection errors
│   └── No → Verify LLMHOST_OLLAMA_HOST in .env matches Ollama's address
└── Is the model loaded? → Check: curl localhost:11434/api/tags
    └── Model not listed → ollama pull llama2:7b
```

### 6. Connection refused

**Symptom**: `Connection refused` when hitting LLMHosts from another machine.

**Diagnostic**:
```bash
llmhosts doctor
curl http://localhost:4000/health           # Works locally?
curl http://<server-ip>:4000/health         # Works remotely?
sudo ufw status                             # Firewall rules
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Works locally, not remotely | Open firewall: `sudo ufw allow 443/tcp` (or 4000 for direct) |
| `LLMHOST_SERVER_HOST=127.0.0.1` | Change to `0.0.0.0` in `.env` to bind all interfaces |
| Docker network isolation | Ensure containers are on the same Docker network |
| Traefik not routing | Check `docker compose logs traefik` and verify labels/routes |

### 7. TLS certificate errors

**Symptom**: Browser shows certificate warning, or `curl` fails with TLS error.

**Diagnostic**:
```bash
# Check certificate
openssl s_client -connect llmhosts.com:443 -servername llmhosts.com </dev/null 2>/dev/null | openssl x509 -noout -dates

# Check Traefik logs
docker compose logs traefik | grep -i "acme\|cert\|tls"
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Certificate expired | Restart Traefik: `docker compose restart traefik`. Let's Encrypt auto-renews. |
| Self-signed certificate | Verify `DOMAIN` and `ACME_EMAIL` are set in `.env`. Traefik needs a real domain. |
| DNS not pointing to server | Check `dig llmhosts.com` resolves to your server IP |
| Port 80 blocked | Let's Encrypt HTTP-01 challenge needs port 80 open: `sudo ufw allow 80/tcp` |

### 13. Tunnel not connecting

**Symptom**: `llmhosts tunnel start` fails or tunnel is unreachable.

**Diagnostic**:
```bash
llmhosts doctor                  # Check tunnel status
tailscale status                # If using Tailscale
cloudflared tunnel info         # If using Cloudflare
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| `tailscale: command not found` | Install Tailscale: `curl -fsSL https://tailscale.com/install.sh \| sh` |
| Tailscale not authenticated | Run `tailscale up` and complete auth in browser |
| Cloudflare tunnel token invalid | Re-generate token in Cloudflare dashboard, update `CLOUDFLARE_TUNNEL_TOKEN` in `.env` |
| Both tunnels detected | LLMHosts auto-detects. Set `LLMHOST_TUNNEL_PROVIDER=tailscale` to force one. |

---

## Authentication

### 2. 401 Unauthorized

**Symptom**: API requests return `401 Unauthorized`.

**Diagnostic**:
```bash
llmhosts doctor
# Test with key
curl -H "Authorization: Bearer llmh_yourkey" http://localhost:4000/v1/models
```

**Decision tree**:
```
401 Unauthorized
├── Is auth enabled? → Check LLMHOST_SECURITY_REQUIREAUTH in .env
│   └── Set to false → Requests should work without key (dev only)
├── Is the API key valid? → llmhosts keys list
│   └── Key not listed → llmhosts keys add <provider> <key>
├── Is the header format correct?
│   └── Must be: Authorization: Bearer <key> (not "Token" or "ApiKey")
└── Key works via curl but not from app?
    └── Check your client's base_url points to the correct host
```

### 3. 429 Too Many Requests

**Symptom**: Requests return `429` with `Retry-After` header.

**Diagnostic**:
```bash
llmhosts doctor
# Check rate limit config
grep -i rate config.toml
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Per-key rate limit hit | Increase `rate_limit_rpm` in config or wait for window to reset |
| IP rate limit on auth | Wait 60 seconds, or check if you're sending too many registration/reset requests |
| Legitimate high traffic | Create multiple API keys and distribute requests |

---

## Performance

### 5. High latency

**Symptom**: Responses take >5 seconds for simple queries.

**Diagnostic**:
```bash
llmhosts doctor
# Check response time
time curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama2:7b","messages":[{"role":"user","content":"hi"}]}'

# Check backend health
curl http://localhost:4000/api/backends
```

**Decision tree**:
```
High latency
├── Is it the first request? → Cold start: model loading takes 10-30s. Subsequent requests are faster.
├── Is GPU being used? → Check nvidia-smi during request
│   └── CPU inference → See #12 GPU not detected
├── Is the model too large? → Check GPU VRAM vs model size
│   └── Model exceeds VRAM → Use a smaller quantization (Q4_K_M) or smaller model
├── Is the cache hitting? → Check X-LLMHosts-Cache header or /api/cache/stats
│   └── Cache disabled → Enable: LLMHOST_CACHE_ENABLED=true
└── Network latency? → Is backend remote (cloud API)?
    └── Check /api/topology for backend latency
```

### 8. Cache not working

**Symptom**: Cache hit rate is 0%, or `X-LLMHosts-Cache: MISS` on every request.

**Diagnostic**:
```bash
llmhosts doctor
curl http://localhost:4000/api/cache/stats
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Cache disabled | Set `LLMHOST_CACHE_ENABLED=true` in `.env` |
| Max size too small | Increase `LLMHOST_CACHE_MAXSIZEMB` (default: 512) |
| Streaming requests | Streaming responses are not cached by default |
| Different temperatures | Exact cache requires identical parameters. Use semantic cache for fuzzy matching. |
| System prompts differ | Namespace cache groups by entity. Ensure consistent system prompts. |

### 16. Streaming cuts off

**Symptom**: SSE stream stops mid-response or browser/client disconnects.

**Diagnostic**:
```bash
# Test streaming directly
curl -N http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama2:7b","messages":[{"role":"user","content":"Write a long story"}],"stream":true}'

# Check proxy logs
docker compose logs llmhost | grep -i "disconnect\|timeout\|error"
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Traefik buffering | Add `X-Accel-Buffering: no` header (LLMHosts sets this automatically) |
| Client timeout | Increase client-side timeout. OpenAI SDK: `timeout=300` |
| Nginx proxy buffering | Set `proxy_buffering off;` in nginx config |
| `max_tokens` too low | Increase `max_tokens` in request |

---

## Frontend / Dashboard

### 4. Model not found

**Symptom**: `Model 'xyz' not found` error when making requests.

**Diagnostic**:
```bash
llmhosts doctor
curl http://localhost:4000/v1/models          # List available models
curl http://localhost:11434/api/tags          # Ollama models specifically
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Model not pulled | `ollama pull llama2:7b` (or your desired model) |
| Model name mismatch | Use exact name from `/v1/models` (e.g., `llama2:7b` not `llama2`) |
| Backend not configured | Add backend in config or set `LLMHOST_OLLAMA_HOST` |
| Cloud model without API key | Add provider key: `llmhosts keys add openai sk-...` |

### 14. Dashboard shows no data

**Symptom**: Dashboard at `https://llmhosts.com/dashboard` shows empty charts or loading forever.

**Diagnostic**:
```bash
# Check frontend health
curl http://localhost:3000/api/health

# Check backend metrics API
curl http://localhost:4000/api/metrics

# Check frontend logs
docker compose logs frontend
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| No requests made yet | Make a test request to populate data |
| Frontend can't reach backend | Check Docker network: both containers need `llmhosts-internal` network |
| CORS error in browser | Check browser console. Backend should allow frontend origin. |
| WebSocket not connecting | Check `/api/ws/live` endpoint. May need Traefik WebSocket configuration. |

### 15. Email verification not received

**Symptom**: Sign up succeeds but no verification email arrives.

**Diagnostic**:
```bash
# Check frontend logs for email sending
docker compose logs frontend | grep -i "email\|resend\|verification"

# Verify Resend API key is set
docker compose exec frontend sh -c 'echo RESEND_API_KEY=$RESEND_API_KEY'
```

**Decision tree**:
```
Email not received
├── Is RESEND_API_KEY set? → Check container env
│   └── Not set → Add to .env and restart: docker compose restart frontend
├── Do logs show "✅ Verification email sent"?
│   ├── Yes → Email sent but not delivered:
│   │   ├── Check spam/junk folder
│   │   ├── Verify SPF record: dig TXT yourdomain.com (need v=spf1 include:amazonses.com ~all)
│   │   ├── Verify DKIM: dig TXT resend._domainkey.yourdomain.com
│   │   └── Check Resend dashboard for bounces
│   └── No → Check for error in logs
├── Is the domain verified in Resend? → Check resend.com dashboard
│   └── Not verified → Add DNS records (SPF, DKIM, DMARC) in your DNS provider
└── Do logs show "Resend not configured"?
    └── RESEND_API_KEY not reaching the container → Check docker-compose.yml environment section
```

### 17. Docker container won't start

**Symptom**: `docker compose up` fails or container keeps restarting.

**Diagnostic**:
```bash
docker compose ps                           # Check container status
docker compose logs <service>               # Check logs
docker compose config                       # Validate compose file
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| Port already in use | `lsof -i :4000` to find conflicting process, then stop it |
| Image not built | `docker compose build` before `docker compose up` |
| `.env` missing | Copy from template: `cp .env.example .env` and fill in values |
| Volume permission error | `sudo chown -R 1000:1000 ./data` or check volume mount paths |
| Out of disk space | `docker system prune -a` to clean up old images |

### 18. Prisma / database errors

**Symptom**: Frontend crashes with Prisma errors or "table not found".

**Diagnostic**:
```bash
docker compose logs frontend | grep -i "prisma\|database\|sqlite"
docker compose exec frontend sh -c 'ls -la /data/llmhosts.db'
```

**Fixes**:

| Issue | Fix |
|-------|-----|
| DB file doesn't exist | Run migrations: `docker compose exec frontend npx prisma migrate deploy` |
| Schema out of date | Regenerate client: `npx prisma generate` then rebuild image |
| Permission denied on `/data` | Check volume mount ownership: `docker compose exec frontend ls -la /data/` |
| "prepared statement already exists" | Restart the frontend container: `docker compose restart frontend` |

---

## Still Stuck?

1. Run `llmhosts doctor` and share the full output
2. Collect logs: `docker compose logs > logs.txt 2>&1`
3. Check the [API Reference](./api-reference.md) for correct request format
4. Open an issue at [github.com/LookNoHandsMom/LLMHosts.com](https://github.com/LookNoHandsMom/LLMHosts.com/issues) with:
   - `llmhosts doctor` output
   - Relevant log snippets
   - Steps to reproduce
