# ADR-008: CLI-First Distribution Strategy

## STATUS
REVISED (2026-02-14) -- course correction adds lightweight install tiers + npx scaffolder

## CHANGE LOG
- 2026-02-14: Original proposal -- `pip install llmhosts` as single monolithic package
- 2026-02-14 (course correction): **REVISED** -- lightweight install tiers to solve the "pip install is 3-5GB" problem. Core is ~50MB. Smart adds ONNX-based ML. Full adds PyTorch. npx scaffolder for JS/TS devs. Anthropic API dialect added to proxy.

## CONTEXT
LLMHosts's original distribution strategy was platform-first: sign up on website, download agent, install Docker. Debate-001 identified critical adoption barriers (Holes 10-13). The pivot moved to `pip install llmhosts` but the course correction identified that a single monolithic pip package risks being 3-5GB with all ML dependencies.

### The Install Size Problem
A single `pip install llmhosts` that includes sentence-transformers, PyTorch, FAISS, and all ML dependencies could be 3-5GB. This kills the "value in under 3 minutes" promise. Solution: tiered install with lightweight default.

## DECISION
**Tiered pip install + Docker + npx scaffolder**

### Install Paths (Priority Order)

#### Path 1: `pip install llmhosts` (Core, ~50MB)
```
pip install llmhosts
llmhosts serve
```
- Proxy, rule-based router, local dashboard, BYOK management
- No ML dependencies (no PyTorch, no sentence-transformers)
- Runs immediately, value in <1 minute
- Python 3.10+ (Linux, macOS, Windows WSL)

#### Path 2: `pip install llmhosts[smart]` (~150MB total)
```
pip install llmhosts[smart]
llmhosts serve
```
- Everything in core PLUS:
- sentence-transformers via ONNX Runtime (NOT full PyTorch)
- kNN embedding router (Tier 1)
- ModernBERT classifier (Tier 2)
- vCache verified semantic caching
- Install time: <3 minutes

#### Path 3: `pip install llmhosts[full]` (~2GB)
```
pip install llmhosts[full]
llmhosts serve
```
- Everything in smart PLUS:
- Full PyTorch + FAISS for maximum routing intelligence
- Qwen-0.5B reasoning router support (Tier 3)
- vLLM integration support
- For power users with GPU

#### Path 4: Docker (pre-built, everything included)
```
docker run -p 4000:4000 llmhost/llmhost
```
- Pre-built image with all dependencies (`[full]` tier)
- No dependency resolution, instant start (after initial pull)
- Volume mounts for config persistence
- GPU passthrough: `--gpus all`

#### Path 5: `npx create-llmhosts` (Phase 2, JS/TS devs)
```
npx create-llmhosts my-ai-cloud
```
- Scaffolder for JavaScript/TypeScript developers
- Generates docker-compose.yml + config files
- Asks questions: "Do you have a GPU? Which cloud providers?"
- Produces ready-to-run setup

#### Path 6: Homebrew/APT/Snap (Phase 2+)
```
brew install llmhost
```
- Native package management per OS
- Phase 2+ once adoption justifies maintenance

### API Dialects
The proxy speaks TWO API formats, both routing through the same Conductor internally:
- **OpenAI format**: `/v1/chat/completions` (90%+ of tools: Cursor, Aider, Continue.dev, Open WebUI, LangChain)
- **Anthropic format**: `/v1/messages` (Claude Code, Anthropic-native tools)

### First-Run Experience
```
$ pip install llmhosts
$ llmhosts serve

  Discovering local AI infrastructure...
   OK  Ollama found (3 models: llama3.2:3b, codellama:7b, mistral:7b)
   OK  GPU detected: NVIDIA RTX 4070 (12GB VRAM)
   --  No cloud API keys (add with: llmhosts keys add openai)

  LLMHosts proxy running on http://localhost:4000
   OpenAI API:    http://localhost:4000/v1/chat/completions
   Anthropic API: http://localhost:4000/v1/messages
   Dashboard:     http://localhost:4000/dashboard

  Quick start:
   export OPENAI_API_BASE=http://localhost:4000/v1
   Now all your AI tools route through LLMHosts!

  Upgrade routing intelligence:
   pip install llmhosts[smart]  (adds ML-powered router + semantic cache)
```

### Cloud-First Hybrid Onboarding
Value starts BEFORE local models are ready:
```
Minute 0:   pip install llmhosts && llmhosts serve
            Proxy starts, discovers Ollama models
            Cloud API fallback available immediately (BYOK)
            Dashboard live

Minute 1:   User sets OPENAI_API_BASE in Cursor/Claude Code
            Requests flow through LLMHosts
            Dashboard: "Routed to local Llama:8b. Cost: $0.00"

Minute 5:   If no local models: "Run: ollama pull llama3.2:8b"
            Model downloads in background
            Cloud API continues serving

Minute 30+: Local model ready
            Router shifts to local
            Dashboard shows cost drop
            The moment: "$0.00... $0.00... $0.00..."
```

## RATIONALE
- Core install is <1 minute (50MB), smart is <3 minutes (150MB)
- ONNX Runtime for ML models avoids the 2GB PyTorch dependency
- Tiered approach: users install what they need, nothing more
- Docker as zero-config alternative for those who prefer containers
- npx scaffolder captures JS/TS ecosystem (different audience than pip)
- Anthropic API dialect ensures Claude Code compatibility alongside OpenAI tools
- Cloud-first hybrid delivers value from Minute 0

## CONSEQUENCES
- Must maintain three pip extras (`[smart]`, `[full]`) with tested dependency sets
- ONNX Runtime adds maintenance for model conversion pipeline
- npx scaffolder is a separate JS package to maintain
- Docker image includes everything (larger initial pull, but one-time)
- Must test all install paths in CI (pip core, pip smart, pip full, Docker, npx)
