# Product Requirements Document -- LLMHosts.com

## META
- version: 1.0
- created: 2026-02-14
- type: PRD (synthesis document, references all L2 artifacts)
- references: lean-canvas.md, competitive-analysis.md, personas-jtbd.md, financial-model.md
- consumed_by: L4 architecture, L5 backlog, L10 implementation blueprint

---

## 1. EXECUTIVE SUMMARY

LLMHosts is an open-source CLI tool and SaaS platform that turns developer-owned GPUs into production AI infrastructure accessible from anywhere. One command (`pip install llmhosts && llmhosts serve`) creates an OpenAI/Anthropic-compatible API proxy on localhost. A second command (`llmhosts tunnel`) makes that GPU accessible from any device over the internet via encrypted tunnel. A three-tier intelligent router automatically selects the best model for each request. A verified semantic cache reduces costs and latency. A real-time dashboard shows honest savings compared to cloud-only spending.

**Target users**: Developers who own GPUs and spend $50-200+/month on AI tools.
**Core value**: "Your 4090 at home is serving your laptop right now. Cost: $0."
**Business model**: Open source core (free, unlimited local use). SaaS from $19/mo for cloud intelligence and multi-device coordination.
**Success criteria**: 15,000 free users and 1,000 paying users by end of Year 1. $19K MRR. NPS >50.

---

## 2. STRATEGIC CONTEXT

### Why Build This NOW (Feb 2026)
- AI inference market: $106B (2025), growing 19% CAGR to $255B by 2030
- Developers spend avg $97/month on AI tools (SlashData Q4 2025, n=800+)
- 34% pay for tools they use <5 times/month -- waste is endemic
- OpenRouter hit $100M GMV with 5M users by stitching APIs together -- validates the proxy/routing market
- Tailscale doubled paying customers in 10 months (5K → 10K) -- validates the "access your stuff from anywhere" market
- No product combines: local inference + intelligent routing + remote tunnel access + verified caching in one tool
- AI coding agents (Cursor, Claude Code, Aider) all consume OpenAI-compatible endpoints -- our proxy slots in with zero config change

### Market Gap
Every developer tool treats local inference and cloud inference as separate worlds. LLMHosts unifies them: one API endpoint, one router, one dashboard, accessible from anywhere. The "holy shit" moment is realizing your home GPU works from a coffee shop.

### Competitive Position
- vs. Ollama: Ollama is the runtime. We WRAP Ollama and add routing, caching, remote access, cost tracking.
- vs. OpenRouter: They route between cloud providers. We route between YOUR hardware and cloud.
- vs. LiteLLM: They're a library for backend developers. We're a product for any developer.
- vs. Tailscale: They solve networking. We solve AI inference routing + networking.
- Our unique combination: local-first AI + intelligent routing + tunnel access + verified caching + honest cost tracking.

---

## 3. TARGET USERS & JOBS

### Persona 1: Solo Developer ("Alex")
- **Profile**: Full-stack dev, owns a gaming PC with RTX 3080/4070/4090, uses Cursor or Claude Code daily
- **Monthly AI spend**: $60-120 (Cursor Pro + Claude API + ChatGPT Plus)
- **JTBD**: "When I'm coding at a coffee shop, I want to use my home GPU for inference, so I can stop paying $0.015/1K tokens to Anthropic for things my GPU handles for free."
- **Switching trigger**: Sees the `llmhosts tunnel` demo, realizes they can cut their AI bill by 40-60%
- **Success criteria**: Working proxy in <5 minutes. Tunnel in <2 minutes. Measurable cost savings in first week.

### Persona 2: AI Engineer ("Jordan")
- **Profile**: ML engineer, has multiple GPUs (workstation + server), runs experiments and serves models
- **Monthly AI spend**: $200-500 (API costs for evaluation, testing, team serving)
- **JTBD**: "When I need to serve 3 different models to my team, I want one unified API that routes intelligently, so I don't have to manage separate endpoints and explain to each person which model to hit."
- **Switching trigger**: Sees the three-tier router picking the right model automatically
- **Success criteria**: Unified /v1/chat/completions endpoint serving multiple models. Cost dashboard showing team usage.

### Persona 3: Tech Lead ("Sam")
- **Profile**: Leads a team of 5-15 devs, manages AI tool budget, needs cost visibility
- **Monthly AI spend**: $500-2000 (team Cursor licenses + API costs)
- **JTBD**: "When I'm justifying AI tool spend to my VP, I want clear data on what we're spending, what we could save by routing some traffic locally, so I can make an evidence-based case."
- **Switching trigger**: Cloud cost dashboard shows exact savings opportunity with existing team hardware
- **Success criteria**: Team-wide dashboard, per-developer cost breakdown, month-over-month trend.

---

## 4. VALUE PROPOSITION & MESSAGING

### UVP
"Your hardware. Real AI infrastructure. From anywhere."

### One-liner
LLMHosts turns your GPU into an AI cloud you can access from any device.

### Elevator Pitch
Developers spend $100+/month on AI tools, but their GPUs sit idle 90% of the time. LLMHosts wraps your existing Ollama installation with an intelligent routing proxy, verified semantic cache, and encrypted tunnel -- giving you a personal AI cloud accessible from anywhere. Open source. One command to install. Save 30-50% on AI costs while keeping your data private.

### Positioning Statement
For developers who own GPUs and use AI coding tools daily, LLMHosts is the open-source AI infrastructure layer that turns local hardware into a remotely-accessible, intelligently-routed AI service. Unlike OpenRouter (cloud-only routing) or raw Ollama (local-only, single model), LLMHosts bridges local and cloud inference with fit-for-purpose routing, verified caching, and zero-config remote access.

---

## 5. PRODUCT SCOPE

### P0 -- Must Ship (Phase 1, Weeks 1-6)

| ID | Feature | Persona | User Story | Acceptance Criteria | Revenue Connection |
|----|---------|---------|-----------|-------------------|-------------------|
| F-001 | OpenAI API Proxy | All | As a developer, I want to point my tools at localhost:4000, so I can use any OpenAI-compatible tool with my local models | 1. POST /v1/chat/completions returns valid response. 2. Streaming works. 3. /v1/models lists all available. 4. Response includes model used and cost metadata. | Enabler: core product hook |
| F-002 | Anthropic API Proxy | All | As a developer, I want to send Anthropic-format requests to the same proxy, so I can use Claude Code and other Anthropic tools | 1. POST /v1/messages returns valid Anthropic response. 2. Streaming works. 3. Same backend routing as OpenAI dialect. | Enabler: doubles compatible tool surface |
| F-003 | Ollama Auto-Discovery | All | As a developer, I want LLMHosts to find my Ollama automatically, so I don't configure anything | 1. Detect Ollama at localhost:11434. 2. List all installed models. 3. Register in routing table. 4. Works if Ollama starts after LLMHosts. | Enabler: zero-config promise |
| F-004 | BYOK Cloud Keys | All | As a developer, I want to add my OpenAI/Anthropic API keys, so the router can fall back to cloud | 1. `llmhosts keys add openai sk-xxx` encrypts and stores. 2. Keys validated on add. 3. Never shown in logs. 4. `llmhosts keys list` shows providers only. | Enabler: hybrid routing |
| F-005 | Rule-Based Router | All | As a developer, I want requests automatically sent to the best available backend, so I get good results without manual model selection | 1. Explicit model → that model. 2. "auto" → best local match. 3. No local match → cloud fallback. 4. Decision logged with reason. | Enabler: intelligence layer |
| F-006 | Cost Tracking | All | As a developer, I want to see what each request costs, so I know my actual AI spend | 1. Local = $0. 2. Cloud = provider pricing. 3. Running total by day/week/month. 4. "Estimated cloud cost" for local requests. | Direct: proves savings → upgrade motivation |
| F-007 | Exact Hash Cache | All | As a developer, I want identical queries to return cached responses instantly, so I save time and money | 1. Same query = same response in <5ms. 2. 0% false positive rate. 3. Hit/miss logged. 4. `llmhosts cache stats` works. | Enabler: cost reduction |
| F-008 | TUI Dashboard | Alex | As a developer, I want a terminal dashboard showing real-time savings and request flow, so I feel the value | 1. Shows savings ticker. 2. Shows recent requests with routing decisions. 3. Shows cache hit rate. 4. Shows backend health. 5. Updates in real-time. | Direct: "wow" moment → word of mouth |
| F-009 | Web Dashboard | All | As a developer, I want a browser dashboard at localhost:4000/dashboard, so I can check stats from any device on my LAN | 1. React SPA served by proxy. 2. Same data as TUI. 3. WebSocket real-time updates. 4. Baseline capture on first visit. | Direct: visual proof of savings |
| F-010 | `llmhosts serve` | All | As a developer, I want one command to start everything, so I can go from install to working in seconds | 1. Starts proxy, discovers Ollama, loads keys, starts dashboard. 2. Prints API endpoint URLs. 3. <5 second startup. 4. Ctrl+C graceful shutdown. | Enabler: first-run experience |
| F-011 | `llmhosts doctor` | All | As a developer, I want to verify my setup, so I know everything is working | 1. Checks Python, Ollama, GPU, keys, proxy, cache, router. 2. Clear pass/fail per component. 3. Actionable fix suggestions. 4. <10 seconds. | Enabler: reduces support burden |
| F-012 | Hardware Detection | All | As a developer, I want LLMHosts to know my GPU specs, so the router makes hardware-aware decisions | 1. Detect GPU name, VRAM, driver. 2. Detect CPU, RAM, disk. 3. Store in profile table. 4. Refresh on serve start. | Enabler: router quality |

### P1 -- Should Ship (Phase 2, Weeks 7-14)

| ID | Feature | Persona | User Story | Acceptance Criteria | Revenue Connection |
|----|---------|---------|-----------|-------------------|-------------------|
| F-013 | `llmhosts tunnel` | Alex | As a developer, I want to access my home GPU from my laptop anywhere, so I stop paying for cloud API calls on the go | 1. Auto-detects Tailscale or Cloudflare Tunnel. 2. Generates HTTPS URL. 3. Works with Cursor/Claude Code by setting OPENAI_API_BASE. 4. Encrypted end-to-end. | Direct: THIS IS THE CONVERSION TRIGGER. Free→Pro at $19/mo. |
| F-014 | kNN Router (Tier 1) | Jordan | As an AI engineer, I want the router to learn from my usage patterns, so it gets better at picking models over time | 1. Embeds queries (ONNX, <15ms). 2. FAISS nearest-K lookup. 3. Handles 75% of routing decisions. 4. 5% exploration rate. 5. Retrainable. | Enabler: intelligence differentiator |
| F-015 | Entity Namespace Cache | All | As a developer, I want "sort in Python" and "sort in JavaScript" to cache separately, so cache hits are relevant | 1. Extracts language/framework entities. 2. Queries isolated by namespace. 3. Cross-namespace never matches. 4. <10ms extraction. | Enabler: cache quality |
| F-016 | vCache Semantic (Tier 2) | All | As a developer, I want semantically similar queries to hit cache, so I save even more, with confidence the answers are right | 1. Adaptive per-embedding thresholds. 2. <1% false positive rate (formally guaranteed). 3. Online logistic regression learns from feedback. 4. `llmhosts cache stats` shows FP rate. | Enabler: verified cost reduction |
| F-017 | Cloud Cost Dashboard | Sam | As a tech lead, I want a hosted dashboard showing my team's AI spending across cloud providers, so I can optimize our budget | 1. Imports OpenAI/Anthropic billing data (or manual entry). 2. Shows: current spend, projected spend, savings with LLMHosts. 3. Works without local install (web-only). | Direct: $19/mo Pro revenue from Phase 1. The dashboard alone justifies the subscription. |
| F-018 | Multi-Device Mesh | Alex | As a developer, I want to register up to 5 devices with LLMHosts Pro, so all my machines route through the best available backend | 1. Devices register via `llmhosts device add`. 2. Routing decisions consider which device has GPU. 3. Dashboard shows per-device usage. | Direct: Pro feature, increases stickiness |
| F-019 | Honest Savings Display | All | As a developer, I want to see REAL savings, not inflated marketing numbers, so I trust LLMHosts | 1. Baseline captured from user. 2. Savings = actual spend - estimated cloud equivalent. 3. Shows routing decision per request. 4. Never shows >100% savings. | Trust: prevents churn from "this doesn't match my bill" |
| F-020 | ModernBERT Router (Tier 2) | Jordan | As an AI engineer, I want the router to analyze complex requests with multi-signal classification, so edge cases route correctly | 1. 6-signal analysis (keywords, embeddings, domain, complexity, PII, preference). 2. <50ms GPU, <100ms CPU. 3. Handles 20% of routing decisions. 4. PII detection → local-only routing. | Enabler: intelligence + privacy |

### P2 -- Could Ship (Phase 3-4)

| ID | Feature | Persona | User Story | Acceptance Criteria | Revenue Connection |
|----|---------|---------|-----------|-------------------|-------------------|
| F-021 | Hive Mode | Alex, Jordan | As a developer, I want to pool GPUs with friends, so we can share capacity | 1. `llmhosts hive join CODE`. 2. Headscale mesh. 3. Karma system (contributed vs consumed). 4. PII stays local. | Direct: Team tier at $49/seat/mo |
| F-022 | Qwen-0.5B Router (Tier 3) | Jordan | As an AI engineer, I want the router to reason about ambiguous requests, so the hardest 5% route correctly | 1. Small LLM reasoning (<500ms). 2. Falls back to CPU. 3. <400MB VRAM. 4. Handles 5% edge cases. | Enabler: peak intelligence |
| F-023 | vLLM Integration | Jordan | As an AI engineer, I want to opt into vLLM for continuous batching and better throughput | 1. `pip install llmhosts[full]` includes vLLM. 2. Auto-detects vLLM alongside Ollama. 3. Routes high-throughput to vLLM. | Enabler: power user retention |
| F-024 | RBAC + Team Management | Sam | As a tech lead, I want to control who can use which models and set spending limits per developer | 1. Roles: admin, developer, viewer. 2. Per-user model allowlists. 3. Spending alerts. 4. SSO (SAML). | Direct: Team/Enterprise feature |
| F-025 | npx Scaffolder | Alex | As a JS/TS developer, I want `npx create-llmhosts` to scaffold a project with LLMHosts pre-configured | 1. Creates config file. 2. Sets OPENAI_API_BASE in .env. 3. Tests connection. | Enabler: expands TAM to JS ecosystem |

---

## 6. NON-FUNCTIONAL REQUIREMENTS

| Category | Requirement | Target | Measurement |
|----------|------------|--------|-------------|
| Latency | Proxy overhead | <20ms added to any request | p95 measured via internal timing |
| Latency | Router Tier 0 (rules) | <5ms | Internal timer |
| Latency | Router Tier 1 (kNN) | <15ms | Internal timer |
| Latency | Router Tier 2 (ModernBERT) | <50ms GPU, <100ms CPU | Internal timer |
| Latency | Cache hit (exact) | <5ms | Internal timer |
| Latency | Cache hit (semantic) | <30ms | Internal timer |
| Accuracy | vCache false positive rate | <1% (formally guaranteed) | Online FP tracking |
| Reliability | Proxy uptime (local) | 99.9% (when machine is on) | Health check monitoring |
| Scalability | Concurrent connections | 100+ simultaneous streams | Load test |
| Security | BYOK key encryption | Fernet at rest, TLS in transit | Crypto audit |
| Security | PII detection | >95% recall on standard PII patterns | Test suite |
| Security | Tunnel encryption | WireGuard (Tailscale) or TLS (Cloudflare) | Network audit |
| Performance | Startup time | <5 seconds | Timer from CLI invoke to ready |
| Performance | Install time (core) | <60 seconds on broadband | Manual test |
| Performance | Install time (smart) | <3 minutes on broadband | Manual test |
| Storage | Database overhead | <100MB for 10,000 requests/day over 30 days | Measured |
| Accessibility | Web dashboard | WCAG 2.1 AA | axe-core audit |
| Compatibility | OS support | Linux (primary), macOS (secondary), Windows WSL (best-effort) | CI matrix |
| Compatibility | Python | 3.10+ | CI matrix |

---

## 7. OUT OF SCOPE (v1)

- Mobile app (web dashboard via tunnel is sufficient)
- Model training or fine-tuning (we are inference-only)
- Self-hosting the cloud dashboard (Phase 2+ only)
- Multi-user authentication on the local proxy (single-user in Phase 1)
- GPU sharing with strangers (Hive Mode is friends-only, Phase 3)
- Windows native support (WSL only)
- Billing/payments infrastructure (Phase 2 with Stripe)
- Self-evolution / automated architecture changes (Phase 4)

---

## 8. SUCCESS METRICS (AARRR)

| Metric | Phase 1 Target | Phase 2 Target | Measurement |
|--------|---------------|---------------|-------------|
| **Acquisition** | 500 installs/month | 5,000 installs/month | PyPI download stats |
| **Activation** | 60% route first request within 10 min | 70% | Telemetry (opt-in) |
| **Revenue** | $950 MRR (50 Pro) | $47K MRR (2K Pro + 200 Team) | Stripe |
| **Retention** | 55% monthly (paid) | 65% monthly (paid) | Subscription data |
| **Referral** | 10% of users share tunnel URL screenshot | 20% | Social media tracking |
| NPS | >30 | >50 | Survey |

---

## 9. RELEASE PLAN

| Phase | Timeline | Ships | Go/No-Go |
|-------|---------|-------|----------|
| Phase 1 | Weeks 1-6 | F-001 through F-012. Local proxy + cache + dashboard. | 500+ installs, 50+ paying users, NPS >30 |
| Phase 2 | Weeks 7-14 | F-013 through F-020. Tunnel + ML router + cloud dashboard. | 3,000+ installs, 500+ paying users |
| Phase 3 | Weeks 15-22 | F-021 through F-024. Hive Mode + Team features. | Team tier revenue, 3+ Hive groups active |
| Phase 4 | Weeks 23-30 | F-025. Polish, scale, npx scaffolder, enterprise. | $100K+ MRR trajectory |

---

## 10. RISKS & MITIGATIONS

| Risk | Probability | Impact | Mitigation |
|------|------------|--------|-----------|
| Ollama adds routing natively | Medium | High | Our moat is tunnel + cache + dashboard + intelligence, not just routing |
| Cloud GPU prices drop 50% | Medium | Medium | "Access from anywhere" + privacy + latency are price-immune value props |
| Low free→paid conversion | High | High | Tunnel is the conversion trigger. Cloud dashboard has standalone value even without local install. |
| Users don't trust semantic cache | Medium | Medium | vCache <1% FP guarantee. Transparent "why this was cached" in dashboard. Opt-out available. |
| Tailscale/Cloudflare break API | Low | High | Abstraction layer. Can swap tunnel provider. Headscale as self-hosted fallback. |
| Solo founder burnout | High | Critical | Bootstrap, don't raise until PMF. Hire first engineer at $5K MRR. Community contributors. |

---

## 11. OPEN QUESTIONS

| # | Question | Assigned To | Phase |
|---|----------|------------|-------|
| 1 | What's the minimum VRAM for the kNN router to be useful? | Research | Phase 2 |
| 2 | Should tunnel URLs be persistent or ephemeral? | Design | Phase 2 |
| 3 | How do we handle Ollama model updates mid-session? | Engineering | Phase 1 |
| 4 | Is there a legal risk in proxying Anthropic API format? | Legal | Phase 1 |
| 5 | Should free tier have a request/day limit? | Business | Phase 2 |
