# Epic: Compute Fabric (Ollama Inside + Your GPU From Anywhere)

## DESCRIPTION
Distributed compute layer that discovers, wraps, and orchestrates user's existing Ollama installations, cloud API fallback, and SECURE REMOTE ACCESS into unified inference fabric. "Ollama Inside" -- enhance, don't replace. The KILLER feature: `llmhosts tunnel` turns your home GPU into real AI infrastructure accessible from anywhere.

**COURSE CORRECTION (2026-02-14)**: Restored `llmhosts tunnel` as the transformative feature. vLLM elevated to real upgrade path (not afterthought). Networking tiered: localhost → Tailscale/Cloudflare → Headscale for Hive only. See ADR-004 (REVISED).

## BACKLOG ITEMS

### CF-001: Ollama Auto-Discovery
- **Priority**: P0
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: Detect running Ollama instance via API (localhost:11434). List installed models, capabilities, VRAM usage. Zero configuration.
- **Acceptance**: `llmhosts serve` discovers existing Ollama and lists all models within 10 seconds. No user action needed.
- **Dependencies**: None

### CF-002: Hardware Discovery
- **Priority**: P0
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: Detect GPU type/VRAM, CPU cores, RAM, disk. Used for model recommendations and routing intelligence. Also detects installed tunneling tools (Tailscale, Cloudflare Tunnel).
- **Acceptance**: Accurate hardware + networking inventory reported within 5 seconds of `llmhosts serve` start.
- **Dependencies**: None

### CF-003: BYOK (Bring Your Own Key) Management
- **Priority**: P0
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: `llmhosts keys add openai <key>` / `llmhosts keys add anthropic <key>`. Validate keys. Store encrypted locally. Cloud routing via LiteLLM when needed.
- **Acceptance**: User adds API key via CLI → validated → stored encrypted → cloud routing available immediately.
- **Dependencies**: LiteLLM config

### CF-004: LiteLLM Proxy Integration
- **Priority**: P0
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: LiteLLM as unified proxy backend. Routes to local Ollama + cloud APIs through single interface. Model aliasing (user requests "gpt-4" → routed per Conductor decision).
- **Acceptance**: Single API endpoint routes to any backend (local Ollama, cloud API) seamlessly.
- **Dependencies**: CF-001, CF-003

### CF-005: Model Suggestion Engine
- **Priority**: P1
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: `llmhosts suggest-models` -- recommend optimal models based on hardware profile. "You have 12GB VRAM → try: llama3.2:8b, codellama:7b, mistral:7b". User pulls via `ollama pull`.
- **Acceptance**: Suggestions match hardware. User runs ollama pull themselves (LLMHosts doesn't distribute weights).
- **Dependencies**: CF-002

### CF-006: Compute Node Health Monitoring
- **Priority**: P0
- **Phase**: 1
- **Status**: BACKLOG
- **Description**: Heartbeat for all backends (Ollama, cloud APIs, remote tunnel nodes). Detect failures within 3 seconds. Feed into fallback chain.
- **Acceptance**: Unhealthy backend removed from routing pool within 3 seconds. Dashboard shows health status.
- **Dependencies**: CF-001

### CF-016: `llmhosts tunnel` -- Remote Access (THE KILLER FEATURE)
- **Priority**: P0
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: `llmhosts tunnel` auto-detects installed tunneling tools (Tailscale or Cloudflare Tunnel) and creates a secure HTTPS endpoint so the user can access their home GPU from ANYWHERE. No networking knowledge required. Auto-detection flow: check for `tailscale` → use `tailscale funnel 4000`. Check for `cloudflared` → use `cloudflared tunnel`. Neither found → guide user to install (one-liner per OS). `--public` flag creates a shareable URL with rate limiting + API key auth.
- **Acceptance**: User runs `llmhosts tunnel` on home machine → gets HTTPS URL → accesses home GPU from laptop at coffee shop. Works with Cursor, Claude Code, any OpenAI-compatible tool. <30 seconds to working remote access.
- **Dependencies**: CF-001, Tailscale OR Cloudflare Tunnel installed

### CF-017: Multi-Device Mesh
- **Priority**: P0
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: Connect multiple user devices (desktop + laptop + VPS) into a personal mesh. `llmhosts` on laptop discovers `llmhosts` on desktop via Tailscale network. Router sees all devices' models as one pool.
- **Acceptance**: User's laptop routes request to desktop's 4090 transparently. Dashboard shows all devices.
- **Dependencies**: CF-016, Tailscale mesh

### CF-007: LAN Node Discovery (mDNS)
- **Priority**: P1
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: Auto-discover other LLMHosts instances on same LAN via mDNS. Pool their models into routing decisions.
- **Acceptance**: Two LLMHosts instances on same network auto-discover and can route requests to each other.
- **Dependencies**: CF-001

### CF-008: exo LAN Cluster Integration
- **Priority**: P1
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: Integrate exo for distributed inference across LAN GPUs. LAN-ONLY (not WAN -- latency kills it).
- **Acceptance**: 70B model sharded across 2 LAN GPUs runs at acceptable latency (<200ms first-token).
- **Dependencies**: CF-007

### CF-009: vLLM Integration (Real Upgrade Path)
- **Priority**: P1
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: vLLM as a REAL upgrade path for power users, NOT an afterthought. `llmhosts config set engine vllm` enables: continuous batching (higher throughput), prefix caching (massive savings for repetitive prompts), PagedAttention (better VRAM utilization), speculative decoding. Discovers existing vLLM instances or helps configure new ones. Dashboard shows throughput comparison: "vLLM: 45 tok/s vs Ollama: 28 tok/s on your hardware."
- **Acceptance**: Power user opts into vLLM → measurable throughput improvement. Prefix caching active. Clear dashboard comparison showing the value. Migration path from Ollama to vLLM is smooth.
- **Dependencies**: CF-001, `pip install llmhosts[full]`

### CF-010: WAN Node Support (Headscale -- Hive Only)
- **Priority**: P1
- **Phase**: 3
- **Status**: BACKLOG
- **Description**: Headscale coordination for Hive Mode ONLY. Connects compute from DIFFERENT users via WireGuard mesh. Pre-auth keys for auto-join. Whole-model routing (send REQUEST, not tensor shards).
- **Acceptance**: Hive member's GPU accessible via WireGuard mesh. ACL-controlled.
- **Dependencies**: Headscale deployment, Hive Mode

### CF-011: Network Topology Mapping
- **Priority**: P1
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: Map latency and bandwidth between all nodes (local, LAN, tunnel, WAN). Feed into latency-aware routing.
- **Acceptance**: Router knows: local=0ms, LAN=1ms, tunnel=30ms, WAN=80ms and factors this into decisions.
- **Dependencies**: CF-007, CF-016

### CF-012: GPU Memory Management
- **Priority**: P1
- **Phase**: 2
- **Status**: BACKLOG
- **Description**: Track GPU memory allocation per model on each node. Prevent OOM. Support model swapping.
- **Acceptance**: Dashboard shows VRAM usage per model. Router avoids overloaded nodes.
- **Dependencies**: CF-001, CF-002

### CF-013: Hive Mode Compute Routing
- **Priority**: P1
- **Phase**: 3
- **Status**: BACKLOG
- **Description**: Route to Hive member's shared node when user's own nodes are insufficient. Whole-model routing. Privacy routing gates eligibility. Karma system for fairness.
- **Acceptance**: User A (offline GPU) → routes to Hive member B's shared GPU → response returned. Privacy enforced. Karma tracked.
- **Dependencies**: CF-010, Hive Mode, Headscale ACLs

### CF-014: exo GPL 3.0 License Analysis
- **Priority**: P1
- **Phase**: 1
- **Status**: RESEARCH
- **Description**: Legal analysis: does bundling exo (GPL 3.0) in a Docker container trigger copyleft?
- **Acceptance**: Written legal opinion. Architecture adjusted if needed.
- **Dependencies**: None

### CF-015: Hardware Compatibility Matrix
- **Priority**: P1
- **Phase**: 1
- **Status**: RESEARCH
- **Description**: Benchmark matrix: GPU model → supported quantizations → VRAM per model size → throughput → recommended models.
- **Acceptance**: Published matrix used by CF-005 for model recommendations.
- **Dependencies**: CF-002
