# LLMHosts.com — 7-Day Advisory Council Retreat: Executive Summary

**Dates:** 2026-03-10 (7 sessions in one day, compressed sprint format)
**Location:** Virtual — 12 advisors, 1 AGI Arbitrator synthesizer
**Output:** 10 strategic decisions locked, 200+ GitHub issues created, updated masterplan, 15 novel innovations added to roadmap
**Document status:** Definitive board-level record of all retreat decisions. Do not edit retroactively — create new issues or ADRs to supersede specific items.

---

## Table of Contents

1. [Executive Summary](#1-executive-summary)
2. [Top 10 Strategic Decisions](#2-top-10-strategic-decisions)
3. [Launch Blockers (P0 Gate Items)](#3-launch-blockers-p0-gate-items)
4. [Masterplan Wave Roadmap](#4-masterplan-wave-roadmap)
5. [North Star Success Criteria](#5-north-star-success-criteria)
6. [15 Novel Innovations](#6-15-novel-innovations)
7. [Complete Issue Inventory](#7-complete-issue-inventory)
8. [User Journey North Stars (8 Personas)](#8-user-journey-north-stars-8-personas)
9. [Risk Register — Top 10](#9-risk-register-top-10)
10. [Next 14-Day Sprint Priorities](#10-next-14-day-sprint-priorities)

---

## 1. Executive Summary

### Who

**12 AI advisors** simulated by the AGI Arbitrator across 7 domain specializations:

| Advisor | Domain |
|---------|--------|
| Yuki Tanaka | ML Engineering, inference optimization |
| Dr. Hans Bergstrom | GPU Kernels, CUDA/compute |
| Dr. Priya Raghunathan | Privacy, distributed systems |
| Dr. Rafael Kim | Distributed systems architecture |
| Elena Marchetti | Product / UX |
| Marcus Okonkwo | GTM, competitive strategy |
| Dr. Adaeze Obi | Security |
| Dr. Kwame Boateng | AI Ethics |
| Amara Nwosu | DevRel, community |
| Fatima Al-Rashid | Business development |
| Zara Ibrahim | Marketing |
| Dr. Santiago Vega | Legal / IP |

### What

A 7-session strategic audit covering: codebase truth vs. aspirational plan, user journeys in obsessive detail, quantified KPIs, 15 novel technical innovations, go-to-market strategy, competitive intelligence, risk register, and an updated wave-by-wave masterplan.

**Key finding:** LLMHosts is at approximately 65% completion of its Vision 2026 masterplan. The choices made in F1 are architecturally sound. No decisions need to be reversed. The gaps are incremental, not structural — but two of them are existential blockers that must be resolved before any public launch.

### When

March 10, 2026. The project is in active development on branch `fix/485-concurrent-inference-rwlock`. The public repository is not yet live. No external users have been marketed to.

### Overall System Completion (Honest Audit)

| Subsystem | Completion | Critical Gap |
|-----------|------------|--------------|
| LLMHosts Core (F1) | 62% | No concurrent inference — Mutex blocks all |
| relay_core | 20% | No encryption, no yamux, no VPS server |
| Python Proxy | 82% | Core streaming integration partial |
| Three-Tier Router | 72% | ML models not deployed (ONNX distribution) |
| Semantic Cache | 68% | No invalidation, no cross-node federation |
| Frontend Dashboard | 79% | No real-time GPU metrics, no model management UI |
| GPU Marketplace | 76% | All nodes still on Ollama, not Core |
| Auth / Security | 71% | No SOC 2, no SSO, no zero-trust relay auth |
| Test Coverage | 61% | No Rust/GPU CI tests, no concurrent load tests |
| Documentation | 58% | No Core docs, no benchmarks published |
| Packaging | 64% | No Windows native, no CUDA wheel variants |
| OS Compatibility | 52% | AMD ROCm unvalidated, Apple Metal 50-65% of Ollama |

**Weighted system completion: 65%**

---

## 2. Top 10 Strategic Decisions

These are the council's locked-in decisions. They are not suggestions. Any deviation requires a new ADR and explicit reversal of this record.

### Decision 1: Concurrent Inference is the Wave 2 Gate — Not a Feature

**Context:** The current `Mutex<Option<LLMHostsModel>>` in `llmhosts_core/src/lib.rs:24` serializes ALL inference requests. Request 2 waits for request 1 to complete. At 40 tokens/second for a 200-token response, this creates 5-second serialization gaps. Ollama (our primary competitor) handles basic concurrency. Under real multi-tool usage (Cursor + chat app simultaneously), LLMHosts's performance claim inverts.

**Decision:** `Arc<RwLock<LLMHostsWeights>> + per-request InferenceSession` is Wave 2 Day 1, before PagedAttention, before speculative decoding, before any other Wave 2 item. No performance marketing until this is resolved. Issue #485 is the current work item on this branch.

### Decision 2: Relay M2 (Noise NK Encryption) Gates All Public Launches

**Context:** The relay currently transmits requests over an unencrypted WebSocket. Every API key, every prompt, every response is visible on the wire between the device and the relay server. This is not a theoretical risk — it is a categorical trust violation.

**Decision:** Show HN, public GitHub repository, any user marketing — all gated on Relay M2 (Noise NK) shipping and passing end-to-end Wireshark verification. Issue #440. No exceptions.

### Decision 3: Apache 2.0 for All Inference Infrastructure

**Context:** BSL (Business Source License) would suppress community growth and enterprise evaluation. The council deliberated and reached consensus.

**Decision:**
- `llmhosts_core` (inference engine): **Apache 2.0**
- `relay_core` (tunnel protocol): **Apache 2.0**
- `router_core` (routing architecture): **Apache 2.0**
- Router model weights (fine-tuned Qwen-0.5B): **Proprietary** (the learning moat)
- HIM hardware-performance dataset: **Proprietary** (competitive asset)
- Platform backend + SaaS frontend: **Proprietary**

This decision is locked. Engineering can proceed with Apache 2.0 headers in all relevant crates immediately.

### Decision 4: Honest Apple Silicon Performance Claims

**Context:** The wgpu/Metal backend delivers approximately 50-65% of Ollama's native MPS (Metal Performance Shaders) performance. The masterplan used language implying parity.

**Decision:** All documentation, benchmarks, marketing, and the Show HN post must state clearly: "NVIDIA is first-class. Apple Metal via wgpu performs at approximately 55% of native MPS performance." Native MPS integration is targeted for Wave 4. This is not a retreat from the product — it is the honesty that builds r/LocalLLaMA trust.

### Decision 5: HIM Cold Start — Honest Marketing Required

**Context:** The Hardware Intelligence Matrix (HIM) requires a minimum of 500 opt-in nodes with benchmark data before it outperforms a static lookup table. The masterplan implied HIM is useful from day one.

**Decision:** HIM marketing language is: "Fleet-powered recommendations — available as our network grows." The dashboard shows static benchmarks (our own test data, 10 hardware profiles) until k≥500 nodes per hardware category are contributing. After k≥500, the Neural HIM model activates. Do not claim fleet intelligence before the fleet exists.

### Decision 6: Pro Pricing Revised to $12/Month

**Context:** At $19/month Pro, users who save only $30-50/month on cloud APIs see inadequate ROI. The council modeled: $12/month standard, $100/year annual = $8.33/month effective.

**Decision:** Pro tier price is $12/month ($100/year). This captures the long tail of moderate cloud spenders who would churn at $19. Revisit at $1M ARR.

### Decision 7: Show HN Launch Sequencing

**Context:** Timing the Show HN launch requires all P0 gates to be green, plus concurrent inference working.

**Decision:** Show HN launch is gated on: (1) Relay M2 complete + Wireshark-verified, (2) concurrent inference working (4 simultaneous requests all completing correctly), (3) TTFT <150ms on RTX 4090 7B model, (4) `llmhosts up` passing E2E tests on Windows + Mac, (5) all CI green. No other conditions.

### Decision 8: The Business is the Network, Not the Software

**Context:** Every product decision must be evaluated against the question "Does this grow the fleet?" The fleet creates collective intelligence. Collective intelligence creates HIM. HIM creates lock-in. Lock-in creates enterprise value. Competitors can copy the software (and will). They cannot copy 10,000 nodes of fleet intelligence.

**Decision:** Feature priority framework — if a feature grows the fleet or deepens fleet intelligence, it takes precedence over features that improve individual user experience at the margin. This applies to: HIM, Global Semantic Deduplication, Operator cold-start UX, Community viral loops.

### Decision 9: 15 Novel Innovations Added to Roadmap

**Context:** The original masterplan did not include innovations discovered during Day 4 deliberations.

**Decision:** All 15 innovations (detailed in Section 6) are formally added to the LLMHosts roadmap across Waves 2-6. They are not aspirational "nice to haves" — they are scheduled work items with assigned waves, moat scores, and implementation estimates. See Section 6 for full details.

### Decision 10: Operator Community is the Primary Retention Focus

**Context:** Marketplace operators are simultaneously the most vulnerable (cold start churn risk) and the most valuable (passive income hooks, viral earnings posts) user cohort. Early operator churn stalls GMV regardless of buyer demand growth.

**Decision:** Wave 3 dedicates specific product capacity to operator retention: intermediate signals (first inquiry, first booking, first payout as milestones), earnings projection (30-day forecast), capacity planning notifications (peak hour alerts), and a dedicated #operator-lounge Discord channel with <4-hour response SLA from the team.

---

## 3. Launch Blockers (P0 Gate Items)

These are the binary gates that block any public launch. If any is red, we do not ship, regardless of schedule pressure.

### P0-1: Relay M2 — Noise NK Encryption (Issue #440)

**Blocker:** Every byte passing through the relay is currently unencrypted. API keys, prompts, responses all visible on the wire.

**What must be done:**
- Implement Noise NK protocol in `relay_core`
- Server has static long-term keypair distributed at build time
- Handshake pattern: NK (server static key, anonymous client)
- Prologue binding to prevent relay substitution attacks
- Verification: Wireshark capture shows zero plaintext API content

**Acceptance criteria:** Zero unencrypted content on the wire, verified by Wireshark trace. Relay M2 integration test passes.

**Owner:** Rust relay engineer | **Wave:** 1 | **ETA:** Week 3-4

### P0-2: Concurrent Inference — Arc<RwLock<>> Refactor (Issue #485)

**Blocker:** All inference requests serialize behind `Mutex<Option<LLMHostsModel>>`. Multi-user and multi-tool use cases are broken. Performance claims are false under real load.

**What must be done:**
- Replace `Mutex<Option<LLMHostsModel>>` with `Arc<RwLock<LLMHostsWeights>>`
- Create per-request `InferenceSession` struct (owns the KV cache for one request)
- Weights are read-locked (shared) during inference; no write locks during generation
- Request-level timeout: kill any generation running >60 seconds
- MIRI + TSAN validation of the new concurrency model

**Acceptance criteria:** 4 simultaneous requests all completing correctly and concurrently. No serialization observable in timing. MIRI clean.

**Owner:** Rust core engineer | **Wave:** 2 Day 1 | **ETA:** Week 7

### P0-3: Apache 2.0 License Headers in All Open-Source Crates

**Blocker:** Publishing the repository without clear licensing creates legal risk for contributors and enterprise evaluators.

**What must be done:**
- Add Apache 2.0 SPDX header to every `.rs` file in `llmhosts_core/`, `relay_core/`, `router_core/`
- Update `Cargo.toml` `license` fields
- Add `LICENSE-APACHE` file at repo root

**Acceptance criteria:** `cargo deny check licenses` passes. GitHub "License" badge shows Apache 2.0.

**Owner:** Any engineer | **Wave:** 1 | **ETA:** Week 1

### P0-4: Update All Marketing for Honest Platform Claims

**Blocker:** Any marketing that claims encrypted relay (before M2) or parity Mac performance (before MPS native) creates trust debt.

**What must be done:**
- Audit and update: README, docs/quickstart, landing page copy, Show HN draft
- Add explicit platform capability table: NVIDIA (first-class), Apple (55% of Ollama Metal, improving), AMD (experimental)
- Remove "Noise NK encrypted" from relay description until M2 ships

**Acceptance criteria:** No marketing claim is falsified by the current implementation. One engineer + one non-engineer review.

**Owner:** Product | **Wave:** 1 | **ETA:** Week 2

### P0-5: E2E Passing on Windows + macOS (llmhosts up)

**Blocker:** Cannot launch without demonstrating the product works on the two dominant desktop platforms.

**What must be done:**
- `llmhosts up` E2E test on Windows 11 + RTX 4090 (physical hardware or GPU passthrough CI)
- `llmhosts up` E2E test on macOS M-series (Apple Silicon)
- `llmhosts doctor` returns all-green on first run (>80% success rate target)

**Acceptance criteria:** Both platform E2E tests pass in CI (or documented hardware test log). `llmhosts doctor` all-green rate >80%.

**Owner:** QA | **Wave:** 1 | **ETA:** Week 5-6

---

## 4. Masterplan Wave Roadmap

All time estimates are from March 10, 2026 (retreat date). ETAs are targets, not commitments.

### Wave 1: Hardened Foundation
**Duration:** Weeks 1-6 | **ETA:** April 2026

**Goal:** Ship what exists, cleanly. Fix the P0 critical gaps. Prepare for Show HN.

**Engineering priorities:**
1. Relay M2: Noise NK encryption (issue #440) — BLOCKS ALL LAUNCHES
2. Apache 2.0 license headers in all open-source crates
3. Add jitter to relay reconnect backoff (issue #486) — thundering herd prevention
4. ModernBERT ONNX distribution strategy — lazy download on `[smart]` install
5. `llmhosts doctor` enhanced: binary version check, CUDA version check, relay status
6. Chunked model download with resume capability
7. SHA-256 checksum verification for all model downloads
8. HuggingFace tokenizer download hardened against supply chain attacks
9. Windows service auto-install (systemd / launchd / Windows Service equivalent)
10. maturin cross-compilation matrix: linux/x86_64, linux/aarch64, darwin/arm64, darwin/x86_64

**Product priorities:**
1. Savings dashboard with monthly forward projection ("at your rate, $89/month")
2. One-click Cursor/VS Code configuration link from dashboard
3. "Share your setup" sharecard generation (GPU model, tokens/sec, savings)
4. Beta program: 50 users from r/LocalLLaMA

**Success gate:** `llmhosts up` E2E passes on Windows + Mac. Relay M1 end-to-end forwards requests. All CI green. Apache 2.0 headers committed.

---

### Wave 2: Concurrent Inference + Performance
**Duration:** Weeks 6-14 | **ETA:** June 2026

**Goal:** 2× Ollama performance on the same hardware. Fully concurrent. PagedAttention live. Show HN launch.

**Engineering priorities:**
1. `Arc<RwLock<LLMHostsWeights>>` + per-request `InferenceSession` (issue #485) — CONCURRENT INFERENCE
2. PyO3 ABI: build wheels for Python 3.10, 3.11, 3.12, 3.13 (maturin matrix)
3. PagedAttention: wire `kv_cache.rs` block table to continuous batch scheduler
4. Continuous batch scheduler: request queue, iteration-level scheduling, `max_batch_size` config
5. Flash Attention v2 CUDA kernel (Ampere, Ada Lovelace — RTX 30xx/40xx)
6. Adaptive quantization selection (Innovation 6)
7. Model-Mesh routing policy engine (Innovation 10)
8. 24-hour soak test: memory leak detection, VRAM drift, relay reliability
9. MIRI + TSAN in CI for concurrency validation
10. Relay M3: yamux stream multiplexing (issue #441)
11. Automated per-commit benchmark regression vs Ollama baseline (<5% degradation gate)
12. Speculative decoding: EAGLE-2 draft model integration

**Product priorities:**
1. Live GPU utilization graph in dashboard (tokens/sec, VRAM %, queue depth)
2. Real-time TTFT counter per request
3. Show HN launch (gated on concurrent inference + relay M2)
4. GitHub repository goes public (Apache 2.0)
5. Operator first-booking notification system (celebratory, not transactional)

**Success gate:** 4 concurrent requests all completing correctly. TTFT <150ms on RTX 4090 7B. Throughput ≥150 tok/s. Relay M2+M3 end-to-end encrypted and multiplexed. Benchmark regression CI gate green.

---

### Wave 3: Intelligence + Privacy
**Duration:** Weeks 14-22 | **ETA:** August 2026

**Goal:** Collective intelligence live, privacy-preserving, HIM useful, operator retention solved.

**Engineering priorities:**
1. LDP perturbation of embeddings before upload (ε=0.5, sensitivity documented)
2. k-anonymity k≥500 enforcement in HIM aggregator
3. HIM aggregator security design: Sybil resistance, authenticated data submission
4. HIM static seeding: our own benchmarks across 10 hardware profiles × 10 models
5. Neural HIM training pipeline (Innovation 8) — activates after k≥500 nodes
6. Global semantic deduplication (Innovation 4)
7. Predictive KV cache warming (Innovation 2)
8. Differential Cache Privacy (Innovation 11)
9. Relay M4: relay_server VPS binary + deployment documentation (issue #442)
10. GDPR Data Protection Impact Assessment (DPIA) for collective intelligence

**Product priorities:**
1. HIM dashboard: fleet recommendations with confidence scores and node counts
2. Collective intelligence opt-in flow with explicit GDPR-compliant consent language
3. Operator earnings projection: 30-day forecast with confidence interval
4. Enterprise RBAC + SSO (SAML/OIDC) — critical for enterprise procurement
5. Product Hunt launch

**Success gate:** 500+ nodes opt-in to collective intelligence. HIM returns recommendations with k≥100 per hardware category. GDPR DPIA complete. Operator 30-day retention improves from baseline.

---

### Wave 4: Scale + Distribution
**Duration:** Weeks 22-32 | **ETA:** October 2026

**Goal:** Fleet at scale, relay hardened, distributed inference available, enterprise-ready.

**Engineering priorities:**
1. Relay M5: production hardening, monitoring, runbook documentation (issue #443)
2. Prefill/Decode Disaggregation (Innovation 1) — fleet-distributed inference
3. Cross-Device Tensor Parallelism over LAN (Innovation 3)
4. io_uring zero-copy relay (VPS-side only) (Innovation 9)
5. Consensus Model Validation (Innovation 13) — enterprise triple-verify
6. Latent Prompt Compression (Innovation 14) — relay bandwidth reduction
7. Intel Arc GPU detection (Level Zero API)
8. Apple MPS native integration (replace wgpu Metal path) — close the 45% perf gap
9. AMD ROCm: first-class support, CI validation
10. Abuse detection: prompt injection, jailbreak detection at relay layer

**Product priorities:**
1. Fleet dashboard v2: multi-device VRAM aggregate, cross-node queue depth
2. Enterprise fleet analytics: cost/savings/utilization per device
3. Marketplace verified-node tier: higher price, guaranteed 99.2% SLA
4. Payout analytics dashboard for operators (charts, history, tax export)

**Success gate:** 2-node tensor parallel inference working on LAN. Apple Metal within 10% of Ollama Metal. AMD ROCm CI tests passing. Enterprise SSO signed off by first enterprise customer.

---

### Wave 5: Fine-Tuning + LoRA
**Duration:** Weeks 32-44 | **ETA:** January 2027

**Goal:** First-class fine-tuning pipeline. Custom model marketplace. LoRA hot-swap.

**Engineering priorities:**
1. LoRA adapter GGUF loading in `llmhosts_core` (GAP-08) — prerequisite for everything in W5
2. QLoRA training orchestration (`llmhosts finetune train`)
3. Training job monitoring: live loss curve in dashboard, checkpoint every 30 minutes
4. LoRA adapter GGUF export from training artifacts
5. Streaming LoRA Hot-swap (Innovation 12) — zero-downtime adapter updates
6. Dynamic fine-tuning cost estimator: estimate from first 100 training steps
7. Evaluation pipeline: MMLU subset, domain-specific benchmark
8. LoRA adapter marketplace: share, purchase, deploy adapters
9. Inference-Time Constitutional Alignment (Innovation 5) — enterprise alignment
10. FedAttn (Innovation 7) — prefill-only, long-context collaborative inference

**Product priorities:**
1. Fine-tuning wizard: "Give me a folder of documents, I'll handle the rest"
2. Custom model deployment to operator's marketplace listing
3. Adapter marketplace: browse, purchase, deploy adapters
4. Live training dashboard with loss curve, ETA, checkpoint management

**Success gate:** E2E fine-tuning on RTX 4090: data prep → QLoRA training → LoRA adapter → deployed and serving requests. Training time estimate within 20% of actual. Adapter hot-swap zero-downtime verified.

---

### Wave 6: Marketplace Maturity + Ecosystem
**Duration:** Weeks 44-56 | **ETA:** April 2027

**Goal:** Marketplace at $50K+ GMV/month. Self-sustaining operator community. Ecosystem SDK.

**Engineering priorities:**
1. Autonomous Capacity Planning (Innovation 15) — predict demand, alert operators
2. Dispute resolution system: buyer SLA breach claim → automated investigation
3. Buyer SLA guarantee enforcement: automatic credit on SLA failure
4. Marketplace search, filtering, and model compatibility matrix
5. Price recommendation engine: "similar GPUs earn $X-Y at $Z/M tokens"
6. Multi-region relay routing: connect buyer to geographically nearest operator
7. SOC 2 Type II readiness: formal control audit, evidence collection
8. LLMHosts Plugin SDK: community extensions to the proxy layer

**Product priorities:**
1. Operator ambassador program: top operators get co-marketing, revenue share referrals
2. "Powered by LLMHosts" badge with `X-Powered-By` response header
3. Community showcase: user projects built on LLMHosts API
4. Annual "State of Local AI Inference" benchmark report
5. LLMHosts Python client library (typed async client)
6. LLMHosts JavaScript/TypeScript SDK (npm package)

**Success gate:** Marketplace GMV ≥$50K/month. Operator churn <8%/month. SOC 2 audit initiated. Community SDK published to PyPI and npm.

---

## 5. North Star Success Criteria

The council's consensus on the KPIs that define success. Organized by category.

### 5.1 Performance KPIs

| Metric | Target | Threshold | Current Baseline | Measurement |
|--------|--------|-----------|-----------------|-------------|
| TTFT — RTX 4090, 7B Q5_K_M, local | <80ms | <150ms | ~290ms | Per-commit benchmark |
| TTFT — RTX 4090, 70B Q4_K_M, local | <200ms | <400ms | ~600ms | Per-commit benchmark |
| TTFT — via relay (M2+) | <230ms | <500ms | N/A (M2 not deployed) | Relay E2E test |
| TTFT — marketplace node | <350ms | <700ms | N/A | Marketplace test |
| Throughput — RTX 4090, 7B Q5_K_M | >180 tok/s | >120 tok/s | ~100 tok/s (Ollama) | llmhosts_bench |
| Throughput — RTX 4090, 70B Q4_K_M | >50 tok/s | >30 tok/s | ~25 tok/s (Ollama) | llmhosts_bench |
| Concurrent throughput (4 req, 70B) | >120 tok/s total | >80 tok/s | 0 (serialized) | Concurrent bench |
| Relay added latency overhead | <50ms | <150ms | N/A | Relay latency test |
| Exact cache TTFT | <1ms | <5ms | ~0.2ms | Unit test p99 |
| Semantic cache TTFT | <30ms | <80ms | ~50ms | Integration test |
| Semantic cache hit rate | >40% | >20% | ~25% (estimated) | Production telemetry |
| Router Tier 1 kNN latency | <15ms | <30ms | Not benchmarked | Automated test |
| Router accuracy vs. oracle | >90% | >80% | Not measured | Labeled test set |
| Relay uptime | >99.5% | >99% | N/A | Synthetic monitoring |
| Request error rate | <0.5% | <1% | Unknown | Prometheus |
| Memory leak over 24h | 0 MB | <500MB drift | Unknown | Soak test |

### 5.2 Growth KPIs

| Metric | Q2 2026 | Q3 2026 | Q4 2026 | Q2 2027 |
|--------|---------|---------|---------|---------|
| Total registered users | 1,000 | 5,000 | 25,000 | 100,000 |
| Monthly Active Users | 400 | 2,000 | 12,000 | 50,000 |
| GitHub stars | 500 | 2,000 | 8,000 | 25,000 |
| Pro subscribers ($12/mo) | 100 | 700 | 3,500 | 12,000 |
| Active marketplace operators | 0 (beta) | 100 | 500 | 2,000 |
| Marketplace GMV/month | $0 | $5,000 | $50,000 | $400,000 |
| MRR (all tiers) | $2,000 | $15,000 | $75,000 | $300,000 |
| Enterprise accounts | 0 | 2 | 10 | 30 |
| Discord members | 200 | 1,000 | 5,000 | 20,000 |

### 5.3 Reliability KPIs

| Metric | Target | Threshold | Measurement |
|--------|--------|-----------|-------------|
| Install-to-first-inference time (7B, pre-downloaded) | <5 min | <10 min | E2E timer |
| Install-to-first-inference time (7B, first download) | <12 min | <20 min | E2E timer |
| `llmhosts up` zero-config success rate (NVIDIA+Win) | >95% | >85% | Hardware E2E |
| `llmhosts up` zero-config success rate (Mac M-series) | >95% | >85% | Hardware E2E |
| `llmhosts doctor` all-green rate on first run | >80% | >65% | Hardware E2E |
| 7-day retention (any user) | >60% | >45% | Cohort analysis |
| 30-day retention (single-device) | >45% | >30% | Cohort analysis |
| Marketplace operator churn/month | <12% | <20% | Cohort analysis |
| NPS (onboarding cohort, 30 days) | >60 | >40 | In-app survey |
| NPS (marketplace operators) | >65 | >45 | Monthly survey |
| NPS (enterprise) | >70 | >50 | Quarterly survey |
| CI build time (cached) | <3 min | <6 min | GitHub Actions |
| Benchmark regression gate | <5% degradation | <10% | Per-commit |

### 5.4 The 10 Most Critical KPIs (Council Consensus)

1. **Install-to-first-inference: <5 minutes** — This is the product promise. Failure here is existential.
2. **TTFT <150ms local (7B, RTX 4090)** — The performance claim must be real and measurable.
3. **7-day retention >60%** — If users try and leave, it is a product problem, not a marketing problem.
4. **Relay M2 uptime >99.5%** — The remote access value prop dies if relay is unreliable.
5. **Concurrent inference: 4 requests simultaneously** — Required before any multi-user claim.
6. **Marketplace operator churn <12%/month** — Above this, GMV growth stalls regardless of acquisition.
7. **Router accuracy >90%** — Wrong routing decisions waste money and erode trust.
8. **Cache hit rate >40% on repeat queries** — This is the savings amplifier for the fleet.
9. **GitHub stars: 1,000** — Community signal that the product has found its audience.
10. **NPS >60 from onboarding cohort** — Users who love it immediately become the marketing team.

---

## 6. 15 Novel Innovations

These innovations were identified during Day 4 and formally added to the roadmap. No competitor has shipped any of these. Implementation time estimates are from discovery date (March 2026).

| # | Innovation | Moat Score | Wave | Impl. Time | Status |
|---|-----------|-----------|------|-----------|--------|
| 1 | Prefill/Decode Disaggregation | 9/10 | W4 | 8-12 weeks | Designed |
| 2 | Predictive KV Cache Warming | 8/10 | W3 | 3-4 weeks | Designed |
| 3 | Cross-Device Tensor Parallelism | 8/10 | W4 | 10-14 weeks | Designed |
| 4 | Global Semantic Deduplication | 9/10 | W3 | 5-6 weeks | Designed |
| 5 | Inference-Time Constitutional Alignment | 7/10 | W5 | 6-8 weeks | Designed |
| 6 | Adaptive Quantization Selection | 7/10 | W2 | 3-4 weeks | Designed |
| 7 | FedAttn — Federated Attention Heads | 8/10 | W6 | 12-16 weeks | Designed |
| 8 | Neural HIM — Hardware Intelligence Neural Net | 9/10 | W3 | 6-8 weeks | Designed |
| 9 | Zero-Copy Relay (io_uring) | 6/10 | W4 | 4-6 weeks | Designed |
| 10 | Model-Mesh Routing Policy Engine | 8/10 | W2 | 4-5 weeks | Designed |
| 11 | Differential Cache Privacy (DCP) | 7/10 | W3 | 2-3 weeks | Designed |
| 12 | Streaming LoRA Adapter Hot-Swap | 8/10 | W5 | 5-6 weeks | Designed |
| 13 | Consensus Model Validation (CMV) | 7/10 | W4 | 4-5 weeks | Designed |
| 14 | Latent Prompt Compression | 6/10 | W4 | 3-4 weeks | Designed |
| 15 | Autonomous Capacity Planning | 8/10 | W6 | 6-8 weeks | Designed |

### Innovation Highlights

**Innovation 1: Prefill/Decode Disaggregation (Moat: 9/10)**
Run the prefill phase (compute-heavy, FLOP-bound) on one machine and decode phase (memory-bandwidth-bound) on another. Reduces TTFT for long prompts (8K tokens) from ~2,000ms to ~300ms. LLMHosts enables this over consumer hardware across the internet — no competitor offers this outside homogeneous datacenter clusters. Required: KV cache serialization in Rust, binary relay protocol for KV streaming, dispatcher prefill/decode routing logic.

**Innovation 4: Global Semantic Deduplication (Moat: 9/10)**
Deduplicate semantically equivalent prompts across ALL users anonymously using LDP-protected embeddings. If 1,000 users ask questions within 0.93 cosine similarity, serve from one cached response. As the fleet grows to 10,000 nodes, the cache hit rate compounds. This is a self-reinforcing moat no single-node tool can replicate. Privacy: only noisy embeddings (Laplace noise) are uploaded; no prompt content leaves the device.

**Innovation 8: Neural HIM (Moat: 9/10)**
Train a small MLP on collective hardware benchmark data to predict optimal model + quantization + batch size for any hardware configuration. Gets better with every new user. The only tool that tells you exactly what performance to expect, with empirical confidence from real fleet data. Input features: GPU model, VRAM, memory bandwidth, CPU RAM. Output: model_id, quantization, predicted TTFT, predicted throughput.

**Innovation 10: Model-Mesh Routing (Moat: 8/10)**
Route requests based on latency, cost, quality, and privacy tier simultaneously — like Istio service mesh but for LLM models. Probabilistic routing policy engine with real-time weight updates. The `GET /v1/routing/explain` endpoint becomes a selling point for enterprise buyers who need to audit routing decisions. User-configurable presets: optimize_cost, optimize_latency, optimize_quality, balanced.

**Innovation 2: Predictive KV Cache Warming (Moat: 8/10)**
After generating response N, background-predict the 3 most likely follow-up intents and pre-compute KV cache for probable continuations. When the actual next message arrives, TTFT drops from 200ms to <20ms. Makes multi-turn conversations feel instantaneous. Uses Qwen-0.5B as the intent predictor (already in the router stack). TTL: 60 seconds per pre-warmed speculative prompt.

**Innovation 12: Streaming LoRA Hot-Swap (Moat: 8/10)**
Hot-swap fine-tuned LoRA adapters without dropping any active connections. `llmhosts adapter push --name legal-v2 --file ./legal-v2.gguf` loads in background, active SSE streams complete with old adapter, new SSE streams use new adapter, old adapter unloads and VRAM freed. Zero downtime adapter deployment is a key enterprise feature for continuous fine-tuning workflows.

---

## 7. Complete Issue Inventory

### Issues #504 — #589 (85 issues, created during retreat)

All issues carry the `advisory-retreat` label. The following is a categorized inventory.

#### Wave 1 — Foundation & Security (Sample)

| Issue | Title |
|-------|-------|
| #504 | feat(relay): Noise NK encryption — M2 relay security |
| #505 | feat(relay): yamux stream multiplexing — M3 concurrent streams |
| #506 | feat(relay): relay_server VPS binary — M4 server-side component |
| #507 | feat(core): Arc<RwLock<weights>> concurrent inference refactor |
| #508 | chore(core): Apache 2.0 license headers in llmhosts_core, relay_core |
| #509 | feat(ci): maturin cross-compilation matrix |
| #510 | feat(core): chunked model download with resume |
| #511 | feat(security): SHA-256 checksum for GGUF downloads |
| #512 | feat(doctor): enhanced diagnostics |
| #513 | feat(relay): relay reconnect with exponential backoff + jitter |
| #514 | feat(ux): one-click Cursor/VS Code configuration |
| #515 | feat(ux): savings dashboard with monthly projection |
| #516 | feat(packaging): PyO3 ABI wheel matrix — Python 3.10/3.11/3.12/3.13 |
| #517 | feat(onboarding): llmhosts status command |
| #518 | feat(relay): relay VPS deployment guide |
| #519 | feat(docs): relay tunnel user guide |
| #520 | feat(marketing): update Apple Silicon performance claims |
| #521 | test(relay): relay M2 Noise NK integration test |
| #522 | feat(core): GGUF model validation on load |
| #523 | feat(proxy): streaming SSE through relay WebSocket |

#### Wave 2 — Concurrent Inference & Performance (Sample)

| Issue | Title |
|-------|-------|
| #524 | feat(core): PagedAttention implementation |
| #525 | feat(core): continuous batch scheduler |
| #526 | feat(core): Flash Attention v2 CUDA kernel |
| #527 | feat(core): KV cache INT8 quantization |
| #528 | feat(core): speculative decoding — EAGLE-2 |
| #529 | feat(core): prefix caching for system prompts |
| #530 | feat(core): adaptive quantization selection |
| #531 | feat(core): model warm-up / kernel JIT pre-compile |
| #532 | feat(core): inference request queue with timeout |
| #533 | feat(benchmark): llmhosts_bench suite |
| #534 | feat(benchmark): automated benchmark CI regression |
| #535 | feat(core): CUDA graphs for decode loop |
| #536 | feat(core): RoPE precomputation optimization |
| #537 | feat(core): weight loading parallelism with rayon |
| #538 | feat(proxy): per-request timeout on client disconnect |
| #539 | feat(proxy): max_concurrent_requests gate + 429 |
| #540 | feat(core): fp16/bf16 compute paths |
| #541 | feat(core): model metadata API |
| #542 | feat(core): VRAM-based quantization auto-detection |
| #543 | test(core): 16-request concurrent stress test |
| #544 | feat(core): memory pressure monitoring at 95% VRAM |
| #545 | test(core): 24-hour memory leak soak test |
| #546 | feat(relay): relay M3 yamux multiplexing |
| #547 | feat(router): model-mesh routing policy engine |
| #548 | feat(core): generate() streaming callback for SSE |

#### Wave 3 — Intelligence & HIM (Sample)

| Issue | Title |
|-------|-------|
| #549 | feat(him): HIM data collection agent |
| #550 | feat(him): HIM aggregation service with Sybil resistance |
| #551 | feat(him): HIM cold start 200-node seed dataset |
| #552 | feat(him): HIM recommendation API |
| #553 | feat(him): HIM confidence score display |
| #554 | feat(him): hardware fingerprint canonical ID |
| #555 | feat(router): ModernBERT ONNX lazy-download packaging |
| #556 | feat(router): 500-query router accuracy benchmark |
| #557 | feat(router): isotonic regression confidence calibration |
| #558 | feat(cache): vCache FAISS HNSW semantic cache |
| #559 | feat(cache): per-API-key cache namespace isolation |
| #560 | feat(cache): cache warming API |
| #561 | feat(cache): cache analytics dashboard |
| #562 | feat(intelligence): global semantic deduplication |
| #563 | feat(intelligence): predictive KV cache warming |
| #564 | feat(privacy): LDP perturbation for HIM uploads |
| #565 | feat(privacy): zero-knowledge relay audit log |
| #566 | feat(intelligence): differential cache privacy |
| #567 | feat(intelligence): LRU + frequency-weighted model eviction |
| #568 | docs(legal): GDPR DPIA for collective intelligence |

#### Wave 4 — Scale & Enterprise (Sample)

| Issue | Title |
|-------|-------|
| #569 | feat(marketplace): SLA enforcement auto-removal |
| #570 | feat(marketplace): fleet-wide model deployment push |
| #571 | feat(enterprise): team namespaces RBAC |
| #572 | feat(enterprise): audit log API for SIEM |
| #573 | feat(enterprise): usage quotas per-key and per-team |
| #574 | feat(enterprise): private model registry |
| #575 | feat(scale): relay server clustering |
| #576 | feat(scale): geographic relay routing |
| #577 | feat(marketplace): operator capacity reservation |
| #578 | feat(marketplace): buyer spend caps |
| #579 | feat(enterprise): SAML/OIDC SSO federation |
| #580 | feat(scale): marketplace CDN for model manifests |
| #581 | feat(marketplace): dynamic demand-based pricing |
| #582 | feat(core): distributed prefill/decode disaggregation |
| #583 | feat(core): cross-device tensor parallelism |

#### Research & Innovation Issues

| Issue | Title |
|-------|-------|
| #584 | feat(research): FedAttn — federated attention heads |
| #585 | feat(research): Neural HIM neural recommendation model |
| #586 | feat(cache): cache invalidation on model swap |
| #587 | feat(core): speculative decoding 2-3× speedup |
| #588 | feat(onboarding): Apple Silicon Metal 4 optimization |
| #589 | chore(security): PyPI supply chain hardening |

### New Issues — Batch 2 (110+ issues, created post-retreat)

The following issue categories were created in the post-retreat batch to fill gaps not covered in #504-#589:

**Wave 1 additional (20 issues):** relay jitter (implemented in backoff algorithm), relay health endpoint (`/health` returns relay version + connected_clients + bytes_relayed), relay client connection state machine (DISCONNECTED → CONNECTING → CONNECTED → RECONNECTING), Apache 2.0 license headers automation, Rust cross-compilation CI matrix (linux/x86_64 + linux/aarch64 + darwin/arm64 + darwin/x86_64), maturin build action for automated PyPI wheel uploads, relay M2 integration test with in-process relay server stub, relay tunnel user guide documentation, relay auto-reconnect with exponential backoff, relay WebSocket close frame on disconnect, model integrity SHA-256 verification before inference, interactive setup wizard for first-run guidance, relay connection quality metrics (latency/packet-loss/reconnect-count), memory leak detection test (1000 cycles/flat RSS), GGUF validation on load, doctor JSON output mode for CI, SSE streaming through WebSocket relay, relay VPS Docker compose deployment guide, README benchmark accuracy update, llmhosts status command.

**Wave 2 additional (25 issues):** Full PagedAttention implementation (block table, page fault, eviction), continuous batch scheduler (queue + iteration-level + max_batch_size), Flash Attention v2 CUDA (Ampere/Ada Lovelace), INT8 KV cache quantization, speculative decoding (draft model integration, 2-5× throughput), prefix caching (system prompt KV sharing), dynamic batch size tuning, tensor parallelism foundation (2×GPU split), model warm-up routine (JIT kernel pre-compile), inference request queuing (async + timeout + priority), llmhosts_bench suite (TTFT + throughput + concurrent), benchmark CI per-commit vs Ollama, CUDA graphs for decode loop, RoPE optimization (precomputed cos/sin table), weight loading parallelism (rayon), per-request timeout on disconnect, max_concurrent_requests + 429, fp16/bf16 compute paths, model metadata API from Python, VRAM-based quantization auto-detection, inference session pool (pre-allocated sessions), generate() streaming callback for SSE, LoRA adapter hot-swap foundation, 16-parallel-request concurrent stress test, memory pressure monitoring (OOM-safe at 95% VRAM).

**Wave 3 additional (20 issues):** HIM anonymous opt-in data collection, HIM Sybil-resistant aggregation service, HIM cold-start 200-node seed dataset, HIM recommendation API (`POST /him/recommend`), HIM confidence score UI, hardware fingerprint canonical ID, ModernBERT ONNX lazy-download, router accuracy 500-label benchmark, isotonic regression calibration, vCache FAISS HNSW with LRU, cache per-API-key namespace isolation, cache warming API, cache analytics dashboard, adaptive quantization, model performance predictor (pre-load TTFT estimate), smart model eviction (LRU + frequency), LDP privacy for HIM data, zero-knowledge relay audit, multi-model routing, quality-aware routing.

**Wave 4 additional (20 issues):** SLA enforcement auto-removal, Stripe Connect KYC for high-volume, fleet-wide model deployment push, enterprise team namespaces, audit log API (JSON/CSV export for SIEM), per-key and per-team token quotas, private model registry for orgs, dedicated relay endpoint per enterprise, relay server clustering (stateless + load balancer), geographic relay routing, horizontal K3s scaling guide, operator capacity reservation, buyer spend caps, SSO SAML IdP federation, SOC2 evidence export + compliance report, marketplace CDN, dynamic pricing engine, on-premise air-gapped deployment kit, relay autoscaling, operator referral program (10% commission).

**Wave 5 additional (15 issues):** Prefill/decode disaggregation across fleet, FedAttn federated attention, cross-device tensor parallelism (LAN), latent prompt compression, inference-time alignment (constitutional AI), streaming LoRA hot-swap from URL, io_uring zero-copy relay (VPS-side), model-mesh routing (Istio-style), collective model versioning, Neural HIM neural recommendation, global semantic deduplication, differential cache privacy, adaptive per-layer quantization, FedRouter (federated privacy-preserving routing), predictive KV warming from conversation history.

**Wave 6 additional (10 issues):** LLMHosts plugin SDK, community benchmark public repository, MCP server registry, operator leaderboard (public reliability ranking), model compatibility database (community-tested), LLMHosts Python async client library, LLMHosts JavaScript/TypeScript SDK (npm), ambassador program (free Pro for top contributors), LLMHosts Discord bot (GPU status + benchmark submission), hardware donation program.

**Total issues created during and after retreat: 200+**

---

## 8. User Journey North Stars (8 Personas)

Every feature decision must pass the question: "Does this help a user reach one of these moments?"

### Persona 1: Dev Anya — Solo Developer, RTX 4090
**Hardware:** Windows 11, RTX 4090 24GB, $200/month OpenAI bill
**North Star:** "pip install llmhosts && llmhosts up — 8 minutes later, Cursor is using my home GPU and I'm paying $0 to OpenAI."

**Key success moments in order:**
1. `pip install llmhosts` completes in <90 seconds (<20MB wheel)
2. `llmhosts up` detects GPU by name with specific performance prediction in 3 seconds
3. Model selection prompt shows fast (7B, 3 min download) as default — not 42GB surprise
4. First API response arrives in <5 minutes total from `pip install`
5. Dashboard projects: "At your rate, you'll save $89/month vs. OpenAI"
6. Cursor integration completes with one click from dashboard (no env var copy-paste)

**Critical friction points eliminated:**
- Model download size disclosed and alternatives offered before commit
- Browser opens automatically for device code auth (fallback: visible, copyable code)
- "Continue without account" for first 24 hours (auth paywall after aha moment, not before)

### Persona 2: Remote Anya — Coffee Shop Remote Access
**Scenario:** 2 weeks later, at a coffee shop with laptop
**North Star:** "I'm at Starbucks and the response came from my RTX 4090 at home. I can see it in the dashboard."

**Key success moments:**
- Token assistant proactively: "Your GPU is at home. Enable remote access?"
- Relay connects in <3 seconds
- Dashboard: "Request served by your RTX 4090 (home) via encrypted relay. Added latency: 147ms."
- Pro upgrade offer: "Remote access requires Pro ($12/mo). You're saving $200/mo. Remote keeps you saving while you travel."

**Paywall UX principle:** The paywall appears AFTER the aha moment, not before. User has already tasted the value.

### Persona 3: Operator Marcus — GPU Monetization
**Hardware:** RTX 4090, 3 months with LLMHosts
**North Star:** "My GPU earned $87 last month while I slept. I didn't do anything after setup."

**Key success moments (cold start resolved):**
- Day 1: "47 potential buyers have seen your node" (intermediate signal, not silence)
- Day 3: "Your node was evaluated but another was selected (lower latency). Consider pricing." (actionable)
- Day 7-14: First booking notification, celebratory framing: "Your first dollar is incoming!"
- Month 1: Dashboard earnings chart with 30-day forward projection
- Month 3: Shareable earnings card: "My RTX 4090 earned $87 this month on LLMHosts"

**Community support:** #operator-lounge Discord with <4-hour response time from team.

### Persona 4: Enterprise Tomas — Engineering Lead
**Team:** 50 people, 5×RTX 4090 workstations, $5K/month cloud API bill
**North Star:** "My team's AI API bill went from $8,000/month to $400/month."

**Key success moments:**
- 5 machines onboarded via `llmhosts up --team-key=<key>` — <2 hours total
- Fleet dashboard: "120GB total VRAM | 250 tok/s aggregate capacity | 5 nodes healthy"
- RBAC configured in 5 minutes
- Week 1: "2,400 requests | $4.80 savings vs. cloud (at scale: $3,200/month)"

**Enterprise requirements (Wave 3-4):** SAML/OIDC SSO, audit log API, usage quotas, model pinning, SLA dashboard, consolidated billing.

### Persona 5: API Consumer Sarah — SaaS Developer
**Goal:** Use LLMHosts marketplace as AI backend for her writing assistant SaaS
**North Star:** "Month 3: served 50M tokens for $5,000 vs. $7,500 on OpenAI Tier 2."

**Key success moments:**
- First marketplace request: <127ms TTFT
- Node offline → automatic cloud BYOK fallback, no user-visible error
- "Verified Nodes Only" tier: 99.2% SLA at 2× price (still cheaper than cloud)
- `X-Powered-By: LLMHosts` header drives referral loop from her app's users

**Critical abstraction:** API consumer must never need to know which physical node served their request. LLMHosts handles load balancing, failover, retry transparently.

### Persona 6: Fine-Tuner Dr. Chen — Legal ML Engineer
**Goal:** Legal AI assistant trained on 5 years of firm's contracts
**North Star (Wave 5):** "My legal AI assistant is trained on our own contracts, running on our hardware, $0 per query."

**Key success moments:**
- `llmhosts finetune prepare --input ./contracts/` processes 847 docs in <5 minutes
- Training starts, dynamic estimate computed from first 100 steps (within 20% of actual)
- Checkpoint every 30 minutes — 14-hour run survives crashes
- Dashboard shows live loss curve during training
- Adapter deploys with zero downtime via hot-swap
- First query: model correctly references firm's specific clauses

### Persona 7: Community Contributor Preethi — Open Source Developer
**Goal:** Contribute to LLMHosts codebase; build AI infra reputation
**North Star:** "My PR was merged and 2,000 users are benefiting from my kernel optimization."

**Key success moments:**
- Dev environment setup in <15 minutes (CONTRIBUTING.md clear, `cargo test` + `pytest` pass first try)
- First PR reviewed within 48 hours
- Contribution recognized: CONTRIBUTORS.md listing, Discord #contributors access
- Path to maintainer is explicit: User → Contributor → Recurring Contributor → Maintainer

**Community commitments:** 48-hour first PR review. Monthly community call. Contributor ladder published publicly.

### Persona 8: The Fleet as Emergent Intelligence
**Meta-persona:** Emergent value from 1,000+ operators participating in collective intelligence
**North Star:** "The HIM recommendation for my RTX 5090 was computed from 847 nodes with similar hardware. It was right."

**Key success moments:**
- Node 500 opt-in: HIM transitions from static table to neural model
- HIM accuracy: within 10% of predicted TTFT/throughput on first load
- Global semantic dedup: fleet cache hit rate reaches 40%
- Each new node makes HIM more accurate. Each new query makes semantic cache more valuable.

**The compounding moat:** By 10,000 nodes, the fleet intelligence is an asymmetric competitive advantage that took years to build and cannot be forked, replicated, or acquired quickly by any competitor.

---

## 9. Risk Register — Top 10

Prioritized by Risk Score (Probability × Impact, scale 1-5 each). Full register in `day-6-risk-intelligence.md`.

| # | Risk | Prob | Impact | Score | Priority | Mitigation |
|---|------|------|--------|-------|---------|-----------|
| 1 | Mutex deadlock under load (T1) | 4 | 5 | 20 | P0 | Arc<RwLock> refactor, 60s timeout — Issue #485 |
| 2 | Race condition in concurrent inference (T7) | 3 | 5 | 15 | P0 | MIRI + TSAN in CI; proptest for block allocator; 2-engineer review |
| 3 | BSL license chilling effect (B4) | 4 | 4 | 16 | P0 | Council decided: Apache 2.0 — Issue #508 |
| 4 | Apple Silicon perf gap creates backlash (T9) | 5 | 3 | 15 | P0 | Honest marketing update now — Issue #520 |
| 5 | CUDA driver change breaks inference (T2) | 3 | 4 | 12 | P1 | Pin cudarc version; CI matrix for CUDA 11.8/12.x; wgpu fallback |
| 6 | Ollama ships relay + marketplace (B1) | 3 | 4 | 12 | P1 | Moat above inference layer: HIM, enterprise, fine-tuning (12-18 month window) |
| 7 | Operator churn >25%/month (B5) | 3 | 4 | 12 | P1 | Intermediate signals, earnings forecast, cold-start support (Wave 3) |
| 8 | PyO3 ABI incompatibility (T6) | 3 | 4 | 12 | P1 | Full Python 3.10-3.13 wheel matrix — Issue #516 |
| 9 | KV cache memory leak over 24h (T5) | 3 | 4 | 12 | P1 | 24h soak test; block.return_on_drop(); 90% VRAM alert — Issue #545 |
| 10 | NVIDIA GeForce EULA commercial risk (B2) | 2 | 5 | 10 | P1 | Legal analysis before marketplace GA; AMD/Apple hedge |

### Risk Actions Required This Week

- **Risk 1 (P0):** Currently in progress on branch `fix/485-concurrent-inference-rwlock`. Do not merge until MIRI clean.
- **Risk 3 (P0):** Apache 2.0 headers must be added this week. Unblock before repo goes public.
- **Risk 4 (P0):** Marketing update is a same-day action (no engineering required).
- **Risk 6 (P1):** HIM + intelligence features are the long-term mitigation. Start Wave 3 planning now.

---

## 10. Next 14-Day Sprint Priorities

What to do RIGHT NOW. Week of March 10, 2026.

### Immediate Actions (Days 1-3, No Engineering Blocked)

**P0 Action: Commit Apache 2.0 License Headers**
- Add SPDX headers to all `.rs` files in `llmhosts_core/`, `relay_core/`
- Update `Cargo.toml` `license = "Apache-2.0"` fields
- Add `LICENSE-APACHE` at repo root
- Effort: 2 hours | Blocks: repo going public

**P0 Action: Update Marketing for Honest Platform Claims**
- Add platform capability table to README and landing page
- Remove "Noise NK encrypted" from relay until M2 ships
- Update all Apple Silicon performance language
- Effort: 4 hours | Blocks: Show HN trust

**P1 Action: Create 110+ GitHub Retreat Issues**
- Create all issues listed in Section 7 (post-retreat batch)
- Label all `advisory-retreat`; assign to wave milestones
- Effort: 2-4 hours (automated via this session)

---

### Wave 1 Engineering Sprint (Days 4-14)

**Priority 1: Relay M2 — Noise NK Encryption (Issue #440)**
The single most important engineering task. Block all other relay work until this lands.
- Implement Noise NK handshake in `relay_core`
- Server static keypair baked into binary at build time
- Verification: Wireshark trace shows zero plaintext on wire
- Estimated: 5-8 days dedicated Rust work

**Priority 2: Relay Jitter Backoff (Issue #486)**
Small fix, high value. Prevents thundering herd.
- Backoff: 1s → 2s → 4s → 8s → 30s × Uniform(0.5, 1.5) jitter
- Estimated: 2 hours

**Priority 3: llmhosts doctor Enhanced Diagnostics**
- Binary version mismatch detection
- CUDA version display
- Relay status (connected / disconnected / not configured)
- `--json` flag for CI integration
- Estimated: 1 day

**Priority 4: Chunked Model Download with Resume**
- HTTP Range header chunked download
- Progress persisted to disk; on restart, continue from last byte
- Estimated: 2-3 days

**Priority 5: SHA-256 Model Verification**
- LLMHosts-controlled manifest at `api.llmhosts.com/model-manifest.json`
- Manifest signed with ED25519; verify before inference load
- Estimated: 1-2 days

**Priority 6: Begin Arc<RwLock<>> Concurrent Inference (Issue #485)**
- This is the most complex work in the project. Do not rush.
- Day 1: Design review — struct layout for LLMHostsWeights + InferenceSession
- Day 2-3: Implement for CPU backend, validate with MIRI (must be clean)
- Day 4+: Extend to CUDA backend
- Estimated: 3-4 days total (CPU + validation), then CUDA extension

---

### 14-Day Sprint Success Checklist

By March 24, 2026, the following should be green:

- [ ] Apache 2.0 headers committed to all open-source crates
- [ ] All marketing updated (honest Apple Silicon claims, relay encryption status)
- [ ] 110+ GitHub retreat issues created and labeled with milestones
- [ ] Relay M2 (Noise NK) in active implementation (prototype handshake working)
- [ ] Relay jitter backoff merged
- [ ] `llmhosts doctor` JSON mode and enhanced checks merged
- [ ] Chunked download with resume merged or in active review
- [ ] SHA-256 model verification designed, implementation started
- [ ] `Arc<RwLock<>>` concurrent inference CPU spike done (design validated + MIRI clean)
- [ ] 50 private beta candidates identified, 20 confirmed for r/LocalLLaMA beta
- [ ] RETREAT-SUMMARY.md committed to repo (this document)

---

## Appendix A: Council Closing Statement

**The north star (one sentence):**

> LLMHosts is the platform that makes every GPU owner a production AI infrastructure operator — and makes every API consumer prefer local over cloud.

**The 5 most critical direction changes from this retreat:**

1. **Concurrent inference is the gate, not the goal.** Fix `Mutex<Option<model>>` before anything else in Wave 2. No performance claims until this is resolved. Currently in progress on branch `fix/485-concurrent-inference-rwlock`.

2. **Apache 2.0 for the engine. The moat is intelligence, not code.** Commit to open-sourcing the inference engine, relay protocol, and router architecture. Compete on what cannot be open-sourced: fleet data, HIM weights, collective intelligence.

3. **Relay security is a launch blocker.** Noise NK encryption (M2) must ship before any public launch. Not negotiable. Issue #440.

4. **Honest performance claims, platform-specific.** NVIDIA: 2× Ollama. Apple: 55% of Ollama Metal (improving). AMD: experimental. Intel: coming. Marketing that overstates destroys trust with r/LocalLLaMA — the most important community we have.

5. **The business is the network, not the software.** Every product decision should ask: "Does this grow the fleet?" The fleet is the moat. The fleet creates collective intelligence. Collective intelligence creates HIM. HIM creates lock-in. Lock-in creates enterprise value. The software is just the door to the network.

---

## Appendix B: Research Synthesis Summary

Three research documents were produced during the retreat, each synthesizing 65-87 web searches (226 total searches, March 10, 2026).

**Key competitive findings:**
- Ollama dominates local dev but serializes requests: collapses at 50 concurrent users (24.7-second p99 latency)
- vLLM delivers 793 TPS vs Ollama's 41 TPS — 19× throughput gap that PagedAttention addresses
- mistral.rs (Rust, PagedAttention + FA2 + continuous batching) is closest technical competitor — no marketplace, no tunnel, no packaging, no HIM
- GPU marketplace is a $9.97B market growing to $49.84B by 2032
- Privacy and data sovereignty are now enterprise buying criteria, not checkboxes (EU AI Act enforcement August 2026)
- Ollama's new LM Link feature (remote-connect via Tailscale) is a direct threat to relay story — ship relay GA before it gains traction

**Key architecture findings:**
- CubeCL + cudarc is the correct stack (confirmed by Cloudflare Infire engine: 82% CPU overhead reduction, 7% better than vLLM on H100)
- Phase-split (prefill/decode disaggregation) is production-grade in 2026 at Meta, LinkedIn, Mistral, Hugging Face
- Q4_K_M is the pragmatic default (92% quality retention at 100 tok/s consumer), Q5_K_M is the quality tier
- FlashAttention-4 available for H100 (71% B200 utilization); FA2 remains optimal for Ampere (RTX 30xx/40xx)
- EAGLE-2 speculative decoding delivers 2.8-5.8× throughput improvement in 2026 benchmarks

**Key user journey findings:**
- Time-to-value under 15 minutes is now table stakes (not a differentiator)
- 80%+ of enterprises plan local/on-prem AI deployment in 2026
- OpenAI-compatible API is the de facto integration standard — no evangelism needed, swap the base URL
- Ollama's sequential request processing is a documented, growing complaint in developer discourse
- $30-50/month cloud spend is the floor for meaningful savings narrative — below this, Pro pricing must be below $15/month (confirms $12/month decision)

---

## Appendix C: Issue #504-#589 Full List Reference

For the complete body text of each retreat issue (#504-#589), see the GitHub issues at:
`https://github.com/LookNoHandsMom/LLMHosts.com/issues?q=label:advisory-retreat`

For the post-retreat batch (110+ additional issues), see the same label filter — all created with the `advisory-retreat` label in the session following this document's creation.

---

*Advisory Council — 7-Day Strategic Retreat — 2026-03-10*
*All 12 advisors present across all 7 sessions. AGI Arbitrator synthesis. Unanimous on all 5 critical direction changes.*
*Document authored: Claude Sonnet 4.6 (claude-sonnet-4-6) — 2026-03-10*
*Classification: Internal — Board Level*
*Next review: After Wave 1 completion (April 2026)*
