# Day 7: Grand Synthesis — Updated Masterplan

**Date:** 2026-03-10
**Retreat:** 7-Day Advisory Council Strategic Retreat
**Session Lead:** AGI Arbitrator (synthesis of all 12 advisors, 6 sessions of deliberation)
**Purpose:** The definitive strategic output of the 7-Day Retreat.

---

## 1. What Changed From the Original Masterplan

The original Vision 2026 Masterplan (March 10, 2026) was architecturally correct but strategically incomplete in six areas. The council deliberated on each and reached the following revised positions:

### 1.1 Concurrency is the Gating Feature (NEW PRIORITY)
The masterplan correctly describes the target architecture (`Arc<dyn LLMHostsModel + Send + Sync>` + `ContinuousBatchScheduler`). What it underemphasizes is that the **current code (`Mutex<Option<model>>`) cannot do concurrent inference at all**. This is not a Wave 2 "nice to have" — it is a Wave 2 prerequisite before any performance claims can be made honestly.

**Change:** Concurrent inference (Arc<RwLock<weights>> + InferenceSession) is now the first engineering priority of Wave 2, above PagedAttention and speculative decoding.

### 1.2 Relay Security is a Launch Blocker (CLARIFIED)
The masterplan lists relay M2 (Noise NK) as "open issue #440." The gap report correctly identified this as P0 existential. A product that claims to let users "access their home GPU from anywhere" over an unencrypted channel is a trust violation.

**Change:** Relay M2 (Noise NK encryption) is gated on launch. No Show HN, no public repository, no user marketing until M2 is complete.

### 1.3 Apache 2.0 for Inference Engine (DECIDED)
The masterplan hedged on licensing (BSL vs Apache 2.0). The council deliberated and reached consensus: Apache 2.0 for all inference infrastructure (llmhosts_core, relay_core, router_core). Proprietary for: router model weights, HIM dataset, platform backend, SaaS frontend.

**Change:** License decision is locked. Engineering can proceed with Apache 2.0 headers in all relevant crates.

### 1.4 Apple Silicon Performance Claims (CORRECTED)
The masterplan uses performance language that implies parity with Ollama Metal. The wgpu/Metal backend is 50-65% of native MPS performance. This must be reflected in all documentation, benchmarks, and marketing.

**Change:** All Apple Silicon claims updated to reflect wgpu performance reality. MPS native implementation targeted for Wave 4.

### 1.5 HIM Cold Start (HONEST MARKETING)
The masterplan implies HIM is useful from day one. HIM requires minimum 500 opt-in nodes with benchmark data before it provides better recommendations than a static lookup table.

**Change:** HIM marketing language: "Fleet-powered recommendations — available as the network grows." Dashboard shows static table until k≥500 nodes per hardware category.

### 1.6 15 Novel Innovations Added to Roadmap
The original masterplan did not include: Prefill/Decode Disaggregation, Predictive KV Cache Warming, FedAttn, Neural HIM, Global Semantic Deduplication, DCP, Adaptive Quantization, Model-Mesh Routing, Streaming LoRA Hot-swap, CMV, Latent Prompt Compression, io_uring Zero-Copy Relay, Cross-Device Tensor Parallelism, Inference-Time Alignment, Autonomous Capacity Planning.

**Change:** All 15 innovations added to the implementation roadmap across Waves 2-5.

---

## 2. Wave Implementation Blueprints (Updated)

### Wave 1: Hardened Foundation (Weeks 1-6)
**Goal:** Ship what we have, cleanly. Fix the critical gaps. Launch-ready.

**Engineering priorities:**
1. Relay M2: Noise NK encryption (issue #440) — BLOCKS ALL LAUNCHES
2. Add jitter to relay reconnect backoff (GAP-10)
3. Update all marketing to remove "Noise NK encrypted" until M2 ships
4. ModernBERT ONNX distribution strategy — lazy download on `[smart]` install
5. `llmhosts doctor` enhanced: check binary version, CUDA version, relay status
6. Chunked download with resume for model files (GAP-19)
7. SHA-256 checksum verification for model downloads (GAP-15)
8. Fix HuggingFace tokenizer download to verify against checksum before using
9. Windows service auto-install (GAP-17 → systemd/launchd/Windows Service)
10. Apache 2.0 headers in all open-source crates

**Product priorities:**
1. Savings dashboard with monthly projection
2. One-click Cursor/VS Code configuration link
3. "Share your setup" sharecard generation
4. Beta program: 50 users from r/LocalLLaMA

**Success gate:** `llmhosts up` completes E2E on Windows + Mac with M2 relay. All CI green.

### Wave 2: Concurrent Inference + Performance (Weeks 6-14)
**Goal:** 2× Ollama performance, fully concurrent, PagedAttention live.

**Engineering priorities:**
1. Arc<RwLock<LLMHostsWeights>> + per-request InferenceSession (GAP-03) — CONCURRENT INFERENCE
2. PyO3 ABI: build wheels for Python 3.10-3.13 (GAP-T6)
3. PagedAttention: wire kv_cache.rs to continuous batch scheduler
4. Continuous batch scheduler: request queue, iteration-level scheduling
5. Flash Attention v2 CUDA kernel (consumer GPUs: Ampere, Ada Lovelace)
6. Adaptive quantization selection (Innovation 6)
7. Model-Mesh Routing policy engine (Innovation 10)
8. 24-hour soak test suite (memory leak, VRAM drift, relay reliability)
9. MIRI + TSAN in CI for concurrency validation
10. Relay M3: yamux stream multiplexing (issue #441)
11. Benchmark framework: automated per-commit regression vs Ollama baseline
12. Speculative decoding: EAGLE-2 draft model integration

**Product priorities:**
1. Live GPU utilization graph in dashboard
2. Real-time tokens/second counter
3. Show HN launch (gated on concurrent inference + relay M2)
4. GitHub repository public (Apache 2.0)
5. Operator first-booking notification system

**Success gate:** 4 concurrent requests all completing correctly. TTFT <150ms on RTX 4090 7B. Throughput ≥150 tok/s. Relay M2+M3 end-to-end encrypted and multiplexed.

### Wave 3: Intelligence + Privacy (Weeks 14-22)
**Goal:** Collective intelligence live, privacy-preserving, HIM useful.

**Engineering priorities:**
1. LDP perturbation of embeddings (ε=0.5, sensitivity analysis documented)
2. k-anonymity k≥500 enforcement in aggregator
3. HIM aggregator security design (GAP-13)
4. HIM static seeding (our benchmarks → 10 hardware profiles × 10 models)
5. Neural HIM training pipeline (Innovation 8, requires 500+ nodes)
6. Global semantic deduplication (Innovation 4)
7. Predictive KV cache warming (Innovation 2)
8. Differential Cache Privacy (Innovation 11)
9. Relay M4: relay_server VPS binary + deployment (issue #442)
10. GDPR DPIA documentation for collective intelligence

**Product priorities:**
1. HIM dashboard: show fleet insights
2. Collective intelligence opt-in flow with clear consent language
3. Operator earnings projection (30-day forecast)
4. Enterprise RBAC + SSO (SAML/OIDC)
5. Product Hunt launch

**Success gate:** 500+ nodes opt-in to collective intelligence. HIM returns recommendations with k≥100 confidence. GDPR DPIA complete.

### Wave 4: Scale + Relay Completion (Weeks 22-32)
**Goal:** Fleet at scale, relay hardened, distributed inference available.

**Engineering priorities:**
1. Relay M5: production hardening + docs
2. Prefill/Decode Disaggregation (Innovation 1)
3. Cross-Device Tensor Parallelism over LAN (Innovation 3)
4. io_uring zero-copy relay (VPS side only) (Innovation 9)
5. Consensus Model Validation (Innovation 13) — enterprise
6. Latent Prompt Compression (Innovation 14)
7. Intel Arc GPU detection (Level Zero API)
8. Apple MPS native integration (replace wgpu for Metal path)
9. AMD ROCm: first-class support
10. Abuse detection: prompt injection, jailbreak detection at relay

**Product priorities:**
1. Fleet dashboard v2: multi-device management, VRAM aggregate view
2. Enterprise fleet analytics: cost/savings/utilization per device
3. Marketplace premium tiers: Verified Nodes (higher price, 99.2% SLA)
4. Payout analytics dashboard for operators

**Success gate:** 2-node tensor parallel inference working on LAN. Apple Metal native performing within 10% of Ollama Metal. AMD ROCm first-class CI tests.

### Wave 5: Fine-Tuning + LoRA (Weeks 32-44)
**Goal:** First-class fine-tuning pipeline, custom model marketplace.

**Engineering priorities:**
1. LoRA adapter GGUF loading in llmhosts_core (GAP-08) — must come first
2. QLoRA training orchestration (llmhosts finetune train)
3. Training job monitoring: live loss curve in dashboard
4. LoRA adapter GGUF export from training artifacts
5. Streaming LoRA Hot-swap (Innovation 12)
6. Fine-tuning cost calculator (dynamic estimate from first 100 steps) (GAP-12)
7. Training checkpoint management (30-min auto-save)
8. Evaluation pipeline: MMLU subset, domain-specific benchmarks
9. LoRA adapter marketplace (share adapters with other operators)
10. Inference-Time Constitutional Alignment (Innovation 5) — enterprise

**Product priorities:**
1. Fine-tuning wizard: "Give me a folder, I'll handle the rest"
2. Custom model deployment to marketplace
3. Adapter marketplace: browse, purchase, deploy adapters

**Success gate:** E2E fine-tuning on RTX 4090: data prep → training → adapter → deployed. Training time estimate within 20% of actual. Adapter hot-swap zero-downtime verified.

### Wave 6: Marketplace Maturity (Weeks 44-56)
**Goal:** Marketplace at $50K+ GMV/month, self-sustaining operator community.

**Engineering priorities:**
1. Autonomous Capacity Planning (Innovation 15)
2. Dispute resolution system
3. Buyer SLA guarantee enforcement
4. Marketplace search and discovery
5. Price recommendation engine
6. Multi-region marketplace routing
7. FedAttn (Innovation 7) — research to production
8. SOC 2 Type II readiness

**Product priorities:**
1. Operator ambassador program
2. "Powered by LLMHosts" badge for API consumers
3. Community showcase: user projects built on LLMHosts
4. Annual "State of Local AI" benchmark report

---

## 3. The 15 Novel Innovations Timeline (Summary)

| Wave | Innovations |
|------|-------------|
| W2 | Adaptive Quantization, Model-Mesh Routing |
| W3 | Predictive KV Warming, Global Semantic Dedup, Neural HIM, DCP |
| W4 | Prefill/Decode Disagg, Cross-Device TP, io_uring Relay, CMV, Latent Compression |
| W5 | Inference-Time Alignment, LoRA Hot-swap |
| W6 | FedAttn, Autonomous Capacity Planning |

---

## 4. Updated User Journey North Stars

The council affirms these as the product north stars. Every feature decision should pass the question: "Does this help a user reach one of these moments?"

1. **"pip install llmhosts && llmhosts up — 8 minutes, Cursor is using my home GPU, I'm paying $0 to OpenAI."**
2. **"I'm at a coffee shop and my RTX 4090 at home is answering my questions. I can see it in the dashboard."**
3. **"My GPU earned $87 last month while I slept. I didn't do anything after setup."**
4. **"My team's AI API bill went from $8,000/month to $400/month."**
5. **"My legal AI assistant is trained on 10 years of our own contracts, running on our hardware, costing us $0 per query."**

---

## 5. Top 20 Success Criteria (Synthesized from Day 3)

1. Install-to-first-inference: <5 minutes (7B, pre-downloaded model)
2. TTFT <150ms local (RTX 4090, 7B Q5_K_M)
3. Throughput >150 tok/s (RTX 4090, 7B) — 2× Ollama baseline
4. Concurrent inference: 4 simultaneous requests, all completing correctly
5. Relay M2 encrypted: zero unencrypted bytes traverse relay
6. Relay overhead: <50ms added latency vs. direct
7. Cache hit rate >40% on semantically similar queries
8. Router accuracy >90% vs. human-labeled optimal routing
9. 7-day retention >60%
10. NPS from onboarding cohort >60
11. Marketplace operator churn <12%/month
12. `llmhosts doctor` all-green rate >80% on first run
13. Model download resume: successful after connection drop mid-download
14. GitHub stars: 1,000 (public repository milestone)
15. HIM recommendations: k≥100 nodes per hardware category
16. Relay uptime >99.5%
17. 90-day retention >35%
18. Enterprise accounts: 2 signed before Wave 4
19. Benchmark regression gate: <5% performance degradation per commit
20. Zero unhandled errors in production (all errors have user-visible explanation)

---

## 6. Community + GTM Execution Plan (90 Days)

**Days 1-30: Private Beta**
- Wave 1 engineering priorities (relay M2, concurrent inference prep)
- 50 private beta users from r/LocalLLaMA
- Benchmark collection: RTX 4090, RTX 4080, RTX 5090, M4 Max, RTX 4060
- License decision: Apache 2.0 confirmed in code headers
- Blog post: "Why we built our own inference engine"

**Days 31-60: Limited Public Beta**
- Wave 2 engineering priorities (concurrent inference, PagedAttention)
- GitHub repository goes public (Apache 2.0)
- Discord community launched (200+ members target)
- 3 benchmark posts on r/LocalLLaMA (authentic, detailed)
- Operator onboarding: first 20 operators manually onboarded

**Days 61-90: Show HN Launch**
- Show HN post (gated on: relay M2 complete, concurrent inference working, E2E tests passing)
- Synchronized Reddit, Twitter, YouTube launch
- Product Hunt submission
- Target: 500 GitHub stars in first 48 hours
- Target: 1,000 installs in first week

---

## 7. Risk Mitigation Roadmap (Top 10)

| Risk | Mitigation | Deadline | Status |
|------|-----------|----------|--------|
| Relay unencrypted | Relay M2 (Noise NK) | Before launch | Issue #440 open |
| Mutex serialization | Arc<RwLock> refactor | Wave 2 Day 1 | Issue #485 open |
| BSL license chilling effect | Apache 2.0 decision | Before repo public | Council decided |
| Apple Metal perf gap | Honest marketing | Before launch | Update needed |
| Model supply chain | SHA-256 manifest | Wave 2 | Design needed |
| Ollama competitive | HIM + intelligence | Wave 3 | Long-term |
| NVIDIA EULA | Legal analysis | Before marketplace GA | Counsel needed |
| GDPR collective intel | DPIA | Before Wave 4 | Issue #491 open |
| Operator cold start | Intermediate signals | Wave 3 | Design needed |
| Thundering herd reconnect | Jitter in backoff | Wave 1 | Issue #486 |

---

## Final Council Verdict

**The north star (one sentence):**

> LLMHosts is the platform that makes every GPU owner a production AI infrastructure operator — and makes every API consumer prefer local over cloud.

**The 5 most critical direction changes from this retreat:**

1. **Concurrent inference is the gate, not the goal.** Fix `Mutex<Option<model>>` before anything else in Wave 2. No performance claims until this is resolved.

2. **Apache 2.0 for the engine. The moat is intelligence, not code.** Commit to open-sourcing the inference engine, relay protocol, and router architecture. Compete on what cannot be open-sourced: fleet data, HIM weights, collective intelligence.

3. **Relay security is a launch blocker.** Noise NK encryption (M2) must ship before any public launch. This is not negotiable.

4. **Honest performance claims, platform-specific.** NVIDIA: 2× Ollama. Apple: 55% of Ollama Metal (improving). AMD: experimental. Intel: coming. Marketing that overstates destroys trust with the r/LocalLLaMA community — the most important community we have.

5. **The business is the network, not the software.** Every product decision should ask: "Does this grow the fleet?" The fleet is our moat. The fleet creates the collective intelligence. The collective intelligence creates HIM. HIM creates lock-in. Lock-in creates enterprise value. The software is just the door to the network.

---

*Advisory Council — Day 7 Session — Final Synthesis — 2026-03-10*
*All 12 advisors present. AGI Arbitrator synthesis. Unanimous on all 5 critical direction changes.*
