# Day 6: Risk & Competitive Intelligence

**Date:** 2026-03-10
**Retreat:** 7-Day Advisory Council Strategic Retreat
**Session Lead:** Dr. Adaeze Obi (Security), Dr. Priya Raghunathan (Privacy), Dr. Santiago Vega (Legal), Marcus Okonkwo (Competitive)
**Purpose:** Know every way this can fail. Design mitigation for each.

---

## Risk Assessment Framework

**Probability:** 1 (unlikely) to 5 (near-certain)
**Impact:** 1 (minor) to 5 (existential)
**Risk Score:** Probability × Impact
**Priority:** P0 (≥20), P1 (12-19), P2 (8-11), P3 (≤7)

---

## CATEGORY 1: Technical Risks

### T1: Mutex<Option<model>> Deadlock Under Load
- **Description:** The current Mutex prevents concurrent requests. Under load, the queue grows unboundedly. If an inference hangs (infinite generation), all subsequent requests deadlock.
- **Probability:** 4 | **Impact:** 5 | **Score:** 20 | **Priority:** P0
- **Mitigation:** Replace with Arc<RwLock<LLMHostsWeights>> + per-request InferenceSession (Wave 2 Day 1). Add request timeout: kill any generation running >60s.
- **Owner:** Engineering (Rust team)
- **Deadline:** Before any concurrent load testing

### T2: CUDA Driver Change Breaks Inference
- **Description:** NVIDIA updates CUDA toolkit (12.x → 13.x) or changes cudarc ABI. Our compiled binary stops working overnight.
- **Probability:** 3 | **Impact:** 4 | **Score:** 12 | **Priority:** P1
- **Mitigation:** Pin cudarc to a specific version in Cargo.toml. Test against CUDA 11.8, 12.0, 12.3, 12.6 in CI matrix. Maintain wgpu fallback for when CUDA breaks.
- **Early warning:** Monitor NVIDIA developer forums. Subscribe to cudarc GitHub releases.

### T3: GGUF Format Breaking Change (v5)
- **Description:** llama.cpp ships GGUF v5 with an incompatible header format. Our parser breaks silently (wrong outputs instead of errors), leading to garbage responses.
- **Probability:** 3 | **Impact:** 3 | **Score:** 9 | **Priority:** P2
- **Mitigation:** Parse GGUF magic bytes and version field first. Reject unknown versions with clear error. Subscribe to llama.cpp releases. Add GGUF version compatibility matrix to CI.

### T4: WebSocket Relay Blocked by Corporate Firewall
- **Description:** Enterprise users behind strict firewalls cannot establish WebSocket connections. This blocks the entire remote access value prop for enterprise.
- **Probability:** 4 | **Impact:** 3 | **Score:** 12 | **Priority:** P1
- **Mitigation:** Implement HTTP/2 long-polling fallback. Implement CONNECT proxy traversal. Test in corporate proxy simulator in CI. For extreme cases: HTTP/1.1 chunked transfer fallback.

### T5: KV Cache Memory Leak (Long-Running)
- **Description:** Over 24+ hours, the PagedAttention block allocator leaks pages (blocks not returned after request completion). VRAM fills; inference crashes.
- **Probability:** 3 | **Impact:** 4 | **Score:** 12 | **Priority:** P1
- **Mitigation:** Automated 24-hour soak test: run 1,000 requests over 24 hours, monitor VRAM usage. Add explicit block.return_on_drop() in InferenceSession Drop impl. Memory usage Prometheus metric with alert at 90% VRAM.

### T6: PyO3 ABI Incompatibility Across Python Versions
- **Description:** User runs Python 3.12, but our wheel was compiled for 3.11. PyO3 ABI changes break the bridge silently.
- **Probability:** 3 | **Impact:** 4 | **Score:** 12 | **Priority:** P1
- **Mitigation:** Build wheels for Python 3.10, 3.11, 3.12, 3.13 via maturin matrix. CI must test all versions. PyPI wheel matrix covers all supported Python versions.

### T7: Race Condition in Concurrent Inference
- **Description:** After implementing Arc<RwLock<weights>>, a subtle race condition exists where two requests simultaneously access the KV cache paged allocator, causing corruption or double-free.
- **Probability:** 3 | **Impact:** 5 | **Score:** 15 | **Priority:** P0
- **Mitigation:** Extensive Rust concurrency testing with MIRI (Undefined Behavior sanitizer). Thread sanitizer (TSAN) in CI. Property-based testing with proptest for the allocator. Code review requirement: 2 Rust engineers sign off on any concurrency change.

### T8: Supply Chain Compromise of GGUF Files
- **Description:** Attacker registers a deleted HuggingFace username (e.g., "bartowski") and uploads a poisoned GGUF file with identical filename. Our registry.py pulls it trusting the namespace.
- **Probability:** 2 | **Impact:** 5 | **Score:** 10 | **Priority:** P1
- **Mitigation:** SHA-256 checksum verification from LLMHosts-controlled manifest (hosted at api.llmhosts.com/model-manifest.json, signed). Never trust third-party GGUF without checksum. Implement "verified model" badge in UI.
- **Timeline:** Wave 2 — before marketplace where untrusted models might appear.

### T9: Apple Silicon wgpu Performance Gap
- **Description:** Our wgpu Metal backend is 50-65% of Ollama's Metal (MPS) performance. Users on Mac expecting parity will be disappointed and write negative reviews.
- **Probability:** 5 | **Impact:** 3 | **Score:** 15 | **Priority:** P0 (marketing/honesty)
- **Mitigation:** Already flagged in masterplan (GAP-06). Marketing must state clearly: "NVIDIA is first-class. Apple Metal via wgpu is ~55% of native MPS performance." We cannot fix the performance gap without implementing MPS directly (timeline: Wave 4). Honest disclosure is the mitigation.

---

## CATEGORY 2: Business Risks

### B1: Ollama Ships Relay + Marketplace (6-12 months)
- **Description:** Ollama (120K GitHub stars) adds a relay tunnel and GPU marketplace feature. 90% of our current value prop disappears.
- **Probability:** 3 | **Impact:** 4 | **Score:** 12 | **Priority:** P1
- **Competitive intelligence:** Ollama's current focus is on local model management, not distribution. Their 2026 roadmap shows "Ollama Cloud" (hybrid mode) but marketplace is not confirmed. Their Go implementation makes PagedAttention harder to add than our Rust implementation.
- **Mitigation strategy:** Our moat must be **above** the inference layer (collective intelligence, HIM, fine-tuning, enterprise features) — not at it. If Ollama adds relay, we compete on: (1) 2× performance via PagedAttention; (2) fleet intelligence (HIM); (3) enterprise features (RBAC, SSO, audit log); (4) the fine-tuning pipeline.
- **Counter-timeline:** Even if Ollama starts relay development today, they're 12-18 months from production relay. We need to establish user loyalty before then.

### B2: NVIDIA Changes Consumer GPU EULA (Commercial Resale)
- **Description:** NVIDIA adds language prohibiting commercial AI inference as a service on consumer GeForce cards (they already prohibit "datacenter deployment"). This could invalidate our entire marketplace.
- **Probability:** 2 | **Impact:** 5 | **Score:** 10 | **Priority:** P1
- **Mitigation:** Legal analysis: current NVIDIA GeForce EULA (January 2026) prohibits "datacenter deployment" but does not explicitly prohibit individual operators serving AI inference from home. Our legal position: operators are using their personally-owned hardware for personal commercial activity, not running a datacenter. Engage legal counsel to assess this before marketplace GA.
- **Hedge:** Support AMD GPUs (ROCm) and Apple Silicon equally. NVIDIA cannot change AMD's EULA.

### B3: Meta Llama 3 License — 700M MAU Restriction
- **Description:** Meta's Llama 3 Community License restricts commercial use above 700M monthly active users. Our marketplace serves Llama 3.3 models. This restriction is ambiguous — does "700M users" refer to end-users of apps built with Llama, or users of LLMHosts itself?
- **Probability:** 2 | **Impact:** 4 | **Score:** 8 | **Priority:** P2
- **Mitigation:** Engage IP counsel (GAP-14). Legal interpretation: the 700M threshold applies to monthly active users of the licensee's product — not token counts. LLMHosts will be well below 700M MAU for years. However, get written legal opinion before marketplace GA.
- **Fallback:** Qwen 2.5 (Apache 2.0), Mistral (Apache 2.0), Phi-4 (MIT). Llama 3 is not the only option.

### B4: BSL License Suppresses Community Growth
- **Description:** If we ship any component under BSL (Business Source License), enterprise buyers will refuse to evaluate it. Open source communities will fork or ignore it. The advisory council consensus is Apache 2.0 for inference engine.
- **Probability:** 4 | **Impact:** 4 | **Score:** 16 | **Priority:** P0 (decision required now)
- **Council recommendation:**
  - `llmhosts_core` (inference engine): Apache 2.0. This maximizes adoption, shows confidence, enables community contributions.
  - `relay_core` + `router_core`: Apache 2.0. Both have been solved by others; our differentiation is the product, not the protocol.
  - Router model weights (fine-tuned Qwen-0.5B): Proprietary. This IS the moat — the distillation of fleet intelligence.
  - HIM data: Proprietary. The hardware performance database is a competitive asset.
  - Platform code (backend, frontend): Proprietary. Not open-sourced.
- **Timeline:** License decision must be made before repository goes public.

### B5: Operator Churn >25%/Month Makes Marketplace Unviable
- **Description:** If operators leave the marketplace faster than they join, GMV stagnates and buyers leave due to availability issues.
- **Probability:** 3 | **Impact:** 4 | **Score:** 12 | **Priority:** P1
- **Root causes:** Low earnings (cold start), technical issues (relay downtime), NVIDIA EULA concerns
- **Mitigation:**
  1. Intermediate signals (first inquiry → first booking → first payout) — GAP-17
  2. Earnings projection ("your GPU should earn $X in 30 days")
  3. Capacity planning notifications (earn more during peak hours)
  4. Operator support: dedicated Discord channel, response within 4 hours

### B6: Stripe Connect KYC Blocks International Operators
- **Description:** Stripe Connect requires identity verification for payouts. In 30+ countries, Stripe requires additional documents or is not available. Operators in India, Nigeria, Turkey, Vietnam cannot receive payouts.
- **Probability:** 4 | **Impact:** 3 | **Score:** 12 | **Priority:** P1
- **Mitigation:**
  1. Expand payout providers: Wise (TransferWise) as alternative for international operators
  2. Crypto payout option (USDC on Ethereum/Solana — operator choice)
  3. Be transparent about payout availability by country before signup

---

## CATEGORY 3: Regulatory Risks

### R1: GDPR Enforcement Against Collective Intelligence (German DPA)
- **Description:** The Bundesdatenschutzbeauftragter (German DPA) is known for aggressive GDPR enforcement. Our collective intelligence system — even with LDP — could be classified as processing personal data (prompt embeddings could identify individuals).
- **Probability:** 2 | **Impact:** 4 | **Score:** 8 | **Priority:** P2
- **Mitigation:**
  1. GDPR Data Protection Impact Assessment (DPIA) before Wave 4 EU launch
  2. Opt-in consent (not opt-out) for collective intelligence — Article 6(1)(a) lawful basis
  3. LDP epsilon documentation: prove that with ε≤0.5, individual re-identification probability is below the GDPR "reasonable means" threshold
  4. Data minimization: collect minimum signals needed (token counts, not text)
  5. Engage EU GDPR counsel before Wave 4

### R2: EU AI Act — High-Risk Classification
- **Description:** The EU AI Act (full compliance August 2026) could classify LLMHosts as a "high-risk AI system" if used in regulated sectors (healthcare, legal, employment decisions).
- **Probability:** 2 | **Impact:** 3 | **Score:** 6 | **Priority:** P3
- **Mitigation:**
  1. Our product is infrastructure (a tool), not an AI system with autonomous decision-making
  2. The AI Act likely classifies the models (e.g., Llama 3.3), not the inference infrastructure
  3. Enterprise terms of service: operators are responsible for ensuring appropriate use in regulated sectors
  4. Include AI Act risk classification guidance in enterprise documentation

### R3: US Export Controls on AI Technology
- **Description:** US BIS (Bureau of Industry and Security) increasingly regulates AI model exports. Serving large Llama models to users in OFAC-sanctioned countries could violate EAR.
- **Probability:** 2 | **Impact:** 3 | **Score:** 6 | **Priority:** P3
- **Mitigation:**
  1. Block marketplace access from OFAC-sanctioned countries (Iran, North Korea, Cuba, Russia, Syria)
  2. IP-based geofencing at the relay server level
  3. Terms of service explicitly prohibit use in sanctioned countries
  4. Consult US export counsel before marketplace international expansion

### R4: CCPA Enforcement on Usage Telemetry
- **Description:** California users may exercise CCPA right to know / right to delete against our telemetry data. Our rotating HMAC system means we cannot link telemetry to a specific user, making deletion logistically impossible.
- **Probability:** 2 | **Impact:** 2 | **Score:** 4 | **Priority:** P3
- **Mitigation:**
  1. Explicit CCPA disclosure: "Collective intelligence signals are anonymized such that we cannot link them to individual users. Deletion is not technically possible, but re-identification is also not technically possible."
  2. Right to opt-out: `telemetry: false` config flag + UI toggle in dashboard
  3. Data retention: signals auto-purge after 12 months

---

## CATEGORY 4: Competitive Intelligence

### Competitive Landscape (March 2026 state)

**Ollama** (120K+ stars, primary threat):
- Current velocity: 2-3 releases/week, primarily model support additions
- 2026 roadmap includes: "Ollama Cloud" (hybrid mode between local and cloud)
- What they have that we don't: vastly larger community, Apple MPS integration, simpler UX
- What we have that they don't: PagedAttention, continuous batching, relay tunnel, marketplace, collective intelligence
- Threat level: HIGH (community size advantage) but DECLINING (technical moat growing)
- Our window: 12-18 months before they ship relay + marketplace equivalents

**vLLM** (45K+ stars, production inference):
- Current velocity: high (major releases monthly)
- 2026 vLLM Semantic Router (v0.1) shows they're expanding from pure inference to routing intelligence
- What they have: best throughput at datacenter scale, OpenAI-compatible
- What they don't have: consumer install, relay tunnel, marketplace, cross-platform (no Mac/Windows)
- Threat level: LOW for consumer market, MEDIUM for enterprise (they're targeting the same enterprise buyers)
- Our position: We are "vLLM for the rest of us" — same performance, any hardware

**llama.cpp** (70K+ stars, indirect threat, potential R11):
- As flagged in GAP-16, llama.cpp could ship relay + marketplace in 12-18 months if the community prioritizes it
- They have the inference engine. They lack: distribution infrastructure, marketplace code, collective intelligence, business model
- Threat level: MEDIUM-HIGH (large community, fast development)
- Counter-strategy: Our moat must be the **network effects** (fleet intelligence), not the inference engine (they already have a competitive one)

**SaladCloud** (60K+ GPUs, marketplace only):
- Not building a local inference tool — they ARE the cloud for consumer GPUs
- What they have: scale (60K GPUs), SOC 2, enterprise sales
- What they don't have: the "operator installs product on their machine" model — SaladCloud requires VMs
- Threat level: LOW (different product category) but GROWING (they're the most funded competitor)
- Our position: We win on operator UX. `pip install llmhosts` vs. SaladCloud's container deployment process.

**LM Studio / GPT4All** (consumer tools):
- Local-only inference, no marketplace, no relay, no enterprise
- Threat: indirect (they satisfy the "use local model" need without the full LLMHosts stack)
- No trajectory toward our feature set

### Early Warning System

The council recommends monitoring these signals weekly:

| Competitor | Watch for | Action trigger |
|------------|-----------|----------------|
| Ollama | "relay" or "marketplace" in GitHub issues/PRs | Accelerate Wave 3-4 features |
| llama.cpp | "tunnel" or "distribution" in GitHub | Prioritize HIM moat features |
| SaladCloud | "pip install" or "operator tool" in blog | Aggressive operator acquisition |
| vLLM | "consumer" or "Windows" or "Mac" in roadmap | Accelerate enterprise features |
| Hugging Face | "local inference marketplace" announcement | Emergency board meeting |

---

## Risk Register Summary

### P0 Risks (Immediate Action Required)

| Risk | Action | Owner | Deadline |
|------|--------|-------|----------|
| T1: Mutex deadlock | Arc<RwLock> refactor | Rust engineer | Wave 2 Day 1 |
| T7: Concurrency race condition | MIRI + TSAN in CI | Rust engineer | Before W2 testing |
| T9: Apple Metal perf gap | Update all marketing | Product | Before HN launch |
| B4: BSL license decision | Choose Apache 2.0 | Legal + founders | Before repo goes public |

### P1 Risks (Wave 1-2)

| Risk | Action | Owner | Deadline |
|------|--------|-------|----------|
| T2: CUDA driver | Pin cudarc version, CI matrix | Rust engineer | Wave 1 |
| T4: WebSocket blocked | HTTP fallback | Relay engineer | Wave 2 |
| T5: KV cache leak | 24h soak test | QA engineer | Wave 2 |
| T6: PyO3 ABI | Full Python version matrix | CI engineer | Wave 1 |
| T8: Model supply chain | SHA-256 manifest | Security engineer | Wave 2 |
| B1: Ollama competitive | HIM + intelligence features | PM | Wave 3 focus |
| B2: NVIDIA EULA | Legal analysis | IP counsel | Before marketplace GA |
| B5: Operator churn | Intermediate signals | Product | Wave 3 |
| B6: International payouts | Wise integration | BD | Wave 3 |

### P2 Risks (Wave 2-4)

| Risk | Action | Owner | Deadline |
|------|--------|-------|----------|
| T3: GGUF format | Version-aware parser | Rust engineer | Wave 2 |
| B3: Llama license | IP counsel opinion | Legal | Before marketplace GA |
| R1: GDPR collective intel | DPIA + EU counsel | Legal | Before Wave 4 EU launch |

### P3 Risks (Wave 3-6)

| Risk | Action | Owner | Deadline |
|------|--------|-------|----------|
| R2: EU AI Act | Classification analysis | Legal | August 2026 deadline |
| R3: Export controls | Geofencing + ToS | Legal | Before international expansion |
| R4: CCPA telemetry | Privacy policy + opt-out | Legal + eng | Before US public launch |

---

## Day 6 Council Verdict

**DR. ADAEZE:** "The security profile of LLMHosts has two acute vulnerabilities: the unencrypted relay (GAP-02) and the model supply chain (GAP-15). Both are fixable in Wave 1-2. After those, the attack surface is well-managed."

**DR. PRIYA:** "The distributed systems risks are concentrated in the transition from single-node to concurrent inference. MIRI testing is non-negotiable — Rust's type system prevents many races, but not all. The concurrency review process must be rigorous."

**DR. SANTIAGO:** "The licensing decision (B4) must happen today, not next week. Shipping even a single commit with an ambiguous license creates cleanup work that delays launch. Apache 2.0 for the open-source components; proprietary for the moat assets."

**MARCUS:** "The Ollama competitive threat is real but manageable. Our 12-18 month window to establish user loyalty is the critical constraint. Everything that slows us down is existential; everything that grows the community is survival."

---

*Advisory Council — Day 6 Session — 2026-03-10*
*Lead: Dr. Adaeze Obi, Dr. Priya Raghunathan, Dr. Santiago Vega, Marcus Okonkwo*
