# Confidence Assessment 003 -- LLMHosts.com

## META
- date: 2026-02-14
- trigger: Sentinel v1.0 proof-of-concept validation (first assessment using Sentinel protocol)
- phase: SPEC (full IdeaFactory output exists, project skeleton built, no functional features)
- assessor: Sentinel v1.0
- previous: 2026-02-14_confidence-002.md (composite: 66, manual methodology)
- methodology: Sentinel 4-layer DAG (Ingest→Score→Diagnose→Prescribe) with 24 sub-dimensions, evidence tiers, interaction matrix, phase caps

---

## L0: INGEST -- Project State Summary

**Documentation**: Complete IdeaFactory output (PRD, 8 ADRs, roadmap, backlog, knowledge base, implementation blueprint, competitive analysis). Course correction v3.0 applied. 2 prior confidence assessments + improvement plan.
**Codebase**: Project skeleton (pyproject.toml, cli.py, config.py, constants.py, Dockerfile, docker-compose.yml, CI pipeline). Zero functional features. All linting/type-checking passes.
**Metrics**: Zero users, zero installs, zero revenue. GitHub repo not yet public. 
**Deferred items**: 6 (P0: GitHub unflag, P0: GitHub Pro, P1: branch protection, P1: Copilot agent, P1: Project board, P2: SSH keys).
**Phase classification**: **SPEC** (full specs exist + skeleton code, but no functional features or users).

---

## L1: SCORE -- 24 Sub-Dimensions

### TECHNICAL FEASIBILITY (15%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| TF-1: Architecture Soundness | 72 | T3 | 8 ADRs with alternatives analysis, system architecture doc, tiered install design. No production validation. | Strong architecture DESIGN but zero production proof. ADRs well-reasoned (three-tier router, vCache, Tailscale/Cloudflare). Tech stack (FastAPI+Pydantic) appropriate. Cap: T3 evidence → max 70, scoring at 70. Adjusted to 72 per T4 evidence from working Docker build. |
| TF-2: Dependency Risk | 65 | T4 | pyproject.toml with pinned versions. All deps verified: FastAPI, Click, LiteLLM, textual, rich all actively maintained. FAISS-cpu stable. PyTorch/transformers for [full] tier. | Good dep hygiene. No known CVEs at install time. Lock file exists. Risk: torch dependency in [full] tier is heavy but well-tested. |
| TF-3: Integration Complexity | 55 | T3 | Proxy→Ollama→Router→Cache pipeline defined in architecture but no integration code exists. API contracts partially defined (OpenAI format documented). | Integration is the hardest technical risk. 4 systems (proxy, router, cache, Ollama) must interoperate. Zero integration code written. Defined but untested. |
| TF-4: Scalability Headroom | 60 | T3 | Architecture supports horizontal scaling conceptually. FastAPI async by default. Phase 1 is localhost (single machine). K3s planned for Phase 3. | Adequate for Phase 1 (single user). Scaling design exists on paper but unproven. Pre-product, this is acceptable. |

**TF Dimension Average**: 63.0

### MARKET VALIDITY (20%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| MV-1: Problem Evidence | 65 | T3 | OpenRouter $500M valuation validates proxy market. LiteLLM 36K stars, 240M+ Docker pulls. Reddit communities (r/LocalLLaMA 500K+, r/selfhosted) discuss GPU utilization problem. No first-party user interviews. | Strong MARKET evidence from competitors and communities. But no evidence of LLMHosts-specific demand. Market saturation note: large market BUT OpenRouter, LiteLLM, Helicone are well-established. |
| MV-2: Willingness to Pay | 48 | T2 | Pricing based on competitor analysis ($10-29 dev tools). No user surveys. No waitlist. Van Westendorp not run. | Competitor-based pricing only. Zero direct WTP validation. Cap: T2 → max 50. |
| MV-3: Market Timing | 62 | T3 | Local LLM adoption growing (Ollama 100K+ daily users). Consumer GPUs increasingly powerful. Privacy concerns driving on-prem. AI cost awareness emerging. | Good timing argument with supporting data. But no evidence that timing is right for THIS product specifically. |
| MV-4: Distribution Channel | 45 | T2 | Channels identified (HN, Reddit, PyPI, GitHub). Zero channels tested. CAC unknown. GTM strategy exists in docs but nothing executed. | Plan exists but untested. "Open source" is not a distribution strategy by itself. Zero real distribution data. |

**MV Dimension Average**: 55.0

### EXECUTION VELOCITY (20%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| EV-1: Shipped Features | 12 | T4 | 0/22 blueprint steps functional. Project skeleton exists (CLI stub, pyproject.toml, Docker). `llmhosts --version` works. Zero functional proxy, router, cache, or discovery. | Skeleton is not a feature. Phase-appropriate but score reflects reality: nothing works yet. |
| EV-2: Milestone Accuracy | 20 | T2 | No milestones have been hit or missed yet. Blueprint exists with estimated timeline. Two-wave strategy defined. | No data point on execution accuracy. Plan exists but unproven. |
| EV-3: Scope Discipline | 55 | T3 | Two-wave release strategy. Explicit scope cut (Wave 1 = proxy+router+serve). No-build list exists. Deferred items documented. 10 epics, 110+ backlog items with priorities. | Good scope management on paper. Wave 1 scope is focused. But: 110+ backlog items total suggests ambition may outpace capacity. |
| EV-4: Feedback Loop | 10 | T1 | No users. No feedback mechanism. No analytics. GitHub not public. | Cannot have a feedback loop without users. Phase-appropriate 0. |

**EV Dimension Average**: 24.25

### COMPETITIVE MOAT (15%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| CM-1: Unique Combination | 58 | T3 | Feature matrix documented: local-first + tunnel + verified cache (vCache) + three-tier router. No single competitor offers all four. Helicone lacks local. OpenRouter lacks self-hosting. LiteLLM lacks smart routing. | Unique combination exists on paper. No competitor occupies the exact niche. But: the combination is unproven. Would users value ALL four together? |
| CM-2: Switching Cost | 30 | T2 | Some switching cost by design (config files, API key management). But OpenAI-compatible format means easy switching to any other proxy. | Low switching cost. API compatibility is a feature but also makes switching easy. |
| CM-3: Data/Network Effects | 35 | T2 | Data flywheel designed (aggregate routing data improves global router in Pro tier). Architecture supports it. But: requires thousands of users to activate. Zero users today. | Designed but will take months/years to materialize. At 0 users, this is a design principle, not a moat. |
| CM-4: Time-to-Replicate | 40 | T3 | Estimated 3-6 months for funded competitor to replicate core functionality. Novel components: three-tier router, vCache integration. But: each individual component is well-documented in research papers. | Individual components are replicable. The COMBINATION adds time. But a well-funded team (e.g., OpenRouter adding local inference) could replicate in months. |

**CM Dimension Average**: 40.75

### REVENUE PATH (20%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| RP-1: Conversion Evidence | 8 | T1 | Zero users. Zero conversions. Zero waitlist. Pricing defined ($19/mo Pro) but untested. | No conversion data of any kind. Score reflects complete absence of evidence. |
| RP-2: Unit Economics | 35 | T2 | Financial model exists in PRD. $19/mo Pro, $49/seat Team. Infrastructure costs estimated. LTV/CAC modeled but with assumed inputs. | Financial model on paper with industry benchmark inputs. All assumptions. Cap: T2 → max 50. |
| RP-3: Retention Signal | 5 | T1 | Zero users. No retention data possible. | Cannot measure retention without users. |
| RP-4: Growth Trajectory | 5 | T1 | Zero growth. Product not launched. | Cannot measure growth without launch. |

**RP Dimension Average**: 13.25

### FOUNDER/TEAM CAPACITY (10%)

| Sub-Dim | Score | Tier | Evidence | Justification |
|---------|-------|------|----------|---------------|
| FC-1: Skill Coverage | 48 | T3 | Solo founder. Strong technical skills (architecture, Python, Docker, AI). Weaker: marketing, community management, content creation. Using AI agents for code execution. | Covers engineering well. Marketing/community are gaps. AI tools help but don't replace domain expertise. |
| FC-2: Bandwidth vs Scope | 40 | T3 | 22-step blueprint for solo founder. Two-wave release helps but still ~4 weeks of sustained development + marketing. 110+ backlog items long-term. | Scope-to-bandwidth ratio is tight even with AI tools. Two-wave strategy helps but it's still ambitious for one person. |
| FC-3: Burnout Risk | 50 | T2 | Ship-and-breathe cadence defined. Two-wave reduces initial sprint. But: solo founder with no backup. GitHub account flagged adds friction. | Cadence management designed but unproven. Solo with no backup is inherently fragile. |
| FC-4: Hiring Pipeline | 35 | T2 | First hire trigger at $3K MRR. Role defined (DevRel/community). No candidates identified. No budget yet ($0 revenue). | Trigger defined but distant. No pipeline started. Appropriate for pre-revenue but still a gap. |

**FC Dimension Average**: 43.25

---

## L1 BIAS CHECKS

**CHECK 1 (Confirmation)**: Am I scoring high because this is the first Sentinel proof-of-concept and I want it to look good? REVIEW: No -- scores are lower than Assessment 002 in several areas because Sentinel's evidence tier caps and dimensional isolation are stricter. Assessment 002 scored Market at 82; Sentinel scores it at 55 because evidence tiers cap most sub-dimensions. This feels more correct.

**CHECK 2 (Anchoring)**: Am I anchored to Assessment 002's scores? REVIEW: Partially. I consciously re-evaluated from zero. The largest delta is Execution (002: 50, Sentinel: 24.25) because Assessment 002 credited the improvement plan as partial execution; Sentinel doesn't -- a plan is not execution.

**CHECK 3 (Halo Effect)**: Is the strong documentation (17+ knowledge base files, 8 ADRs, complete IdeaFactory output) inflating non-documentation scores? REVIEW: Possible risk. Documentation quality does NOT equal market validity or revenue path. Verified: MV and RP scores are low despite great docs. No halo detected.

---

## L2: DIAGNOSE -- Interaction Matrix & Composite

### Step 1: Interaction Matrix Application

| Rule | Condition | Active? | Effect |
|------|-----------|---------|--------|
| Rule 1: Technical Floor | TF(63) < 60? | NO (63 > 60) | Not applied |
| Rule 2: Founder Caps Execution | FC(43.25) < 40? | NO (43.25 > 40) | Not applied |
| Rule 3: Execution Caps Revenue | EV(24.25) < 30? | YES | RP = min(13.25, 40) = 13.25 (already below cap, no change) |
| Rule 4: Market Caps Moat | MV(55) < 50? | NO (55 > 50) | Not applied |
| Rule 5: Revenue Evidence | RP-1(8) = 0? | YES (effectively 0) | RP = min(13.25, 55) = 13.25 (already below cap) |
| Rule 6: Execution Evidence | EV-1(12) < 20? | YES | EV = min(24.25, 50) = 24.25 (already below cap) |
| Rule 7: Market Without Users | Phase=SPEC, no users? | YES | MV = min(55, 80) = 55 (already below cap) |

**Post-interaction scores**: TF=63, MV=55, EV=24.25, CM=40.75, RP=13.25, FC=43.25

### Step 2: Phase Caps (SPEC phase)

| Dimension | Post-Interaction | SPEC Cap | Final |
|-----------|-----------------|----------|-------|
| Technical Feasibility | 63.0 | 90 | 63.0 |
| Market Validity | 55.0 | 80 | 55.0 |
| Execution Velocity | 24.25 | 55 | 24.25 |
| Competitive Moat | 40.75 | 60 | 40.75 |
| Revenue Path | 13.25 | 45 | 13.25 |
| Founder/Team | 43.25 | 75 | 43.25 |

No phase caps are binding (all scores below their caps).

### Step 3: Composite Calculation

```
Technical Feasibility:  63.00 × 0.15 =  9.45
Market Validity:        55.00 × 0.20 = 11.00
Execution Velocity:     24.25 × 0.20 =  4.85
Competitive Moat:       40.75 × 0.15 =  6.11
Revenue Path:           13.25 × 0.20 =  2.65
Founder/Team:           43.25 × 0.10 =  4.33
─────────────────────────────────────────────
COMPOSITE:                             38.39 → 38
```

**Overall: 38 / 100 -- CRITICAL**

### Step 4: Bottleneck Analysis

**Rank by distance from target (90)**:
1. **Revenue Path**: 13.25 → needs +76.75 to reach 90. Root cause: PHASE (no revenue possible at SPEC). No fix until SHIPPED+GROWING.
2. **Execution Velocity**: 24.25 → needs +65.75. Root cause: nothing shipped. FIX: ship Wave 1 (will move EV to 55-65).
3. **Competitive Moat**: 40.75 → needs +49.25. Root cause: unvalidated combination, low switching cost. FIX: ship + get users (can't validate moat without adoption).

**Binding constraint**: Revenue Path and Execution Velocity are both far from target, but EV is the one that CAN be improved at this phase. RP requires users which requires shipping which requires EV.

**Root Cause**: The project is in SPEC phase with zero functional features. The single most impactful improvement is: **ship working code.**

---

## L3: PRESCRIBE -- Improvement Plan

### Tactic Selection (top 5 by ROI, applicable to SPEC/BUILDING phase)

| Rank | Tactic | Impact | Effort | ROI | Route |
|------|--------|--------|--------|-----|-------|
| 1 | T-EV-01: Scope Cut to MVJ | +15 EV, +8 FC | 4 hours | 46.0 | Founder + AI agent |
| 2 | T-EV-02: Two-Wave Release | +10 EV, +5 RP | 4 hours | 30.0 | Already applied (plan exists) |
| 3 | T-TF-01: ADR Sprint | +12 TF, +3 EV | 1 day | 15.0 | Already done (8 ADRs exist) |
| 4 | T-MV-01: Competitive Blind-Spot Audit | +8 MV, +5 CM | 1 day | 13.0 | Already done (improvement plan) |
| 5 | T-EV-10: Parallel Marketing Track | +5 EV, +8 MV | Ongoing | N/A | Founder (start immediately) |

**Conflict check**: No conflicts detected between selected tactics.

**Net new actions (excluding already-applied)**:

| # | Action | Who | Route | Expected Outcome | Verification |
|---|--------|-----|-------|-----------------|--------------|
| 1 | **Ship Wave 1**: Implement blueprint steps 2-9 (config→db→discovery→keys→backend→proxy→serve→rule-router). Target: `llmhosts serve` routes to Ollama and returns responses. | AI agent + Founder | Get-er-DONE L3-L4 | EV: 24→55 (+31), TF: 63→72 (+9) | `curl localhost:8000/v1/chat/completions` returns valid response |
| 2 | **Start marketing in parallel**: Make GitHub public, join r/LocalLLaMA daily, draft Show HN, start content (blog post on proxy architecture). | Founder | Human action | MV: 55→62 (+7), EV: 24→28 (+4) | GitHub public, 3+ community interactions, Show HN draft ready |
| 3 | **Landing page with email capture**: Simple landing page at LLMHosts.com with value prop and email signup. Measure conversion rate. | Founder | Human action | MV: 55→60 (+5), RP: 13→20 (+7) | Landing page live, 50+ email signups |

### Projected Post-Improvement Scores

| Dimension | Current | Projected | Delta |
|-----------|---------|-----------|-------|
| Technical Feasibility | 63.0 | 72 | +9 |
| Market Validity | 55.0 | 62 | +7 |
| Execution Velocity | 24.25 | 55 | +31 |
| Competitive Moat | 40.75 | 45 | +4 |
| Revenue Path | 13.25 | 25 | +12 |
| Founder/Team | 43.25 | 50 | +7 |

**Projected Composite**: 72 × 0.15 + 62 × 0.20 + 55 × 0.20 + 45 × 0.15 + 25 × 0.20 + 50 × 0.10 = 10.8 + 12.4 + 11.0 + 6.75 + 5.0 + 5.0 = **50.95 → 51**

**Delta from current**: 38 → 51 (+13 points). Phase would transition from SPEC to BUILDING/SHIPPED.

---

## COMPARISON: Sentinel v1.0 vs Manual Assessment 002

| Dimension | Assessment 002 (manual) | Sentinel 003 | Delta | Explanation |
|-----------|------------------------|--------------|-------|-------------|
| Technical | 88 | 63 | -25 | Sentinel applies evidence tier caps. T3 evidence caps most TF scores at 70. Manual assessment credited "plans" as partial evidence. |
| Market | 82 | 55 | -27 | Sentinel requires direct WTP validation, not just competitor proxies. Evidence tier caps bite hard here. |
| Execution | 50 | 24 | -26 | Sentinel doesn't credit plans as execution. 0 features = low EV. |
| Moat | 63 | 41 | -22 | Sentinel penalizes paper-only moat. No switching cost measured. No data flywheel active. |
| Revenue | 55 | 13 | -42 | Sentinel scores near 0 for zero revenue, zero users, zero conversions. Manual credited the financial model. |
| Founder | 58 | 43 | -15 | Sentinel penalizes bandwidth-to-scope ratio more heavily. |
| **Composite** | **66** | **38** | **-28** | Sentinel is significantly stricter due to evidence tier caps and dimensional isolation. |

### Why the Difference?
Assessment 002 used forward-looking confidence (crediting plans and intent). Sentinel scores CURRENT STATE only, with evidence quality determining score ceilings. A plan backed by Tier 2-3 evidence caps at 50-70. Zero revenue/users caps entire RP dimension near floor.

**This is by design.** Sentinel's strictness prevents false confidence. The 38 composite for a SPEC-phase project with zero shipped features is ACCURATE. The plan's benchmarks show SPEC-phase projects typically score 50-65 -- LLMHosts.com scores below this range because several sub-dimensions (RP, EV) have near-zero evidence.

The path forward is clear: **ship code, get users, measure everything.** Every point of real evidence will raise scores faster than any amount of planning.

---

## NEXT ASSESSMENT TRIGGER
- After Wave 1 ships (`llmhosts serve` functional), OR 2 weeks, whichever comes first.
- Expected phase transition: SPEC → BUILDING (after Wave 1 starts) or SHIPPED (after Wave 1 completes).
