# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What This Is

LLMHosts.com is a CLI-first tool + SaaS platform that turns a home GPU into production AI infrastructure accessible from anywhere. Distributed as a PyPI package (`llmhosts`) with a three-tier intelligent router, verified semantic cache, and OpenAI + Anthropic compatible proxy.

**Tagline**: "Your hardware. Real AI infrastructure. From anywhere."

## Session Startup Protocol

GitHub is the single source of truth. Every session, before doing any work:

1. **Handoff / new context:** If switching from another AI or dev instance, read `docs/SSOT-AND-HANDOFF.md` first.
2. **GitHub state** — run these to understand current project state:
   - `gh issue list --state open --limit 20` — what's open
   - `gh pr list --state open` — what's in progress
   - `gh project item-list 5 --owner LookNoHandsMom --limit 20` — board overview
   - `gh run list --limit 5` — recent CI status
2. **Scan `deferred/`** for blocked items that may have been unblocked. Check programmatic triggers where possible.
3. **Read `CLAUDE.md`** (this file) for architecture, patterns, and commands.
4. **Read `MEMORY.md`** (auto-memory) for gotchas and learned patterns.
5. **Pick work** from open issues or the project board. Prioritize by label and project field values.

## Session End Protocol

All work flows through GitHub PRs. No manual tracker files needed:
- Commit on a feature branch, push, create PR with `gh pr create`
- Branch protection requires PR + CI pass before merge to main
- Close relevant issues via PR body (`Closes #123`) or `gh issue close`
- Update `deferred/` if new blocks are discovered or existing ones resolved

## Build & Dev Commands

### Backend (Python)
```bash
# Install (editable, with dev tools)
pip install -e ".[dev]"

# Run the server
llmhosts serve
llmhosts serve --port 8080 --no-tui

# Other CLI commands
llmhosts doctor          # Diagnostics
llmhosts keys add openai sk-...
llmhosts tunnel start
llmhosts suggest-models
llmhosts setup           # First-run wizard

# Lint & format
ruff check .            # Lint
ruff format .           # Format
mypy llmhosts/ --ignore-missing-imports  # Type check

# Tests
pytest tests/ -v                                    # All tests
pytest tests/ -v --cov=llmhosts --cov-report=term-missing  # With coverage
pytest tests/test_router.py -v                      # One file
pytest tests/test_router.py::test_rule_based_routing -v   # One test
pytest -m unit                                      # By marker (unit/integration/e2e/spec/contract/performance/security/slow)

# Build wheel
python -m build --wheel --outdir dist/

# Docker
docker compose build dev && docker compose run --rm dev   # Dev container
docker build -f Dockerfile -t llmhosts:latest .            # Production image

# Monitoring stack (Prometheus + Grafana + Loki)
docker compose -f monitoring/docker-compose.monitoring.yml up -d    # Start monitoring
docker compose -f monitoring/docker-compose.monitoring.yml ps       # Check health
docker compose -f monitoring/docker-compose.monitoring.yml down     # Stop monitoring
```

### Frontend (Next.js — run from `frontend/`)
```bash
npm run dev             # Dev server
npm run build           # Production build
npm run lint            # ESLint
npm run test            # Vitest (unit)
npm run test:coverage   # Vitest with coverage
npm run test:e2e        # Playwright E2E
npm run test:e2e:ui     # Playwright with UI
npm run test:e2e:debug  # Playwright debug mode

# Prisma (run from frontend/)
npx prisma migrate dev  # Apply migrations locally
npx prisma generate     # Regenerate client (required before Docker build)
npx prisma studio       # Visual DB browser
npm run db:seed         # Seed database
```

## Architecture Overview

**Two Python packages** ship together (both in the wheel):

### `llmhosts/` — Core CLI tool (PyPI: `llmhosts`)
- **`cli.py`** — Click CLI entrypoint (`llmhosts` command). Routes to subcommands.
- **`server.py`** — FastAPI app orchestrator. Mounts proxy, dashboard, and hive admin routes.
- **`config.py`** — TOML config loader. Priority: defaults → `~/.config/llmhosts/config.toml` → `--config` flag → `LLMHOST_*` env vars → CLI flags.
- **`proxy/`** — FastAPI routes for OpenAI (`/v1/chat/completions`, `/v1/models`) and Anthropic (`/v1/messages`) compatible APIs. `dispatcher.py` manages backend selection and request routing.
- **`router/`** — Three-tier hybrid router. Tier 1: rule-based + kNN (FAISS/ONNX). Tier 2: ModernBERT classifier. Tier 3: Qwen2.5-0.5B (Phase 4). Each tier has a confidence threshold; falls through to next tier if unmet.
- **`cache/`** — Three-tier: exact hash → entity namespace → vCache semantic (ICLR 2026). `manager.py` coordinates the tiers.
- **`discovery/`** — Auto-discovers Ollama instances and hardware (GPU detection).
- **`tunnel/`** — Self-hosted Rust relay (primary); Tailscale or Cloudflare Tunnel as fallback.
- **`keys/`** — BYOK encrypted key storage using Fernet symmetric encryption.
- **`dashboard/`** — Textual TUI dashboard + web dashboard API bridge.
- **`backends/`** — LiteLLM adapter and model registry.
- **`metrics/`** — Cost tracking, savings calculation, Prometheus export.
- **`audit/`** — SQLite-backed audit logging.
- **`health/`** — Backend health monitoring.

### `hive/` — Embedded agent system (Hive.CEO)
- **Separate from `llmhosts/`** — has its own agents, admin routes, LLM providers, GitHub integration.
- Agents: CEO, Analyst, Watcher, Builder, Advisor (all inherit `hive.agents.base.BaseAgent`).
- Admin panel mounts at `/admin/hive/` on the host FastAPI app.
- All code changes go through GitHub PRs. Human approval required.
- Progressive enhancement — LLMHosts works without it.
- Minimizes dependencies (prefers stdlib: `ast`, `json`, `pathlib`, `dataclasses`).

### `.hive/` — Agent identity & state (git-tracked)
- `identity.md`, `soul.md`, `config.yaml` — agent personality and settings.
- `memory/` — institutional memory (patterns, incidents, decisions).
- `backlog/items/` — agent-managed improvement backlog.

### `frontend/` — Next.js 16 web dashboard (Phase 2+)
- App Router, React 19, TypeScript, TailwindCSS 4, Prisma, Recharts.
- **State**: Tanstack Query for server state, Zustand for client state.
- **Auth**: NextAuth v5 (Auth.js). Must set `AUTH_TRUST_HOST=true` behind Traefik.
- **DB**: Prisma + SQLite (`frontend/prisma/schema.prisma`). Run `npx prisma generate` before Docker build.

### `tests/` — pytest suite
- Uses ASGI transport (no real server). Fixtures in `conftest.py` create the FastAPI app and wire `BackendDispatcher` manually.
- `asyncio_mode = "auto"` in pyproject.toml — all async tests run automatically.
- Pydantic models need `model_rebuild()` in conftest due to `from __future__ import annotations`.

## Key Patterns

- **Tiered pip install**: `llmhosts` (core ~50MB), `llmhosts[smart]` (+ONNX/FAISS ~150MB), `llmhosts[full]` (+PyTorch ~2GB). Guard optional imports with try/except.
- **async/await everywhere** — no blocking I/O in the FastAPI paths.
- **Pydantic v2** for all request/response validation and serialization.
- **Response headers**: `X-Request-ID` (tracing), `X-LLMHosts-Cost`, `X-LLMHosts-Model`.
- **Build system**: Hatchling (PEP 517), version sourced from `llmhosts/__init__.py`.
- **Type hints on all functions**, docstrings on public functions (concise).
- **IslandRun privacy routing**: PII detected before external API routing. Tiered trust — personal=1.0, private=0.7, external=0.4. See `llmhosts/router/engine.py`.
- **Router performance targets**: Tier 1 kNN <15ms CPU; Tier 2 ModernBERT <50ms; Tier 3 Qwen2.5-0.5B Q4 <500ms. Total router VRAM budget: <1GB alongside inference models.

## Coding Standards

- **Python**: 3.12+, FastAPI, Pydantic v2, async/await everywhere, ruff for lint+format.
- **Frontend**: Next.js 15+ App Router, TypeScript strict, TailwindCSS, shadcn/ui.
- **Docker always**: every service runs in containers. Never install natively.
- **Research first**: before implementation, research current best practices.
- **Full implementation**: no partial work or shortcuts. Break into smaller edits if needed, but complete the full scope.

## Knowledge Sync Protocol

Changes must cascade. When modifying one layer, update all affected layers:

- **Architecture change** → update `.cursor/rules/architecture-decisions.mdc`, `project-context.mdc`, roadmap, backlog, KB docs, `AGENTS.md`
- **Requirement change** → update KB doc, backlog, roadmap, `project-context.mdc`
- **Code implementation** → verify matches ADRs, update backlog item status, update roadmap progress
- **Hive.CEO change** → update `.hive/identity.md`, `capabilities.md`, `constraints.md`, `AGENTS.md`, `CONTEXT.md`
- **Cross-cutting change (3+ systems)** → run sync-completeness audit per `.cursor/rules/sync-completeness.mdc` BEFORE declaring done

## Naming Discipline

- **Hive.CEO** = embedded agent system (code in `hive/`, state in `.hive/`). Active now.
- **Hive Mode** = Phase 3 GPU pooling feature (`llmhosts/hive/`). Built in commit b3cd87b.
- Never conflate these.

## Deferred Items

`deferred/` tracks work blocked on human action or external dependencies. On session start, scan for items that may have been unblocked. On encountering new blocks, create `deferred/P[N]-[short-id].md` with unblock trigger and integration steps.

## Current Project State

- **Phase**: Phase 2 complete, Phase 3 in progress (Headscale deprecated per ADR-009, Rust relay is primary tunnel).
- **GitHub**: `LookNoHandsMom/LLMHosts.com` (private). Pro plan active. Branch protection on main. Projects V2 board active.
- **Deferred blockers**: None active in `deferred/`. See `deferred/README.md` for resolved items.
- **Customer readiness**: All production secrets configured (including `RESEND_API_KEY` — issue #187 closed 2026-02-21). Signup E2E works, new customers can onboard.
- **Current state**: Check `gh issue list` and `gh pr list` — GitHub is the only reliable source for live counts and status.

## Configuration

Ruff: `line-length = 120`, `target-version = "py310"`. MyPy: `disallow_untyped_defs = true`. See `pyproject.toml` for full tool config.

## Git Conventions

- Conventional commits: `feat(scope):`, `fix(scope):`, `docs:`, `refactor:`, `test:`, `chore:`
- Branch naming: `feature/<id>-<desc>`, `fix/<id>-<desc>`, `hive/<desc>`
- Squash merge to main (linear history)
- PRs required for develop and main. No direct push.

## Key Documentation

- **START HERE** → `gh issue list`, `gh pr list`, `gh project item-list 5 --owner LookNoHandsMom`
- `CONTEXT.md` — project classification, tech stack summary, key document index (for cross-tool sync)
- `AGENTS.md` — universal grounding document shared across all AI tools (Cursor, Antigravity, etc.)
- `docs/architecture/` — 8 ADRs, system architecture, 22-step implementation blueprint, OpenAPI spec
- `docs/knowledge-base/` — technology deep-dives (vCache, LiteLLM, Rust relay, MCP, etc.)
- `docs/roadmap/` — 3-layer synchronized roadmap (executive, detailed, full PM)
- `deferred/` — blocked items with unblock triggers (format: `[P0-P3]-[short-id].md`)
- `.hive/` — agent identity, soul, config, capabilities, constraints, evolution log

## ADR Quick Reference

| ADR | Decision |
|-----|----------|
| 001 | Three-tier hybrid router (kNN + ModernBERT + Qwen-0.5B) |
| 002 | SUPERSEDED by ADR-010: custom Rust inference engine (LLMHosts Core) replaces Ollama dependency |
| 003 | LiteLLM + Anthropic dialect API proxy |
| 004 | Phase 2-3 SUPERSEDED by ADR-009: self-hosted Rust relay (primary), Tailscale/Cloudflare (fallback) |
| 005 | Next.js + shadcn/ui frontend |
| 006 | Model Context Protocol (MCP) for tools |
| 007 | Docker Compose (dev) + K3s (scale) |
| 008 | Tiered pip install (core ~50MB, smart ~150MB, full ~2GB) |
| 009 | Self-hosted Rust relay tunnel (WSS + yamux + Noise NK), ships in pip wheel via maturin |
| 010 | LLMHosts Core custom Rust inference engine (CubeCL + cudarc), PagedAttention, continuous batching, phase-split |
