# Core-First Wheel Packaging — Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Ship LLMHosts as a single `pip install llmhosts` wheel with pre-compiled Rust extensions, GPU-first inference, no Ollama dependency — ready for Alienware dogfooding.

**Architecture:** Maturin replaces Hatchling. Bridge crate `_llmhosts` unifies 4 Rust crates. CubeCL provides GPU kernels. Core-first pipeline replaces Ollama-based flow.

**Tech Stack:** Rust (pyo3 0.28, maturin 1.8+, CubeCL), Python 3.10+ (FastAPI, Click), HuggingFace Hub

**Design doc:** `docs/plans/2026-03-09-core-first-wheel-packaging-design.md`

---

## Phase A: Packaging Foundation

### Task 1: Fix pyo3 version mismatch in router_core

`router_core` uses pyo3 0.25 while all other crates use 0.28. All crates must match for the bridge crate to compile.

**Files:**
- Modify: `router_core/Cargo.toml:11`

**Step 1: Check current version**

```bash
grep "pyo3" router_core/Cargo.toml
```
Expected: `pyo3 = "0.25"` or similar

**Step 2: Update to workspace dependency**

In `router_core/Cargo.toml`, change the pyo3 dependency to use the workspace version:

```toml
[dependencies]
pyo3 = { workspace = true }
```

If `router_core` uses any pyo3 APIs that changed between 0.25 and 0.28, fix compilation errors. Key changes:
- `#[pyfunction]` syntax unchanged
- `PyModule` bound references: `&PyModule` → `&Bound<'_, PyModule>` (0.28 uses Bound API)
- `#[pymodule]` function signature: `fn name(m: &Bound<'_, PyModule>)` in 0.28

**Step 3: Verify compilation**

```bash
cd /home/dev/projects/LLMHosts.com && cargo build --workspace
```
Expected: Compiles with zero errors

**Step 4: Run tests**

```bash
cargo test --workspace
```
Expected: All tests pass

**Step 5: Commit**

```bash
git add router_core/Cargo.toml router_core/src/
git commit -m "fix(router): upgrade pyo3 0.25 → 0.28 for workspace alignment"
```

---

### Task 2: Add `register_module` functions to each Rust crate

Each crate currently has a `#[pymodule]` function that registers directly as a standalone Python module. We need to add a `register_module` public function that the bridge crate can call to register as a submodule.

**Files:**
- Modify: `llmhosts_core/src/lib.rs`
- Modify: `llmhosts_gpu/src/lib.rs`
- Modify: `relay_core/src/lib.rs`
- Modify: `router_core/src/lib.rs`

**Step 1: Add `register_module` to `llmhosts_core/src/lib.rs`**

Keep the existing `#[pymodule] fn _llmhosts_core` for standalone `maturin develop` usage. Add a public function for the bridge:

```rust
/// Register llmhosts_core classes/functions into a parent module.
pub fn register_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add("LLMHOSTS_CORE_VERSION", LLMHOSTS_CORE_VERSION)?;
    m.add_class::<LLMHostsEngine>()?;
    m.add_function(wrap_pyfunction!(generate_test_model, m)?)?;
    Ok(())
}
```

The existing `_llmhosts_core` pymodule function should call `register_module` internally to avoid duplication:

```rust
#[pymodule]
fn _llmhosts_core(m: &Bound<'_, PyModule>) -> PyResult<()> {
    register_module(m)
}
```

**Step 2: Repeat for `llmhosts_gpu/src/lib.rs`**

```rust
pub fn register_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add("LLMHOSTS_GPU_VERSION", LLMHOSTS_GPU_VERSION)?;
    m.add_function(wrap_pyfunction!(detect_gpus, m)?)?;
    Ok(())
}

#[pymodule]
fn _llmhosts_gpu(m: &Bound<'_, PyModule>) -> PyResult<()> {
    register_module(m)
}
```

**Step 3: Repeat for `relay_core/src/lib.rs`**

```rust
pub fn register_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add("RELAY_VERSION", RELAY_VERSION)?;
    m.add_class::<RelayClient>()?;
    Ok(())
}

#[pymodule]
fn _relay_core(m: &Bound<'_, PyModule>) -> PyResult<()> {
    register_module(m)
}
```

**Step 4: Repeat for `router_core/src/lib.rs`**

```rust
pub fn register_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add("RUST_VERSION", RUST_VERSION)?;
    m.add_function(wrap_pyfunction!(knn_search, m)?)?;
    m.add_function(wrap_pyfunction!(extract_features, m)?)?;
    m.add_function(wrap_pyfunction!(classify_privacy, m)?)?;
    Ok(())
}

#[pymodule]
fn _router_core(m: &Bound<'_, PyModule>) -> PyResult<()> {
    register_module(m)
}
```

**Step 5: Verify all compile and tests pass**

```bash
cargo build --workspace && cargo test --workspace
```

**Step 6: Commit**

```bash
git add llmhosts_core/src/lib.rs llmhosts_gpu/src/lib.rs relay_core/src/lib.rs router_core/src/lib.rs
git commit -m "refactor(core): add register_module to all Rust crates for bridge integration"
```

---

### Task 3: Create the bridge crate

New `src/lib.rs` at project root that re-exports all 4 crates as submodules of `_llmhosts`.

**Files:**
- Create: `src/lib.rs`
- Modify: `Cargo.toml` (root — add `[package]` and `[lib]` sections alongside `[workspace]`)

**Step 1: Update root `Cargo.toml`**

The root `Cargo.toml` currently only has `[workspace]`. Add package metadata for the bridge crate:

```toml
[package]
name = "llmhosts"
version = "0.5.0"
edition = "2021"
publish = false

[lib]
name = "_llmhosts"
crate-type = ["cdylib", "rlib"]

[dependencies]
pyo3 = { workspace = true }
llmhosts_core = { path = "llmhosts_core" }
llmhosts_gpu = { path = "llmhosts_gpu" }
relay_core = { path = "relay_core" }
router_core = { path = "router_core" }

[workspace]
members = ["llmhosts_core", "llmhosts_gpu", "relay_core", "router_core"]

[workspace.dependencies]
pyo3 = { version = "0.28", features = ["extension-module"] }
```

**Step 2: Create `src/lib.rs`**

```rust
use pyo3::prelude::*;

/// Unified LLMHosts Python extension module.
///
/// Re-exports all Rust crates as submodules:
///   _llmhosts.core   — inference engine (GGUF, transformer, generation)
///   _llmhosts.gpu    — GPU detection (NVIDIA, AMD, Apple, Intel)
///   _llmhosts.relay  — tunnel client (WebSocket relay)
///   _llmhosts.router — kNN router + privacy classification
#[pymodule]
fn _llmhosts(m: &Bound<'_, PyModule>) -> PyResult<()> {
    // Core inference engine
    let core = PyModule::new(m.py(), "core")?;
    llmhosts_core::register_module(&core)?;
    m.add_submodule(&core)?;

    // GPU detection
    let gpu = PyModule::new(m.py(), "gpu")?;
    llmhosts_gpu::register_module(&gpu)?;
    m.add_submodule(&gpu)?;

    // Relay tunnel client
    let relay = PyModule::new(m.py(), "relay")?;
    relay_core::register_module(&relay)?;
    m.add_submodule(&relay)?;

    // kNN router + privacy
    let router = PyModule::new(m.py(), "router")?;
    router_core::register_module(&router)?;
    m.add_submodule(&router)?;

    // Top-level version
    m.add("__version__", env!("CARGO_PKG_VERSION"))?;

    Ok(())
}
```

**Step 3: Verify compilation**

```bash
cargo build --workspace
```
Expected: Bridge crate compiles, produces `lib_llmhosts.so` (or `.dll` on Windows)

**Step 4: Test with maturin develop**

```bash
maturin develop --release
python -c "from _llmhosts import core, gpu, relay, router; print('Bridge OK:', core.LLMHOSTS_CORE_VERSION)"
```
Expected: Prints version string

**Step 5: Commit**

```bash
git add src/lib.rs Cargo.toml
git commit -m "feat(bridge): unified _llmhosts extension module re-exporting all Rust crates"
```

---

### Task 4: Migrate build backend from Hatchling to maturin

Replace Hatchling with maturin in `pyproject.toml`. This is the core packaging change.

**Files:**
- Modify: `pyproject.toml` (root)

**Step 1: Update `[build-system]`**

Change lines 1-3 from:
```toml
[build-system]
requires = ["hatchling>=1.26"]
build-backend = "hatchling.build"
```
To:
```toml
[build-system]
requires = ["maturin>=1.8"]
build-backend = "maturin"
```

**Step 2: Replace `[tool.hatch.*]` sections with `[tool.maturin]`**

Remove:
```toml
[tool.hatch.version]
path = "llmhosts/__init__.py"

[tool.hatch.build.targets.wheel]
packages = ["llmhosts", "hive"]
```

Add:
```toml
[tool.maturin]
features = ["pyo3/extension-module"]
module-name = "_llmhosts"
python-packages = ["llmhosts", "hive"]
manifest-path = "Cargo.toml"
```

**Step 3: Update version to be dynamic via Cargo.toml**

In `[project]`, ensure version is dynamic:
```toml
[project]
name = "llmhosts"
dynamic = ["version"]
```

Or set it explicitly to match `Cargo.toml`:
```toml
version = "0.5.0"
```

**Step 4: Move `hatchling` from `[build-system].requires` to `[project.optional-dependencies].dev`**

The dev extras already include maturin. Remove hatchling from build requires (already done in step 1). Keep it in dev deps if needed for any tooling.

**Step 5: Test the build**

```bash
maturin develop --release
python -c "import llmhosts; print(llmhosts.__version__)"
python -c "from _llmhosts import core; print(core.LLMHOSTS_CORE_VERSION)"
```
Expected: Both print version strings

**Step 6: Build a wheel**

```bash
maturin build --release --out dist/
ls -la dist/*.whl
```
Expected: Platform-specific wheel (e.g., `llmhosts-0.5.0-cp310-abi3-manylinux_2_17_x86_64.whl`)

**Step 7: Verify wheel contents**

```bash
unzip -l dist/*.whl | grep -E "\.so|\.pyd|\.py"
```
Expected: Contains `_llmhosts.cpython-310-x86_64-linux-gnu.so` (or similar) + `llmhosts/` + `hive/`

**Step 8: Run full test suite to check nothing broke**

```bash
pytest tests/ -v --tb=short
```
Expected: All tests pass (the standalone `_llmhosts_core` import in tests still works because `maturin develop` installs the bridge module)

**Step 9: Commit**

```bash
git add pyproject.toml
git commit -m "feat(build): migrate from Hatchling to maturin for Rust+Python wheel packaging"
```

---

### Task 5: Update Python imports from standalone modules to bridge submodules

All Python code that imports `_llmhosts_core`, `_llmhosts_gpu`, `_relay_core`, or `_router_core` must be updated to import from `_llmhosts.core`, `_llmhosts.gpu`, `_llmhosts.relay`, `_llmhosts.router`.

**Files:**
- Modify: `llmhosts/inference/core.py:14-21`
- Modify: all files importing `_llmhosts_gpu`
- Modify: all files importing `_relay_core`
- Modify: all files importing `_router_core`
- Modify: all test files importing these modules

**Step 1: Find all imports**

```bash
grep -rn "_llmhosts_core\|_llmhosts_gpu\|_relay_core\|_router_core" llmhosts/ hive/ tests/ --include="*.py"
```

**Step 2: Update each import with graceful fallback**

Pattern for each module (example for `llmhosts/inference/core.py`):

```python
LLMHOSTS_CORE_AVAILABLE: bool = False
_LLMHostsEngineNative: type | None = None

try:
    from _llmhosts import core as _llmhosts_core_mod
    _LLMHostsEngineNative = _llmhosts_core_mod.LLMHostsEngine
    LLMHOSTS_CORE_AVAILABLE = True
    logger.debug("LLMHosts Core engine loaded (_llmhosts.core %s)", _llmhosts_core_mod.LLMHOSTS_CORE_VERSION)
except ImportError:
    # Fallback: try standalone module (development mode with individual maturin develop)
    try:
        import _llmhosts_core
        _LLMHostsEngineNative = _llmhosts_core.LLMHostsEngine
        LLMHOSTS_CORE_AVAILABLE = True
        logger.debug("LLMHosts Core engine loaded (standalone _llmhosts_core %s)", _llmhosts_core.LLMHOSTS_CORE_VERSION)
    except ImportError:
        logger.debug("LLMHosts Core engine not available")
```

The dual-import fallback allows both:
- Production: `pip install llmhosts` → imports from `_llmhosts.core`
- Development: `cd llmhosts_core && maturin develop` → imports from `_llmhosts_core`

**Step 3: Update test files similarly**

Tests that import `_llmhosts_core` directly should use the same fallback pattern.

**Step 4: Run full test suite**

```bash
pytest tests/ -v --tb=short
```
Expected: All tests pass

**Step 5: Commit**

```bash
git add llmhosts/ hive/ tests/
git commit -m "refactor: update Python imports to use _llmhosts bridge module with fallback"
```

---

### Task 6: Update Dockerfile for maturin builds

The Dockerfile currently uses `python -m build` with Hatchling. Update it for maturin.

**Files:**
- Modify: `Dockerfile`

**Step 1: Update builder stage**

Replace:
```dockerfile
RUN pip install --no-cache-dir "hatchling>=1.26" build
RUN python -m build --wheel --outdir /build/dist
```

With:
```dockerfile
RUN pip install --no-cache-dir maturin>=1.8
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
COPY Cargo.toml Cargo.lock ./
COPY src/ ./src/
COPY llmhosts_core/ ./llmhosts_core/
COPY llmhosts_gpu/ ./llmhosts_gpu/
COPY relay_core/ ./relay_core/
COPY router_core/ ./router_core/
RUN maturin build --release --out /build/dist
```

**Step 2: Verify Docker build**

```bash
docker build -f Dockerfile -t llmhosts:test .
```
Expected: Build succeeds, wheel contains compiled Rust extensions

**Step 3: Commit**

```bash
git add Dockerfile
git commit -m "build(docker): update Dockerfile for maturin Rust compilation"
```

---

## Phase B: Fix RoPE Precision (Coherent Output)

### Task 7: Fix RoPE frequency computation precision

The garbled output is caused by f32 precision loss in RoPE frequency computation. Large `rope_theta` values (Qwen uses 1,000,000) amplify the error.

**Files:**
- Modify: `llmhosts_core/src/rope.rs:6-21`
- Test: `llmhosts_core/src/rope.rs` (existing tests)

**Step 1: Write a precision test**

Add a test in `llmhosts_core/src/rope.rs` that validates frequency precision:

```rust
#[test]
fn test_rope_frequency_precision_high_base() {
    // Qwen2.5 uses rope_theta = 1_000_000.0
    // With f32, pos * freq at high positions loses precision
    let head_dim = 64;
    let max_seq_len = 8192;
    let base = 1_000_000.0_f32;

    let (cos_table, sin_table) = precompute_freqs(head_dim, max_seq_len, base);

    // At position 8191, the lowest frequency should still be distinguishable
    let half_dim = head_dim / 2;
    let last_pos = max_seq_len - 1;
    let cos_val = cos_table[last_pos * half_dim];
    let sin_val = sin_table[last_pos * half_dim];

    // cos² + sin² must equal 1.0 (rotation property)
    let sum = cos_val * cos_val + sin_val * sin_val;
    assert!((sum - 1.0).abs() < 1e-5, "Rotation property violated: cos²+sin²={sum}");

    // Adjacent positions must produce different values
    let prev_cos = cos_table[(last_pos - 1) * half_dim];
    assert!((cos_val - prev_cos).abs() > 1e-10, "Adjacent positions indistinguishable");
}
```

**Step 2: Run test — expect it may fail with current f32 code**

```bash
cd llmhosts_core && cargo test test_rope_frequency_precision_high_base -- --nocapture
```

**Step 3: Fix: compute frequencies in f64, downcast to f32**

```rust
pub fn precompute_freqs(head_dim: usize, max_seq_len: usize, base: f32) -> (Vec<f32>, Vec<f32>) {
    let half_dim = head_dim / 2;
    let mut cos_table = vec![0.0f32; max_seq_len * half_dim];
    let mut sin_table = vec![0.0f32; max_seq_len * half_dim];

    let base_f64 = base as f64;
    let head_dim_f64 = head_dim as f64;

    for pos in 0..max_seq_len {
        for i in 0..half_dim {
            // Use f64 for the frequency computation to avoid precision loss
            let freq = 1.0_f64 / base_f64.powf(2.0 * i as f64 / head_dim_f64);
            let angle = pos as f64 * freq;
            cos_table[pos * half_dim + i] = angle.cos() as f32;
            sin_table[pos * half_dim + i] = angle.sin() as f32;
        }
    }

    (cos_table, sin_table)
}
```

**Step 4: Run ALL rope tests**

```bash
cd llmhosts_core && cargo test rope -- --nocapture
```
Expected: All pass

**Step 5: Run full Rust test suite**

```bash
cargo test --workspace
```
Expected: All pass

**Step 6: Test with real model (if available)**

```bash
maturin develop --release
python -c "
from llmhosts.inference.core import LLMHostsInferenceEngine, LLMHOSTS_CORE_AVAILABLE
if LLMHOSTS_CORE_AVAILABLE:
    e = LLMHostsInferenceEngine()
    # Load a test model if available, generate tokens, check output isn't garbled
    print('Core available, ready for real model test')
"
```

**Step 7: Commit**

```bash
git add llmhosts_core/src/rope.rs
git commit -m "fix(core): use f64 precision for RoPE frequency computation

Large rope_theta values (Qwen: 1M) caused f32 precision loss in frequency
tables, producing garbled output at longer positions. Computing in f64 and
downcasting to f32 fixes the issue while maintaining GPU-compatible f32 tensors."
```

---

## Phase C: Core-First Pipeline

### Task 8: Create model registry

Maps friendly model names to HuggingFace repos, GGUF filenames, VRAM requirements.

**Files:**
- Create: `llmhosts/models/__init__.py`
- Create: `llmhosts/models/registry.py`
- Test: `tests/test_model_registry.py`

**Step 1: Write tests**

```python
# tests/test_model_registry.py
"""Tests for model registry."""
from __future__ import annotations

import pytest

@pytest.mark.unit
class TestModelRegistry:
    def test_get_model_by_name(self) -> None:
        from llmhosts.models.registry import get_model
        model = get_model("qwen2.5:7b")
        assert model is not None
        assert model["repo"].startswith("Qwen/")
        assert model["file"].endswith(".gguf")
        assert model["min_vram_gb"] > 0

    def test_unknown_model_returns_none(self) -> None:
        from llmhosts.models.registry import get_model
        assert get_model("nonexistent:model") is None

    def test_list_models(self) -> None:
        from llmhosts.models.registry import list_models
        models = list_models()
        assert len(models) >= 5
        assert "qwen2.5:0.5b" in models
        assert "qwen2.5:7b" in models

    def test_recommend_for_vram(self) -> None:
        from llmhosts.models.registry import recommend_for_vram
        # 8 GB VRAM should recommend something in the 3-7B range
        model = recommend_for_vram(8192)
        assert model is not None
        assert model["min_vram_gb"] <= 8

    def test_recommend_for_low_vram(self) -> None:
        from llmhosts.models.registry import recommend_for_vram
        model = recommend_for_vram(1024)  # 1 GB
        assert model is not None
        assert model["min_vram_gb"] <= 1

    def test_recommend_for_cpu_only(self) -> None:
        from llmhosts.models.registry import recommend_for_vram
        model = recommend_for_vram(0)  # No GPU
        assert model is not None  # Should still recommend a small model

    def test_model_has_required_fields(self) -> None:
        from llmhosts.models.registry import get_model
        model = get_model("qwen2.5:7b")
        for field in ["repo", "file", "tokenizer_repo", "size_gb", "min_vram_gb", "architecture"]:
            assert field in model, f"Missing field: {field}"

    def test_model_path(self) -> None:
        from llmhosts.models.registry import model_path
        import os
        path = model_path("qwen2.5:7b")
        # Should be under ~/.llmhosts/models/
        assert ".llmhosts" in path or "llmhosts" in path.lower()
        assert path.endswith(".gguf")
```

**Step 2: Run tests — expect failure**

```bash
pytest tests/test_model_registry.py -v
```
Expected: ImportError

**Step 3: Implement**

Create `llmhosts/models/__init__.py` (empty) and `llmhosts/models/registry.py`:

```python
"""LLMHosts model registry — maps friendly names to GGUF files on HuggingFace."""
from __future__ import annotations

import os
from typing import Any

# Default model storage location
_MODELS_DIR = os.path.join(os.path.expanduser("~"), ".llmhosts", "models")

MODELS: dict[str, dict[str, Any]] = {
    "qwen2.5:0.5b": {
        "repo": "Qwen/Qwen2.5-0.5B-Instruct-GGUF",
        "file": "qwen2.5-0.5b-instruct-q4_k_m.gguf",
        "tokenizer_repo": "Qwen/Qwen2.5-0.5B-Instruct",
        "size_gb": 0.4,
        "min_vram_gb": 1,
        "architecture": "llama",
        "context_length": 32768,
    },
    "qwen2.5:3b": {
        "repo": "Qwen/Qwen2.5-3B-Instruct-GGUF",
        "file": "qwen2.5-3b-instruct-q4_k_m.gguf",
        "tokenizer_repo": "Qwen/Qwen2.5-3B-Instruct",
        "size_gb": 2.0,
        "min_vram_gb": 3,
        "architecture": "llama",
        "context_length": 32768,
    },
    "qwen2.5:7b": {
        "repo": "Qwen/Qwen2.5-7B-Instruct-GGUF",
        "file": "qwen2.5-7b-instruct-q4_k_m.gguf",
        "tokenizer_repo": "Qwen/Qwen2.5-7B-Instruct",
        "size_gb": 4.4,
        "min_vram_gb": 6,
        "architecture": "llama",
        "context_length": 32768,
    },
    "qwen2.5:14b": {
        "repo": "Qwen/Qwen2.5-14B-Instruct-GGUF",
        "file": "qwen2.5-14b-instruct-q4_k_m.gguf",
        "tokenizer_repo": "Qwen/Qwen2.5-14B-Instruct",
        "size_gb": 8.7,
        "min_vram_gb": 10,
        "architecture": "llama",
        "context_length": 32768,
    },
    "llama3.1:8b": {
        "repo": "bartowski/Meta-Llama-3.1-8B-Instruct-GGUF",
        "file": "Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf",
        "tokenizer_repo": "meta-llama/Llama-3.1-8B-Instruct",
        "size_gb": 4.9,
        "min_vram_gb": 6,
        "architecture": "llama",
        "context_length": 131072,
    },
    "llama3.3:70b": {
        "repo": "bartowski/Llama-3.3-70B-Instruct-GGUF",
        "file": "Llama-3.3-70B-Instruct-Q4_K_M.gguf",
        "tokenizer_repo": "meta-llama/Llama-3.3-70B-Instruct",
        "size_gb": 42.0,
        "min_vram_gb": 44,
        "architecture": "llama",
        "context_length": 131072,
    },
    "mistral:7b": {
        "repo": "TheBloke/Mistral-7B-Instruct-v0.3-GGUF",
        "file": "mistral-7b-instruct-v0.3.Q4_K_M.gguf",
        "tokenizer_repo": "mistralai/Mistral-7B-Instruct-v0.3",
        "size_gb": 4.4,
        "min_vram_gb": 6,
        "architecture": "llama",
        "context_length": 32768,
    },
    "deepseek-r1:7b": {
        "repo": "bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF",
        "file": "DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf",
        "tokenizer_repo": "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
        "size_gb": 4.7,
        "min_vram_gb": 6,
        "architecture": "llama",
        "context_length": 32768,
    },
}


def get_model(name: str) -> dict[str, Any] | None:
    """Get model info by friendly name."""
    return MODELS.get(name)


def list_models() -> list[str]:
    """List all available model names."""
    return list(MODELS.keys())


def recommend_for_vram(vram_mb: int) -> dict[str, Any] | None:
    """Recommend the largest model that fits in available VRAM.

    Returns the model dict, or the smallest model if vram_mb is 0 (CPU only).
    """
    vram_gb = vram_mb / 1024
    candidates = sorted(MODELS.items(), key=lambda x: x[1]["size_gb"], reverse=True)

    for name, info in candidates:
        if info["min_vram_gb"] <= vram_gb:
            return {**info, "name": name}

    # CPU only — return smallest model
    smallest = min(MODELS.items(), key=lambda x: x[1]["size_gb"])
    return {**smallest[1], "name": smallest[0]}


def model_path(name: str) -> str:
    """Return the expected local path for a model's GGUF file."""
    info = MODELS.get(name)
    if info is None:
        raise ValueError(f"Unknown model: {name}")
    return os.path.join(_MODELS_DIR, info["file"])


def models_dir() -> str:
    """Return the models storage directory."""
    return _MODELS_DIR
```

**Step 4: Run tests**

```bash
pytest tests/test_model_registry.py -v
```
Expected: All pass

**Step 5: Commit**

```bash
git add llmhosts/models/ tests/test_model_registry.py
git commit -m "feat(models): add model registry with HuggingFace GGUF mappings and VRAM-based selection"
```

---

### Task 9: Implement GGUF model download

Download GGUF models from HuggingFace with progress, resume, and SHA256 verification.

**Files:**
- Create: `llmhosts/models/download.py`
- Test: `tests/test_model_download.py`

**Step 1: Write tests**

```python
# tests/test_model_download.py
"""Tests for model download (mocked — no real network)."""
from __future__ import annotations

from unittest.mock import AsyncMock, patch

import pytest

@pytest.mark.unit
class TestModelDownload:
    def test_download_creates_models_dir(self, tmp_path) -> None:
        from llmhosts.models.download import ensure_models_dir
        models_dir = tmp_path / "models"
        ensure_models_dir(str(models_dir))
        assert models_dir.exists()

    def test_is_downloaded_false_when_missing(self, tmp_path) -> None:
        from llmhosts.models.download import is_downloaded
        assert is_downloaded("qwen2.5:7b", models_dir=str(tmp_path)) is False

    def test_is_downloaded_true_when_exists(self, tmp_path) -> None:
        from llmhosts.models.download import is_downloaded
        # Create a fake model file
        (tmp_path / "qwen2.5-7b-instruct-q4_k_m.gguf").write_bytes(b"fake")
        assert is_downloaded("qwen2.5:7b", models_dir=str(tmp_path)) is True

    def test_download_unknown_model_raises(self) -> None:
        from llmhosts.models.download import download_model
        with pytest.raises(ValueError, match="Unknown model"):
            download_model("nonexistent:model")

    def test_list_local_models(self, tmp_path) -> None:
        from llmhosts.models.download import list_local_models
        (tmp_path / "qwen2.5-7b-instruct-q4_k_m.gguf").write_bytes(b"fake")
        local = list_local_models(models_dir=str(tmp_path))
        assert len(local) >= 1
        assert any("qwen2.5:7b" in m for m in local)
```

**Step 2: Implement `llmhosts/models/download.py`**

```python
"""Model download from HuggingFace Hub with progress and verification."""
from __future__ import annotations

import logging
import os

from llmhosts.models.registry import MODELS, model_path, models_dir

logger = logging.getLogger(__name__)


def ensure_models_dir(path: str | None = None) -> str:
    """Create the models directory if it doesn't exist."""
    d = path or models_dir()
    os.makedirs(d, exist_ok=True)
    return d


def is_downloaded(name: str, models_dir: str | None = None) -> bool:
    """Check if a model's GGUF file exists locally."""
    info = MODELS.get(name)
    if info is None:
        return False
    d = models_dir or globals()["models_dir"]()
    return os.path.exists(os.path.join(d, info["file"]))


def list_local_models(models_dir: str | None = None) -> list[str]:
    """List model names that have been downloaded."""
    d = models_dir or globals()["models_dir"]()
    if not os.path.exists(d):
        return []
    local_files = set(os.listdir(d))
    return [name for name, info in MODELS.items() if info["file"] in local_files]


def download_model(name: str, models_dir: str | None = None) -> str:
    """Download a GGUF model from HuggingFace Hub.

    Returns the local file path.
    """
    info = MODELS.get(name)
    if info is None:
        raise ValueError(f"Unknown model: {name}")

    d = ensure_models_dir(models_dir)
    dest = os.path.join(d, info["file"])

    if os.path.exists(dest):
        logger.info("Model already downloaded: %s", dest)
        return dest

    try:
        from huggingface_hub import hf_hub_download
    except ImportError:
        raise RuntimeError(
            "huggingface_hub is required for model downloads. "
            "Install with: pip install llmhosts[smart]"
        ) from None

    logger.info("Downloading %s from %s...", info["file"], info["repo"])
    path = hf_hub_download(
        repo_id=info["repo"],
        filename=info["file"],
        local_dir=d,
        revision="main",
    )

    # Also download tokenizer
    try:
        from huggingface_hub import hf_hub_download as _dl
        _dl(
            repo_id=info["tokenizer_repo"],
            filename="tokenizer.json",
            local_dir=d,
            revision="main",
        )
        logger.info("Tokenizer downloaded for %s", name)
    except Exception as exc:
        logger.warning("Could not download tokenizer for %s: %s", name, exc)

    return path
```

**Step 3: Run tests**

```bash
pytest tests/test_model_download.py -v
```

**Step 4: Commit**

```bash
git add llmhosts/models/download.py tests/test_model_download.py
git commit -m "feat(models): GGUF model download from HuggingFace with auto-tokenizer"
```

---

### Task 10: Rewrite `llmhosts up` pipeline — Core-first

Replace the Ollama-centric Phase 1 (engine scanning) and Phase 2 (Ollama install + pull) with GPU detection and GGUF download.

**Files:**
- Modify: `llmhosts/up.py:146-300` (Phase 1 and Phase 2)

**Step 1: Rewrite Phase 1 — Detect Hardware (not engines)**

Replace the `UniversalScanner` engine probing with `_llmhosts.gpu.detect_gpus()`:

```python
async def _phase_assess(self) -> None:
    """Phase 1: Detect hardware capabilities."""
    self._report(UpPhase.ASSESS, "Scanning hardware...")

    # GPU detection via Rust extension
    gpu_info = None
    try:
        from _llmhosts import gpu
        gpu_info = gpu.detect_gpus()
        # Parse JSON result into structured data
    except ImportError:
        try:
            import _llmhosts_gpu
            gpu_info = _llmhosts_gpu.detect_gpus()
        except ImportError:
            logger.debug("GPU detection not available")

    # CPU info
    import os
    cpu_count = os.cpu_count() or 1
    # ... RAM detection

    # Model recommendation
    from llmhosts.models.registry import recommend_for_vram
    vram_mb = gpu_info.get("vram_free_mb", 0) if gpu_info else 0
    recommended = recommend_for_vram(vram_mb)

    self._report(UpPhase.ASSESS, f"GPU: {gpu_info.get('name', 'None')} ({vram_mb} MB)")
    self._report(UpPhase.ASSESS, f"Recommended: {recommended['name']}")
```

**Step 2: Rewrite Phase 2 — Download Model (not install Ollama)**

```python
async def _phase_provision(self) -> None:
    """Phase 2: Download recommended model if none present."""
    from llmhosts.models.download import download_model, list_local_models

    local = list_local_models()
    if local:
        self._report(UpPhase.PROVISION, f"Found {len(local)} model(s): {', '.join(local)}")
        return

    # No models — download recommended
    recommended = self._recommended_model  # Set in Phase 1
    self._report(UpPhase.PROVISION, f"Downloading {recommended['name']} ({recommended['size_gb']} GB)...")
    path = download_model(recommended["name"])
    self._report(UpPhase.PROVISION, f"Saved to {path}")
```

**Step 3: Update Phase 3 — Load model into Core**

```python
async def _phase_load(self) -> None:
    """Phase 3: Load model into LLMHosts Core engine."""
    from llmhosts.inference.core import LLMHostsInferenceEngine
    from llmhosts.models.download import list_local_models
    from llmhosts.models.registry import model_path

    local = list_local_models()
    if not local:
        self._report(UpPhase.LOAD, "No models available. Skipping engine load.")
        return

    model_name = local[0]  # Use first available
    path = model_path(model_name)
    engine = LLMHostsInferenceEngine()
    info = engine.load_model(path)
    self._report(UpPhase.LOAD, f"Loaded {model_name}: {info.get('architecture', 'unknown')}")
```

**Step 4: Run existing up tests**

```bash
pytest tests/test_up*.py -v
```

Existing tests mock the scanner/provisioner — they'll need updating to match the new phase logic. Update mocks to match new imports.

**Step 5: Commit**

```bash
git add llmhosts/up.py
git commit -m "feat(up): Core-first pipeline — GPU detect, GGUF download, Core engine load"
```

---

### Task 11: Rewrite dispatcher — Core is the default engine

Remove Ollama-as-primary routing. Core is always the local engine.

**Files:**
- Modify: `llmhosts/proxy/dispatcher.py:1217-1264` (dispatch routing logic)
- Modify: `llmhosts/proxy/dispatcher.py:260-267` (initialization)

**Step 1: Simplify dispatch routing**

Replace the current multi-backend selection (lines 1217-1264) with:

```python
# Inside dispatch() method, replace the backend selection block:
if self._core_engine is not None and self._core_engine.is_loaded:
    # Core IS the local engine — always use it for local inference
    messages_raw = [
        {"role": m.role, "content": m.content if isinstance(m.content, str) else str(m.content)}
        for m in request.messages
    ]
    body, extra_hdrs = await self._dispatch_core(
        messages_raw,
        model=request.model,
        max_tokens=request.max_tokens or 256,
        temperature=request.temperature or 0.0,
        top_p=request.top_p or 0.9,
        request=request.extra,
    )
    usage = body.get("usage", {})
    response = UnifiedResponse(
        content=body["choices"][0]["message"]["content"],
        finish_reason=body["choices"][0].get("finish_reason", "stop"),
        model=body.get("model", request.model),
        prompt_tokens=usage.get("prompt_tokens", 0),
        completion_tokens=usage.get("completion_tokens", 0),
        total_tokens=usage.get("total_tokens", 0),
        extra={"core_headers": extra_hdrs},
    )
elif backend.backend_type == "remote":
    response, backend, remote_headers = await self._dispatch_remote_with_retry(...)
elif backend.backend_type in ("litellm", "openai", "anthropic"):
    # Cloud BYOK fallback
    response = await self._dispatch_litellm(request)
else:
    raise BackendUnavailableError("No backend available. Run: llmhosts up")
```

**Step 2: Remove `_is_core_available()` guard**

Delete the `_is_core_available()` method. Core being loaded IS the normal state.

**Step 3: Remove `_dispatch_ollama()` method**

Delete the entire method. No more HTTP routing to localhost:11434.

**Step 4: Update tests**

```bash
pytest tests/test_core_dispatcher.py tests/test_core_e2e.py -v
```

Update any test that mocks Ollama routing.

**Step 5: Commit**

```bash
git add llmhosts/proxy/dispatcher.py tests/
git commit -m "refactor(dispatch): Core-first routing, remove Ollama dispatch path"
```

---

### Task 12: Legacy cleanup — remove Ollama-specific code

**Files:**
- Modify: `llmhosts/discovery/scanner.py` — remove OllamaProbe and other engine probes
- Modify: `llmhosts/discovery/provisioner.py` — remove `install_ollama()`
- Modify: `llmhosts/discovery/watcher.py` — replace Ollama polling with filesystem watcher on `~/.llmhosts/models/`
- Modify: `llmhosts/proxy/dispatcher.py` — remove remaining Ollama backend entries
- Update: tests that reference removed code

**Step 1: Remove engine probes from scanner.py**

Remove: OllamaProbe, VLLMProbe, LlamaCppProbe, LMStudioProbe, LocalAIProbe, TGIProbe, GenericOpenAIProbe

Keep: The `EngineProbe` base class (may be useful for marketplace remote probing) and `UniversalScanner` class (refactored to scan local models instead of engines).

**Step 2: Gut provisioner.py**

Remove `install_ollama()` and all platform-specific Ollama install logic. Replace with a thin wrapper around `llmhosts.models.download.download_model()`.

**Step 3: Replace watcher.py Ollama polling**

Replace Ollama API polling with `watchdog` or simple filesystem polling on `~/.llmhosts/models/`:

```python
async def _poll_models(self) -> None:
    """Poll ~/.llmhosts/models/ for new or removed GGUF files."""
    from llmhosts.models.download import list_local_models
    current = set(list_local_models())
    if current != self._last_models:
        added = current - self._last_models
        removed = self._last_models - current
        for m in added:
            self.on_model_added(m)
        for m in removed:
            self.on_model_removed(m)
        self._last_models = current
```

**Step 4: Fix all broken tests**

```bash
pytest tests/ -v --tb=short 2>&1 | grep FAIL
```

Fix each failure — mostly removing tests that mock Ollama probes.

**Step 5: Run full suite**

```bash
pytest tests/ -v
```
Expected: All pass, zero failures

**Step 6: Commit**

```bash
git add llmhosts/discovery/ llmhosts/proxy/dispatcher.py tests/
git commit -m "refactor: remove Ollama-specific code — Core is the engine

Remove: OllamaProbe, VLLMProbe, LlamaCppProbe, LMStudioProbe, LocalAIProbe,
TGIProbe, GenericOpenAIProbe, install_ollama(), Ollama API polling.
Replace: filesystem watcher for ~/.llmhosts/models/ directory."
```

---

### Task 13: Update `llmhosts pull` command

Replace Ollama-wrapping with direct GGUF download.

**Files:**
- Modify: `llmhosts/cli.py` (the `pull` command)

**Step 1: Rewrite pull command**

```python
@cli.command("pull")
@click.argument("model_name", required=False)
@click.option("--auto", is_flag=True, help="Pull the recommended model for your hardware")
@click.option("--list", "list_models", is_flag=True, help="List available models")
def pull(model_name: str | None, auto: bool, list_models: bool) -> None:
    """Download a GGUF model."""
    from llmhosts.models.registry import list_models as _list, recommend_for_vram
    from llmhosts.models.download import download_model, list_local_models

    if list_models:
        for name in _list():
            local = "✓" if name in list_local_models() else " "
            click.echo(f"  [{local}] {name}")
        return

    if auto:
        # Detect GPU, recommend model
        vram_mb = 0  # Default to CPU
        try:
            from _llmhosts import gpu
            info = gpu.detect_gpus()
            vram_mb = info.get("vram_free_mb", 0)
        except ImportError:
            pass
        rec = recommend_for_vram(vram_mb)
        model_name = rec["name"]
        click.echo(f"Recommended: {model_name} ({rec['size_gb']} GB)")

    if not model_name:
        click.echo("Usage: llmhosts pull <model> or llmhosts pull --auto")
        return

    click.echo(f"Downloading {model_name}...")
    path = download_model(model_name)
    click.echo(f"Saved to {path}")
```

**Step 2: Run CLI tests**

```bash
pytest tests/test_cli*.py -v
```

**Step 3: Commit**

```bash
git add llmhosts/cli.py
git commit -m "feat(cli): rewrite pull command for direct GGUF download from HuggingFace"
```

---

## Phase D: GPU Compute (CubeCL)

### Task 14: Add CubeCL dependency and ComputeBackend trait

**Files:**
- Modify: `llmhosts_core/Cargo.toml` — add `cubecl`, `cubecl-cuda`, `cubecl-wgpu` dependencies
- Create: `llmhosts_core/src/compute/mod.rs` — ComputeBackend trait
- Create: `llmhosts_core/src/compute/cpu.rs` — CPU backend (refactor existing code)
- Create: `llmhosts_core/src/compute/auto.rs` — auto-detect best backend

**Step 1: Add dependencies**

```toml
# llmhosts_core/Cargo.toml
[dependencies]
cubecl = { version = "0.4", features = ["default"] }
cubecl-cuda = { version = "0.4", optional = true }
cubecl-wgpu = { version = "0.4", optional = true }

[features]
default = []
cuda = ["cubecl-cuda"]
wgpu = ["cubecl-wgpu"]  # ROCm, Metal, Vulkan via WGPU
```

**Step 2: Define ComputeBackend trait**

```rust
// llmhosts_core/src/compute/mod.rs
pub mod cpu;
pub mod auto;
#[cfg(feature = "cuda")]
pub mod cuda;
#[cfg(feature = "wgpu")]
pub mod wgpu_backend;

use crate::tensor::Tensor;

pub trait ComputeBackend: Send + Sync {
    fn name(&self) -> &str;
    fn matmul(&self, a: &Tensor, b: &Tensor) -> Tensor;
    fn softmax(&self, x: &mut Tensor, dim: usize);
    fn rope(&self, q: &mut Tensor, k: &mut Tensor, freqs_cos: &Tensor, freqs_sin: &Tensor, start_pos: usize);
    fn rmsnorm(&self, x: &mut Tensor, weight: &Tensor, eps: f32);
    fn silu_elementwise_mul(&self, gate: &mut Tensor, up: &Tensor);
}
```

**Step 3: Refactor existing CPU code into `cpu.rs`**

Move the current tensor operations (from `attention.rs`, `ffn.rs`, `rope.rs`) into `CpuBackend` implementing `ComputeBackend`.

**Step 4: Create auto-detect**

```rust
// llmhosts_core/src/compute/auto.rs
pub fn detect_best_backend() -> Box<dyn ComputeBackend> {
    #[cfg(feature = "cuda")]
    if let Some(cuda) = cuda::CudaBackend::try_new() {
        log::info!("Using CUDA backend");
        return Box::new(cuda);
    }

    #[cfg(feature = "wgpu")]
    if let Some(wgpu) = wgpu_backend::WgpuBackend::try_new() {
        log::info!("Using WGPU backend ({})", wgpu.adapter_name());
        return Box::new(wgpu);
    }

    log::info!("Using CPU backend");
    Box::new(cpu::CpuBackend::new())
}
```

**Step 5: Tests**

```bash
cargo test --workspace
```

**Step 6: Commit**

```bash
git add llmhosts_core/
git commit -m "feat(compute): ComputeBackend trait + CPU backend + auto-detect framework"
```

---

### Task 15: Implement CubeCL CUDA matmul kernel

The most critical GPU kernel — matrix multiplication is 90% of inference compute.

**Files:**
- Create: `llmhosts_core/src/compute/cuda.rs`
- Create: `llmhosts_core/src/kernels/matmul.rs`

**Step 1: Write CubeCL matmul kernel**

```rust
// llmhosts_core/src/kernels/matmul.rs
use cubecl::prelude::*;

#[cube(launch)]
pub fn matmul_kernel<F: Float>(
    a: &Tensor<F>,
    b: &Tensor<F>,
    out: &mut Tensor<F>,
    #[comptime] m: u32,
    #[comptime] n: u32,
    #[comptime] k: u32,
) {
    let row = ABSOLUTE_POS_X;
    let col = ABSOLUTE_POS_Y;

    if row < m && col < n {
        let mut sum = F::new(0.0);
        for i in 0..k {
            sum += a[row * k + i] * b[i * n + col];
        }
        out[row * n + col] = sum;
    }
}
```

**Step 2: Implement CudaBackend**

```rust
// llmhosts_core/src/compute/cuda.rs
use cubecl_cuda::CudaRuntime;

pub struct CudaBackend {
    device: CudaDevice,
}

impl CudaBackend {
    pub fn try_new() -> Option<Self> {
        // Try to initialize CUDA runtime
        match CudaRuntime::new(0) {
            Ok(device) => Some(Self { device }),
            Err(_) => None,
        }
    }
}

impl ComputeBackend for CudaBackend {
    fn name(&self) -> &str { "cuda" }

    fn matmul(&self, a: &Tensor, b: &Tensor) -> Tensor {
        // Launch CubeCL kernel on CUDA device
        // ...
    }
    // ... other operations
}
```

**Step 3: Test with GPU (if available)**

```bash
cargo test --workspace --features cuda
```

On machines without CUDA, test falls back to CPU:
```bash
cargo test --workspace
```

**Step 4: Commit**

```bash
git add llmhosts_core/src/compute/ llmhosts_core/src/kernels/
git commit -m "feat(cuda): CubeCL CUDA matmul kernel + CudaBackend implementation"
```

---

### Task 16: Implement remaining GPU kernels

Same pattern as matmul — CubeCL kernels for softmax, RoPE, RMSNorm, SiLU, attention.

**Files:**
- Create: `llmhosts_core/src/kernels/softmax.rs`
- Create: `llmhosts_core/src/kernels/rope_gpu.rs`
- Create: `llmhosts_core/src/kernels/rmsnorm.rs`
- Create: `llmhosts_core/src/kernels/silu.rs`
- Create: `llmhosts_core/src/kernels/attention.rs`

Each kernel follows the same pattern:
1. Write CubeCL kernel (write-once, compiles to CUDA/ROCm/Metal/Vulkan)
2. Add to CudaBackend (and WgpuBackend for AMD/Apple/Intel)
3. Test against CPU reference implementation
4. Verify numerical accuracy (output within 1e-5 of CPU)

**Step 1: Implement all kernels**

(Detailed kernel code in design doc Section 3)

**Step 2: Wire into forward pass**

Update `llmhosts_core/src/model/transformer.rs` to use `ComputeBackend` trait instead of direct CPU function calls.

**Step 3: Run inference with GPU backend**

```bash
maturin develop --release --features cuda
python -c "
from llmhosts.inference.core import LLMHostsInferenceEngine
e = LLMHostsInferenceEngine()
# Load model, generate, verify coherent output on GPU
"
```

**Step 4: Commit**

```bash
git add llmhosts_core/src/
git commit -m "feat(cuda): complete CubeCL GPU kernel suite — softmax, RoPE, RMSNorm, SiLU, attention"
```

---

### Task 17: Wire GPU memory management

Allocate model weights and KV cache in VRAM. Avoid CPU↔GPU copies during inference.

**Files:**
- Create: `llmhosts_core/src/memory/allocator.rs`
- Modify: `llmhosts_core/src/model/transformer.rs` — load weights to GPU
- Modify: `llmhosts_core/src/kv_cache.rs` — allocate cache in VRAM

**Step 1: GPU memory allocator**

```rust
pub struct GpuAllocator {
    device: CudaDevice,
    allocated_bytes: usize,
    total_bytes: usize,
}

impl GpuAllocator {
    pub fn new(device: CudaDevice) -> Self {
        let total = device.memory_total();
        Self { device, allocated_bytes: 0, total_bytes: total }
    }

    pub fn allocate(&mut self, bytes: usize) -> Result<GpuBuffer, MemoryError> {
        if self.allocated_bytes + bytes > self.total_bytes {
            return Err(MemoryError::OutOfMemory { requested: bytes, available: self.total_bytes - self.allocated_bytes });
        }
        // ...
    }
}
```

**Step 2: Load weights to GPU on model load**

**Step 3: KV cache in VRAM**

**Step 4: Commit**

```bash
git commit -m "feat(memory): GPU memory manager with VRAM allocation for weights and KV cache"
```

---

## Phase E: Polish & Dogfooding

### Task 18: Implement `llmhosts uninstall` command

**Files:**
- Create: `llmhosts/uninstall.py`
- Modify: `llmhosts/cli.py` — add `uninstall` command

**Step 1: Write tests**

```python
# tests/test_uninstall.py
"""Tests for llmhosts uninstall command."""
from __future__ import annotations

import os
import pytest

@pytest.mark.unit
class TestUninstall:
    def test_removes_config_dir(self, tmp_path) -> None:
        from llmhosts.uninstall import uninstall
        config_dir = tmp_path / ".config" / "llmhosts"
        config_dir.mkdir(parents=True)
        (config_dir / "config.toml").write_text("[server]\nport = 11400\n")
        uninstall(config_dir=str(config_dir), data_dir=str(tmp_path / ".llmhosts"), models_dir=str(tmp_path / "models"), confirm=False)
        assert not config_dir.exists()

    def test_keeps_models_without_all_flag(self, tmp_path) -> None:
        from llmhosts.uninstall import uninstall
        models_dir = tmp_path / "models"
        models_dir.mkdir()
        (models_dir / "test.gguf").write_bytes(b"model data")
        uninstall(config_dir=str(tmp_path / "config"), data_dir=str(tmp_path / "data"), models_dir=str(models_dir), confirm=False, remove_all=False)
        assert models_dir.exists()
        assert (models_dir / "test.gguf").exists()

    def test_removes_models_with_all_flag(self, tmp_path) -> None:
        from llmhosts.uninstall import uninstall
        models_dir = tmp_path / "models"
        models_dir.mkdir()
        (models_dir / "test.gguf").write_bytes(b"model data")
        uninstall(config_dir=str(tmp_path / "config"), data_dir=str(tmp_path / "data"), models_dir=str(models_dir), confirm=False, remove_all=True)
        assert not (models_dir / "test.gguf").exists()
```

**Step 2: Implement `llmhosts/uninstall.py`**

```python
"""LLMHosts uninstall — clean removal of config, cache, and optionally models."""
from __future__ import annotations

import logging
import os
import shutil

logger = logging.getLogger(__name__)

_DEFAULT_CONFIG_DIR = os.path.join(os.path.expanduser("~"), ".config", "llmhosts")
_DEFAULT_DATA_DIR = os.path.join(os.path.expanduser("~"), ".llmhosts")
_DEFAULT_MODELS_DIR = os.path.join(_DEFAULT_DATA_DIR, "models")

# Windows: use APPDATA
if os.name == "nt":
    _appdata = os.environ.get("APPDATA", os.path.expanduser("~"))
    _DEFAULT_CONFIG_DIR = os.path.join(_appdata, "llmhosts")
    _DEFAULT_DATA_DIR = os.path.join(_appdata, "llmhosts", "data")
    _DEFAULT_MODELS_DIR = os.path.join(_appdata, "llmhosts", "models")


def uninstall(
    config_dir: str = _DEFAULT_CONFIG_DIR,
    data_dir: str = _DEFAULT_DATA_DIR,
    models_dir: str = _DEFAULT_MODELS_DIR,
    confirm: bool = True,
    remove_all: bool = False,
) -> dict[str, bool]:
    """Remove LLMHosts data directories.

    Returns dict of what was removed.
    """
    removed: dict[str, bool] = {}

    for label, path in [("config", config_dir), ("data", data_dir)]:
        if os.path.exists(path):
            shutil.rmtree(path)
            removed[label] = True
            logger.info("Removed %s: %s", label, path)
        else:
            removed[label] = False

    if remove_all and os.path.exists(models_dir):
        shutil.rmtree(models_dir)
        removed["models"] = True
        logger.info("Removed models: %s", models_dir)
    else:
        removed["models"] = False

    return removed
```

**Step 3: Add CLI command**

```python
@cli.command("uninstall")
@click.option("--all", "remove_all", is_flag=True, help="Also remove downloaded models")
@click.option("--yes", "-y", is_flag=True, help="Skip confirmation prompt")
def uninstall_cmd(remove_all: bool, yes: bool) -> None:
    """Remove LLMHosts config, cache, and optionally models."""
    from llmhosts.uninstall import uninstall, _DEFAULT_CONFIG_DIR, _DEFAULT_DATA_DIR, _DEFAULT_MODELS_DIR

    click.echo("This will remove:")
    click.echo(f"  ✓ Config:  {_DEFAULT_CONFIG_DIR}")
    click.echo(f"  ✓ Data:    {_DEFAULT_DATA_DIR}")
    if remove_all:
        click.echo(f"  ✓ Models:  {_DEFAULT_MODELS_DIR}")
    else:
        click.echo(f"  ✗ Models:  {_DEFAULT_MODELS_DIR} (use --all to remove)")
    click.echo(f"\nTo remove the package: pip uninstall llmhosts")

    if not yes:
        if not click.confirm("\nContinue?"):
            click.echo("Cancelled.")
            return

    result = uninstall(confirm=False, remove_all=remove_all)
    for what, removed in result.items():
        if removed:
            click.echo(f"  Removed: {what}")
    click.echo("Done.")
```

**Step 4: Run tests**

```bash
pytest tests/test_uninstall.py -v
```

**Step 5: Commit**

```bash
git add llmhosts/uninstall.py llmhosts/cli.py tests/test_uninstall.py
git commit -m "feat(cli): add llmhosts uninstall command with two-tier cleanup"
```

---

### Task 19: Update CI/CD release workflow for maturin

**Files:**
- Modify: `.github/workflows/release.yml`

**Step 1: Replace `python -m build` with maturin-action**

Add platform matrix and `PyO3/maturin-action@v1`. See design doc Section 11 for full workflow YAML.

**Step 2: Test locally**

```bash
maturin build --release --out dist/
ls -la dist/*.whl
```

**Step 3: Commit**

```bash
git add .github/workflows/release.yml
git commit -m "ci(release): maturin-action platform matrix for Linux + Windows wheels"
```

---

### Task 20: Full verification — all checks clean

**Step 1: Run every check**

```bash
# Rust
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace

# Python
ruff check .
ruff format --check .
mypy llmhosts/ --ignore-missing-imports
pytest tests/ -v --tb=short

# Build
maturin build --release --out dist/
```

**Step 2: Verify wheel**

```bash
pip install dist/*.whl --force-reinstall
python -c "from _llmhosts import core, gpu, relay, router; print('All modules loaded')"
llmhosts --version
llmhosts doctor
```

**Step 3: Zero warnings, zero errors, zero failures**

If anything fails, fix it before proceeding.

**Step 4: Push and create PR**

```bash
git push -u origin feature/core-first-wheel
gh pr create --title "feat: Core-first wheel packaging with GPU compute" --body "..."
```

---

## Task Dependency Graph

```
Task 1 (pyo3 fix) ─┐
Task 2 (register)  ─┼─→ Task 3 (bridge) → Task 4 (maturin) → Task 5 (imports) → Task 6 (Docker)
                    │
Task 7 (RoPE fix) ─┘

Task 8 (registry) → Task 9 (download) → Task 10 (up rewrite) ─┐
                                                                ├─→ Task 12 (legacy cleanup) → Task 13 (pull cmd)
Task 11 (dispatcher) ──────────────────────────────────────────┘

Task 14 (CubeCL trait) → Task 15 (matmul) → Task 16 (all kernels) → Task 17 (GPU memory)

Task 18 (uninstall) — independent
Task 19 (CI/CD) — independent
Task 20 (verification) — depends on all above
```

---

## Success Criteria

All of these must be true before declaring done:

- [ ] `pip install dist/*.whl` installs LLMHosts with compiled Rust extensions
- [ ] `python -c "from _llmhosts import core, gpu, relay, router"` succeeds
- [ ] `llmhosts up` detects GPU, downloads model, loads engine, starts proxy
- [ ] `curl localhost:11400/v1/chat/completions` returns coherent response
- [ ] No Ollama dependency anywhere in the codebase
- [ ] `llmhosts uninstall` removes config/data, `--all` removes models
- [ ] `cargo clippy --workspace` — zero warnings
- [ ] `cargo test --workspace` — all pass
- [ ] `ruff check . && ruff format --check .` — clean
- [ ] `mypy llmhosts/ --ignore-missing-imports` — clean
- [ ] `pytest tests/ -v` — all pass, zero failures
- [ ] CI/CD release workflow builds Linux + Windows wheels
- [ ] Wheel size <50 MB
- [ ] All 14 dogfood tests pass on Alienware (design doc Section 14)
