# Research: NAT Traversal & Secure Tunneling for Home GPU Inference

**Date**: 2026-02-14
**Purpose**: Evaluate the simplest, most reliable way to make "I have a 4090 at home → everything uses MY GPU from anywhere" work.
**Core Question**: What's the best NAT traversal + secure tunneling approach a solopreneur could operate?

---

## 1. The Problem Space

The user installs LLMHosts on their home machine with a GPU. They want:
1. **Local use**: `localhost:4000` OpenAI-compatible API (solved by Phase 1 proxy)
2. **Remote personal use**: From a coffee shop, their Cursor IDE hits their home GPU
3. **Multi-device**: Phone, laptop, work machine -- all route to home GPU
4. **GPU pooling**: Friend's 4090 joins the pool (Phase 3 "Hive Mode")
5. **Cloud fallback**: If home GPU is overloaded/offline, seamlessly fall back to cloud APIs

The hard part is #2-4: NAT traversal. Most home networks are behind CGNAT (RFC 6598), meaning no port forwarding possible. Even without CGNAT, exposing Ollama directly is a security disaster.

---

## 2. Tunneling Solutions: The Full Landscape

### Tier 1: Managed Services (Zero Config)

#### Tailscale (+ Tailscale Funnel)

| Attribute | Details |
|-----------|---------|
| **What it does** | WireGuard-based mesh VPN. Devices appear on same LAN. Funnel exposes to public internet. |
| **Setup complexity** | **1/5** -- `curl install.sh`, `tailscale up`, done |
| **Latency added** | 10-80ms (direct P2P), higher via DERP relay |
| **Throughput** | ~400 Mbps (userspace WireGuard) |
| **Free tier** | 3 users, 100 devices, Funnel included |
| **Auth** | Identity-based ACLs, SSO, MFA |
| **Self-hostable?** | Via Headscale (open-source control server) |
| **NAT traversal** | UDP hole punching + DERP relay fallback |
| **Best for LLMHosts** | Personal remote access (user's own devices) |

**Key insight**: Tailscale is the **default recommendation** for "access your home AI from anywhere." The Tailscale blog literally has a guide called "Self-host a local AI stack and access it from anywhere." Multiple community guides exist for Tailscale + Ollama + Open WebUI.

**Funnel specifically**: `tailscale funnel 4000` exposes LLMHosts's proxy to the public internet with automatic TLS. One command. No DNS config, no certificates, no VPS. But: traffic routes through Tailscale's relay (not P2P), adding latency. Good for demos and external tool access, not ideal for high-throughput inference.

**For LLMHosts's use case**:
- **Personal multi-device**: Perfect. Install Tailscale on all your devices, access `http://100.x.y.z:4000` from anywhere.
- **External tools (Cursor, etc.)**: Funnel gives a public HTTPS URL. Or configure Tailscale on the machine running Cursor.
- **GPU pooling with friends**: Each person installs Tailscale, share a tailnet. Works but requires trust + same Tailscale account (or Tailscale sharing features).
- **Scaling**: Free tier handles the personal use case. Gets expensive at team scale.

#### Cloudflare Tunnel

| Attribute | Details |
|-----------|---------|
| **What it does** | Outbound tunnel to Cloudflare edge. Public HTTPS endpoint. |
| **Setup complexity** | **2/5** -- install `cloudflared`, authenticate, create tunnel, configure DNS |
| **Latency added** | 15-45ms (to nearest Cloudflare PoP) |
| **Throughput** | ~200 Mbps (HTTP/2 encapsulation), free tier: 200 concurrent requests |
| **Free tier** | Unlimited tunnels, no bandwidth cap (but ToS limits non-HTML content) |
| **Auth** | Cloudflare Access (Zero Trust policies, SSO, service tokens) |
| **Self-hostable?** | No -- requires Cloudflare infrastructure |
| **NAT traversal** | Outbound connection, works everywhere |
| **Best for LLMHosts** | Public API endpoint with auth |

**Key insight**: Someone already wrote a Medium article "Access Your Local LLM via Cloudflare Tunnels" for exactly this use case. Works well for API access.

**CRITICAL WARNING**: Cloudflare ToS Section 2.8 prohibits serving "disproportionate non-HTML content" on free plans. LLM inference responses are text/JSON (not video), so this is likely fine. But streaming large responses could be borderline. The 200 concurrent request limit on Quick Tunnels is also restrictive.

**For LLMHosts's use case**:
- **Personal API access**: Works great. `https://llm.yourdomain.com` → home GPU.
- **Auth**: Cloudflare Access provides excellent zero-trust auth (service tokens for Cursor, SSO for web UI).
- **Throughput**: 200 Mbps is fine for LLM inference (text generation, not video).
- **Reliability**: Cloudflare's global edge network = very reliable.
- **Downside**: Requires a domain on Cloudflare. Not self-hostable. Vendor lock-in. ToS risk.

#### ngrok

| Attribute | Details |
|-----------|---------|
| **What it does** | Instant public URL for local services |
| **Setup complexity** | **1/5** -- `ngrok http 4000` |
| **Latency added** | Variable (relay-based) |
| **Throughput** | Limited on free tier |
| **Free tier** | 1 agent, random URLs, rate limits |
| **Auth** | Basic auth, OAuth, IP restrictions (paid) |
| **Self-hostable?** | No |
| **Best for LLMHosts** | Quick demos only |

**Verdict**: Too expensive and limited for production use. Good for demos.

---

### Tier 2: Self-Hosted (Full Control)

#### Headscale (Self-Hosted Tailscale)

| Attribute | Details |
|-----------|---------|
| **What it does** | Open-source Tailscale control server. Uses standard Tailscale clients. |
| **Setup complexity** | **3/5** -- needs VPS, Docker, DNS, HTTPS cert |
| **Latency** | Same as Tailscale (WireGuard P2P) |
| **Throughput** | Same as Tailscale (~400 Mbps userspace) |
| **Cost** | VPS only (~$5/mo for a small instance) |
| **Auth** | OIDC, pre-auth keys |
| **Self-hostable?** | Yes -- that's the whole point |
| **NAT traversal** | Same as Tailscale (needs DERP relay config) |
| **GitHub stars** | 35,238 (very active, v0.28.0 Feb 2026) |

**Key insight**: Headscale is the current plan in LLMHosts's architecture (ADR-004). It works, but adds operational complexity that may not be necessary for Phase 1-2.

**For LLMHosts's use case**:
- **GPU pooling**: Best option for multi-user mesh without trusting a third party.
- **Operational burden**: User needs a VPS, or LLMHosts provides a coordination server.
- **Recommendation**: Defer to Phase 3 (Hive Mode). Use Tailscale/Cloudflare for Phase 2.

#### FRP (Fast Reverse Proxy)

| Attribute | Details |
|-----------|---------|
| **What it does** | Layer 4 reverse proxy. Forward TCP/UDP blindly through a VPS. |
| **Setup complexity** | **3/5** -- needs VPS, config on both ends |
| **Latency** | VPS hop only (minimal) |
| **Throughput** | **Line speed** (limited only by VPS bandwidth) |
| **Cost** | VPS only |
| **Auth** | Token-based, TLS optional |
| **Self-hostable?** | Yes |
| **NAT traversal** | Reverse tunnel through VPS |
| **Best for** | Maximum throughput, no restrictions |

**For LLMHosts's use case**: Overkill for text inference. Best throughput of any tunnel, but requires VPS management. Good fallback option.

#### Rathole (Rust-based FRP alternative)

| Attribute | Details |
|-----------|---------|
| **What it does** | Same as FRP but written in Rust. Higher performance, lower resource use. |
| **Setup complexity** | **3/5** -- same as FRP |
| **Throughput** | Higher than FRP, more stable under load |
| **Binary size** | ~500 KiB (vs FRP's larger Go binary) |
| **Best for** | Resource-constrained environments |

**Verdict**: If we need a self-hosted tunnel, Rathole > FRP for performance. But same operational model.

#### Wireport

| Attribute | Details |
|-----------|---------|
| **What it does** | Self-hosted ingress proxy + WireGuard VPN + Caddy + CoreDNS |
| **Setup complexity** | **3/5** -- needs gateway with public IP |
| **Features** | Auto-SSL, Docker service discovery, HTTP/1/2/3, WebSocket, gRPC |
| **Best for** | Docker-native deployments |

**Verdict**: Interesting for Docker-heavy setups but relatively new and less proven.

#### Portr

| Attribute | Details |
|-----------|---------|
| **What it does** | Self-hosted ngrok alternative with admin dashboard |
| **Setup complexity** | **2/5** -- Docker Compose setup |
| **Features** | HTTP/TCP/WebSocket tunneling, request inspector, team features |
| **License** | Apache 2.0 |
| **Best for** | Teams needing tunnel management UI |

---

### Tier 3: Full Mesh VPNs (Maximum Performance)

#### Netmaker (Kernel WireGuard)

| Attribute | Details |
|-----------|---------|
| **Throughput** | **~950 Mbps** (kernel-level WireGuard) |
| **Setup complexity** | **4/5** -- Docker, wildcard DNS, MQTT broker |
| **Best for** | Wire-speed transfers, serious homelabs |

**Verdict**: Massive overkill for LLM inference. Text generation doesn't need 950 Mbps.

---

## 3. Competitive Landscape: Who's Building This?

### Direct Competitors

#### rbee (rbee.dev)
- **What**: Multi-GPU orchestration across machines, OpenAI-compatible API
- **Architecture**: Keeper (GUI) → Queen (orchestrator) → Hives (machines) → Workers (inference)
- **Setup**: SSH-based, 5-minute setup, no Kubernetes
- **Hardware**: NVIDIA, Apple Silicon, AMD ROCm, CPU
- **Status**: Beta (95% complete), open source
- **Gap**: **No remote access / tunneling**. LAN-only. This is the key differentiator for LLMHosts.
- **Threat level**: Medium. Good local orchestration but doesn't solve the "access from anywhere" problem.

#### LocalPortal (localportal.io)
- **What**: "Remote-first OS for AI compute" -- browser-based access to GPU machines
- **Architecture**: CLI install → public HTTPS URL → browser-based desktop/terminal
- **Features**: Port exposure, Jupyter, file management, terminal
- **Status**: 200+ developers, planning open source
- **Gap**: **Not an inference router**. It's remote desktop for GPU machines, not an AI API proxy.
- **Threat level**: Low. Different problem (remote desktop vs. API routing).

#### Tailscale Blog: "Self-host a local AI stack"
- **What**: Official Tailscale guide for running Ollama + Open WebUI behind Tailscale
- **Status**: This is a blog post/guide, not a product
- **Insight**: **Tailscale sees this as a use case but isn't building a product around it.**
- **Opportunity**: LLMHosts can be the product that makes this guide unnecessary.

### Adjacent Solutions

| Solution | What It Does | LLMHosts Differentiator |
|----------|-------------|----------------------|
| Ollama | Local inference server | No routing, no caching, no remote access |
| LiteLLM | OpenAI-compatible proxy | No tunneling, no discovery, no caching |
| vCache | Semantic caching | Standalone library, not integrated |
| Open WebUI | Chat interface | UI only, no API routing or tunneling |
| vLLM | High-performance serving | Production inference, complex setup, no tunneling |

**The gap in the market**: Nobody stitches together inference + routing + caching + tunneling + cloud fallback into a single `pip install`.

---

## 4. Recommended Architecture for LLMHosts

### The Layered Approach

Don't pick ONE tunneling solution. Offer a **tiered connectivity model** that matches user sophistication:

```
Layer 0: localhost (Phase 1)
  └── llmhosts serve → localhost:4000
  └── Zero config. Works immediately.
  
Layer 1: Tailnet / LAN (Phase 2, Easy Mode)  
  └── llmhosts tunnel --tailscale
  └── Uses Tailscale (managed). One command.
  └── Access from all your devices on your tailnet.
  └── Or: auto-detect if Tailscale is already running.
  
Layer 2: Public URL (Phase 2, External Tools)
  └── llmhosts tunnel --public
  └── Option A: tailscale funnel (simplest)
  └── Option B: cloudflare tunnel (best auth)
  └── Gives a public HTTPS URL for Cursor, custom apps, etc.
  └── API key auth on the LLMHosts side.
  
Layer 3: Mesh / Hive (Phase 3)
  └── llmhost hive join <invite-code>
  └── Headscale-based mesh for GPU pooling.
  └── Or: Tailscale sharing for simpler version.
  └── LLMHosts provides coordination server.
```

### Phase-by-Phase Recommendation

#### Phase 1 (Weeks 1-6): localhost only
- **Tunneling**: None needed
- **Focus**: Get the proxy, router, and caching working locally
- **User value**: `pip install llmhosts && llmhosts serve` → OpenAI-compatible API on localhost

#### Phase 2 (Weeks 7-14): Personal Remote Access
- **Primary**: Tailscale integration (`llmhosts tunnel`)
  - Detect if Tailscale is installed
  - If yes: Advertise LLMHosts service via MagicDNS
  - If no: Offer to install, or use Cloudflare Tunnel as alternative
- **Secondary**: Cloudflare Tunnel integration
  - For users who want a public URL without VPN
  - Better auth model (service tokens for tools)
- **API key auth**: Always required for remote access, regardless of tunnel
- **Complexity for user**: ONE command (`llmhosts tunnel`) to go from local to remote

#### Phase 3 (Weeks 15-24): GPU Pooling (Hive Mode)
- **Coordination**: Headscale (self-hosted by LLMHosts as a service) OR Tailscale sharing
- **Discovery**: Nodes announce available models + GPU capacity
- **Routing**: kNN router considers latency to peer when making routing decisions
- **Fallback**: Cloud API if all peers are busy/offline

### Why NOT Just Headscale From the Start?

The current ADR-004 specifies Headscale. But for Phase 2 (personal remote access), this is **over-engineered**:

| Requirement | Headscale | Tailscale (managed) | Cloudflare Tunnel |
|-------------|-----------|--------------------|--------------------|
| User installs LLMHosts + tunnels in | VPS + Docker + DNS + certs + Headscale | `tailscale funnel 4000` | `cloudflared tunnel` |
| Operational burden on user | HIGH | ZERO | LOW |
| Operational burden on LLMHosts | Must run coordination infra | None | None |
| Works behind CGNAT | Yes | Yes | Yes |
| Auth | Manual config | Built-in ACLs | Cloudflare Access |
| Cost to user | VPS ($5/mo) | Free (100 devices) | Free (with domain) |

**Recommendation**: Use managed Tailscale/Cloudflare for Phase 2 personal access. Reserve Headscale for Phase 3 Hive Mode where LLMHosts needs to run its own coordination plane for multi-user GPU pooling.

---

## 5. The Simplest Viable Path

For a solopreneur building LLMHosts, here's the minimum viable tunneling:

### Step 1: Ship Phase 1 with zero tunneling
- `llmhosts serve` on localhost:4000
- OpenAI-compatible, works with any local tool

### Step 2: Add Tailscale detection (2-3 days of work)
```python
# Pseudocode for llmhosts tunnel
def setup_tunnel():
    if tailscale_installed():
        # Already on tailnet, just advertise the service
        print(f"LLMHosts available at http://{tailscale_ip()}:4000")
        print(f"Or run: tailscale funnel 4000 (for public URL)")
    elif cloudflared_installed():
        # Use Cloudflare Tunnel
        run("cloudflared tunnel --url http://localhost:4000")
    else:
        print("Install Tailscale (recommended) or Cloudflare Tunnel")
        print("  tailscale: https://tailscale.com/download")
        print("  cloudflare: https://developers.cloudflare.com/cloudflare-one/")
```

### Step 3: Built-in tunnel management (Phase 2 proper)
- `llmhosts tunnel start` -- auto-selects best available method
- `llmhosts tunnel status` -- shows connection info
- `llmhosts tunnel stop` -- tears down tunnel
- API key management: `llmhosts keys create --name "cursor-ide"`

### Step 4: Hive Mode (Phase 3)
- `llmhosts hive create` -- starts a Headscale-based mesh
- `llmhosts hive join <code>` -- joins a friend's mesh
- LLMHosts runs a lightweight coordination service

---

## 6. Operational Complexity Ratings (Solopreneur Perspective)

| Solution | Setup Time | Ongoing Maintenance | Things That Can Break | Rating |
|----------|-----------|--------------------|-----------------------|--------|
| **Tailscale (managed)** | 5 min | None | Tailscale outage (rare) | **Easy** |
| **Cloudflare Tunnel** | 15 min | Certificate rotation (auto) | Cloudflare policy changes, ToS | **Easy** |
| **Tailscale Funnel** | 1 min | None | Same as Tailscale | **Trivial** |
| **FRP/Rathole** | 30 min | VPS patching, monitoring | VPS goes down, config drift | **Medium** |
| **Headscale** | 1-2 hours | VPS, Docker, certs, DNS, DERP relay | Many moving parts | **Hard** |
| **Wireport** | 30 min | Similar to FRP | Newer project, less community | **Medium** |
| **Netmaker** | 2-3 hours | Docker, MQTT, DNS | Complex stack | **Hard** |

---

## 7. Key Findings

### Finding 1: The Market is Wide Open
Nobody has built "install one thing → your GPU is an AI server accessible from anywhere with cloud fallback." rbee does multi-GPU orchestration but LAN-only. LocalPortal does remote desktop but not AI routing. Tailscale has a blog post but no product. **This is LLMHosts's opportunity.**

### Finding 2: Tailscale Is the Default Answer
Every "how do I access my home AI remotely" guide in 2025-2026 says "use Tailscale." Multiple blog posts, community guides, and even Tailscale's own blog cover this exact use case. LLMHosts should **embrace this** rather than fight it -- integrate with Tailscale rather than replacing it.

### Finding 3: Cloudflare Tunnel Is the Best "Public URL" Option
For the specific case of "I need my Cursor IDE at the coffee shop to hit my home GPU via a public URL," Cloudflare Tunnel with API key auth is the cleanest solution. Free, reliable, good auth layer.

### Finding 4: Headscale Is Right for Phase 3, Wrong for Phase 2
Headscale makes sense when LLMHosts needs to run its own coordination plane (GPU pooling, multi-user mesh). It's overkill for personal remote access where Tailscale/Cloudflare already work perfectly.

### Finding 5: 400 Mbps Is More Than Enough
LLM inference generates text at maybe 50-100 tokens/second. Even a long response is kilobytes, not megabytes. The throughput "problem" with userspace WireGuard (Tailscale's ~400 Mbps) is completely irrelevant for this use case. Don't optimize for throughput -- optimize for **latency** and **reliability**.

### Finding 6: The "One Command" Matters More Than the Technology
The technology choice (Tailscale vs Cloudflare vs Headscale) matters less than the UX. The user should type `llmhosts tunnel` and have it work. LLMHosts's value is making the right choice automatically and hiding the complexity.

---

## 8. Revised ADR-004 Recommendation

**Current ADR-004**: Headscale for all networking (Phase 2+)

**Proposed revision**:
- **Phase 2**: Tailscale (managed) as primary, Cloudflare Tunnel as secondary. LLMHosts detects what's available and configures automatically. No infrastructure for LLMHosts to operate.
- **Phase 3**: Headscale for Hive Mode coordination. LLMHosts operates a lightweight coordination server. Users who don't want Hive Mode continue using Tailscale/Cloudflare.
- **Rationale**: Reduces operational burden from "run a VPS with Headscale" to "type one command." Dramatically lowers barrier to the "access from anywhere" value prop.

---

## 9. The Pitch, Refined

> "I have a 4090 at home. I run `pip install llmhosts && llmhosts serve`. Now localhost:4000 is an OpenAI-compatible API using my GPU.
> 
> I type `llmhosts tunnel`. Now my Cursor IDE at the coffee shop, my phone, my work laptop -- everything uses MY GPU at home. Encrypted. Automatic. Zero config.
> 
> My GPU is overloaded? LLMHosts seamlessly routes to GPT-4o and tells me exactly what it cost.
> 
> My friend has a 4090 too? `llmhosts hive join`. Now we share a pool of 48GB of VRAM. For free."

This is the product. The tunneling technology is invisible.
