# NAT Traversal & Firewall Solutions -- LLMHosts.com

**NOTE (2026-02-14 COURSE CORRECTION)**: This document was written before ADR-004 was REVISED. The current architecture uses: Phase 1 localhost only, Phase 2 `llmhosts tunnel` (auto-detects Tailscale or Cloudflare Tunnel for personal remote access), Phase 3 Headscale for Hive Mode coordination only. OpenZiti is deferred to Phase 4+ enterprise. See ADR-004 for current decision.

## META
- relevance: CRITICAL -- required for home PCs behind firewalls to participate in compute fabric
- related_adr: ADR-004 (Security Overlay)
- created: 2026-02-13

## THE PROBLEM

Most home PCs sit behind NAT (Network Address Translation) and firewalls:
- Home routers use NAT: devices have private IPs (192.168.x.x), not reachable from internet
- ISPs increasingly use CGNAT (Carrier-Grade NAT): multiple customers share one public IP
- Corporate VPNs add additional network layers
- Users should NEVER need to configure port forwarding, firewall rules, or DNS

For LLMHosts's "10 minutes to first inference" goal, the node agent must connect back to the platform and to other nodes with ZERO network configuration.

## SOLUTIONS EVALUATED

### Option A: OpenZiti (SELECTED in ADR-004)
- **How it works**: Software-defined overlay network with embedded SDK
- **NAT traversal**: Built-in. Nodes connect outbound to OpenZiti fabric (standard HTTPS port)
- **No port forwarding**: Nodes initiate outbound connections; overlay routes traffic through fabric
- **Dark services**: Compute nodes completely invisible to internet
- **Security**: mTLS + E2E encryption via libsodium, cryptographic identity
- **P2P when possible**: Direct node-to-node via UDP hole punching when NAT allows
- **Fallback**: Relay through OpenZiti routers when direct connection impossible
- **Deployment**: Controller + routers hosted by LLMHosts; SDK embedded in node agent
- **Cost**: Open source (Apache 2.0), self-hosted controller

### Option B: Tailscale / Headscale
- **How it works**: WireGuard-based mesh VPN with coordination server
- **NAT traversal**: Excellent. Uses DERP relay servers, STUN, UDP hole punching
- **Strengths**: Very well-tested, 99.9%+ connection success rate, minimal latency
- **Weaknesses**: IP-based (not identity-based), separate from application layer
- **Headscale**: Self-hosted open-source control server (avoids Tailscale dependency)
- **Cost**: Headscale is free; Tailscale paid for >3 users in mesh

### Option C: Nebula (Slack/Defined Networking)
- **How it works**: Overlay network using Noise protocol
- **NAT traversal**: Good. Uses lighthouse nodes for discovery, UDP hole punching
- **Strengths**: Simple, self-hosted, lightweight
- **Weaknesses**: Less mature than Tailscale, fewer features
- **Cost**: Open source

### Option D: FRP (Fast Reverse Proxy)
- **How it works**: Client behind NAT connects to public server, server proxies traffic
- **NAT traversal**: Guaranteed (all traffic relayed through public server)
- **Strengths**: Simple, works everywhere
- **Weaknesses**: All traffic goes through relay (latency, bandwidth bottleneck), not P2P
- **Cost**: Open source, but requires public server

### Option E: connet
- **How it works**: P2P reverse proxy with NAT traversal, built in Go
- **NAT traversal**: Direct P2P when possible, relay fallback
- **Strengths**: Modern design, TLS + token auth, QUIC protocol
- **Weaknesses**: Alpha status (not production-ready as of Feb 2026)
- **Cost**: Open source

## RECOMMENDATION: OpenZiti (Primary) + Tailscale Patterns (Fallback Concepts)

### Why OpenZiti Wins
1. **Application-embedded**: SDK goes directly into node agent (not a separate VPN layer)
2. **Identity-based**: Aligns with zero-trust philosophy (ADR-004)
3. **Dark services**: Compute nodes invisible to internet attackers
4. **Built-in PKI**: Certificate management, identity lifecycle built in
5. **NAT traversal + relay**: Handles all NAT scenarios (standard NAT, CGNAT, symmetric NAT)
6. **No user configuration**: Node agent embeds OpenZiti SDK, connects outbound, done

### Architecture for LLMHosts
```
LLMHosts Cloud
├── OpenZiti Controller (manages identities, policies)
├── OpenZiti Router(s) (data plane, relay when needed)
├── Platform Services (Conductor, LiteLLM, Dashboard)
│   └── Connected via OpenZiti fabric
│
User's Home Network (behind NAT/firewall)
├── Node Agent (Docker container)
│   ├── OpenZiti SDK (embedded)
│   ├── Outbound connection to OpenZiti fabric
│   ├── vLLM / Ollama / exo (model serving)
│   └── MCP Server (capability reporting)
│
User's VPS (cloud, static IP but still secured)
├── Node Agent (Docker container)
│   ├── OpenZiti SDK (embedded)
│   └── Same architecture as home node
```

### Connection Flow
1. User installs node agent (Docker container)
2. Agent starts → embeds OpenZiti SDK → initiates outbound HTTPS connection to OpenZiti fabric
3. Agent authenticates with LLMHosts-issued identity certificate
4. Agent registers as compute node on platform
5. All subsequent traffic flows through OpenZiti overlay:
   - Platform → Node: routed through overlay (no inbound ports needed)
   - Node → Node (P2P): direct when possible (UDP hole punch), relay otherwise
   - Node → External API: direct (not through overlay, BYOK keys used)

### Zero-Config Requirements Met
- No port forwarding: all connections outbound
- No firewall rules: standard HTTPS (port 443) outbound only
- No DNS configuration: overlay handles all routing
- No VPN client installation: SDK embedded in Docker container
- Works behind: home NAT, CGNAT, corporate firewalls, hotel WiFi, mobile hotspot

## PERFORMANCE CONSIDERATIONS

### Latency
- Direct P2P (UDP hole punch success): ~1-5ms overhead
- Relay through OpenZiti router: ~10-50ms overhead depending on relay location
- For inference (tokens streamed over seconds): overhead is negligible

### Bandwidth
- Model weights: NOT transferred through overlay (downloaded directly from HuggingFace/source)
- Inference traffic: prompt + response tokens (small, <1MB typically)
- Distributed inference (exo pipeline parallel): activation tensors between nodes (can be larger)
  - Recommend nodes on same local network for distributed inference
  - Cross-WAN distributed inference possible but slower

### Optimization
- Deploy OpenZiti routers in multiple regions for lower relay latency
- Prefer P2P connections (faster) with relay as fallback
- Group nodes by network proximity for distributed inference assignments
