# NEUROBRIX TECHNICAL SPECIFICATION

> **Version**: 0.1.0 (February 2026)
> **Status**: Production Ready + Public Registry
> **Core**: Universal Graph Executor for ANY Deep Learning Model
> **Hardware Ref**: Dell C4140 4xV100 NVLink (32GB each)
> **Last Update**: Package Restructure + Registry + DtypeEngine AMP + CompiledSequenceV2

---

## ARCHITECTURE OVERVIEW (FEBRUARY 2026)

### System Separation

```
PUBLIC — neurobrix package (pip install neurobrix)
  Entry: neurobrix run --model ... --hardware ... --prompt ...
  Source: src/neurobrix/{cli.py, core/, kernels/, nbx/, config/}
  Install: pip install neurobrix (or pip install -e . for dev)

PRIVATE — Forge (build-time tooling)
  Entry: python forge/forge.py {snap, build, trace, publish}
  Source: forge/{forge.py, importer/, tracer/, vendors/, config/families/}
  Requires: pip install -e . (neurobrix package must be installed)
```

**ZERO COUPLING:** forge/ has zero imports from src/neurobrix/core/. Communication is via files only (.cache/graphs/, .nbx, models/).

### Execution Modes

| Mode | Flag | Implementation | Use Case |
|------|------|----------------|----------|
| **compiled** | (default) | CompiledSequenceV2 + DtypeEngine AMP | Production (fastest, 80-95% GPU util) |
| **native** | `--seq_aten` | SequentialDispatcher (PyTorch ATen) | Debug mode |
| **triton** | `--triton` | Custom Triton kernels | R&D/benchmarking |

### Cache Architecture

```
BUILD (forge):
  python forge/forge.py build → models/<family>/<model>/model.nbx

REGISTRY:
  neurobrix import org/model → ~/.neurobrix/store/<model>.nbx
                             → ~/.neurobrix/cache/<model>/

RUNTIME (reads from):
  ~/.neurobrix/cache/<model>/  (registry-installed models)
  ./models/<family>/<model>/   (dev workspace models)
```

**NEVER modify cache files manually** — they are regenerated by forge build.

---

## TABLE DES MATIERES

1. [Philosophie Fondamentale](#1-philosophie-fondamentale)
2. [Les Principes ZERO](#2-les-principes-zero)
3. [Architecture Globale](#3-architecture-globale)
4. [Phase 1: TRACE (Capture)](#4-phase-1-trace-capture)
5. [Phase 2: IMPORT (Packaging)](#5-phase-2-import-packaging)
6. [Phase 3: PRISM (Hardware)](#6-phase-3-prism-hardware)
7. [Phase 4: RUNTIME (Execution)](#7-phase-4-runtime-execution)
8. [Format NBX](#8-format-nbx)
9. [TensorDAG](#9-tensordag)
10. [Gestion du Dtype](#10-gestion-du-dtype)
11. [VAE Tiling (Haute Resolution)](#11-vae-tiling-haute-resolution)
12. [Classifier-Free Guidance (CFG)](#12-classifier-free-guidance-cfg)
13. [High Performance I/O](#13-high-performance-io)
14. [Multi-GPU Strategies](#14-multi-gpu-strategies)
15. [Structure des Fichiers](#15-structure-des-fichiers)
16. [Historique des Corrections](#16-historique-des-corrections)
17. [Modeles Verifies](#17-modeles-verifies)
18. [Registry & Distribution](#18-registry--distribution)
19. [Dtype Engine — AMP Autocast](#19-dtype-engine--amp-autocast-critical-lesson)
20. [Known Issues & Learnings](#20-known-issues--learnings)

---

## 1. PHILOSOPHIE FONDAMENTALE

```
+===========================================================================================+
|                                                                                           |
|   "NeuroBrix n'est pas un framework de modeles."                                          |
|   "C'est un moteur d'execution de graphes declaratifs universels."                        |
|                                                                                           |
|   Un seul moteur execute TOUS les modeles:                                                |
|   - Image (Diffusion: Sana, PixArt, SDXL, Flux)                                          |
|   - Audio (TTS, Speech-to-Text)                                                          |
|   - LLM (Autoregressive: Llama, Mistral)                                                 |
|   - Video (Diffusion video: AnimateDiff, CogVideo)                                       |
|                                                                                           |
+===========================================================================================+
```

### Comment ca fonctionne

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                          NEUROBRIX: LE FLUX COMPLET                              │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  Modele HuggingFace                                                              │
│         │                                                                        │
│         ▼                                                                        │
│  ┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐   │
│  │   TRACE     │────►│   IMPORT    │────►│   PRISM     │────►│  RUNTIME    │   │
│  │  (Capture)  │     │ (Packaging) │     │ (Hardware)  │     │ (Execute)   │   │
│  └─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘   │
│         │                   │                   │                   │           │
│         ▼                   ▼                   ▼                   ▼           │
│   ┌───────────┐       ┌───────────┐       ┌───────────┐       ┌───────────┐    │
│   │ TensorDAG │       │   .nbx    │       │ Execution │       │  Output   │    │
│   │  V3 JSON  │       │ Container │       │   Plan    │       │  (Image)  │    │
│   └───────────┘       └───────────┘       └───────────┘       └───────────┘    │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### La Metaphore du Restaurant

> **"Le Chef ecrit la recette. Le Cuisinier la suit aveuglement."**

| Concept | Description |
|---------|-------------|
| **Chef** | Phase IMPORT - comprend les modeles HuggingFace, cree les recettes |
| **Recette** | Fichiers JSON (topology.json, graph.json) - instructions d'execution |
| **Cuisinier** | Phase RUNTIME - execute mecaniquement, ne comprend pas le "sens" |
| **Ingredients** | Tenseurs (poids, inputs, activations) |
| **Plat Final** | Output (image, texte, audio) |

---

## 2. LES PRINCIPES ZERO

Ces principes sont **ABSOLUS** et ne doivent **JAMAIS** etre violes.

### 2.1 ZERO HARDCODE

**Aucune valeur codee en dur. Tout vient des fichiers de configuration.**

```python
# ❌ INTERDIT
if model_name == "sana":
    sample_size = 128
hidden_dim = 2048
batch_size = 2

# ✅ REQUIS
sample_size = profile.get("sample_size")
if sample_size is None:
    raise ValueError(f"ZERO FALLBACK: 'sample_size' not in {profile_path}")
```

**Sources de donnees valides:**
- `~/.neurobrix/cache/<model>/topology.json` - Flow, connections (runtime)
- `~/.neurobrix/cache/<model>/components/<name>/runtime.json` - Config composant
- `~/.neurobrix/cache/<model>/components/<name>/graph.json` - TensorDAG
- `src/neurobrix/config/hardware/<profile>.yml` - Profil hardware (package)

### 2.2 ZERO FALLBACK

**Pas de valeur par defaut. Donnee manquante = crash explicite.**

```python
# ❌ INTERDIT
value = config.get("key", 128)        # Default silencieux
x = x or 77                           # Fallback implicite
batch_size = batch_size if batch_size else 1

# ✅ REQUIS
value = config.get("key")
if value is None:
    raise ValueError(
        f"ZERO FALLBACK: 'key' not found in config.\n"
        f"Expected in: {config_path}\n"
        f"This value should be populated during IMPORT phase."
    )
```

**Pourquoi?** Les erreurs explicites permettent de debugger rapidement.
Une valeur par defaut peut masquer un bug pendant des semaines.

### 2.3 ZERO RENAME

**Ne jamais renommer. Toujours corriger le code existant.**

```python
# ❌ INTERDIT
def _execute_v2():           # Nouvelle fonction
    pass

new_sample_size = 32         # Nouvelle variable

# ✅ REQUIS
def _execute():              # Corriger la fonction existante
    # Fix applique ici
    pass

sample_size = 32             # Corriger la valeur existante
```

**Pourquoi?** Le renommage cree du code mort, de la confusion, et des bugs.

### 2.4 ZERO SEMANTIQUE METIER

**Le runtime ne connait pas "image", "audio", "texte". Il connait des tenseurs.**

```python
# ❌ INTERDIT (dans le runtime)
if family == "image":
    height = inputs["height"]
    width = inputs["width"]

# ✅ REQUIS (semantique abstraite)
state_extent_0 = variable_resolver.resolve("state_extent_0")
state_extent_1 = variable_resolver.resolve("state_extent_1")
```

| Concept Metier (Interdit) | Concept Abstrait (NBX) |
|---------------------------|------------------------|
| `image`, `audio`, `text`  | `state_tensor` |
| `height`, `width`         | `state_extent_N` |
| `channels`, `hidden_dim`  | `state_channels` |
| `batch_size`              | `axis_0` |
| `timesteps`               | `loop_index` |

---

## 3. ARCHITECTURE GLOBALE

### 3.1 Structure du Projet (Phase 16 - Fevrier 2026)

```
/home/mlops/NeuroBrix_System/
│
├── pyproject.toml                  # Build config (hatchling)
├── src/neurobrix/                  # PACKAGE PIP INSTALLABLE
│   ├── __init__.py                 # __version__ = "0.1.0"
│   ├── cli.py                      # CLI: run, info, inspect, validate, import, list, remove
│   ├── core/                       # Runtime engine
│   │   ├── runtime/                # Execution engine
│   │   │   ├── executor.py         # RuntimeExecutor (orchestrator)
│   │   │   ├── graph_executor.py   # TensorDAG execution
│   │   │   └── graph/
│   │   │       ├── compiled_sequence.py  # CompiledSequenceV2 (zero overhead)
│   │   │       └── native_dispatcher.py  # Native PyTorch ATen
│   │   ├── flow/                   # Flow handlers
│   │   │   ├── iterative_process.py     # Diffusion loop
│   │   │   ├── autoregressive.py        # LLM/VQ generation
│   │   │   ├── forward_pass.py          # Single-pass
│   │   │   └── static_graph.py          # Static execution
│   │   ├── dtype/                  # Dtype engine
│   │   │   ├── engine.py           # DtypeEngine (AMP autocast rules)
│   │   │   └── converter.py        # Dtype conversion
│   │   ├── prism/                  # Hardware solver
│   │   │   └── solver.py           # PrismSolver (GPU allocation, dtype)
│   │   └── strategies/             # Multi-GPU strategies
│   ├── kernels/                    # Triton kernels (--triton flag)
│   │   ├── adapter.py              # ATen → kernel translation
│   │   ├── classification.py       # METADATA vs TRITON classification
│   │   └── ops/                    # Pure Triton kernels
│   ├── nbx/                        # NBX format
│   └── config/                     # YAML configs (hardware/, vendors/)
│
├── forge/                          # PRIVE — Forge CLI
│   ├── forge.py                    # CLI: snap, build, trace, publish
│   ├── importer/                   # NBXBuilder, model analyzer
│   ├── tracer/                     # Graph tracing (was: trace/)
│   │   ├── orchestrator.py         # Multi-component orchestration
│   │   ├── capture.py              # ATenHookManager
│   │   └── stimulus/               # Input generation
│   ├── vendors/                    # Vendored diffusers
│   └── config/families/            # Family YAML templates
│
├── models/                         # Modeles locaux (.nbx) — dev workspace
│   ├── image/
│   │   └── Sana_1600M_1024px_MultiLing_diffusers/
│   │       └── model.nbx
│   └── llm/
│
├── .cache/                         # CACHE LOCAL DEV (forge-only)
│   └── graphs/<model>/             # Graphes traces
│
├── ~/.neurobrix/                   # CACHE UTILISATEUR (post-registry)
│   ├── store/                      # .nbx telecharges depuis registry
│   └── cache/                      # .nbx extraits pour runtime
│       └── <model_name>/
│           ├── manifest.json
│           ├── topology.json
│           ├── runtime/
│           │   ├── variables.json
│           │   └── defaults.json
│           ├── components/
│           │   └── <name>/
│           │       ├── graph.json
│           │       ├── runtime.json
│           │       └── weights/
│           └── modules/
│
├── docs/                           # Documentation
└── tests/                          # Tests
```

### 3.2 Execution Modes

NeuroBrix supports exactly 3 execution modes:

| Mode | Flag | Description | Use Case |
|------|------|-------------|----------|
| **compiled** | (default) | CompiledSequenceV2 + DtypeEngine AMP | Production (fastest) |
| **native** | `--seq_aten` | SequentialDispatcher, PyTorch ATen | Debug mode |
| **triton** | `--triton` | Custom Triton kernels | R&D/benchmarking |

**IMPORTANT:** There is NO `--cpp`, NO `--native` flag (native is `--seq_aten`), NO `--family` flag.

### 3.3 Registry Infrastructure

```
neurobrix.es (10.0.0.39:3000)       ← API (Next.js + Prisma)
  ├── PostgreSQL (10.0.0.35:5432)   ← DB (User, Model, ApiToken)
  └── MinIO (10.0.0.36:9000)        ← Object storage (bucket: neurobrix)

hub.neurobrix.es                     ← Public downloads (Cloudflare → HAProxy → MinIO)
  └── OPNsense ACL: path /neurobrix/ only
```

---

## 4. PHASE 1: TRACE (Capture)

### 4.1 Objectif

Capturer le graphe de calcul d'un modele HuggingFace en format TensorDAG.

### 4.2 Architecture

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                              TRACE ARCHITECTURE                                  │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  tracer_orchestrator.py (main process)                                           │
│       │                                                                          │
│       ├──► spawn Worker(text_encoder)                                            │
│       │         │                                                                │
│       │         ├─► blind_solver.py: genere inputs stimulus                      │
│       │         ├─► capture.py: ATenHookManager capture ops                      │
│       │         └─► Output: graphs/<model>/components/text_encoder/graph.json    │
│       │                                                                          │
│       ├──► spawn Worker(transformer)                                             │
│       │         │                                                                │
│       │         ├─► blind_solver.py: genere inputs depuis profile.json           │
│       │         ├─► capture.py: capture ops                                      │
│       │         └─► Output: graphs/<model>/components/transformer/graph.json     │
│       │                                                                          │
│       └──► spawn Worker(vae)                                                     │
│                 │                                                                │
│                 ├─► VAE trace a petite taille (32x32 latent)                     │
│                 └─► Output: graphs/<model>/components/vae/graph.json             │
│                                                                                  │
│  ISOLATION: Chaque Worker est un subprocess Python separe                        │
│  AVANTAGE: Pas de pollution memoire entre composants                             │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 4.3 Commande CLI

```bash
# Trace via forge CLI (PRIVATE)
python forge/forge.py trace \
  --family image \
  --model Sana_1600M_4Kpx_BF16_diffusers

# Le CLI cherche automatiquement dans /home/mlops/hf_snapshots/
# Output: .cache/graphs/<model>/components/*/graph.json
```

### 4.4 Fichiers Cles

| Fichier | Role |
|---------|------|
| `forge/tracer/orchestrator.py` | Orchestre les Workers |
| `forge/tracer/worker.py` | TraceWorker (subprocess) |
| `forge/tracer/capture.py` | ATenHookManager, GraphBuilder |
| `forge/tracer/stimulus/blind_solver.py` | Genere inputs depuis config |
| `forge/tracer/stimulus/structure.py` | Infere structure model |
| `forge/tracer/storage.py` | Sauvegarde TensorDAG |

### 4.5 Mecanisme de Capture

```python
# forge/tracer/capture.py

class ATenHookManager:
    """Capture ATen ops via TorchDispatchMode."""

    def __enter__(self):
        self._mode = TorchFunctionModeCapture(self)
        self._mode.__enter__()

    def __torch_dispatch__(self, func, types, args, kwargs):
        """Intercepte CHAQUE operation ATen."""
        op_name = str(func._opname)  # ex: "aten::matmul"

        # Capture inputs
        input_ids = [self._register_tensor(t) for t in args if isinstance(t, torch.Tensor)]

        # Execute vraiment l'operation
        result = func(*args, **kwargs)

        # Capture output
        output_ids = [self._register_tensor(result)]

        # Enregistre dans le graphe
        self._graph_builder.add_op(op_name, input_ids, output_ids, attrs)

        return result
```

### 4.6 Blind Solver (Generation d'Inputs)

Le Blind Solver genere des inputs de test depuis la configuration du modele.

```python
# forge/tracer/stimulus/blind_solver.py

def solve_input_shapes(component_name: str, profile: dict) -> dict:
    """
    Genere les shapes d'input depuis le profile.json du composant.

    ZERO HARDCODE: Toutes les valeurs viennent de profile.json
    """
    shapes = {}

    # Exemple pour transformer
    if "hidden_states" in expected_inputs:
        # Lit depuis profile.json
        sample_size = profile.get("sample_size")  # ex: 128 pour 4K
        in_channels = profile.get("in_channels")  # ex: 32

        if sample_size is None:
            raise ValueError("ZERO FALLBACK: 'sample_size' not in profile")

        shapes["hidden_states"] = [batch_size, in_channels, sample_size, sample_size]

    return shapes
```

### 4.7 Strategie de Trace par Composant

| Composant | Trace Size | Raison |
|-----------|------------|--------|
| **text_encoder** | Sequence complete | Tokens = longueur variable |
| **transformer** | config.sample_size | Supporte resolutions dynamiques |
| **vae** | Petite taille (32x32) | Tiling a l'execution pour 4K+ |

---

## 5. PHASE 2: IMPORT (Packaging)

### 5.1 Objectif

Packager les graphes traces + poids HuggingFace dans un conteneur NBX.

### 5.2 Architecture

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                              IMPORT FLOW                                         │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  HuggingFace Snapshot                                                            │
│       │                                                                          │
│       ▼                                                                          │
│  ┌─────────────┐                                                                 │
│  │  Safetensors│  (poids originaux, generalement fp32 ou bf16)                   │
│  └─────────────┘                                                                 │
│         │                                                                        │
│         ▼                                                                        │
│  ┌─────────────┐     ┌─────────────┐     ┌─────────────┐                        │
│  │  NeuroTax   │────►│   Sharder   │────►│  Packager   │                        │
│  │ (normalize) │     │ (2GB max)   │     │   (ZIP)     │                        │
│  └─────────────┘     └─────────────┘     └─────────────┘                        │
│         │                   │                   │                                │
│         │                   │                   ▼                                │
│         │                   │            ┌─────────────┐                        │
│         │                   │            │   model.nbx │                        │
│         │                   │            └─────────────┘                        │
│         │                   │                                                    │
│  NeuroTax normalise:        Sharder divise:                                      │
│  - model.layers.0.attn      - shard_000.safetensors (2GB max)                   │
│  - → blocks.0.attn          - shard_001.safetensors                             │
│                             - shard_002.safetensors                             │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 5.3 Commande CLI

```bash
# Build .nbx via forge CLI (PRIVATE)
python forge/forge.py build \
  --snapshot-path /home/mlops/hf_snapshots/Sana_1600M_4Kpx_BF16_diffusers \
  --family image \
  --overwrite

# Output: models/image/<model>/model.nbx
```

### 5.4 NeuroTax: Normalisation des Noms

NeuroTax traduit les noms de poids HuggingFace en format canonique NeuroBrix.

```
HuggingFace Original                    NeuroTax Canonical
────────────────────────────────────────────────────────────────
model.layers.0.self_attn.q_proj.weight  → blocks.0.attn.q_proj.weight
text_model.encoder.layers.5.mlp.fc1     → layers.5.mlp.fc1
decoder.mid_block.resnets.0.conv1       → mid.resnets.0.conv1
```

### 5.5 Stockage des Poids en fp32

**IMPORTANT**: Les poids sont stockes en fp32 dans le NBX.

**Pourquoi?**
1. Preserver la precision maximale
2. Un seul NBX fonctionne sur tous les hardwares
3. La conversion dtype se fait au RUNTIME par Prism

```
NBX (fp32 on disk)
       │
       ▼ Runtime sur V100
   Convert to fp16
       │
       ▼ Runtime sur A100
   Convert to bf16 (ou garder fp32)
```

---

## 6. PHASE 3: PRISM (Hardware)

### 6.1 Objectif

Determiner le dtype optimal et allouer les composants aux GPUs.

### 6.2 Architecture

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                              PRISM ARCHITECTURE                                  │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  src/neurobrix/config/hardware/v100-32g.yml                                      │
│  ┌───────────────────────────────────────────────────────────────┐              │
│  │ id: v100-32g                                                   │              │
│  │ vendor: nvidia                                                 │              │
│  │ architecture: volta                                            │              │
│  │ devices:                                                       │              │
│  │   - index: 0                                                   │              │
│  │     memory_mb: 32768                                           │              │
│  │     supports_dtypes:                                           │              │
│  │       - float32                                                │              │
│  │       - float16     # bf16 NOT supported on V100              │              │
│  └───────────────────────────────────────────────────────────────┘              │
│         │                                                                        │
│         ▼                                                                        │
│  ┌─────────────────────────────────────────────────────────────────────────┐    │
│  │                         PrismSolver                                      │    │
│  │               (src/neurobrix/core/prism/solver.py)                       │    │
│  │                                                                          │    │
│  │  1. _resolve_dtype()                                                     │    │
│  │     Model dtype: bfloat16                                                │    │
│  │     Hardware supports: [float32, float16]                                │    │
│  │     → bfloat16 NOT supported → fallback to float32                       │    │
│  │     → JAMAIS float16 comme fallback de bfloat16!                         │    │
│  │                                                                          │    │
│  │  2. _allocate_components()                                               │    │
│  │     Components: [text_encoder, transformer, vae]                         │    │
│  │     Strategy: SINGLE_GPU (tout sur cuda:3)                               │    │
│  │                                                                          │    │
│  │  3. _estimate_kv_cache_bytes() [LLM only]                                │    │
│  │     cache_len = max_tokens + prompt_margin (128)                         │    │
│  │     Validates KV cache fits in chosen strategy                           │    │
│  │                                                                          │    │
│  └─────────────────────────────────────────────────────────────────────────┘    │
│         │                                                                        │
│         ▼                                                                        │
│  ComponentAllocation(                                                            │
│      name="transformer",                                                         │
│      devices=["cuda:3"],                                                         │
│      dtype=torch.float32                                                         │
│  )                                                                               │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 6.3 Regle de Dtype (CRITIQUE)

```
┌────────────────────────────────────────────────────────────────┐
│                    DTYPE HIERARCHY                              │
├────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Modele en bfloat16                                            │
│         │                                                       │
│         ▼ Hardware supporte bf16?                               │
│         │                                                       │
│    ┌────┴────┐                                                  │
│   OUI       NON                                                 │
│    │         │                                                  │
│    ▼         ▼                                                  │
│  bf16     float32                                               │
│           (JAMAIS float16!)                                     │
│                                                                 │
│  RAISON: bf16 → fp16 cause overflow                            │
│  bf16 range: ±3.4e38                                           │
│  fp16 range: ±65504                                            │
│  fp32 range: ±3.4e38 (meme que bf16)                           │
│                                                                 │
└────────────────────────────────────────────────────────────────┘
```

### 6.4 Strategies d'Allocation

| Strategy | Condition | Description |
|----------|-----------|-------------|
| `SINGLE_GPU` | Model fits 1 GPU | Tous composants sur 1 device |
| `COMPONENT_AFFINITY` | Components fit GPUs | text_encoder:cuda:0, transformer:cuda:1 |
| `PIPELINE_PARALLEL` | Sequential execution | Pipeline across GPUs (generic) |
| `PP_NVLINK` | NVLink available | Pipeline with NVLink interconnect |
| `PP_PCIE` | PCIe only | Pipeline with PCIe interconnect |
| `FGP_NVLINK` | Block-level | Fine-Grained Pipeline (NVLink) |
| `FGP_PCIE` | Block-level | Fine-Grained Pipeline (PCIe) |
| `TP_INTENT` | Weight sharding | Tensor Parallel (placeholder) |
| `ZERO3_OFFLOAD` | Exceeds VRAM | CPU offload (last resort) |

**Strategy Selection:** Prism scores strategies and picks highest. If KV cache doesn't fit, falls back to next candidate.

---

## 7. PHASE 4: RUNTIME (Execution)

### 7.1 Architecture Modulaire

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                         RUNTIME (MODULAR)                                         │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  NBXRuntimeLoader (src/neurobrix/core/runtime/loader.py)                         │
│       │                                                                          │
│       ▼                                                                          │
│  RuntimeExecutor (src/neurobrix/core/runtime/executor.py)                        │
│       │                                                                          │
│       ├──► 1. Setup modules (scheduler, tokenizer)                               │
│       ├──► 2. Setup executors (GraphExecutor per component)                      │
│       ├──► 3. Initialize strategy (SINGLE_GPU, PP_NVLINK, etc.)                  │
│       ├──► 4. Detect flow_type from topology.json                                │
│       └──► 5. Dispatch to FlowHandler                                            │
│                                                                                  │
│            ┌─────────────────────────────────────────┐                           │
│            │    FLOW HANDLERS (src/neurobrix/core/flow/)  │                      │
│            ├─────────────────────────────────────────┤                           │
│            │                                         │                           │
│            │ iterative_process.py                    │ → Diffusion models        │
│            │   pre_loop: text_encoder (ONCE)         │   (Sana, PixArt, SDXL)    │
│            │   loop: transformer (N steps)           │                           │
│            │   post_loop: vae (ONCE)                 │                           │
│            │                                         │                           │
│            │ autoregressive.py                       │ → LLM + VQ generation     │
│            │   prepare: embed_tokens                 │   (Llama, Mistral, Janus) │
│            │   loop: decode token by token           │                           │
│            │   Family-aware branching (CRITICAL)     │                           │
│            │                                         │                           │
│            │ forward_pass.py                         │ → Single-pass             │
│            │   execute all components once           │   (Encoders)              │
│            │                                         │                           │
│            │ static_graph.py                         │ → Static execution        │
│            │   execute pre-compiled graph            │                           │
│            │                                         │                           │
│            └─────────────────────────────────────────┘                           │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 7.2 Flow Iteratif (Diffusion)

```python
# Defini dans topology.json
"flow": {
    "type": "iterative_process",
    "pre_loop": ["text_encoder"],
    "loop": {
        "driver": "scheduler",
        "components": ["transformer"],
        "state_variable": "global.hidden_states"
    },
    "post_loop": ["vae"]
}
```

**Execution:**

```
1. PRE_LOOP
   ├─► Load text_encoder weights
   ├─► Execute text_encoder(input_ids, attention_mask)
   ├─► Store encoder_hidden_states
   ├─► CFG: Execute negative encoding (empty prompt)
   └─► Unload text_encoder weights

2. LOOP (20-50 steps)
   FOR step in scheduler.timesteps:
       ├─► Load transformer weights (first iteration only)
       ├─► CFG batch: stack(negative_latent, positive_latent)
       ├─► Execute transformer(hidden_states, encoder_hidden_states, timestep)
       ├─► CFG unbatch + guidance: output = negative + scale * (positive - negative)
       └─► scheduler.step(model_output) → update hidden_states

3. POST_LOOP
   ├─► Unload transformer weights
   ├─► Load vae weights
   ├─► Execute vae.decode(hidden_states)
   ├─► Apply VAE tiling if 4K+ resolution
   └─► Output: image tensor
```

### 7.3 GraphExecutor

Execute un TensorDAG (une seule composante) sur GPU.

```python
# src/neurobrix/core/runtime/graph_executor.py

class GraphExecutor:
    def __init__(self, dag: dict, dtype: torch.dtype, device: str, mode: str = "compiled"):
        self.dag = dag
        self.dtype = dtype
        self.device = device
        self.mode = mode  # "compiled" (default), "native"
        self._tensor_store = {}
        self._weights = {}

        # Compile sequence for zero-overhead execution
        if mode == "compiled":
            from neurobrix.core.runtime.graph.compiled_sequence import CompiledSequenceV2
            self._compiled_seq = CompiledSequenceV2(dag, self._tensor_store, dtype, device)

    def run(self, inputs: Dict[str, torch.Tensor]) -> Dict[str, torch.Tensor]:
        """Execute le DAG complet."""

        # CRITIQUE: inference_mode desactive autograd
        # Sans ca, PyTorch garde tous les tenseurs pour backward = OOM
        with torch.inference_mode():

            # Bind inputs
            for name, tensor in inputs.items():
                self._tensor_store[f"input::{name}"] = tensor

            # Execute via compiled sequence (zero overhead) or native dispatcher
            if self.mode == "compiled":
                self._compiled_seq.run()
            else:
                # Native mode: dict-based sequential execution
                for op_uid in self.dag["execution_order"]:
                    op = self.dag["ops"][op_uid]
                    self._execute_op(op_uid, op)

            # Gather outputs
            return self._gather_outputs()

    def unload_weights(self):
        """CRITIQUE: Libere TOUS les tenseurs."""
        self._tensor_store.clear()  # DOIT etre clear en premier!
        self._weights.clear()
        self._op_outputs.clear()
        torch.cuda.empty_cache()
```

### 7.4 WeightLoader avec Dtype Conversion

```python
# src/neurobrix/core/runtime/weight_loader.py

class WeightLoader:
    def load_component(self, component: str, device: str, dtype: torch.dtype):
        """Charge les poids avec conversion dtype."""

        # Charge depuis safetensors (stocke en fp32)
        weights = safetensors.torch.load_file(shard_path, device=device)

        # Convertit au dtype determine par Prism
        if dtype is not None:
            weights = {
                k: v.to(dtype) if v.is_floating_point() else v
                for k, v in weights.items()
            }

        return weights

# Parallel shard loading (8 workers)
# Uses ThreadPoolExecutor for high-performance I/O
```

### 7.5 DtypeEngine — AMP Autocast Rules (CRITICAL)

```python
# src/neurobrix/core/dtype/engine.py

class DtypeEngine:
    """
    Implements PyTorch AMP autocast rules for compiled mode.

    Source: PyTorch aten/src/ATen/autocast_mode.h

    Categories:
    - FP32 ops: pow, rsqrt, softmax, sum, log... (numerical stability)
    - FP16 ops: mm, bmm, conv... (performance)
    - Promote ops: add, mul... (match widest dtype)
    """

    def apply_autocast(self, op_type: str, inputs: List[torch.Tensor]) -> torch.dtype:
        """Determine compute dtype for operation following AMP rules."""

        if op_type in self.AMP_FP32_OPS:
            return torch.float32  # Upcast for stability
        elif op_type in self.AMP_FP16_OPS:
            return self.compute_dtype  # fp16/bf16 for performance
        else:
            # Promote ops: use widest dtype
            return self._promote_dtype(inputs)

# Without AMP: RMSNorm's pow→mean→rsqrt chain overflows in fp16 → rsqrt(Inf)=0 → ALL ZEROS
# With AMP: pow upcasts to fp32, cascades through mean/rsqrt, matmul brings back to fp16
```

---

## 8. FORMAT NBX

### 8.1 Structure du Conteneur

```
model.nbx (archive ZIP)
│
├── manifest.json                  # Identite du modele
├── topology.json                  # Flow d'execution, connections
│
├── runtime/
│   ├── variables.json             # Recettes de resolution
│   └── defaults.json              # Defaults (batch, seed)
│
├── components/
│   ├── text_encoder/
│   │   ├── graph.json             # TensorDAG
│   │   ├── profile.json           # Config canonique
│   │   ├── runtime.json           # Config runtime
│   │   └── weights/
│   │       ├── shard_000.safetensors
│   │       └── weights_index.json
│   │
│   ├── transformer/
│   │   ├── graph.json
│   │   ├── profile.json
│   │   └── weights/
│   │       ├── shard_000.safetensors
│   │       ├── shard_001.safetensors
│   │       └── weights_index.json
│   │
│   └── vae/
│       ├── graph.json
│       ├── profile.json
│       └── weights/
│
└── modules/
    ├── scheduler/
    │   └── scheduler_config.json
    └── tokenizer/
        ├── tokenizer_config.json
        ├── tokenizer.model
        └── vocab.json
```

### 8.2 NBX vs Cache

**IMPORTANT**: Le runtime ne lit JAMAIS directement du .nbx

```
BUILD (forge):
  python forge/forge.py build → Cree model.nbx (ZIP archive)

IMPORT (registry):
  neurobrix import org/model → Download .nbx → Extract to ~/.neurobrix/cache/

RUNTIME (TOUJOURS):
  Lit depuis ~/.neurobrix/cache/<model_name>/ (registry)
  OU depuis ./models/<family>/<model_name>/ (dev workspace)
  JAMAIS depuis model.nbx directement
```

**Fichier cle:** `src/neurobrix/nbx/cache.py:ensure_extracted()`

**Note:** Le systeme de tracage est maintenant independant dans `forge/tracer/` (zero imports de core/).

---

## 9. TENSORDAG

### 9.1 Format

```json
{
  "format": "tensor_dag",
  "version": "0.1",
  "component_name": "transformer",

  "symbolic_context": {
    "symbols": {
      "s0": {
        "name": "batch",
        "min": 1,
        "max": 64,
        "trace_value": 2,
        "source": "input::hidden_states::dim_0"
      },
      "s1": {
        "name": "height",
        "min": 8,
        "max": 256,
        "trace_value": 128
      }
    }
  },

  "tensors": {
    "input::hidden_states": {
      "tensor_id": "input::hidden_states",
      "shape": ["s0", 32, "s1", "s1"],
      "shape_concrete": [2, 32, 128, 128],
      "dtype": "torch.float32",
      "is_input": true
    },
    "param::blocks.0.attn.q_proj.weight": {
      "tensor_id": "param::blocks.0.attn.q_proj.weight",
      "shape": [2048, 2048],
      "dtype": "torch.float32",
      "is_parameter": true,
      "weight_name": "blocks.0.attn.q_proj.weight"
    }
  },

  "ops": {
    "blocks.0.attn::aten.matmul::0": {
      "op_uid": "blocks.0.attn::aten.matmul::0",
      "op_type": "aten::matmul",
      "input_tensor_ids": ["tensor_123", "param::blocks.0.attn.q_proj.weight"],
      "output_tensor_ids": ["blocks.0.attn::aten.matmul::0::out_0"],
      "attributes": {}
    }
  },

  "execution_order": ["op_uid_1", "op_uid_2", "..."],
  "input_tensor_ids": ["input::hidden_states", "input::timestep"],
  "output_tensor_ids": ["output::sample"]
}
```

### 9.2 Symbolic Shapes

Les shapes utilisent des symboles qui sont resolus au runtime.

```python
# Trace time: shape = ["s0", 32, "s1", "s1"] avec s0=2, s1=128

# Runtime 1K: s1=32
shape_resolved = [2, 32, 32, 32]  # 1024x1024 / 32

# Runtime 4K: s1=128
shape_resolved = [2, 32, 128, 128]  # 4096x4096 / 32
```

---

## 10. GESTION DU DTYPE

### 10.1 Flow Complet

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                          DTYPE FLOW (ZERO HARDCODE)                              │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  1. Hardware Profile (config/hardware/v100-32g.yml)                              │
│     supports_dtypes: ["float32", "float16"]                                      │
│              │                                                                   │
│              ▼                                                                   │
│  2. Model dtype (from topology.json extracted_values)                            │
│     dtype: "bfloat16"                                                            │
│              │                                                                   │
│              ▼                                                                   │
│  3. Prism._resolve_dtype()                                                       │
│     bf16 not in [fp32, fp16] → fallback to fp32                                  │
│              │                                                                   │
│              ▼                                                                   │
│  4. ComponentAllocation.dtype = torch.float32                                    │
│              │                                                                   │
│              ▼                                                                   │
│  5. ExecutorFactory extracts dtype from allocation                               │
│              │                                                                   │
│              ▼                                                                   │
│  6. GraphExecutor.__init__(dtype=torch.float32)                                  │
│              │                                                                   │
│              ▼                                                                   │
│  7. WeightLoader.load_component(dtype=torch.float32)                             │
│     → safetensors (fp32 on disk) → stays fp32                                    │
│              │                                                                   │
│              ▼                                                                   │
│  8. GPU Memory: Optimal (no precision loss)                                      │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 10.2 DtypeAdapter

```python
# src/neurobrix/core/runtime/dtype_adapter.py

class DtypeAdapter:
    """Adapte les dtypes selon le hardware."""

    def adapt(self, model_dtype: str, hardware_supports: List[str]) -> torch.dtype:
        """
        Determine le dtype optimal.

        REGLE CRITIQUE: bf16 not supported → fp32 (JAMAIS fp16)
        RAISON: bf16 range = ±3.4e38, fp16 range = ±65504
                bf16 → fp16 = overflow catastrophique
                bf16 → fp32 = safe (same exponent range)
        """
        if model_dtype == "bfloat16":
            if "bfloat16" in hardware_supports:
                return torch.bfloat16
            else:
                # CRITIQUE: fp32, pas fp16!
                return torch.float32

        if model_dtype == "float16":
            if "float16" in hardware_supports:
                return torch.float16
            else:
                return torch.float32

        return torch.float32
```

---

## 11. VAE TILING (HAUTE RESOLUTION)

### 11.1 Probleme

Le VAE DC-AE de Sana est trace a 32x32 latent (1024px output).
Pour 4K (128x128 latent), on doit utiliser le tiling.

### 11.2 Solution

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                          VAE TILING STRATEGY                                     │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  Latent 4K: [1, 32, 128, 128]                                                   │
│                                                                                  │
│  ┌────┬────┬────┬────┬────┐                                                     │
│  │ T1 │ T2 │ T3 │ T4 │ T5 │  5 tiles horizontaux                               │
│  ├────┼────┼────┼────┼────┤                                                     │
│  │ T6 │ T7 │ T8 │ T9 │T10 │  5 tiles verticaux                                 │
│  ├────┼────┼────┼────┼────┤                                                     │
│  │T11 │T12 │T13 │T14 │T15 │  = 25 tiles total                                  │
│  ├────┼────┼────┼────┼────┤                                                     │
│  │T16 │T17 │T18 │T19 │T20 │                                                     │
│  ├────┼────┼────┼────┼────┤                                                     │
│  │T21 │T22 │T23 │T24 │T25 │                                                     │
│  └────┴────┴────┴────┴────┘                                                     │
│                                                                                  │
│  Chaque tile: 32x32 latent                                                      │
│  Overlap: 2 latent pixels (64 output pixels)                                    │
│  Stride: 30 latent pixels                                                       │
│                                                                                  │
│  1. Split latent en tiles overlapping                                           │
│  2. Decode chaque tile avec le VAE trace a 32x32                                │
│  3. Blend les tiles avec interpolation lineaire                                 │
│  4. Assemble l'image finale 4096x4096                                           │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 11.3 Implementation

```python
# src/neurobrix/core/components/vae_tiling.py

class VAETilingStrategy:
    def __init__(self, trace_latent_size: int, spatial_compression_ratio: int):
        """
        Args:
            trace_latent_size: Size VAE was traced at (e.g., 32)
            spatial_compression_ratio: VAE compression (e.g., 32 for DC-AE)
        """
        self.tile_latent_size = trace_latent_size  # 32
        self.tile_pixel_size = trace_latent_size * spatial_compression_ratio  # 1024
        self.overlap_latent = 2  # overlap in latent space
        self.stride_latent = trace_latent_size - self.overlap_latent  # 30

    def should_tile(self, latent: torch.Tensor) -> bool:
        """Check if latent requires tiling."""
        _, _, h, w = latent.shape
        return h > self.tile_latent_size or w > self.tile_latent_size

    def tiled_decode(self, latent: torch.Tensor, decode_fn) -> torch.Tensor:
        """Decode with overlapping tiles + blending."""
        # 1. Split into overlapping tiles
        # 2. Decode each tile
        # 3. Blend overlapping regions with linear interpolation
        # 4. Assemble final image
```

### 11.4 Configuration Data-Driven

La strategie de tiling est configuree depuis les fichiers de modele:

```python
# Lecture depuis profile.json et graph.json
trace_latent_size = graph["tensors"]["input::z"]["shape"][2]  # 32
decoder_blocks = profile["decoder_block_out_channels"]  # [128, 256, 512, 512, 1024, 1024]
spatial_compression_ratio = 2 ** (len(decoder_blocks) - 1)  # 32
```

---

## 12. CLASSIFIER-FREE GUIDANCE (CFG)

### 12.1 Principe

CFG combine une prediction conditionnelle et inconditionnelle:

```
output = uncond + guidance_scale * (cond - uncond)
```

### 12.2 Implementation Batched

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                          CFG BATCHED EXECUTION                                   │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  1. PRE_LOOP: Generer embeddings                                                │
│     ├─► positive_embed = text_encoder(prompt)                                   │
│     └─► negative_embed = text_encoder("")  # empty prompt                       │
│                                                                                  │
│  2. LOOP: Batched execution                                                     │
│     FOR step in timesteps:                                                      │
│         │                                                                        │
│         ├─► Stack inputs: batch = [negative_latent, positive_latent]            │
│         │   encoder_hidden = [negative_embed, positive_embed]                   │
│         │                                                                        │
│         ├─► Execute transformer(batch, encoder_hidden, timestep)                │
│         │   → output shape: [2, channels, H, W]                                 │
│         │                                                                        │
│         ├─► Split: uncond = output[0], cond = output[1]                         │
│         │                                                                        │
│         └─► Apply guidance:                                                      │
│             noise_pred = uncond + 4.5 * (cond - uncond)                         │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 12.3 Code

```python
# src/neurobrix/core/cfg/cfg_executor.py

class CFGExecutor:
    def execute_with_cfg(self, latent, encoder_hidden, timestep):
        """Execute transformer with CFG batching."""

        # Stack positive and negative
        batched_latent = torch.cat([latent, latent], dim=0)
        batched_hidden = torch.cat([self.negative_embed, encoder_hidden], dim=0)
        batched_timestep = torch.cat([timestep, timestep], dim=0)

        # Single forward pass (batch=2)
        output = self.transformer.run({
            "hidden_states": batched_latent,
            "encoder_hidden_states": batched_hidden,
            "timestep": batched_timestep
        })

        # Split and apply guidance
        noise_pred = output["output_0"]
        uncond, cond = noise_pred.chunk(2, dim=0)

        guided = uncond + self.guidance_scale * (cond - uncond)
        return guided
```

---

## 13. HIGH PERFORMANCE I/O

### 13.1 Architecture

NeuroBrix utilise un systeme I/O haute performance pour le chargement des poids.

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                         HIGH PERFORMANCE I/O PIPELINE                            │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  Disk (SSD/NVMe)                                                                │
│       │                                                                          │
│       ▼ ThreadPoolExecutor (8 workers parallel)                                 │
│  ┌─────────────────────────────────────────────────────────────────────────┐    │
│  │ Worker 1 ──► shard_000.safetensors ─┐                                   │    │
│  │ Worker 2 ──► shard_001.safetensors ─┤                                   │    │
│  │ Worker 3 ──► shard_002.safetensors ─┼──► Parallel Load                  │    │
│  │ Worker 4 ──► shard_003.safetensors ─┤                                   │    │
│  │ Worker 5-8 ──► (available for more) ─┘                                  │    │
│  └─────────────────────────────────────────────────────────────────────────┘    │
│       │                                                                          │
│       ▼ Pinned Memory (optional DMA)                                            │
│  ┌─────────────────────────────────────────────────────────────────────────┐    │
│  │ CUDA Pinned RAM                                                          │    │
│  │ - Enables 16 GB/s PCIe DMA transfer                                     │    │
│  │ - Used for prefetch queue                                               │    │
│  └─────────────────────────────────────────────────────────────────────────┘    │
│       │                                                                          │
│       ▼ Direct GPU Load (safetensors mmap)                                      │
│  ┌─────────────────────────────────────────────────────────────────────────┐    │
│  │ VRAM (cuda:0, cuda:1, ...)                                               │    │
│  │ load_file(path, device="cuda") → Zero-copy GPU load                     │    │
│  └─────────────────────────────────────────────────────────────────────────┘    │
│                                                                                  │
│  SPEEDUP: 2.5-3x faster than sequential loading                                 │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 13.2 Configuration (config/system.yml)

```yaml
io:
  num_workers: 8              # Parallel loading workers
  use_pinned_memory: true     # Enable pinned memory for DMA
  enable_prefetch: true       # Prefetch next component while executing
  prefetch_queue_size: 2      # Number of components to prefetch
  log_performance: true       # Log I/O performance metrics
```

### 13.3 Modules I/O

| Fichier | Role |
|---------|------|
| `src/neurobrix/core/io/loader.py` | WeightLoader with parallel shard loading |
| `src/neurobrix/core/runtime/weight_loader.py` | Parallel shard loading (8 workers) |
| `src/neurobrix/core/strategies/tp_sharding.py` | Parallel TP shard loading |
| `src/neurobrix/nbx/loader.py` | FastNBXLoader (parallel extraction) |

### 13.4 Parallel Loading

```python
# src/neurobrix/core/io/loader.py

from neurobrix.core.io import WeightLoader

# Create loader
loader = WeightLoader()

# Parallel shard loading (8 workers via ThreadPoolExecutor)
shard_paths = [Path("shard_000.safetensors"), Path("shard_001.safetensors")]
weights = loader.load_shards_parallel(shard_paths, device="cuda:0", dtype=torch.float16)

# Direct GPU load via safetensors.torch.load_file (zero-copy)
```

### 13.5 Performance Notes

**Direct GPU Loading:** safetensors.torch.load_file supports direct GPU loading (mmap + zero-copy transfer).

**Parallel Sharding:** ThreadPoolExecutor with 8 workers provides 2.5-3x speedup on multi-shard models.

**No Pinned Memory Queue:** Current implementation uses direct GPU load without explicit pinned memory staging.

### 13.6 Performance Comparison

| Method | Time (21GB model) | Throughput |
|--------|-------------------|------------|
| Sequential CPU load | ~108s | 0.2 GB/s |
| Parallel (8 workers) | ~42s | 0.5 GB/s |
| Direct GPU (safetensors mmap) | ~30s | 0.7 GB/s |
| Parallel + Direct GPU | ~15s | 1.4 GB/s |

---

## 14. MULTI-GPU STRATEGIES

### 14.1 Prism (SmartSolver)

Le SmartSolver est le moteur d'allocation multi-GPU intelligent de NeuroBrix.

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                           PRISM                                                  │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  Hardware Profile (YAML)                                                        │
│       │                                                                          │
│       ▼                                                                          │
│  ┌─────────────────────────────────────────────────────────────────────────┐    │
│  │                         SmartSolver                                      │    │
│  │                                                                          │    │
│  │  1. Memory Estimation                                                    │    │
│  │     - Symbolic profiler for activations                                  │    │
│  │     - Weight size from manifest                                          │    │
│  │     - 20% safety margin                                                  │    │
│  │                                                                          │    │
│  │  2. Strategy Selection                                                   │    │
│  │     ┌────────────────────────────────────────────────────────────┐      │    │
│  │     │ Model fits 1 GPU?    → SINGLE_GPU                          │      │    │
│  │     │ Components fit GPUs? → COMPONENT_AFFINITY                  │      │    │
│  │     │ Need block split?    → FGP_NVLINK or FGP_PCIE             │      │    │
│  │     │ Need weight shard?   → TP_INTENT                          │      │    │
│  │     │ Exceeds all VRAM?    → ZERO3_OFFLOAD                      │      │    │
│  │     └────────────────────────────────────────────────────────────┘      │    │
│  │                                                                          │    │
│  │  3. Interconnect-Aware Placement                                         │    │
│  │     - Prefer same NVLink group for consecutive blocks                   │    │
│  │     - Minimize cross-group (PCIe) hops                                  │    │
│  │     - Bandwidth-weighted spillover                                       │    │
│  │                                                                          │    │
│  └─────────────────────────────────────────────────────────────────────────┘    │
│       │                                                                          │
│       ▼                                                                          │
│  SmartExecutionPlan(                                                            │
│      strategy=FGP_NVLINK,                                                       │
│      allocations=[...],                                                         │
│      estimated_peak_mb=28500,                                                   │
│      dtype=torch.float32                                                        │
│  )                                                                              │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 14.2 Strategies Disponibles

| Strategy | Description | Condition |
|----------|-------------|-----------|
| `SINGLE_GPU` | Tout sur 1 GPU | Model fits in largest GPU |
| `COMPONENT_AFFINITY` | 1 composant = 1 GPU | Each component fits 1 GPU |
| `PP_NVLINK` | Pipeline Parallel (NVLink) | Sequential execution across NVLink GPUs |
| `PP_PCIE` | Pipeline Parallel (PCIe) | Sequential execution across PCIe GPUs |
| `FGP_NVLINK` | Fine-Grained Pipeline (NVLink) | Block-level distribution with NVLink |
| `FGP_PCIE` | Fine-Grained Pipeline (PCIe) | Block-level distribution with PCIe |
| `TP_INTENT` | Tensor Parallel | Weight sharding (placeholder) |
| `ZERO3_OFFLOAD` | CPU Offload | Model exceeds total VRAM |

### 14.3 Interconnect Topology

```python
# src/neurobrix/core/prism/structure.py

class InterconnectTopology:
    """Manages GPU interconnect information."""

    def get_bandwidth(self, dev_a: int, dev_b: int) -> float:
        """Get bandwidth between devices in Gbps."""

    def get_interconnect_type(self, dev_a: int, dev_b: int) -> InterconnectType:
        """Get interconnect type (P2P_DIRECT, CPU_BOUNCE, etc.)."""

    def devices_have_fast_interconnect(self, devices: List[int]) -> bool:
        """Check if all devices are in same NVLink/xGMI group."""

    def get_fastest_group(self) -> List[int]:
        """Get device indices of fastest interconnect group."""

# Interconnect Technologies
InterconnectTech.NVLINK   # NVIDIA NVLink (300 Gbps)
InterconnectTech.XGMI     # AMD xGMI (400 Gbps)
InterconnectTech.PCIE     # PCIe (32 Gbps)
InterconnectTech.CXL      # CXL memory interconnect
InterconnectTech.UALINK   # UALink (future standard)
```

### 14.4 Hardware Profiles (Multi-GPU)

```yaml
# src/neurobrix/config/hardware/c4140-4xv100-custom-nvlink.yml

id: c4140-4xv100-custom-nvlink
vendor: nvidia

summary:
  total_gpus: 4
  total_vram_gb: 128
  topology: "NVLink partial mesh"

devices:
  - index: 0
    brand: nvidia
    model: "NVIDIA Tesla V100-SXM2-32GB"
    memory_mb: 32768
    supports_dtypes: ["float32", "float16"]
    architecture: volta
  # ... devices 1-3 (mixed 32GB + 64GB) ...

interconnect:
  groups:
    - index: 0
      members: [0, 1, 2, 3]
      type: partial_mesh
      tech: nvlink
      bandwidth_gbps: 300

pcie_fallback:
  version: "3.0"
  bandwidth_gbps: 32
```

### 14.5 FGP (Fine-Grained Pipeline) Example

```
┌─────────────────────────────────────────────────────────────────────────────────┐
│                     FGP_NVLINK: Block Distribution                               │
├─────────────────────────────────────────────────────────────────────────────────┤
│                                                                                  │
│  Transformer (28 blocks)                                                        │
│                                                                                  │
│  NVLink Group: [cuda:0, cuda:1, cuda:2, cuda:3]                                 │
│                                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │
│  │   cuda:0     │  │   cuda:1     │  │   cuda:2     │  │   cuda:3     │        │
│  │              │  │              │  │              │  │              │        │
│  │ blocks 0-6   │  │ blocks 7-13  │  │ blocks 14-20 │  │ blocks 21-27 │        │
│  │ (7 blocks)   │  │ (7 blocks)   │  │ (7 blocks)   │  │ (7 blocks)   │        │
│  │              │  │              │  │              │  │              │        │
│  │ ~7.5 GB      │  │ ~7.5 GB      │  │ ~7.5 GB      │  │ ~7.5 GB      │        │
│  └──────────────┘  └──────────────┘  └──────────────┘  └──────────────┘        │
│        │                │                 │                 │                   │
│        └───────────────┬┴─────────────────┴────────────────┘                   │
│                        │                                                        │
│                   NVLink 300 Gbps                                              │
│                   (activations transfer)                                        │
│                                                                                  │
│  Execution: Sequential block execution with NVLink activation transfer         │
│                                                                                  │
└─────────────────────────────────────────────────────────────────────────────────┘
```

### 14.6 TP Block Sharding

```python
# src/neurobrix/core/strategies/tp_sharding.py

def create_block_shards(
    cache_path: Path,
    component_name: str,
    n_shards: int,
    force: bool = False,
) -> Path:
    """
    Create TP block shards from original weights.

    LAZY: Only creates if source weights changed.
    ZERO HARDCODE: Shard axis detected from tensor shapes.
    """

# Cache structure:
# ~/.neurobrix/cache/<model>/components/<component>/
# ├── weights/                    # Original merged weights
# │   └── shard_000.safetensors
# └── block_shards/              # TP shards (created lazily)
#     ├── manifest.json          # Shard metadata + source hash
#     ├── block.0/
#     │   ├── shard_0.safetensors
#     │   └── shard_1.safetensors
#     └── block.N/
```

### 14.7 Vendor Support (VENDORLESS Architecture)

```python
# src/neurobrix/core/prism/structure.py

class DeviceBrand(Enum):
    NVIDIA = "nvidia"      # → device prefix: "cuda:"
    AMD = "amd"            # → device prefix: "hip:"
    INTEL = "intel"        # → device prefix: "xpu:"
    TENSTORRENT = "tt"     # → device prefix: "tt:"
```

**Hardcoded vendor strings:** 2 instances only (fallback error messages)

**Note:** In practice, only NVIDIA devices are currently supported. AMD/Intel/Tenstorrent are architectural placeholders.

---

## 15. STRUCTURE DES FICHIERS (Phase 16 - Fevrier 2026)

### 15.1 Fichiers Cles par Fonction

**NOTE Phase 15-16:** Le projet est maintenant un package pip installable (`src/neurobrix/`). Forge est separe dans `forge/`.

| Fonction | Fichier | Description |
|----------|---------|-------------|
| **CLI (Public)** | `src/neurobrix/cli.py` | neurobrix: run, info, inspect, validate, import, list, remove |
| **Forge (Prive)** | `forge/forge.py` | forge: snap, build, trace, publish |
| **Trace** | `forge/tracer/orchestrator.py` | Multi-component orchestration |
| | `forge/tracer/capture.py` | ATenHookManager |
| | `forge/tracer/topology.py` | Topology generator |
| | `forge/tracer/storage.py` | Graph cache storage |
| | `forge/tracer/stimulus/blind_solver.py` | Generate inputs from config |
| **Import** | `forge/importer/builder.py` | Build .nbx container |
| | `forge/importer/analyzer.py` | Model analyzer |
| **Prism** | `src/neurobrix/core/prism/solver.py` | GPU allocation, dtype, KV cache |
| | `src/neurobrix/core/prism/loader.py` | Hardware profile loading |
| **Runtime** | `src/neurobrix/core/runtime/executor.py` | Pipeline orchestrator |
| | `src/neurobrix/core/runtime/graph_executor.py` | TensorDAG execution |
| | `src/neurobrix/core/runtime/graph/compiled_sequence.py` | Zero-overhead execution (CompiledSequenceV2) |
| | `src/neurobrix/core/runtime/graph/native_dispatcher.py` | Native PyTorch ATen (--seq_aten) |
| | `src/neurobrix/core/runtime/debug.py` | Debug control system (NBX_DEBUG) |
| | `src/neurobrix/core/runtime/loader.py` | NBXRuntimeLoader |
| | `src/neurobrix/core/runtime/dtype_adapter.py` | Dtype adaptation |
| **Flow** | `src/neurobrix/core/flow/iterative_process.py` | Diffusion loop |
| | `src/neurobrix/core/flow/autoregressive.py` | LLM/VQ generation (family-aware) |
| | `src/neurobrix/core/flow/forward_pass.py` | Single-pass |
| | `src/neurobrix/core/flow/static_graph.py` | Static execution |
| **Dtype** | `src/neurobrix/core/dtype/engine.py` | DtypeEngine (AMP autocast rules) |
| | `src/neurobrix/core/dtype/converter.py` | Dtype conversion |
| **CFG** | `src/neurobrix/core/cfg/cfg_executor.py` | CFG batching |
| **Strategies** | `src/neurobrix/core/strategies/base.py` | Strategy base classes |
| | `src/neurobrix/core/strategies/pipeline.py` | Pipeline Parallel |
| | `src/neurobrix/core/strategies/fgp.py` | Fine-Grained Pipeline |
| | `src/neurobrix/core/strategies/tp_sharding.py` | Tensor Parallel |
| | `src/neurobrix/core/strategies/zero3.py` | ZeRO-3 Offload |
| **Kernels** | `src/neurobrix/kernels/adapter.py` | ATen → Triton translation (--triton) |
| | `src/neurobrix/kernels/classification.py` | METADATA vs TRITON classification |
| | `src/neurobrix/kernels/ops/*.py` | Pure Triton kernels |
| **Modules** | `src/neurobrix/core/module/scheduler/factory.py` | Scheduler creation |
| | `src/neurobrix/core/module/tokenizer/factory.py` | Tokenizer creation |
| | `src/neurobrix/core/module/autoregressive/generator.py` | AutoregressiveGenerator |
| **Config** | `src/neurobrix/config/hardware/*.yml` | Hardware profiles |
| | `src/neurobrix/config/vendors/*.yml` | Vendor configs |
| | `forge/config/families/*.yml` | Family configs (forge-only) |
| **Registry** | neurobrix.es `/var/www/neurobrix.es/` | Next.js API (10.0.0.39) |
| | `src/lib/storage.ts` | Dual S3 client (internal + hub.neurobrix.es) |
| | `src/lib/api-auth.ts` | Bearer token + NextAuth auth |

### 15.2 [ARCHIVED] C++ Runtime

**The C++ runtime and native/ directory have been removed.** NeuroBrix is now a pure Python system with three execution modes:
- **compiled** (default): CompiledSequenceV2 with zero overhead
- **native** (--seq_aten): Sequential PyTorch ATen dispatch
- **triton** (--triton): Custom Triton kernels (R&D only)

**[ARCHIVED] ONNX Export:** ONNX export is not part of the system.

---

## 16. HISTORIQUE DES CORRECTIONS

### 16.3 Fevrier 2026 - Phase 16: Registry CLI + Forge Publish

**REGISTRY COMPLETE: End-to-end publish -> import -> run via neurobrix.es + hub.neurobrix.es.**

| Changement | Description |
|------------|-------------|
| `neurobrix import` | Telecharge .nbx via signed URL hub.neurobrix.es -> ~/.neurobrix/cache/ |
| `neurobrix list` | Liste modeles installes (cache registry + workspace dev) |
| `neurobrix remove` | Supprime modeles du cache local |
| `forge publish` | Upload .nbx vers MinIO via presigned PUT interne |
| API Tokens | `nbx_...` format, Prisma ApiToken, Bearer auth |
| hub.neurobrix.es | Domaine public pour downloads signes (Cloudflare -> HAProxy -> MinIO) |
| Dual S3 Client | Interne pour uploads, public pour signed download URLs |

**Performance verifiee:**
- Publish Sana 9.1 GB: ~10s @ 580 MB/s (reseau interne)
- Import Sana 9.1 GB: ~2m09s @ 70 MB/s (via Cloudflare)

---

### 16.2 Fevrier 2026 - Phase 15: Package Restructuration

**PACKAGE PIP: `pip install neurobrix` fonctionne.**

| Changement | Description |
|------------|-------------|
| `src/neurobrix/` | Layout standard PEP 517/518 |
| `pyproject.toml` | Hatchling build backend |
| Import rename | `from core.` -> `from neurobrix.core.` (~260 occurrences) |
| neurobrix.py supprime | Shadowait le package — utiliser commande `neurobrix` |
| Config dans package | hardware/ et vendors/ dans src/neurobrix/config/ |

---

### 16.1 Fevrier 2026 - Phase 14: Forge Extraction

**FORGE UNIFIE: `forge/forge.py` = snap + build + trace + publish.**

| Changement | Description |
|------------|-------------|
| `forge/forge.py` | CLI unifie pour toutes les operations privees |
| `forge/importer/` | Deplace depuis racine importer/ |
| `forge/tracer/` | Deplace depuis racine trace/, renomme |
| `forge/vendors/` | Deplace depuis racine vendors/ |
| `forge/config/families/` | Deplace depuis racine config/families/ |

---

### 16.0 Janvier 2026 - Phase 12: Architecture Cleanup & Code Hygiene

**ARCHITECTURE CLEANUP: Zero wrappers, zero deprecated code, direct imports everywhere.**

Complete refactoring of `core/` directory following ZERO WRAPPERS principle.

**Changements cles:**

| Changement | Description |
|------------|-------------|
| **ZERO WRAPPERS** | Suppression des modules wrapper, imports directs aux sites d'usage |
| **Deprecated Removal** | Suppression des fonctions et alias deprecies |
| **Dtype Consolidation** | Source unique dans `core/dtype/` |
| **Memory Manager** | Centralise dans `core/runtime/memory/` |

**Wrappers supprimes:**
- `core/runtime/vendor_config.py` - Wrapper inutile, utiliser `core/config/` directement
- `core/runtime/family_config.py` - Wrapper inutile, utiliser `core/config/` directement

**Fonctions deprecies supprimees:**
- `core/hardware/detector.py:detect_hardware()` → Utiliser `get_device_info()` avec device explicite
- `core/hardware/detector.py:get_optimal_device()` → Specification explicite du device (ZERO AUTO-DETECTION)

**Alias deprecies supprimes:**
- `AutoregressiveScheduler` → Utiliser `AutoregressiveGenerator`
- `VQImageScheduler` → Utiliser `VQImageGenerator`
- `AutoregressiveState` → Utiliser `GenerationState`

**Principe ZERO WRAPPERS:**
```python
# AVANT (wrapper)
from core.runtime.vendor_config import load_hardware_profile

# APRES (import direct)
from core.config.hardware import load_hardware_profile
```

**NOTE:** Le registre factory dans `core/module/autoregressive/factory.py` maintient la compatibilite avec les valeurs string dans les fichiers NBX.

---

### 16.1 Janvier 2026 - Phase 11: Zero-Overhead Execution

**PERFORMANCE BREAKTHROUGH: NeuroBrix atteint la parite avec Diffusers (~23s E2E).**

**Probleme identifie:**
- GPU-Util NeuroBrix: 40-60% (contre 100% Diffusers)
- ~15000 dict lookups par step transformer
- `tensor.item()` causait des syncs GPU a chaque iteration

**Solutions implementees:**

1. **CompiledSequence** (`core/runtime/graph/compiled_sequence.py`)
   - Pre-compilation de tous les ops au chargement
   - TensorArena avec `__slots__` pour acces O(1)
   - Closures pour resolution des inputs (zero string lookup)
   - Dead tensor analysis pour liberer la memoire pendant l'execution

2. **Debug Control System** (`core/runtime/debug.py`)
   - `NBX_DEBUG=0` (default): Pas de logging verbose
   - `NBX_DEBUG=1`: Logging detaille (force GPU sync)
   - Elimine 75% de l'overhead cause par `tensor.item()`

**Architecture:**
```
AVANT (runtime):
for op_uid in execution_order:        # Python loop
    op_data = ops[op_uid]              # Dict lookup
    inputs = resolve(op_data)           # String -> tensor
    result = dispatch(op_type, ...)     # String dispatch
    store(output_id, result)            # String -> storage

APRES (runtime):
for op in compiled_ops:
    args = tuple(memory[i] for i in op.input_slots)  # int -> tensor
    memory[op.output_slot] = op.func(args, op.attrs)  # int -> storage
```

**Resultats:**
| Metrique | Avant | Apres |
|----------|-------|-------|
| GPU Utilization | 40-60% | 80-95% |
| E2E Time (20 steps) | ~45s | ~23s |
| Step Time | ~2000ms | ~1150ms |

**Fichiers ajoutes:**
- `src/neurobrix/core/runtime/graph/compiled_sequence.py` - CompiledSequenceV2
- `src/neurobrix/core/runtime/debug.py` - Debug control system (NBX_DEBUG)

**Key APIs:**
```python
# Compiled mode (default)
executor = RuntimeExecutor(pkg, plan, mode="compiled")

# Native mode (debug)
executor = RuntimeExecutor(pkg, plan, mode="native")

# Debug control
NBX_DEBUG=1 neurobrix run ...  # Verbose logging (slow, forces GPU sync)
NBX_DEBUG=0 neurobrix run ...  # No logging (default, fast)
```

### 16.1 Janvier 2026 - Phase 9: Root-Level Module Extraction [ARCHIVED]

**[ARCHIVED]** This phase is superseded by Phase 14 (Forge extraction).

**Key change:** `trace/` moved to `forge/tracer/`, `importer/` moved to `forge/importer/`.

### 16.1 Janvier 2026 - Sana 4K Fix

**Probleme:** Transformer trace a 32x32 au lieu de 128x128 pour 4K.

**Cause:** `blind_solver.py` forcait sample_size = compression_ratio (32) pour tous les composants.

**Solution:** Transformer utilise config.sample_size (128), VAE reste a 32 + tiling.

**Fichiers modifies:**
- `forge/tracer/stimulus/blind_solver.py` - Removed compression_ratio forcing
- `forge/tracer/stimulus/structure.py` - Same fix
- `src/neurobrix/core/runtime/executor.py` - Added logger, fixed cache_path
- `src/neurobrix/core/components/vae_tiling.py` - Fixed padding mode + tensor cloning

### 16.2 Janvier 2026 - Runtime Modular Refactoring

**Changement:** executor.py decompose de 3022 LOC → ~800 LOC (75% reduction)

**Nouveaux modules:**
- `src/neurobrix/core/flow/iterative_process.py` - Diffusion loop
- `src/neurobrix/core/flow/autoregressive.py` - LLM + VQ generation (family-aware)
- `src/neurobrix/core/flow/forward_pass.py` - Single-pass
- `src/neurobrix/core/flow/static_graph.py` - Static execution
- `src/neurobrix/core/cfg/cfg_executor.py` - CFG batching
- `src/neurobrix/core/runtime/resolution/input_resolver.py` - Input resolution
- `src/neurobrix/core/runtime/synthesis/input_synthesizer.py` - Input synthesis
- `src/neurobrix/core/runtime/extraction/output_extractor.py` - Output extraction

### 16.3 Janvier 2026 - Trace System Separation [SUPERSEDED]

**[SUPERSEDED BY PHASE 14]** Le systeme de tracage est maintenant dans `forge/tracer/`.

**Modifications finales:**
- CLI: `python forge/forge.py trace` (forge CLI unifie)
- `forge/tracer/` (was: `trace/`)
- forge/tracer/ a ZERO imports de src/neurobrix/core/
- Separation complete: forge (build-time) vs neurobrix (runtime)

### 16.4 Decembre 2025 - Memory Fixes

**Problemes:**
1. Memory leak dans unload_weights (tensor_store non clear)
2. Pas de inference_mode (autograd garde tous les tenseurs)
3. Dtype non converti (fp32 charge au lieu de fp16)

**Solutions:**
```python
# Fix 1: Clear tensor_store FIRST
# src/neurobrix/core/runtime/graph_executor.py
def unload_weights(self):
    self._tensor_store.clear()  # CRITIQUE!
    self._weights.clear()
    torch.cuda.empty_cache()

# Fix 2: inference_mode
# src/neurobrix/core/runtime/graph_executor.py
def run(self, inputs):
    with torch.inference_mode():
        # ... execution ...

# Fix 3: Dtype flow
# src/neurobrix/core/runtime/weight_loader.py
weights = weights.to(dtype)  # Convert at load time
```

---

## 17. MODELES VERIFIES

| Modele | Resolution | Hardware | Status | Source |
|--------|------------|----------|--------|--------|
| Sana_1600M_1024px (sana/1600m-1024) | 1024x1024 | V100 32GB | Working | Registry |
| Sana_1600M_4Kpx_BF16_diffusers | 4096x4096 | V100 32GB | Working | Local |
| PixArt-Sigma-XL-2-1024-MS | 1024x1024 | V100 32GB | Working | Local |
| PixArt-XL-2-1024-MS | 1024x1024 | V100 32GB | Working | Local |
| deepseek-moe-16b-chat | N/A (LLM) | 4xV100 NVLink | Working | Local |
| Janus-Pro-7B | 384x384 (VQ) | 4xV100 NVLink | Working | Local |

### Commandes de Test

```bash
# Import depuis registry + run (PUBLIC)
neurobrix import sana/1600m-1024
neurobrix run --model 1600m-1024 --hardware v100-32g --prompt "A sunset" --steps 10

# Run with dev mode (pip install -e .)
PYTHONPATH=src python -m neurobrix run \
  --model 1600m-1024 \
  --hardware v100-32g \
  --prompt "A sunset" \
  --steps 10

# Trace (forge - PRIVATE)
python forge/forge.py trace --family image --model PixArt-Sigma-XL-2-1024-MS

# Build .nbx (forge - PRIVATE)
python forge/forge.py build \
  --snapshot-path /home/mlops/hf_snapshots/Sana_1600M_4Kpx_BF16_diffusers \
  --family image \
  --overwrite

# Sana 4K (dev workspace)
neurobrix run \
  --model Sana_1600M_4Kpx_BF16_diffusers \
  --height 4096 --width 4096 \
  --prompt "a photorealistic cat" \
  --steps 20 --hardware v100-32g \
  --output sana_4k.png

# LLM
neurobrix run \
  --model deepseek-moe-16b-chat \
  --hardware c4140-4xv100-custom-nvlink \
  --prompt "Hello, Who are you"

# Native mode (debug)
neurobrix run --model 1600m-1024 --hardware v100-32g --prompt "test" --seq_aten

# Triton mode (R&D)
neurobrix run --model 1600m-1024 --hardware v100-32g --prompt "test" --triton

# Publish to registry (forge - PRIVATE)
NEUROBRIX_API_TOKEN=nbx_... python forge/forge.py publish \
  models/image/Sana_1600M_1024px_MultiLing_diffusers/model.nbx \
  --org sana --name 1600m-1024 --category IMAGE \
  --tags "diffusion,text-to-image,sana,1024px" \
  --description "Sana 1600M 1024px MultiLingual"

# List installed models
neurobrix list

# Remove model
neurobrix remove 1600m-1024
neurobrix remove 1600m-1024 --store  # Also delete .nbx from store/
```

---

---

## 18. REGISTRY & DISTRIBUTION (Phase 16 - Fevrier 2026)

### 18.1 Architecture

```
┌─────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│  forge publish   │────►│   neurobrix.es   │────►│      MinIO       │
│  (presigned PUT) │     │   (API + Auth)   │     │  10.0.0.36:9000  │
└─────────────────┘     └──────────────────┘     └────────┬─────────┘
                                                           │
┌─────────────────┐     ┌──────────────────┐              │
│ neurobrix import │◄────│ hub.neurobrix.es │◄─────────────┘
│  (signed GET)    │     │ (Cloudflare/HAP) │
└─────────────────┘     └──────────────────┘
```

### 18.2 Flux Publish (Forge → MinIO)

1. `forge publish model.nbx --org sana --name 1600m-1024 --category IMAGE`
2. CLI envoie `POST /api/admin/upload` avec Bearer token → recoit presigned PUT URL (interne)
3. CLI upload le .nbx directement vers MinIO via PUT (~580 MB/s sur reseau interne)
4. CLI envoie `POST /api/admin/models` → enregistre les metadonnees en base (upsert)

### 18.3 Flux Import (User → MinIO via hub.neurobrix.es)

1. `neurobrix import sana/1600m-1024`
2. CLI appelle `GET /api/models/sana/1600m-1024` → metadonnees du modele
3. CLI appelle `GET /api/models/sana/1600m-1024/download` → signed URL (hub.neurobrix.es, 1h expiry)
4. CLI telecharge le .nbx via signed URL (~70 MB/s via Cloudflare)
5. Sauvegarde dans `~/.neurobrix/store/1600m-1024.nbx`
6. Extraction vers `~/.neurobrix/cache/1600m-1024/`

### 18.4 Signed URLs — Pourquoi et Comment

**Pourquoi:** Protection contre les bots et telechargements non autorises. Les URLs expirent apres 1h.

**Comment:** Dual S3 client dans `storage.ts`:
- `s3` (interne): endpoint `http://10.0.0.36:9000` — pour uploads et operations admin
- `s3Public`: endpoint `https://hub.neurobrix.es` — pour generer les signed URLs de telechargement

Les signatures S3 sont liees au header Host. Une URL signee avec `10.0.0.36:9000` ne fonctionne pas via `hub.neurobrix.es` (et inversement). D'ou le dual client.

### 18.5 Securite

| Couche | Protection |
|--------|-----------|
| **API** | Bearer tokens `nbx_...` pour publish (Prisma ApiToken) |
| **Downloads** | Signed URLs (expirent apres 1h) |
| **Network** | OPNsense ACL: hub.neurobrix.es restreint a `/neurobrix/` |
| **Auth** | Admin routes acceptent NextAuth (navigateur) OU Bearer token (CLI) |

### 18.6 Cache Utilisateur

```
~/.neurobrix/
├── store/              ← .nbx telecharges (transport)
│   └── 1600m-1024.nbx
└── cache/              ← Modeles extraits (runtime)
    └── 1600m-1024/
        ├── manifest.json
        ├── topology.json
        ├── runtime/
        ├── components/
        └── modules/
```

`find_model()` cherche dans cet ordre:
1. `~/.neurobrix/cache/{name}/` (registry)
2. `./models/{family}/{name}/` (dev workspace)

---

## 19. DTYPE ENGINE — AMP AUTOCAST (CRITICAL LESSON)

### 19.1 The Problem Without AMP

**RMSNorm overflow in fp16:** Without AMP, operations like `pow(2) → mean → rsqrt` overflow in fp16:
- `pow(2)` on large values → Inf
- `rsqrt(Inf)` → 0
- Result: ALL ZEROS propagate through the network → gibberish output

### 19.2 The Solution: DtypeEngine

`src/neurobrix/core/dtype/engine.py` implements PyTorch AMP autocast rules (source: `aten/src/ATen/autocast_mode.h`).

**Three categories:**
1. **FP32 ops** (numerical stability): pow, rsqrt, softmax, sum, log, exp, sqrt, mean, std, layer_norm, batch_norm, group_norm, polar, view_as_complex
2. **FP16 ops** (performance): mm, bmm, addmm, baddbmm, conv1d, conv2d, conv3d, linear
3. **Promote ops** (match widest): add, sub, mul, div, cat, stack

**Flow:**
```
pow (fp16 input) → AMP upcasts to fp32
  ↓
mean (fp32) → stays fp32 (FP32 op)
  ↓
rsqrt (fp32) → stays fp32 (FP32 op)
  ↓
mul (fp32, fp16) → promotes to fp32
  ↓
matmul (fp32 input) → AMP downcasts to fp16 (FP16 op)
```

**Key insight:** FP32 outputs stay fp32 (no automatic downcast). Downstream FP16 ops (matmul) cast back to compute_dtype.

### 19.3 Complex Tensor Protection

**CRITICAL:** `_to_copy` must NEVER cast complex → real dtype (discards imaginary part).

```python
# src/neurobrix/core/dtype/engine.py
def _handle_aten___to_copy(self, inputs, attrs):
    inp = inputs[0]
    target_dtype = attrs.get("dtype")

    # Guard: complex → real is forbidden
    if inp.is_complex() and not target_dtype.is_complex:
        return inp  # Pass through unchanged

    return inp.to(dtype=target_dtype)
```

**Why:** DeepSeek-V2 RoPE uses `polar` + `view_as_complex` → complex64 tensors. Casting to fp16 silently drops imaginary part → corrupted RoPE frequencies → ALL attention wrong.

### 19.4 Compiled vs Native Mode

| Mode | Dtype Control | Use Case |
|------|---------------|----------|
| **compiled** (default) | DtypeEngine (AMP) | Production (fastest) |
| **native** (--seq_aten) | Ad-hoc per-op fixes | Debug |
| **triton** (--triton) | Custom kernels | R&D |

**Performance:** Compiled mode achieves 80-95% GPU utilization (~1150ms/step). Native mode is slower (40-60% GPU util, ~2000ms/step with debug logging).

---

## 20. KNOWN ISSUES & LEARNINGS

### 20.1 Autoregressive Flow — Family-Aware Branching

**CRITICAL:** `src/neurobrix/core/flow/autoregressive.py` handles BOTH image-family (Janus VQ) and LLM-family (DeepSeek). Every change MUST be conditioned on family.

```python
# Family detection (data-driven from manifest.json)
gen_type = self.ctx.pkg.manifest.get("generation_type")  # "autoregressive_image" or "autoregressive_text"
family = self.ctx.pkg.manifest.get("family")              # "image" or "llm"
```

**Key differences:**
- **Image VQ:** CFG batch=2 (cond+uncond), VQ token offset, gen_head component
- **LLM:** CFG not used, batch=1, lm_head weight matrix, configurable sampling

### 20.2 KV Cache Position Handling

**Position IDs Contract:**
```
PREFILL:
  position_ids = [0, 1, 2, ..., seq_len-1]     # Standard sequential

DECODE (each step):
  cache_len = kv_wrapper.get_cache_len()        # Grows each step
  position_ids = [[cache_len]] (per batch item) # ABSOLUTE position
```

**Why relative position_ids=[[0]] was wrong:** Every decode step used RoPE for position 0 → attention pattern identical → incoherent generation.

**arange Interceptor:** Only shifts float-dtype aranges (RoPE frequency computation). Integer aranges (position indices) pass through unmodified.

### 20.3 Cache Files Are Generated, Not Editable

```
WRONG APPROACH (manual cache edit):
  ~/.neurobrix/cache/Janus-Pro-7B/runtime/defaults.json ← Manual edit
  → LOST on next import!

CORRECT APPROACH (fix at source):
  1. Fix in forge/importer/ (for defaults.json, manifest.json, variables.json)
  2. Re-run: python forge/forge.py build --overwrite
  → Cache regenerated with correct values
```

**Source of Generation:**
| Cache File | Generated By | Fix Location |
|------------|--------------|--------------|
| `defaults.json` | `forge/importer/builder.py` | forge/importer/ |
| `manifest.json` | `forge/importer/builder.py` | forge/importer/ |
| `variables.json` | `forge/importer/builder.py` | forge/importer/ |
| `weights/` | `forge/importer/weights/sharder.py` | forge/importer/ |
| `graph.json` | Trace system (forge/tracer/) | `python forge/forge.py trace` |
| `topology.json` | Trace system (forge/tracer/) | `python forge/forge.py trace` |

---

*NeuroBrix System - Universal Deep Learning Infrastructure*
*Version 0.1.0 - February 2026*
