# NeMo Guardrails - Deep Dive

> **TL;DR**: NeMo Guardrails is NVIDIA's open-source toolkit (5,500+ stars) for adding programmable safety controls to LLM applications. Unlike content filters, it uses Colang - a custom language for defining conversation flows and policies. It controls what LLMs can discuss, how they respond, and what actions they can take. Works at 5 levels: input, dialog, retrieval, execution, and output rails. Apache 2.0 licensed.

---

## What Is NeMo Guardrails?

**NeMo Guardrails** is a toolkit for adding **programmable guardrails** to LLM-based conversational systems. Instead of relying on fixed safety classifiers, you define policies in code that control:

- What topics the LLM can discuss
- How conversations flow
- What external tools can be called
- What responses are acceptable

### The Key Difference

```
┌─────────────────────────────────────────────────────────────────────┐
│                    FIXED CLASSIFIERS vs PROGRAMMABLE GUARDRAILS      │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   FIXED CLASSIFIERS                  PROGRAMMABLE GUARDRAILS        │
│   ├── Pre-trained on fixed data      ├── Define your own rules      │
│   ├── Can't customize categories     ├── Domain-specific policies   │
│   ├── Black box decisions            ├── Transparent logic          │
│   ├── Retrain to change              ├── Change in configuration    │
│   └── One-size-fits-all              └── Tailored to your use case  │
│                                                                     │
│   EXAMPLE                                                           │
│   "Block discussions about competitors"                             │
│                                                                     │
│   Fixed Classifier: Need training data + model retraining           │
│   NeMo Guardrails: Add 3 lines of Colang config                    │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Quick Stats

| Metric | Value |
|--------|-------|
| **GitHub Stars** | 5,500+ |
| **Forks** | 579 |
| **License** | Apache 2.0 |
| **Latest Version** | 0.19.0 |
| **Creator** | NVIDIA |
| **Language** | Python + Colang DSL |

---

## The Five Types of Rails

NeMo Guardrails operates at 5 different points in the conversation:

```
┌─────────────────────────────────────────────────────────────────────┐
│                    RAIL TYPES                                        │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   1. INPUT RAILS                                                    │
│      └── Filter user messages BEFORE LLM processing                 │
│      └── PII masking, injection detection, profanity filtering      │
│                                                                     │
│   2. DIALOG RAILS                                                   │
│      └── Control conversation FLOW and LLM prompting                │
│      └── Topic constraints, conversation steering                   │
│                                                                     │
│   3. RETRIEVAL RAILS                                                │
│      └── Filter RAG chunks BEFORE they reach the LLM                │
│      └── Fact-checking, removing irrelevant content                 │
│                                                                     │
│   4. EXECUTION RAILS                                                │
│      └── Monitor TOOL/ACTION invocations                            │
│      └── Validate inputs to external APIs, prevent unsafe actions   │
│                                                                     │
│   5. OUTPUT RAILS                                                   │
│      └── Validate responses BEFORE returning to users               │
│      └── Hallucination detection, toxicity filtering                │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Colang: The Policy Language

**Colang** is a domain-specific language for defining guardrails. It has Python-like syntax designed to be intuitive.

### Basic Syntax

```colang
# Define what users might say
define user ask about competitors
  "What do you think about [competitor]?"
  "How do you compare to [competitor]?"
  "Is [competitor] better?"

# Define how the bot should respond
define bot refuse competitor discussion
  "I'm focused on helping you with our products."
  "I'd be happy to discuss our features instead."

# Define the flow
define flow handle competitor questions
  user ask about competitors
  bot refuse competitor discussion
```

### Colang 2.0 Features

| Feature | Description |
|---------|-------------|
| **Python-like syntax** | Familiar to developers |
| **Parallel execution** | Multiple flows run simultaneously |
| **Modular imports** | Reuse flow definitions |
| **Variables** | Use `$variable` for dynamic values |

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                    NEMO GUARDRAILS ARCHITECTURE                      │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   Application                                                       │
│        ↓                                                            │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  NeMo Guardrails Layer                                      │   │
│   │  ├── Input Rails                                            │   │
│   │  ├── Dialog Rails (Colang state machine)                    │   │
│   │  ├── LLM Interaction                                        │   │
│   │  ├── Retrieval Rails (for RAG)                             │   │
│   │  ├── Execution Rails (for tools)                           │   │
│   │  └── Output Rails                                           │   │
│   └─────────────────────────────────────────────────────────────┘   │
│        ↓                                                            │
│   LLM Provider (OpenAI, HuggingFace, NVIDIA, etc.)                 │
│        ↓                                                            │
│   External Tools & Actions                                          │
│        ↓                                                            │
│   User Response                                                     │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Key Features

### Built-in Guardrails

| Guardrail | What It Does |
|-----------|--------------|
| **Jailbreak Detection** | Detects prompt injection attempts |
| **Content Moderation** | Filters inappropriate content |
| **Fact-Checking** | Validates against trusted sources |
| **Hallucination Detection** | Prevents false information |
| **PII Masking** | Protects personal data |
| **Topic Control** | Keeps conversations on-topic |

### Framework Integrations

| Framework | Support |
|-----------|---------|
| **LangChain** | Full integration |
| **LangGraph** | Multi-agent support |
| **LlamaIndex** | RAG pipeline integration |
| **OpenAI** | Native support |
| **HuggingFace** | Model support |
| **NVIDIA NIMs** | GPU-optimized deployment |

### Third-Party Safety

| Integration | Purpose |
|-------------|---------|
| **ActiveFence** | Content moderation API |
| **LlamaGuard** | Meta's safety classifier |
| **NVIDIA Safety Models** | Enterprise safety |

---

## Getting Started

### Installation

```bash
pip install nemoguardrails
```

### Project Structure

```
my_app/
├── config/
│   ├── config.yml          # Model and rails configuration
│   ├── rails.co            # Colang dialogue flows
│   └── actions.py          # Custom Python actions
└── main.py                 # Application entry point
```

### Configuration (config.yml)

```yaml
models:
  - type: main
    engine: openai
    model: gpt-4

rails:
  input:
    flows:
      - check jailbreak
      - mask pii
  output:
    flows:
      - check hallucination
      - filter toxicity
```

### Usage

```python
from nemoguardrails import RailsConfig, LLMRails

# Load configuration
config = RailsConfig.from_path("config/")

# Create rails instance
rails = LLMRails(config)

# Generate with guardrails
response = rails.generate(messages=[
    {"role": "user", "content": "What's your opinion on competitors?"}
])
```

---

## Comparison with LLM Guard

| Aspect | NeMo Guardrails | LLM Guard |
|--------|-----------------|-----------|
| **Focus** | Conversation flow control | Content filtering |
| **Approach** | State machine + Colang | Modular scanners |
| **Configuration** | DSL (Colang) | Python/YAML |
| **Best For** | Chatbots, multi-turn | API protection |
| **Learning Curve** | Medium (learn Colang) | Low |
| **Company** | NVIDIA | ProtectAI |

### When to Use Which?

```
USE NEMO GUARDRAILS WHEN:
├── Building conversational AI
├── Need dialogue flow control
├── Topic steering is important
├── Multi-turn context matters
└── Want programmable policies

USE LLM GUARD WHEN:
├── Need content scanning
├── Drop-in security layer
├── API protection focus
├── Per-request validation
└── Compliance requirements
```

---

## NVIDIA NIM Microservices (Enterprise)

NVIDIA offers GPU-optimized guardrails as services:

| NIM | Purpose |
|-----|---------|
| **Content Safety NIM** | 23-category content detection |
| **Topic Control NIM** | Keep conversations on-topic |
| **Jailbreak Detection NIM** | Prompt injection detection |

### Performance

| Metric | Value |
|--------|-------|
| **Latency** | ~0.5s for 5 parallel rails |
| **Protection** | 50% better than single-tool |
| **GPU Optimized** | H100/A100 support |

### Pricing

| Option | Cost |
|--------|------|
| **Open Source** | Free (Apache 2.0) |
| **NVIDIA AI Enterprise** | $4,500/GPU/year |
| **Free Trial** | Available on build.nvidia.com |

---

## Enterprise Integrations

### Cisco AI Defense (November 2025)

Validated reference architecture combining:
- NVIDIA NeMo Guardrails
- Cisco AI Defense platform
- On-premises or cloud deployment

### Palo Alto Networks (March 2025)

AI Runtime Security integration:
- Centralized policy enforcement
- Defense-in-depth approach
- Semantic guardrails + threat detection

---

## Use Cases

### 1. Customer Service Bot

```colang
define flow customer_service
  user asks about products
  bot provide product information

  user asks about competitors
  bot redirect to our products

  user gets angry
  bot offer human escalation
```

### 2. Healthcare Assistant

```colang
define flow medical_assistant
  user asks for diagnosis
  bot refuse medical diagnosis
  bot recommend professional consultation

  user asks about symptoms
  bot provide general information
  bot add medical disclaimer
```

### 3. Financial Advisor

```colang
define flow financial_assistant
  user asks for investment advice
  bot provide educational information
  bot add financial disclaimer

  user asks about specific stocks
  bot provide factual data only
  bot refuse predictions
```

### 4. RAG Application

```colang
define flow rag_qa
  user asks question
  retrieve relevant documents
  check document relevance
  generate grounded response
  verify factual consistency
```

---

## Recent Developments (2025)

| Date | Development |
|------|-------------|
| **Jan 2025** | NIM Microservices launch |
| **Mar 2025** | Palo Alto Networks integration |
| **Nov 2025** | Cisco AI Defense integration |
| **Nov 2025** | Nemotron Safety Guard 8B V3 (23 categories, 9 languages) |
| **Ongoing** | Multi-agent workflow support |

---

## How This Connects to Our Other Deep Dives

| Deep Dive | Connection |
|-----------|------------|
| **LLM Guard** | Complementary (content vs flow) |
| **Awesome LLMOps** | Listed in Security category |
| **OpenHands** | Could add conversation safety |
| **Context Engineering** | Guardrails as context control |
| **Agentic Design Patterns** | Safety patterns for agents |

---

## Key Links

| Resource | URL |
|----------|-----|
| GitHub | https://github.com/NVIDIA/NeMo-Guardrails |
| Documentation | https://docs.nvidia.com/nemo/guardrails |
| NVIDIA Developer | https://developer.nvidia.com/nemo-guardrails |
| NIMs | https://build.nvidia.com |

---

## Bottom Line

NeMo Guardrails represents **programmable AI safety** - the idea that guardrails should be code, not black boxes.

**Why it matters**:
- Define policies specific to your domain
- Control conversation flow, not just content
- Transparent, auditable rules
- NVIDIA enterprise backing

**The key insight**: Safety isn't just about filtering bad content. It's about controlling the entire conversation - what topics are discussed, how the AI responds, what actions it can take. NeMo Guardrails provides that complete control.

For our repository focused on AI-assisted development, NeMo Guardrails represents the production-grade approach to AI safety that enterprises need. Combined with LLM Guard for content scanning, it provides comprehensive protection.

---

*Document created: January 2026*
*Sources: [GitHub - NVIDIA/NeMo-Guardrails](https://github.com/NVIDIA/NeMo-Guardrails), [NVIDIA Developer](https://developer.nvidia.com/nemo-guardrails), [Colang Documentation](https://docs.nvidia.com/nemo/guardrails/latest/colang-2/)*
