---
title: HallucinationGuard-Env
emoji: 🛡️
colorFrom: red
colorTo: indigo
sdk: docker
app_port: 7860
pinned: true
tags:
  - openenv
  - reinforcement-learning
  - hallucination-detection
  - grounded-generation
  - question-answering
  - fact-checking
  - llm-training
  - llm-evaluation
  - benchmark
  - ai-safety
---

# 🛡️ HallucinationGuard-Env v4.1

> **The production-grade OpenEnv RL environment for training and evaluating LLMs on hallucination avoidance.**

[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compatible-blue)](https://github.com/meta-pytorch/OpenEnv)
[![PyPI](https://img.shields.io/pypi/v/openenv-halluguard)](https://pypi.org/project/openenv-halluguard/)
[![License](https://img.shields.io/badge/License-MIT-green)](LICENSE)
[![Dataset](https://img.shields.io/badge/Dataset-1M%2B_examples-orange)](#datasets)
[![Grader](https://img.shields.io/badge/Grader-research--grade-purple)](#grader)

---

## 💡 The Inspiration

During research for a Hackathon, an AI model confidently hallucinated a **"golden ticket backdoor"** — claiming that Ideathon winners could skip directly to the Grand Finale. This information existed nowhere in the official sources. The AI stated it with high confidence and even fabricated a supporting quote.

That moment made one thing clear: hallucination isn't just an academic problem. It causes real confusion in high-stakes situations.

**HallucinationGuard-Env** was built to fix that — training AI models to say *"I don't know"* when they don't, cite real sources when they do, and never fabricate with confidence.

---

## 🚀 Quick Start

### Python SDK

```bash
pip install openenv-halluguard
```

```python
from openenv_halluguard import HallucinationGuardEnv

# Define your model
def my_model(question, context):
    # Call your LLM here — answer from context only
    return "your answer based on context"

# Evaluate in 3 lines
env = HallucinationGuardEnv()
results = env.evaluate(my_model, episodes=5, model_name="my-model")
env.print_report(results)
env.submit_to_leaderboard(results, organization="MyCompany")
```

### Raw HTTP

```python
import requests

BASE = "https://samsankar-hallucination-guard-env.hf.space"

# 1. Start episode
obs = requests.post(f"{BASE}/reset").json()
print(obs["question"], obs["context"])

# 2. Answer from context only
result = requests.post(f"{BASE}/step", json={"answer": "your answer"}).json()
print(result["reward"], result["is_hallucination"])

# 3. View leaderboard
print(requests.get(f"{BASE}/leaderboard").json())
```

### Run Locally

```bash
git clone https://huggingface.co/spaces/SamSankar/hallucination-guard-env
cd hallucination-guard-env
pip install -r server/requirements.txt
uvicorn server.app:app --reload --port 7860
curl http://localhost:7860/health
```

---

## Tasks

HallucinationGuard-Env exposes **3 named tasks** in difficulty order.
Each task maps to a curated subset of the 38 loaded datasets and uses
task-specific grader weights.

| # | task_id | Difficulty | Primary Datasets | Frontier LLM Score |
|---|---------|-----------|-----------------|-------------------|
| 1 | `task_1_factual_grounding` | 🟢 Beginner | SQuAD, BoolQ, ARC, OpenBookQA | 0.70–0.85 |
| 2 | `task_2_multi_hop_synthesis` | 🟡 Intermediate | HotpotQA, CoQA, NQ-Open, MS-MARCO | 0.55–0.70 |
| 3 | `task_3_adversarial_resistance` | 🔴 Advanced | HaluEval, TruthfulQA, FEVER, AdversarialQA | 0.40–0.60 |

Retrieve the full task list and action schema:
```bash
curl https://samsankar-hallucination-guard-env.hf.space/tasks
```

---

## 🎮 How The Environment Works

The agent receives a **question** and a **source document**. It must answer using only what the document says, provide a direct quote supporting its answer, and state how confident it is.

### Action Space

Every `POST /step` call accepts this JSON body (only `answer` is required):

```json
{
    "answer":           "string  — derived ONLY from the provided context",
    "confidence":       0.5,     // float 0.0–1.0, calibrated estimate
    "source_quote":     "string  — verbatim phrase from context supporting the answer",
    "reasoning":        "string  — optional chain-of-thought",
    "uncertainty_flags": []      // list of aspects the agent is unsure about
}
```

### Observation Space

```python
@dataclass
class HallucinationObservation(Observation):
    question: str                  # The question to answer
    context: str                   # Source document to answer from
    reward: float                  # Step reward (-1.0 to 1.0)
    feedback: str                  # Detailed human-readable feedback
    is_hallucination: bool         # Was hallucination detected?
    hallucination_type: str        # Type of hallucination detected
    hallucination_severity: str    # NONE / MINOR / MODERATE / SEVERE / CRITICAL
    grounding_score: float         # How well answer is grounded in context
    accuracy_so_far: float         # Running accuracy this episode
    skill_rating: float            # ELO-style skill rating
    attempts_remaining: int        # Steps left in episode
    done: bool                     # Episode complete?
```

### Episode Flow

```
reset()
  → Sample question + context from dataset (curriculum-aware)
  → Return initial observation

step(action)
  → Grade answer across 9 research-grade components
  → Detect hallucination type and severity
  → Compute multi-factor reward with ROUGE + BERTScore + AlignScore
  → Adapt difficulty based on performance
  → Return observation with reward + rich feedback

state()
  → Return episode metadata: ID, step count, skill rating, curriculum stage
```

---

## OpenEnv Required Endpoints

### `GET /tasks`
Returns all 3 task definitions and the complete action schema.

```python
import requests
BASE = "https://samsankar-hallucination-guard-env.hf.space"
tasks = requests.get(f"{BASE}/tasks").json()
print(tasks["tasks"])   # list of 3 task objects
print(tasks["action_schema"])  # JSON Schema for step actions
```

### `POST /grader`
Score a completed episode. Pass the per-step rewards and info dicts collected during the episode.

```python
grade = requests.post(f"{BASE}/grader", json={
    "task_id": "task_1_factual_grounding",
    "step_rewards": [0.82, 0.55, 0.91, 0.43, 0.78],
    "step_infos": [
        {"correctness": 0.9, "grounding": 0.8, "calibration": 0.7,
         "hallucination_score": 0.1, "is_hallucination": False},
        # ... one dict per step
    ]
}).json()
print(grade["score"])    # float in [0.0, 1.0]
print(grade["breakdown"])
```

### `POST /baseline`
Run the built-in heuristic agent across all 3 tasks. No API key needed.

```python
baseline = requests.post(f"{BASE}/baseline", json={
    "steps_per_task": 5,
    "seed": 42
}).json()
print(baseline["summary"])  # overall_score, avg_reward, hallucination_rate
for task in baseline["tasks"]:
    print(task["task_id"], task["score"])
```

---

## Baseline Scores

Run `run_baseline.py` to reproduce these scores:

```bash
# Heuristic baseline (no API key)
python run_baseline.py --heuristic --episodes 3 --steps 5 --seed 42

# GPT-3.5-turbo baseline
export OPENAI_API_KEY=sk-...
python run_baseline.py --model gpt-3.5-turbo --episodes 3 --steps 5 --seed 42
```

### Heuristic Baseline (reproducible, no LLM required)

| Task | Score | Hallucination Rate |
|------|-------|--------------------|
| task_1_factual_grounding | 0.38 | 28% |
| task_2_multi_hop_synthesis | 0.28 | 41% |
| task_3_adversarial_resistance | 0.19 | 58% |
| **Overall** | **0.28** | **42%** |

### GPT-3.5-turbo Baseline (3 episodes × 5 steps, seed=42)

| Task | Score | Hallucination Rate |
|------|-------|--------------------|
| task_1_factual_grounding | 0.58 ± 0.08 | 14% |
| task_2_multi_hop_synthesis | 0.47 ± 0.09 | 22% |
| task_3_adversarial_resistance | 0.34 ± 0.10 | 38% |
| **Overall** | **0.46** | **25%** |

*Scores are averaged across episodes. Higher is better. Lower hallucination rate is better.*

---

## 🏆 Reward System (v4.1 — Research-Grade)

| Component | Weight | Description |
|-----------|--------|-------------|
| Factual correctness | 0.30 | Exact/fuzzy match + semantic similarity to ground truth |
| Source grounding | 0.20 | Verifies answer is supported by context |
| Citation accuracy | 0.15 | source_quote found verbatim in context |
| Confidence calibration | 0.15 | ECE between stated confidence and correctness |
| Semantic consistency | 0.10 | NLI entailment score (DeBERTa-v3-large) |
| Hallucination penalty | 0.10 | Penalises detected hallucinations |
| ROUGE (1/2/L) | 0.05 | Surface-form overlap with reference answer |
| BERTScore (DeBERTa-v3) | 0.05 | Token-level semantic similarity |
| AlignScore | 0.05 | Faithfulness to context (RoBERTa, ACL 2023) |

Difficulty multiplier: `beginner × 0.9`, `intermediate × 1.0`, `advanced × 1.1`, `expert × 1.2`

```
reward = clamp(Σ(weight × score) × difficulty_multiplier + consistency_bonus, 0.0, 1.0)
```

**In practice:**
- Hallucinated answer with false citation → reward ≈ **0.002–0.10**, CRITICAL severity
- Grounded correct answer with real quote → reward ≈ **0.85–1.00**

---

## 🔬 Hallucination Detection <a name="grader"></a>

### 8 Types Classified

| Type | What It Catches |
|---|---|
| `FABRICATED_FACT` | Information stated that is not in the source |
| `FALSE_CITATION` | `source_quote` that does not exist in the document |
| `OVERCONFIDENT_WRONG` | High confidence on an incorrect answer |
| `CONTEXT_DRIFT` | Answer gradually drifts away from source |
| `NUMERICAL_FABRICATION` | Made-up statistics or numbers |
| `ENTITY_CONFUSION` | Wrong names, organisations, or places |
| `TEMPORAL_ERROR` | Incorrect dates or timelines |
| `RELATIONSHIP_ERROR` | Incorrect relationships between entities |

### 5 Severity Levels

| Level | Score | Meaning |
|---|---|---|
| NONE | 0.0 | Fully grounded answer |
| MINOR | 0.1–0.3 | Slight deviation from source |
| MODERATE | 0.3–0.5 | Noticeable unsupported claims |
| SEVERE | 0.5–0.7 | Significantly fabricated content |
| CRITICAL | 0.7+ | Answer largely invented |

### Detection Algorithms (v4.0)

- **Word coverage** — fraction of meaningful content words in answer found in context
- **Entity hallucination** — novel entities in answer not found in source
- **Numerical fabrication** — numbers in answer absent from context
- **Sliding window fuzzy matching** — citation verification (threshold 0.7)
- **Negation mismatch** — contradiction detection via negation word analysis
- **Confidence calibration error** — `|confidence − correctness|` with 50% overconfidence surcharge
- **NLI cross-encoder** — `cross-encoder/nli-deberta-v3-large` for semantic entailment (upgraded from small in v4.0)
- **ROUGE-1/2/L** — standard token overlap metric in QA/summarisation research (Lin 2004)
- **BERTScore** — contextual embedding similarity via DeBERTa-v3-base (Zhang et al. 2020)
- **AlignScore** — faithfulness/grounding scorer via RoBERTa (Zha et al. ACL 2023)

---

## 📚 Datasets <a name="datasets"></a>

**1,090,163 total examples** across 38 real-world QA datasets — cached permanently, instant boot:

| Source | Examples | Domain |
|---|---|---|
| SQuAD + SQuAD-v2 | 100,000 | Reading comprehension |
| TriviaQA | 50,000 | Open-domain factual QA |
| HotpotQA | 50,000 | Multi-hop reasoning |
| DROP | 50,000 | Numerical reasoning |
| RACE | 50,000 | Exam reading comprehension |
| NewsQA | 50,000 | News article QA |
| FaithDial / HH-RLHF | 49,649 | Faithful dialogue |
| FEVER / SNLI | 49,947 | Fact verification |
| NQ Open | 50,000 | Natural questions |
| AQUA-RAT | 97,467 | Math word problems |
| XSum | 49,994 | Extreme summarisation |
| CNN/DailyMail | 50,000 | News summarisation |
| HellaSwag | 39,905 | Commonsense completion |
| AdversarialQA | 30,000 | Adversarial reading comprehension |
| WinoGrande | 40,398 | Commonsense inference |
| CommonsenseQA | 9,741 | Commonsense reasoning |
| BoolQ | 9,427 | Boolean yes/no QA |
| CoQA | 7,199 | Conversational QA |
| MedQA | 10,000 | Medical licensing exam |
| MedMCQA | 20,000 | Medical entrance exam |
| SciTail | 23,596 | Science entailment |
| HaluEval | 10,000 | Hallucination evaluation |
| TruthfulQA | 817 | Factuality benchmark |
| QASC | 8,134 | Multi-hop science |
| QUAIL | 10,246 | Reading comprehension |
| SciQ | 11,679 | Science QA |
| Circa | 31,525 | Social context QA |
| ARC | 2,590 | Science exam |
| OpenBookQA | 4,957 | Common knowledge |
| AG News | 50,000 | News classification |
| QuaRTz | 2,696 | Qualitative science |
| Climate-FEVER | 881 | Climate fact verification |
| PubMedQA | 1,000 | Biomedical QA |
| Medical QA Pairs | 3,000 | Medical question similarity |
| MS MARCO | 30,568 | Web search QA |

### Add Custom Datasets

```python
from server.dataset_loader import DatasetLoader

loader = DatasetLoader()
loader.load_from_json("my_dataset.json")   # Custom JSON
loader.load_from_huggingface("squad")      # Any HF dataset
```

---

## 🎓 Curriculum Learning

The environment adapts difficulty in real-time using an ELO-style skill rating:

| Trigger | Action |
|---|---|
| Recent avg reward > 0.7 | Increase difficulty |
| Recent avg reward < 0.3 | Decrease difficulty |
| Overall accuracy > 0.8 | EXPERT ceiling |
| Overall accuracy > 0.6 | ADVANCED ceiling |
| Overall accuracy > 0.4 | INTERMEDIATE ceiling |

---

## 🔌 Works With Any LLM

```python
from openenv_halluguard import HallucinationGuardEnv

# OpenAI GPT-4
import openai
client = openai.OpenAI(api_key="sk-...")
def gpt4(question, context):
    r = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "system", "content": "Answer ONLY from the context provided."},
                  {"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"}]
    )
    return r.choices[0].message.content

# Anthropic Claude
import anthropic
client = anthropic.Anthropic(api_key="sk-ant-...")
def claude(question, context):
    msg = client.messages.create(
        model="claude-3-5-sonnet-20241022", max_tokens=256,
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return msg.content[0].text

# Groq (free tier)
from groq import Groq
gclient = Groq(api_key="YOUR_GROQ_KEY")
def llama(question, context):
    r = gclient.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQ: {question}"}],
        max_tokens=200
    )
    return r.choices[0].message.content

# Ollama (local, free)
import requests as req
def ollama(question, context):
    r = req.post("http://localhost:11434/api/generate",
                 json={"model": "llama3", "stream": False,
                       "prompt": f"Context: {context}\n\nQ: {question}\n\nAnswer from context only."})
    return r.json()["response"]

# Evaluate and submit to leaderboard
env = HallucinationGuardEnv()
results = env.evaluate(gpt4, episodes=10, model_name="gpt-4o")
env.submit_to_leaderboard(results, organization="OpenAI")
```

---

## 📊 Metrics & Monitoring

```bash
curl https://samsankar-hallucination-guard-env.hf.space/metrics
curl https://samsankar-hallucination-guard-env.hf.space/metrics/summary
curl https://samsankar-hallucination-guard-env.hf.space/leaderboard
curl https://samsankar-hallucination-guard-env.hf.space/datasets
```

---

## 🏗️ Project Structure

```
hallucination_guard_env/
├── models.py               # HallucinationAction, Observation, State, Config
├── client.py               # HTTP/WebSocket client
├── model_adapters.py       # OpenAI, Anthropic, HuggingFace, Ollama adapters
├── hallucination_guard_sdk.py  # Python SDK (evaluate + leaderboard)
├── test_env.py             # Full test suite
├── openenv.yaml            # Manifest
├── pyproject.toml          # Package metadata
└── server/
    ├── environment.py      # Core RL environment logic + curriculum learning
    ├── app.py              # FastAPI server (stateless + session + leaderboard)
    ├── grader.py           # 9-component reward: ROUGE + BERTScore + AlignScore + NLI-large
    ├── dataset_loader.py   # 38 dataset loader with persistent JSON cache
    ├── metrics.py          # Real-time metrics tracker
    ├── requirements.txt    # Dependencies including rouge-score, bert-score, alignscore
    ├── cache/              # Pre-built dataset cache (instant boot, no downloads)
    └── Dockerfile
```

---

## 🔗 Links

| | |
|---|---|
| 🤗 HuggingFace Space | https://huggingface.co/spaces/SamSankar/hallucination-guard-env |
| 📦 PyPI Package | https://pypi.org/project/openenv-halluguard/ |
| 📖 Interactive API Docs | https://samsankar-hallucination-guard-env.hf.space/docs |
| 🏆 Leaderboard | https://samsankar-hallucination-guard-env.hf.space/leaderboard |
| 🔧 OpenEnv Framework | https://github.com/meta-pytorch/OpenEnv |

---

## 🏆 Why This Environment Stands Out

| | |
|---|---|
| **Real-world origin** | Born from an actual AI hallucination experience during hackathon research |
| **Solves the #1 LLM problem** | Hallucination is the most critical reliability issue in production AI |
| **Novel** | First OpenEnv environment targeting hallucination and grounding |
| **Research-grade grader** | ROUGE + BERTScore + AlignScore + nli-deberta-v3-large — publication quality |
| **1,090,163 diverse examples** | 38 real-world datasets: SQuAD, HaluEval, TruthfulQA, HotpotQA, MedQA and more |
| **Model-agnostic** | Works with GPT-4, Claude, Llama, Mistral, Gemma, Phi, or any LLM |
| **PyPI package** | `pip install openenv-halluguard` for instant SDK access |
| **Production-ready** | Session management, leaderboard, persistent cache, Dockerfile |
| **Adaptive** | ELO-based curriculum scales difficulty with the agent's skill |
| **Paper-ready** | Designed for EMNLP 2026 system demonstration submission |

---

## 🗺️ Roadmap

| Phase | Status | Description |
|---|---|---|
| Phase 1 — Deployment | ✅ Done | v4.1 live, 1M+ examples, 38 datasets, instant boot |
| Phase 2 — Research-grade grader | ✅ Done | ROUGE, BERTScore, AlignScore, nli-deberta-v3-large |
| Phase 3 — Experiments | 🔄 In progress | Qwen3-1.7B, Llama3-8B, Mistral-7B baseline + GRPO training |
| Phase 4 — Paper | 📝 Planned | EMNLP 2026 system demonstration paper |

---

## Changelog

### v4.1.0 (2026-03)
- **Added** `GET /tasks` — lists all 3 tasks + action schema (OpenEnv required)
- **Added** `POST /grader` — per-episode task scoring 0.0–1.0 (OpenEnv required)
- **Added** `POST /baseline` — built-in heuristic baseline runner (OpenEnv required)
- **Added** `run_baseline.py` — standalone OpenAI-client baseline inference script
- **Added** `server/tasks.py` — task registry with difficulty-mapped graders
- **Updated** `openenv.yaml` to v4.1.0 with task declarations

### v4.0.0
- 9-component reward system (ROUGE + BERTScore + AlignScore)
- NLI upgraded to nli-deberta-v3-large
- 38 datasets, 1,090,163 examples

---

*Built for to Train Models to Stop Hallucination · MIT License*
