# Entropy-RAG
Entropy-Regulated Retrieval-Augmented Reasoning
Author: Steven Reid (RRG314 Research Group)
License: Apache 2.0
Version: 1.0.0

---

## Overview

Entropy-RAG is a retrieval-augmented reasoning (RAG) framework based on physical and mathematical principles of entropy regulation.
Conventional RAG systems retrieve the top-k most similar documents using cosine similarity alone, which often leads to *semantic collapse* — overly narrow or repetitive results.

Entropy-RAG models retrieval as an entropy-regulated diffusion process, balancing focus and diversity using an adaptive coupling parameter Ω (Omega).
This allows retrieval to behave like a self-stabilizing physical system, maintaining semantic equilibrium across reasoning tasks.

---

## Theoretical Basis

The system draws from:
- Recursive Entropic Field Theory (REFT): entropy as a regulator of recursive system stability
- RDT Kernel: nonlinear diffusion equation ∂L/∂t = -α·log(L) + D·∇²L
- Topological Adam: optimizer coupling gradients through energy feedback

Entropy-RAG treats retrieval as a diffusion field evolving toward equilibrium, governed by information entropy rather than temperature or noise.

---

## Design Principles

Entropy-RAG is not a neural network itself.
It is a modular retrieval and reasoning layer that can be attached to any language model (Flan-T5, LLaMA, GPT, etc.).
Its role is to maintain stability and diversity in reasoning chains.

Main design goals:
- Preserve contextual diversity while staying relevant
- Prevent mode collapse in long or multi-hop retrievals
- Quantify semantic entropy in the information field
- Create a bridge between physics-based and AI-based reasoning

---

## Features

- Entropy-balanced retrieval with adaptive Ω regulation
- Topic diversity control using entropy feedback
- Works with Hugging Face sentence-transformers
- Evaluation metrics: semantic diversity, lexical entropy, coherence
- Fully differentiable and compatible with fine-tuning pipelines

---

## Installation

```bash
git clone https://github.com/rrg314/entropy-rag.git
cd entropy-rag
pip install -e .
```

---

## Quick Start Example

```python
from entropy_rag.entropy_index import build_index
from entropy_rag.retriever import EntropyBalancedSelector, EntropySelectorConfig

docs = [
    "Entropy stabilizes diffusion by regulating potential energy.",
    "Topological Adam introduces alpha-beta coupling in optimization.",
    "Nonlinear PDEs improve stability in differentiable physics."
]

index = build_index(docs, n_topics=3)
selector = EntropyBalancedSelector(index, EntropySelectorConfig(Omega_mode='median'))

results = selector.retrieve('how does entropy stabilize diffusion?', k=5)
for r in results:
    print('-', r)
```

---

## Evaluation Example

```python
from entropy_rag.evaluator import evaluate_entropy_rag

queries = [
    'how does entropy regulate diffusion?',
    'why are nonlinear PDEs important for stability?',
]

evaluate_entropy_rag(index, queries)
```

Example output:

| Metric | Mean | Description |
|:--|:--:|:--|
| Semantic Diversity | 0.659 | Topic variety |
| Lexical Entropy | 2.187 | Vocabulary balance |
| Context Coherence | 0.752 | Consistency of retrieved information |

---

## Research Context

Entropy-RAG extends physical entropy models to machine learning.
It combines energy-based optimization, diffusion physics, and information theory to form a retrieval engine that self-stabilizes like a physical field system.

Applications include:
- Physics-informed reasoning in neural architectures
- Long-context document retrieval
- Entropy-regulated AI systems for interpretability and balance

---

## Future Work

- Adaptive Ω-learning from corpus feedback
- Multi-modal (text + image) retrieval
- Symbolic reasoning integration
- Empirical benchmarking vs. standard RAG frameworks

---

## Citation

Reid, S. (2025). *Entropy-Regulated Retrieval-Augmented Reasoning (Entropy-RAG).*
RRG314 Research Group.

---

## License

Apache License 2.0 © 2025 Steven Reid
See LICENSE for details.