Metadata-Version: 2.4
Name: raglint
Version: 0.1.0
Summary: Production-ready RAG pipeline evaluation and observability platform
Author-email: RAGlint Team <maintainers@raglint.dev>
License: MIT
Project-URL: Homepage, https://github.com/raglint/raglint
Project-URL: Bug Tracker, https://github.com/raglint/raglint/issues
Project-URL: Repository, https://github.com/raglint/raglint
Project-URL: Changelog, https://github.com/raglint/raglint/blob/main/CHANGELOG.md
Keywords: rag,llm,evaluation,observability,monitoring,langchain,openai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.0.0
Requires-Dist: anthropic>=0.18.0
Requires-Dist: requests>=2.31.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: click>=8.1.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: jinja2>=3.1.0
Requires-Dist: pip>=25.3
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=10.0.0
Requires-Dist: aiohttp>=3.8.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn>=0.20.0
Requires-Dist: sqlalchemy>=2.0.0
Requires-Dist: aiosqlite>=0.19.0
Requires-Dist: python-multipart>=0.0.6
Requires-Dist: passlib[bcrypt]>=1.7.4
Requires-Dist: python-jose[cryptography]>=3.3.0
Requires-Dist: scikit-learn
Requires-Dist: sentence-transformers
Requires-Dist: numpy
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: black>=23.7.0; extra == "dev"
Requires-Dist: ruff>=0.0.285; extra == "dev"
Requires-Dist: pre-commit>=3.3.3; extra == "dev"
Requires-Dist: pip-audit>=2.6.0; extra == "dev"
Requires-Dist: bandit[toml]>=1.7.5; extra == "dev"
Requires-Dist: psutil>=5.9.0; extra == "dev"
Provides-Extra: integrations
Requires-Dist: langchain>=0.1.0; extra == "integrations"
Requires-Dist: langchain-community>=0.0.10; extra == "integrations"
Requires-Dist: llama-index>=0.9.0; extra == "integrations"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=2.0; extra == "docs"
Requires-Dist: myst-parser>=2.0; extra == "docs"
Provides-Extra: all
Requires-Dist: raglint[dev,docs,integrations]; extra == "all"
Dynamic: license-file

# 🔍 RAGLint

**The Honest RAG Evaluation Platform**

[![PyPI version](https://badge.fury.io/py/raglint.svg)](https://badge.fury.io/py/raglint)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![Code Coverage](https://img.shields.io/badge/coverage-27%25-yellow.svg)](https://github.com/yourusername/raglint)

RAGLint is the **all-in-one evaluation platform** for Retrieval-Augmented Generation (RAG) systems. Built with production in mind, it provides comprehensive metrics, a beautiful dashboard, and enterprise-grade security.

## ✨ Why RAGLint?

| Feature | RAGLint | Ragas | Arize Phoenix | DeepEval |
|---------|---------|-------|---------------|----------|
| **Plugin Marketplace** | ✅ Unique | ❌ | ❌ | ❌ |
| **Local LLM (Ollama)** | ✅ Native | ⚠️ Partial | ⚠️ Partial | ⚠️ Partial |
| **Dashboard** | ✅ Excellent | ⚠️ Basic | ✅ Excellent | ⚠️ Good |
| **Real-time Analytics** | ✅ WebSockets | ❌ | ✅ | ❌ |
| **Security** | ✅ High | ⚠️ Moderate | ✅ High | ⚠️ Moderate |
| **CI/CD Integration** | ✅ Native Action | ⚠️ Manual | ✅ | ⚠️ Manual |
| **Open Source** | ✅ MIT | ✅ Apache | ⚠️ Partial | ✅ MIT |

## 🚀 Quick Start

```bash
# Install
pip install raglint

# Analyze your RAG pipeline
raglint analyze data.json --smart --provider ollama

# Launch dashboard
raglint dashboard
```

## 📊 Features

### Core Metrics
- **Faithfulness**: Verify answers match source documents
- **Relevance**: Context & answer relevance scoring
- **Precision/Recall**: Retrieval quality metrics
- **Bias Detection**: Identify biased responses
- **Toxicity**: Safety & appropriateness checks
- **PII Detection**: Security & privacy compliance

### Advanced Analytics
- **Drift Detection**: Monitor metric changes over time
- **Cohort Analysis**: Compare configurations
- **Embedding Visualization**: UMAP projections
- **Real-time Updates**: WebSocket-powered dashboard

### Integrations
- ✅ OpenAI, Azure OpenAI, AWS Bedrock
- ✅ Ollama (local LLM)
- ✅ LangChain, LlamaIndex, Haystack
- ✅ Pinecone, Weaviate, Chroma, Qdrant

## 💡 Example

```python
from raglint.core import RAGPipelineAnalyzer

# Your RAG data
data = [
    {
        "query": "What is machine learning?",
        "retrieved_contexts": ["ML is a subset of AI..."],
        "response": "Machine learning is..."
    }
]

# Analyze
analyzer = RAGPipelineAnalyzer(use_smart_metrics=True)
results = analyzer.analyze(data)

print(f"Faithfulness: {results.faithfulness_scores[0]:.2f}")
print(f"Relevance: {results.semantic_scores[0]:.2f}")
```

## 🔌 Plugin Marketplace

Extend RAGLint with custom metrics:

```bash
# Browse plugins
raglint plugins list

# Install a plugin
raglint plugins install raglint-pii-advanced

# Create your own
raglint plugins create my-custom-metric
```

**Built-in Plugins:**
- Citation Accuracy Checker
- PII Detector
- SQL Injection Detector
- Bias Detector
- Hallucination Confidence

## 🎯 CI/CD Integration

```yaml
# .github/workflows/raglint.yml
- name: RAGLint Analysis
  uses: yourusername/raglint-action@v1
  with:
    data-file: 'eval_data.json'
    threshold: 0.7
```

## 📈 Dashboard

Beautiful, real-time analytics dashboard:

- 📊 Interactive charts & visualizations
- 🔍 Drill-down into individual runs
- 📉 Drift detection & alerting
- 🎨 Dark mode & modern UI
- 🚀 WebSocket real-time updates

```bash
raglint dashboard
# Open http://localhost:8000
```

## 🔐 Enterprise-Ready

- **Security**: RestrictedPython sandbox, code signing
- **Scalability**: PostgreSQL support, async processing
- **Privacy**: Self-hosted, local LLM support
- **Compliance**: PII detection, audit logs

## 📚 Documentation

- [Quick Start Guide](docs/QUICKSTART.md)
- [API Reference](docs/API.md)
- [Cookbooks](docs/cookbooks/)
  - [Pinecone Integration](docs/cookbooks/pinecone_integration.md)
  - [LlamaIndex Advanced](docs/cookbooks/llamaindex_advanced.md)
  - [Haystack Integration](docs/cookbooks/haystack_advanced.md)

## 🤝 Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## 📄 License

MIT License - see [LICENSE](LICENSE) for details.

## ⭐ Star Us!

If you find RAGLint useful, please consider starring the repo to help others discover it!

---

**Made with ❤️ by the RAGLint team** | [Website](https://raglint.io) | [Discord](https://discord.gg/raglint) | [Twitter](https://twitter.com/raglint)
