# Evaluatr


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

[![PyPI](https://img.shields.io/pypi/v/evalstack.png)](https://pypi.org/project/evalstack/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Documentation](https://img.shields.io/badge/docs-GitHub%20Pages-blue.png)](https://franckalbinet.github.io/evalstack/)

## What is Evaluatr?

`Evaluatr` is an AI-powered system that automates the creation of
**Evidence Maps** - structured, visual tools that organize what we know
(and don’t know) about programs, policies, and interventions. It
transforms hundreds of hours of manual document review into an
automated, intelligent process.

### The Challenge We Solve

Traditional evidence mapping requires hundreds of staff-hours manually
reviewing and tagging evaluation documents. Evaluatr automates this
process while maintaining **accuracy** through human-AI collaboration,
making evidence mapping accessible and efficient.

## Key Features

### Document Processing

- **Multi-Format Repository Reading**: Seamlessly process evaluation
  repositories from diverse organizations with standardized outputs
- **OCR Processing**: Extract text from scanned PDFs and images to
  ensure no valuable information is missed
- **Intelligent Document Chunking**: Break down documents into
  meaningful segments for optimal analysis

### AI-Powered Analysis

- **Automated Information Extraction**: Extract key program details,
  context, and findings using advanced AI models
- **Smart Document Tagging**: AI-assisted categorization and labeling of
  evaluation reports
- **Enhanced RAG Implementation**: Enriched chunking strategy for more
  accurate retrieval and generation

### Flexibility & Openness

- **Framework Agnostic Design**: Compatible with multiple evaluation
  frameworks including IOM Strategic Results Framework and Global
  Compact on Migration
- **Open Training Data**: Contribute to and benefit from
  community-curated training datasets

## ️ Installation

### From PyPI (Recommended)

``` bash
pip install evaluatr
```

### From GitHub

``` bash
pip install git+https://github.com/franckalbinet/evaluatr.git
```

### Development Installation

``` bash
# Clone the repository
git clone https://github.com/franckalbinet/evaluatr.git
cd evaluatr

# Install in development mode
pip install -e .

# Make changes in nbs/ directory, then compile:
nbdev_prepare
```

## Quick Start

### Reading an IOM Evaluation Repository

``` python
from evaluatr.readers import IOMRepoReader

# Initialize reader with your Excel file
reader = IOMRepoReader('files/test/eval_repo_iom.xlsx')

# Process the repository
evaluations = reader()

# Each evaluation is a standardized dictionary
for eval in evaluations[:3]:  # Show first 3
    print(f"ID: {eval['id']}")
    print(f"Title: {eval['meta']['Title']}")
    print(f"Documents: {len(eval['docs'])}")
    print("---")
```

    ID: 1a57974ab89d7280988aa6b706147ce1
    Title: EX-POST EVALUATION OF THE PROJECT:  NIGERIA: STRENGTHENING REINTEGRATION FOR RETURNEES (SRARP)  - PHASE II
    Documents: 2
    ---
    ID: c660e774d14854e20dc74457712b50ec
    Title: FINAL EVALUATION OF THE PROJECT: STRENGTHEN BORDER MANAGEMENT AND SECURITY IN MALI AND NIGER THROUGH CAPACITY BUILDING OF BORDER AUTHORITIES AND ENHANCED DIALOGUE WITH BORDER COMMUNITIES
    Documents: 2
    ---
    ID: 2cae361c6779b561af07200e3d4e4051
    Title: Final Evaluation of the project "SUPPORTING THE IMPLEMENTATION OF AN E RESIDENCE PLATFORM IN CABO VERDE"
    Documents: 2
    ---

Exporting it to JSON:

    reader.to_json('processed_evaluations.json')

## Downloading evaluation documents

``` python
from evaluatr.downloaders import download_docs
from pathlib import Path

fname = 'files/test/evaluations.json'
base_dir = Path("files/test/pdf_library")
download_docs(fname, base_dir=base_dir, n_workers=0, overwrite=True)
```

    (#24) ['Downloaded Internal%20Evaluation_NG20P0516_MAY_2023_FINAL_Abderrahim%20EL%20MOULAT.pdf','Downloaded RR0163_Evaluation%20Brief_MAY_%202023_Abderrahim%20EL%20MOULAT.pdf','Downloaded IB0238_Evaluation%20Brief_FEB_%202023_Abderrahim%20EL%20MOULAT.pdf','Downloaded Internal%20Evaluation_IB0238__FEB_2023_FINAL%20RE_Abderrahim%20EL%20MOULAT.pdf','Downloaded IB0053_Evaluation%20Brief_SEP_%202022_Abderrahim%20EL%20MOULAT.pdf','Downloaded Internal%20Evaluation_IB0053_OCT_2022_FINAL_Abderrahim%20EL%20MOULAT_0.pdf','Downloaded Internal%20Evaluation_NC0030_JUNE_2022_FINAL_Abderrahim%20EL%20MOULAT_0.pdf','Downloaded NC0030_Evaluation%20Brief_June%202022_Abderrahim%20EL%20MOULAT.pdf','Downloaded CD0015_Evaluation%20Brief_May%202022_Abderrahim%20EL%20MOULAT.pdf','Downloaded Projet%20CD0015_Final%20Evaluation%20Report_May_202_Abderrahim%20EL%20MOULAT.pdf','Downloaded Internal%20Evaluation_Retour%20Vert_JUL_2021_Fina_Abderrahim%20EL%20MOULAT.pdf','Downloaded NC0012_Evaluation%20Brief_JUL%202021_Abderrahim%20EL%20MOULAT.pdf','Downloaded Nigeria%20GIZ%20Internal%20Evaluation_JANUARY_2021__Abderrahim%20EL%20MOULAT.pdf','Downloaded Nigeria%20GIZ%20Project_Evaluation%20Brief_JAN%202021_Abderrahim%20EL%20MOULAT_0.pdf','Downloaded Evaluation%20Brief_ARCO_Shiraz%20JERBI.pdF','Downloaded Final%20evaluation%20report_ARCO_Shiraz%20JERBI_1.pdf','Downloaded Management%20Response%20Matrix_ARCO_Shiraz%20JERBI.pdf','Downloaded IOM%20MANAGEMENT%20RESPONSE%20MATRIX.pdf','Downloaded IOM%20Niger%20-%20MIRAA%20III%20-%20Final%20Evaluation%20Report%20%28003%29.pdf','Downloaded CE.0369%20-%20IDEE%20-%20ANNEXE%201%20-%20Rapport%20Recherche_Joanie%20DUROCHER_0.pdf'...]

## Documentation

- **Full Documentation**: [GitHub
  Pages](https://fr.anckalbi.net/evalstack/)
- **API Reference**: Available in the documentation
- **Examples**: See the `nbs/` directory for Jupyter notebooks

## Contributing

We welcome contributions! Here’s how you can help:

1.  **Fork** the repository
2.  **Create** a feature branch
    (`git checkout -b feature/amazing-feature`)
3.  **Make** your changes in the `nbs/` directory
4.  **Compile** with `nbdev_prepare`
5.  **Commit** your changes (`git commit -m 'Add amazing feature'`)
6.  **Push** to the branch (`git push origin feature/amazing-feature`)
7.  **Open** a Pull Request

### Development Setup

``` bash
# Install development dependencies
pip install -e .

# Make changes in nbs/ directory
# ...

# Compile changes to evalstack package
nbdev_prepare
```

## License

This project is licensed under the MIT License - see the
[LICENSE](LICENSE) file for details.

## Acknowledgments

- Built with [nbdev](https://nbdev.fast.ai/) for literate programming
- Uses [pandas](https://pandas.pydata.org/) for data processing
- Powered by [rich](https://rich.readthedocs.io/) for beautiful terminal
  output

## Support

- **Issues**: [GitHub
  Issues](https://github.com/franckalbinet/evalstack/issues)
- **Discussions**: [GitHub
  Discussions](https://github.com/franckalbinet/evalstack/discussions)
- **Email**: \[Your email here\]
