
<div align="center">

<img src="https://raw.githubusercontent.com/lerdmann1601/HERA-Matlab/main/assets/hera_logo.svg" alt="HERA Logo" width="300"/>

# HERA: Hierarchical-Compensatory, Effect-Size-Driven Ranking Algorithm

[![MATLAB](https://img.shields.io/badge/MATLAB-R2025b-orange.svg)](https://www.mathworks.com/products/matlab.html)
[![File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/183089-hera)
[![View on PyPI](https://img.shields.io/badge/PyPI-View_on_PyPI-blue?logo=pypi)](https://pypi.org/project/hera-matlab/)
[![View on GitHub](https://img.shields.io/badge/GitHub-View_on_GitHub-blue?logo=github)](https://github.com/lerdmann1601/HERA-Matlab)
[![Issues](https://img.shields.io/github/issues/lerdmann1601/HERA-Matlab)](https://github.com/lerdmann1601/HERA-Matlab/issues)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.18274870-blue.svg)](https://doi.org/10.5281/zenodo.18274870)
[![ORCID](https://img.shields.io/badge/ORCID-0009--0009--3758--7363-green.svg)](https://orcid.org/0009-0009-3758-7363)

**Made for Scientific Benchmarking**

[Key Features](#key-features) • [Installation](#installation) • [Quick Start](#quick-start) • [Documentation](#documentation) • [Citation](#citation)

</div>

---

## Overview

**HERA** is a MATLAB-based scientific ranking framework for paired benchmarking,
designed to automate the objective comparison of algorithms, experimental
conditions, or other methods with repeated measurements across single or
multiple quality metrics.
Unlike traditional ranking methods that rely solely on mean values or
p-values, HERA employs a **hierarchical-compensatory logic** that integrates:

* **Significance Testing**: Wilcoxon signed-rank tests for paired data.
* **Effect Sizes**: Cliff's Delta and Relative Mean Difference for practical relevance.
* **Bootstrapping**: Data-driven thresholds and confidence intervals.

This ensures that a "win" is only counted if it is both **statistically significant** and
**practically relevant**, providing a robust and nuanced
ranking system.

For more information please refer to the Project
[Website](https://lerdmann1601.github.io/HERA-Matlab/).

---

## Key Features

* **Hierarchical-Compensatory Logic**: Define primary and secondary metrics
  for the sequential comparison. Secondary metrics can act as tie-breakers or
  iterative rank correctors (e.g., `M1_M2`, `M1_M2_M3`).
* **Data-Driven Thresholds**: Automatically calculates adaptive effect size
  thresholds via Percentile Bootstrapping, utilizing a dynamic SEM guardrail to
  maintain practical relevance relative to measurement noise.
* **Robustness**: Utilizes Bias-Corrected and Accelerated (BCa) confidence
  intervals for effect sizes and Cluster Bootstrapping using the percentile method
  for rank stability.
* **Automated Reporting**: Generates PDF reports with high-resolution graphics
  (e.g., Win-Loss matrices, Sankey diagrams) and machine-readable JSON/CSV
  exports including the complete analysis and statistics.
* **Reproducibility**: Features automated convergence control to determine
  optimal bootstrap iterations ($B$) without guesswork, alongside
  fixed-seed execution and configuration-based workflows.

---

## Installation

### Requirements

* **MATLAB** (R2020a or later Required)
* **Statistics and Machine Learning Toolbox** (Required)
* **Parallel Computing Toolbox** (Required for performance)

### Setup

#### Option A: MATLAB Toolbox (Recommended for MATLAB Users)

1. Download the latest `HERA_v1.4.1.mltbx` from the
   [Releases](https://github.com/lerdmann1601/HERA-Matlab/releases) page.
2. Double-click the file to install it.
3. Done! HERA is now available as a command (`HERA.start_ranking`) in MATLAB.

#### Option B: Git Clone (for Developers)

1. **Clone the repository:**

    ```bash
    git clone https://github.com/lerdmann1601/HERA-Matlab.git
    ```

2. **Install/Configure Path:**

    Navigate to the repository folder and run the setup script to add HERA to
    your MATLAB path.

    ```matlab
    cd HERA-Matlab
    setup_HERA
    ```

#### Option C: Standalone Runtime and Python Integration

👉 [Standalone Runtime](https://lerdmann1601.github.io/HERA-Matlab/Standalone_Runtime)

👉 [Python Integration](https://lerdmann1601.github.io/HERA-Matlab/Python_Integration)

---

## Quick Start

### 1. Interactive Mode (Recommended for Beginners)

The interactive command-line interface guides you through every step of the configuration,
from data selection to statistical parameters.
If you are new to HERA, this is the recommended mode.
At any point, you can exit the interface by typing `exit` or `quit` or `q`.

```matlab
HERA.start_ranking()
```

### 2. Batch Mode (Reproducible / Server)

For automated analysis or reproducible research, use a JSON configuration file.
For more details on configuration parameters, see [Configuration & Parameters](https://lerdmann1601.github.io/HERA-Matlab/Configuration_&_Parameters).

```matlab
HERA.start_ranking('configFile', 'config.json')
```

### 3. Unit Test Mode

Run the built-in validation suite to ensure HERA is working correctly on your system.
For more details, see the [Testing](#testing) section.

```matlab
% Run tests and save log to default location
HERA.start_ranking('runtest', 'true')

% Run tests and save log to a specific folder
HERA.start_ranking('runtest', 'true', 'logPath', '/path/to/logs')
```

### 4. Convergence Analysis

Perform a robust scientific validation of the default convergence parameters.
For more details, see [Convergence Analysis](https://lerdmann1601.github.io/HERA-Matlab/Convergence_Analysis).

```matlab
% Run analysis and save log to default location
HERA.start_ranking('convergence', 'true')

% Run analysis and save log to a specific folder
HERA.start_ranking('convergence', 'true', 'logPath', '/path/to/logs')

% Run analysis using a JSON configuration file
HERA.start_ranking('convergence', 'path/to/config.json')
```

---
> [!NOTE]
> Example use cases with synthetic datasets and results are
> provided in the `data/examples` directory. See [Example Analysis](https://lerdmann1601.github.io/HERA-Matlab/Example_Analysis) for a
> walkthrough of the example use cases and visual examples of the ranking
> outputs.
---
> [!NOTE]
> HERA is designed for high-performance scientific computing, featuring
> **fully parallelized bootstrap procedures** and **automatic memory management**
> to optimize efficiency. However, specifically due to the extensive use of
> bootstrapping, it remains a **CPU-intensive application**. Please ensure you
> have access to enough CPU cores for reasonable performance.
---

## Documentation

👉 [Version History (Changelog)](https://github.com/lerdmann1601/HERA-Matlab/blob/main/CHANGELOG.md)

👉 [Repository Structure](https://lerdmann1601.github.io/HERA-Matlab/Repository_Structure)

👉 [Theoretical Background](https://lerdmann1601.github.io/HERA-Matlab/Methodology)

👉 [Ranking Modes Explained](https://lerdmann1601.github.io/HERA-Matlab/Ranking_Modes_Explained)

👉 [Convergence Modes Explained](https://lerdmann1601.github.io/HERA-Matlab/Convergence_Modes)

👉 [Methodological Guidelines & Limitations](https://lerdmann1601.github.io/HERA-Matlab/Methodological_Guidelines_&_Limitations)

👉 [Example Analysis](https://lerdmann1601.github.io/HERA-Matlab/Example_Analysis)

👉 [Input Data Specification](https://lerdmann1601.github.io/HERA-Matlab/Input_Data_Specification)

👉 [Configuration & Parameters](https://lerdmann1601.github.io/HERA-Matlab/Configuration_&_Parameters)

👉 [Bootstrap Configuration](https://lerdmann1601.github.io/HERA-Matlab/Bootstrap_Configuration)

👉 [Convergence Analysis](https://lerdmann1601.github.io/HERA-Matlab/Convergence_Analysis)

👉 [Advanced Usage (MATLAB Users)](https://lerdmann1601.github.io/HERA-Matlab/Advanced_Usage)

👉 [Results Structure Reference](https://lerdmann1601.github.io/HERA-Matlab/Results_Structure_Reference)

---

## Outputs

HERA generates a timestamped directory containing:

```text
Ranking_<Timestamp>/
├── Output/
│   ├── results_*.csv                 % Final ranking table (Mean ± SD of metrics and rank CI)
│   ├── data_*.json                   % Complete analysis record (Inputs, Config, Stats, Results)
│   ├── log_*.csv                     % Detailed log of pairwise comparisons and logic
│   ├── sensitivity_details_*.csv     % Results of the Borda sensitivity analysis
│   ├── BCa_Correction_Factors_*.csv  % Correction factors (Bias/Skewness) for BCa CIs
│   └── bootstrap_rank_*.csv          % Complete distribution of bootstrapped ranks
├── Graphics/                         % High-res PNGs organized in subfolders
│   ├── Ranking/
│   ├── Detail_Comparison/
│   ├── CI_Histograms/
│   └── Threshold_Analysis/
├── PDF/                              % Specialized reports
│   ├── Ranking_Report.pdf
│   ├── Convergence_Report.pdf
│   └── Bootstrap_Report.pdf
├── Final_Ranking_*.png               % Summary graphic of ranking result
├── Final_Report_*.pdf                % Consolidated graphical report of the main results
├── Ranking_*.txt                     % Complete console log of the session
└── configuration.json                % Reusable configuration file to reproduce the run
```

---

## Testing

HERA includes a comprehensive validation framework (`run_unit_test.m`)
comprising **46 test cases** organized into four suites:

1. **Unit Tests** (19 cases): Checks individual components, helper functions, and
    execution logic (Run/Start packages) to ensure specific parts of the code work
    correctly.
2. **Statistical Tests** (5 cases): Verifies the core mathematical functions
    (e.g., Jackknife, Cliff's Delta) and ensures the performance optimizations
    (hybrid switching) work as intended.
3. **Scientific Tests** (19 cases): Comprehensive validation of ranking logic,
    statistical accuracy, and robustness against edge cases (e.g., zero
    variance, outliers).
4. **System Tests** (3 cases): Runs the entire HERA pipeline from start to
    finish to ensure that the JSON configuration (batch), Developer API and NaN
    Data handling are working correctly.

### Running Tests

You can run the test suite in three ways:

1. **Auto-Log Mode (Default)**
    Automatically finds a writable folder (e.g., Documents) to save the log
    file.

    ```matlab
    HERA.run_unit_test()
    ```

2. **Interactive Mode**
    Opens a dialog to select where to save the log file.

    ```matlab
    HERA.run_unit_test('interactive')
    ```

3. **Custom Path Mode**
    Saves the log file to a specific directory.

    ```matlab
    HERA.run_unit_test('/path/to/my/logs')
    ```

### GitHub Actions (Cloud Testing)

For reviewers or users without a local MATLAB license, you can run the test
suite directly on GitHub:

1. Go to the **Actions** tab in this repository.
2. Select **Testing HERA** from the left sidebar.
3. Click **Run workflow**.

---

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](https://github.com/lerdmann1601/HERA-Matlab/blob/main/CONTRIBUTING.md) for details.

1. Fork the repository.
2. Create a feature branch.
3. Commit your changes.
4. Open a Pull Request.

---

## Citation

If you use HERA in your research, please cite:

```bibtex
@software{HERA_Matlab,
  author = {von Erdmannsdorff, Lukas},
  title = {HERA: Hierarchical-Compensatory, Effect-Size-Driven
  Ranking Algorithm},
  url = {https://github.com/lerdmann1601/HERA-Matlab},
  version = {1.4.1},
  doi = {10.5281/zenodo.18274870},
  year = {2026}
}
```

---

## License

This project is licensed under the MIT License - see the [LICENSE](https://github.com/lerdmann1601/HERA-Matlab/blob/main/license.txt) file
for details.

---
