Metadata-Version: 2.1
Name: moval
Version: 0.3.17
Summary: Model evaluation without manual labels
Home-page: https://github.com/ZerojumpLine/MOVAL
Author: Zeju Li
Author-email: lizeju8@gmail.com
Project-URL: Bug Tracker, https://github.com/ZerojumpLine/MOVAL/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: GPU :: NVIDIA CUDA
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: Free for non-commercial use
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: scipy>=1.8.0
Requires-Dist: pytest
Requires-Dist: gdown
Requires-Dist: pandas
Requires-Dist: nibabel

<h1 align="center">
  <a href="https://zerojumpline.github.io/moval">
    <img src="https://github.com/ZerojumpLine/MOVAL/blob/main/docs/logo.png?raw=True" alt="Logo" width="180" height="180">
  </a>
  <br/>
  MOVAL
</h1>

<p align="center"><strong>Estimating performance for safe deployment of machine learning models</strong></p>


<div align="center">

[📖 Documentation](https://moval.readthedocs.io/en/latest/index.html) |
[🏠 Home Page](https://zerojumpline.github.io/moval)


[![PyPI version](https://badge.fury.io/py/moval.svg)](https://badge.fury.io/py/moval)


</div>



**MOVAL** is a Python package designed for assessing model performance in the absence of ground truth labels. It computes and calibrated confidence scores to accurately reflect the likelihood of predictions, leveraging these calibrated confidence scores to estimate the model's overall performance. Notably, MOVAL operates without the need for ground truth labels in the target domains and supports the evaluation of model performance in classification, 2D segmentation, and 3D segmentation.

**MOVAL** highlights a key feature—class-wise calibration, recognized as essential for addressing long-tailed distributions commonly found in real-world datasets. This proves especially significant in segmentation tasks where background samples often outnumber foregrounds. The inclusion of class-specific variants becomes crucial for accurately estimating segmentation performance. Additionally, MOVAL offers support for various types of confidence scores, enhancing its versatility.

What it offers:
<br/> <div align=center><img src="https://github.com/ZerojumpLine/MOVAL/blob/main/docs/software_features.png?raw=True" width="600px"/></div>

## User Document

The latest documentation can be found [here](https://moval.readthedocs.io/en/latest/index.html).

## Reference

```
@inproceedings{li2022estimating,
  title={Estimating model performance under domain shifts with class-specific confidence scores},
  author={Li, Zeju and Kamnitsas, Konstantinos and Islam, Mobarakol and Chen, Chen and Glocker, Ben},
  booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
  pages={693--703},
  year={2022},
  organization={Springer}
}
```
