Metadata-Version: 2.1
Name: vision-evaluation
Version: 0.1.0
Summary: Evaluation code for vision tasks.
Home-page: https://github.com/pj-ms/vision-evaluation
Author: Ping Jin, Shohei Ono, I-Ting Fang
License: MIT
Keywords: vision metric evaluation classification detection
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: numpy (~=1.18.3)
Requires-Dist: sklearn

# Introduction 
This repo contains evaluation metric codes used in Microsoft Cognitive Services Computer Vision for tasks such as classification and object detection.

# Functionalities
This repo currently offers evaluation metrics for two vision tasks:

- Image classification:
    - `evaluators.TopKAccuracyEvaluator`: computes the top-k accuracy, i.e., accuracy of the top k predictions with highest confidence.
    - `evaluators.AveragePrecisionEvaluator`: computes the average precision, precision averaged across different confidence thresholds.
    - `evaluators.ThresholdAccuracyEvaluator`: computes the threshold based accuracy, i.e., accuracy of the predictions with confidence over a certain threshold.
    - `evaluators.EceLossEvaluator`: computes the [ECE loss](https://arxiv.org/pdf/1706.04599.pdf), i.e., the expected calibration error, given the model confidence and true labels for a set of data points. 
- Object detection:
    - `evaluators.MeanAveragePrecisionEvaluatorForSingleIOU`, `evaluators.MeanAveragePrecisionEvaluatorForMultipleIOUs`: computes the mean average precision (mAP), i.e. mean average precision across different classes, under single or multiple [IoU(s)](https://en.wikipedia.org/wiki/Jaccard_index).


