# EvalDNN

EvalDNN is an open-source toolbox for model evaluation of deep learning systems, supporting multiple frameworks and metrics.

Author: Yongqiang Tian, Zhihua Zeng, Ming Wen, Yepang Liu, Tzu-yang Kuo,
and Shing-Chi, Cheung.

The first two author contribute equally. 

This project is mainly supported by Microsoft Asia Cloud Research Software Fellow Award 2019



a photo of EvalDNN (overvierw5.pdf in bitbucket)



A video is here: the link to youtube



## Frameworks and Metrics
EvalDNN supports the model based on following frameworks:

- TensorFlow
- PyTorch
- Keras
- MXNet

EvalDNN supports the model based on following metrics:

- Top-K accuracy
- Neuron Coverage
- Robustness

## Usage

### Installation

pip install EvalDNN

### Evaluate a model

Show an simple example

### Extension

#### Add a new framework

#### Add a new metric





## Benchmark

The full benchmark is available here: a link to excel file

The code to reproduce the results in benchmark is in xxxx (a folder in this repo)