# P-TAME

<p align="center">
  <img width="350" src="https://github.com/IDT-ITI/P-TAME/blob/main/ptame_logo.png">
</p>
<!--<h1 align="center"><b>Quantus</b></h1>-->
<h3 align="center"><b>Explain Any Image Classifier with Trained Perturbations</b></h3>
<p align="center">

[![arXiv](https://img.shields.io/badge/arXiv-2501.17813-b31b1b.svg?style=flat)](https://arxiv.org/abs/2501.17813)
[![python](https://img.shields.io/badge/-Python_3.10-blue?logo=python&logoColor=white)](https://github.com/pre-commit/pre-commit)
[![pytorch](https://img.shields.io/badge/PyTorch_2.0+-ee4c2c?logo=pytorch&logoColor=white)](https://pytorch.org/get-started/locally/)
[![lightning](https://img.shields.io/badge/-Lightning_2.0+-792ee5?logo=pytorchlightning&logoColor=white)](https://pytorchlightning.ai/)
[![hydra](https://img.shields.io/badge/Config-Hydra_1.3-89b8cd)](https://hydra.cc/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

_Don't hesitate to open a new issue if you encounter problems while using this method!_

## Table of contents

- [Introduction](#introduction)
- [Installation](#installation)
  - [Package requirements](#package-requirements)
- [Project structure](#project-structure)
- [Tutorials](#tutorials)
- [Citation](#citation)
- [Acknowledgement](#acknowledgement)
- [License](#license)

## Introduction

The adoption of Deep Neural Networks (DNNs) in critical fields where predictions need justifications is limited by their black-box nature. P-TAME (Perturbation-based Trainable Attention Mechanism for Explanations) is a model-agnostic method for explaining DNN-based image classifiers. It uses an auxiliary image classifier to extract features from the input image, bypassing the need to modify the explanation method to the architecture of the classifier being explained. Unlike traditional perturbation-based methods, P-TAME  generates high-resolution explanations efficiently in a single forward pass during inference. We demonstrate its performance by applying it to VGG-16, ResNet-50, and ViT-B-16. The quantitative and qualitative results included in our work can be reproduced using this code, as explained below. Additionally, the procedure to apply this method to any classifier is also described.

## Installation

If you already have [PyTorch](https://pytorch.org/) installed on your machine,
the latest version of P-TAME can be obtained from [PyPI](https://pypi.org/project/ptame/) as follows:

```bash
pip install ptame
```

### Package requirements

The package requirements are as follows:

```text
python>=3.10.0
torch>=2.0.0
```

## Project structure

The directory structure of new project looks like this:

```tree
├── data               <- Project data
│   └── datalists         <- Validation and test subsets
├── logs               <- Logs generated by loggers
├── README.md
├── requirements.txt   <- File for installing python dependencies
├── setup.py           <- File for installing project as a package
└── src                <- Source code
    └── ptame
        ├── configs          <- Hydra configs
        │   ├── callbacks
        │   ├── data
        │   ├── debug
        │   ├── experiment      <- Experiment configs
        │   ├── extras
        │   ├── hparams_search  <- Hyperparameter search configs
        │   ├── hydra
        │   ├── local
        │   ├── logger
        │   ├── model           <- Model configs
        │   ├── paths
        │   ├── trainer
        │   ├── print_map.yaml
        │   ├── ax_sweep.yaml
        │   └── train.yaml
        ├── data             <- Data module
        ├── models           <- Model module
        ├── print_map.py     <- Script for printing explanation maps
        ├── ax_sweep.py      <- Script for hyperparameter optimization
        ├── train.py         <- Script for training
        └── utils            <- Utility module
```

## Tutorials

To learn how to reproduce the results in tables 1 and 2 of the [paper](https://arxiv.org/abs/2501.17813), and how to apply the method to classifiers besides the ones included in the experiments, refer to this page in the [documentation](<>).

## Citation

If you fine this XAI explainability method interesting or useful in your research, use the following Bibtex annotation to cite us:

```bibtex
@misc{ntrougkas2025ptameexplainimageclassifier,
      title={P-TAME: Explain Any Image Classifier with Trained Perturbations},
      author={Mariano V. Ntrougkas and Vasileios Mezaris and Ioannis Patras},
      year={2025},
      eprint={2501.17813},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2501.17813},
}
```

## Acknowledgement

The P-TAME repository was built using the [lightning-hydra-template](https://github.com/ashleve/lightning-hydra-template). The results of the comparison study are obtained using the following implementations:

- The [pytorch-gradcam](https://github.com/jacobgil/pytorch-grad-cam) library's implementations of the Grad-CAM, Grad-CAM++, Score-CAM, and Ablation-CAM methods.
- The [original](https://github.com/eclique/RISE) implementation of the RISE method.
- The [original](https://github.com/hanwei0912/OptiCAM) implementation of the Opti-CAM method.
- The [original](https://github.com/IDT-ITI/T-TAME) implementation of the T-TAME method.

The weights for the pretrained ResNet-50, VGG-16, ViT-B-16, ResNet-18, MobileNet-V3, and MnasNet were taken from [torchvision.models](https://pytorch.org/vision/stable/models.html).

This work was supported by the EU Horizon 2020 programme under grant agreement 101070190 AI4TRUST.

## License

This code is provided for academic, non-commercial use only. Please also check for any restrictions applied in the code parts and datasets used here from other sources. For the materials not covered by any such restrictions, redistribution and use in source and binary forms, with or without modification, are permitted for academic non-commercial use provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation provided with the distribution.

This software is provided by the authors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the authors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.
