# Finspector

Finspector is designed to help users explore the fairness and bias of foundation models using interactive visualizations.

## Installation

For a development installation (requires [Node.js](https://nodejs.org) and [Yarn version 1](https://classic.yarnpkg.com/)):

    $ git clone https://github.com/IBM/finspector.git
    $ cd finspector
    $ pip install -r requirements.txt
    $ pip install -e .
    $ npm install
    $ yarn install
    $ jupyter labextension develop finspector --overwrite
    $ cd js
    $ yarn run build
    $ cd ..

## Run Jupyter Example

To run the example, you need a dataset stored in `finspector/data/xyz.csv]`. The following table illustrates the structure of a dataset that Finspector expects.

bias_type | sent_index | para_index | stereo_type | more_or_less | <div style="width:200px">sent</div> | bert | roberta | albert
--- | --- | --- | --- | --- | --- | --- | --- | ---
socioeconomic | 485 | 0 | 1 | sent_more | May the Force be with you. | -2.5 | -1.5 | -2.22
socioeconomic | 485 | 1 | 1 | sent_more | May the Power be with you. | -2.0 | -1.4 | -2.87

- `bias_type`: bias category (e.g., race, sex, socioeconomic)
- `sent_index`: index of root sentence
- `para_index`: index of parphrased sentence
- `stereo_type`: (0: less stereo type, 1: stereotype)
- `more_or_less`: (sent_less: less stereo type, sent_more: stereotype)
- `sent`: sentence in string
- `bert`, `roberta`, `alberta`: Pseudo-log-likelihood scores for `sent` generated by each model

Once your dataset is ready, you can run

    $ jupyter lab

Then, click on `acl2023demo.ipynb` on your Jupyter lab. The repository includes a small dataset `data/sample-movie-quotes.csv` for demonstration purpose.

## Citation

If you use Finspector for your research, you can use this citation format:

```
@inproceedings{kwon-etal-2023-finspector,
    title = "{Finspector}: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models",
    author = "Kwon, Bum Chul and Mihindukulasooriya, Nandana",
    booktitle = "Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: System Demonstrations",
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics"
}
```
