# LangDiversity

[![PyPI version](https://badge.fury.io/py/diversity-measures.svg)](LINK_TO_PYPI)
[![Python version](https://img.shields.io/badge/python-3.7%2B-blue)](LINK_TO_PYTHON_VERSION)
[![License](https://img.shields.io/badge/license-MIT-green)](LINK_TO_LICENSE)

Elevate your language models with insightful diversity metrics.

## Table of Contents

- [Introduction](#introduction)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Documentation](#documentation)
- [Contributing](#contributing)
- [Bibtex](#bibtex)
- [License](#license)
- [Contacts](#contacts)

## Introduction

LangDiversity is a package that provides tools to calculate diversity measures for a given set of data. Specifically, it can compute measures like Shannon's entropy and Gini impurity. It also offers utilities to select prompts based on their diversity scores when interacting with models like OpenAI's GPT-3.5 Turbo.

The primary goal of this project is to assist researchers and developers in analyzing the diversity of responses generated by language models, thereby aiding in the evaluation and fine-tuning of such models.

## Installation

```bash
pip install diversity-measures
```

## Quick Start

```python
from langdiversity.models import OpenAIModel
from langdiversity.measures import ShannonEntropyMeasure
from langdiversity.utils import PromptSelection

# Initialize the OpenAI model and diversity measure
model = OpenAIModel(openai_api_key="YOUR_OPENAI_API_KEY")
diversity_measure = ShannonEntropyMeasure()

# Use the PromptSelection utility
prompt_selection = PromptSelection(model=model, diversity_measure=diversity_measure)
selected_prompt, selected_diversity = prompt_selection.generate(["Your list of prompts here..."])

print("Selected Prompt:", selected_prompt)
print("Selected Diversity:", selected_diversity)
```

## Documentation

Detailed documentation is available [here]().

## Contributing

We welcome contributions! Please see our [contribution guidelines]() for more details.

## Bibtex

If you used this software in your work please cite our paper

```bibtex
@article{ngu2023diversity,
  title={Diversity Measures: Domain-Independent Proxies for Failure in Language Model Queries},
  author={Ngu, Noel and Lee, Nathaniel and Shakarian, Paulo},
  journal={arXiv preprint arXiv:2308.11189},
  year={2023},
  url={https://arxiv.org/abs/2308.11189},
}
```

## License

This repository is licensed under [BSD-3-Clause](LICENSE)

## Contacts

For any inquiries or feedback, please contact:

- Noel Ngu: [nngu2@asu.edu]()
- Nathaniel Lee: [nlee51@asu.edu]()
- Paulo Shakarian: [pshak02@asu.edu]()
