# Examples

Detailed, runnable examples are available as Jupyter Notebooks in the [`examples/`](https://github.com/ai4society/GenAIResultsComparator/tree/main/examples) folder:

- [`quickstart.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/quickstart.ipynb):

    Using GAICo's `Experiment` module to provide a simple, quickstart workflow.

- [`example-1.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/example-1.ipynb):

    Evaluating multiple models (LLMs, Google, and Custom) using a single metric.

- [`example-2.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/example-2.ipynb):

    Evaluating a single model on multiple metrics.

- [`DeepSeek-example.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/DeepSeek-example.ipynb)

    The aim for this notebook was to aid with evaluating DeepSeek R1 for [AI4Society's Point of View (POV)](https://drive.google.com/file/d/1ErR1xT7ftvmHiUyYrdUbjyd4qCK_FxKX/view?usp=sharing).


## Advanced Examples

The `advanced-examples` directory contains advances notebooks showcasing more complex use cases and metrics. These examples are intended for users who are already familiar with the basics of GAICo. Please refer to the README.md file in that directory for details. A quick description:

- [`llm_faq-example.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/advanced-examples/llm_faq-example.ipynb)

    Comparison of various LLM responses (Phi, Mixtral, etc.) on FAQ dataset from USC.

- [`threshold-example.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/advanced-examples/threshold-example.ipynb)

    Exploration of default and custom thresholding techniques for LLM responses.

- [`viz-example.ipynb`](https://github.com/ai4society/GenAIResultsComparator/blob/main/examples/advanced-examples/viz-example.ipynb)

    Hands-on visualizations for LLM results.
