---
title: Examples
---

## Overview
An `Example` is a basic unit of data in `judgeval` that allows you to run evaluation scorers on your LLM system. 
An `Example` is composed of seven fields:
- `input`
- `actual_output`
- [Optional] `expected_output`
- [Optional] `retrieval_context`
- [Optional] `context`
- [Optional] `tools_called`
- [Optional] `expected_tools`

**Here's a sample of creating an `Example`:**

```python example_test.py
from judgeval.data import Example

example = Example(
    input="Who founded Microsoft?",
    actual_output="Bill Gates and Paul Allen.",
    expected_output="Bill Gates and Paul Allen founded Microsoft in New Mexico in 1975.",
    retrieval_context=["Bill Gates co-founded Microsoft with Paul Allen in 1975."],
    context=["Bill Gates and Paul Allen are the founders of Microsoft."],
    tools_called=["Google Search"],
    expected_tools=["Google Search", "Perplexity"],
)
```

<Note>
The `input` and `actual_output` fields are required for all examples. **However**, you don't need to use them in your evaluations. 

For example, if you're evaluating whether a chatbot's response is friendly, you don't need to use `input`. 

Other fields are optional and depend on the type of evaluation. If you want to detect hallucinations in a RAG system, you'd use the `retrieval_context` field for the [**Faithfulness**](/evaluation/scorers/faithfulness) scorer.
</Note>

## Example Fields 

Here, we cover the seven fields that make up an `Example`.

### Input 
The `input` field represents a sample interaction between a user and your LLM system. The input should represent the direct input to your prompt template(s), and **SHOULD NOT CONTAIN** your prompt template itself.

<Tip> 

Prompt templates are **hyperparameters** that you optimize for based on the scorer you're executing. 

Evaluation is always tied with optimization, so you should try to **isolate your system's independent variables** (e.g. prompt template, model, etc.) from your evaluation. 

</Tip>

### Actual Output 

The `actual_output` field represents what your LLM system outputs based on the `input`. 
This is the actual output of your LLM system created either at evaluation time or with saved answers.

```python actual_output.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question)
)
```

### Expected Output

The `expected_output` field is `Optional[str]` and represents the ideal output of your LLM system. 

One great part of `judgeval`'s scorers is that they use LLMs which have flexible evaluation criteria. You don't need to worry about your `expected_output` perfectly matching your `actual_output`!

To learn more about how `judgeval`'s scorers work, please see the [scorer docs](evaluation/scorers/introduction).

```python expected_output.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question),
    expected_output="Sparkling water is neither healthy nor unhealthy."
)
```

### Context 

The `context` field is `Optional[List[str]]` and represents information that is supplied to the LLM system as **ground truth**. 

For instance, context could be a list of facts that the LLM system is aware of. However, `context` should not be confused with `retrieval_context`.

<Tip>
In RAG systems, contextual information is retrieved from a vector database and is represented in `judgeval` by `retrieval_context`, 
not `context`. **If you're building a RAG system, you'll want to use `retrieval_context`.**
</Tip>

```python context.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question),
    expected_output="Sparkling water is neither healthy nor unhealthy.",
    context=["Sparkling water is a type of water that is carbonated."]
)
```

### Retrieval Context 

The `retrieval_context` field is `Optional[List[str]]` and represents the context that is actually retrieved from a vector database. 
This is often the context that is used to generate the `actual_output` in a RAG system.

<Tip>
Some common cases for using `retrieval_context` are:
- Checking for hallucinations in a RAG system
- Evaluating the quality of a retriever model by comparing `retrieval_context` to `context`
</Tip>

```python retrieval_context.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question),
    expected_output="Sparkling water is neither healthy nor unhealthy.",
    context=["Sparkling water is a type of water that is carbonated."],
    retrieval_context=["Sparkling water is carbonated and has no calories."]
)
```

<Note>
`context` is the ideal retrieval result for a specific `input`, whereas `retrieval_context` is the actual retrieval result at runtime. While they are similar, they are not always interchangeable.
</Note>
### Tools Called 

The `tools_called` field is `Optional[List[str]]` and represents the tools that were called by the LLM system. This is particularly useful for evaluating whether agents are properly using tools available to them.

```python tools_called.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question),
    expected_output="Sparkling water is neither healthy nor unhealthy.",
    context=["Sparkling water is a type of water that is carbonated."],
    retrieval_context=["Sparkling water is carbonated and has no calories."],
    tools_called=["Perplexity", "GoogleSearch"]
)
```

### Expected Tools 

The `expected_tools` field is `Optional[List[str]]` and represents the tools that are expected to be called by the LLM system. This is particularly useful for evaluating whether agents are properly using tools available to them.

```python expected_tools.py
# Sample app implementation
import medical_chatbot

question = "Is sparkling water healthy?"
example = Example(
    input=question,
    actual_output=medical_chatbot.chat(question),
    expected_output="Sparkling water is neither healthy nor unhealthy.",
    context=["Sparkling water is a type of water that is carbonated."],
    retrieval_context=["Sparkling water is carbonated and has no calories."],
    tools_called=["Perplexity", "GoogleSearch"],
    expected_tools=["Perplexity", "DBQuery"]
)
```

## Conclusion 

Congratulations! 🎉

You've learned how to create an `Example` and can begin using them to execute evaluations or create [datasets](/evaluation/data_datasets).
