---
title: Unit Testing
---

CI pipelines are the core of all mature software engineering practices. 

**With LLMs, developers should expect nothing less.** 
Using `judgeval`, you can easily unit test your LLM applications for consistency and quality in any metric of your choice.

Unit testing is natively supported in `judgeval` through the `client.assert_test` (Python) or `client.assertTest` (Typescript) method. 
**This also integrates with popular testing frameworks like `pytest` (Python) or `jest`/`vitest` (Typescript), meaning you won't have to learn any new testing frameworks!**

<CodeGroup>
```Python Python
import pytest # Added import
from judgeval import JudgmentClient
from judgeval.data import Example
from judgeval.scorers import FaithfulnessScorer

def test_faithfulness():
    client = JudgmentClient()
    
    example = Example(
        input="What is the capital of France?",
        actual_output="The capital of France is Lyon.", # Hallucinated output
        retrieval_context=["Come tour Paris' museums in the capital of France!"],
    )

    # Example contains a hallucination, so we should expect an exception/assertion error
    # when the threshold is 1.0 (expecting perfect faithfulness)
    with pytest.raises(AssertionError):
        client.assert_test(
            eval_run_name="test_faithfulness_fail",
            examples=[example],
            scorers=[FaithfulnessScorer(threshold=1.0)],
            model="gpt-4o" # Added model parameter
        )
    
    # This should pass as the threshold is low
    client.assert_test(
        eval_run_name="test_faithfulness_pass",
        examples=[example],
        scorers=[FaithfulnessScorer(threshold=0.1)],
        model="gpt-4o" # Added model parameter
    )
```
```Typescript Typescript
import { JudgmentClient, ExampleBuilder, FaithfulnessScorer } from 'judgeval';
import { describe, it, expect } from 'vitest'; // Or import from 'jest'

describe('LLM Unit Tests', () => {
    const client = JudgmentClient.getInstance();

    it('should fail faithfulness test when threshold is high', async () => {
        const example = new ExampleBuilder()
            .input("What is the capital of France?")
            .actualOutput("The capital of France is Lyon.") // Hallucinated output
            .context(["Come tour Paris' museums in the capital of France!"])
            .build();

        // Example contains a hallucination, so assertTest should throw an error
        // when the threshold is 1.0 (expecting perfect faithfulness)
        await expect(client.assertTest({
            evalRunName: "test_faithfulness_fail_ts",
            examples: [example],
            scorers: [new FaithfulnessScorer(1.0)],
            model: "gpt-4o"
        })).rejects.toThrow(); // Check for any thrown error (AssertionError equivalent)
    });

    it('should pass faithfulness test when threshold is low', async () => {
        const example = new ExampleBuilder()
            .input("What is the capital of France?")
            .actualOutput("The capital of France is Lyon.") // Hallucinated output
            .context(["Come tour Paris' museums in the capital of France!"])
            .build();
        
        // This should pass as the threshold is low
        await expect(client.assertTest({
            evalRunName: "test_faithfulness_pass_ts",
            examples: [example],
            scorers: [new FaithfulnessScorer(0.1)],
            model: "gpt-4o"
        })).resolves.toBeUndefined(); // Check that it completes without throwing
    });
});
```
</CodeGroup>

`judgeval` naturally integrates into your CI pipelines, allowing you to execute robust unit tests across your entire codebase. 
**This allows you to catch regressions in your LLM applications before they make it to production!**


