---
title: Tracing
---

## Overview ##

`judgeval`'s tracing module allows you to view your LLM application's execution from **end-to-end**. 

Using tracing, you can:
- Gain observability into **every layer of your agentic system**, from database queries to tool calling and text generation.
- Measure the performance of **each system component in any way** you want to measure it. For instance:
    - Catch regressions in **retrieval quality, factuality, answer relevance**, and 10+ other [**research-backed metrics**](/evaluation/scorers/introduction).
    - Quantify the **quality of each tool call** your agent makes
    - Track the latency of each system component
    - Count the token usage of each LLM generation
- Export your workflow runs to the Judgment platform for **real-time analysis** or as a dataset for [**offline experimentation**](/evaluation/introduction).


## Tracing Your Workflow ##

Setting up tracing with `judgeval` takes two simple steps:

### 1. Initialize a tracer with your API keys and project name

<CodeGroup>
```Python Python
from judgeval.tracer import Tracer

# loads from JUDGMENT_API_KEY and JUDGMENT_ORG_ID env vars
judgment = Tracer(project_name="my_project")
```
```Typescript Typescript
import { Tracer } from 'judgeval';

// The getInstance method ensures a singleton Tracer is used
const judgment = Tracer.getInstance({ projectName: "my_project" });
```
</CodeGroup>

<Note>
    The [Judgment tracer](/api_reference/trace) is a singleton object that should be shared across your application. 
    Your project name will be used to organize your traces in one place on the Judgment platform.
</Note>


### 2. Wrap your workflow components

`judgeval` provides wrapping mechanisms for your workflow components:

#### `wrap()` ####
The `wrap()` function goes over your LLM client (e.g. OpenAI, Anthropic, etc.) and captures metadata surrounding your LLM calls, such as:
- Latency
- Token usage
- Prompt/Completion
- Model name

Here's an example of using `wrap()` on an OpenAI client:
<CodeGroup>
```Python Python
from openai import OpenAI
from judgeval.tracer import wrap

client = wrap(OpenAI())
```
```Typescript Typescript
import OpenAI from 'openai';
import { wrap } from 'judgeval';

const client = wrap(new OpenAI());
```
</CodeGroup>

<Note>
    When using OpenAI streaming with a wrapped client, you need to explicitly enable token usage tracking by setting `stream_options={"include_usage": True}`. Otherwise, token counts won't be captured for streaming calls.

    ```python
    # Enable token counting with streaming
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello"}],
        stream=True,
        stream_options={"include_usage": True}  # Required for token counting
    )
    ```
</Note>

#### `@observe` (Python) / `observe()` (Typescript) ####
The `@observe` decorator (Python) or the `observe()` higher-order function (Typescript) wraps your functions/tools and captures metadata surrounding your function calls, such as:
- Latency
- Input/Output
- Span type (e.g. `retriever`, `tool`, `LLM call`, etc.)

Here's an example of using the observer mechanism:
<CodeGroup>
```Python Python
from judgeval.tracer import Tracer

# loads from JUDGMENT_API_KEY env var
judgment = Tracer(project_name="my_project")

@judgment.observe(span_type="tool")
def my_tool():
    print("Hello world!")

```
```Typescript Typescript
import { Tracer } from 'judgeval';

const judgment = Tracer.getInstance({ projectName: "my_project" });

async function myTool(): Promise<void> {
    console.log("Hello world!");
}

const observedMyTool = judgment.observe({ spanType: "tool" })(myTool);

// You would then call observedMyTool() instead of myTool()
// await observedMyTool();
```
</CodeGroup>

<Note>
    `span_type` is a string that you can use to categorize and organize your trace spans. 
    Span types are displayed on the trace UI to easily navigate a visualization of your workflow. 
    Common span types include `tool`, `function`, `retriever`, `database`, `web search`, etc.
</Note>

#### Automatic Deep Tracing

Judgeval includes automatic deep tracing, which significantly reduces the amount of instrumentation needed in your code. With deep tracing enabled (the default), you only need to observe top-level functions, and all nested function calls will be automatically traced.

##### How Deep Tracing Works

When you decorate a function with `@observe` (Python) or wrap it with `observe()` (TypeScript), the tracer automatically instruments all functions called within that function, creating a complete trace of your execution flow without requiring explicit decorators on every function.

<CodeGroup>
```Python Python
# Deep tracing is enabled by default
judgment = Tracer(project_name="my_project")

# Only need to observe the top-level function
@judgment.observe(span_type="function")
def main():
    # These functions will be automatically traced without @observe
    result = helper_function()
    return process_result(result)

def helper_function():
    return "Helper result"

def process_result(result):
    return f"Processed: {result}"

main()  # Traces main, helper_function, and process_result
```
</CodeGroup>

##### Disabling Deep Tracing

If you prefer more control over what gets traced, you can disable deep tracing:

<CodeGroup>
```Python Python
# Disable deep tracing globally
judgment = Tracer(project_name="my_project", deep_tracing=False)

# Or disable for specific functions
@judgment.observe(span_type="function", deep_tracing=False)
def selective_function():
    helper_function()  # Won't be traced automatically
```
</CodeGroup>

With deep tracing disabled, you'll need to explicitly observe each function you want to trace. You can still name and declare span types for each function using jdugement.observe().


#### Putting it all Together
Here's a complete example of using judgeval's tracing mechanisms:
<CodeGroup>
```Python Python
from judgeval.tracer import Tracer, wrap
from openai import OpenAI

openai_client = wrap(OpenAI())
# loads from JUDGMENT_API_KEY and JUDGMENT_ORG_ID env vars
judgment = Tracer(project_name="my_project")

@judgment.observe(span_type="tool")
def my_tool():
    return "Hello world!"

@judgment.observe(span_type="function")
def my_llm_call():
    message = my_tool()
    res = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": message}]
    )
    return res.choices[0].message.content

# This implicitly starts a trace if one isn't active
# and saves it upon completion or error.
main_result = my_llm_call()
```
```Typescript Typescript
import { Tracer, wrap } from 'judgeval';
import OpenAI from 'openai';

const openaiClient = wrap(new OpenAI());
const judgment = Tracer.getInstance({ projectName: "my_project" });

async function myTool(): Promise<string> {
    return "Hello world!";
}
const observedMyTool = judgment.observe({ spanType: "tool" })(myTool);

async function myLlmCall(): Promise<string | null | undefined> {
    const message = await observedMyTool();
    const res = await openaiClient.chat.completions.create({
        model: "gpt-4o",
        messages: [{ role: "user", content: message }],
    });
    return res.choices[0]?.message?.content;
}
const observedLlmCall = judgment.observe({ spanType: "function" })(myLlmCall);

// Calling the observed function implicitly starts and saves the trace
async function runImplicitTrace() {
    try {
        const result = await observedLlmCall();
        console.log("Implicit trace completed:", result);
    } catch (error) {
        console.error("Implicit trace failed:", error);
        // Trace saved automatically by observe on completion/error
    }
}

runImplicitTrace();
```
</CodeGroup>

And the trace will appear on the Judgment platform as follows:

![Alt text](/images/basic_trace_example.png "Basic Trace Example")

### Using Streaming with Token Counting

When using streaming responses with a wrapped client, you need to explicitly enable token usage tracking:

<CodeGroup>
```Python Python
@judgment.observe(span_type="function")
def my_llm_streaming_call():
    # Enable token counting with streaming API calls
    stream = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Write a poem"}],
        stream=True,
        stream_options={"include_usage": True}  # Required for token counting
    )
    
    # Process the stream
    full_response = ""
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta and chunk.choices[0].delta.content:
            content = chunk.choices[0].delta.content
            full_response += content
            print(content, end="", flush=True)
    
    return full_response
```
```Typescript Typescript
const observedStreamingCall = judgment.observe({ spanType: "function" })(async () => {
    // Enable token counting with streaming API calls
    const stream = await openaiClient.chat.completions.create({
        model: "gpt-4o",
        messages: [{ role: "user", content: "Write a poem" }],
        stream: true,
        stream_options: { include_usage: true }  // Required for token counting
    });
    
    // Process the stream
    let fullResponse = "";
    for await (const chunk of stream) {
        if (chunk.choices[0]?.delta?.content) {
            const content = chunk.choices[0].delta.content;
            fullResponse += content;
            process.stdout.write(content);
        }
    }
    
    return fullResponse;
});
```
</CodeGroup>

<Warning>
  Without setting `stream_options={"include_usage": True}`, token counts will not be captured for streaming API calls, and your usage metrics in traces will be incomplete.
</Warning>

### 3. Running Production Evaluations

Optionally, you can run asynchronous evaluations directly inside your traces.

This enables you to run evaluations on your **production data in real-time**, which can be useful for:
- **Guardrailing your production system** against quality regressions (hallucinations, toxic responses, revealing private data, etc.).
- Exporting production data for **offline experimentation** (e.g for A/B testing your workflow versions on relevant use cases).
- Getting **actionable insights** on how to fix common failure modes in your workflow (e.g. missing knowledge base info, suboptimal prompts, etc.). 

To execute an asynchronous evaluation, you can use the `trace.asyncEvaluate()` method (Typescript) or `judgment.async_evaluate()` (Python, assuming it operates on the currently active trace).

<CodeGroup>
```Python Python
from judgeval.tracer import Tracer
from judgeval.scorers import AnswerRelevancyScorer
from judgeval.data import Example

judgment = Tracer(project_name="my_project")

@judgment.observe(span_type="function")
def main():
    query = "What is the capital of France?"
    res = "The capital of France is Paris."  # Replace with your workflow logic
    
    # Create an Example object to pass to async_evaluate
    example = Example(
        input=query,
        actual_output=res
    )
    
    # Run the evaluation with the Example object
    judgment.async_evaluate(
        scorers=[AnswerRelevancyScorer(threshold=0.5)],
        example=example,
        model="gpt-4o"
    )
    return res

main() # Call the observed function
```
```Typescript Typescript
import { Tracer, AnswerRelevancyScorer } from 'judgeval';

const judgment = Tracer.getInstance({ projectName: "my_project" });

async function mainWithEval(): Promise<string> {
    const query = "What is the capital of France?";
    const res = "The capital of France is Paris."; // Replace with your workflow logic

    const currentTrace = judgment.getCurrentTrace();
    if (currentTrace) {
        await currentTrace.asyncEvaluate(
            [new AnswerRelevancyScorer(1.0)], 
            {
                input: query,
                actualOutput: res,
                model: "gpt-4o"
            }
        );
    } else {
        console.warn("No active trace for async evaluation.");
    }
    return res;
}

// Explicitly start a trace for the evaluation
async function runMainWithEvalTrace() {
    const trace = judgment.startTrace("main_eval_trace");
    const observedMainWithEval = judgment.observe({ spanType: "function" })(mainWithEval);
    try {
        const result = await observedMainWithEval();
        console.log("Trace with eval completed:", result);
        await trace.save();
    } catch (error) {
        console.error("Trace with eval failed:", error);
        await trace.save();
    }
}

runMainWithEvalTrace();
```
</CodeGroup>

<Tip>
    Your async evaluations will be logged to the Judgment platform as part of the original trace and 
    a new evaluation will be created on the Judgment platform.
</Tip>

## Example: Music Recommendation Agent

In this video, we'll walk through all of the topics covered in this guide by tracing over a simple OpenAI API-based music recommendation agent.

<iframe 
    width="560" 
    height="315" 
    src="https://www.youtube.com/embed/7g0fut06UxQ"
    title="YouTube video player" 
    frameborder="0" 
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" 
    referrerpolicy="strict-origin-when-cross-origin" 
    allowfullscreen
></iframe>


## Advanced: Customizing Traces Using the Context Manager (Python) / Explicit Trace Client (Typescript) ##

In Python, if you need to customize your tracing context beyond the implicit behavior of `@observe`, you can use the `with judgment.trace()` context manager.
In Typescript, you achieve similar control by explicitly creating a `TraceClient` instance using `judgment.startTrace()` and manually calling methods like `save()` or `print()` on it.

The explicit trace client allows you to **save or print the state of the trace at any point in the workflow**.
This is useful for debugging or exporting any state of your workflow to run an evaluation from!

<Tip>
    Any functions wrapped with `judgment.observe()` called within the scope where the `TraceClient` is active will automatically be associated with that trace.
</Tip>

Here's an example of using explicit trace management:
<CodeGroup>
```Python Python
from judgeval.tracer import Tracer, wrap
from openai import OpenAI

judgment = Tracer(project_name="my_project")
client = wrap(OpenAI())

@judgment.observe(span_type="tool")
def my_tool():
    return "Hello world!"

def main():
    with judgment.trace(name="my_workflow") as trace:
        res = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": f"{my_tool()}"}]
        )
    
        trace.print()  # prints the state of the trace to console
        trace.save()  # saves the current state of the trace to the Judgment platform
        # Note: Python trace context likely saves automatically on exit

    return res.choices[0].message.content
```
```Typescript Typescript
import { Tracer, wrap } from 'judgeval';
import OpenAI from 'openai';

const judgment = Tracer.getInstance({ projectName: "my_project" });
const client = wrap(new OpenAI());

async function myTool(): Promise<string> {
    return "Hello world!";
}
const observedMyTool = judgment.observe({ spanType: "tool" })(myTool);

async function main() {
    // Start the trace explicitly
    const trace = judgment.startTrace("my_workflow"); 
    let resultMessage: string | null | undefined;

    try {
        const toolOutput = await observedMyTool(); // observedMyTool is associated with the active trace
        const res = await client.chat.completions.create({
            model: "gpt-4o",
            messages: [{ role: "user", content: toolOutput }],
        });
        resultMessage = res.choices[0]?.message?.content;

        trace.print(); // Prints the trace to console
        await trace.save(); // Saves the trace to the Judgment platform
        
    } catch (error) {
        console.error("Workflow failed:", error);
        await trace.save(); // Still save trace on error
    } finally {
        // Ensure save is called if not already done (optional, depends on desired behavior)
        // await trace.save(); // Could be redundant if already saved in try/catch
    }

    return resultMessage;
}

main();
```
</CodeGroup>

<Warning>
    In Python, the `with judgment.trace()` context manager should only be used if you need fine-grained control 
    over the trace lifecycle. In Typescript, explicit management via `startTrace()` and `trace.save()` is the standard way to gain this control.
    In most simple cases in Python, the `@observe` decorator is sufficient.
</Warning>
