[ Skip to content ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#logfire-integration)
**Join us at the inaugural PyAI Conf in San Francisco on March 10th![Learn More](https://pyai.events/?utm_source=pydantic-ai) **
[ ![logo](https://ai.pydantic.dev/img/logo-white.svg) ](https://ai.pydantic.dev/ "Pydantic AI")
Pydantic AI
Logfire Integration
Type to start searching
[ pydantic/pydantic-ai
  * v1.63.0
  * 15.1k
  * 1.7k

](https://github.com/pydantic/pydantic-ai "Go to repository")
[ ![logo](https://ai.pydantic.dev/img/logo-white.svg) ](https://ai.pydantic.dev/ "Pydantic AI") Pydantic AI
[ pydantic/pydantic-ai
  * v1.63.0
  * 15.1k
  * 1.7k

](https://github.com/pydantic/pydantic-ai "Go to repository")
  * [ Pydantic AI  ](https://ai.pydantic.dev/)
  * [ Installation  ](https://ai.pydantic.dev/install/)
  * [ Getting Help  ](https://ai.pydantic.dev/help/)
  * [ Troubleshooting  ](https://ai.pydantic.dev/troubleshooting/)
  * [ Pydantic AI Gateway  ](https://ai.pydantic.dev/gateway/)
  * Documentation
    * Core Concepts
      * [ Agents  ](https://ai.pydantic.dev/agent/)
      * [ Dependencies  ](https://ai.pydantic.dev/dependencies/)
      * [ Function Tools  ](https://ai.pydantic.dev/tools/)
      * [ Output  ](https://ai.pydantic.dev/output/)
      * [ Messages and chat history  ](https://ai.pydantic.dev/message-history/)
      * [ Direct Model Requests  ](https://ai.pydantic.dev/direct/)
    * Models & Providers
      * [ Overview  ](https://ai.pydantic.dev/models/overview/)
      * [ OpenAI  ](https://ai.pydantic.dev/models/openai/)
      * [ Anthropic  ](https://ai.pydantic.dev/models/anthropic/)
      * [ Google  ](https://ai.pydantic.dev/models/google/)
      * [ xAI  ](https://ai.pydantic.dev/models/xai/)
      * [ Bedrock  ](https://ai.pydantic.dev/models/bedrock/)
      * [ Cerebras  ](https://ai.pydantic.dev/models/cerebras/)
      * [ Cohere  ](https://ai.pydantic.dev/models/cohere/)
      * [ Groq  ](https://ai.pydantic.dev/models/groq/)
      * [ Hugging Face  ](https://ai.pydantic.dev/models/huggingface/)
      * [ Mistral  ](https://ai.pydantic.dev/models/mistral/)
      * [ OpenRouter  ](https://ai.pydantic.dev/models/openrouter/)
      * [ Outlines  ](https://ai.pydantic.dev/models/outlines/)
    * Tools & Toolsets
      * [ Function Tools  ](https://ai.pydantic.dev/tools/)
      * [ Advanced Tool Features  ](https://ai.pydantic.dev/tools-advanced/)
      * [ Toolsets  ](https://ai.pydantic.dev/toolsets/)
      * [ Deferred Tools  ](https://ai.pydantic.dev/deferred-tools/)
      * [ Built-in Tools  ](https://ai.pydantic.dev/builtin-tools/)
      * [ Common Tools  ](https://ai.pydantic.dev/common-tools/)
      * [ Third-Party Tools  ](https://ai.pydantic.dev/third-party-tools/)
    * Advanced Features
      * [ Image, Audio, Video & Document Input  ](https://ai.pydantic.dev/input/)
      * [ Thinking  ](https://ai.pydantic.dev/thinking/)
      * [ HTTP Request Retries  ](https://ai.pydantic.dev/retries/)
    * MCP
      * [ Overview  ](https://ai.pydantic.dev/mcp/overview/)
      * [ Client  ](https://ai.pydantic.dev/mcp/client/)
      * [ FastMCP Client  ](https://ai.pydantic.dev/mcp/fastmcp-client/)
      * [ Server  ](https://ai.pydantic.dev/mcp/server/)
    * [ Multi-Agent Patterns  ](https://ai.pydantic.dev/multi-agent-applications/)
    * [ Web Chat UI  ](https://ai.pydantic.dev/web/)
    * [ Embeddings  ](https://ai.pydantic.dev/embeddings/)
    * [ Testing  ](https://ai.pydantic.dev/testing/)
  * Pydantic Evals
    * [ Overview  ](https://ai.pydantic.dev/evals/)
    * Getting Started
      * [ Quick Start  ](https://ai.pydantic.dev/evals/quick-start/)
      * [ Core Concepts  ](https://ai.pydantic.dev/evals/core-concepts/)
    * Evaluators
      * [ Overview  ](https://ai.pydantic.dev/evals/evaluators/overview/)
      * [ Built-in Evaluators  ](https://ai.pydantic.dev/evals/evaluators/built-in/)
      * [ LLM Judge  ](https://ai.pydantic.dev/evals/evaluators/llm-judge/)
      * [ Custom Evaluators  ](https://ai.pydantic.dev/evals/evaluators/custom/)
      * [ Report Evaluators  ](https://ai.pydantic.dev/evals/evaluators/report-evaluators/)
      * [ Span-Based  ](https://ai.pydantic.dev/evals/evaluators/span-based/)
    * How-To Guides
      * Logfire Integration  [ Logfire Integration  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/)
        * [ Overview  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#overview)
        * [ Installation  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#installation)
        * [ Basic Setup  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#basic-setup)
        * [ What Gets Sent to Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#what-gets-sent-to-logfire)
        * [ Viewing Results in Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#viewing-results-in-logfire)
          * [ Evaluation Overview  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#evaluation-overview)
          * [ Individual Case Details  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#individual-case-details)
          * [ Full Trace View  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#full-trace-view)
        * [ Analyzing Traces  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#analyzing-traces)
          * [ Comparing Runs  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#comparing-runs)
          * [ Debugging Failed Cases  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#debugging-failed-cases)
        * [ Span-Based Evaluation  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#span-based-evaluation)
        * [ Troubleshooting  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#troubleshooting)
          * [ No Data Appearing in Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#no-data-appearing-in-logfire)
          * [ Traces Missing Spans  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#traces-missing-spans)
        * [ Best Practices  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#best-practices)
          * [ 1. Configure Early  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#1-configure-early)
          * [ 2. Use Descriptive Service Names And Environments  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#2-use-descriptive-service-names-and-environments)
          * [ 3. Review Periodically  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#3-review-periodically)
        * [ Next Steps  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#next-steps)
      * [ Dataset Management  ](https://ai.pydantic.dev/evals/how-to/dataset-management/)
      * [ Dataset Serialization  ](https://ai.pydantic.dev/evals/how-to/dataset-serialization/)
      * [ Concurrency & Performance  ](https://ai.pydantic.dev/evals/how-to/concurrency/)
      * [ Multi-Run Evaluation  ](https://ai.pydantic.dev/evals/how-to/multi-run/)
      * [ Retry Strategies  ](https://ai.pydantic.dev/evals/how-to/retry-strategies/)
      * [ Metrics & Attributes  ](https://ai.pydantic.dev/evals/how-to/metrics-attributes/)
    * Examples
      * [ Simple Validation  ](https://ai.pydantic.dev/evals/examples/simple-validation/)
  * Pydantic Graph
    * [ Overview  ](https://ai.pydantic.dev/graph/)
    * [ Beta API  ](https://ai.pydantic.dev/graph/beta/)
      * [ Steps  ](https://ai.pydantic.dev/graph/beta/steps/)
      * [ Joins & Reducers  ](https://ai.pydantic.dev/graph/beta/joins/)
      * [ Decisions  ](https://ai.pydantic.dev/graph/beta/decisions/)
      * [ Parallel Execution  ](https://ai.pydantic.dev/graph/beta/parallel/)
  * Integrations
    * [ Debugging & Monitoring with Pydantic Logfire  ](https://ai.pydantic.dev/logfire/)
    * Durable Execution
      * [ Overview  ](https://ai.pydantic.dev/durable_execution/overview/)
      * [ Temporal  ](https://ai.pydantic.dev/durable_execution/temporal/)
      * [ DBOS  ](https://ai.pydantic.dev/durable_execution/dbos/)
      * [ Prefect  ](https://ai.pydantic.dev/durable_execution/prefect/)
    * UI Event Streams
      * [ Overview  ](https://ai.pydantic.dev/ui/overview/)
      * [ AG-UI  ](https://ai.pydantic.dev/ui/ag-ui/)
      * [ Vercel AI  ](https://ai.pydantic.dev/ui/vercel-ai/)
    * [ Agent2Agent (A2A)  ](https://ai.pydantic.dev/a2a/)
  * Related Packages
    * [ clai  ](https://ai.pydantic.dev/cli/)
  * Examples
    * [ Setup  ](https://ai.pydantic.dev/examples/setup/)
    * Getting Started
      * [ Pydantic Model  ](https://ai.pydantic.dev/examples/pydantic-model/)
      * [ Weather agent  ](https://ai.pydantic.dev/examples/weather-agent/)
    * Conversational Agents
      * [ Chat App with FastAPI  ](https://ai.pydantic.dev/examples/chat-app/)
      * [ Bank support  ](https://ai.pydantic.dev/examples/bank-support/)
    * Data & Analytics
      * [ SQL Generation  ](https://ai.pydantic.dev/examples/sql-gen/)
      * [ Data Analyst  ](https://ai.pydantic.dev/examples/data-analyst/)
      * [ RAG  ](https://ai.pydantic.dev/examples/rag/)
    * Streaming
      * [ Stream markdown  ](https://ai.pydantic.dev/examples/stream-markdown/)
      * [ Stream whales  ](https://ai.pydantic.dev/examples/stream-whales/)
    * Complex Workflows
      * [ Flight booking  ](https://ai.pydantic.dev/examples/flight-booking/)
      * [ Question Graph  ](https://ai.pydantic.dev/examples/question-graph/)
    * Business Applications
      * [ Slack Lead Qualifier with Modal  ](https://ai.pydantic.dev/examples/slack-lead-qualifier/)
    * UI Examples
      * [ Agent User Interaction (AG-UI)  ](https://ai.pydantic.dev/examples/ag-ui/)
  * API Reference
    * pydantic_ai
      * [ pydantic_ai.ag_ui  ](https://ai.pydantic.dev/api/ag_ui/)
      * [ pydantic_ai.agent  ](https://ai.pydantic.dev/api/agent/)
      * [ pydantic_ai.builtin_tools  ](https://ai.pydantic.dev/api/builtin_tools/)
      * [ pydantic_ai.common_tools  ](https://ai.pydantic.dev/api/common_tools/)
      * [ pydantic_ai — Concurrency  ](https://ai.pydantic.dev/api/concurrency/)
      * [ pydantic_ai.direct  ](https://ai.pydantic.dev/api/direct/)
      * [ pydantic_ai.durable_exec  ](https://ai.pydantic.dev/api/durable_exec/)
      * [ pydantic_ai.embeddings  ](https://ai.pydantic.dev/api/embeddings/)
      * [ pydantic_ai.exceptions  ](https://ai.pydantic.dev/api/exceptions/)
      * [ pydantic_ai.ext  ](https://ai.pydantic.dev/api/ext/)
      * [ pydantic_ai.format_prompt  ](https://ai.pydantic.dev/api/format_prompt/)
      * [ pydantic_ai.mcp  ](https://ai.pydantic.dev/api/mcp/)
      * [ pydantic_ai.messages  ](https://ai.pydantic.dev/api/messages/)
      * [ pydantic_ai.models.anthropic  ](https://ai.pydantic.dev/api/models/anthropic/)
      * [ pydantic_ai.models  ](https://ai.pydantic.dev/api/models/base/)
      * [ pydantic_ai.models.bedrock  ](https://ai.pydantic.dev/api/models/bedrock/)
      * [ pydantic_ai.models.cerebras  ](https://ai.pydantic.dev/api/models/cerebras/)
      * [ pydantic_ai.models.cohere  ](https://ai.pydantic.dev/api/models/cohere/)
      * [ pydantic_ai.models.fallback  ](https://ai.pydantic.dev/api/models/fallback/)
      * [ pydantic_ai.models.function  ](https://ai.pydantic.dev/api/models/function/)
      * [ pydantic_ai.models.google  ](https://ai.pydantic.dev/api/models/google/)
      * [ pydantic_ai.models.xai  ](https://ai.pydantic.dev/api/models/xai/)
      * [ pydantic_ai.models.groq  ](https://ai.pydantic.dev/api/models/groq/)
      * [ pydantic_ai.models.huggingface  ](https://ai.pydantic.dev/api/models/huggingface/)
      * [ pydantic_ai.models.instrumented  ](https://ai.pydantic.dev/api/models/instrumented/)
      * [ pydantic_ai.models.mcp_sampling  ](https://ai.pydantic.dev/api/models/mcp-sampling/)
      * [ pydantic_ai.models.mistral  ](https://ai.pydantic.dev/api/models/mistral/)
      * [ pydantic_ai.models.openai  ](https://ai.pydantic.dev/api/models/openai/)
      * [ pydantic_ai.models.openrouter  ](https://ai.pydantic.dev/api/models/openrouter/)
      * [ pydantic_ai.models.outlines  ](https://ai.pydantic.dev/api/models/outlines/)
      * [ pydantic_ai.models.test  ](https://ai.pydantic.dev/api/models/test/)
      * [ pydantic_ai.models.wrapper  ](https://ai.pydantic.dev/api/models/wrapper/)
      * [ pydantic_ai.output  ](https://ai.pydantic.dev/api/output/)
      * [ pydantic_ai.profiles  ](https://ai.pydantic.dev/api/profiles/)
      * [ pydantic_ai.providers  ](https://ai.pydantic.dev/api/providers/)
      * [ pydantic_ai.result  ](https://ai.pydantic.dev/api/result/)
      * [ pydantic_ai.retries  ](https://ai.pydantic.dev/api/retries/)
      * [ pydantic_ai.run  ](https://ai.pydantic.dev/api/run/)
      * [ pydantic_ai.settings  ](https://ai.pydantic.dev/api/settings/)
      * [ pydantic_ai.tools  ](https://ai.pydantic.dev/api/tools/)
      * [ pydantic_ai.toolsets  ](https://ai.pydantic.dev/api/toolsets/)
      * [ pydantic_ai.ui.ag_ui  ](https://ai.pydantic.dev/api/ui/ag_ui/)
      * [ pydantic_ai.ui  ](https://ai.pydantic.dev/api/ui/base/)
      * [ pydantic_ai.ui.vercel_ai  ](https://ai.pydantic.dev/api/ui/vercel_ai/)
      * [ pydantic_ai.usage  ](https://ai.pydantic.dev/api/usage/)
    * pydantic_evals
      * [ pydantic_evals.dataset  ](https://ai.pydantic.dev/api/pydantic_evals/dataset/)
      * [ pydantic_evals.evaluators  ](https://ai.pydantic.dev/api/pydantic_evals/evaluators/)
      * [ pydantic_evals.reporting  ](https://ai.pydantic.dev/api/pydantic_evals/reporting/)
      * [ pydantic_evals.otel  ](https://ai.pydantic.dev/api/pydantic_evals/otel/)
      * [ pydantic_evals.generation  ](https://ai.pydantic.dev/api/pydantic_evals/generation/)
    * pydantic_graph
      * [ pydantic_graph  ](https://ai.pydantic.dev/api/pydantic_graph/graph/)
      * [ pydantic_graph.nodes  ](https://ai.pydantic.dev/api/pydantic_graph/nodes/)
      * [ pydantic_graph.persistence  ](https://ai.pydantic.dev/api/pydantic_graph/persistence/)
      * [ pydantic_graph.mermaid  ](https://ai.pydantic.dev/api/pydantic_graph/mermaid/)
      * [ pydantic_graph.exceptions  ](https://ai.pydantic.dev/api/pydantic_graph/exceptions/)
      * Beta API
        * [ pydantic_graph.beta  ](https://ai.pydantic.dev/api/pydantic_graph/beta/)
        * [ pydantic_graph.beta.graph  ](https://ai.pydantic.dev/api/pydantic_graph/beta_graph/)
        * [ pydantic_graph.beta.graph_builder  ](https://ai.pydantic.dev/api/pydantic_graph/beta_graph_builder/)
        * [ pydantic_graph.beta.step  ](https://ai.pydantic.dev/api/pydantic_graph/beta_step/)
        * [ pydantic_graph.beta.join  ](https://ai.pydantic.dev/api/pydantic_graph/beta_join/)
        * [ pydantic_graph.beta.decision  ](https://ai.pydantic.dev/api/pydantic_graph/beta_decision/)
        * [ pydantic_graph.beta.node  ](https://ai.pydantic.dev/api/pydantic_graph/beta_node/)
    * fasta2a
      * [ fasta2a  ](https://ai.pydantic.dev/api/fasta2a/)
  * Project
    * [ Contributing  ](https://ai.pydantic.dev/contributing/)
    * [ Upgrade Guide  ](https://ai.pydantic.dev/changelog/)
    * [ Version policy  ](https://ai.pydantic.dev/version-policy/)


  * [ Overview  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#overview)
  * [ Installation  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#installation)
  * [ Basic Setup  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#basic-setup)
  * [ What Gets Sent to Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#what-gets-sent-to-logfire)
  * [ Viewing Results in Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#viewing-results-in-logfire)
    * [ Evaluation Overview  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#evaluation-overview)
    * [ Individual Case Details  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#individual-case-details)
    * [ Full Trace View  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#full-trace-view)
  * [ Analyzing Traces  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#analyzing-traces)
    * [ Comparing Runs  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#comparing-runs)
    * [ Debugging Failed Cases  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#debugging-failed-cases)
  * [ Span-Based Evaluation  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#span-based-evaluation)
  * [ Troubleshooting  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#troubleshooting)
    * [ No Data Appearing in Logfire  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#no-data-appearing-in-logfire)
    * [ Traces Missing Spans  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#traces-missing-spans)
  * [ Best Practices  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#best-practices)
    * [ 1. Configure Early  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#1-configure-early)
    * [ 2. Use Descriptive Service Names And Environments  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#2-use-descriptive-service-names-and-environments)
    * [ 3. Review Periodically  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#3-review-periodically)
  * [ Next Steps  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/#next-steps)


  1. [ Pydantic AI  ](https://ai.pydantic.dev/)
  2. [ Pydantic Evals  ](https://ai.pydantic.dev/evals/)
  3. [ How-To Guides  ](https://ai.pydantic.dev/evals/how-to/logfire-integration/)


# Logfire Integration
Visualize and analyze evaluation results using Pydantic Logfire.
## Overview
Pydantic Evals uses OpenTelemetry to record traces of the evaluation process. These traces contain all the information from your evaluation reports, plus full tracing from the execution of your task function.
You can send these traces to any OpenTelemetry-compatible backend, including [Pydantic Logfire](https://logfire.pydantic.dev/docs/guides/web-ui/evals/).
## Installation
Install the optional logfire dependency:
```
pip install 'pydantic-evals[logfire]'

```

## Basic Setup
Configure Logfire before running evaluations:
basic_logfire_setup.py```
import logfire

from pydantic_evals import Case, Dataset

# Configure Logfire
logfire.configure(
    send_to_logfire='if-token-present',  [](https://ai.pydantic.dev/evals/how-to/logfire-integration/#__code_1_annotation_1)
)


# Your evaluation code
def my_task(inputs: str) -> str:
    return f'result for {inputs}'


dataset = Dataset(cases=[Case(name='test', inputs='example')])
report = dataset.evaluate_sync(my_task)

```

That's it! Your evaluation traces will now appear in the Logfire web UI as long as you have the `LOGFIRE_TOKEN` environment variable set.
## What Gets Sent to Logfire
When you run an evaluation, Logfire receives:
  1. **Evaluation metadata**
    1. Dataset name
    2. Number of cases
    3. Evaluator names
  2. **Per-case data**
    1. Inputs and outputs
    2. Expected outputs
    3. Metadata
    4. Execution duration
  3. **Evaluation results**
    1. Scores, assertions, and labels
    2. Reasons (if included)
    3. Evaluator failures
  4. **Task execution traces**
    1. All OpenTelemetry spans from your task function
    2. Tool calls (for Pydantic AI agents)
    3. API calls, database queries, etc.


## Viewing Results in Logfire
### Evaluation Overview
Logfire provides a special table view for evaluation results on the root evaluation span:
[![Logfire Evals Overview](https://ai.pydantic.dev/img/logfire-evals-overview.png)](https://ai.pydantic.dev/img/logfire-evals-overview.png)
This view shows:
  * Case names
  * Pass/fail status
  * Scores and assertions
  * Execution duration
  * Quick filtering and sorting


### Individual Case Details
Click any case to see detailed inputs and outputs:
[![Logfire Evals Case](https://ai.pydantic.dev/img/logfire-evals-case.png)](https://ai.pydantic.dev/img/logfire-evals-case.png)
### Full Trace View
View the complete execution trace including all spans generated during evaluation:
[![Logfire Evals Case Trace](https://ai.pydantic.dev/img/logfire-evals-case-trace.png)](https://ai.pydantic.dev/img/logfire-evals-case-trace.png)
This is especially useful for:
  * Debugging failed cases
  * Understanding performance bottlenecks
  * Analyzing tool usage patterns
  * Writing span-based evaluators


## Analyzing Traces
### Comparing Runs
Run the same evaluation multiple times and compare in Logfire:
```
from pydantic_evals import Case, Dataset


def original_task(inputs: str) -> str:
    return f'original result for {inputs}'


def improved_task(inputs: str) -> str:
    return f'improved result for {inputs}'


dataset = Dataset(cases=[Case(name='test', inputs='example')])

# Run 1: Original implementation
report1 = dataset.evaluate_sync(original_task)

# Run 2: Improved implementation
report2 = dataset.evaluate_sync(improved_task)

# Compare in Logfire by filtering by timestamp or attributes

```

### Debugging Failed Cases
Find failed cases quickly:
  1. Search for `service_name = 'my_service_evals' AND is_exception` (replace with the actual service name you are using)
  2. View the full span tree to see where the failure occurred
  3. Inspect attributes and logs for error messages


## Span-Based Evaluation
Logfire integration enables powerful span-based evaluators. See [Span-Based Evaluation](https://ai.pydantic.dev/evals/evaluators/span-based/) for details.
Example: Verify specific tools were called:
```
import logfire

from pydantic_evals import Case, Dataset
from pydantic_evals.evaluators import HasMatchingSpan

logfire.configure(send_to_logfire='if-token-present')


def my_agent(inputs: str) -> str:
    return f'result for {inputs}'


dataset = Dataset(
    cases=[Case(name='test', inputs='example')],
    evaluators=[
        HasMatchingSpan(
            query={'name_contains': 'search_tool'},
            evaluation_name='used_search',
        ),
    ],
)

report = dataset.evaluate_sync(my_agent)

```

The span tree is available in both:
  * Your evaluator code (via `ctx.span_tree`)
  * Logfire UI (visual trace view)


## Troubleshooting
### No Data Appearing in Logfire
Check:
  1. **Token is set** : `echo $LOGFIRE_TOKEN`
  2. **Configuration is correct** :
```
import logfire

logfire.configure(send_to_logfire='always')  # Force sending

```

  3. **Network connectivity** : Check firewall settings
  4. **Project exists** : Verify project name in Logfire UI


### Traces Missing Spans
If some spans are missing:
  1. **Ensure logfire is configured before imports** :
```
import logfire

logfire.configure()  # Must be first

```

  2. **Check instrumentation** : Ensure your code has enabled all instrumentations you want:
```
import logfire

logfire.instrument_pydantic_ai()
logfire.instrument_httpx(capture_all=True)

```



## Best Practices
### 1. Configure Early
Always configure Logfire before running evaluations:
```
import logfire

from pydantic_evals import Case, Dataset

logfire.configure(send_to_logfire='if-token-present')


# Now import and run evaluations
def task(inputs: str) -> str:
    return f'result for {inputs}'


dataset = Dataset(cases=[Case(name='test', inputs='example')])
dataset.evaluate_sync(task)

```

### 2. Use Descriptive Service Names And Environments
```
import logfire

logfire.configure(
    service_name='rag-pipeline-evals',
    environment='development',
)

```

### 3. Review Periodically
  * Check Logfire regularly to identify patterns
  * Look for consistently failing cases
  * Analyze performance trends
  * Adjust evaluators based on insights


## Next Steps
  * **[Span-Based Evaluation](https://ai.pydantic.dev/evals/evaluators/span-based/)** - Use OpenTelemetry spans in evaluators
  * **[Logfire Documentation](https://logfire.pydantic.dev/docs/guides/web-ui/evals/)** - Complete Logfire guide
  * **[Metrics& Attributes](https://ai.pydantic.dev/evals/how-to/metrics-attributes/)** - Add custom data to traces


© Pydantic Services Inc. 2024 to present
