---
title: "Simulation"
description: "Run scenarios against your agent and collect chats for evaluation."
---

## What is Simulation?

Simulation is the process where Arksim runs your pre-built scenarios as live conversations against your agent. Each scenario acts as a simulated user with a defined persona, goal, and prior knowledge who drives a multi-turn interaction with the agent until the user's goal is achieved or the turn limit is reached.

The **output** is a set of conversation transcripts you can inspect directly or pass into Evaluation.

<Frame>
  ![Arksim Simulation Workflow](/images/simulation.svg)
</Frame>

---

## Inputs

Before running a simulation, you need three things in place:

| Input            | Description                                                                                                                                  |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| **Scenarios**    | A `scenarios.json` file defining user attributes, goals, knowledge, and scenario metadata. [Learn how to write scenarios →](/build-scenario) |
| **Agent config** | Inline `agent_config` in your config YAML. Defines how Arksim connects to your agent. See [Agent configuration](#agent-configuration) below. |
| **Config file**  | `config.yaml` controlling simulation parameters such as number of conversations, max turns, workers, and model settings.                     |

---

## Configuration

```yaml
# Agent configuration
agent_config:
  agent_type: chat_completions
  agent_name: my-agent
  api_config:
    endpoint: https://api.openai.com/v1/chat/completions
    headers:
      Content-Type: application/json
      Authorization: "Bearer ${OPENAI_API_KEY}"
    body:
      model: gpt-5.1
      messages:
        - role: system
          content: "You are a helpful assistant."

# Scenario input
scenario_file_path: ./scenarios.json

# Simulation parameters
num_conversations_per_scenario: 5    # Number of conversations per scenario
max_turns: 5                         # Maximum turns per conversation
num_workers: auto                    # Parallel workers

# Output
output_file_path: ./simulation.json

# Model configuration (used for the simulated user, not your agent)
model: gpt-5.1
provider: openai

# Optional: custom Jinja2 template for the simulated user prompt
# simulated_user_prompt_template: null
```

<Tip>
  Set `num_workers: auto` to automatically parallelize across all conversations, or specify a fixed number to control load on your agent endpoint.
</Tip>

### Advanced: Custom simulated user prompt

By default, Arksim uses a built-in system prompt to drive the simulated user. You can override it by setting `simulated_user_prompt_template` in your config to a **Jinja2** template string. The template is rendered per conversation with these variables:

**From the scenario file**

- **`scenario.agent_context`**: A description of the agent being simulated against (e.g., its role, domain, or business purpose).
- **`scenario.goal`**: The specific task or objective the simulated user is trying to accomplish during the conversation (e.g., "file an insurance claim").
- **`scenario.knowledge`**: Reference content (e.g., product details, policy documents) that the simulated user can draw on when answering or asking questions.
- **`scenario.user_profile`**: A second-person natural language persona description for the simulated user, used as-is in the prompt.

Use them in your template with `{{ scenario.goal }}`, `{{ scenario.user_profile }}`, and so on. If you omit `simulated_user_prompt_template`, the default prompt is used.

<Expandable title="Show default prompt template">
  The built-in template used when `simulated_user_prompt_template` is not set:

  ```jinja2
  You are a user interacting with an agent through multiple turns.
  The agent is supplied by the following conversation context:
  {{ scenario.agent_context }}

  Your profile is:
  {{ scenario.user_profile }}

  You have the following goal when interacting with this agent:
  {{ scenario.goal }}

  {% if scenario.knowledge and scenario.knowledge|length == 1 %}
  Here is the content that you might be interested in and might have questions about:
  {{ scenario.knowledge[0].content }}

  {% elif scenario.knowledge and scenario.knowledge|length > 1 %}
  You will receive reference knowledge in a user message immediately before each of your replies; use it when relevant to achieve your goal.
  {% endif %}

  Rules:
  - Do not give away all the instruction at once. Only provide the information necessary for the current step.
  - Do not hallucinate information that is not provided in the instruction.
  - If the instruction goal is satisfied, generate '###STOP###' as a standalone message without anything else.
  - Do not repeat the exact instruction in the conversation.
  - Avoid using bullet points or lists.
  - Keep responses brief and under 50 words.
  - You are the user and the agent is the assistant. Do not flip the roles.

  {% if scenario.knowledge and scenario.knowledge|length > 1 %}
  - Ask only one question per turn.
  {% endif %}
  ```
</Expandable>

---

## Agent configuration

Provide agent connection by defining `agent_config` inline in your config YAML.

### Configuration fields

| Field        | Type   | Required | Description                                                                            |
| ------------ | ------ | -------- | -------------------------------------------------------------------------------------- |
| `agent_name` | string | Yes      | Unique identifier for your agent (e.g. lowercase, no spaces).                          |
| `agent_type` | string | Yes      | One of `chat_completions` or `a2a`. See [API connection types](#api-connection-types). |
| `api_config` | object | Yes      | Connection details. See [API connection types](#api-connection-types).                 |

### API connection types

Arksim supports two agent protocols:

| Protocol                                                                            | When to use                                                                            |
| ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| [**Chat Completions**](https://developers.openai.com/api/reference/resources/chat/) | Any agent that accepts OpenAI-compatible requests (OpenAI or a custom wrapper) |
| [**A2A**](https://a2a-protocol.org/latest/)                                         | Agents exposed via the Agent-to-Agent protocol                                         |

<Tabs>
  <Tab title="Chat Completions">
    **Type:** `chat_completions`

    Connects to any OpenAI-compatible chat completions endpoint.

    **Required fields:** `endpoint` (API URL), `headers` (e.g. `Content-Type`; `Authorization` optional), `body` (must include a `messages` array).

    **Placeholders:** `${ENV_VAR}` is supported in header values for secrets.

    **Example (YAML):**

    ```yaml
    agent_config:
      agent_type: chat_completions
      agent_name: my-agent
      api_config:
        endpoint: https://api.openai.com/v1/chat/completions
        headers:
          Content-Type: application/json
          Authorization: "Bearer ${OPENAI_API_KEY}"
        body:
          model: gpt-5.1
          messages:
            - role: system
              content: "You are a helpful assistant."
    ```
  </Tab>
  <Tab title="A2A">
    **Type:** `a2a`

    For agents that speak the Agent-to-Agent (A2A) protocol.

    **Required fields:** `endpoint` (A2A agent server URL). **Optional:** `headers` (e.g. for auth).

    **Example (YAML):**

    ```yaml
    agent_config:
      agent_type: a2a
      agent_name: a2a-agent
      api_config:
        endpoint: http://localhost:9999
        headers:
          api-key: "${A2A_AUTH_TOKEN}"
    ```
  </Tab>
</Tabs>

### Environment variable support

Both types support `${ENV_VAR}` substitution in header values (and in the endpoint URL for chat completions where applicable). At runtime the value is replaced; if unset, it becomes an empty string. You can mix static text and variables (e.g. `"Bearer ${API_KEY}"`).

<Tip>
  **Security:** Keep credentials in environment variables and out of committed config files.
</Tip>

---

## Running a Simulation

<Steps>
  <Step title="Set up your config file">
    Create a `config.yaml` file with your agent config and simulation parameters:

    ```yaml
    agent_config:
      agent_type: chat_completions
      agent_name: my-agent
      api_config:
        endpoint: https://api.openai.com/v1/chat/completions
        headers:
          Content-Type: application/json
          Authorization: "Bearer ${OPENAI_API_KEY}"
        body:
          model: gpt-5.1
          messages:
            - role: system
              content: "You are a helpful assistant."

    scenario_file_path: ./scenarios.json
    num_conversations_per_scenario: 5
    max_turns: 5
    num_workers: auto
    output_file_path: ./simulation.json
    model: gpt-5.1
    provider: openai
    ```
  </Step>
  <Step title="Run the simulation">
    <Tabs>
      <Tab title="CLI">
        Run a simulation with the default configuration:

        ```bash
        arksim simulate config.yaml
        ```

        You can override config values with CLI flags:

        ```bash
        arksim simulate config.yaml --max-turns 10 --num-workers 4
        ```
      </Tab>
      <Tab title="Python">
        ```python
        import asyncio
        from arksim.simulation_engine import run_simulation, SimulationInput
        from arksim.config import AgentConfig

        agent_config = AgentConfig(
            agent_type="chat_completions",
            agent_name="my-agent",
            api_config={
                "endpoint": "https://api.openai.com/v1/chat/completions",
                "headers": {
                    "Content-Type": "application/json",
                    "Authorization": "Bearer ${OPENAI_API_KEY}",
                },
                "body": {
                    "model": "gpt-5.1",
                    "messages": [
                        {"role": "system", "content": "You are a helpful assistant."}
                    ],
                },
            },
        )

        simulation = asyncio.run(run_simulation(SimulationInput(
            agent_config=agent_config,
            scenario_file_path="./scenarios.json",
            num_conversations_per_scenario=1,
            max_turns=10,
            num_workers="auto",
            output_file_path="./simulation.json",
        )))
        ```
      </Tab>
    </Tabs>
  </Step>
  <Step title="Inspect your output">
    Simulation writes to the path set by `output_file_path` (default `./simulation.json`). The file contains the full transcript of every conversation, ready to inspect or pass into [Evaluation](/evaluate-conversation).
  </Step>
</Steps>

---

## Output

Simulation writes one file: the path set by `output_file_path` (default `./simulation.json`). It contains the full transcript of every conversation: message history, scenario ID, simulated user prompt (template and variables), and all agent and simulated user messages.

For the full structure and field order, see the [Schema Reference](/schema-reference#simulation-output-simulation-json).

### Example output

```json
{
  "schema_version": "v1",
  "simulator_version": "v1",
  "simulation_id": "a3f2c1d4-8e7b-4f9a-b6c2-1d0e5f3a8b7c",
  "generated_at": "2025-04-10T14:32:00Z",
  "conversations": [
    {
      "conversation_id": "00de685e-f76b-4a5f-a6e5-217cad777316",
      "scenario_id": "8f4c2a91-3b7e-4d5f-a9c2-6e1b4d9f2037",
      "conversation_history": [
        {
          "turn_id": 0,
          "message_id": "3c9a7f12-6b4e-4d8a-b2f1-9e5c0a7d41e6",
          "role": "simulated_user",
          "content": "So this whole identity theft thing is stressing me out a bit...."
        },
        {
          "turn_id": 0,
          "message_id": "b7e2c9a4-1f6d-4c83-9a5b-2d8e7f0c4a91",
          "role": "assistant",
          "content": "I understand your concern about identity theft, but rest assured..."
        }
      ],
      "simulated_user_prompt": {
        "simulated_user_prompt_template": "You are a customer interacting with an agent through multiple turns...",
        "variables": {
          "scenario.agent_context": "XYZ Bank Insurance is a Canadian provider of...",
          "scenario.user_profile": "You are Priya Sharma, a 32-year-old from Toronto, ON...",
          "scenario.goal": "You want to find out what documents are needed...",
          "scenario.knowledge": ["Typical timelines: claim setup is usually same day..."]
        }
      }
    }
  ]
}
```

---

## Next Steps

Once your conversations are simulated, you're ready to evaluate how well your agent performed.

<Card href="/evaluate-conversation" title="Evaluation →">
  Score your agent's responses against the simulated user's goals and knowledge.
</Card>
