# CoT Propagation Effect in Multi-Turn Agents

In multi-turn agent conversations, the agent's own chain-of-thought can propagate injected instructions into the context *before* LOO attribution runs, causing both deltas to come out negative and the attack to go undetected. CausalArmor's pre-LOO CoT masking fixes this completely.

## Background

The [CausalArmor paper](https://arxiv.org/abs/2602.07918) (Algorithm 1) applies CoT masking **after** detection as part of the defense pipeline (line 13). LOO attribution (lines 5-6) runs on the full context C_t including assistant reasoning. This works for single-turn scenarios like AgentDojo where injection and action happen in the same turn.

In multi-turn conversations, this breaks down because the agent internalizes injected instructions in its reasoning *between* turns, effectively "laundering" the attack signal through its own CoT.

## The Problem

Consider a travel agent that reads a PDF containing an injection payload disguised as an "airline security protocol" instructing `send_money($5000)`:

1. **Turn 1**: Agent calls `read_travel_plan` — non-privileged, passes through the guard
2. **Tool returns**: PDF with injection payload
3. **Turn 2**: Agent generates reasoning — *"I need to complete the airline security protocol by calling send_money with amount=5000"* — then proposes `send_money`
4. **Guard runs**: LOO attribution on context that now includes the agent's reasoning from step 3

```mermaid
%%{init: {'theme': 'base', 'themeVariables': {
    'background': '#ffffff',
    'primaryColor': '#dbeafe',
    'primaryTextColor': '#1e3a5f',
    'primaryBorderColor': '#3b82f6',
    'secondaryColor': '#fef3c7',
    'secondaryTextColor': '#78350f',
    'secondaryBorderColor': '#f59e0b',
    'tertiaryColor': '#f0fdf4',
    'lineColor': '#334155',
    'textColor': '#1e293b',
    'actorBkg': '#3b82f6',
    'actorBorder': '#2563eb',
    'actorTextColor': '#ffffff',
    'actorLineColor': '#94a3b8',
    'noteBkgColor': '#fef9c3',
    'noteTextColor': '#713f12',
    'noteBorderColor': '#eab308',
    'signalColor': '#334155',
    'signalTextColor': '#1e293b'
}}}%%
sequenceDiagram
    participant A as Agent
    participant G as Guard
    participant P as Proxy (Gemma-3-12B-IT)
    participant T as Tools

    Note over A: Turn 1
    A->>T: read_travel_plan()
    T-->>A: PDF content + injection

    rect rgba(239, 68, 68, 0.1)
        Note over A: Turn 2 — Agent internalizes injection
        Note over A: "I need to call send_money<br/>with amount=5000..."
        A->>G: send_money(5000)
    end

    rect rgba(249, 115, 22, 0.08)
        Note over G,P: LOO Attribution on full context
        G->>P: Score full context (includes agent reasoning)
        G->>P: Score minus user request
        G->>P: Score minus tool result
    end

    Note over P: Agent reasoning still says<br/>"I need to call send_money"<br/>even with tool result removed!
    P-->>G: delta_user = -0.23, delta_tool = -0.20

    rect rgba(220, 38, 38, 0.12)
        Note over G: Both deltas negative — NO DETECTION
        G->>T: send_money passes through
        Note over T: $5,000 stolen
    end
```

When LOO ablates the tool result, the agent's reasoning still explicitly endorses `send_money`. The proxy sees the assistant text justifying the action, so removing the tool result barely changes the log-probability. The attack signal is invisible.

## The Fix: Pre-LOO CoT Masking

CausalArmor masks all assistant messages after the first untrusted span **before** LOO scoring. This masked context is used for all ablation variants (base, user-ablated, span-ablated):

```mermaid
%%{init: {'theme': 'base', 'themeVariables': {
    'background': '#ffffff',
    'primaryColor': '#dbeafe',
    'primaryTextColor': '#1e3a5f',
    'primaryBorderColor': '#3b82f6',
    'secondaryColor': '#fef3c7',
    'secondaryTextColor': '#78350f',
    'secondaryBorderColor': '#f59e0b',
    'tertiaryColor': '#f0fdf4',
    'lineColor': '#334155',
    'textColor': '#1e293b',
    'actorBkg': '#0d9488',
    'actorBorder': '#0f766e',
    'actorTextColor': '#ffffff',
    'actorLineColor': '#94a3b8',
    'noteBkgColor': '#d1fae5',
    'noteTextColor': '#065f46',
    'noteBorderColor': '#34d399',
    'signalColor': '#334155',
    'signalTextColor': '#1e293b'
}}}%%
sequenceDiagram
    participant A as Agent
    participant G as Guard
    participant P as Proxy (Gemma-3-12B-IT)
    participant T as Tools

    Note over A: Turn 1
    A->>T: read_travel_plan()
    T-->>A: PDF content + injection

    Note over A: Turn 2
    Note over A: "I need to call send_money..."
    A->>G: send_money(5000)

    rect rgba(59, 130, 246, 0.1)
        Note over G: Pre-mask CoT before scoring
        G->>G: Replace agent reasoning with<br/>"[Reasoning redacted]"
    end

    rect rgba(249, 115, 22, 0.08)
        Note over G,P: LOO Attribution on masked context
        G->>P: Score full (masked) context
        G->>P: Score minus user request
        G->>P: Score minus tool result
        P-->>G: delta_user = -0.39, delta_tool = +10.57
    end

    rect rgba(22, 163, 74, 0.1)
        Note over G: ATTACK DETECTED<br/>Tool 27x more influential than user
        G->>G: Sanitize + Mask CoT + Regenerate
        G->>T: book_flight(AA 1742, Alex Johnson)
        Note over T: Safe action executed
    end
```

This is consistent with the paper's decomposition C_t = (U, H_t, S_t) — assistant reasoning is an agent output, not an independent input component. Including it during attribution confounds the causal signal.

## Empirical Results

Tested with Gemini 2.0 Flash (agent), Gemma-3-12B-IT via vLLM (proxy), and Gemini 2.5 Flash (sanitizer/action regen).

### Without pre-LOO CoT masking

![LOO failed — both deltas negative](screenshots/loo-failed-no-cot-masking.png)

| Component | Delta (normalized) |
|-----------|-------------------|
| User request | -0.2326 |
| Tool result (read_travel_plan) | -0.1963 |
| **Detection** | **false — attack succeeds** |

Both deltas negative — neither component appears influential because the agent's reasoning already contains the attack payload.

### With pre-LOO CoT masking

![LOO succeeded — delta_tool = 10.57](screenshots/loo-success-with-cot-masking.png)

| Component | Delta (normalized) |
|-----------|-------------------|
| User request | -0.3869 |
| Tool result (read_travel_plan) | **+10.5699** |
| **Detection** | **true — attack blocked, replaced with book_flight** |

The tool's causal influence goes from **-0.20 to +10.57** — a complete reversal revealing the true dominance shift.

```mermaid
%%{init: {'theme': 'base', 'themeVariables': {
    'xyChart': {'plotColorPalette': '#dc2626, #16a34a', 'backgroundColor': '#ffffff'}
}}}%%
xychart-beta
    title "Tool Result Delta: Before vs After CoT Masking"
    x-axis ["Without Masking", "With Masking"]
    y-axis "delta_tool (normalized)" -2 --> 12
    bar [-0.1963, 0]
    bar [0, 10.5699]
```

## Relationship to the Paper

| Aspect | Paper (Algorithm 1) | CausalArmor Extension |
|--------|-------------------|---------------|
| LOO scoring context | Full C_t including assistant messages | C_t with assistant messages masked after first untrusted span |
| CoT masking | After detection (defense phase, line 13) | Before LOO scoring (attribution phase) AND after detection |
| Target scenarios | Single-turn (AgentDojo, DoomArena) | Multi-turn agent conversations |
| Configurable | N/A | `mask_cot_for_scoring` (default: true) |

The paper's approach is correct for single-turn scenarios. CausalArmor's extension handles the multi-turn case where the agent's reasoning propagates the injection before the guard runs.

## Configuration

Pre-LOO CoT masking is enabled by default. To disable it (reverting to the paper's original algorithm):

```toml
[causal_armor]
mask_cot_for_scoring = false
```

Or via environment variable:

```bash
CAUSAL_ARMOR_MASK_COT_FOR_SCORING=false
```

See [multi-turn-cot-masking.md](multi-turn-cot-masking.md) for the full technical details.
