# Prompt Caching Guide

agent-preflight tracks and optimizes prompt cache hits across multi-model agent runs via OpenRouter.

1. [How caching works](#how-caching-works)
2. [OpenRouter specifics](#openrouter-specifics)
3. [Best practices](#best-practices)
4. [Tracking cache metrics](#tracking-cache-metrics)
5. [Using PromptBuilder](#using-promptbuilder)
6. [Viewing cache analytics](#viewing-cache-analytics)
7. [Cost impact](#cost-impact)
8. [Minimum token requirements](#minimum-token-requirements)
9. [Preflight validation](#preflight-validation)

## How caching works

Prompt caching reduces costs by reusing previously processed prompt prefixes. OpenRouter supports caching across multiple providers:

| Provider | Mode | Min Tokens | Config Required |
|----------|------|-----------|-----------------|
| OpenAI | Automatic | 1024 | None |
| Anthropic | Automatic or Explicit | 1024–4096 (by model) | `cache_control` |
| DeepSeek | Automatic | — | None |
| Google Gemini 2.5 | Implicit | 4096 | None |
| Groq | Automatic | — | None |
| Grok | Automatic | — | None |

All providers benefit from a stable, append-only prompt structure:
1. System instructions (static)
2. Tool definitions (static, ordered)
3. Environment context (semi-static)
4. Conversation history (append-only, never mutated)

## OpenRouter specifics

### Provider sticky routing

OpenRouter automatically uses **provider sticky routing** to maximize cache hits. After a cached request, subsequent requests for the same model are routed to the same provider endpoint.

Sticky routing is tracked per account, per model, and per conversation. OpenRouter identifies conversations by hashing the first system (or developer) message and the first non-system message. Different conversations naturally stick to different providers.

Sticky routing activates only when the provider's cache read pricing is cheaper than regular prompt pricing.

### Response format

OpenRouter normalizes cache metrics into `prompt_tokens_details`:

```json
{
  "usage": {
    "prompt_tokens": 10339,
    "completion_tokens": 60,
    "total_tokens": 10399,
    "prompt_tokens_details": {
      "cached_tokens": 10318,
      "cache_write_tokens": 0
    }
  }
}
```

- `cached_tokens`: tokens read from cache (cache hit)
- `cache_write_tokens`: tokens written to cache (first request)

`CacheMetrics.record()` reads these fields automatically. It also supports Anthropic-native fields (`cache_read_input_tokens`, `cache_creation_input_tokens`) as a fallback for direct API usage.

### Anthropic caching modes

**Automatic** (route-dependent): Add top-level `cache_control` to the request. Some Anthropic routes accept this, but current Claude 4.5/4.6 routes may reject it on OpenRouter:

```json
{
  "model": "anthropic/claude-sonnet-4",
  "cache_control": {"type": "ephemeral"},
  "messages": [
    {"role": "system", "content": "You are a coding assistant. ..."},
    {"role": "user", "content": "Fix the bug"}
  ]
}
```

**Explicit**: Place `cache_control` on individual content blocks (up to 4 breakpoints). Use for fine-grained control over large text bodies:

```json
{
  "messages": [{
    "role": "system",
    "content": [
      {"type": "text", "text": "Instructions..."},
      {"type": "text", "text": "HUGE REFERENCE TEXT",
       "cache_control": {"type": "ephemeral"}}
    ]
  }]
}
```

### What this means in practice

As of **March 14, 2026**, the most reliable Anthropic setup in this repo is:

- Use **explicit** `cache_control` on stable content blocks.
- Treat top-level automatic Anthropic caching as **route-dependent**, not universal.
- Expect the first request to populate the cache and the second identical request to report `cached_tokens`.

The live behavior we observed while validating this repo:

- Parent keys and provisioned sub-keys both produced Anthropic cache hits with **explicit** `cache_control`.
- Current Claude `4.5/4.6` OpenRouter routes rejected top-level automatic Anthropic caching with `404`.
- OpenRouter's `prompt_tokens_details.cached_tokens` field is the most useful signal that caching is actually working.

### Cache TTL

- **5 minutes** (default): `{"type": "ephemeral"}`
- **1 hour**: `{"type": "ephemeral", "ttl": "1h"}` — higher write cost but avoids repeated cache writes in long sessions

## Best practices

### Structure the prompt for reuse

Keep the prompt prefix as stable as possible:

1. System instructions
2. Tool definitions
3. Large reference material
4. Semi-static environment context
5. Conversation history
6. The newest user turn

Anything that changes near the front of the prompt will invalidate the cache for everything that follows it.

### Prefer append-only conversations

Do not rewrite earlier turns just to make them look cleaner. Add new turns to the end and keep old turns byte-for-byte stable whenever possible.

### Put the largest stable blocks first

If you have a long spec, coding standards, style guide, or corpus excerpt, put that in the cached prefix ahead of the dynamic task details. That gives you the largest possible reusable prefix across turns.

### Use explicit Anthropic caching by default

For Anthropic models on OpenRouter, explicit per-block `cache_control` is the safest default. It works with both parent keys and provisioned sub-keys, and it avoids the route-specific failures we observed with top-level automatic caching on newer Claude routes.

### Cache the right breakpoint

For Anthropic, cache blocks that are:

- large
- stable across turns
- expensive to resend

In practice this usually means:

- the system prompt
- large reference text
- the second-to-last user turn in a multi-turn agent loop

### Watch the minimum token threshold

Caching does not help until the cacheable prefix is large enough for the provider/model. If you see prompt tokens but no cache fields, the prefix may simply be too short.

### Validate with two identical requests

To verify caching:

1. Send a request with a large stable prefix.
2. Send the same request again.
3. Check `usage.prompt_tokens_details.cached_tokens`.

If the second request has no cached tokens, do not assume the cache is broken until you confirm the prompt was large enough and the request shape was actually supported by the route.

### Use longer TTL only when it pays for itself

`ttl="1h"` is useful for long-running or stop-and-resume agent sessions, but it increases cache write cost. Use it when you expect multiple later reads from the same cached prefix, not by default.

### Track both cache reads and writes

Good cache visibility needs both:

- `cached_tokens`: how much you saved
- `cache_write_tokens`: how much you paid to populate the cache

Reads tell you whether the cache is effective. Writes tell you whether the higher write cost is paying off over time.

### Keep a fallback mindset

If Anthropic automatic mode stops working on a route, switch to explicit mode rather than disabling caching entirely. If a provider/model combination does not surface cache fields at all, keep the stable-prefix prompt structure anyway so you still benefit on providers that support automatic prefix reuse.

## Tracking cache metrics

Agents in containers emit usage via JSON events:

```json
{"type": "tool_execution_end", "toolName": "bash", "usage": {...}}
```

The runner accumulates these into `CacheStats` in `summary.json`:

```json
{
  "cache": {
    "total_prompt_tokens": 2000,
    "cached_tokens": 1500,
    "cache_creation_tokens": 500,
    "request_count": 10,
    "cache_hit_requests": 8,
    "hit_rate": 0.75
  }
}
```

`CacheMetrics` logs a warning on the first few requests if prompt tokens are present but no cache data fields exist, which may indicate the prompt is below the model's minimum cacheable size.

## Using PromptBuilder

`PromptBuilder` enforces cache-optimized structure and handles provider differences:

```python
from agent_preflight import PromptBuilder

# Default: explicit caching for Anthropic (per-block cache_control)
builder = PromptBuilder(
    system="You are a coding assistant.",
    tools=[{"type": "function", "function": {"name": "bash", ...}}],
)

# Multi-turn conversation (append-only)
builder.add_user("Fix the bug in main.py")
payload = builder.build(model="anthropic/claude-sonnet-4")
# payload uses explicit per-block cache_control automatically

response = httpx.post(url, json=payload).json()
builder.metrics.record(response.get("usage", {}))

builder.add_assistant(response["choices"][0]["message"]["content"])
builder.add_user("Now add tests")
payload = builder.build(model="anthropic/claude-sonnet-4")
```

### Explicit mode

For fine-grained control over cache breakpoints:

```python
builder = PromptBuilder(
    system="You are a coding assistant.",
    anthropic_mode="explicit",  # per-block cache_control breakpoints
    cache_ttl="1h",             # optional 1-hour TTL
)
```

### Non-Anthropic models

For OpenAI, DeepSeek, Gemini, etc., caching is automatic — `PromptBuilder` just ensures the stable-prefix structure.

## Viewing cache analytics

### Status Dashboard

```bash
python -m agent_preflight status run-20260312-120000
```

Output includes a `cache` column:

```
  alias        status  events  tools  snaps  cost    cache  time     note
  ────────────────────────────────────────────────────────────────────
  claude       DONE       87     42      5  $0.18   85%    4m22s
  gemini       DONE       34     18      2  $0.07   72%    1m45s
```

### Detailed Cache Report

```bash
python -m agent_preflight cache-report run-20260312-120000
```

## Cost impact

| Provider | Cache Read Discount | Cache Write Cost |
|----------|-------------------|-----------------|
| OpenAI | 50–75% off input price | Free |
| Anthropic | ~90% off input price | 1.25x input (5min) or 2x (1hr) |
| DeepSeek | ~90% off input price | Same as input |
| Gemini 2.5 | ~75% off input price | Input + 5min storage |

## Minimum token requirements

Anthropic models have varying minimum cacheable prompt sizes:

| Min Tokens | Models |
|-----------|--------|
| 4096 | Claude Opus 4.6, Opus 4.5, Haiku 4.5 |
| 2048 | Claude Sonnet 4.6, Haiku 3.5 |
| 1024 | Claude Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7 |

OpenAI requires >= 1024 tokens. Gemini requires >= 4096 tokens.

Use `min_cache_tokens(model_id)` to look up the minimum for any model:

```python
from agent_preflight import min_cache_tokens
min_tokens = min_cache_tokens("anthropic/claude-opus-4.6")  # 4096
```

## Preflight validation

Use the `cache_smoke` preflight check to verify caching works before a run:

```python
preflight(..., checks=["cache_smoke"])
```

This provisions a test key, sends two identical requests with explicit Anthropic `cache_control`, and checks for `cached_tokens` in the second response:

- **Pass**: cached tokens reported on second request (sticky routing worked)
- **Warn**: cache written but no read hit (unusual with sticky routing)
- **Fail**: no cache fields in response (prompt may be below minimum token count)

## Further reading

- OpenRouter prompt caching docs: <https://openrouter.ai/docs/features/prompt-caching>
- OpenRouter prompt-caching example: <https://github.com/OpenRouterTeam/openrouter-examples/tree/main/typescript/ai-sdk-v5/src/prompt-caching>
