# Token Profiling

Crucible's `--profile` flag tracks token usage, prompt composition, and cache efficiency across experiment iterations. Use it to understand where your API budget goes and identify optimization opportunities.

## Usage

### During experiment runs

```bash
crucible run --tag run1 --profile
```

Each iteration logs a `[profile]` line:

```
[profile] prompt: ~653 tok (instructions: 24%, state: 15%, history: 20%, directive: 35%, preamble: 5%) | cache: 91%
```

### Post-run analysis

```bash
crucible postmortem --tag run1 --tokens          # table view
crucible postmortem --tag run1 --tokens --json   # machine-readable
```

## Reading the Output

### Per-iteration table

```
Token Profile (2 iterations)
===========================================================================
 Iter   In Tok  Out Tok  Cache%  Agent(s)  Run(s)   Status
---------------------------------------------------------------------------
    1       37     1725     96%      21.5     0.5     keep
    2       37     6975     91%      63.9     0.0    crash
---------------------------------------------------------------------------
  avg       37     4350
```

| Column | Meaning |
|--------|---------|
| **In Tok** | Input tokens reported by the SDK (excludes cached portion) |
| **Out Tok** | Output tokens generated by Claude. Higher = more code written |
| **Cache%** | Prompt caching hit rate. Cached tokens cost 10% of normal input tokens |
| **Agent(s)** | Seconds spent waiting for Claude to think and generate code |
| **Run(s)** | Seconds spent executing the experiment script (evaluate.py) |
| **Status** | `keep` = improved metric, `discard` = worse, `crash` = execution failed |

### Prompt breakdown

```
Prompt Breakdown (avg tokens per section):
             directive:   233 (38%) ████████████
          instructions:   157 (25%) ████████
               history:    99 (16%) █████
                 state:    83 (13%) ████
              preamble:    33 ( 5%) █
```

| Section | What it contains |
|---------|------------------|
| **directive** | READ-THINK-EDIT-EXPLAIN workflow steps and hard rules |
| **instructions** | Content from `.crucible/program.md` (task description, goals, constraints) |
| **history** | Past iteration results — grows with each iteration |
| **state** | Current best metric, iteration count, editable file list |
| **preamble** | One-line role description for the agent |
| **file_inline** | Editable file contents (only when agent lacks read tools) |

### Cache efficiency

```
Cache Efficiency: avg 93% hit rate
```

Anthropic's API automatically caches repeated prompt content. Sections that stay the same across iterations (directive, instructions, preamble) get cached; sections that change (history, state) don't. A high cache rate (>80%) means you're paying mostly for new content.

## When to Act

| Signal | What it means | Action |
|--------|---------------|--------|
| **history > 50%** | Too many past iterations in prompt | Consider truncating history or starting a new tag |
| **Cache% dropping** | More unique content per iteration | Expected as history grows; no action needed |
| **Out Tok consistently high** | Agent writes verbose code | Acceptable if iterations succeed; wasteful if they crash |
| **Agent(s) > 60s regularly** | Agent is overthinking | Check if program.md is too vague |
| **Run(s) near timeout** | Experiment script is slow | Optimize evaluate.py or increase timeout |

## Cost Perspective

For a typical 50-iteration experiment with ~600 token prompts and 93% cache rate:

- Input cost: negligible (most tokens cached at 10% rate)
- Output cost: ~3000 avg tokens x 50 iterations x $15/M tokens = ~$2.25
- **Output tokens dominate cost**, not prompt size

Optimizing prompt size has minimal cost impact. The real levers are: reducing crash rate (wasted iterations) and limiting unnecessarily verbose agent output.

## Technical Notes

- Token counts in the prompt breakdown are **estimates** (~4 chars per token). Actual tokenization varies.
- SDK-reported `input_tokens` may differ from the estimated prompt size because of system prompt and tool definitions added by the SDK.
- Profile data is persisted in `results-{tag}.jsonl` alongside other experiment data.
