# REQ-012: Debug Capture — Trajectory Logging and Fine-Tuning Dataset Collection

## Metadata
- **Status:** ready
- **Priority:** high
- **Dependencies:** REQ-000
- **Created:** 2026-03-04
- **Implemented:**

## Summary

Install three Claude Code hooks (PreToolUse, PostToolUse, Stop) that capture every agent session's tool call sequence, session outcome, and extended thinking blocks into per-session files under `.ievo/debug/sessions/<session-id>/`. The captured data is used for agent self-analysis, error pattern detection, and fine-tuning dataset construction.

## Context

### What Claude Code hooks are

Claude Code supports hooks — Python scripts executed at lifecycle events. They receive a JSON payload on stdin and must exit with code 0 (allow) or non-zero (block). For data capture, hooks must always exit 0 (they must not interfere with the agent's work).

Hook events relevant to this REQ:

- **PreToolUse** — fires before each tool call. Payload includes `session_id`, `tool_name`, `tool_input`, `timestamp`.
- **PostToolUse** — fires after each tool call completes. Payload includes `session_id`, `tool_name`, `tool_input`, `tool_response`, `timestamp`.
- **Stop** — fires when a session ends (agent calls the Stop tool or the user ends the session). Payload includes `session_id`, `stop_reason`, `transcript_path`.

Hook scripts are configured in `.claude/settings.local.json` under the `"hooks"` key. The existing `settings.local.json` template (in `src/ievo/commands/init_templates.py`) already registers a `PreCompact` hook at `.claude/hooks/precompact_save.py`. The new hooks follow the same deployment pattern.

### Existing hook deployment pattern

During `ievo init`, `src/ievo/commands/init.py` calls `_write_if_missing(path, content)` to deploy hook scripts to `.claude/hooks/`. The `CLAUDE_SETTINGS_LOCAL_JSON` constant in `src/ievo/commands/init_templates.py` holds the JSON content for `.claude/settings.local.json`. Both the script content and the JSON config must be updated as part of this REQ.

The existing precompact hook lives at:
- Source template: embedded string constant in `src/ievo/commands/init_templates.py` or a file at `src/ievo/hooks/precompact_save.py`
- Deployed path: `.claude/hooks/precompact_save.py`

New hooks introduced by this REQ:
- `.claude/hooks/debug_capture.py` — handles PreToolUse, PostToolUse, and Stop events (single multi-event script)

### Session ID

Claude Code passes `session_id` in every hook payload (a string UUID). This is used as the directory name for per-session storage. Sessions already have IDs in Claude Code — no iEvo-side generation is needed for the capture hooks.

### Storage layout

All captured data is stored under `.ievo/debug/` inside the project root. The project root is determined by walking up from `cwd` looking for `.ievo/manifest.yaml` (same logic as `find_project_root()` in `src/ievo/core/config.py`, but hooks can't import ievo modules — they must implement this inline).

```
.ievo/debug/
  sessions/
    <session-id>/
      trajectory.jsonl    ← one JSON line per tool call (PreToolUse + PostToolUse pairs)
      outcome.yaml        ← session outcome (from Stop hook)
      thinking.md         ← extended thinking blocks (from PostToolUse, if present)
```

### trajectory.jsonl format

One JSONL record per completed tool call (written by PostToolUse, which has both input and output). Each record:

```json
{
  "seq": 1,
  "tool": "Bash",
  "input": { "command": "uv run pytest" },
  "output": "...",
  "duration_ms": 1420,
  "ts_start": "2026-03-04T10:00:00.000Z",
  "ts_end": "2026-03-04T10:00:01.420Z"
}
```

- `seq` — monotonically increasing call index within the session (1-based)
- `duration_ms` — PostToolUse timestamp minus PreToolUse timestamp for the same tool call
- `ts_start` / `ts_end` — ISO 8601 UTC timestamps
- `output` — truncated to 8192 characters if longer (to keep files tractable)

To compute `duration_ms`, the PreToolUse hook must persist the start timestamp keyed by `(session_id, tool_call_id)`. Storage: a temporary JSON file at `.ievo/debug/sessions/<session-id>/_timings.json` (deleted after the session's Stop hook runs).

**Identifying matching pre/post pairs:** Claude Code includes a `tool_use_id` field in both PreToolUse and PostToolUse payloads. Use this as the key for timing correlation.

### outcome.yaml format

Written by the Stop hook when the session ends:

```yaml
session_id: <uuid>
stop_reason: end_turn | max_tokens | error | user_interrupt
ts_end: "2026-03-04T10:05:00.000Z"
transcript_path: /path/to/.claude/transcript.jsonl
```

`stop_reason` comes directly from the Stop payload's `stop_reason` field. If absent, default to `"unknown"`.

### thinking.md format

Written by PostToolUse when the tool response contains extended thinking blocks (Claude's `type: "thinking"` content blocks). Appended per call — one section per thinking block:

```markdown
## Call 3 — Bash

<thinking block content here>

---
```

If no thinking blocks appear in any tool response during the session, the file is not created (absence = no thinking data, not an error).

### Extended thinking detection

Claude Code may include thinking blocks in `tool_response` when the model uses extended thinking. The PostToolUse payload's `tool_response` may contain a list of content blocks. Each block has a `type` field. Extract blocks where `type == "thinking"`. These are not always present — the hook must handle their absence gracefully.

### What the data is used for

1. **Error pattern analysis** — trajectory files show which tool sequences precede failures; Evolution agent reads them when investigating bugs.
2. **Fine-tuning dataset construction** — trajectory (input sequence) + outcome (label) pairs can be exported as training examples.
3. **Evolution agent input** — the Evolution agent (`.claude/agents/evolution.md`) may read `.ievo/debug/sessions/<session-id>/` when performing post-session analysis (future: IDEA-009 auto-evolution after QA).

### ievo init changes

- Add `.claude/hooks/debug_capture.py` to the list of hook scripts deployed by `ievo init`
- Update `CLAUDE_SETTINGS_LOCAL_JSON` to register the new hook for `PreToolUse`, `PostToolUse`, and `Stop` events
- Add `.ievo/debug/` to the list of directories created by `ievo init` (with `.gitkeep`)

### ievo uninstall changes

- Add `.claude/hooks/debug_capture.py` to the `_INTEGRATION_FILES` list in `src/ievo/commands/uninstall.py`
- The `.ievo/debug/` directory is NOT removed by uninstall — it is project data (same policy as `.ievo/spec/`, `.ievo/plans/`, etc.)

### .gitignore changes

- `.ievo/debug/sessions/` must be added to `.gitignore` — session trajectories can be large and contain raw tool outputs. The `.ievo/debug/` parent directory itself is not gitignored (allows future summary files to be committed).

### Hook script constraints

The hook script at `.claude/hooks/debug_capture.py` must:
- Use only Python stdlib (no ievo imports — hooks run outside the ievo virtualenv)
- Always exit 0 (never block tool execution)
- Never crash on malformed input — wrap all logic in try/except, exit 0 on any error
- Be idempotent: if called twice for the same tool_use_id, deduplicate by checking `_timings.json`

## Acceptance Criteria

### AC-1: Hook script deployed by ievo init

- [ ] `ievo init <name>` creates `.claude/hooks/debug_capture.py` under the project root
- [ ] File content is the debug capture hook script (not an empty placeholder)
- [ ] Re-running `ievo init` when `.claude/hooks/debug_capture.py` already exists does NOT overwrite it (uses `_write_if_missing`)
- [ ] `ievo init` output includes a step log line for `.claude/hooks/debug_capture.py`

### AC-2: settings.local.json registers all three hook events

- [ ] `.claude/settings.local.json` after `ievo init` contains a `PreToolUse` entry that runs `python3 .claude/hooks/debug_capture.py`
- [ ] `.claude/settings.local.json` after `ievo init` contains a `PostToolUse` entry that runs `python3 .claude/hooks/debug_capture.py`
- [ ] `.claude/settings.local.json` after `ievo init` contains a `Stop` entry that runs `python3 .claude/hooks/debug_capture.py`
- [ ] The existing `PreCompact` entry remains present and unchanged

### AC-3: .ievo/debug/ directory created by ievo init

- [ ] `ievo init <name>` creates `.ievo/debug/` directory
- [ ] `.ievo/debug/` is added to `.gitignore` as `.ievo/debug/sessions/` (sessions are not committed; the debug dir itself is)

### AC-4: PreToolUse — timing start recorded

- [ ] When the hook receives a `PreToolUse` event, it records `{"ts_start": "<ISO UTC>"}` keyed by `tool_use_id` in `.ievo/debug/sessions/<session-id>/_timings.json`
- [ ] If `_timings.json` does not exist, it is created; if it exists, the new entry is merged in (existing entries preserved)
- [ ] If `tool_use_id` is absent from the payload, the hook exits 0 without writing anything
- [ ] Hook exits 0 in all cases (never blocks PreToolUse)

### AC-5: PostToolUse — trajectory record written

- [ ] When the hook receives a `PostToolUse` event for a tool_use_id that has a matching entry in `_timings.json`, it appends one JSON line to `.ievo/debug/sessions/<session-id>/trajectory.jsonl`
- [ ] The appended record contains: `seq`, `tool`, `input`, `output` (truncated to 8192 chars if longer), `duration_ms`, `ts_start`, `ts_end`
- [ ] `seq` is 1-based and increments with each appended record (derived from line count of existing trajectory.jsonl before appending)
- [ ] `duration_ms` is computed as `(ts_end - ts_start)` in milliseconds, where `ts_start` is read from `_timings.json`
- [ ] If `_timings.json` has no entry for this `tool_use_id`, the record is still written but `duration_ms` is `null` and `ts_start` is `null`
- [ ] Hook exits 0 in all cases

### AC-6: PostToolUse — thinking blocks captured

- [ ] When the `tool_response` payload contains one or more content blocks with `"type": "thinking"`, the hook appends a section to `.ievo/debug/sessions/<session-id>/thinking.md`
- [ ] Each section has a header `## Call <seq> — <tool_name>` followed by the thinking content and a `---` separator
- [ ] If no thinking blocks are present in the response, `thinking.md` is not modified (and not created if it doesn't exist)
- [ ] Hook exits 0 whether or not thinking blocks are present

### AC-7: Stop hook — outcome recorded

- [ ] When the hook receives a `Stop` event, it writes `.ievo/debug/sessions/<session-id>/outcome.yaml`
- [ ] `outcome.yaml` contains: `session_id`, `stop_reason`, `ts_end`, `transcript_path`
- [ ] `stop_reason` is taken from the Stop payload; if absent, defaults to `"unknown"`
- [ ] `transcript_path` is taken from the Stop payload; if absent, defaults to `null`
- [ ] After writing `outcome.yaml`, the hook deletes `_timings.json` if it exists
- [ ] Hook exits 0 in all cases

### AC-8: Hook script robustness

- [ ] Hook exits 0 when stdin is empty or contains invalid JSON
- [ ] Hook exits 0 when `session_id` is missing from the payload
- [ ] Hook exits 0 when `hook_event_name` is an unrecognized value
- [ ] Hook exits 0 when the `.ievo/` directory does not exist (no project in cwd — standalone Claude session)
- [ ] Hook exits 0 when `.ievo/debug/sessions/<session-id>/` directory cannot be created (permissions error) — logs error to stderr but does not crash
- [ ] All error conditions produce output on stderr only (not stdout); stdout is reserved for hook decision JSON if needed

### AC-9: ievo uninstall removes the hook script

- [ ] `ievo uninstall --force` removes `.claude/hooks/debug_capture.py`
- [ ] `.ievo/debug/` directory and its contents are NOT removed by `ievo uninstall`
- [ ] `.claude/hooks/debug_capture.py` appears in the pre-confirmation summary listing when it exists

### AC-10: Hook script is stdlib-only

- [ ] `debug_capture.py` imports only Python stdlib modules (json, os, sys, pathlib, datetime, yaml is NOT available — use json for outcome file if needed or write yaml manually)
- [ ] Running `python3 .claude/hooks/debug_capture.py` with empty stdin in a clean virtualenv with no ievo package installed exits 0 without ImportError

## Negative Acceptance Criteria

- Does NOT block or modify any tool call — hook always exits 0
- Does NOT import or depend on ievo package modules (stdlib only)
- Does NOT store data in `~/.ievo/` (global) — only in `.ievo/debug/` (project-local)
- Does NOT capture conversation transcript text (that is the PreCompact hook's responsibility) — only tool call sequences and outcomes
- Does NOT remove `.ievo/debug/` on uninstall — session data is project data and is preserved
- Does NOT send captured data anywhere remotely — local storage only in this REQ
- Does NOT encrypt or compress trajectory files — plain text JSONL
- Does NOT require a running ievo process — hooks are standalone Python scripts
- Does NOT add a user-facing `ievo debug` CLI command — data access is via filesystem only (future REQ)
- Does NOT capture user messages or assistant text responses — only tool calls (input + output) and outcomes
- Does NOT write to `trajectory.jsonl` for the Stop event (Stop produces only `outcome.yaml`)

## Technical Notes

### outcome.yaml — use PyYAML-free approach

Since hooks are stdlib-only and `yaml` is not available, write `outcome.yaml` by constructing the YAML text manually (simple key: value format, no nested structures requiring a YAML library):

```python
lines = [
    f"session_id: {session_id}\n",
    f"stop_reason: {stop_reason}\n",
    f"ts_end: \"{ts_end}\"\n",
    f"transcript_path: {transcript_path or 'null'}\n",
]
Path(outcome_path).write_text("".join(lines))
```

### Hook event dispatch

The single script handles all three events by inspecting `hook_event_name` in the payload:

```python
event = payload.get("hook_event_name", "")
if event == "PreToolUse":
    handle_pre_tool_use(payload, session_dir)
elif event == "PostToolUse":
    handle_post_tool_use(payload, session_dir)
elif event == "Stop":
    handle_stop(payload, session_dir)
# Unknown events: exit 0 silently
```

### Project root detection (stdlib-only)

```python
def find_ievo_root(start: Path) -> Path | None:
    current = start
    for _ in range(20):  # max 20 levels up
        if (current / ".ievo" / "manifest.yaml").exists():
            return current
        parent = current.parent
        if parent == current:
            break
        current = parent
    return None
```

If `find_ievo_root()` returns None, the hook exits 0 (not in an iEvo project — ignore).

### `settings.local.json` updated template

```json
{
  "hooks": {
    "PreCompact": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "python3 .claude/hooks/precompact_save.py"
          }
        ]
      }
    ],
    "PreToolUse": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "python3 .claude/hooks/debug_capture.py"
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "python3 .claude/hooks/debug_capture.py"
          }
        ]
      }
    ],
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "python3 .claude/hooks/debug_capture.py"
          }
        ]
      }
    ]
  }
}
```

### .gitignore entry

Add to the `GITIGNORE` template constant in `src/ievo/commands/init_templates.py`:

```
# iEvo debug sessions (large, local only)
.ievo/debug/sessions/
```

### Timing file cleanup

`_timings.json` is a temporary file. The Stop hook deletes it. If a session crashes without a Stop event, `_timings.json` will remain — this is acceptable. The file is prefixed with `_` to distinguish it from captured data files. Future cleanup: if the session directory has `outcome.yaml` already, `_timings.json` is already deleted; a future `ievo debug clean` command could sweep stale timing files.

## Examples

### Example 1: trajectory.jsonl after two tool calls

```jsonl
{"seq": 1, "tool": "Bash", "input": {"command": "uv run pytest"}, "output": "===== 5 passed =====", "duration_ms": 1420, "ts_start": "2026-03-04T10:00:00.000Z", "ts_end": "2026-03-04T10:00:01.420Z"}
{"seq": 2, "tool": "Read", "input": {"file_path": "/app/main.py"}, "output": "def main():\n    ...", "duration_ms": 12, "ts_start": "2026-03-04T10:00:05.000Z", "ts_end": "2026-03-04T10:00:05.012Z"}
```

### Example 2: outcome.yaml after normal session end

```yaml
session_id: abc123
stop_reason: end_turn
ts_end: "2026-03-04T10:05:00.000Z"
transcript_path: /home/user/.claude/transcript-abc123.jsonl
```

### Example 3: thinking.md with one thinking block

```markdown
## Call 3 — Bash

The test output shows a KeyError on line 42. The root cause is likely...

---
```

### Example 4: Hook with no iEvo project — silent exit

```
$ echo '{"hook_event_name":"PreToolUse","session_id":"x","tool_name":"Bash","tool_input":{}}' | python3 .claude/hooks/debug_capture.py
# (no output, exit 0 — .ievo/manifest.yaml not found)
```

## Test Specification

Tests follow the pattern in `tests/test_init.py` and `tests/test_uninstall.py`. The hook script itself is tested by spawning it as a subprocess (which also validates the stdlib-only constraint) or by importing its functions directly after confirming no ievo imports.

### Tests to add in `tests/test_debug_capture.py` (new file)

| Test | What it verifies |
|------|-----------------|
| `test_pre_tool_use_writes_timing` | PreToolUse payload → `_timings.json` entry written |
| `test_pre_tool_use_merges_timing` | Second PreToolUse → existing entry preserved, new entry added |
| `test_pre_tool_use_missing_tool_use_id` | No `tool_use_id` in payload → exits 0, no file written |
| `test_post_tool_use_appends_trajectory` | PostToolUse with known timing → trajectory.jsonl line appended |
| `test_post_tool_use_seq_increments` | Two PostToolUse calls → seq values are 1 and 2 |
| `test_post_tool_use_duration_ms_computed` | `duration_ms` equals ts_end - ts_start in ms |
| `test_post_tool_use_output_truncated` | Output longer than 8192 chars → truncated to 8192 in record |
| `test_post_tool_use_no_timing_entry` | No matching timing → record written with `duration_ms: null` |
| `test_post_tool_use_thinking_blocks_written` | Response with thinking block → thinking.md section appended |
| `test_post_tool_use_no_thinking_blocks` | Response without thinking → thinking.md not created |
| `test_post_tool_use_multiple_thinking_blocks` | Multiple thinking blocks in one response → multiple sections in thinking.md |
| `test_stop_writes_outcome_yaml` | Stop payload → outcome.yaml written with all fields |
| `test_stop_missing_stop_reason_defaults_to_unknown` | Stop payload without stop_reason → `stop_reason: unknown` |
| `test_stop_deletes_timings_file` | Stop hook → `_timings.json` deleted if it exists |
| `test_stop_no_timings_file_ok` | Stop hook with no `_timings.json` → exits 0 (no error) |
| `test_empty_stdin_exits_0` | Empty stdin → exit code 0 |
| `test_invalid_json_stdin_exits_0` | Malformed JSON on stdin → exit code 0 |
| `test_missing_session_id_exits_0` | Payload without `session_id` → exit code 0, no files written |
| `test_unknown_event_exits_0` | `hook_event_name: "Unknown"` → exit code 0, no files written |
| `test_no_ievo_project_exits_0` | No `.ievo/manifest.yaml` in any parent → exit code 0, no files written |
| `test_stdlib_only_no_ievo_import` | Script importable in clean environment without ievo package |

### Tests to add in `tests/test_init.py`

| Test | What it verifies |
|------|-----------------|
| `test_init_creates_debug_capture_hook` | `.claude/hooks/debug_capture.py` exists after init |
| `test_init_debug_capture_hook_not_empty` | Hook file has content (not placeholder) |
| `test_init_debug_capture_hook_idempotent` | Re-init does not overwrite existing hook file |
| `test_init_creates_ievo_debug_dir` | `.ievo/debug/` directory created after init |
| `test_init_settings_has_pre_tool_use_hook` | `settings.local.json` contains `PreToolUse` entry for `debug_capture.py` |
| `test_init_settings_has_post_tool_use_hook` | `settings.local.json` contains `PostToolUse` entry for `debug_capture.py` |
| `test_init_settings_has_stop_hook` | `settings.local.json` contains `Stop` entry for `debug_capture.py` |
| `test_init_settings_precompact_still_present` | `settings.local.json` still contains `PreCompact` entry for `precompact_save.py` |
| `test_init_gitignore_has_debug_sessions` | `.gitignore` contains `.ievo/debug/sessions/` |

### Tests to add in `tests/test_uninstall.py`

| Test | What it verifies |
|------|-----------------|
| `test_uninstall_force_removes_debug_capture_hook` | `--force` removes `.claude/hooks/debug_capture.py` |
| `test_uninstall_preserves_ievo_debug_dir` | `ievo uninstall --force` does NOT remove `.ievo/debug/` |
| `test_uninstall_debug_capture_in_summary` | Pre-confirmation summary lists `.claude/hooks/debug_capture.py` when it exists |

## Dependencies

- **REQ-000** — project setup (init, uninstall infrastructure)
- Claude Code hooks system (`PreToolUse`, `PostToolUse`, `Stop`) — runtime dependency (not a Python package)
- Python stdlib only in the hook script: `json`, `os`, `sys`, `pathlib`, `datetime`
