# CLAUDE.md

## Debug Snapshots (llmdebug)

This project uses `llmdebug` for structured crash diagnostics. The pytest plugin auto-captures snapshots on test failures.

### On any failure:
1. **Read `.llmdebug/latest.json` first** (or run `llmdebug show --json`) - never patch before reading the snapshot
2. Analyze the snapshot:
   - **Exception type/message** - what went wrong
   - **Closest frame** - where it happened (index 0 is crash site)
   - **Locals** - variable values at crash time
   - **Array shapes** - look for empty arrays, shape mismatches, wrong dtypes
3. **Produce 2-4 ranked hypotheses** based on evidence before patching
4. Apply minimal fix for the most likely hypothesis
5. Re-run to verify

### CLI tools for debugging:
- `llmdebug show` - View latest snapshot (defaults to crash-level detail)
- `llmdebug show --detail full` - Show all frames
- `llmdebug show --detail context` - Everything including repro, git, env
- `llmdebug hypothesize` - Auto-generate ranked debugging hypotheses
- `llmdebug diff` - Compare latest vs previous snapshot to see what changed

### Key signals to look for:
- `shape: [0, ...]` - empty array, upstream data issue
- `None` where object expected - initialization/ordering bug
- Type mismatch in locals - wrong function called or bad return value
- Index vs length - `i=10` with `len(arr)=10` means off-by-one

### When the snapshot isn't enough:
If the crash frame locals show the symptom but not the cause:
1. **Add targeted instrumentation** (not random fixes):
   - Wrap suspect code in `with snapshot_section("stage_x")`
   - Focus on the boundary where values go wrong
2. Re-run to generate a better snapshot
3. Repeat the hypothesis→patch loop

### Don't:
- Guess without reading the snapshot first
- Make multiple speculative changes - one hypothesis at a time
- Refactor until tests pass - prefer small, targeted fixes
- Ignore array shapes - they're often the key clue

### Debugging workflow summary:
```
fail → read snapshot → ranked hypotheses → minimal patch → rerun
```

This replaces:
```
fail → guess patch → rerun → repeat (the LLM roulette)
```

## Project Architecture

### Core modules (`src/llmdebug/`):

**Capture pipeline:**
- `capture.py` - Exception capture engine
- `config.py` - `SnapshotConfig` dataclass with all configuration options
- `serialize.py` - Safe serialization of locals (arrays, dataframes, etc.)
- `output.py` - Atomic file writes, snapshot storage, `get_latest_snapshot()`
- `compact_keys.py` - Key abbreviation for `json_compact` format
- `detail_filter.py` - Layered disclosure (`crash`/`full`/`context` detail levels)
- `error_categories.py` - Auto-classification with suggestions
- `hypotheses.py` - Hypothesis generation engine (10 pattern detectors)
- `_debug_session.py` - DebugSession envelope handling and normalization
- `_snapshot_loader.py` - Snapshot loading utilities (used by CLI and MCP)
- `_formatting.py` - Shared formatting functions for text output
- `snapshot_types.py` - TypedDict definitions for snapshot payload types (strict typing support)

**Context collectors:**
- `coverage_context.py` - pytest-cov integration for coverage data
- `git_context.py` - Git commit/branch/dirty capture
- `async_context.py` - Asyncio task info capture
- `log_capture.py` - Recent log record capture

**RCA workflow:**
- `rca_state.py` - RCA state machine and workflow tracking
- `gates.py` - RCA gate logic (strict/soft/off modes)
- `state_db.py` - SQLite state database for RCA and snapshot metadata persistence
- `rca_store.py` - RCA persistence layer (queries/writes `state_db`)

**Production integration:**
- `hooks.py` - Production exception hooks with rate limiting and PII redaction
- `middleware.py` - WSGI/ASGI middleware for web frameworks
- `redaction_policy.py` - PII redaction policies

**Comparison:**
- `snapshot_diff.py` - Snapshot comparison logic

**Interfaces (kept thin):**
- `cli.py` - Click-based CLI (`show`, `list`, `frames`, `diff`, `hypothesize`, `git-context`, `clean`)
- `mcp_server.py` - MCP server with 10 tools for IDE integration
- `pytest_plugin.py` - Automatic capture on test failures
- `jupyter_plugin.py` - Jupyter/IPython integration plugin
- `toon_encoder.py` - TOON format output encoder
- `_presentation.py` - Shared presentation helpers for HTML/text snapshot rendering

### Code Quality Checks

All tool configs live in `pyproject.toml`. Install dev dependencies with `uv sync --extra dev`.

#### Quick check (pre-commit essentials)
```bash
uv run ruff check src tests                                          # Lint (style, bugs, perf)
uv run ruff format --check src tests                                 # Format check
uv run pyright                                                       # Static type checking
uv run pytest -x -q                                                  # Tests (fail-fast)
```

#### Full suite
```bash
# Lint & format (evals/cases and evals/artifacts auto-excluded via pyproject.toml force-exclude)
uv run ruff check src tests evals                                     # Lint all non-buggy code
uv run ruff format src tests evals                                    # Auto-format

# Type checking
uv run pyright                                                       # Strict mode, src/ only

# Testing & coverage
uv run pytest                                                        # Full test suite
uv run pytest --cov=src/llmdebug --cov-report=term-missing           # With coverage report
uv run pytest --cov=src/llmdebug --cov-report=xml --cov-fail-under=75  # CI gate

# Security
uv run bandit -r src/llmdebug -c pyproject.toml                      # Security scan

# Dead code
uv run vulture src/llmdebug vulture_whitelist.py --min-confidence 90  # Dead code detection

# Dependency hygiene
uv run deptry src/llmdebug                                           # Unused/missing/transitive deps

# Complexity
uv run radon cc -s -a src/llmdebug                                   # Cyclomatic complexity report
uv run xenon --max-absolute C --max-modules C --max-average B src/llmdebug  # Complexity gate

# PR coverage enforcement
uv run diff-cover coverage.xml --compare-branch=origin/main --fail-under=80

# Mutation testing (targeted, slow — run on specific modules)
uv run mutmut run --paths-to-mutate=src/llmdebug/compact_keys.py
```

#### Tool summary

| Tool | What it does | Config location |
|------|-------------|-----------------|
| **ruff** | Linting (E/W/F/I/UP/B/SIM/C4/RUF/PERF/FURB) + formatting | `[tool.ruff]` |
| **pyright** | Static type checking (strict mode) | `[tool.pyright]` |
| **pytest-cov** | Test coverage measurement | `[tool.coverage.*]` |
| **bandit** | Security vulnerability scanning | `[tool.bandit]` |
| **vulture** | Dead/unreachable code detection | CLI flags only |
| **deptry** | Dependency hygiene (unused/missing/transitive) | `[tool.deptry]` |
| **radon** | Cyclomatic complexity reporting | CLI flags only |
| **xenon** | Complexity gate (fails if thresholds exceeded) | CLI flags only |
| **diff-cover** | Coverage enforcement on changed lines only | CLI flags only |
| **mutmut** | Mutation testing for pure-function modules | CLI flags only |

#### Baselines & thresholds (ratchet over time)

- **Coverage floor**: `>=75%` (currently ~77%). Ratchet up as coverage improves.
- **Changed-line coverage**: `>=80%` on PRs via diff-cover.
- **Complexity**: xenon gate at `--max-absolute C --max-modules C --max-average B` (average B ~6). Ratchet down toward A.
- **Type checking**: pyright `strict` mode. 0 errors, 0 warnings.
- **Vulture**: `--min-confidence 90`. Treat findings as cleanup candidates.
- **Bandit**: All findings suppressed with `# nosec BXXX` + justification. Zero tolerance for new unsuppressed findings.
- **Deptry**: All legitimate exceptions documented in `[tool.deptry.per_rule_ignores]`.

### Quality Metrics Policy:
- Enforce package-level coverage floor: `>=75%`.
- Enforce changed-line coverage on PRs: `>=80%`.
- Treat vulture findings (`--min-confidence 90`) as candidates for cleanup or explicit justification.
- Treat deptry findings as dependency declaration/cleanup tasks.
- Watch complexity trend using radon; xenon gate prevents regressions beyond current baseline.

### Architecture Guardrails:
- Keep interface/entrypoint modules (`cli.py`, `mcp_server.py`, `pytest_plugin.py`, `jupyter_plugin.py`, `middleware.py`) thin.
- Keep reusable core logic in composable modules (`capture.py`, `serialize.py`, `output.py`, `snapshot_diff.py`, `rca_state.py`, `gates.py`, `_formatting.py`, `_snapshot_loader.py`, `_debug_session.py`, and supporting utilities).
- Avoid importing interface modules from core modules; dependencies should point from interfaces to core.

### Eval framework (`evals/`):

See `evals/README.md` for the comprehensive guide. Key components:
- `run_eval.py` - A/B eval harness (traceback-only vs with-snapshot)
- `analyze_results.py` - Results analysis with category/source slicing
- `validate_cases.py` - Case schema and execution validation
- `stats.py` - Statistical analysis (McNemar's, Cohen's h, bootstrap CI, Holm-Bonferroni)
- `category_mapping.py` - External dataset type → llmdebug category mappings
- `patchers/` - Patch generation backends (command, openai, heuristic, null)
- `importers/` - Dataset importers (DebugBench, HumanEvalPack, ConDefects, mutmut)
- `cases/` - benchmark cases (see `evals/README.md` for live inventory counts)

### Extras:
- `pip install llmdebug[cli]` - CLI (click, rich)
- `pip install llmdebug[mcp]` - MCP server
- `pip install llmdebug[toon]` - TOON output format
- `pip install llmdebug[jupyter]` - Jupyter/IPython plugin
- `pip install llmdebug[evals]` - Eval dataset importers (HuggingFace datasets)

## Release Workflow

This project uses **python-semantic-release (PSR)** for automated releases.

### Conventional Commits

Use conventional commit messages - PSR parses them to determine version bumps:

| Commit Type | Example | Version Bump |
|-------------|---------|--------------|
| `feat:` | `feat: add GPU memory tracking` | Minor (0.1.4 → 0.2.0) |
| `fix:` | `fix: handle empty arrays` | Patch (0.1.4 → 0.1.5) |
| `perf:` | `perf: optimize serialization` | Patch |
| `feat!:` or `BREAKING CHANGE:` | `feat!: change API` | Major (0.1.4 → 1.0.0) |
| `docs:`, `test:`, `chore:` | `docs: update README` | None |

Format: `<type>(<optional scope>): <description>`

### How Releases Work

1. Push commits to `main` with conventional commit messages
2. GitHub Actions runs CI (tests, lint, typecheck)
3. Release workflow analyzes commits since last tag
4. If releasable commits found: bumps version, updates CHANGELOG.md, creates tag, publishes to PyPI

### Manual Commands

```bash
# Preview what would be released (dry run)
uv run semantic-release version --noop

# Check current version
uv run semantic-release version --print
```

### CI/CD Files

- `.github/workflows/ci-cd.yml` - Runs on PR/push: test matrix + quality gates (coverage, complexity, dead code, dependency hygiene)
- `.github/workflows/release.yml` - Runs on main: auto-release to PyPI
