# llmdebug Static Codebase Review (Round 4)

Date: 2026-02-27

## Scope
- Architecture and code-quality review of the `src/llmdebug` runtime + MCP/CLI interfaces.
- Focus on module boundaries, contract consistency, quality gates, and maintainability hotspots.

## Findings

### P1: MCP JSON mode has a non-uniform error contract
- Evidence:
  - `src/llmdebug/mcp_server.py:419`
  - `src/llmdebug/mcp_server.py:546`
  - `src/llmdebug/mcp_server.py:770`
  - `src/llmdebug/mcp_server.py:972`
  - `tests/test_mcp_server.py:598`
- Risk:
  - MCP clients requesting `response_format="json"` could still receive plain text errors, forcing brittle parsing and creating avoidable integration failures.
- Recommendation:
  - For `response_format="json"`, return structured JSON envelopes for both success and failure, with stable `error.code` values.

### P2: Interface modules are overloaded (transport + orchestration + presentation)
- Evidence:
  - `src/llmdebug/mcp_server.py` (1160 LOC)
  - `src/llmdebug/cli.py` (856 LOC)
  - High-complexity entrypoints/functions from `radon` (e.g. C(20), C(19), C(18)).
- Risk:
  - Harder onboarding and higher regression risk because behavior is concentrated in adapter layers.
- Recommendation:
  - Extract shared application services (snapshot/evidence/rca orchestration) and keep CLI/MCP adapters thin.

### P2: RCA domain and persistence concerns are tightly coupled
- Evidence:
  - `src/llmdebug/rca_state.py:90`
  - `src/llmdebug/rca_state.py:208`
  - `src/llmdebug/rca_state.py:364`
  - `src/llmdebug/rca_state.py:567`
  - Maintainability: `radon mi` reports `C (8.12)` for `rca_state.py`.
- Risk:
  - High change coupling and slower safe iteration on RCA behavior.
- Recommendation:
  - Separate pure RCA transition logic from storage/migration adapters.

### P3: Quality gates are strong but local/CI alignment has gaps
- Evidence:
  - Local `just` coverage threshold: `70` (`justfile:26`, `justfile:114`)
  - Canonical/CI coverage threshold: `75` (`pyproject.toml:168`, `.github/workflows/ci-cd.yml:67`)
  - `deptry` fails on eval/report deps from package metadata (`pyproject.toml:33-35`, `pyproject.toml:193`).
- Risk:
  - Avoidable friction and mismatched local-vs-CI expectations.
- Recommendation:
  - Align thresholds and deptry policy for eval-only dependencies.

## Architecture Snapshot
- Internal layering is generally healthy:
  - No detected import cycles in `src/llmdebug`.
  - No reverse dependencies from core modules into interface modules (`cli`, `mcp_server`, `pytest_plugin`, `jupyter_plugin`, `middleware`).
- Main architectural risk is concentration, not cyclic coupling.

## Quality Report

### Commands and outcomes
- `uv run ruff check src tests`: **PASS**
- `uv run ruff format --check src tests`: **FAIL**
  - 9 files would be reformatted
- `uv run pyright`: **PASS**
  - `0 errors, 0 warnings, 0 informations`
- `uv run pytest -q`: **PASS**
  - `1068 passed, 5 skipped, 1 warning`
- `uv run radon cc -s -a src/llmdebug`: **PASS**
  - Average complexity: `B (5.1227)`
- `uv run xenon --max-absolute F --max-modules C --max-average B src/llmdebug`: **PASS**
- `uv run bandit -c pyproject.toml -r src/llmdebug`: **PASS**
  - `0 issues`
- `uv run vulture src/llmdebug vulture_whitelist.py --min-confidence 90`: **PASS**
- `uv run deptry src/llmdebug`: **FAIL**
  - `DEP002` for `polars`, `scikit-learn`, `scipy`

### Coverage snapshot
- Existing `coverage.xml` line coverage: `79.14%`

## Residual Risks
- Large adapter modules (`mcp_server.py`, `cli.py`) remain complex even though tests are strong.
- Dependency-hygiene failures remain unresolved.
- Formatting drift blocks full green local quality pipeline until fixed.

## Recommended remediation order
1. P1: Enforce structured JSON error contract for MCP evidence tools.
2. P2: Split interface-layer orchestration into shared services.
3. P2: Decouple RCA domain transitions from storage backends.
4. P3: Align local vs CI quality gates and deptry policy.
