# Repository Guidelines

## Project Structure & Module Organization
- `src/llmdebug/`: core package code (capture pipeline, serialization, RCA workflow, CLI, pytest plugin, MCP server, Jupyter plugin, middleware).
- `tests/`: pytest suite for library and eval tooling (`test_*.py` files).
- `evals/`: local benchmark harness — see `evals/README.md` for comprehensive guide.
  - `run_eval.py`, `analyze_results.py`, `validate_cases.py` — eval pipeline.
  - `patchers/` — patch generation backends (command, openai, heuristic, null).
  - `importers/` — dataset importers (DebugBench, HumanEvalPack, ConDefects, mutmut).
  - `stats.py` — statistical analysis (McNemar's, Cohen's h, bootstrap CI).
  - `category_mapping.py` — external dataset type mappings.
  - `cases/<case_id>/` — benchmark cases (see `evals/README.md` for live inventory counts).
- `docs/` and `reports/`: roadmap notes and analysis outputs.
- Generated artifacts should not be committed: `.llmdebug/`, `evals/artifacts/`, `evals/results/*.jsonl`, `reports/`, `dist/`.

## Build, Test, and Development Commands
Use `uv` for dependency and task management.

```bash
uv sync --extra dev                 # Install runtime + dev dependencies
uv run ruff check src tests         # Lint and import/order checks
uv run pyright                      # Static type checking (standard mode)
uv run pytest                       # Full test suite
uv run pytest -k snapshot_diff      # Targeted test run
uv run pytest --cov=src/llmdebug --cov-report=xml --cov-fail-under=70  # Coverage gate
uv run vulture src --min-confidence 90                                  # Dead code scan (high confidence)
uv run deptry src/llmdebug                                                     # Dependency hygiene
uv run radon cc -s -a src/llmdebug                                      # Cyclomatic complexity report
uv run xenon --max-absolute F --max-modules C --max-average B src/llmdebug  # Complexity guardrail
uv run diff-cover coverage.xml --compare-branch=origin/main --fail-under=80  # PR diff coverage
uv build --no-sources               # Build wheel/sdist (release-style build)
```

Eval case validation before benchmark runs:

```bash
uv run python -m evals.validate_cases --schema-only
```

## Research Compute Best-Practices (Hybrid)

### Generic best-practices (portable)
1. Put a hard smoke test in front of every long run.
- Check end-to-end data loading, minimal training/eval step, and output paths.
2. Track effective GPU utilization, not just runtime.
- Low utilization usually means dataloader, CPU preprocessing, or storage bottlenecks.
3. Keep a tiny experiment contract per run.
- Deterministic seed controls logged.
- One config snapshot (JSON/YAML) stored with results.
- Unique run ID plus git commit hash recorded.
- Metrics written in one machine-readable format (JSON/CSV).
4. Use successive halving for sweeps.
- Many cheap short-budget configs first, then promote top fraction only.
5. Optimize queue strategy for throughput.
- Prefer modular 1-4h jobs when possible.
- Use checkpoint/resume so walltime limits are not destructive.
6. Make failures informative automatically.
- Persist traceback + config + last known batch/shape/NaN signals.

### HAICore + llmdebug examples (this repo)
```bash
slurm/check_env_gguf_eval.sh
slurm/run_gguf_smoke.sh
sbatch slurm/eval_gguf_eval.sh
slurm/submit_gguf_sweep.sh
slurm/submit_gguf_resume_chain.sh <RUN_ID> 4
slurm/collect_gguf_debug_bundle.sh <JOBID>
```

## Coding Style & Naming Conventions
- Python 3.10+ with 4-space indentation and max line length 100 (configured in `pyproject.toml`).
- Follow Ruff rule sets `E`, `F`, `I`, `UP`, `B`; keep imports sorted and avoid broad suppressions.
- Prefer type annotations for new/changed APIs to keep `pyright` clean.
- Naming: `snake_case` for modules/functions/variables, `PascalCase` for classes, `UPPER_SNAKE_CASE` for constants.
- Keep patches focused: small, test-backed changes over broad refactors.

## Testing Guidelines
- Framework: `pytest` (configured `testpaths = ["tests"]`).
- Test files: `test_*.py`; test functions: `test_*`.
- Bugs: add regression test when it fits.
- Add regression tests for each bug fix and behavior change.
- Use async tests when touching coroutine paths (`pytest-asyncio` is included in dev dependencies).
- CI enforces a package coverage floor (`--cov-fail-under=70`) and PR diff coverage (`diff-cover --fail-under=80`).

## Code Quality Metrics
- Coverage: use `pytest-cov` for total package coverage and `diff-cover` for changed-line coverage.
- Dead code: use `vulture` with `--min-confidence 90` for low-noise findings.
- Dependency hygiene: use `deptry` on `src/llmdebug` to detect undeclared/unused dependencies.
- Complexity: use `radon` for reports and `xenon` for hard gates (`max-absolute F`, `max-modules C`, `max-average B`).
- Architecture/cohesion: keep interface modules (`cli`, `mcp_server`, `pytest_plugin`, `jupyter_plugin`, `middleware`) thin and avoid introducing reverse dependencies from core modules into those interfaces.

## Commit & Pull Request Guidelines
- Prefer Conventional Commits (`feat:`, `fix:`, `perf:`, `docs:`, `test:`, `chore:`). Releases are automated via semantic-release.
- Use `feat!:` or a `BREAKING CHANGE:` footer for incompatible changes.
- PRs should include a concise problem/solution summary, linked issue(s), and evidence (tests or command output).
- Before opening a PR, run `ruff`, `pyright`, `pytest`, and the quality metric commands listed above.
