# Quality Map

This guide maps the repository's local and CI quality checks to the parts of
the codebase they protect. Use it alongside [CONTRIBUTING.md](../CONTRIBUTING.md)
when deciding which commands to run before a PR.

## Quick command map

| Command | Scope | Blocking? | Notes |
|---------|-------|-----------|-------|
| `just check` | `src/llmdebug` + default tests | Yes | Fastest full pre-PR pass |
| `just quality` | Repo-wide quality contract | Yes | Mirrors the shared GitHub quality gate |
| `just test-integration` | `@pytest.mark.integration` tests | No | Heavy locally; CI runs this lane in the 3.13 quality job |
| `just type-coverage` | Public package type completeness | No | Informational snapshot of `pyright --verifytypes` |
| `just type-coverage-strict` | Public package type completeness | Yes | Fails on regression vs `quality/type_coverage_baseline.json` |
| `just typecheck-evals` | Bounded eval modules | No | Staged `pyright` pass in `basic` mode |

## Blocking contract for `src/llmdebug`

These checks are part of the normal shipping gate for the package:

- `ruff check` and `ruff format --check`
- strict `pyright` on `src/llmdebug`
- pytest with coverage over `src/llmdebug`
- line coverage floor `>= 85`
- branch coverage no-regression against `quality/coverage_baseline.json`
- public type-completeness no-regression against
  `quality/type_coverage_baseline.json`
- `bandit`, `deptry`, `pip-audit`, `vulture`, `radon`, `xenon`
- docs drift checks and SLURM script validation where applicable

## Staged contract for `evals/`

The eval harness is intentionally being brought under stronger static checks in
stages rather than all at once.

- `just typecheck-evals` runs `pyright -p pyrightconfig.evals.json`
- the initial module set is:
  - `evals/eval_config.py`
  - `evals/eval_types.py`
  - `evals/eval_summary.py`
  - `evals/select_cases.py`
  - `evals/validate_cases.py`
  - `evals/stats.py`
  - `evals/report_run_metrics.py`
- this pass uses `basic` mode and is currently surfaced in CI as
  informational (`continue-on-error: true`)

## Test routing

- Default `just test-quick`, `just test`, and `just test-no-cov` runs exclude
  `@pytest.mark.integration`.
- Use `@pytest.mark.integration` for tests that need container wrappers, stub
  services, or other heavier infrastructure.
- Run those tests explicitly with `just test-integration`.
- GitHub Actions also runs `just test-integration` in the 3.13 quality job so
  integration-marked eval smoke coverage stays automated.

## Updating baselines

Coverage and type-completeness baselines are both committed quality assets.

- Coverage baseline: `quality/coverage_baseline.json`
- Type-completeness baseline: `quality/type_coverage_baseline.json`

When intentionally ratcheting the package quality floor, regenerate the metric,
update the committed JSON file, and include the reason in the PR description.
