# Quality Testing Guide

This guide explains how to use Logos ADK's quality testing features.

## Overview

The quality system provides:
- **Regression Testing** — Validate agent behavior against examples
- **Load Testing** — Test performance under concurrency
- **Run Comparison** — Compare results between runs

## Quality Endpoints

### Run Quality Suite

```
POST /quality/run
```

**Regression Test:**
```json
{
  "kind": "regression",
  "suite": {
    "examples": [
      {
        "name": "greeting-1",
        "prompt": "Say hello",
        "expected": "Hello!"
      },
      {
        "name": "greeting-fail",
        "prompt": "Say goodbye",
        "expected": "Goodbye!"
      }
    ]
  }
}
```

Response:
```json
{
  "status": "completed",
  "run_id": "run-quality-123",
  "summary": {
    "passed": 1,
    "failed": 1,
    "requests": 2
  },
  "reports": {
    "html": "/quality/runs/run-quality-123/report.html"
  }
}
```

**Load Test:**
```json
{
  "kind": "load-test",
  "suite": {
    "prompts": [
      "What is the weather?",
      "Tell me a joke"
    ],
    "iterations": 3
  },
  "background": true
}
```

For background runs, response is HTTP 202:
```json
{
  "status": "queued",
  "run_id": "run-quality-456",
  "message": "Run queued"
}
```

### List Quality Runs

```
GET /quality/runs?kind=regression&limit=10
```

```json
{
  "items": [
    {
      "run_id": "run-quality-123",
      "status": "completed",
      "kind": "regression",
      "created_at": "2026-03-26T12:00:00Z"
    }
  ]
}
```

### Get Run Details

```
GET /quality/runs/{run_id}
```

```json
{
  "run_id": "run-quality-123",
  "status": "completed",
  "kind": "regression",
  "summary": {
    "passed": 10,
    "failed": 2,
    "requests": 12
  },
  "reports": {
    "html": "/quality/runs/run-quality-123/report.html"
  }
}
```

### Compare Runs

```
GET /quality/compare?run_id_a=run-1&run_id_b=run-2
```

```json
{
  "summary_diff": {
    "passed": {"a": 10, "b": 12}
  },
  "row_count_diff": {
    "a": 12,
    "b": 12
  }
}
```

## Python Usage

### CLI

```bash
# Run regression suite
logos-adk quality run --kind regression --suite examples.json

# Run load test
logos-adk quality run --kind load-test --prompts prompts.txt --iterations 3

# List runs
logos-adk quality runs

# Compare runs
logos-adk quality compare run-1 run-2
```

### Programmatic

```python
from logos_adk.quality import QualitySuite, RegressionRunner

suite = QualitySuite(
    kind="regression",
    examples=[
        {"name": "test-1", "prompt": "Hi", "expected": "Hello!"},
        {"name": "test-2", "prompt": "Bye", "expected": "Goodbye!"},
    ]
)

runner = RegressionRunner(agent)
result = await runner.run(suite)

print(f"Passed: {result.summary['passed']}")
print(f"Failed: {result.summary['failed']}")
```

## Best Practices

1. **Use descriptive example names** — Include `-fail` suffix for expected failures
2. **Test edge cases** — Include boundary conditions
3. **Run load tests in background** — Large tests shouldn't block
4. **Compare runs regularly** — Track improvements/regressions
5. **Store examples in version control** — Track test changes