---
title: "Test Cases"
description: "Managing test cases, test suites, and verification artifacts in TraceRTM"
---

# Test Cases

Test artifacts verify that the system meets its requirements. They form the verification side of traceability, ensuring every requirement has appropriate test coverage.

## Test Artifact Types

| Type | Purpose | Example |
|------|---------|---------|
| Test | Individual test case | `test_login_valid_credentials` |
| TestSuite | Collection of tests | `AuthenticationTestSuite` |
| TestPlan | Testing strategy | `v2.0 Test Plan` |
| TestResult | Execution record | `Build #123 Results` |

## Creating Test Artifacts

### Basic Test Case

```python
from tracertm import TraceRTMClient

client = TraceRTMClient()

# Create test case
test = client.create_item(
    title="Test valid login credentials",
    item_type="test",
    description="""
    ## Objective
    Verify that users can login with valid credentials.

    ## Preconditions
    - User account exists
    - User is not locked

    ## Steps
    1. Navigate to login page
    2. Enter valid email
    3. Enter valid password
    4. Click Login button

    ## Expected Result
    - User redirected to dashboard
    - JWT token returned
    - Session created
    """,
    metadata={
        "test_type": "integration",
        "automated": True,
        "file_path": "tests/auth/test_login.py::test_login_valid",
        "priority": "high",
        "tags": ["smoke", "auth"]
    }
)

# Link to requirement it verifies
client.create_link(test.id, feature_auth.id, "verifies")
```

### Test with Expected Results

```python
# Test case with detailed expectations
test_edge = client.create_item(
    title="Test login with locked account",
    item_type="test",
    description="""
    ## Objective
    Verify locked accounts cannot login.

    ## Steps
    1. Lock user account (3 failed attempts)
    2. Attempt login with correct credentials

    ## Expected Result
    - HTTP 403 returned
    - Error message: "Account locked. Contact support."
    - No session created
    - Security event logged
    """,
    metadata={
        "test_type": "security",
        "automated": True,
        "file_path": "tests/auth/test_login.py::test_login_locked_account",
        "priority": "critical",
        "security_relevant": True
    }
)
```

### Test Suite

```python
# Create test suite
suite = client.create_item(
    title="Authentication Test Suite",
    item_type="testsuite",
    description="""
    Comprehensive tests for authentication functionality.

    ## Scope
    - Login flow
    - Logout flow
    - Password reset
    - Session management
    - OAuth2 integration

    ## Not in Scope
    - MFA (separate suite)
    - Admin authentication (separate suite)
    """,
    metadata={
        "test_count": 45,
        "estimated_duration": "15 minutes",
        "parallel_safe": True
    }
)

# Link tests to suite
for test_id in test_ids:
    client.create_link(test_id, suite.id, "part_of")
```

### Test Plan

```python
# Test plan for release
test_plan = client.create_item(
    title="v2.0 Release Test Plan",
    item_type="testplan",
    description="""
    ## Objective
    Ensure v2.0 release meets quality standards.

    ## Scope
    - All new features in v2.0
    - Regression on existing functionality
    - Performance under load

    ## Schedule
    - Unit tests: Continuous (CI)
    - Integration tests: Daily
    - E2E tests: Per build
    - Performance tests: Weekly
    - Security scan: Pre-release

    ## Entry Criteria
    - All features implemented
    - Code review complete
    - Unit test coverage > 80%

    ## Exit Criteria
    - All critical tests passing
    - No P1/P2 bugs open
    - Performance baseline met
    """,
    metadata={
        "release": "v2.0",
        "start_date": "2024-12-01",
        "end_date": "2024-12-15"
    }
)
```

## CLI Usage

```bash
# Create test case
tracertm create test "Test valid login" \
  --metadata '{"test_type": "integration", "automated": true}'

# Link test to requirement
tracertm link TEST-001 FEAT-001 --type verifies

# Create test suite
tracertm create testsuite "Auth Test Suite"

# List tests
tracertm list --type test
tracertm list --type test --metadata '{"automated": true}'

# Search tests
tracertm search "login" --type test

# Run coverage check
tracertm coverage --from feature --to test
```

## Test Types

### By Test Level

```yaml
# .tracertm/config.yaml
test_types:
  unit:
    description: "Tests individual functions/methods"
    scope: "Single module"
    automation: required
    coverage_target: 80%

  integration:
    description: "Tests component interactions"
    scope: "Multiple modules"
    automation: required
    coverage_target: 70%

  system:
    description: "Tests full system behavior"
    scope: "End-to-end"
    automation: recommended

  acceptance:
    description: "Validates against requirements"
    scope: "User scenarios"
    automation: optional
```

### Test Classification

```python
# Different test types for a feature
tests = [
    {
        "title": "Unit: validate_password returns True for valid",
        "type": "unit",
        "file": "tests/unit/test_validation.py"
    },
    {
        "title": "Integration: login API returns JWT",
        "type": "integration",
        "file": "tests/integration/test_auth_api.py"
    },
    {
        "title": "E2E: User can login via web",
        "type": "e2e",
        "file": "tests/e2e/test_login_flow.py"
    },
    {
        "title": "Performance: Login under 200ms at 1000 RPS",
        "type": "performance",
        "file": "tests/perf/test_login_load.py"
    },
    {
        "title": "Security: SQL injection in login form",
        "type": "security",
        "file": "tests/security/test_auth_injection.py"
    }
]

for test_data in tests:
    test = client.create_item(
        title=test_data["title"],
        item_type="test",
        metadata={
            "test_type": test_data["type"],
            "file_path": test_data["file"]
        }
    )
    client.create_link(test.id, feature.id, "verifies")
```

## Traceability for Tests

### Linking Tests to Requirements

```python
# Link test to multiple requirements
test = client.create_item(
    title="Test password complexity validation",
    item_type="test"
)

# This test verifies multiple requirements
client.create_link(test.id, "FEAT-001", "verifies")  # Login feature
client.create_link(test.id, "SEC-003", "verifies")   # Security requirement
client.create_link(test.id, "REG-005", "verifies")   # Regulatory requirement
```

### Linking Tests to Code

```python
# Link test to code it tests
client.create_link(test.id, code.id, "tests")

# Query: What tests cover this code?
tests = client.trace_backward(
    item_id="CODE-001",
    link_type="tests",
    source_type="test"
)
```

### Coverage Queries

```python
# Get test coverage for requirements
coverage = client.get_coverage(
    source_type="feature",
    target_type="test",
    link_type="verified_by"
)

print(f"Requirement coverage: {coverage.percentage}%")
for gap in coverage.gaps:
    print(f"  Missing tests: {gap.id} - {gap.title}")

# Get test coverage for code
code_coverage = client.get_coverage(
    source_type="code",
    target_type="test",
    link_type="tested_by"
)
```

## Auto-Linking Tests

### From Docstrings

```python
# Test file with traceability markers
def test_user_login():
    """
    Verifies: FEAT-001, FEAT-002
    Tests: CODE-001
    Type: integration
    Priority: high
    """
    response = client.post("/login", json=credentials)
    assert response.status_code == 200
```

### Scanner Configuration

```yaml
# .tracertm/scanner.yaml
test_scanner:
  frameworks:
    - pytest
    - unittest
    - jest

  patterns:
    # Docstring markers
    verifies_pattern: "Verifies:\\s*([\\w-,\\s]+)"
    tests_pattern: "Tests:\\s*([\\w-,\\s]+)"

    # File naming convention
    # tests/test_FEAT_001_login.py → links to FEAT-001
    file_pattern: "test_([A-Z]+-\\d+)_"

  auto_link: true
```

### Running Scanner

```bash
# Scan tests and create links
tracertm scan tests/ --framework pytest --auto-link

# Output:
# Scanning tests/...
# Found 142 test cases
# Created 89 new links
# Updated 12 existing links
# Skipped 41 (no traceability markers)
```

## Test Execution Tracking

### Recording Results

```python
# Record test execution
result = client.create_item(
    title="Build #456 Test Results",
    item_type="test_result",
    metadata={
        "build_id": 456,
        "executed_at": "2024-12-01T10:30:00Z",
        "duration_seconds": 847,
        "environment": "staging",
        "results": {
            "total": 142,
            "passed": 138,
            "failed": 3,
            "skipped": 1
        },
        "failed_tests": [
            {"id": "TEST-042", "error": "Timeout"},
            {"id": "TEST-089", "error": "Assertion failed"},
            {"id": "TEST-101", "error": "Connection refused"}
        ]
    }
)
```

### Test History

```bash
# View test execution history
tracertm history TEST-001 --results

# Output:
# TEST-001: Test valid login credentials
# ═══════════════════════════════════════
#
# Recent Executions:
#   Build #456 (Dec 1): ✓ Passed (1.2s)
#   Build #455 (Nov 30): ✓ Passed (1.1s)
#   Build #454 (Nov 29): ✗ Failed (timeout)
#   Build #453 (Nov 28): ✓ Passed (1.3s)
#
# Pass Rate: 75% (last 4 runs)
# Avg Duration: 1.2s
```

## Test Coverage Reports

### Requirement Coverage

```bash
# Generate test coverage report
tracertm report coverage --from feature --to test

# Output:
# Test Coverage Report
# ════════════════════
#
# Overall: 87% (52/60 features have tests)
#
# By Priority:
#   Critical: 100% (12/12) ✓
#   High: 95% (19/20)
#   Medium: 80% (16/20)
#   Low: 62% (5/8)
#
# Gaps (features without tests):
#   FEAT-042: Export to CSV [medium]
#   FEAT-055: Bulk delete [low]
#   FEAT-067: Archive items [low]
#   ...
```

### Code Coverage Integration

```bash
# Import code coverage from pytest-cov
tracertm import coverage --file coverage.xml --format cobertura

# Map to traceability
tracertm coverage analyze --source code --target test

# Output includes:
# - Line coverage per module
# - Branch coverage
# - Untested code paths
```

## Test Workflows

### Test Case Workflow

```yaml
# .tracertm/workflows/test.yaml
name: test
states:
  - draft
  - review
  - approved
  - automated
  - deprecated

transitions:
  - from: draft
    to: review
    requires:
      - field: description
        min_length: 50
      - link_type: verifies
        target_type: feature
        min_count: 1

  - from: approved
    to: automated
    requires:
      - field: metadata.file_path
        not_null: true
```

## Best Practices

### 1. Link Tests to Requirements

```python
# Every test should verify at least one requirement
def create_test_with_traceability(title, verifies):
    test = client.create_item(title=title, item_type="test")
    for req_id in verifies:
        client.create_link(test.id, req_id, "verifies")
    return test
```

### 2. Use Meaningful Test Names

```
✓ test_login_with_valid_credentials_returns_jwt
✓ test_login_with_locked_account_returns_403
✓ test_password_reset_sends_email_within_30s

✗ test_login_1
✗ test_feature_works
✗ test_bug_fix
```

### 3. Categorize Tests Appropriately

```python
# Tag tests for selective execution
test = client.create_item(
    title="Test login performance",
    item_type="test",
    metadata={
        "test_type": "performance",
        "tags": ["slow", "nightly", "load"],
        "timeout": 300
    }
)
```

### 4. Maintain Test-Requirement Ratio

```
Guidelines:
- Critical requirements: 5+ tests
- High priority: 3+ tests
- Medium priority: 2+ tests
- Low priority: 1+ test
```

## Related Topics

- [Requirements](/docs/wiki/concepts/artifacts/requirements) - What tests verify
- [Code](/docs/wiki/concepts/artifacts/code) - What tests exercise
- [Coverage Reports](/docs/getting-started/first-project/reports) - Gap analysis
- [Test Automation](/docs/wiki/concepts/workflows/automation) - CI/CD integration

