---
name: client-testing
description: >
  Testing patterns specific to the Avatar Python SDK client (services/client/) focusing on
  Manager/Runner interfaces and HTTP mocking. Use when testing SDK client for:
  (1) writing unit tests with httpx.MockTransport for API responses,
  (2) testing Manager/Runner workflow orchestration, (3) validating client-side processors,
  (4) running integration tests with notebook execution, or (5) testing SDK behavior without
  real API dependencies. DO NOT test auto-generated code (api.py, client.py, models.py).
---

# Client Testing - Avatar Monorepo

## Overview

This skill covers testing patterns specific to the **Avatar Python SDK** (`services/client/`), which provides high-level Manager/Runner interfaces and auto-generated API client code. Tests focus on SDK behavior, HTTP interactions, and integration with notebooks.

## Key Principles

- **Mock HTTP layer**: Use `httpx.MockTransport` to simulate API responses
- **Auto-generated code exempt**: DO NOT test auto-generated `api.py`, `client.py`, `models.py`
- **Manager/Runner focused**: Test high-level SDK patterns and workflows
- **Notebook validation**: Integration tests verify tutorial notebooks execute correctly
- **Fast unit tests**: No real API dependencies in unit tests

## Test Organization

```
services/client/tests/
├── conftest.py              # Mock HTTP client fixtures
├── fixtures/                # Test data files
│   ├── iris.csv
│   └── sample_data/
├── unit/                    # Fast, isolated tests
│   ├── conftest.py          # FakeApiClient fixture
│   ├── manager_test.py      # Manager class tests
│   ├── runner_test.py       # Runner class tests
│   ├── processors/          # Processor logic tests
│   └── lib/                 # Utility function tests
├── integration/             # Full workflow tests
│   ├── processors/          # Processor integration tests
│   └── notebooks/           # Notebook execution tests
└── e2e/                     # End-to-end with real API (optional)
```

## Important: Auto-Generated Code

### DO NOT Test These Files

```python
# ❌ DO NOT create tests for auto-generated code
services/client/src/avatars/api.py       # Auto-generated
services/client/src/avatars/client.py    # Auto-generated
services/client/src/avatars/models.py    # Auto-generated
```

**Why**: These files are regenerated from OpenAPI schema. Changes must be made in the API service, not the client.

**Instead**: Test the **Manager** and **Runner** classes that use these auto-generated clients.

### Regeneration Process

```bash
# When API schema changes
cd services/api
just export-open-api

cd ../client
just generate-python
```

## Mock HTTP Testing

### Basic Mock Pattern

**Pattern**: Use `httpx.MockTransport` to simulate API responses without real server.

```python
# tests/unit/conftest.py
import httpx
from avatars.client import ApiClient

RequestHandle = Callable[[httpx.Request], httpx.Response]


def mock_httpx_client(handler: RequestHandle | None = None) -> httpx.Client:
    """Generate HTTPX client with mock transport."""
    if handler is None:
        handler = lambda request: httpx.Response(200, json={})

    transport = httpx.MockTransport(handler)
    return httpx.Client(
        base_url="http://localhost:8000",
        transport=transport,
    )


def api_client_factory(handler: RequestHandle | None = None) -> ApiClient:
    """Generate API client with mock transport."""
    http_client = mock_httpx_client(handler)
    return ApiClient(
        base_url="http://localhost:8000/api",
        http_client=http_client,
        verify_auth=False,
    )
```

### Custom Response Handlers

```python
# tests/unit/manager_test.py
def test_manager_creates_job():
    """Test Manager creates job via API."""

    def handler(request: httpx.Request) -> httpx.Response:
        """Custom handler for job creation endpoint."""
        if request.url.path == "/api/jobs" and request.method == "POST":
            return httpx.Response(
                201,
                json={
                    "id": str(uuid4()),
                    "status": "created",
                    "name": "job-123",
                },
            )
        return httpx.Response(404, json={"error": "Not found"})

    client = api_client_factory(handler)
    manager = Manager(api_client=client)

    job = manager.create_job(set_name="test", parameters_name="params")

    assert job.status == "created"
```

## FakeApiClient Pattern

### Creating Test Doubles

```python
# tests/unit/conftest.py
from polyfactory.factories.pydantic_factory import ModelFactory
from avatars.models import JobWithDisplayNameResponse, JobKind


def create_fake_job(
    name: str = "test_job",
    set_name: UUID | None = None,
    display_name: str = "test_dataset",
    kind: JobKind = JobKind.standard,
    status: str = "finished",
    **kwargs,
) -> JobWithDisplayNameResponse:
    """Create a fake JobWithDisplayNameResponse for testing.

    See conftest.py for full parameter list and documentation.
    """
    # Implementation in conftest.py
    ...


class JobResponseFactory(ModelFactory[JobWithDisplayNameResponse]):
    """Factory for creating job responses with random data."""
    __model__ = JobWithDisplayNameResponse


class FakeJobs:
    """Fake Jobs API client for testing.

    Supports both direct initialization and builder pattern.
    """

    def __init__(self, jobs: list[JobWithDisplayNameResponse] | None = None):
        """Initialize with optional job list."""
        self._jobs = jobs if jobs is not None else []

    def with_job(
        self,
        name: str = "test_job",
        display_name: str = "test_dataset",
        **kwargs,
    ) -> "FakeJobs":
        """Add a job using builder pattern (chainable)."""
        job = create_fake_job(name=name, display_name=display_name, **kwargs)
        self._jobs.append(job)
        return self

    def get_jobs(self):
        """Return configured jobs as JobResponseList."""
        return JobResponseList(jobs=self._jobs)

    def get_job_status(self, name: str):
        """Return job status (checks configured jobs first)."""
        for job in self._jobs:
            if job.name == name:
                return job
        return create_fake_job(name=name, status="finished")


class FakeApiClient:
    """Fake API client for unit testing.

    Examples:
        # Default (empty jobs)
        client = FakeApiClient()

        # With pre-configured jobs
        jobs = [create_fake_job(), create_fake_job()]
        client = FakeApiClient(jobs=jobs)

        # Builder pattern
        client = FakeApiClient()
        client.jobs.with_job(name="job1").with_job(name="job2")
    """

    def __init__(
        self,
        tables: list[str] | None = None,
        jobs: list[JobWithDisplayNameResponse] | None = None,
    ):
        """Initialize fake API client.

        Parameters:
            tables: Table names for fake results
            jobs: Pre-configured jobs list
        """
        self.jobs = FakeJobs(jobs=jobs)
        self.results = FakeResults(tables=tables or [])
        self.resources = FakeResources()
        # ... other attributes
```

### Using FakeApiClient

**Pattern 1: Simple usage (no jobs needed)**

```python
# tests/unit/manager_test.py
class TestManager:
    def setup_method(self):
        """Create fresh manager for each test."""
        self.manager = Manager(api_client=FakeApiClient())

    def test_create_runner(self):
        """Test Manager creates Runner."""
        runner = self.manager.create_runner("test_set")
        assert isinstance(runner, Runner)
```

**Pattern 2: With pre-configured jobs**

```python
def test_get_jobs():
    """Test Manager lists jobs."""
    jobs = [
        create_fake_job(name="job1", display_name="dataset1"),
        create_fake_job(name="job2", display_name="dataset2"),
    ]
    client = FakeApiClient(jobs=jobs)
    manager = Manager(api_client=client)

    results = manager.get_jobs()
    assert len(results) == 2
```

**Pattern 3: Builder pattern**

```python
def test_with_builder():
    """Test using builder pattern for dynamic job creation."""
    client = FakeApiClient()
    client.jobs.with_job(
        name="job1",
        display_name="dataset1",
        kind=JobKind.standard,
    ).with_job(
        name="job2",
        display_name="dataset2",
        kind=JobKind.privacy_metrics,
    )

    manager = Manager(api_client=client)
    results = manager.get_jobs()
    assert len(results) == 2
```

**Pattern 4: Using create_fake_job helper**

```python
def test_find_jobs_by_display_name():
    """Test filtering jobs by display name."""
    now = datetime.now(UTC)
    jobs = [
        create_fake_job(
            name="job1",
            set_name=uuid4(),
            created_at=now - timedelta(days=1),
            display_name="my_dataset",
        ),
        create_fake_job(
            name="job2",
            set_name=uuid4(),
            created_at=now - timedelta(days=5),
            display_name="my_dataset",
        ),
    ]
    client = FakeApiClient(jobs=jobs)
    manager = Manager(api_client=client)

    found = manager.find_ids_by_name("my_dataset")
    assert len(found) == 2
```

## Testing Manager Class

### Manager Setup

```python
# tests/unit/manager_test.py
class TestManager:
    @classmethod
    def setup_class(cls):
        """Setup once for all tests in class."""
        cls.manager = Manager(api_client=FakeApiClient())

    def test_list_jobs(self):
        """Test Manager lists jobs."""
        jobs = self.manager.list_jobs()
        assert isinstance(jobs, list)

    def test_get_job_status(self):
        """Test Manager retrieves job status."""
        status = self.manager.get_job_status("job-123")
        assert status.status in ["created", "running", "finished"]
```

## Testing Runner Class

### Runner Test Pattern

```python
# tests/unit/runner_test.py
import pandas as pd
from avatars.manager import Manager
from avatars.runner import Runner


class TestRunner:
    df = pd.DataFrame({
        "col1": [1, 2, 3, 4, 5],
        "col2": [3, 4, 5, 6, 7],
    })

    @classmethod
    def setup_class(cls):
        """Setup Manager once."""
        cls.manager = Manager(api_client=FakeApiClient())

    def setup_method(self):
        """Create fresh runner for each test."""
        self.runner = self.manager.create_runner("test")

    def test_add_table_from_dataframe(self):
        """Test adding table from DataFrame."""
        self.runner.add_table("test_table", data=self.df)

        assert "test_table" in self.runner.config.tables.keys()

    def test_add_table_from_file(self):
        """Test adding table from file path."""
        self.runner.add_table("test_table", data="../fixtures/iris.csv")

        assert "test_table" in self.runner.config.tables.keys()

    def test_add_table_with_avatar(self):
        """Test adding table with avatar data."""
        self.runner.add_table(
            "test_table",
            data="fixtures/iris.csv",
            avatar_data="fixtures/iris.csv",
        )

        assert "test_table" in self.runner.config.tables.keys()
        assert "test_table" in self.runner.config.avatar_tables.keys()

    def test_run_raises_without_avatar_table(self):
        """Test run() fails when no avatar table specified."""
        self.runner.add_table("test", data=self.df)

        with pytest.raises(ValueError, match="No avatar table"):
            self.runner.run()
```

### Testing Runner Parameters

```python
# tests/unit/runner_test.py
def test_runner_set_parameters(self):
    """Test setting avatarization parameters."""
    self.runner.add_table("test_table", data=self.df)
    self.runner.set_parameters(
        "test_table",
        k=10,
        open_dp_epsilon=1.0,
        time_series_method=AlignmentMethod.FAST,
    )

    params = self.runner.config.avatarization["test_table"]
    assert params.k == 10
```

## Testing Processors

### Processor Unit Tests

**Pattern**: Test processor logic independently of API.

```python
# tests/unit/processors/datetime_test.py
from avatars.processors.datetime import DatetimeProcessor


def test_datetime_processor_parses_formats():
    """Test datetime processor handles various formats."""
    processor = DatetimeProcessor(format="%Y-%m-%d")

    df = pd.DataFrame({
        "date": ["2023-01-01", "2023-12-31", "2024-06-15"],
    })

    result = processor.process(df)

    assert result["date"].dtype == "datetime64[ns]"
```

### Processor Integration Tests

```python
# tests/integration/processors/geolocation_test.py
@pytest.mark.integration
def test_geolocation_normalization():
    """Test geolocation processor with real geocoding."""
    processor = GeolocationProcessor()

    df = pd.DataFrame({
        "address": ["1600 Amphitheatre Parkway, Mountain View, CA"],
    })

    result = processor.process(df)

    assert "latitude" in result.columns
    assert "longitude" in result.columns
    assert result["latitude"].notna().all()
```

## Notebook Testing

### Testing Tutorial Notebooks

**Pattern**: Execute notebooks and verify they run without errors.

```python
# tests/integration/notebooks/test_tutorials.py
import pytest
from pathlib import Path
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor


NOTEBOOKS_PATH = Path(__file__).parent.parent.parent.parent / "notebooks"


@pytest.mark.parametrize("notebook_name", [
    "Tutorial_1_Quickstart.ipynb",
    "Tutorial_2_Advanced_Features.ipynb",
    "Tutorial_3_Multi_Table.ipynb",
])
def test_notebook_execution(notebook_name):
    """Test that tutorial notebook executes without errors."""
    notebook_path = NOTEBOOKS_PATH / notebook_name

    with open(notebook_path) as f:
        nb = nbformat.read(f, as_version=4)

    # Execute notebook
    ep = ExecutePreprocessor(timeout=600, kernel_name="python3")
    ep.preprocess(nb, {"metadata": {"path": str(NOTEBOOKS_PATH)}})

    # Verify no errors in cells
    for cell in nb.cells:
        if cell.cell_type == "code":
            outputs = cell.get("outputs", [])
            for output in outputs:
                assert output.get("output_type") != "error", (
                    f"Error in {notebook_name}: {output.get('evalue', '')}"
                )
```

### Notebook Output Validation

```python
def test_notebook_produces_expected_output():
    """Test notebook produces expected visualizations."""
    notebook_path = NOTEBOOKS_PATH / "Tutorial_1_Quickstart.ipynb"

    with open(notebook_path) as f:
        nb = nbformat.read(f, as_version=4)

    ep = ExecutePreprocessor(timeout=600)
    ep.preprocess(nb, {"metadata": {"path": str(NOTEBOOKS_PATH)}})

    # Check specific cell outputs
    results_cell = nb.cells[10]  # Cell that displays results
    outputs = results_cell.get("outputs", [])

    assert len(outputs) > 0
    assert any("data" in output for output in outputs)
```

## Test Data Factories

### Using Polyfactory

```python
# tests/unit/conftest.py
from polyfactory.factories.pydantic_factory import ModelFactory
from avatars.models import (
    JobWithDisplayNameResponse,
    ResourceSetResponse,
    FileCredentials,
)


class JobResponseFactory(ModelFactory[JobWithDisplayNameResponse]):
    """Factory for job responses."""
    __model__ = JobWithDisplayNameResponse


class ResourceSetFactory(ModelFactory[ResourceSetResponse]):
    """Factory for resource set responses."""
    __model__ = ResourceSetResponse


class FileCredentialsFactory(ModelFactory[FileCredentials]):
    """Factory for file credentials."""
    __model__ = FileCredentials


# Usage
def test_with_factory():
    job = JobResponseFactory.build(
        name="custom-name",
        status="finished",
    )
    assert job.name == "custom-name"
```

### Metrics Factories

```python
# tests/unit/conftest.py
import json
import numpy as np


def privacy_metrics_factory(table_name: str) -> str:
    """Generate fake privacy metrics JSON."""
    return json.dumps({
        "local_cloaking": np.random.uniform(0, 100),
        "hidden_rate": np.random.uniform(0, 100),
        "metadata": {
            "table_name": table_name,
            "computation_type": "standalone",
        },
    })


def signal_metrics_factory(table_name: str) -> str:
    """Generate fake signal metrics JSON."""
    return json.dumps({
        "hellinger_mean": np.random.uniform(0, 100),
        "hellinger_std": np.random.uniform(0, 100),
        "metadata": {
            "table_name": table_name,
        },
    })
```

## Running Tests

```bash
# All tests
cd services/client
just test

# Unit tests only (fast)
just test tests/unit/

# Integration tests
just test tests/integration/

# Specific test
just test tests/unit/runner_test.py::TestRunner::test_add_table

# Skip notebook tests (slow)
just test -k "not notebook"

# With coverage
just test-coverage

# Verbose
just test -v
```

## Best Practices

### ✅ DO

```python
# Use FakeApiClient for unit tests
manager = Manager(api_client=FakeApiClient())


# Use mock_httpx_client for custom responses
def handler(request):
    return httpx.Response(200, json={...})

client = api_client_factory(handler)


# Test Manager/Runner behavior, not API internals
def test_runner_validates_input():
    runner = manager.create_runner("test")
    with pytest.raises(ValueError):
        runner.add_table("", data=df)  # Empty name


# Use setup_method for fresh state
def setup_method(self):
    self.runner = self.manager.create_runner("test")


# Test processor logic independently
processor = DatetimeProcessor(format="%Y-%m-%d")
result = processor.process(df)
```

### ❌ DON'T

```python
# Don't test auto-generated code
def test_api_client_get_jobs():  # Wrong! api.py is auto-generated
    ...


# Don't make real API calls in unit tests
manager = Manager(
    base_url="https://real-api.com",  # Wrong!
    username="real-user",
)


# Don't share mutable state between tests
class TestRunner:
    runner = manager.create_runner("test")  # Wrong! Shared state

    def test_a(self):
        self.runner.add_table(...)  # Modifies shared state


# Don't hardcode test data paths
df = pd.read_csv("/absolute/path/data.csv")  # Wrong!


# Don't test implementation details
assert runner._internal_cache == {}  # Wrong! Private API
```

## Common Patterns

### Testing Error Handling

```python
def test_runner_validates_table_name():
    """Test Runner rejects invalid table names."""
    runner = manager.create_runner("test")

    with pytest.raises(ValueError, match="Table name cannot be empty"):
        runner.add_table("", data=df)


def test_manager_handles_api_errors():
    """Test Manager handles API errors gracefully."""

    def handler(request):
        return httpx.Response(500, json={"error": "Internal server error"})

    manager = Manager(api_client=api_client_factory(handler))

    with pytest.raises(ApiError, match="Internal server error"):
        manager.create_job(...)
```

### Testing Configuration Building

```python
def test_runner_builds_correct_config():
    """Test Runner generates valid avatar-yaml config."""
    runner = manager.create_runner("test")
    runner.add_table("iris", data=df)
    runner.set_parameters("iris", k=10)

    config = runner.config

    assert config.avatar_metadata.spec.display_name == "test"
    assert "iris" in config.tables
    assert config.avatarization["iris"].k == 10
```

### Testing File Downloads

```python
# tests/unit/file_downloader_test.py
def test_file_downloader_downloads_results():
    """Test FileDownloader retrieves job results."""

    def handler(request):
        if "/results/" in request.url.path:
            return httpx.Response(
                200,
                content=b"fake,csv,data\n1,2,3",
                headers={"content-type": "text/csv"},
            )
        return httpx.Response(404)

    client = api_client_factory(handler)
    downloader = FileDownloader(client)

    data = downloader.download_result("job-123", "table_name")

    assert b"fake,csv,data" in data
```

## Quick Reference

| Task                    | Pattern                                     |
| ----------------------- | ------------------------------------------- |
| Mock HTTP client        | `api_client_factory(handler)`               |
| Fake API for unit tests | `FakeApiClient(jobs=[...])`                 |
| Fake API with builder   | `client.jobs.with_job(name=...).with_job()` |
| Create fake job         | `create_fake_job(name="job1", ...)`         |
| Test Runner setup       | `runner = manager.create_runner("test")`    |
| Test notebook execution | `ExecutePreprocessor().preprocess(nb)`      |
| Generate test data      | `JobResponseFactory.build()`                |
| Custom HTTP response    | Define handler function for `MockTransport` |

## Common Fixtures

- `api_client_factory(handler)` - Create API client with mock transport
- `FakeApiClient(jobs=...)` - Fake API client for unit tests with optional jobs list
- `create_fake_job(...)` - Create single fake job with specific attributes
- `JobResponseFactory` - Generate job response with random data
- `ResourceSetFactory` - Generate resource set test data
- `privacy_metrics_factory()` - Generate fake privacy metrics
- `signal_metrics_factory()` - Generate fake signal metrics

## Common Assertions

```python
# Configuration built correctly
assert "table" in runner.config.tables
assert runner.config.parameters.avatarization.k == 10

# Manager/Runner behavior
assert isinstance(runner, Runner)
assert runner.set_name == "test"

# HTTP interactions
assert request.method == "POST"
assert request.url.path == "/api/jobs"

# Notebook execution
assert output.get("output_type") != "error"
assert len(outputs) > 0
```

## References

- Global testing standards: `/.claude/skills/testing-strategy.md`
- SDK delivery patterns: `.claude/skills/sdk-delivery.md`
- Manager/Runner implementation: `services/client/src/avatars/{manager,runner}.py`
- Auto-generated code: `services/client/src/avatars/{api,client,models}.py` (DO NOT EDIT)
