---
name: sdk-delivery
description: >
  Python SDK patterns for the Avatar client library with auto-generated low-level client
  and hand-crafted Manager/Runner high-level interfaces. Use when working with Avatar SDK
  (services/client/) for: (1) using or extending Manager/Runner workflows,
  (2) regenerating client code from OpenAPI schema, (3) implementing client-side data processors,
  (4) handling avatarization results, or (5) understanding auto-generation boundaries.
  NEVER edit auto-generated files (api.py, client.py, models.py).
---

# SDK Delivery - Avatar Python Client

## Overview

Avatar Python SDK provides high-level abstractions over the API with **auto-generated** low-level client and **hand-crafted** Manager/Runner patterns.

## Auto-Generation Boundary

### ❌ NEVER Edit These Files

```
src/avatars/api.py      # Generated from OpenAPI
src/avatars/client.py   # Generated from OpenAPI
src/avatars/models.py   # Generated from OpenAPI
```

### ✅ Edit These Files

```
src/avatars/manager.py    # High-level facade
src/avatars/runner.py     # Workflow orchestrator
src/avatars/processors/   # Client-side transforms
```

## Regeneration Workflow

```bash
# 1. Update API schema
cd services/api && just export-open-api

# 2. Regenerate client
cd ../client && just generate-python
```

## Pattern: Manager/Runner

### Basic Usage

```python
from avatars import Manager

manager = Manager(base_url="https://api.octopize.io")
manager.authenticate("user", "password")

runner = manager.create_runner(set_name="my_project")
runner.add_table("patients", data="patients.csv", primary_key="patient_id")
runner.set_parameters("patients", k=20, ncp=30)
runner.run()
results_df = runner.shuffled("patients")
```

## Runner State Machine

```
Configuration → Execution → Retrieval
     ↓              ↓           ↓
add_table()     run()      shuffled()
set_parameters()           privacy_metrics()
```

**Key insight**: `set_name` (UUID) only assigned server-side after first `put_resources()` call. Until then, config is local-only.

**State phases**:

1. **Configuration**: `add_table()` → `set_parameters()` → `get_yaml()` builds `avatar_yaml.Config`
2. **Execution**: `run(jobs_to_run=[...])` submits jobs with dependency chains
3. **Retrieval**: Results lazy-loaded via `ResultsOrganizer` on first access

## Job Dependency Order

```python
[JobKind.standard, JobKind.privacy_metrics, JobKind.signal_metrics, JobKind.report]
```

Privacy/signal metrics depend on `standard` completing first.

## Quick Reference

| Task              | Command                 |
| ----------------- | ----------------------- |
| Run tests         | `just test`             |
| Test tutorials    | `just test-tutorials`   |
| Format notebooks  | `just format-notebooks` |
| Sync notebook .py | `just generate-py`      |
| Build docs        | `just doc-build`        |
| Regenerate client | `just generate-python`  |

## Dual Parameter Systems

### ✅ Standard Avatarization

```python
runner.set_parameters("table", k=20, ncp=30)
```

### ✅ Differential Privacy

```python
runner.set_parameters("table", open_dp_epsilon=1.0)
```

### ❌ Never Both

```python
# Raises ValueError
runner.set_parameters("table", k=20, open_dp_epsilon=1.0)
```

## Testing Patterns

See `.claude/skills/testing-strategy.md` (root) for comprehensive testing patterns.

### Unit Tests with Fakes

```python
# tests/unit/conftest.py
class FakeApiClient:
    def get_results(self, url):
        return in_memory_data[url]
```

### Integration Tests (Notebooks)

Notebooks are executed via `nbconvert` - see `tests/integration/test_tutorials.py`.

## Notebook Development

```bash
just notebook  # Launch Jupyter

# After editing:
just format-notebooks  # Clear output, strip metadata
just generate-py       # Sync to .py format
```

## Processor Pattern

Client-side transforms run BEFORE upload:

```python
from avatars.processors import GeolocationNormalizationProcessor

processor = GeolocationNormalizationProcessor()
df_processed = processor.process(df)
```

## Best Practices

1. **Never edit auto-generated code** - Update schema instead
2. **Test notebooks** - Use `just test-tutorials`
3. **Sync notebooks** - Always run `just generate-py` after editing
4. **Document committed changes** - `sphinx-multiversion` requires commits
5. **Use processors** for client-side transforms
6. **SSL verification** - Can disable with `should_verify_ssl=False` for on-premise
7. **File uploads** - Handled via pre-signed S3 URLs from `/results/upload-url`

## Results Handling

**Caching**: Results cached in `ResultsOrganizer` after first download.

```python
runner.shuffled(table_name)                        # → pandas DataFrame
runner.privacy_metrics(table_name)                 # → list[dict]
runner.render_plot(table_name, PlotKind.PROJECTION_2D)  # → HTML
```

## Multi-Table Workflows

```python
runner.add_table("visits", "visits.csv", foreign_keys=["patient_id"])
runner.add_link(
    parent_table_name="patients", parent_field="patient_id",
    child_table_name="visits", child_field="patient_id"
)
```

## Test Fixtures

`tests/unit/conftest.py` provides `FakeApiClient` for mocking S3 downloads.

**When adding endpoints**: Update `FakeResults.get_results()` to return expected URLs.

## Package Dependencies

- **Main**: `httpx`, `pydantic`, `pandas`, `avatar-yaml`
- **Dev**: `ruff`, `mypy`, `pytest`
- **Notebook**: `jupyter`, `matplotlib`, `seaborn`
- **Pinned**: `aiobotocore <=2.22.0`, `botocore <=1.40.17` (compatibility)

## When Modifying Code

- **Adding API endpoints**: Regenerate from schema (don't hand-write)
- **Adding result types**: Update `constants.py::Results` enum + `RESULTS_TO_STORE`
- **Adding job types**: Add to `models.py::JobKind` + `constants.py::JOB_EXECUTION_ORDER`
- **Changing Runner state**: Keep `_extract_current_parameters()` and `update_parameters()` in sync
- **New processors**: Follow naming `*Processor`, add to `processors/__init__.py`

## AI Developer Notes

1. **Don't edit ApiClient/api.py** - Code-generated, change OpenAPI schema instead
2. **Job polling**: `Runner._retrieve_job_result_urls()` polls with `DEFAULT_RETRY_INTERVAL` (5s)
3. **YAML config**: Uses `avatar-yaml` library (external), Runner builds, server interprets
4. **Notebook testing**: `tests/integration/test_tutorials.py` executes via `nbconvert`
5. **Docs**: `sphinx-multiversion` only builds committed changes (git checkout mechanism)

## Docs Review Checklist (on every code change)

When modifying `manager.py` or `runner.py`:

1. **Check `doc/source/`** — scan `.md` and `.rst` files for references to changed methods,
   signatures, or behaviors. Key files: `user_guide.md`, `manager.rst`, `runner.rst`,
   `api_reference.rst`
2. **Update affected docs** — fix outdated method names, return types, or examples in the
   source `.md` files (`.rst` files are generated — do not edit them directly)
3. **Add new sections if needed** — new public methods or workflows should have a how-to
   entry in `doc/source/user_guide.md`
4. **Regenerate `.rst` files** — run `just format-doc` after editing any `.md` source file
5. **Check `docs/`** — also scan the `docs/` folder for any developer-facing markdown that
   references changed APIs

> `doc/source/user_guide.md` is the primary user-facing reference. Keep it in sync with
> code changes. The `.rst` files in `doc/source/` are auto-generated — always edit the
> `.md` source.

## Common Pitfalls

| Issue                      | Solution                           |
| -------------------------- | ---------------------------------- |
| Edited api.py              | Regenerate: `just generate-python` |
| Notebook not synced        | Run `just generate-py`             |
| Docs not updated           | Commit changes first               |
| Both k and open_dp_epsilon | Choose one parameter system        |

## File Organization

```
src/avatars/
├── api.py          # AUTO-GENERATED
├── client.py       # AUTO-GENERATED
├── models.py       # AUTO-GENERATED
├── manager.py      # Hand-crafted
├── runner.py       # Hand-crafted
└── processors/     # Hand-crafted
```

## Environment Variables

```bash
AVATAR_BASE_API_URL=http://localhost:8080/api
AVATAR_USERNAME=user_integration
AVATAR_PASSWORD=password_integration
```

## Integration Points

- **API Schema**: `../../services/api/schema.json` (source of truth)
- **Generator**: `../../client/generator-config/python/generate.py`
- **Shared justfiles**: `../../justfiles/python.just`
- **S3 Storage**: Pre-signed URLs from `/results/upload-url`

## References

- Manager implementation: `src/avatars/manager.py`
- Runner implementation: `src/avatars/runner.py`
- Tutorial notebooks: `tutorials/`
- API documentation: `services/api/`
