# Interactive Discovery

## Overview

Interactive discovery refers to how easily developers can explore and learn the SDK through direct interaction—in the Python REPL, IPython console, or Jupyter notebooks. A well-designed SDK is self-documenting through introspection, allowing users to discover functionality without constantly referring to documentation.

This matters for:
- **Human developers**: Faster learning, less context switching, better development flow
- **AI coding agents**: Clear patterns enable accurate code generation and API usage
- **Jupyter users**: Rich representations make exploration delightful

**Key principle**: The SDK should guide users toward correct usage through clear, discoverable interfaces.

## msgspec.Struct Default Behavior

Before implementing custom interactive features, understand what msgspec.Struct provides automatically:

- ✅ `__repr__()`: Shows all fields with values (e.g., `Usage(total_tokens=5)`)
- ✅ `__rich_repr__()`: Support for the `rich` library
- ✅ `__dir__()`: Standard Python behavior (all attributes)
- ❌ `_repr_pretty_()` and `_repr_html_()`: Not provided by default

**When msgspec defaults are sufficient:**
- Simple data classes (e.g., `Usage` with 1-2 fields)
- Small nested objects without large arrays
- Internal request models

**When to override defaults:** See class-specific guidance below.

## Class-Specific Patterns

### Client/Namespace Classes

**Examples:** `Pinecone`, `AsyncPinecone`, `Inference`, `AsyncInference`

**Required:**

`__repr__()` - Show configuration without secrets:

```python
def __repr__(self) -> str:
    """Return string representation of client."""
    return f"Inference(base_url='{self._base_url}')"
```

`__dir__()` - Filter to public API only:

```python
def __dir__(self) -> list[str]:
    """Return public API, filtering out internal attributes."""
    attrs = set(super().__dir__())
    public = {name for name in attrs if not x.startswith('_')}
    return sorted(public)
```

**Skip:**
- `_repr_pretty_()` and `_repr_html_()` - Not useful for service classes

**Example output:**

```python
>>> pc = Pinecone(api_key="secret-key")
>>> pc
Pinecone(base_url='https://api.pinecone.io')  # No API key shown!

>>> dir(pc)
['close', 'inference']  # Only public API

>>> pc.inference
Inference(base_url='https://api.pinecone.io')
```

### Top-Level Response Classes

**Examples:** `EmbedResponse`, `RerankResponse`, `ListModelsResponse`

**Required:**

`__repr__()` - Show summary, truncate arrays:

```python
def __repr__(self) -> str:
    """Return summary representation."""
    return (
        f"EmbedResponse("
        f"model={self.model!r}, "
        f"embeddings={len(self.data)}, "
        f"usage={self.usage!r})"
    )
```

`__dir__()` - Hide internal cache attributes:

```python
def __dir__(self) -> list[str]:
    """Return public API, hiding cache attributes."""
    attrs = set(super().__dir__())
    public = {name for name in attrs if not name.startswith('_')}
    return sorted(public)
```

**Recommended (for Jupyter/IPython UX):**

`_repr_pretty_()` - IPython console formatting:

```python
def _repr_pretty_(self, p, cycle):
    """IPython pretty-printer support.
    
    Args:
        p: Pretty printer instance
        cycle: True if circular reference detected
    """
    if cycle:
        p.text('EmbedResponse(...)')
        return
    
    p.text('EmbedResponse(')
    with p.group(2, '', ')'):
        p.breakable()
        p.text(f'model={self.model!r},')
        p.breakable()
        p.text(f'embeddings={len(self.data)},')
        p.breakable()
        p.text(f'usage={self.usage!r}')
```

`_repr_html_()` - Jupyter notebook display:

```python
def _repr_html_(self) -> str:
    """Jupyter notebook HTML representation."""
    import html
    
    model_safe = html.escape(self.model)
    
    return f"""
    <div style="font-family: monospace; padding: 10px; border: 1px solid #ddd; 
                border-radius: 4px; background-color: #f9f9f9;">
        <div style="font-weight: bold; margin-bottom: 8px;">EmbedResponse</div>
        <table style="border-collapse: collapse; width: 100%;">
            <tr>
                <td style="padding: 4px; font-weight: bold;">Model:</td>
                <td style="padding: 4px;">{model_safe}</td>
            </tr>
            <tr>
                <td style="padding: 4px; font-weight: bold;">Embeddings:</td>
                <td style="padding: 4px;">{len(self.data)}</td>
            </tr>
            <tr>
                <td style="padding: 4px; font-weight: bold;">Vector Type:</td>
                <td style="padding: 4px;">{self.vector_type}</td>
            </tr>
            <tr>
                <td style="padding: 4px; font-weight: bold;">Total Tokens:</td>
                <td style="padding: 4px;">{self.usage.total_tokens}</td>
            </tr>
        </table>
    </div>
    """
```

**Rich representation requirements:**
- `_repr_pretty_`: Must handle `cycle` parameter for circular references
- `_repr_html_`: Must escape user data with `html.escape()` to prevent XSS
- Both should handle all edge cases gracefully (no exceptions)
- Keep HTML styling minimal and readable

**Example output:**

```python
>>> response = pc.inference.embed(model="e5", texts=["hello"] * 1000)
>>> response
EmbedResponse(model='multilingual-e5-large', embeddings=1000, usage=Usage(total_tokens=5000))

>>> dir(response)
['data', 'embeddings', 'model', 'usage', 'vector_type']  # No _cached_embeddings
```

### Nested Models with Large Arrays

**Examples:** `Embedding` (with 1536-dimensional vectors)

**Required:**

`__repr__()` - Truncate large arrays:

```python
def __repr__(self) -> str:
    """Return repr with truncated values array."""
    if len(self.values) <= 3:
        values_repr = repr(self.values)
    else:
        # Show first 2 values + ellipsis
        values_repr = f"[{self.values[0]}, {self.values[1]}, ...]"
    
    return f"Embedding(values={values_repr}, index={self.index})"
```

**Skip:**
- `__dir__()` - Standard msgspec behavior is fine
- `_repr_pretty_()` / `_repr_html_()` - Less useful for nested objects

**Example output:**

```python
>>> emb = Embedding(values=[0.1, 0.2, 0.3, ..., 0.999], index=0)  # 1536 values
>>> emb
Embedding(values=[0.1, 0.2, ...], index=0)  # Truncated!
```

### Simple msgspec.Struct Models

**Examples:** `Usage`, `RerankResult`, `ModelInfoSupportedParameter`

**Use msgspec defaults** - No custom implementations needed:

```python
>>> usage = Usage(total_tokens=42)
>>> usage
Usage(total_tokens=42)  # msgspec default is perfect!

>>> dir(usage)
['total_tokens']  # Standard behavior is fine
```

## IDE Integration

### Type Hints

Complete type hints enable accurate IDE autocomplete:

```python
def embed(
    self,
    model: str,
    *,
    texts: list[str],
    parameters: dict[str, Any] | None = None,
) -> EmbeddingResponseProtocol:
    """Generate embeddings for texts."""
    ...
```

- Use modern syntax: `|` instead of `Union`, `list[str]` instead of `List[str]`
- Return Protocol types for flexibility
- Enforced by `mypy --strict`

### Docstring Quality

Google-style docstrings with Examples section:

```python
def embed(...) -> EmbeddingResponseProtocol:
    """Generate embeddings for texts.
    
    Args:
        model: Model name to use (e.g., "multilingual-e5-large")
        texts: List of text strings to embed (non-empty)
        parameters: Optional model-specific parameters
    
    Returns:
        EmbeddingResponse with embeddings and usage info
    
    Raises:
        BadRequestError: If inputs are invalid
        APIError: If the API returns an error
    
    Example:
        >>> pc = Pinecone(api_key="...")
        >>> response = pc.inference.embed(
        ...     model="multilingual-e5-large",
        ...     texts=["Hello world"]
        ... )
        >>> len(response.embeddings)
        1
    """
    ...
```

IDE hover tooltips show this content for inline help.

### Namespace Organization

Clear namespace boundaries aid discovery:

```python
# ✅ Good: Feature-based namespaces
pc.inference.embed()
pc.inference.rerank()

# ❌ Bad: Flat namespace
pc.embed()
pc.rerank()
```

## API Exploration & Introspection

### Custom `__dir__()` Implementation

Override `__dir__()` to surface useful methods:

```python
def __dir__(self) -> list[str]:
    """Return public API, filtering out internal attributes."""
    attrs = set(super().__dir__())
    public = {name for name in attrs if not name.startswith('_')}
    return sorted(public)
```

This dynamic approach doesn't go stale as methods are added.

### List Available Resources

Methods that list available options aid discovery:

```python
# Helps users discover what's available
>>> pc.inference.list_models()
['multilingual-e5-large', 'bge-reranker-v2-m3', ...]
```

- Return simple types (`list[str]`) for easy inspection
- Consider properties for quick reference: `.available_models`

### Help Text

Ensure `help()` shows useful information:

```python
>>> help(pc.inference.embed)
# Shows signature, docstring, examples
```

Leverage comprehensive docstrings—no need for custom `__doc__`.

### Introspection-Friendly

- Typed attributes enable `inspect.signature()` to work
- Properties with clear return types
- Avoid magic that breaks introspection

## Tab Completion Support

### Predictable Naming

Follow naming conventions for guess-ability:

- Consistent prefixes: `list_*`, `create_*`, `delete_*`
- Users can guess: "It's probably `pc.inference.list_models()`"

See [naming.md](./naming.md) for full conventions.

### Avoid Deep Nesting

Keep API surface flat:

```python
# ✅ Good: 2-3 levels
pc.inference.embed()

# ❌ Bad: Too deep
pc.services.inference.operations.embed()
```

### Property vs Method

- **Properties** for attributes: `response.embeddings` (no parens)
- **Methods** for actions: `pc.inference.embed()` (with parens)

## Interactive Debugging Helpers

### Meaningful Error Messages

Follow [errors.md](./errors.md) conventions:

- Include suggested fixes
- Provide context
- Actionable guidance

### Request/Response Inspection

- Log requests at DEBUG level (already implemented)
- Consider properties like `.request_id` on responses for tracing

### Validation Before API Calls

Fail fast with clear validation:

```python
def embed(self, model: str, *, texts: list[str], ...) -> ...:
    if not texts:
        raise BadRequestError("texts must be a non-empty list", None)
    if not model:
        raise BadRequestError("model must be specified", None)
    
    # Now make API call...
```

## Testing Interactive Features

### Validation Helpers

The SDK provides validation helpers in `tests/unit/helpers/interactive_validation.py` that enforce interactive discovery patterns:

```python
from tests.unit.helpers import validate_client_class, validate_response_class

def test_my_client_interactive_features():
    """Validate all interactive features at once."""
    pc = MyClient(api_key="test-key")
    validate_client_class(MyClient, pc, "test-key")
    # Checks: custom __repr__, custom __dir__, no secrets, no private attrs

def test_my_response_interactive_features():
    """Validate response class interactive features."""
    response = MyResponse(...)
    validate_response_class(MyResponse, response, require_rich_repr=True)
    # Checks: custom __repr__, custom __dir__, truncated arrays,
    #         _repr_pretty_, _repr_html_
```

These helpers catch common issues:
- Missing custom `__repr__()` or `__dir__()`
- API keys leaked in repr output
- Private attributes exposed in dir()
- Arrays not truncated in repr
- Missing or broken rich representations
- XSS vulnerabilities in HTML output

### Individual Validation Functions

For more granular testing:

```python
from tests.unit.helpers import (
    assert_has_custom_repr,
    assert_has_custom_dir,
    assert_repr_hides_secrets,
    assert_dir_hides_private_attrs,
    assert_repr_truncates_arrays,
    assert_has_repr_pretty,
    assert_has_repr_html,
    assert_repr_html_no_xss,
)
```

### Required Tests

```python
def test_client_repr_no_api_key():
    """Verify __repr__ doesn't leak secrets."""
    pc = Pinecone(api_key="secret-key-12345")
    repr_str = repr(pc)
    assert "secret-key" not in repr_str
    assert "Pinecone" in repr_str

def test_embed_response_repr_truncates_long_arrays():
    """Verify large responses show summary, not full data."""
    embeddings = [Embedding(values=[0.1] * 1536, index=i) for i in range(1000)]
    response = EmbedResponse(
        data=embeddings,
        model="test-model",
        vector_type="dense",
        usage=Usage(total_tokens=5000)
    )
    
    repr_str = repr(response)
    
    # Should show summary
    assert "1000" in repr_str or "embeddings=1000" in repr_str
    assert "test-model" in repr_str
    
    # Should NOT contain full array contents
    assert repr_str.count("0.1") < 100  # Not all 1,536,000 values
    assert len(repr_str) < 500  # Reasonable length

def test_inference_dir_public_only():
    """Verify __dir__ shows only public API."""
    pc = Pinecone(api_key="test")
    dir_result = dir(pc.inference)
    
    # Verify public methods included
    assert 'embed' in dir_result
    assert 'rerank' in dir_result
    
    # Verify internals excluded
    assert '_http' not in dir_result
    assert '_adapter' not in dir_result
    
    # Verify no underscore-prefixed items
    private_items = [x for x in dir_result if x.startswith('_')]
    assert len(private_items) == 0, f"Found private items: {private_items}"
```

### Optional Tests (for rich representations)

```python
def test_repr_pretty_no_crash():
    """Ensure _repr_pretty_ doesn't crash."""
    from io import StringIO
    
    class MockPrinter:
        def __init__(self):
            self.buffer = StringIO()
        def text(self, s):
            self.buffer.write(s)
        def breakable(self):
            pass
        def group(self, *args, **kwargs):
            return self
        def __enter__(self):
            return self
        def __exit__(self, *args):
            pass
    
    response = EmbedResponse(...)
    if hasattr(response, '_repr_pretty_'):
        p = MockPrinter()
        response._repr_pretty_(p, cycle=False)  # Should not raise
        response._repr_pretty_(p, cycle=True)  # Should handle cycle

def test_repr_html_escapes_user_data():
    """Ensure _repr_html_ prevents XSS."""
    response = EmbedResponse(model="<script>alert('xss')</script>", ...)
    
    if hasattr(response, '_repr_html_'):
        html = response._repr_html_()
        
        assert isinstance(html, str)
        assert '<script>' not in html.lower()
        assert '&lt;script&gt;' in html  # Properly escaped
```

## Examples

### Good Patterns

```python
# Auto-discovery through dir()
>>> pc = Pinecone(api_key="...")
>>> [x for x in dir(pc) if not x.startswith('_')]
['close', 'inference']

# Helpful repr
>>> pc
Pinecone(base_url='https://api.pinecone.io')
>>> pc.inference
Inference(base_url='https://api.pinecone.io')

# Tab completion works
>>> pc.inference.<TAB>
embed  rerank

# Help is useful
>>> help(pc.inference.embed)
# Shows: signature, docstring, examples

# Response is inspectable with summary
>>> response = pc.inference.embed(model="e5", texts=["hello"])
>>> response
EmbedResponse(model='multilingual-e5-large', embeddings=1, usage=Usage(total_tokens=5))

# Long arrays truncated
>>> response = pc.inference.embed(model="e5", texts=["text"] * 1000)
>>> response
EmbedResponse(model='multilingual-e5-large', embeddings=1000, usage=Usage(total_tokens=5000))
```

### Anti-Patterns

```python
# ❌ Uninformative repr
>>> pc
<pinecone.client.Pinecone object at 0x7f8b9c>

# ❌ Hidden functionality
>>> pc.inference._secret_method()  # Users won't discover this

# ❌ Overwhelming output
>>> response
EmbedResponse(data=[Embedding(values=[0.1, 0.2, ...], index=0), Embedding(...), ...998 more...], ...)

# ❌ No tab completion (only internals shown)
>>> pc.inference.<TAB>
_http  _adapter  _base_url
```

## Enforcement

The SDK uses **runtime validation helpers** (`tests/unit/helpers/interactive_validation.py`) to enforce these patterns. When adding new classes:

1. Implement the required interactive features
2. Add validation tests using the helpers
3. Run tests to verify compliance

This ensures consistency as the SDK grows and catches issues early.

## Implementation Checklist

Use this checklist when creating new classes:

### Client/Namespace Classes
- [ ] `__repr__()` shows config without secrets
- [ ] `__dir__()` returns public API only
- [ ] Type hints on all methods
- [ ] Docstrings with Examples
- [ ] Skip `_repr_pretty_()` and `_repr_html_()`

### Top-Level Response Classes
- [ ] `__repr__()` shows summary, truncates arrays
- [ ] `__dir__()` hides cache attributes
- [ ] Type hints on all properties
- [ ] Properties provide clean data access
- [ ] (Optional) `_repr_pretty_()` for IPython
- [ ] (Optional) `_repr_html_()` for Jupyter

### Nested Models with Large Arrays
- [ ] `__repr__()` truncates large arrays
- [ ] Skip `__dir__()` (msgspec default OK)
- [ ] Skip rich representations

### Simple msgspec.Struct Models
- [ ] Use msgspec defaults (no custom code needed)

## References

- [usability.md](./usability.md) - 12 usability principles
- [naming.md](./naming.md) - Predictable naming conventions
- [errors.md](./errors.md) - Actionable error messages
- [documentation.md](./documentation.md) - Docstring quality standards
- [types.md](./types.md) - Type hints and msgspec patterns
