# API Structure

## Introduction

The Pinecone Python SDK uses a three-tier API structure that separates public interfaces, type contracts, and implementation details. This document explains the complete API organization and how the tiers interact.

## Three-Tier Structure

### Overview

```mermaid
graph TB
    subgraph Tier1["Tier 1: Public API"]
        direction TB
        T1A["__init__.py"]
        T1B["User-facing exports"]
        T1C["Client construction"]
        T1D["Convenience functions"]
    end

    subgraph Tier2["Tier 2: Protocol Layer"]
        direction TB
        T2A["__interface__/"]
        T2B["Type protocols"]
        T2C["Method signatures"]
        T2D["Return types"]
    end

    subgraph Tier3["Tier 3: Implementation"]
        direction TB
        T3A["_internal/"]
        T3B["Concrete classes"]
        T3C["HTTP clients"]
        T3D["Serialization"]
        T3E["Error handling"]
    end

    User[User Code] --> Tier1
    Tier1 --> Tier2
    Tier2 --> Tier3
    Tier3 --> API[Pinecone API]

    style Tier1 fill:#e1f5ff
    style Tier2 fill:#fff3e0
    style Tier3 fill:#f3e5f5
```

## Tier 1: Public API

### Purpose

The public API tier provides the user-facing interface to the SDK. It defines what users can import and use.

### Location

`pinecone/__init__.py`

### Responsibilities

1. **Export stable APIs** - Only expose tested, documented interfaces
2. **Re-export implementations** - Bring internal classes to public namespace
3. **Construct clients** - Provide factory functions for client creation
4. **Version management** - Expose SDK version information

### Structure

```python
# pinecone/__init__.py

# Version information
__version__ = "1.0.0"

# Import protocols for type hints
from pinecone.__interface__ import (
    IPineconeClient,
    IIndexClient,
    IDataClient,
)

# Import implementations and re-export
from pinecone._internal.client import PineconeClient
from pinecone._internal.models import (
    Vector,
    ScoredVector,
    QueryResponse,
    UpsertResponse,
)

# Public exports (what users can import)
__all__ = [
    # Main client
    "PineconeClient",

    # Type protocols (for type hints)
    # Export these so users can write type-safe code
    # without depending on concrete implementations
    "IPineconeClient",
    "IIndexClient",
    "IDataClient",

    # Data models
    "Vector",
    "ScoredVector",
    "QueryResponse",
    "UpsertResponse",

    # Version
    "__version__",
]
```

### Why Export Protocols?

Protocols should be exported alongside implementations for several key reasons:

1. **Type Hints in User Code** - Users can write type-safe functions:
   ```python
   from pinecone import IIndexClient

   def analyze_index(index: IIndexClient) -> dict:
       """Accept any object implementing IIndexClient."""
       stats = index.describe_index_stats()
       return {"vector_count": stats.get("total_vector_count")}
   ```

2. **Duck Typing Support** - Users can create their own implementations:
   ```python
   from pinecone import IIndexClient

   class MockIndexClient:
       """Test double that satisfies IIndexClient protocol."""
       def query(self, vector, top_k=10, **kwargs):
           return MockQueryResponse()

       def upsert(self, vectors, namespace=""):
           return MockUpsertResponse()

   # Works with any function expecting IIndexClient
   def test_my_function():
       mock = MockIndexClient()
       assert isinstance(mock, IIndexClient)  # Runtime check passes
       result = analyze_index(mock)  # Type checker is happy
   ```

3. **Better IDE Support** - Type checkers and IDEs provide:
   - Autocomplete for protocol methods
   - Type error detection
   - Better refactoring support
   - Documentation on hover

4. **Dependency Injection** - Enables flexible architecture:
   ```python
   from pinecone import IPineconeClient, IIndexClient

   class DataPipeline:
       def __init__(self, client: IPineconeClient):
           """Accept any client implementation."""
           self.client = client

       def process(self, index_name: str):
           index = self.client.index(index_name)
           # Work with protocol, not concrete class
   ```

### Design Rules

1. **No implementation logic** - Only imports and exports
2. **Stable interface** - Changes require careful consideration
3. **Explicit exports** - Use `__all__` to control public API
4. **Type availability** - Export protocols for user type hints
5. **Protocol naming** - Prefix with `I` to distinguish from implementations

### User Experience

```python
# Basic usage - concrete implementation
from pinecone import PineconeClient, Vector

client = PineconeClient(api_key="...")
index = client.index("my-index")

# Advanced usage - type hints with protocols
from pinecone import IIndexClient, IPineconeClient

def process_index(index: IIndexClient) -> int:
    """Function accepts any IIndexClient implementation.

    This works with:
    - Real IndexClient from the SDK
    - Mock implementations for testing
    - Custom implementations from users
    """
    response = index.query(vector=[0.1] * 128, top_k=10)
    return len(response.matches)

# Type-safe client parameter
def create_pipeline(client: IPineconeClient, index_name: str):
    """Accept any client satisfying IPineconeClient."""
    index = client.index(index_name)
    return process_index(index)

# All of these work:
create_pipeline(PineconeClient(api_key="..."), "my-index")  # Real client
create_pipeline(MockPineconeClient(), "my-index")            # Test double
```

### When to Use Protocols vs Concrete Types

```python
# ✅ Good: Use protocols for parameters (flexible)
def analyze_vectors(index: IIndexClient) -> dict:
    pass

# ✅ Good: Use concrete types for construction (clear)
def create_client(api_key: str) -> PineconeClient:
    return PineconeClient(api_key=api_key)

# ❌ Avoid: Concrete types for parameters (rigid)
def analyze_vectors(index: IndexClient) -> dict:
    pass  # Can't pass mock or alternative implementation

# ❌ Avoid: Protocols for construction (vague)
def create_client(api_key: str) -> IPineconeClient:
    return PineconeClient(api_key=api_key)  # Unclear what's returned
```

## Tier 2: Protocol Layer

### Purpose

The protocol layer defines type contracts for all public interfaces using Python's Protocol system. It specifies what methods and properties implementations must provide.

### Location

`pinecone/__interface__/`

### Module Organization

```
__interface__/
├── __init__.py           # Protocol exports
├── client.py            # Client protocols
├── index.py             # Index protocols
├── data.py              # Data operation protocols
└── models.py            # Data model protocols
```

### Protocol Structure

```mermaid
graph TB
    subgraph Protocols["Protocol Definitions"]
        IPC[IPineconeClient]
        IIC[IIndexClient]
        IDC[IDataClient]
        IVP[IVectorProtocol]
    end

    subgraph Methods["Method Contracts"]
        M1["index(name: str) -> IIndexClient"]
        M2["query(vector: List[float]) -> QueryResponse"]
        M3["upsert(vectors: List[Vector]) -> UpsertResponse"]
    end

    subgraph Types["Type Contracts"]
        T1["Return types"]
        T2["Parameter types"]
        T3["Exception types"]
    end

    IPC --> M1
    IIC --> M2
    IDC --> M3

    M1 --> T1
    M2 --> T1
    M3 --> T1

    style Protocols fill:#fff3e0
    style Methods fill:#f3e5f5
    style Types fill:#e8f5e9
```

### Example Protocol

```python
# pinecone/__interface__/client.py

from typing import Protocol, runtime_checkable
from pinecone.__interface__.index import IIndexClient

@runtime_checkable
class IPineconeClient(Protocol):
    """Protocol defining the Pinecone client interface.

    This protocol specifies the contract that all Pinecone client
    implementations must satisfy.
    """

    def index(self, name: str, host: str | None = None) -> IIndexClient:
        """Get an index client for the specified index.

        Args:
            name: Name of the index
            host: Optional host URL override

        Returns:
            Index client implementing IIndexClient protocol

        Raises:
            PineconeException: If index access fails
        """
        ...

    def list_indexes(self) -> list[str]:
        """List all available indexes.

        Returns:
            List of index names

        Raises:
            PineconeException: If API call fails
        """
        ...

    def create_index(
        self,
        name: str,
        dimension: int,
        metric: str = "cosine",
        spec: dict | None = None,
    ) -> None:
        """Create a new index.

        Args:
            name: Index name
            dimension: Vector dimension
            metric: Distance metric (cosine, euclidean, dotproduct)
            spec: Index specification

        Raises:
            PineconeException: If index creation fails
        """
        ...
```

### Design Rules

1. **Runtime checkable** - Use `@runtime_checkable` decorator
2. **Complete signatures** - Include all parameters and types
3. **Document contracts** - Explain expected behavior
4. **No implementation** - Only method signatures with `...`

### Benefits

1. **Type safety** - mypy and IDEs understand the contracts
2. **Testability** - Easy to create mocks implementing protocols
3. **Flexibility** - Multiple implementations can coexist
4. **Documentation** - Protocols are self-documenting

## Tier 3: Implementation Layer

### Purpose

The implementation layer contains all concrete implementations of the protocols, including HTTP communication, serialization, and error handling.

### Location

`pinecone/_internal/`

### Module Organization

```
_internal/
├── __init__.py
├── client.py              # Client implementation
├── index.py               # Index implementation
├── data.py                # Data operations
├── http/
│   ├── __init__.py
│   ├── client.py         # HTTP client
│   ├── retry.py          # Retry logic
│   └── transport.py      # Transport layer
├── models/
│   ├── __init__.py
│   ├── vector.py         # Vector models
│   ├── request.py        # Request models
│   └── response.py       # Response models
├── serialization/
│   ├── __init__.py
│   ├── json.py           # JSON serialization
│   └── protobuf.py       # Protobuf serialization
└── utils/
    ├── __init__.py
    ├── validation.py     # Input validation
    └── config.py         # Configuration
```

### Implementation Architecture

```mermaid
graph TB
    subgraph Public["Public Layer"]
        PC[PineconeClient]
    end

    subgraph Protocol["Protocol Layer"]
        IPC[IPineconeClient]
    end

    subgraph Implementation["Implementation Layer"]
        PCI[PineconeClientImpl]

        subgraph Components["Internal Components"]
            HTTP[HTTPClient]
            SER[Serializer]
            VAL[Validator]
            ERR[ErrorHandler]
            RETRY[RetryLogic]
        end
    end

    PC -.implements.-> IPC
    IPC --> PCI
    PCI --> HTTP
    PCI --> SER
    PCI --> VAL
    PCI --> ERR
    HTTP --> RETRY

    style Public fill:#e1f5ff
    style Protocol fill:#fff3e0
    style Implementation fill:#f3e5f5
```

### Example Implementation

```python
# pinecone/_internal/client.py

from pinecone.__interface__.client import IPineconeClient
from pinecone.__interface__.index import IIndexClient
from pinecone._internal.index import IndexClient
from pinecone._internal.http.client import HTTPClient
from pinecone._internal.utils.validation import validate_api_key

class PineconeClient:
    """Implementation of IPineconeClient protocol."""

    def __init__(
        self,
        api_key: str,
        environment: str | None = None,
        http_client: HTTPClient | None = None,
    ):
        validate_api_key(api_key)
        self._api_key = api_key
        self._environment = environment
        self._http_client = http_client or HTTPClient(api_key=api_key)

    def index(self, name: str, host: str | None = None) -> IIndexClient:
        """Get an index client."""
        return IndexClient(
            name=name,
            host=host or self._get_index_host(name),
            http_client=self._http_client,
        )

    def list_indexes(self) -> list[str]:
        """List all indexes."""
        response = self._http_client.get("/indexes")
        return [idx["name"] for idx in response["indexes"]]

    def create_index(
        self,
        name: str,
        dimension: int,
        metric: str = "cosine",
        spec: dict | None = None,
    ) -> None:
        """Create a new index."""
        payload = {
            "name": name,
            "dimension": dimension,
            "metric": metric,
        }
        if spec:
            payload["spec"] = spec

        self._http_client.post("/indexes", json=payload)

    def _get_index_host(self, name: str) -> str:
        """Internal method to resolve index host."""
        indexes = self.list_indexes()
        # ... implementation details ...
```

### Design Rules

1. **Implement protocols** - Must satisfy protocol contracts
2. **Hide internals** - Not directly importable by users
3. **Encapsulate logic** - All business logic lives here
4. **Use internal utilities** - Share common functionality

### Interactive Introspection

Implementation classes follow the [Interactive Discovery Convention](../conventions/interactive-discovery.md) to support REPL and notebook environments.

**Architectural Pattern**:

Namespace implementation classes inherit from `InteractiveIntrospectionMixin` to provide enhanced error messages:

```python
from pinecone._internal.introspection import InteractiveIntrospectionMixin

class Inference(InteractiveIntrospectionMixin, InferenceNamespaceProtocol):
    """Inference namespace with helpful REPL experience."""

    def embed(self, ...) -> EmbeddingResponse:
        ...
```

**Mixin Benefits**:

- **`__dir__()`** - Filters to public API only, improving tab completion
- **`__getattr__()`** - Provides helpful errors listing available methods when users typo
- **Zero maintenance** - Uses dynamic introspection so stays current automatically
- **Code reuse** - Single mixin shared across all namespace classes

**Example User Experience**:

```python
>>> dir(pc.inference)
['embed', 'list_models', 'rerank']  # Clean public API only

>>> pc.inference.embed()  # Typo
AttributeError: Inference has no attribute 'embedd'.
Available methods: embed, list_models, rerank.
Use dir(inference) or help(inference) to explore the API.
```

**Requirements**:

- All user-facing namespace implementations must use this pattern
- Internal utilities (`_internal/`) do not need it
- See [interactive-discovery.md](../conventions/interactive-discovery.md) for full patterns
- See [usability.md](../conventions/usability.md#interactive-discoverability-pattern) for mixin details

## Inter-Tier Communication

### Dependency Flow

```mermaid
graph LR
    subgraph Dependencies["Allowed Dependencies"]
        T1[Tier 1<br/>Public API]
        T2[Tier 2<br/>Protocols]
        T3[Tier 3<br/>Implementation]

        T1 -->|imports| T2
        T1 -->|re-exports| T3
        T2 -->|no imports| None
        T3 -->|implements| T2
    end

    subgraph Forbidden["Forbidden"]
        F1[Protocol → Implementation]
        F2[Implementation → Public]
        F3[User → Implementation]
    end

    style Dependencies fill:#e8f5e9
    style Forbidden fill:#ffebee
```

### Rules

1. **Public API can import:**
   - Protocols (for type hints)
   - Implementations (to re-export)

2. **Protocols can import:**
   - Other protocols
   - Standard library types
   - No implementations!

3. **Implementations can import:**
   - Protocols (to implement)
   - Other implementations
   - Internal utilities
   - No public API!

4. **Users can import:**
   - Public API only
   - Never directly from `_internal`

## Type Flow

### How Types Flow Through Tiers

```mermaid
sequenceDiagram
    participant User
    participant Public as Tier 1<br/>Public API
    participant Protocol as Tier 2<br/>Protocol
    participant Impl as Tier 3<br/>Implementation

    Note over User,Impl: Type Definition Phase
    Protocol->>Protocol: Define IPineconeClient
    Protocol->>Protocol: Define method signatures

    Note over User,Impl: Implementation Phase
    Impl->>Protocol: Implement IPineconeClient
    Impl->>Impl: Add concrete logic

    Note over User,Impl: Export Phase
    Public->>Protocol: Import protocols
    Public->>Impl: Import implementations
    Public->>Public: Re-export as public API

    Note over User,Impl: Usage Phase
    User->>Public: from pinecone import PineconeClient
    User->>Public: client = PineconeClient(...)
    Public->>Impl: Delegate to implementation
    Impl->>Impl: Execute logic
    Impl->>Public: Return typed result
    Public->>User: Return to user
```

## API Evolution

### Adding New Features

```mermaid
graph LR
    subgraph Process["Feature Addition Process"]
        S1[1. Define Protocol]
        S2[2. Implement in _internal]
        S3[3. Add tests]
        S4[4. Export in __init__.py]
        S5[5. Document]

        S1 --> S2 --> S3 --> S4 --> S5
    end

    style Process fill:#e8f5e9
```

### Deprecation Strategy

```mermaid
graph TB
    subgraph Deprecation["Deprecation Process"]
        D1[Mark as deprecated]
        D2[Add warnings]
        D3[Update docs]
        D4[Provide migration path]
        D5[Remove in major version]

        D1 --> D2 --> D3 --> D4 --> D5
    end

    style Deprecation fill:#fff3e0
```

## Module Import Patterns

### Correct Import Patterns

```python
# ✅ User imports from public API
from pinecone import PineconeClient, Vector

# ✅ User uses protocols for type hints
from pinecone import IPineconeClient

def my_function(client: IPineconeClient):
    pass

# ✅ Internal modules import protocols
from pinecone.__interface__.client import IPineconeClient

# ✅ Internal modules import other internal modules
from pinecone._internal.http.client import HTTPClient
```

### Incorrect Import Patterns

```python
# ❌ User imports from _internal
from pinecone._internal.client import PineconeClient

# ❌ Protocol imports from implementation
from pinecone._internal.client import PineconeClient

# ❌ Implementation imports from public API
from pinecone import PineconeClient
```

## Testing Across Tiers

### Test Organization

```mermaid
graph TB
    subgraph Tests["Test Structure"]
        UT[Unit Tests]
        IT[Integration Tests]
        ET[End-to-End Tests]
    end

    subgraph Tiers["Tier Testing"]
        T1T[Public API Tests]
        T2T[Protocol Tests]
        T3T[Implementation Tests]
    end

    UT --> T3T
    IT --> T2T
    ET --> T1T

    style Tests fill:#e3f2fd
    style Tiers fill:#f3e5f5
```

## Performance Implications

The three-tier structure has minimal performance overhead:

1. **No runtime cost** - Protocols use structural subtyping
2. **Import time** - Slightly higher due to re-exports
3. **Type checking** - Only at development time with mypy
4. **Memory** - Minimal extra memory for protocol definitions

## Related Documentation

- [Overview](./overview.md) - System architecture overview
- [Specifications](./specifications.md) - Detailed protocol specifications
- [Verification](./verification.md) - Testing strategy
- [Architecture Conventions](../conventions/architecture.md) - Architecture coding rules
- [Naming Conventions](../conventions/naming.md) - Naming patterns
