# Development Workflow Guide

## Introduction

This guide provides step-by-step workflows for common development tasks in the Pinecone Python SDK. Each workflow includes checklists, command examples, and links to relevant documentation.

**Before starting any development task:**
- Read the [Architecture Overview](./architecture/overview.md) to understand the system design
- Review the [Conventions](./conventions/README.md) for coding standards
- Ensure your development environment is set up correctly

## Quick Reference

| Task | Estimated Time | Complexity |
|------|---------------|------------|
| [Adding a new method](#workflow-1-adding-a-new-method) | 2-4 hours | Medium |
| [Updating API version](#workflow-2-updating-api-version) | 1-3 days | High |
| [Adding a new namespace](#workflow-3-adding-a-new-namespace) | 4-8 hours | High |
| [Performance optimization](#workflow-4-performance-optimization) | Variable | Medium-High |

---

## Workflow 1: Adding a New Method

**Goal:** Add a new method to an existing client (e.g., add `configure_index` to `PineconeClient`)

**Time:** 2-4 hours
**Complexity:** Medium

### Prerequisites

- [ ] Understand the [three-layer architecture](./architecture/overview.md#three-layer-architecture)
- [ ] Review [protocol specifications](./architecture/specifications.md)
- [ ] Read [testing conventions](./conventions/testing.md)

### Step 1: Define the Protocol

**Location:** `pinecone/__interface__/[module].py`

Add the method signature to the appropriate protocol interface.

```python
# Example: pinecone/__interface__/client.py

@runtime_checkable
class IPineconeClient(Protocol):
    # ... existing methods ...

    def configure_index(
        self,
        name: str,
        replicas: int | None = None,
        pod_type: str | None = None,
    ) -> None:
        """Configure index settings.

        Args:
            name: Index name
            replicas: Number of replicas (optional)
            pod_type: Pod type specification (optional)

        Raises:
            ValidationError: If parameters are invalid
            PineconeApiException: If configuration fails

        Example:
            >>> client.configure_index("my-index", replicas=3)
        """
        ...
```

**Checklist:**
- [ ] Use `@runtime_checkable` decorator
- [ ] Add complete type annotations
- [ ] Write comprehensive docstring (Args, Returns, Raises, Example)
- [ ] Use `...` for method body (no implementation)
- [ ] Follow [naming conventions](./conventions/naming.md)

### Step 2: Implement the Method

**Location:** `pinecone/_internal/[module].py`

Implement the method in the concrete class.

```python
# Example: pinecone/_internal/client.py

class PineconeClient:
    # ... existing methods ...

    def configure_index(
        self,
        name: str,
        replicas: int | None = None,
        pod_type: str | None = None,
    ) -> None:
        """Configure index settings."""
        # 1. Validate inputs
        if not name:
            raise ValidationError("Index name is required")

        if replicas is not None and replicas < 1:
            raise ValidationError("Replicas must be >= 1")

        # 2. Build request payload
        payload = {}
        if replicas is not None:
            payload["replicas"] = replicas
        if pod_type is not None:
            payload["pod_type"] = pod_type

        # 3. Make API call
        try:
            self._http_client.patch(
                f"/indexes/{name}",
                json=payload,
            )
        except HTTPError as e:
            # 4. Handle errors
            if e.status_code == 404:
                raise PineconeApiException(
                    f"Index '{name}' not found"
                ) from e
            elif e.status_code == 400:
                raise PineconeApiException(
                    f"Invalid configuration: {e.message}"
                ) from e
            else:
                raise PineconeApiException(
                    f"Failed to configure index: {e.message}"
                ) from e
```

**Checklist:**
- [ ] Implement the protocol signature exactly
- [ ] Validate all inputs
- [ ] Use internal HTTP client
- [ ] Handle all expected errors
- [ ] Transform HTTP errors to SDK exceptions
- [ ] Follow [error handling conventions](./conventions/errors.md)
- [ ] Add logging if appropriate

**Commands:**
```bash
# Verify no syntax errors
python -c "import pinecone._internal.client"

# Run type checker
mypy pinecone/_internal/client.py
```

### Step 3: Write Tests

Create three types of tests: protocol, unit, and integration.

#### 3.1 Protocol Compliance Test

**Location:** `tests/protocols/test_[module]_protocol.py`

```python
# Example: tests/protocols/test_client_protocol.py

def test_configure_index_in_protocol():
    """Verify configure_index satisfies protocol."""
    from pinecone import IPineconeClient
    from pinecone._internal.client import PineconeClient

    client = PineconeClient(api_key="test-key")
    assert isinstance(client, IPineconeClient)
    assert hasattr(client, "configure_index")
    assert callable(getattr(client, "configure_index"))
```

#### 3.2 Unit Tests

**Location:** `tests/unit/test_[module].py`

```python
# Example: tests/unit/test_client.py

def test_configure_index_success(mocker):
    """Test configure_index with valid parameters."""
    mock_http = mocker.Mock()
    client = PineconeClient(api_key="test-key", http_client=mock_http)

    client.configure_index(
        name="my-index",
        replicas=3,
        pod_type="p1.x1",
    )

    mock_http.patch.assert_called_once_with(
        "/indexes/my-index",
        json={"replicas": 3, "pod_type": "p1.x1"},
    )

def test_configure_index_validation_error():
    """Test configure_index validates inputs."""
    client = PineconeClient(api_key="test-key")

    with pytest.raises(ValidationError, match="Index name is required"):
        client.configure_index(name="")

    with pytest.raises(ValidationError, match="Replicas must be >= 1"):
        client.configure_index(name="test", replicas=0)

def test_configure_index_not_found_error(mocker):
    """Test configure_index handles 404 errors."""
    mock_http = mocker.Mock()
    mock_http.patch.side_effect = HTTPError(404, "Not found")
    client = PineconeClient(api_key="test-key", http_client=mock_http)

    with pytest.raises(PineconeApiException, match="not found"):
        client.configure_index(name="nonexistent")
```

#### 3.3 Integration Test

**Location:** `tests/integration/test_[module]_integration.py`

```python
# Example: tests/integration/test_client_integration.py

@responses.activate
def test_configure_index_integration():
    """Test configure_index with mocked HTTP."""
    responses.add(
        responses.PATCH,
        "https://api.pinecone.io/indexes/my-index",
        json={"status": "success"},
        status=200,
    )

    client = PineconeClient(api_key="test-key")
    client.configure_index(name="my-index", replicas=2)

    assert len(responses.calls) == 1
    assert responses.calls[0].request.url.endswith("/indexes/my-index")
```

**Checklist:**
- [ ] Protocol compliance test
- [ ] Unit tests for success cases
- [ ] Unit tests for validation errors
- [ ] Unit tests for API errors
- [ ] Integration test with mocked HTTP
- [ ] Test coverage >= 90%

**Commands:**
```bash
# Run all tests for the module
pytest tests/unit/test_client.py -v
pytest tests/protocols/test_client_protocol.py -v
pytest tests/integration/test_client_integration.py -v

# Check coverage
pytest tests/unit/test_client.py --cov=pinecone._internal.client --cov-report=term-missing

# Run type checking
mypy pinecone tests
```

### Step 4: Update Documentation

**If adding to public API:** Update or add docstring examples.

**If significant feature:** Consider adding to examples or guides.

**Checklist:**
- [ ] Docstring is complete and accurate
- [ ] Examples in docstring work correctly
- [ ] Related documentation updated if needed

### Step 5: Run Verification

Run the full verification suite before committing.

**Commands:**
```bash
# Run all checks
make check  # Or equivalent

# Individual checks
ruff check .
black --check .
mypy pinecone
pytest

# Check coverage threshold
pytest --cov=pinecone --cov-report=term --cov-fail-under=90
```

**Checklist:**
- [ ] All linting passes
- [ ] All type checking passes
- [ ] All tests pass
- [ ] Coverage >= 90%
- [ ] No new warnings

### Step 6: Commit and Push

**Commands:**
```bash
git add pinecone/__interface__/client.py
git add pinecone/_internal/client.py
git add tests/

git commit -m "Add configure_index method to PineconeClient

- Add protocol definition to IPineconeClient
- Implement in PineconeClient with validation and error handling
- Add comprehensive test coverage (protocol, unit, integration)
- Update docstrings with examples"

git push -u origin your-branch-name
```

### Common Pitfalls

❌ **Forgetting `@runtime_checkable`** on protocol
- This breaks `isinstance()` checks at runtime

❌ **Incomplete type annotations**
- mypy will complain and type hints won't work properly

❌ **Missing error handling**
- Always handle expected HTTP errors and transform to SDK exceptions

❌ **Insufficient test coverage**
- Must test success, validation errors, and API errors

❌ **Not updating protocol when changing implementation**
- Protocol and implementation must stay in sync

---

## Workflow 2: Updating API Version

**Goal:** Update the SDK to support a new version of the Pinecone API

**Time:** 1-3 days
**Complexity:** High

### Prerequisites

- [ ] Access to new OpenAPI specification
- [ ] Understanding of [API stability conventions](./conventions/api-stability.md)
- [ ] Understanding of [versioning conventions](./conventions/versioning.md)
- [ ] Review existing [architecture](./architecture/overview.md)

### Step 1: Fetch and Review OpenAPI Spec

Obtain the new OpenAPI specification from the API team.

**Commands:**
```bash
# Download new spec (example)
curl -o openapi-spec-new.yaml https://api.pinecone.io/openapi.yaml

# Or use internal tooling
fetch-openapi-spec --version 2024-10 --output openapi-spec-new.yaml
```

**Checklist:**
- [ ] Download new OpenAPI spec
- [ ] Save in `specs/` directory with version number
- [ ] Keep old spec for comparison

### Step 2: Identify Breaking Changes

Compare the new spec with the current version to identify changes.

**Commands:**
```bash
# Use OpenAPI diff tool
openapi-diff specs/openapi-spec-old.yaml specs/openapi-spec-new.yaml

# Manual review
diff -u specs/openapi-spec-old.yaml specs/openapi-spec-new.yaml | less
```

**Look for:**
- [ ] Removed endpoints
- [ ] Changed request/response schemas
- [ ] New required parameters
- [ ] Changed parameter types
- [ ] Removed optional parameters
- [ ] Changed error responses
- [ ] New endpoints (non-breaking)

**Document findings:**
```markdown
# API Version X.Y Breaking Changes

## Removed Endpoints
- DELETE /indexes/{name}/vectors (replaced by new bulk delete)

## Schema Changes
- `Vector.metadata` now required (was optional)
- `QueryRequest.top_k` max value reduced from 10000 to 1000

## New Required Parameters
- `create_index` now requires `spec` parameter

## Deprecated (to be removed)
- `legacy_upsert` method (use `upsert` instead)
```

### Step 3: Plan Migration Strategy

Decide how to handle breaking changes while maintaining backward compatibility.

**Strategy Options:**

1. **Adapter Pattern** - Add adapters to translate between versions
2. **Versioned Methods** - Offer both old and new methods (deprecate old)
3. **Major Version Bump** - Accept breaking changes for major release

**Example: Adapter Pattern**
```python
class APIAdapter:
    """Adapt between API versions."""

    @staticmethod
    def adapt_query_request(request: QueryRequest, api_version: str):
        """Adapt query request for different API versions."""
        if api_version == "2024-07":
            # New version requires different format
            return {
                "queries": [request.to_dict()],
                "namespace": request.namespace,
            }
        else:
            # Old version format
            return request.to_dict()
```

**Checklist:**
- [ ] Document all breaking changes
- [ ] Choose migration strategy
- [ ] Plan deprecation timeline
- [ ] Identify affected users/code

### Step 4: Update Protocol Definitions

Update protocols to reflect new API requirements.

**Location:** `pinecone/__interface__/`

```python
# Example: Adding new required parameter

@runtime_checkable
class IPineconeClient(Protocol):
    def create_index(
        self,
        name: str,
        dimension: int,
        spec: dict,  # Now required (was optional)
        metric: str = "cosine",
    ) -> None:
        """Create index with new spec parameter."""
        ...
```

**For backward compatibility:**
```python
@runtime_checkable
class IPineconeClient(Protocol):
    def create_index(
        self,
        name: str,
        dimension: int,
        spec: dict | None = None,  # Keep optional for now
        metric: str = "cosine",
    ) -> None:
        """Create index.

        Args:
            spec: Index specification. Required for API v2024-10+.
                  Optional for older versions but will be required
                  in next major version.

        .. deprecated:: 1.5.0
            Creating indexes without spec will be removed in v2.0.0
        """
        ...
```

**Checklist:**
- [ ] Update affected protocols
- [ ] Add deprecation warnings where needed
- [ ] Update docstrings
- [ ] Maintain backward compatibility if possible

### Step 5: Update Implementation

Update implementation to work with new API version.

**Location:** `pinecone/_internal/`

```python
class PineconeClient:
    def create_index(
        self,
        name: str,
        dimension: int,
        spec: dict | None = None,
        metric: str = "cosine",
    ) -> None:
        """Create index."""
        # Handle deprecated usage
        if spec is None:
            warnings.warn(
                "Creating indexes without 'spec' is deprecated and will "
                "be removed in v2.0.0. Please provide a spec parameter.",
                DeprecationWarning,
                stacklevel=2,
            )
            # Provide default spec for backward compatibility
            spec = {"serverless": {"cloud": "aws", "region": "us-east-1"}}

        payload = {
            "name": name,
            "dimension": dimension,
            "metric": metric,
            "spec": spec,
        }

        self._http_client.post("/indexes", json=payload)
```

**For adapters:**
```python
class HTTPClient:
    def __init__(self, api_version: str = "2024-10"):
        self.api_version = api_version
        self.adapter = APIAdapter(api_version)

    def post(self, endpoint: str, json: dict):
        # Adapt request for API version
        adapted = self.adapter.adapt_request(endpoint, json)
        # Make request with version header
        headers = {"X-API-Version": self.api_version}
        return requests.post(endpoint, json=adapted, headers=headers)
```

**Checklist:**
- [ ] Update implementations
- [ ] Add deprecation warnings
- [ ] Maintain backward compatibility
- [ ] Add version-specific logic if needed
- [ ] Update error handling

### Step 6: Update Tests

Update tests to cover new API version and backward compatibility.

```python
def test_create_index_with_spec():
    """Test create_index with new required spec."""
    mock_http = mocker.Mock()
    client = PineconeClient(api_key="test-key", http_client=mock_http)

    spec = {"serverless": {"cloud": "aws", "region": "us-east-1"}}
    client.create_index(name="test", dimension=128, spec=spec)

    mock_http.post.assert_called_once()
    call_args = mock_http.post.call_args
    assert call_args[1]["json"]["spec"] == spec

def test_create_index_without_spec_deprecated():
    """Test create_index without spec shows deprecation warning."""
    client = PineconeClient(api_key="test-key")

    with pytest.warns(DeprecationWarning, match="will be removed in v2.0.0"):
        client.create_index(name="test", dimension=128)

def test_adapter_transforms_request():
    """Test adapter transforms requests for new API."""
    adapter = APIAdapter("2024-10")
    request = QueryRequest(vector=[0.1] * 128, top_k=10)

    adapted = adapter.adapt_query_request(request, "2024-10")
    assert "queries" in adapted  # New format
    assert isinstance(adapted["queries"], list)
```

**Checklist:**
- [ ] Test new API version
- [ ] Test backward compatibility
- [ ] Test deprecation warnings
- [ ] Test adapter logic
- [ ] Test error handling for version-specific errors
- [ ] Update integration tests

### Step 7: Update Migration Guide

Document how users should migrate to the new version.

**Location:** `docs/migration-guides/v1-to-v2.md`

```markdown
# Migration Guide: v1.x to v2.0

## Breaking Changes

### create_index now requires spec parameter

**Old code:**
```python
pc.create_index(name="my-index", dimension=128)
```

**New code:**
```python
pc.create_index(
    name="my-index",
    dimension=128,
    spec={
        "serverless": {
            "cloud": "aws",
            "region": "us-east-1"
        }
    }
)
```

### Why this change?

The new API supports multiple deployment types (serverless, pod-based)
and requires explicit configuration to ensure optimal performance.
```

**Checklist:**
- [ ] Document all breaking changes
- [ ] Provide before/after code examples
- [ ] Explain rationale for changes
- [ ] Suggest migration strategies
- [ ] Link to relevant documentation

### Step 8: Update Version and Changelog

**Update version in `pyproject.toml` or `setup.py`:**
```toml
[project]
version = "2.0.0"  # Major bump for breaking changes
```

**Update CHANGELOG.md:**
```markdown
## [2.0.0] - 2024-10-15

### Breaking Changes
- `create_index` now requires `spec` parameter
- Removed deprecated `legacy_upsert` method
- Maximum `top_k` reduced from 10000 to 1000

### Added
- Support for API version 2024-10
- New bulk operations endpoint
- Enhanced error messages

### Migration
See [Migration Guide](docs/migration-guides/v1-to-v2.md)
```

**Checklist:**
- [ ] Bump version appropriately (major/minor/patch)
- [ ] Update changelog with all changes
- [ ] Link to migration guide
- [ ] Document deprecations

### Step 9: Run Full Verification

**Commands:**
```bash
# Run full test suite
pytest

# Run with coverage
pytest --cov=pinecone --cov-report=html

# Type checking
mypy pinecone

# Linting
ruff check .
black --check .

# Test against real API (if available)
pytest tests/e2e -m api_version_2024_10
```

**Checklist:**
- [ ] All tests pass
- [ ] Coverage >= 90%
- [ ] Type checking passes
- [ ] Linting passes
- [ ] Integration tests pass
- [ ] E2E tests pass (if available)

### Common Pitfalls

❌ **Breaking users without warning**
- Always add deprecation warnings before removing features

❌ **Inconsistent versioning**
- Follow semantic versioning strictly

❌ **Missing migration documentation**
- Users need clear guidance on how to upgrade

❌ **Not testing backward compatibility**
- Old code should continue to work (with warnings) until major version

❌ **Forgetting to update error messages**
- Error messages should reference current API version

---

## Workflow 3: Adding a New Namespace

**Goal:** Add a new namespace (e.g., `CollectionClient`) to the SDK

**Time:** 4-8 hours
**Complexity:** High

### Prerequisites

- [ ] Understand [three-layer architecture](./architecture/overview.md)
- [ ] Review [API structure](./architecture/api-structure.md)
- [ ] Read [naming conventions](./conventions/naming.md)

### Step 1: Design the Protocol

Define what operations the new namespace will support.

**Example: Collections namespace**

```python
# Questions to answer:
# - What operations are needed? (list, create, delete, describe)
# - What parameters do they take?
# - What do they return?
# - What errors can they raise?

# Document design decisions
"""
Collections Protocol Design
===========================

Operations:
- list_collections() -> list[str]
- create_collection(name, source_index) -> None
- delete_collection(name) -> None
- describe_collection(name) -> dict

Design decisions:
- Collections are read-only copies of indexes
- Source index must exist before creating collection
- Collection names follow same rules as index names
"""
```

**Checklist:**
- [ ] List all operations needed
- [ ] Define parameters and return types
- [ ] Identify possible errors
- [ ] Document design decisions

### Step 2: Create Protocol Interface

**Location:** `pinecone/__interface__/collection.py`

```python
from typing import Protocol, runtime_checkable

@runtime_checkable
class ICollectionClient(Protocol):
    """Protocol for collection operations."""

    def list_collections(self) -> list[str]:
        """List all collections.

        Returns:
            List of collection names

        Raises:
            PineconeApiException: If API request fails
        """
        ...

    def create_collection(
        self,
        name: str,
        source_index: str,
    ) -> None:
        """Create a new collection from an index.

        Args:
            name: Collection name
            source_index: Source index name

        Raises:
            ValidationError: If parameters are invalid
            PineconeApiException: If creation fails
        """
        ...

    def delete_collection(self, name: str) -> None:
        """Delete a collection.

        Args:
            name: Collection name

        Raises:
            PineconeApiException: If deletion fails
        """
        ...

    def describe_collection(self, name: str) -> dict:
        """Get collection details.

        Args:
            name: Collection name

        Returns:
            Collection metadata

        Raises:
            PineconeApiException: If request fails
        """
        ...
```

**Export in protocol __init__.py:**
```python
# pinecone/__interface__/__init__.py
from pinecone.__interface__.collection import ICollectionClient

__all__ = [
    # ... existing exports ...
    "ICollectionClient",
]
```

**Checklist:**
- [ ] Create protocol file
- [ ] Define all methods with complete signatures
- [ ] Add comprehensive docstrings
- [ ] Use `@runtime_checkable`
- [ ] Export in `__interface__/__init__.py`

### Step 3: Implement the Client

**Location:** `pinecone/_internal/collection.py`

```python
from pinecone.__interface__.collection import ICollectionClient
from pinecone._internal.http.client import HTTPClient
from pinecone._internal.errors import PineconeApiException, ValidationError

class CollectionClient:
    """Implementation of ICollectionClient protocol."""

    def __init__(self, http_client: HTTPClient):
        self._http_client = http_client

    def list_collections(self) -> list[str]:
        """List all collections."""
        try:
            response = self._http_client.get("/collections")
            return [c["name"] for c in response["collections"]]
        except HTTPError as e:
            raise PineconeApiException(
                f"Failed to list collections: {e.message}"
            ) from e

    def create_collection(
        self,
        name: str,
        source_index: str,
    ) -> None:
        """Create a new collection."""
        if not name:
            raise ValidationError("Collection name is required")
        if not source_index:
            raise ValidationError("Source index is required")

        try:
            self._http_client.post(
                "/collections",
                json={
                    "name": name,
                    "source": source_index,
                },
            )
        except HTTPError as e:
            if e.status_code == 404:
                raise PineconeApiException(
                    f"Source index '{source_index}' not found"
                ) from e
            elif e.status_code == 409:
                raise PineconeApiException(
                    f"Collection '{name}' already exists"
                ) from e
            else:
                raise PineconeApiException(
                    f"Failed to create collection: {e.message}"
                ) from e

    def delete_collection(self, name: str) -> None:
        """Delete a collection."""
        try:
            self._http_client.delete(f"/collections/{name}")
        except HTTPError as e:
            if e.status_code == 404:
                raise PineconeApiException(
                    f"Collection '{name}' not found"
                ) from e
            else:
                raise PineconeApiException(
                    f"Failed to delete collection: {e.message}"
                ) from e

    def describe_collection(self, name: str) -> dict:
        """Get collection details."""
        try:
            return self._http_client.get(f"/collections/{name}")
        except HTTPError as e:
            if e.status_code == 404:
                raise PineconeApiException(
                    f"Collection '{name}' not found"
                ) from e
            else:
                raise PineconeApiException(
                    f"Failed to describe collection: {e.message}"
                ) from e
```

**Checklist:**
- [ ] Implement all protocol methods
- [ ] Add input validation
- [ ] Handle all expected errors
- [ ] Transform HTTP errors to SDK exceptions
- [ ] Add internal docstrings

### Step 4: Integrate with Main Client

Add method to access the new namespace from the main client.

**Update protocol:**
```python
# pinecone/__interface__/client.py

@runtime_checkable
class IPineconeClient(Protocol):
    # ... existing methods ...

    def collections(self) -> ICollectionClient:
        """Get collections client.

        Returns:
            Client for collection operations
        """
        ...
```

**Update implementation:**
```python
# pinecone/_internal/client.py

from pinecone._internal.collection import CollectionClient

class PineconeClient:
    def __init__(self, api_key: str):
        self._api_key = api_key
        self._http_client = HTTPClient(api_key=api_key)
        self._collections_client = None

    def collections(self) -> ICollectionClient:
        """Get collections client."""
        if self._collections_client is None:
            self._collections_client = CollectionClient(
                http_client=self._http_client
            )
        return self._collections_client
```

**Checklist:**
- [ ] Add method to main client protocol
- [ ] Implement method in main client
- [ ] Use lazy initialization if appropriate
- [ ] Reuse HTTP client

### Step 5: Export in Public API

**Location:** `pinecone/__init__.py`

```python
# Import protocol
from pinecone.__interface__.collection import ICollectionClient

# Import implementation
from pinecone._internal.collection import CollectionClient

# Export in __all__
__all__ = [
    # ... existing exports ...
    "ICollectionClient",
    # Note: CollectionClient is accessed via PineconeClient.collections()
    # so we typically don't export it directly
]
```

**Checklist:**
- [ ] Import protocol
- [ ] Import implementation (if exposing directly)
- [ ] Add to `__all__`
- [ ] Decide if namespace should be directly importable

### Step 6: Write Comprehensive Tests

Create tests for the new namespace.

**Protocol tests:**
```python
# tests/protocols/test_collection_protocol.py

def test_collection_client_implements_protocol():
    """Verify CollectionClient implements ICollectionClient."""
    from pinecone import ICollectionClient
    from pinecone._internal.collection import CollectionClient
    from pinecone._internal.http.client import HTTPClient

    http_client = HTTPClient(api_key="test")
    client = CollectionClient(http_client=http_client)

    assert isinstance(client, ICollectionClient)
```

**Unit tests:**
```python
# tests/unit/test_collection.py

def test_list_collections(mocker):
    """Test list_collections returns collection names."""
    mock_http = mocker.Mock()
    mock_http.get.return_value = {
        "collections": [
            {"name": "col-1"},
            {"name": "col-2"},
        ]
    }

    client = CollectionClient(http_client=mock_http)
    collections = client.list_collections()

    assert collections == ["col-1", "col-2"]
    mock_http.get.assert_called_once_with("/collections")

def test_create_collection_success(mocker):
    """Test create_collection with valid parameters."""
    mock_http = mocker.Mock()
    client = CollectionClient(http_client=mock_http)

    client.create_collection(name="my-col", source_index="my-index")

    mock_http.post.assert_called_once_with(
        "/collections",
        json={"name": "my-col", "source": "my-index"},
    )

def test_create_collection_validation():
    """Test create_collection validates inputs."""
    mock_http = mocker.Mock()
    client = CollectionClient(http_client=mock_http)

    with pytest.raises(ValidationError, match="name is required"):
        client.create_collection(name="", source_index="my-index")
```

**Integration tests:**
```python
# tests/integration/test_collection_integration.py

@responses.activate
def test_collections_integration():
    """Test collections namespace integration."""
    # Mock collections list
    responses.add(
        responses.GET,
        "https://api.pinecone.io/collections",
        json={"collections": [{"name": "col-1"}]},
        status=200,
    )

    pc = PineconeClient(api_key="test-key")
    collections = pc.collections().list_collections()

    assert collections == ["col-1"]
```

**Checklist:**
- [ ] Protocol compliance tests
- [ ] Unit tests for all methods
- [ ] Validation error tests
- [ ] API error handling tests
- [ ] Integration tests
- [ ] Test integration with main client
- [ ] Coverage >= 90%

### Step 7: Add Documentation

Add usage examples and documentation.

**Example in docstring:**
```python
class IPineconeClient(Protocol):
    def collections(self) -> ICollectionClient:
        """Get collections client.

        Example:
            >>> pc = PineconeClient(api_key="...")
            >>> collections = pc.collections()
            >>> collections.create_collection(
            ...     name="my-collection",
            ...     source_index="my-index"
            ... )
            >>> print(collections.list_collections())
            ['my-collection']
        """
        ...
```

**Checklist:**
- [ ] Add usage examples to docstrings
- [ ] Update main documentation if needed
- [ ] Add to examples directory if significant

### Step 8: Run Verification

**Commands:**
```bash
# Run all tests
pytest

# Check coverage
pytest --cov=pinecone._internal.collection --cov-report=term-missing

# Type checking
mypy pinecone

# Linting
ruff check .
black --check .
```

**Checklist:**
- [ ] All tests pass
- [ ] Coverage >= 90%
- [ ] Type checking passes
- [ ] Linting passes

### Common Pitfalls

❌ **Not integrating with main client**
- Users should access namespace through main client

❌ **Forgetting lazy initialization**
- Create namespace clients only when needed

❌ **Inconsistent error handling**
- All namespaces should handle errors consistently

❌ **Missing protocol exports**
- Protocol must be exported for type hints

❌ **Not testing integration**
- Test that namespace works through main client

---

## Workflow 4: Performance Optimization

**Goal:** Identify and fix performance bottlenecks

**Time:** Variable (hours to days)
**Complexity:** Medium-High

### Prerequisites

- [ ] Understanding of [performance conventions](./conventions/performance.md)
- [ ] Profiling tools installed (`py-spy`, `memory_profiler`)
- [ ] Benchmark environment available

### Step 1: Identify Performance Issue

Define what performance issue you're investigating.

**Common issues:**
- [ ] Slow API calls
- [ ] High memory usage
- [ ] CPU bottlenecks
- [ ] Inefficient algorithms
- [ ] Unnecessary copies
- [ ] Blocking I/O

**Gather baseline metrics:**
```python
import time

def benchmark_operation():
    """Benchmark current performance."""
    start = time.time()

    # Run operation
    client = PineconeClient(api_key="...")
    index = client.index("benchmark-index")
    for i in range(1000):
        index.query(vector=[0.1] * 128, top_k=10)

    elapsed = time.time() - start
    print(f"1000 queries took {elapsed:.2f}s")
    print(f"Average: {elapsed/1000*1000:.2f}ms per query")

benchmark_operation()
```

**Checklist:**
- [ ] Define performance goal
- [ ] Establish baseline metrics
- [ ] Identify affected operations
- [ ] Determine acceptable performance

### Step 2: Profile the Code

Use profiling tools to identify bottlenecks.

#### CPU Profiling with py-spy

```bash
# Install py-spy
pip install py-spy

# Profile a running script
py-spy record -o profile.svg -- python benchmark.py

# View the flame graph
open profile.svg
```

#### Memory Profiling

```bash
# Install memory_profiler
pip install memory_profiler

# Add @profile decorator to functions
# Then run:
python -m memory_profiler benchmark.py
```

#### Line Profiler

```python
# Install line_profiler
pip install line_profiler

# Use @profile decorator
from line_profiler import LineProfiler

def benchmark():
    client = PineconeClient(api_key="...")
    index = client.index("test")
    index.query(vector=[0.1] * 128, top_k=10)

profiler = LineProfiler()
profiler.add_function(benchmark)
profiler.run('benchmark()')
profiler.print_stats()
```

**Checklist:**
- [ ] Run CPU profiler
- [ ] Run memory profiler
- [ ] Identify hot spots (>10% of time)
- [ ] Document findings

### Step 3: Design Optimization

Plan how to fix the identified bottleneck.

**Common optimization patterns:**

#### Connection Pooling

```python
# Before: Creating new connection each time
class PineconeClient:
    def __init__(self, api_key: str):
        self._api_key = api_key

    def index(self, name: str):
        # Creates new HTTP client each time!
        http_client = HTTPClient(self._api_key)
        return IndexClient(name, http_client)

# After: Reuse HTTP client with connection pooling
class PineconeClient:
    def __init__(self, api_key: str):
        self._api_key = api_key
        self._http_client = HTTPClient(
            api_key=api_key,
            pool_connections=10,
            pool_maxsize=100,
        )

    def index(self, name: str):
        # Reuses pooled connections
        return IndexClient(name, self._http_client)
```

#### Batching

```python
# Before: Individual operations
for vector in vectors:
    index.upsert(vectors=[vector])

# After: Batch operations
index.upsert(vectors=vectors, batch_size=100)
```

#### Caching

```python
# Before: Fetch metadata every time
def get_index_dimension(client, name):
    metadata = client.describe_index(name)
    return metadata["dimension"]

# After: Cache metadata
from functools import lru_cache

@lru_cache(maxsize=128)
def get_index_dimension(client, name):
    metadata = client.describe_index(name)
    return metadata["dimension"]
```

#### Lazy Evaluation

```python
# Before: Eager initialization
class PineconeClient:
    def __init__(self, api_key: str):
        self._indexes = self.list_indexes()  # Called immediately!

# After: Lazy initialization
class PineconeClient:
    def __init__(self, api_key: str):
        self._indexes = None

    @property
    def indexes(self):
        if self._indexes is None:
            self._indexes = self.list_indexes()
        return self._indexes
```

**Checklist:**
- [ ] Identify optimization technique
- [ ] Estimate expected improvement
- [ ] Consider trade-offs
- [ ] Plan implementation

### Step 4: Implement Optimization

Implement the optimization carefully.

**Example: Add connection pooling**

```python
# pinecone/_internal/http/client.py

import httpx

class HTTPClient:
    def __init__(
        self,
        api_key: str,
        pool_connections: int = 10,
        pool_maxsize: int = 100,
        timeout: float = 30.0,
    ):
        limits = httpx.Limits(
            max_keepalive_connections=pool_connections,
            max_connections=pool_maxsize,
        )

        self._client = httpx.Client(
            timeout=timeout,
            limits=limits,
            headers={"Authorization": f"Bearer {api_key}"},
        )

    def __enter__(self):
        return self

    def __exit__(self, *args):
        self._client.close()
```

**Checklist:**
- [ ] Implement optimization
- [ ] Add configuration if needed
- [ ] Maintain backward compatibility
- [ ] Add docstrings explaining optimization

### Step 5: Add Benchmarks

Create benchmark tests to verify improvement.

```python
# tests/benchmarks/test_performance.py

import pytest

@pytest.mark.benchmark
def test_query_performance_with_pooling(benchmark):
    """Benchmark query performance with connection pooling."""
    client = PineconeClient(api_key="test-key")
    index = client.index("benchmark")

    def query():
        return index.query(vector=[0.1] * 128, top_k=10)

    result = benchmark(query)

    # Assert performance targets
    assert benchmark.stats.mean < 0.1  # < 100ms average
    assert benchmark.stats.stddev < 0.02  # Low variance

@pytest.mark.benchmark(group="batch-operations")
def test_batch_upsert_performance(benchmark):
    """Benchmark batch upsert performance."""
    client = PineconeClient(api_key="test-key")
    index = client.index("benchmark")
    vectors = [Vector(id=f"v{i}", values=[0.1] * 128) for i in range(100)]

    def upsert():
        return index.upsert(vectors=vectors)

    result = benchmark(upsert)
    assert benchmark.stats.mean < 1.0  # < 1s for 100 vectors
```

**Run benchmarks:**
```bash
# Install pytest-benchmark
pip install pytest-benchmark

# Run benchmarks
pytest tests/benchmarks/ --benchmark-only

# Compare with baseline
pytest tests/benchmarks/ --benchmark-compare=baseline --benchmark-save=optimized
```

**Checklist:**
- [ ] Create benchmark tests
- [ ] Save baseline metrics
- [ ] Run optimized version
- [ ] Compare results

### Step 6: Verify No Regressions

Ensure optimization doesn't break existing functionality.

**Commands:**
```bash
# Run full test suite
pytest

# Run with coverage
pytest --cov=pinecone --cov-report=term

# Type checking
mypy pinecone

# Check for memory leaks (if applicable)
python -m memory_profiler tests/benchmarks/test_memory.py
```

**Checklist:**
- [ ] All tests pass
- [ ] No new warnings
- [ ] Type checking passes
- [ ] Coverage maintained
- [ ] No memory leaks

### Step 7: Document Performance Characteristics

Document the optimization and its impact.

```python
class HTTPClient:
    """HTTP client with connection pooling.

    Performance characteristics:
    - Connection pooling reduces latency by ~30% for repeated requests
    - Supports up to 100 concurrent connections
    - Keep-alive connections maintained for up to 10 idle connections

    Configuration:
        pool_connections: Max idle connections to keep (default: 10)
        pool_maxsize: Max total connections (default: 100)
        timeout: Request timeout in seconds (default: 30.0)

    Example:
        >>> client = HTTPClient(
        ...     api_key="...",
        ...     pool_connections=20,  # More idle connections
        ...     pool_maxsize=200,     # Support more concurrency
        ... )
    """
```

**Update performance conventions:**
```markdown
# docs/conventions/performance.md

## Connection Pooling

The SDK uses connection pooling to improve performance:
- Reduces connection overhead by ~30%
- Configurable via HTTPClient parameters
- Default: 10 keep-alive, 100 max connections

See HTTPClient documentation for configuration options.
```

**Checklist:**
- [ ] Document optimization in code
- [ ] Update performance conventions
- [ ] Add configuration examples
- [ ] Document performance gains

### Step 8: Measure Results

Compare before and after metrics.

**Create comparison report:**
```markdown
# Performance Optimization Results

## Operation: Index queries with connection pooling

### Before
- Average latency: 150ms
- P95 latency: 250ms
- P99 latency: 400ms
- Throughput: 100 queries/second

### After
- Average latency: 105ms (-30%)
- P95 latency: 175ms (-30%)
- P99 latency: 280ms (-30%)
- Throughput: 143 queries/second (+43%)

### Configuration
- pool_connections: 10
- pool_maxsize: 100
- timeout: 30.0s

### Trade-offs
- Slightly higher memory usage (~2MB for connection pool)
- More complex cleanup (must close client properly)

### Recommendation
Enable by default with conservative settings.
```

**Checklist:**
- [ ] Measure baseline
- [ ] Measure optimized version
- [ ] Calculate improvement percentage
- [ ] Document trade-offs
- [ ] Make recommendation

### Common Pitfalls

❌ **Premature optimization**
- Profile first, optimize second

❌ **Breaking API for performance**
- Maintain backward compatibility

❌ **Optimizing the wrong thing**
- Focus on actual bottlenecks, not assumptions

❌ **No benchmarks**
- Can't verify improvement without measurements

❌ **Ignoring trade-offs**
- Every optimization has costs (complexity, memory, etc.)

---

## General Tips

### Before Starting Any Workflow

1. **Read the docs** - Review architecture and conventions
2. **Check existing code** - Look for similar implementations
3. **Plan before coding** - Think through the design
4. **Start with tests** - Consider test-driven development

### During Development

1. **Commit often** - Small, atomic commits
2. **Run tests frequently** - Catch issues early
3. **Use type hints** - Let mypy catch bugs
4. **Follow conventions** - Consistency matters

### Before Completing

1. **Run full verification** - All checks must pass
2. **Review your own code** - Fresh eyes catch issues
3. **Update documentation** - Docs are part of the feature
4. **Clean up** - Remove debug code, TODOs, etc.

## Getting Help

### Documentation Resources

- [Architecture Overview](./architecture/overview.md)
- [API Structure](./architecture/api-structure.md)
- [Protocol Specifications](./architecture/specifications.md)
- [Verification Guide](./architecture/verification.md)
- [AI Development Guide](./architecture/ai-development.md)
- [All Conventions](./conventions/README.md)

### Commands Reference

```bash
# Setup
pip install -e ".[dev]"
pre-commit install

# Testing
pytest                              # All tests
pytest tests/unit                   # Unit tests only
pytest tests/integration            # Integration tests only
pytest --cov=pinecone              # With coverage

# Type checking
mypy pinecone
pyright

# Linting
ruff check .
black --check .

# Formatting
black .
ruff check --fix .

# Full verification
make check  # Or run all of the above

# Pre-commit hooks
pre-commit run --all-files        # Run all hooks manually
pre-commit run <hook-id>          # Run specific hook
pre-commit autoupdate             # Update hook versions
git commit --no-verify            # Skip hooks (emergencies only)
```

### Pre-Commit Hooks

The repository uses pre-commit hooks for automated code quality checks:

**Automatic checks on every commit:**
- Remove trailing whitespace
- Fix end-of-file issues
- Validate YAML and TOML files
- Check for large files (>1MB)
- Run ruff linting and formatting
- Type check with mypy (pinecone/ only)
- Check docstring conventions
- Check spelling
- Detect secrets

**Setup (one-time):**
```bash
pip install pre-commit
pre-commit install
```

**Usage:**
```bash
# Hooks run automatically on git commit
git commit -m "your message"

# Run hooks manually on all files
pre-commit run --all-files

# Run specific hook
pre-commit run ruff

# Skip hooks in emergencies (use sparingly!)
git commit --no-verify -m "emergency fix"

# Update hook versions
pre-commit autoupdate
```

**Performance:**
- First run: ~45 seconds (installs environments)
- Subsequent runs: ~3 seconds (well under 10s target)
- Only checks changed files by default

**Note:** If hooks fail, they often auto-fix issues. Just stage the fixes and commit again.

### When Stuck

1. Check existing implementations for patterns
2. Review relevant documentation sections
3. Look at test files for usage examples
4. Run type checker for hints
5. Ask for help (with context about what you've tried)

## Conclusion

Following these workflows ensures consistency, quality, and maintainability. Each workflow builds on the SDK's architecture and conventions to guide you through complex tasks step by step.

Remember: **Plan, implement, test, document, verify**. This cycle applies to every development task.
