# Pagination

This document describes the pagination pattern used for list operations in the Pinecone SDK.

## Overview

List operations (indexes, collections, backups) use a **transparent iterator pattern** that:

- Fetches pages lazily on demand
- Provides simple item-level iteration by default
- Exposes page-level access when needed
- Supports resumption via pagination tokens

## User-Facing API

### Simple Iteration (Most Common)

```python
# Iterate all items - pages fetched automatically
for index in pc.indexes.list():
    print(index.name)

# Works with list comprehensions
names = [idx.name for idx in pc.indexes.list()]

# Async version
async for index in async_pc.indexes.list():
    print(index.name)
```

### Convert to List

```python
# Fetch all items into a list
all_indexes = pc.indexes.list().to_list()

# Async version
all_indexes = await async_pc.indexes.list().to_list()
```

### Limit Total Items

```python
# Get at most 10 items (across all pages)
for index in pc.indexes.list(limit=10):
    print(index.name)
```

### Page-Level Access

Use `.pages()` when you need access to pagination metadata or batch processing:

```python
for page in pc.indexes.list().pages():
    print(f"Got {len(page.items)} items")
    if page.has_more:
        print(f"More pages available")
    for index in page.items:
        process(index)
```

### Resume Pagination

```python
# Start iteration
iterator = pc.indexes.list()
for index in iterator:
    if should_stop():
        # Save token for later
        token = iterator.pagination_token
        break

# Later: resume from saved token
for index in pc.indexes.list(pagination_token=token):
    print(index.name)
```

## Core Types

### Page[T]

A single page of results from a paginated API.

```python
from pinecone import Page

class Page(Generic[T]):
    items: list[T]                    # Items in this page
    pagination_token: str | None      # Token for next page, None if last

    @property
    def has_more(self) -> bool:       # True if more pages available
```

### Paginator[T]

Lazy iterator over paginated API results (sync).

```python
from pinecone import Paginator

class Paginator(Generic[T]):
    def __iter__(self) -> Iterator[T]:          # Iterate items
    def pages(self) -> Iterator[Page[T]]:       # Iterate pages
    def to_list(self) -> list[T]:               # Fetch all as list

    @property
    def pagination_token(self) -> str | None:   # Current token
```

### AsyncPaginator[T]

Async version with identical API:

```python
from pinecone import AsyncPaginator

class AsyncPaginator(Generic[T]):
    async def __aiter__(self) -> AsyncIterator[T]:
    async def pages(self) -> AsyncIterator[Page[T]]:
    async def to_list(self) -> list[T]:

    @property
    def pagination_token(self) -> str | None:
```

## Implementation Guide

> **Note**: Use `@pagination-review` agent to validate your implementation matches these patterns.

### Adding a Paginated List Method

1. **Update the protocol** in the appropriate namespace module (e.g., `pinecone/__interface__/indexes/namespace.py`):

```python
def list(
    self,
    *,
    limit: int | None = None,
    pagination_token: str | None = None,
) -> Paginator[ItemInfoProtocol]:
    """List all items.

    Args:
        limit: Maximum number of items to return (across all pages).
        pagination_token: Token to resume pagination from a previous call.

    Returns:
        Paginator over ItemInfo objects.
    """
    ...
```

2. **Implement the method**:

```python
from pinecone.models.pagination import Page, Paginator

def list(
    self,
    *,
    limit: int | None = None,
    pagination_token: str | None = None,
) -> Paginator[ItemInfo]:
    def fetch_page(token: str | None) -> Page[ItemInfo]:
        params = {}
        if token:
            params["paginationToken"] = token

        response = self._http.get("/items", params=params)
        return self._adapter.to_item_page(response)

    return Paginator(
        fetch_page=fetch_page,
        initial_token=pagination_token,
        limit=limit,
    )
```

3. **Create the adapter method**:

```python
def to_item_page(self, response: httpx.Response) -> Page[ItemInfo]:
    data = orjson.loads(response.content)
    items = [
        msgspec.convert(item, ItemInfo)
        for item in data.get("items", [])
    ]
    pagination = data.get("pagination", {})
    return Page(
        items=items,
        pagination_token=pagination.get("next"),
    )
```

### Key Implementation Notes

- **Lazy loading**: No API calls until iteration starts
- **`limit` is SDK-side**: Caps total items yielded, not page size
- **Page size is API-controlled**: The API decides items per page
- **Single-use iterators**: Standard Python behavior - iterate once, then exhausted
- **Fresh per call**: Each `pc.list_items()` returns a new iterator

## Testing Paginated Methods

### Unit Tests with Mocked HTTP

```python
import pytest
import respx
from httpx import Response

from pinecone import Pinecone

@respx.mock
def test_list_indexes_pagination() -> None:
    # Mock first page
    respx.get("https://api.pinecone.io/indexes").mock(
        return_value=Response(200, json={
            "indexes": [{"name": "idx-1", ...}],
            "pagination": {"next": "token123"}
        })
    )
    # Mock second page
    respx.get("https://api.pinecone.io/indexes?paginationToken=token123").mock(
        return_value=Response(200, json={
            "indexes": [{"name": "idx-2", ...}],
            "pagination": {}
        })
    )

    pc = Pinecone(api_key="test-key")
    indexes = pc.indexes.list().to_list()

    assert len(indexes) == 2
    assert indexes[0].name == "idx-1"
    assert indexes[1].name == "idx-2"
```

### Testing Page-Level Access

```python
@respx.mock
def test_list_indexes_pages() -> None:
    # Setup mocks...

    pc = Pinecone(api_key="test-key")
    pages = list(pc.indexes.list().pages())

    assert len(pages) == 2
    assert pages[0].has_more is True
    assert pages[1].has_more is False
```

### Testing Limit Parameter

```python
@respx.mock
def test_list_indexes_with_limit() -> None:
    # Mock page with many items
    respx.get("https://api.pinecone.io/indexes").mock(
        return_value=Response(200, json={
            "indexes": [{"name": f"idx-{i}", ...} for i in range(10)],
            "pagination": {"next": "more"}
        })
    )

    pc = Pinecone(api_key="test-key")
    indexes = pc.indexes.list(limit=3).to_list()

    assert len(indexes) == 3
```

## Design Rationale

### Why Transparent Iterator?

We chose this pattern over alternatives because:

1. **Best developer experience**: Simple case is trivially simple
2. **Pythonic**: Works with for loops, list comprehensions, itertools
3. **Memory efficient**: Pages fetched on demand
4. **Flexible**: `.pages()` available when needed
5. **Proven**: Used by Google Cloud, Azure, and other major SDKs

### Alternatives Considered

- **Explicit paginator (boto3-style)**: Requires extra step to get pagination
- **Method-based (Stripe-style)**: Extra `.auto_paging_iter()` call
- **Return list directly**: Loads everything into memory, no pagination control

### API Token Naming

We use `pagination_token` (snake_case) in the SDK to match the Pinecone API's `paginationToken` (camelCase), following Python naming conventions while maintaining API alignment.
