# HTTP Client

**Overview:** Internal HTTP client layer for all API communication

**Location:** `pinecone/_internal/http/`

**Never imported by users** - abstracted behind public API layer

---

## Architecture

The SDK uses `httpx` as its HTTP client library for both sync and async operations:

- **HTTPClient** (`client.py`) - Synchronous client with connection pooling
- **AsyncHTTPClient** (`async_client.py`) - Asynchronous client for concurrent operations
- **HTTPConfig** (`config.py`) - Immutable configuration using `msgspec.Struct`

### Why httpx?

- **HTTP/2 support** - Multiplexing and header compression for better performance
- **Connection pooling** - Reuse connections to reduce latency
- **Async/sync parity** - Same API for both sync and async
- **Type safety** - Full type hints and modern Python support

## Data Plane Client

The `Index` and `AsyncIndex` classes are standalone data plane clients. Each creates its own `HTTPClient` / `AsyncHTTPClient` pointed at the index host, separate from any control plane client.

### Standalone Construction

```python
from pinecone import Index

# Direct construction -- no Pinecone client needed
index = Index(host="my-index-abc123.svc.pinecone.io", api_key="...")
index.upsert(vectors=[{"id": "v1", "values": [0.1, 0.2]}])
results = index.query(vector=[0.1, 0.2], top_k=10)
index.close()
```

### Factory Config Delegation

When using the `pc.index()` factory, all shared configuration is forwarded automatically:

```python
pc = Pinecone(api_key="...", timeout=60)

# Factory forwards api_key, timeout, additional_headers
index = pc.index(host="my-index-abc123.svc.pinecone.io")

# Equivalent to:
index = Index(
    host="my-index-abc123.svc.pinecone.io",
    api_key=pc._api_key,
    timeout=pc._timeout,
    additional_headers=pc._additional_headers,
)
```

The `Index` creates its own HTTP connection pool pointed at the data plane host. It does not share the control plane client's connection pool (different base URL).

## HTTP/2 Configuration

### Why HTTP/2 is Enabled by Default

```python
class HTTPClient:
    def __init__(self, api_key: str, config: HTTPConfig | None = None):
        self._client = httpx.Client(
            http2=True,  # ✅ Enabled by default
            timeout=config.timeout,
            limits=httpx.Limits(...),
        )
```

**Benefits:**

- **Multiplexing** - Multiple requests over single connection
- **Header compression** - Reduced bandwidth usage
- **Server push** - Proactive resource delivery (when supported)
- **Request prioritization** - Better resource utilization

**When to disable:**

```python
from pinecone._internal.http import HTTPConfig

# Disable if debugging connection issues or working with HTTP/1.1-only proxies
config = HTTPConfig(http2=False)
```

## Connection Pooling

### Pool Configuration

```python
config = HTTPConfig(
    max_connections=100,           # Total concurrent connections
    max_keepalive_connections=20,  # Persistent connections
)
```

**Default pool sizes:**

- `max_connections=100` - Suitable for most use cases
- `max_keepalive_connections=20` - Balance between memory and latency

### Connection Reuse Pattern

```python
# ✅ Good: Reuse client for multiple requests
with Pinecone(api_key="...") as pc:
    for doc in documents:
        response = pc.inference.embed(model="...", texts=[doc])
        # Connection is reused from pool

# ✅ Good: Reuse Index for multiple data plane operations
with Index(host="my-index.svc.pinecone.io", api_key="...") as index:
    for batch in vector_batches:
        index.upsert(vectors=batch)
        # Connection is reused from pool

# ❌ Bad: Create new client for each request
for doc in documents:
    with Pinecone(api_key="...") as pc:
        response = pc.inference.embed(model="...", texts=[doc])
        # New connection every time (slow!)
```

### Async Connection Pooling

```python
# ✅ Good: Concurrent requests share connection pool
async with AsyncPinecone(api_key="...") as pc:
    tasks = [
        pc.inference.embed(model="...", texts=batch)
        for batch in text_batches
    ]
    results = await asyncio.gather(*tasks)
    # All requests share the same connection pool

# ✅ Good: Concurrent data plane operations
async with AsyncIndex(host="my-index.svc.pinecone.io", api_key="...") as index:
    tasks = [index.query(vector=vec, top_k=10) for vec in query_vectors]
    results = await asyncio.gather(*tasks)
```

## Retry Configuration

### Default Retry Behavior

Automatic retries for transient failures with exponential backoff:

```python
config = HTTPConfig(
    max_retries=3,                              # Total attempts
    retry_statuses=(429, 500, 502, 503, 504),   # Status codes to retry
    retry_backoff_factor=2.0,                   # Exponential multiplier
    retry_max_wait=60,                          # Max wait between retries
)
```

### Retry Strategy

**Exponential backoff with jitter:**

```python
# First retry: 1s * 2^0 + jitter = ~1s
# Second retry: 1s * 2^1 + jitter = ~2s
# Third retry: 1s * 2^2 + jitter = ~4s
```

**Jitter prevents thundering herd** - Random delay added to avoid synchronized retries from multiple clients

### Retry-After Header

The client respects `Retry-After` headers from rate limit responses:

```python
# Server responds with 429 + Retry-After: 10
# Client waits 10 seconds before retrying (overrides backoff)
```

### Custom Retry Configuration

```python
# More aggressive retries for critical operations
config = HTTPConfig(
    max_retries=5,
    retry_backoff_factor=1.5,  # Slower backoff growth
    retry_max_wait=120,        # Allow longer waits
)

# No retries for fast-fail scenarios
config = HTTPConfig(max_retries=0)
```

## Timeout Configuration

### Default Timeouts

```python
config = HTTPConfig(
    timeout=30,  # 30 seconds for all operations
)
```

Applied to:
- Connection establishment
- Data read operations
- Data write operations

### Custom Timeouts

```python
# Longer timeout for large embeddings
config = HTTPConfig(timeout=60)

# Shorter timeout for health checks
config = HTTPConfig(timeout=5)
```

### Request-Level Timeouts

```python
# Override global timeout for specific request
response = pc.inference.embed(
    model="multilingual-e5-large",
    texts=texts,
    timeout=10.0,  # 10 seconds for this request only
)
```

### Async Timeout Handling

```python
# ✅ Good: Handle timeouts gracefully
async with AsyncPinecone(api_key="...") as pc:
    try:
        response = await pc.inference.embed(
            model="...",
            texts=texts,
            timeout=5.0,
        )
    except TimeoutError:
        logger.warning("Request timed out, falling back to cached embeddings")
        response = get_cached_embeddings(texts)
```

## Error Handling

### Connection Errors

```python
try:
    response = pc.inference.embed(model="...", texts=["test"])
except APIConnectionError as e:
    # Network issues, connection failures
    logger.error(f"Connection failed: {e.message}")
except TimeoutError as e:
    # Request timed out
    logger.error(f"Timeout after {e.timeout}s")
```

### API Errors

```python
try:
    response = pc.inference.embed(model="...", texts=["test"])
except BadRequestError as e:
    # 400 - Invalid request
    logger.error(f"Bad request: {e.message}")
except UnauthorizedError:
    # 401 - Invalid API key
    logger.error("Invalid API key")
except RateLimitError as e:
    # 429 - Rate limit exceeded
    if e.retry_after:
        logger.warning(f"Rate limited, retry after {e.retry_after}s")
except ServerError as e:
    # 500+ - Server error
    logger.error(f"Server error {e.status_code}: {e.message}")
```

## Performance Best Practices

### Reuse Clients

```python
# ✅ Good: Single client for entire application lifecycle
pc = Pinecone(api_key="...")
for batch in data_batches:
    result = pc.inference.embed(model="...", texts=batch)

# ✅ Good: Single Index for repeated data plane operations
index = pc.index(host="my-index.svc.pinecone.io")
for batch in vector_batches:
    index.upsert(vectors=batch)

# ❌ Bad: New client every request
for batch in data_batches:
    pc = Pinecone(api_key="...")
    result = pc.inference.embed(model="...", texts=batch)
    pc.close()
```

### Use Context Managers

```python
# ✅ Good: Automatic cleanup
with Pinecone(api_key="...") as pc:
    result = pc.inference.embed(model="...", texts=texts)
# Client closed automatically

# ✅ Good: Data plane client
with Index(host="my-index.svc.pinecone.io", api_key="...") as index:
    results = index.query(vector=[0.1, 0.2], top_k=10)
# Client closed automatically

# ✅ Good: Async version
async with AsyncIndex(host="my-index.svc.pinecone.io", api_key="...") as index:
    results = await index.query(vector=[0.1, 0.2], top_k=10)
# Client closed automatically
```

### Connection Pool Tuning

For high-concurrency scenarios:

```python
# High-throughput configuration
config = HTTPConfig(
    max_connections=500,           # More concurrent connections
    max_keepalive_connections=100, # More persistent connections
    timeout=60,                    # Longer timeout for queued requests
)

async with AsyncPinecone(api_key="...", config=config) as pc:
    # Can handle hundreds of concurrent requests
    tasks = [pc.inference.embed(...) for _ in range(1000)]
    results = await asyncio.gather(*tasks)
```

## Logging and Debugging

### Enable Request/Response Logging

```python
config = HTTPConfig(
    log_requests=True,   # Log outgoing requests
    log_responses=True,  # Log incoming responses
)

pc = Pinecone(api_key="...", config=config)
```

**Log output:**

```
DEBUG - HTTP request: method=POST url=https://api.pinecone.io/embed attempt=1
DEBUG - HTTP response: method=POST url=https://api.pinecone.io/embed status=200 duration=0.123
```

### Debug Connection Issues

```python
# Check connection pool status
import httpx

# Enable httpx debug logging
import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("httpx").setLevel(logging.DEBUG)
```

## Security Considerations

### SSL/TLS Verification

```python
# ✅ Good: SSL verification enabled by default
pc = Pinecone(api_key="...")

# ❌ Bad: Never disable SSL verification in production
# (httpx doesn't expose this by default, which is good)
```

### API Key Handling

```python
# ✅ Good: API key in headers only
# The HTTP client adds: {"Api-Key": api_key}

# ✅ Good: API key never logged
config = HTTPConfig(log_requests=True)
# Logs never include sensitive headers
```

### Connection Security

- **HTTP/2** - Encrypted by default (TLS required)
- **Connection pooling** - Reuse secure connections
- **Timeouts** - Prevent resource exhaustion attacks

## Testing with HTTP Clients

### Mock with respx

```python
import respx
from httpx import Response

@respx.mock
async def test_embed_success():
    # Mock HTTP response
    respx.post("https://api.pinecone.io/embed").mock(
        return_value=Response(200, json={"data": [...], "model": "..."})
    )

    async with AsyncPinecone(api_key="test") as pc:
        response = await pc.inference.embed(model="...", texts=["test"])
        assert len(response.embeddings) > 0
```

### Test Retry Logic

```python
@respx.mock
async def test_retry_on_500():
    route = respx.post("https://api.pinecone.io/embed").mock(
        side_effect=[
            Response(500),  # First attempt fails
            Response(500),  # Second attempt fails
            Response(200, json={"data": [...]}),  # Third succeeds
        ]
    )

    async with AsyncPinecone(api_key="test") as pc:
        response = await pc.inference.embed(model="...", texts=["test"])
        assert route.call_count == 3
```

## Related Conventions

- [Error Handling](errors.md) - Exception hierarchy and error handling patterns
- [Performance](performance.md) - Performance optimization techniques
- [Async Patterns](async.md) - Async/await best practices
- [Testing](testing.md) - Testing with mocked HTTP clients

## References

- [httpx documentation](https://www.python-httpx.org/)
- [HTTP/2 spec](https://httpwg.org/specs/rfc7540.html)
- [Connection pooling best practices](https://www.python-httpx.org/advanced/#pool-limit-configuration)
