# Story 5.23: PostgreSQL Database Sink

## Status: Complete

## Priority: Medium

## Estimated Effort: Medium (4-5 days)

## Dependencies: None

## Epic: Database Sink Integrations

---

## Context

fapilog currently has no database sink implementations. This story introduces a PostgreSQL sink as the first database sink, establishing patterns for:

1. **Connection pooling** — Managing async database connections efficiently
2. **Schema management** — Auto-creating tables with appropriate indexes
3. **Bulk inserts** — Batching log events into efficient INSERT statements
4. **Async database drivers** — Using `asyncpg` for non-blocking I/O

PostgreSQL is chosen as the first database sink because:

- Widely deployed in production environments
- Excellent async driver support (`asyncpg`)
- JSON/JSONB support for structured log storage
- Strong indexing capabilities for log queries
- Foundation for time-series partitioning (TimescaleDB extension)

This sink complements the existing cloud sinks (CloudWatch, Loki) by providing a self-hosted storage option suitable for:

- On-premises deployments
- Air-gapped environments
- Organizations requiring data sovereignty
- Audit trail storage with SQL queryability

---

## Goals

1. Provide a production-ready PostgreSQL sink with async batching
2. Demonstrate database sink integration patterns
3. Enable efficient log storage with queryable JSON fields
4. Support both simple table and partitioned storage models
5. Document patterns for users building similar database sinks (MySQL, MongoDB, ClickHouse)

---

## Acceptance Criteria

### AC1: Sink Implementation

- [ ] Create `src/fapilog/plugins/sinks/contrib/postgres.py`
- [ ] Implement full `BaseSink` protocol with lifecycle methods
- [ ] Support `write_serialized()` fast path
- [ ] Use `asyncpg` for non-blocking database operations
- [ ] Implement connection pooling with configurable pool size
- [ ] Add circuit breaker integration for fault isolation
- [ ] Handle PostgreSQL-specific errors gracefully (connection loss, deadlocks)

### AC2: Configuration

- [ ] `PostgresSinkConfig` dataclass with all options
- [ ] Environment variable support (`FAPILOG_POSTGRES_*`)
- [ ] Settings integration in `sink_config.postgres`
- [ ] Sensible defaults for common use cases
- [ ] Support for connection string or individual parameters

Configuration options:

```python
@dataclass
class PostgresSinkConfig:
    # Connection settings (either dsn OR individual params)
    dsn: str | None = None                    # e.g., postgresql://user:pass@host/db
    host: str = "localhost"
    port: int = 5432
    database: str = "fapilog"
    user: str = "fapilog"
    password: str | None = None

    # Table settings
    table_name: str = "logs"
    schema_name: str = "public"
    create_table: bool = True                 # Auto-create table if missing

    # Connection pool settings
    min_pool_size: int = 2
    max_pool_size: int = 10
    pool_acquire_timeout: float = 10.0

    # Batching settings
    batch_size: int = 100
    batch_timeout_seconds: float = 5.0

    # Reliability settings
    max_retries: int = 3
    retry_base_delay: float = 0.5
    circuit_breaker_enabled: bool = True
    circuit_breaker_threshold: int = 5

    # Storage options
    use_jsonb: bool = True                    # Use JSONB vs JSON column
    include_raw_json: bool = True             # Store full event as JSON
    extract_fields: list[str] = field(        # Fields to extract as columns
        default_factory=lambda: ["level", "timestamp", "logger", "correlation_id"]
    )
```

### AC3: Schema Design

- [ ] Default table schema with indexed columns
- [ ] JSONB column for full event storage
- [ ] Extracted columns for common query patterns
- [ ] Timestamp-based indexing for time-range queries
- [ ] Optional: TimescaleDB hypertable support

Default schema:

```sql
CREATE TABLE IF NOT EXISTS {schema}.{table} (
    id BIGSERIAL PRIMARY KEY,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    timestamp TIMESTAMPTZ,
    level VARCHAR(10),
    logger VARCHAR(255),
    correlation_id VARCHAR(64),
    message TEXT,
    event JSONB NOT NULL
);

CREATE INDEX IF NOT EXISTS idx_{table}_timestamp ON {schema}.{table} (timestamp DESC);
CREATE INDEX IF NOT EXISTS idx_{table}_level ON {schema}.{table} (level);
CREATE INDEX IF NOT EXISTS idx_{table}_correlation_id ON {schema}.{table} (correlation_id);
CREATE INDEX IF NOT EXISTS idx_{table}_event_gin ON {schema}.{table} USING GIN (event);
```

### AC4: Testing

- [ ] Unit tests with mocked asyncpg connections
- [ ] Integration tests with PostgreSQL in CI (Docker service)
- [ ] Tests cover: table creation, bulk inserts, connection pooling, error handling
- [ ] Tests verify JSONB storage and queryability
- [ ] GitHub Actions workflow with PostgreSQL service

### AC5: Error Handling

- [ ] Handle connection failures with automatic reconnection
- [ ] Handle pool exhaustion gracefully (queue or drop)
- [ ] Handle constraint violations (log and continue)
- [ ] Handle serialization errors (malformed data)
- [ ] Handle PostgreSQL-specific errors (deadlocks, timeouts)
- [ ] Emit diagnostics for all error conditions

### AC6: Registration and Discovery

- [ ] Register as `postgres` in `fapilog.sinks` entry point group
- [ ] Loadable via `core.sinks = ["postgres"]`
- [ ] `PLUGIN_METADATA` with dependencies declared
- [ ] Optional dependency: `fapilog[postgres]` installs asyncpg

### AC7: Documentation

- [ ] `docs/plugins/sinks/postgres.md` — Full reference documentation
- [ ] Configuration examples (env vars, Settings object, programmatic)
- [ ] Schema customization guide
- [ ] Query patterns for log analysis
- [ ] Performance tuning recommendations
- [ ] Link to "Building Database Sinks" pattern guide

### AC8: Example Application

- [ ] Add `examples/postgres_logging/` with complete working example
- [ ] Example includes: FastAPI app, Docker Compose with PostgreSQL, README
- [ ] Include sample queries for log analysis

---

## Technical Design

### 1. File Structure

```
src/fapilog/plugins/sinks/
├── contrib/
│   ├── __init__.py
│   ├── cloudwatch.py
│   ├── loki.py
│   └── postgres.py       # New
```

### 2. Implementation

```python
# src/fapilog/plugins/sinks/contrib/postgres.py

from __future__ import annotations

import asyncio
import json
import os
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any

from ....core import diagnostics
from ....core.circuit_breaker import SinkCircuitBreaker, SinkCircuitBreakerConfig
from ....core.serialization import SerializedView
from .._batching import BatchingMixin

# Lazy import asyncpg to avoid hard dependency
asyncpg: Any = None


def _ensure_asyncpg() -> None:
    global asyncpg
    if asyncpg is None:
        import asyncpg as _asyncpg
        asyncpg = _asyncpg


@dataclass
class PostgresSinkConfig:
    """Configuration for PostgreSQL sink."""

    # Connection settings
    dsn: str | None = field(
        default_factory=lambda: os.getenv("FAPILOG_POSTGRES__DSN")
    )
    host: str = field(
        default_factory=lambda: os.getenv("FAPILOG_POSTGRES__HOST", "localhost")
    )
    port: int = field(
        default_factory=lambda: int(os.getenv("FAPILOG_POSTGRES__PORT", "5432"))
    )
    database: str = field(
        default_factory=lambda: os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog")
    )
    user: str = field(
        default_factory=lambda: os.getenv("FAPILOG_POSTGRES__USER", "fapilog")
    )
    password: str | None = field(
        default_factory=lambda: os.getenv("FAPILOG_POSTGRES__PASSWORD")
    )

    # Table settings
    table_name: str = "logs"
    schema_name: str = "public"
    create_table: bool = True

    # Connection pool
    min_pool_size: int = 2
    max_pool_size: int = 10
    pool_acquire_timeout: float = 10.0

    # Batching
    batch_size: int = 100
    batch_timeout_seconds: float = 5.0

    # Reliability
    max_retries: int = 3
    retry_base_delay: float = 0.5
    circuit_breaker_enabled: bool = True
    circuit_breaker_threshold: int = 5

    # Storage options
    use_jsonb: bool = True
    include_raw_json: bool = True
    extract_fields: list[str] = field(
        default_factory=lambda: ["level", "timestamp", "logger", "correlation_id", "message"]
    )


class PostgresSink(BatchingMixin):
    """PostgreSQL sink with async batching and connection pooling.

    This sink demonstrates the database integration pattern:
    - Uses asyncpg for non-blocking database I/O
    - Manages connection pool for efficient resource usage
    - Supports bulk inserts for high throughput
    - Auto-creates schema with appropriate indexes
    - Stores structured JSON with extracted queryable columns

    Use this as a reference for building sinks for other databases
    (MySQL, MongoDB, ClickHouse, etc.).
    """

    name = "postgres"

    def __init__(
        self,
        config: PostgresSinkConfig | None = None,
        **kwargs: Any,
    ) -> None:
        if config is None:
            config = PostgresSinkConfig(**kwargs) if kwargs else PostgresSinkConfig()

        self._config = config
        self._pool: Any = None
        self._circuit_breaker: SinkCircuitBreaker | None = None
        self._table_created = False

        # Initialize batching
        self._init_batching(config.batch_size, config.batch_timeout_seconds)

    async def start(self) -> None:
        _ensure_asyncpg()

        # Build connection parameters
        if self._config.dsn:
            self._pool = await asyncpg.create_pool(
                dsn=self._config.dsn,
                min_size=self._config.min_pool_size,
                max_size=self._config.max_pool_size,
            )
        else:
            self._pool = await asyncpg.create_pool(
                host=self._config.host,
                port=self._config.port,
                database=self._config.database,
                user=self._config.user,
                password=self._config.password,
                min_size=self._config.min_pool_size,
                max_size=self._config.max_pool_size,
            )

        if self._config.create_table:
            await self._ensure_table()

        if self._config.circuit_breaker_enabled:
            self._circuit_breaker = SinkCircuitBreaker(
                self.name,
                SinkCircuitBreakerConfig(
                    enabled=True,
                    failure_threshold=self._config.circuit_breaker_threshold,
                ),
            )

        await self._start_batching()

    async def stop(self) -> None:
        await self._stop_batching()
        if self._pool:
            await self._pool.close()
            self._pool = None

    async def write(self, entry: dict[str, Any]) -> None:
        await self._enqueue_for_batch(entry)

    async def write_serialized(self, view: SerializedView) -> None:
        """Fast path for pre-serialized payloads."""
        try:
            data = json.loads(bytes(view.data).decode("utf-8"))
            await self._enqueue_for_batch(data)
        except Exception:
            pass  # Contain errors

    async def _send_batch(self, batch: list[dict[str, Any]]) -> None:
        if not batch or self._pool is None:
            return

        if self._circuit_breaker and not self._circuit_breaker.should_allow():
            diagnostics.warn(
                "postgres-sink",
                "circuit breaker open, dropping batch",
                batch_size=len(batch),
                _rate_limit_key="postgres-circuit-open",
            )
            return

        await self._insert_batch_with_retry(batch)

    async def _insert_batch_with_retry(self, batch: list[dict[str, Any]]) -> None:
        """Insert batch with retry logic."""
        for attempt in range(self._config.max_retries):
            try:
                await self._do_bulk_insert(batch)
                if self._circuit_breaker:
                    self._circuit_breaker.record_success()
                return
            except Exception as exc:
                if self._circuit_breaker:
                    self._circuit_breaker.record_failure()

                diagnostics.warn(
                    "postgres-sink",
                    "batch insert failed",
                    error=str(exc),
                    attempt=attempt + 1,
                    batch_size=len(batch),
                    _rate_limit_key="postgres-insert-error",
                )

                if attempt < self._config.max_retries - 1:
                    delay = self._config.retry_base_delay * (2 ** attempt)
                    await asyncio.sleep(delay)

    async def _do_bulk_insert(self, batch: list[dict[str, Any]]) -> None:
        """Execute bulk insert using COPY or multi-row INSERT."""
        schema = self._config.schema_name
        table = self._config.table_name
        json_type = "JSONB" if self._config.use_jsonb else "JSON"

        # Prepare rows
        rows = []
        for entry in batch:
            row = self._prepare_row(entry)
            rows.append(row)

        # Build column list
        columns = ["timestamp", "level", "logger", "correlation_id", "message", "event"]
        placeholders = ", ".join(f"${i+1}" for i in range(len(columns)))

        query = f"""
            INSERT INTO {schema}.{table} ({", ".join(columns)})
            VALUES ({placeholders})
        """

        async with self._pool.acquire() as conn:
            # Use executemany for batch insert
            await conn.executemany(query, rows)

    def _prepare_row(self, entry: dict[str, Any]) -> tuple:
        """Extract fields and prepare row tuple for insertion."""
        # Parse timestamp
        ts = entry.get("timestamp")
        if isinstance(ts, str):
            try:
                ts = datetime.fromisoformat(ts.replace("Z", "+00:00"))
            except Exception:
                ts = datetime.now(timezone.utc)
        elif not isinstance(ts, datetime):
            ts = datetime.now(timezone.utc)

        # Extract common fields
        level = entry.get("level", "INFO")
        logger = entry.get("logger", "root")
        correlation_id = entry.get("correlation_id")
        message = entry.get("message", "")

        # Serialize full event as JSON
        event_json = json.dumps(entry, default=str)

        return (ts, level, logger, correlation_id, message, event_json)

    async def _ensure_table(self) -> None:
        """Create table and indexes if they don't exist."""
        if self._table_created:
            return

        schema = self._config.schema_name
        table = self._config.table_name
        json_type = "JSONB" if self._config.use_jsonb else "JSON"

        create_table_sql = f"""
            CREATE TABLE IF NOT EXISTS {schema}.{table} (
                id BIGSERIAL PRIMARY KEY,
                created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
                timestamp TIMESTAMPTZ,
                level VARCHAR(10),
                logger VARCHAR(255),
                correlation_id VARCHAR(64),
                message TEXT,
                event {json_type} NOT NULL
            )
        """

        create_indexes_sql = [
            f"CREATE INDEX IF NOT EXISTS idx_{table}_timestamp ON {schema}.{table} (timestamp DESC)",
            f"CREATE INDEX IF NOT EXISTS idx_{table}_level ON {schema}.{table} (level)",
            f"CREATE INDEX IF NOT EXISTS idx_{table}_correlation_id ON {schema}.{table} (correlation_id) WHERE correlation_id IS NOT NULL",
        ]

        if self._config.use_jsonb:
            create_indexes_sql.append(
                f"CREATE INDEX IF NOT EXISTS idx_{table}_event_gin ON {schema}.{table} USING GIN (event)"
            )

        async with self._pool.acquire() as conn:
            await conn.execute(create_table_sql)
            for index_sql in create_indexes_sql:
                try:
                    await conn.execute(index_sql)
                except Exception:
                    pass  # Index may already exist

        self._table_created = True

    async def health_check(self) -> bool:
        if not self._pool:
            return False

        if self._circuit_breaker and self._circuit_breaker.is_open:
            return False

        try:
            async with self._pool.acquire(timeout=5.0) as conn:
                await conn.execute("SELECT 1")
            return True
        except Exception:
            return False

    async def flush(self) -> None:
        """Flush any pending batched events."""
        await self._flush_batching()


PLUGIN_METADATA = {
    "name": "postgres",
    "version": "1.0.0",
    "plugin_type": "sink",
    "entry_point": "fapilog.plugins.sinks.contrib.postgres:PostgresSink",
    "description": "PostgreSQL sink with async batching and connection pooling.",
    "author": "Fapilog Core",
    "compatibility": {"min_fapilog_version": "0.4.0"},
    "api_version": "1.0",
    "dependencies": ["asyncpg>=0.28.0"],
}
```

### 3. Settings Integration

```python
# In src/fapilog/core/settings.py

class PostgresSinkSettings(BaseModel):
    """Configuration for PostgreSQL sink."""

    dsn: str | None = Field(default=None, description="PostgreSQL connection string")
    host: str = Field(default="localhost", description="Database host")
    port: int = Field(default=5432, description="Database port")
    database: str = Field(default="fapilog", description="Database name")
    user: str = Field(default="fapilog", description="Database user")
    password: str | None = Field(default=None, description="Database password")
    table_name: str = Field(default="logs", description="Target table name")
    schema_name: str = Field(default="public", description="Database schema")
    create_table: bool = Field(default=True, description="Auto-create table")
    min_pool_size: int = Field(default=2, ge=1, description="Minimum pool connections")
    max_pool_size: int = Field(default=10, ge=1, description="Maximum pool connections")
    batch_size: int = Field(default=100, ge=1, description="Events per batch")
    batch_timeout_seconds: float = Field(default=5.0, gt=0, description="Batch timeout")
    use_jsonb: bool = Field(default=True, description="Use JSONB column type")
    circuit_breaker_enabled: bool = Field(default=True, description="Enable circuit breaker")
    circuit_breaker_threshold: int = Field(default=5, ge=1, description="Failures before open")


class SinkConfig(BaseModel):
    # ... existing sinks ...
    postgres: PostgresSinkSettings = Field(
        default_factory=PostgresSinkSettings,
        description="Configuration for PostgreSQL sink",
    )
```

### 4. CI Testing Workflow

```yaml
# .github/workflows/test-postgres-sink.yml
name: PostgreSQL Sink Tests

on:
  push:
    paths:
      - "src/fapilog/plugins/sinks/contrib/postgres.py"
      - "tests/integration/test_postgres_*.py"
  pull_request:
    paths:
      - "src/fapilog/plugins/sinks/contrib/postgres.py"
      - "tests/integration/test_postgres_*.py"

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        ports:
          - 5432:5432
        env:
          POSTGRES_USER: fapilog
          POSTGRES_PASSWORD: fapilog
          POSTGRES_DB: fapilog_test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: |
          pip install -e ".[postgres,dev]"

      - name: Run PostgreSQL tests
        run: |
          pytest tests/integration/test_postgres_sink.py -v
        env:
          FAPILOG_POSTGRES__HOST: localhost
          FAPILOG_POSTGRES__PORT: 5432
          FAPILOG_POSTGRES__DATABASE: fapilog_test
          FAPILOG_POSTGRES__USER: fapilog
          FAPILOG_POSTGRES__PASSWORD: fapilog
```

### 5. Test Implementation

```python
# tests/integration/test_postgres_sink.py

import os
import pytest
import asyncio

# Skip if asyncpg not available
pytest.importorskip("asyncpg")

import asyncpg


@pytest.fixture
async def postgres_pool():
    """Create a test connection pool."""
    pool = await asyncpg.create_pool(
        host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
        port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
        database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
        user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
        password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
    )
    yield pool
    await pool.close()


@pytest.fixture
async def clean_table(postgres_pool):
    """Clean test table before each test."""
    async with postgres_pool.acquire() as conn:
        await conn.execute("DROP TABLE IF EXISTS public.test_logs")
    yield
    async with postgres_pool.acquire() as conn:
        await conn.execute("DROP TABLE IF EXISTS public.test_logs")


class TestPostgresSink:
    async def test_creates_table_on_start(self, postgres_pool, clean_table):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(
            host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
            port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
            database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
            user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
            password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
            table_name="test_logs",
            create_table=True,
        )
        sink = PostgresSink(config)
        await sink.start()

        # Verify table exists
        async with postgres_pool.acquire() as conn:
            result = await conn.fetchval(
                "SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'test_logs')"
            )
            assert result is True

        await sink.stop()

    async def test_writes_logs_successfully(self, postgres_pool, clean_table):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(
            host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
            port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
            database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
            user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
            password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
            table_name="test_logs",
            batch_size=1,  # Flush immediately for testing
        )
        sink = PostgresSink(config)
        await sink.start()

        await sink.write({
            "level": "INFO",
            "message": "test log message",
            "timestamp": "2024-01-15T10:30:00Z",
            "correlation_id": "test-123",
        })

        # Wait for batch flush
        await asyncio.sleep(0.1)
        await sink.stop()

        # Verify log was written
        async with postgres_pool.acquire() as conn:
            row = await conn.fetchrow("SELECT * FROM public.test_logs LIMIT 1")
            assert row is not None
            assert row["level"] == "INFO"
            assert row["message"] == "test log message"
            assert row["correlation_id"] == "test-123"

    async def test_batch_insert_performance(self, postgres_pool, clean_table):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(
            host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
            port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
            database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
            user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
            password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
            table_name="test_logs",
            batch_size=100,
        )
        sink = PostgresSink(config)
        await sink.start()

        # Write 500 logs
        for i in range(500):
            await sink.write({
                "level": "INFO",
                "message": f"test log {i}",
                "timestamp": "2024-01-15T10:30:00Z",
            })

        await sink.stop()

        # Verify all logs written
        async with postgres_pool.acquire() as conn:
            count = await conn.fetchval("SELECT COUNT(*) FROM public.test_logs")
            assert count == 500

    async def test_jsonb_queryable(self, postgres_pool, clean_table):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(
            host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
            port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
            database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
            user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
            password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
            table_name="test_logs",
            batch_size=1,
            use_jsonb=True,
        )
        sink = PostgresSink(config)
        await sink.start()

        await sink.write({
            "level": "ERROR",
            "message": "database error",
            "error_code": 500,
            "metadata": {"service": "api", "version": "1.2.3"},
        })

        await asyncio.sleep(0.1)
        await sink.stop()

        # Query using JSONB operators
        async with postgres_pool.acquire() as conn:
            row = await conn.fetchrow(
                "SELECT * FROM public.test_logs WHERE event->>'error_code' = '500'"
            )
            assert row is not None

            row = await conn.fetchrow(
                "SELECT * FROM public.test_logs WHERE event->'metadata'->>'service' = 'api'"
            )
            assert row is not None

    async def test_health_check_returns_true_when_healthy(self, clean_table):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(
            host=os.getenv("FAPILOG_POSTGRES__HOST", "localhost"),
            port=int(os.getenv("FAPILOG_POSTGRES__PORT", "5432")),
            database=os.getenv("FAPILOG_POSTGRES__DATABASE", "fapilog_test"),
            user=os.getenv("FAPILOG_POSTGRES__USER", "fapilog"),
            password=os.getenv("FAPILOG_POSTGRES__PASSWORD", "fapilog"),
            table_name="test_logs",
        )
        sink = PostgresSink(config)
        await sink.start()

        assert await sink.health_check() is True

        await sink.stop()

    async def test_health_check_returns_false_when_stopped(self):
        from fapilog.plugins.sinks.contrib.postgres import (
            PostgresSink,
            PostgresSinkConfig,
        )

        config = PostgresSinkConfig(table_name="test_logs")
        sink = PostgresSink(config)

        # Not started
        assert await sink.health_check() is False
```

---

## Documentation

### docs/plugins/sinks/postgres.md

````markdown
# PostgreSQL Sink

Store structured logs in PostgreSQL with async batching and queryable JSON storage.

## Installation

```bash
pip install fapilog[postgres]
```
````

## Quick Start

```python
import fapilog

# Configure via environment
# FAPILOG_POSTGRES__HOST=localhost
# FAPILOG_POSTGRES__DATABASE=myapp
# FAPILOG_POSTGRES__USER=logger
# FAPILOG_POSTGRES__PASSWORD=secret

with fapilog.runtime() as logger:
    logger.info("Hello PostgreSQL!")
```

## Configuration

### Environment Variables

| Variable                         | Description            | Default   |
| -------------------------------- | ---------------------- | --------- |
| `FAPILOG_POSTGRES__DSN`          | Full connection string | None      |
| `FAPILOG_POSTGRES__HOST`         | Database host          | localhost |
| `FAPILOG_POSTGRES__PORT`         | Database port          | 5432      |
| `FAPILOG_POSTGRES__DATABASE`     | Database name          | fapilog   |
| `FAPILOG_POSTGRES__USER`         | Database user          | fapilog   |
| `FAPILOG_POSTGRES__PASSWORD`     | Database password      | None      |
| `FAPILOG_POSTGRES__TABLE_NAME`   | Target table           | logs      |
| `FAPILOG_POSTGRES__BATCH_SIZE`   | Events per batch       | 100       |
| `FAPILOG_POSTGRES__CREATE_TABLE` | Auto-create table      | true      |
| `FAPILOG_POSTGRES__USE_JSONB`    | Use JSONB column       | true      |

### Programmatic Configuration

```python
from fapilog.plugins.sinks.contrib.postgres import PostgresSink, PostgresSinkConfig

config = PostgresSinkConfig(
    dsn="postgresql://user:pass@localhost/myapp",
    table_name="application_logs",
    batch_size=200,
    use_jsonb=True,
)

sink = PostgresSink(config)
```

## Schema

The sink auto-creates a table with this schema:

```sql
CREATE TABLE IF NOT EXISTS public.logs (
    id BIGSERIAL PRIMARY KEY,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    timestamp TIMESTAMPTZ,
    level VARCHAR(10),
    logger VARCHAR(255),
    correlation_id VARCHAR(64),
    message TEXT,
    event JSONB NOT NULL
);
```

### Indexes

- `timestamp DESC` — Time-range queries
- `level` — Filter by severity
- `correlation_id` — Request tracing
- `event` (GIN) — JSONB field queries

## Querying Logs

### By Time Range

```sql
SELECT * FROM logs
WHERE timestamp BETWEEN '2024-01-01' AND '2024-01-02'
ORDER BY timestamp DESC
LIMIT 100;
```

### By Level

```sql
SELECT * FROM logs
WHERE level IN ('ERROR', 'CRITICAL')
ORDER BY timestamp DESC;
```

### By Correlation ID

```sql
SELECT * FROM logs
WHERE correlation_id = 'req-abc123'
ORDER BY timestamp;
```

### JSONB Queries

```sql
-- Find logs with specific error code
SELECT * FROM logs
WHERE event->>'error_code' = '500';

-- Find logs from specific service
SELECT * FROM logs
WHERE event->'metadata'->>'service' = 'api';

-- Full-text search in message
SELECT * FROM logs
WHERE event->>'message' ILIKE '%connection%';
```

## Performance Tuning

### Batch Size

Increase batch size for high-throughput scenarios:

```bash
export FAPILOG_POSTGRES__BATCH_SIZE=500
```

### Connection Pool

Adjust pool size based on concurrent writers:

```python
config = PostgresSinkConfig(
    min_pool_size=5,
    max_pool_size=20,
)
```

### Table Partitioning

For very high volume, consider time-based partitioning:

```sql
-- Using pg_partman or TimescaleDB
SELECT create_hypertable('logs', 'timestamp');
```

## Troubleshooting

### Connection Refused

Verify PostgreSQL is running and accessible:

```bash
psql -h localhost -U fapilog -d fapilog -c "SELECT 1"
```

### Pool Exhaustion

Increase `max_pool_size` or reduce concurrent logger instances.

### Slow Inserts

- Increase `batch_size`
- Check index bloat
- Consider partitioning

## Building Similar Sinks

This sink demonstrates the **database integration pattern**:

1. Use async database drivers (asyncpg, aiomysql, motor)
2. Manage connection pools for efficiency
3. Support bulk inserts for throughput
4. Auto-create schema with appropriate indexes
5. Store full event as JSON for flexibility

See [Building Database Sinks](../patterns/database-sinks.md) for a detailed guide.

```

---

## Example Application

### examples/postgres_logging/

```

examples/postgres_logging/
├── README.md
├── docker-compose.yml
├── main.py
├── queries.sql
└── requirements.txt

````

**docker-compose.yml:**

```yaml
version: '3.8'
services:
  postgres:
    image: postgres:15
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: fapilog
      POSTGRES_PASSWORD: fapilog
      POSTGRES_DB: fapilog
    volumes:
      - postgres_data:/var/lib/postgresql/data

  app:
    build: .
    depends_on:
      - postgres
    environment:
      FAPILOG_POSTGRES__HOST: postgres
      FAPILOG_POSTGRES__DATABASE: fapilog
      FAPILOG_POSTGRES__USER: fapilog
      FAPILOG_POSTGRES__PASSWORD: fapilog
      FAPILOG_CORE__SINKS: '["postgres"]'

volumes:
  postgres_data:
````

**main.py:**

```python
"""Example FastAPI app logging to PostgreSQL."""
import os
from fastapi import FastAPI, Depends
import fapilog
from fapilog import get_async_logger

app = FastAPI()


async def get_logger():
    return await get_async_logger("api")


@app.get("/")
async def root(logger=Depends(get_logger)):
    await logger.info("Request received", path="/", method="GET")
    return {"status": "ok"}


@app.get("/users/{user_id}")
async def get_user(user_id: int, logger=Depends(get_logger)):
    await logger.info("User lookup", user_id=user_id)
    return {"user_id": user_id, "name": "Example User"}
```

**queries.sql:**

```sql
-- Recent errors
SELECT timestamp, message, event->>'error_code' as code
FROM logs
WHERE level = 'ERROR'
ORDER BY timestamp DESC
LIMIT 20;

-- Request volume by hour
SELECT
    date_trunc('hour', timestamp) as hour,
    COUNT(*) as requests
FROM logs
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY 1
ORDER BY 1;

-- Slowest requests (if latency logged)
SELECT
    event->>'path' as path,
    AVG((event->>'latency_ms')::float) as avg_latency
FROM logs
WHERE event->>'latency_ms' IS NOT NULL
GROUP BY 1
ORDER BY 2 DESC
LIMIT 10;
```

---

## pyproject.toml Addition

```toml
[project.optional-dependencies]
# ... existing ...

# PostgreSQL sink
postgres = [
    "asyncpg>=0.28.0",
]

# All optional dependencies
all = [
    "fapilog[dev,docs,fastapi,http,metrics,system,mqtt,aws,postgres]",
]
```

---

## Success Metrics

- [ ] All PostgreSQL integration tests pass
- [ ] Documentation complete with query examples
- [ ] Example application works with Docker Compose
- [ ] No regression in existing functionality
- [ ] Health check correctly reports status
- [ ] Batch inserts achieve >1000 events/second

---

## Related Stories

- Story 5.5: Cloud Sink Examples (pattern reference)
- Story 5.13: SizeGuardProcessor (recommended for large payloads)
- Story 5.14: CloudWatch Sink (SDK-based pattern counterpart)
- Story 5.15: Loki Sink (HTTP-based pattern counterpart)

---

## Future Considerations

- **TimescaleDB support** — Native hypertable creation for time-series optimization
- **Read API** — Query logs via fapilog (for audit/compliance use cases)
- **MySQL sink** — Similar pattern with `aiomysql` driver
- **MongoDB sink** — Document-based variant with `motor` driver
- **ClickHouse sink** — Column-oriented variant for analytics
