# Story 4.16: SealedSink Wrapper and Manifest Generation

## Status

Ready

## Story

**As a** compliance auditor,  
**I want** log files to include signed manifests with chain roots,  
**So that** I can verify file integrity offline and detect truncation or replacement of rotated files.

## Relationship to Other Stories

- **Depends on 4.14**: Uses package types and canonicalization
- **Depends on 4.15**: Receives chain state from IntegrityEnricher
- **Prerequisite for 4.17**: Manifests are input for verification

## Background

The SealedSink is a sink wrapper that:
1. Wraps any existing sink (typically `RotatingFileSink`)
2. Tracks file-level metadata (record count, timestamps, chain roots)
3. Generates signed manifests on file rotation
4. Optionally compresses rotated files

This provides cryptographic proof material for offline verification.

## Acceptance Criteria

### AC1: SealedSink Protocol Compliance

- [ ] Implements `BaseSink` protocol: `start()`, `stop()`, `write(entry)`
- [ ] Wraps an inner sink transparently
- [ ] Delegates `start()` and `stop()` to inner sink
- [ ] Intercepts `write()` to track metadata

### AC2: Metadata Tracking

- [ ] Track per-file:
  - `record_count`: Number of records written
  - `first_seq`: Sequence number of first record
  - `last_seq`: Sequence number of last record
  - `first_ts`: Timestamp of first record
  - `last_ts`: Timestamp of last record
  - `root_chain_hash`: Chain hash of last record (file root)
- [ ] Reset tracking on rotation
- [ ] Thread-safe updates

### AC3: Rotation Detection

- [ ] Detect rotation via inner sink callback or file change
- [ ] Support explicit `rotate()` method for manual trigger
- [ ] Emit manifest before starting new file
- [ ] Handle rotation errors gracefully (warn, don't crash)

### AC4: Manifest Generation

- [ ] Manifest format (JSON):
  ```json
  {
    "version": "1.0",
    "file": "fapilog-20250101-120000.jsonl",
    "created_ts": "2025-01-01T12:00:00Z",
    "closed_ts": "2025-01-01T13:00:00Z",
    "record_count": 1234,
    "first_seq": 1,
    "last_seq": 1234,
    "first_ts": "2025-01-01T12:00:00.123Z",
    "last_ts": "2025-01-01T12:59:59.987Z",
    "root_chain_hash": "base64url...",
    "algo": "HMAC-SHA256",
    "key_id": "audit-key-2025Q1",
    "signature_algo": "HMAC-SHA256",
    "signature": "base64url...",
    "integrity_version": "1.0"
  }
  ```
- [ ] Manifest written to `{file}.manifest.json`
- [ ] Signature computed over canonical manifest (excluding signature field)

### AC5: Manifest Signing

- [ ] Sign with same key as record MACs
- [ ] Algorithm matches `TamperConfig.algorithm`
- [ ] Signature covers all manifest fields except `signature`
- [ ] Verification: recompute signature, compare

### AC6: Optional Compression

- [ ] When `compress_rotated=True`:
  - Gzip data file to `{file}.gz`
  - Include manifest in gzip or as sidecar
  - Delete uncompressed file after successful compression
- [ ] Atomic operations (temp file, fsync, rename)

### AC7: Fsync Controls

- [ ] `fsync_on_write`: Call fsync after each write (high durability)
- [ ] `fsync_on_rotate`: Call fsync before rotation (default: on)
- [ ] Configurable via `TamperConfig`

### AC8: Chain Continuity Options

- [ ] `rotate_chain=False` (default): Chain continues across files
- [ ] `rotate_chain=True`: Reset chain to genesis on each file
- [ ] Manifest includes `continues_from` field when chain is continuous

## Technical Design

### SealedSink Class

```python
@dataclass
class FileMetadata:
    """Tracks metadata for the current file."""
    filename: str
    created_ts: datetime
    record_count: int = 0
    first_seq: int | None = None
    last_seq: int | None = None
    first_ts: str | None = None
    last_ts: str | None = None
    root_chain_hash: bytes | None = None


class SealedSink:
    """Sink wrapper that generates signed manifests on rotation."""

    def __init__(
        self,
        inner_sink: BaseSink,
        config: TamperConfig,
        *,
        key: bytes | None = None,
    ):
        self._inner = inner_sink
        self._config = config
        self._key = key
        self._lock = asyncio.Lock()
        self._current_file: FileMetadata | None = None
        self._manifest_generator = ManifestGenerator(config, key)

    async def start(self) -> None:
        await self._inner.start()
        self._current_file = FileMetadata(
            filename=self._get_current_filename(),
            created_ts=datetime.utcnow(),
        )

    async def stop(self) -> None:
        # Generate final manifest before shutdown
        if self._current_file and self._current_file.record_count > 0:
            await self._emit_manifest()
        await self._inner.stop()

    async def write(self, entry: dict) -> None:
        async with self._lock:
            # Track metadata
            integrity = entry.get("integrity", {})
            seq = integrity.get("seq")
            chain_hash = integrity.get("chain_hash")
            ts = entry.get("timestamp")

            if self._current_file.first_seq is None:
                self._current_file.first_seq = seq
                self._current_file.first_ts = ts

            self._current_file.last_seq = seq
            self._current_file.last_ts = ts
            self._current_file.record_count += 1
            if chain_hash:
                self._current_file.root_chain_hash = b64url_decode(chain_hash)

        # Delegate to inner sink
        await self._inner.write(entry)

        if self._config.fsync_on_write:
            await self._fsync_current_file()

    async def rotate(self) -> None:
        """Manually trigger rotation and manifest generation."""
        async with self._lock:
            await self._emit_manifest()
            # Reset for new file
            self._current_file = FileMetadata(
                filename=self._get_current_filename(),
                created_ts=datetime.utcnow(),
            )

    async def _emit_manifest(self) -> None:
        """Generate and write manifest for current file."""
        if not self._current_file:
            return

        manifest = await self._manifest_generator.generate(
            self._current_file,
            closed_ts=datetime.utcnow(),
        )
        manifest_path = Path(self._current_file.filename + ".manifest.json")
        await asyncio.to_thread(manifest_path.write_text, json.dumps(manifest, indent=2))

        if self._config.compress_rotated:
            await self._compress_file(self._current_file.filename)
```

### ManifestGenerator Class

```python
class ManifestGenerator:
    """Generates signed manifests for rotated files."""

    def __init__(self, config: TamperConfig, key: bytes | None):
        self._config = config
        self._key = key

    async def generate(self, metadata: FileMetadata, closed_ts: datetime) -> dict:
        manifest = {
            "version": "1.0",
            "file": metadata.filename,
            "created_ts": metadata.created_ts.isoformat() + "Z",
            "closed_ts": closed_ts.isoformat() + "Z",
            "record_count": metadata.record_count,
            "first_seq": metadata.first_seq,
            "last_seq": metadata.last_seq,
            "first_ts": metadata.first_ts,
            "last_ts": metadata.last_ts,
            "root_chain_hash": b64url_encode(metadata.root_chain_hash) if metadata.root_chain_hash else None,
            "algo": self._config.algorithm,
            "key_id": self._config.key_id,
            "signature_algo": self._config.algorithm,
            "integrity_version": "1.0",
        }

        # Sign manifest
        if self._key:
            payload = canonicalize(manifest)
            if self._config.algorithm == "HMAC-SHA256":
                sig = hmac.new(self._key, payload, hashlib.sha256).digest()
            else:  # Ed25519
                sig = self._signing_key.sign(payload).signature
            manifest["signature"] = b64url_encode(sig)

        return manifest
```

## Test Cases

| Test | Description |
|------|-------------|
| `test_sealed_sink_implements_protocol` | Satisfies `BaseSink` |
| `test_metadata_tracking` | `record_count`, `first_seq`, `last_seq` correct |
| `test_manifest_generated_on_stop` | Manifest file created on shutdown |
| `test_manifest_generated_on_rotate` | Manifest file created on manual rotate |
| `test_manifest_signature_valid` | Signature verifies with key |
| `test_manifest_signature_hmac` | HMAC-SHA256 signature correct |
| `test_manifest_signature_ed25519` | Ed25519 signature correct |
| `test_chain_hash_in_manifest` | `root_chain_hash` matches last record |
| `test_compression_enabled` | Rotated file compressed to .gz |
| `test_compression_manifest_sidecar` | Manifest preserved after compression |
| `test_fsync_on_write` | Fsync called per write when enabled |
| `test_fsync_on_rotate` | Fsync called before rotation |
| `test_rotate_chain_continuous` | Chain continues across files |
| `test_rotate_chain_reset` | Chain resets on each file |
| `test_concurrent_writes` | No race conditions under load |

## Out of Scope

- Verification of manifests (Story 4.17)
- KMS/Vault key sources (Story 4.18)
- Remote transport security (future story)

## Definition of Done

- [ ] `SealedSink` implements `BaseSink` protocol
- [ ] Wraps inner sink transparently
- [ ] Tracks file metadata correctly
- [ ] Generates manifest on rotation/stop
- [ ] Manifest signature verification works
- [ ] Compression option functional
- [ ] Fsync controls work as documented
- [ ] Chain continuity options work
- [ ] Unit tests pass with >90% coverage
- [ ] Integration test: write, rotate, verify manifest

## Estimation

**Story Points: 5**

## Labels

`enhancement`, `epic:enterprise-compliance`, `priority:high`, `status:ready`, `story`, `compliance`, `cryptography`

## Change Log

| Date       | Version | Description                              | Author   |
|------------|---------|------------------------------------------|----------|
| 2025-12-29 | 0.1     | Story created from addon design document | AI Agent |

