# Story 4.15: IntegrityEnricher and ChainState Persistence

## Status

Ready

## Story

**As a** security engineer using fapilog-tamper,  
**I want** log events to be enriched with cryptographic MACs and hash chain linkage,  
**So that** I can detect any tampering, insertion, deletion, or reordering of log entries.

## Relationship to Other Stories

- **Depends on 4.14**: Uses package structure, types, and canonicalization
- **Prerequisite for 4.16**: Provides chain state for manifest generation
- **Implements chain integrity**: Full hash chain with keyed MACs (replaces cancelled 4.13)

## Background

The IntegrityEnricher is the write-path component that:
1. Computes a per-record MAC (HMAC-SHA256 or Ed25519 signature)
2. Maintains a forward hash chain linking records cryptographically
3. Injects integrity metadata into each event

ChainState persistence ensures the chain can be resumed after process restarts.

## Acceptance Criteria

### AC1: IntegrityEnricher Protocol Compliance

- [ ] Implements `BaseEnricher` protocol: `start()`, `stop()`, `enrich(event)`
- [ ] `start()` loads key material and recovers chain state
- [ ] `stop()` persists chain state and clears sensitive material from memory
- [ ] `enrich()` returns `{"integrity": IntegrityFields}` merged into event

### AC2: MAC Computation

- [ ] HMAC-SHA256 mode (default):
  ```python
  mac = hmac.new(key, canonical_payload, hashlib.sha256).digest()
  ```
- [ ] Ed25519 mode (optional, requires `pynacl`):
  ```python
  signature = signing_key.sign(canonical_payload).signature
  ```
- [ ] Algorithm selection via `TamperConfig.algorithm`
- [ ] MAC stored as base64url-encoded string

### AC3: Chain Hash Computation

- [ ] Forward hash chain formula:
  ```python
  chain_hash = SHA256(prev_chain_hash || mac || seq_bytes || timestamp_bytes)
  ```
- [ ] Genesis chain hash: 32 zero bytes (`b'\x00' * 32`)
- [ ] Sequence number: monotonic, starts at 1
- [ ] Timestamp: ISO 8601 UTC string, encoded as UTF-8

### AC4: Integrity Fields Injection

- [ ] Event enriched with:
  ```json
  {
    "integrity": {
      "seq": 42,
      "mac": "base64url...",
      "algo": "HMAC-SHA256",
      "key_id": "audit-key-2025Q1",
      "chain_hash": "base64url...",
      "prev_chain_hash": "base64url..."
    }
  }
  ```
- [ ] Original event fields preserved unchanged
- [ ] Integrity fields computed atomically (thread-safe)

### AC5: ChainState Persistence

- [ ] State file location: `{state_dir}/{stream_id}.chainstate`
- [ ] State format (JSON):
  ```json
  {
    "seq": 42,
    "prev_chain_hash": "base64url...",
    "key_id": "audit-key-2025Q1",
    "last_updated": "2025-12-29T12:00:00Z"
  }
  ```
- [ ] Atomic writes (write to temp, fsync, rename)
- [ ] Recovery on `start()`: read state file if exists
- [ ] Corruption handling: warn and reset to genesis if state unreadable

### AC6: Key Loading

- [ ] `key_source="env"`: Load from environment variable
- [ ] `key_source="file"`: Load from file path (binary or base64)
- [ ] Validate key length (32 bytes for HMAC-SHA256, 32 bytes for Ed25519 seed)
- [ ] Clear key from memory on `stop()` (best effort)
- [ ] Emit diagnostic warning if key not found (fallback: disabled)

### AC7: Thread/Async Safety

- [ ] Sequence increment protected by `asyncio.Lock`
- [ ] No race conditions under concurrent `enrich()` calls
- [ ] State persistence uses file locking or atomic operations

## Technical Design

### IntegrityEnricher Class

```python
class IntegrityEnricher:
    """Enricher that adds tamper-evident MAC and chain fields."""

    def __init__(self, config: TamperConfig, stream_id: str = "default"):
        self._config = config
        self._stream_id = stream_id
        self._lock = asyncio.Lock()
        self._key: bytes | None = None
        self._signing_key: SigningKey | None = None  # Ed25519
        self._state: ChainState | None = None
        self._persistence: ChainStatePersistence | None = None

    async def start(self) -> None:
        self._key = await self._load_key()
        self._persistence = ChainStatePersistence(
            state_dir=self._config.state_dir,
            stream_id=self._stream_id,
        )
        self._state = await self._persistence.load()

    async def stop(self) -> None:
        if self._state and self._persistence:
            await self._persistence.save(self._state)
        self._key = None  # Clear sensitive material

    async def enrich(self, event: dict) -> dict:
        if not self._config.enabled or not self._key:
            return {}

        async with self._lock:
            self._state.seq += 1
            seq = self._state.seq

            # Compute MAC
            payload = canonicalize(event)
            if self._config.algorithm == "HMAC-SHA256":
                mac = hmac.new(self._key, payload, hashlib.sha256).digest()
            else:  # Ed25519
                mac = self._signing_key.sign(payload).signature

            # Compute chain hash
            timestamp = event.get("timestamp", datetime.utcnow().isoformat())
            chain_input = (
                self._state.prev_chain_hash
                + mac
                + seq.to_bytes(8, "big")
                + timestamp.encode("utf-8")
            )
            chain_hash = hashlib.sha256(chain_input).digest()

            # Update state
            prev_chain_hash = self._state.prev_chain_hash
            self._state.prev_chain_hash = chain_hash

        return {
            "integrity": {
                "seq": seq,
                "mac": b64url_encode(mac),
                "algo": self._config.algorithm,
                "key_id": self._config.key_id,
                "chain_hash": b64url_encode(chain_hash),
                "prev_chain_hash": b64url_encode(prev_chain_hash),
            }
        }
```

### ChainStatePersistence Class

```python
class ChainStatePersistence:
    """Persists chain state to disk for restart recovery."""

    GENESIS_HASH = b'\x00' * 32

    def __init__(self, state_dir: str, stream_id: str):
        self._path = Path(state_dir) / f"{stream_id}.chainstate"

    async def load(self) -> ChainState:
        """Load state from disk or return genesis state."""
        if not self._path.exists():
            return ChainState(seq=0, prev_chain_hash=self.GENESIS_HASH, key_id="")
        try:
            data = json.loads(await asyncio.to_thread(self._path.read_text))
            return ChainState(
                seq=data["seq"],
                prev_chain_hash=b64url_decode(data["prev_chain_hash"]),
                key_id=data.get("key_id", ""),
            )
        except Exception as exc:
            # Warn and reset to genesis
            from fapilog.core import diagnostics
            diagnostics.warn("tamper", "chain state corrupt, resetting", error=str(exc))
            return ChainState(seq=0, prev_chain_hash=self.GENESIS_HASH, key_id="")

    async def save(self, state: ChainState) -> None:
        """Atomically persist state to disk."""
        data = {
            "seq": state.seq,
            "prev_chain_hash": b64url_encode(state.prev_chain_hash),
            "key_id": state.key_id,
            "last_updated": datetime.utcnow().isoformat() + "Z",
        }
        temp_path = self._path.with_suffix(".tmp")
        await asyncio.to_thread(temp_path.write_text, json.dumps(data))
        await asyncio.to_thread(os.fsync, temp_path.open().fileno())
        await asyncio.to_thread(temp_path.rename, self._path)
```

## Test Cases

| Test | Description |
|------|-------------|
| `test_enricher_implements_protocol` | Satisfies `BaseEnricher` |
| `test_hmac_computation` | MAC matches expected value for known input |
| `test_ed25519_signature` | Signature verifiable with public key |
| `test_chain_hash_linkage` | `events[n].chain_hash` computed from `events[n-1]` |
| `test_genesis_chain_hash` | First event uses 32 zero bytes as prev |
| `test_sequence_monotonic` | 100 events have sequences 1..100 |
| `test_concurrent_enrichment` | 50 concurrent enriches have unique sequences |
| `test_state_persistence_roundtrip` | Save then load returns same state |
| `test_state_recovery_on_restart` | Enricher resumes sequence after restart |
| `test_state_corruption_recovery` | Corrupt state file → reset to genesis + warning |
| `test_key_from_env` | Loads key from environment variable |
| `test_key_from_file` | Loads key from file path |
| `test_key_not_found_disabled` | Missing key → enricher disabled + warning |
| `test_thread_safety` | No race conditions under load |

## Out of Scope

- Sink wrapper and manifests (Story 4.16)
- Verification (Story 4.17)
- KMS/Vault key sources (Story 4.18)

## Definition of Done

- [ ] `IntegrityEnricher` implements `BaseEnricher` protocol
- [ ] HMAC-SHA256 MAC computation working
- [ ] Ed25519 signatures working (with optional dep)
- [ ] Chain hash formula implemented correctly
- [ ] `ChainStatePersistence` saves/loads state
- [ ] Atomic state file writes
- [ ] Key loading from env/file
- [ ] Thread-safe sequence management
- [ ] Unit tests pass with >90% coverage
- [ ] Integration test: write 1000 events, restart, verify chain continuity

## Estimation

**Story Points: 5**

## Labels

`enhancement`, `epic:enterprise-compliance`, `priority:high`, `status:ready`, `story`, `compliance`, `cryptography`

## Change Log

| Date       | Version | Description                              | Author   |
|------------|---------|------------------------------------------|----------|
| 2025-12-29 | 0.1     | Story created from addon design document | AI Agent |

