# Story 4.18: Enterprise Key Management Integration

## Status

Ready

## Story

**As an** enterprise security team,  
**I want** tamper-evident logging to integrate with our existing key management infrastructure (KMS, Vault),  
**So that** we can use centrally managed, auditable keys without hardcoding secrets.

## Relationship to Other Stories

- **Depends on 4.14–4.17**: Extends key loading for all tamper-evident components
- **Extends core fapilog**: Aligns with `EncryptionSettings.key_source` patterns

## Background

Enterprise environments require:
1. Centralized key management (AWS KMS, GCP KMS, Azure Key Vault, HashiCorp Vault)
2. Key rotation without service restarts
3. Audit trails for key access
4. Separation of duties (ops manages keys, devs use keys)

This story extends `key_source` options beyond `env` and `file` to support enterprise KMS/Vault backends.

## Acceptance Criteria

### AC1: KMS Key Sources

- [ ] AWS KMS support:
  ```python
  key_source: Literal["aws-kms"]
  key_id: str  # KMS key ARN or alias
  ```
  - Use AWS SDK (`boto3`) for key operations
  - Support `GenerateDataKey` for local caching
  - Respect AWS credential chain (env, profile, IAM role)

- [ ] GCP KMS support:
  ```python
  key_source: Literal["gcp-kms"]
  key_id: str  # projects/{}/locations/{}/keyRings/{}/cryptoKeys/{}
  ```
  - Use Google Cloud SDK (`google-cloud-kms`)
  - Support Application Default Credentials

- [ ] Azure Key Vault support:
  ```python
  key_source: Literal["azure-keyvault"]
  key_id: str  # https://{vault}.vault.azure.net/keys/{key}
  ```
  - Use Azure SDK (`azure-keyvault-keys`)
  - Support managed identity and service principal auth

### AC2: HashiCorp Vault Support

- [ ] Vault Transit secrets engine:
  ```python
  key_source: Literal["vault"]
  vault_addr: str  # VAULT_ADDR
  vault_token: str | None  # VAULT_TOKEN (or use other auth)
  vault_path: str  # transit/keys/{key_name}
  ```
- [ ] Vault auth methods:
  - Token (env var or file)
  - AppRole
  - Kubernetes (for k8s pods)
- [ ] Key operations via Transit engine (sign/verify without exporting key)

### AC3: Key Provider Protocol

- [ ] Abstract `KeyProvider` protocol:
  ```python
  class KeyProvider(Protocol):
      async def get_key(self, key_id: str) -> bytes:
          """Retrieve key material by ID."""

      async def sign(self, key_id: str, data: bytes) -> bytes:
          """Sign data using key (for KMS that don't export keys)."""

      async def verify(self, key_id: str, data: bytes, signature: bytes) -> bool:
          """Verify signature using key."""

      async def rotate_check(self) -> bool:
          """Check if key has rotated and refresh cache."""
  ```
- [ ] Implementations: `EnvKeyProvider`, `FileKeyProvider`, `AwsKmsProvider`, `GcpKmsProvider`, `AzureKeyVaultProvider`, `VaultProvider`

### AC4: Key Caching and Rotation

- [ ] Local key cache with TTL:
  ```python
  cache_ttl_seconds: int = 300  # 5 minutes default
  ```
- [ ] Automatic refresh on cache expiry
- [ ] Rotation detection:
  - Check key version/alias on refresh
  - If key rotated, update `key_id` in integrity fields
  - Support multiple active keys for verification

### AC5: KMS-Native Signing (No Key Export)

- [ ] For KMS that don't export keys (AWS KMS, Vault Transit):
  - Call KMS `Sign` API instead of local HMAC
  - Trade latency for security (keys never leave KMS)
  - Configurable: `use_kms_signing: bool = False`
- [ ] Batch signing for performance:
  ```python
  async def sign_batch(self, key_id: str, items: list[bytes]) -> list[bytes]:
      """Sign multiple items in one API call if supported."""
  ```

### AC6: Configuration Extensions

- [ ] Extended `TamperConfig`:
  ```python
  class TamperConfig(BaseModel):
      # ... existing fields ...

      # KMS settings
      key_source: Literal["env", "file", "aws-kms", "gcp-kms", "azure-keyvault", "vault"] = "env"
      key_cache_ttl_seconds: int = 300
      use_kms_signing: bool = False

      # AWS KMS
      aws_region: str | None = None

      # Vault
      vault_addr: str | None = None
      vault_auth_method: Literal["token", "approle", "kubernetes"] = "token"
      vault_role: str | None = None  # For approle/k8s

      # Azure
      azure_tenant_id: str | None = None
      azure_client_id: str | None = None
  ```

### AC7: Optional Dependencies

- [ ] KMS dependencies are optional:
  ```toml
  [project.optional-dependencies]
  aws = ["boto3>=1.26.0"]
  gcp = ["google-cloud-kms>=2.0.0"]
  azure = ["azure-keyvault-keys>=4.0.0", "azure-identity>=1.0.0"]
  vault = ["hvac>=1.0.0"]
  all-kms = ["fapilog-tamper[aws,gcp,azure,vault]"]
  ```
- [ ] Graceful error if KMS dependency not installed:
  ```
  ImportError: AWS KMS support requires boto3. Install with: pip install fapilog-tamper[aws]
  ```

### AC8: Credential Security

- [ ] Never log credentials or key material
- [ ] Use secure credential loading (IAM roles, managed identity preferred)
- [ ] Clear sensitive data from memory after use (best effort)
- [ ] Warn if using hardcoded credentials

## Technical Design

### KeyProvider Implementations

```python
# fapilog_tamper/providers/aws_kms.py
class AwsKmsProvider:
    """AWS KMS key provider with local caching."""

    def __init__(
        self,
        key_id: str,
        region: str | None = None,
        cache_ttl: int = 300,
        use_kms_signing: bool = False,
    ):
        self._key_id = key_id
        self._cache_ttl = cache_ttl
        self._use_kms_signing = use_kms_signing
        self._cached_key: bytes | None = None
        self._cache_expires: float = 0
        self._client = boto3.client("kms", region_name=region)

    async def get_key(self, key_id: str) -> bytes:
        if self._use_kms_signing:
            raise NotImplementedError("KMS signing mode doesn't export keys")

        now = time.time()
        if self._cached_key and now < self._cache_expires:
            return self._cached_key

        # Generate data key (encrypted + plaintext)
        response = await asyncio.to_thread(
            self._client.generate_data_key,
            KeyId=self._key_id,
            KeySpec="AES_256",
        )
        self._cached_key = response["Plaintext"]
        self._cache_expires = now + self._cache_ttl
        return self._cached_key

    async def sign(self, key_id: str, data: bytes) -> bytes:
        if not self._use_kms_signing:
            key = await self.get_key(key_id)
            return hmac.new(key, data, hashlib.sha256).digest()

        # Use KMS Sign API
        response = await asyncio.to_thread(
            self._client.sign,
            KeyId=self._key_id,
            Message=data,
            MessageType="RAW",
            SigningAlgorithm="HMAC_SHA_256",
        )
        return response["Signature"]

    async def verify(self, key_id: str, data: bytes, signature: bytes) -> bool:
        if not self._use_kms_signing:
            key = await self.get_key(key_id)
            expected = hmac.new(key, data, hashlib.sha256).digest()
            return hmac.compare_digest(signature, expected)

        # Use KMS Verify API
        try:
            await asyncio.to_thread(
                self._client.verify,
                KeyId=self._key_id,
                Message=data,
                Signature=signature,
                MessageType="RAW",
                SigningAlgorithm="HMAC_SHA_256",
            )
            return True
        except self._client.exceptions.KMSInvalidSignatureException:
            return False
```

```python
# fapilog_tamper/providers/vault.py
class VaultProvider:
    """HashiCorp Vault Transit secrets engine provider."""

    def __init__(
        self,
        addr: str,
        key_name: str,
        auth_method: str = "token",
        token: str | None = None,
        role_id: str | None = None,
        secret_id: str | None = None,
    ):
        import hvac
        self._client = hvac.Client(url=addr)
        self._key_name = key_name
        self._authenticate(auth_method, token, role_id, secret_id)

    def _authenticate(self, method: str, token: str | None, role_id: str | None, secret_id: str | None):
        if method == "token":
            self._client.token = token or os.environ.get("VAULT_TOKEN")
        elif method == "approle":
            self._client.auth.approle.login(role_id=role_id, secret_id=secret_id)
        elif method == "kubernetes":
            jwt = Path("/var/run/secrets/kubernetes.io/serviceaccount/token").read_text()
            self._client.auth.kubernetes.login(role=role_id, jwt=jwt)

    async def sign(self, key_id: str, data: bytes) -> bytes:
        import base64
        b64_data = base64.b64encode(data).decode()
        response = await asyncio.to_thread(
            self._client.secrets.transit.sign_data,
            name=self._key_name,
            hash_input=b64_data,
            hash_algorithm="sha2-256",
            signature_algorithm="pkcs1v15",
        )
        sig_b64 = response["data"]["signature"].split(":")[-1]
        return base64.b64decode(sig_b64)

    async def verify(self, key_id: str, data: bytes, signature: bytes) -> bool:
        import base64
        b64_data = base64.b64encode(data).decode()
        b64_sig = f"vault:v1:{base64.b64encode(signature).decode()}"
        response = await asyncio.to_thread(
            self._client.secrets.transit.verify_signed_data,
            name=self._key_name,
            hash_input=b64_data,
            signature=b64_sig,
            hash_algorithm="sha2-256",
        )
        return response["data"]["valid"]
```

### Provider Factory

```python
def create_key_provider(config: TamperConfig) -> KeyProvider:
    """Create appropriate key provider based on config."""
    if config.key_source == "env":
        return EnvKeyProvider(config.key_env_var)
    elif config.key_source == "file":
        return FileKeyProvider(config.key_file_path)
    elif config.key_source == "aws-kms":
        try:
            from .providers.aws_kms import AwsKmsProvider
        except ImportError:
            raise ImportError(
                "AWS KMS support requires boto3. "
                "Install with: pip install fapilog-tamper[aws]"
            )
        return AwsKmsProvider(
            key_id=config.key_id,
            region=config.aws_region,
            cache_ttl=config.key_cache_ttl_seconds,
            use_kms_signing=config.use_kms_signing,
        )
    elif config.key_source == "vault":
        try:
            from .providers.vault import VaultProvider
        except ImportError:
            raise ImportError(
                "Vault support requires hvac. "
                "Install with: pip install fapilog-tamper[vault]"
            )
        return VaultProvider(
            addr=config.vault_addr or os.environ.get("VAULT_ADDR", ""),
            key_name=config.key_id,
            auth_method=config.vault_auth_method,
        )
    # ... similar for gcp-kms, azure-keyvault
    else:
        raise ValueError(f"Unknown key_source: {config.key_source}")
```

## Test Cases

| Test | Description |
|------|-------------|
| `test_aws_kms_provider_get_key` | Key retrieved from AWS KMS |
| `test_aws_kms_provider_sign` | Signing via KMS API |
| `test_aws_kms_provider_cache` | Key cached for TTL |
| `test_aws_kms_provider_rotation` | Cache refreshed on expiry |
| `test_gcp_kms_provider` | GCP KMS integration |
| `test_azure_keyvault_provider` | Azure Key Vault integration |
| `test_vault_provider_token_auth` | Vault with token auth |
| `test_vault_provider_approle` | Vault with AppRole auth |
| `test_vault_provider_kubernetes` | Vault with K8s auth |
| `test_vault_sign_verify` | Sign and verify via Transit |
| `test_provider_factory` | Correct provider created |
| `test_missing_dependency_error` | Clear error for missing SDK |
| `test_key_rotation_detection` | New key version detected |
| `test_multi_key_verification` | Verify with multiple keys |
| `test_no_credential_logging` | Credentials not in logs |

## Out of Scope

- Key generation (use KMS console/CLI)
- Key policy management
- Custom auth plugins
- HSM direct integration (use KMS as intermediary)

## Definition of Done

- [ ] `KeyProvider` protocol defined
- [ ] `AwsKmsProvider` implementation
- [ ] `GcpKmsProvider` implementation
- [ ] `AzureKeyVaultProvider` implementation
- [ ] `VaultProvider` implementation
- [ ] Provider factory with graceful dependency handling
- [ ] Key caching with TTL
- [ ] KMS-native signing option
- [ ] Optional dependencies in pyproject.toml
- [ ] Unit tests with mocked KMS APIs
- [ ] Integration tests with LocalStack/Vault dev mode
- [ ] Documentation for each KMS setup

## Estimation

**Story Points: 8**

## Labels

`enhancement`, `epic:enterprise-compliance`, `priority:medium`, `status:ready`, `story`, `compliance`, `security`, `enterprise`

## Change Log

| Date       | Version | Description                              | Author   |
|------------|---------|------------------------------------------|----------|
| 2025-12-29 | 0.1     | Story created from addon design document | AI Agent |

