# AMSDAL Mail - Architecture Documentation

## Overview

`amsdal_mail` is a universal email integration plugin for the AMSDAL Framework, providing a unified interface for working with various email services. It follows the AMSDAL contrib plugin pattern, similar to `amsdal_storages`, offering multiple backend implementations through a common API.

## Design Philosophy

### Core Principles

1. **Backend Abstraction**: Single API for multiple email services
2. **Familiar API**: Django-like `send_mail()` API for ease of adoption
3. **Async-First**: Native support for both sync and async operations
4. **Minimal Dependencies**: Core functionality with optional backend-specific deps
5. **AMSDAL Integration**: Seamless integration with AMSDAL Framework lifecycle

### Comparison with Similar Projects

| Feature | amsdal_mail | Python email | aiosmtplib |
|---------|-------------|--------------|------------|
| Framework | AMSDAL | stdlib | None |
| Multiple Backends | ✅ | ❌ | ❌ |
| Async Support | ✅ | ❌ | ✅ |
| API Services | ✅ | ❌ | ❌ |
| Type Safety | ✅ (Pydantic) | ❌ | ❌ |

## Architecture Components

### 1. Core Abstractions

```
┌─────────────────────────────────────────┐
│         Public API Layer                │
│  send_mail() / asend_mail()             │
│  get_connection()                       │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│         Backend Registry                │
│  BACKENDS = {...}                       │
│  import_string()                        │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│      BaseEmailBackend (ABC)             │
│  + open() / close()                     │
│  + send_messages()                      │
│  + asend_messages()                     │
└─────────────────┬───────────────────────┘
                  │
        ┌─────────┴─────────┬─────────────┬──────────┐
        ▼                   ▼             ▼          ▼
┌───────────────┐  ┌─────────────┐  ┌──────────┐  ┌─────────┐
│ ConsoleBackend│  │ SMTPBackend │  │   SES    │  │  Dummy  │
│               │  │             │  │ Backend  │  │ Backend │
└───────────────┘  └─────────────┘  └──────────┘  └─────────┘
```

### 2. Data Models

#### EmailMessage

The central data model representing an email message:

```python
class EmailMessage(BaseModel):
    subject: str                           # Email subject
    body: str                             # Plain text body
    from_email: EmailStr | str            # Sender address
    to: list[EmailStr | str]              # Primary recipients
    cc: list[EmailStr | str]              # Carbon copy recipients
    bcc: list[EmailStr | str]             # Blind carbon copy recipients
    reply_to: list[EmailStr | str]        # Reply-to addresses
    attachments: list[Attachment]         # File attachments
    headers: dict[str, str]               # Custom headers
    html_body: str | None                 # HTML version of body
    tags: list[str]                       # Message tags for categorization
    metadata: dict[str, str]              # Custom key-value metadata for tracking
    template_id: str | None               # ESP template ID
    merge_data: dict | None               # Per-recipient template variables
    merge_global_data: dict | None        # Global template variables
    track_opens: bool                     # Enable open tracking
    track_clicks: bool                    # Enable click tracking
```

**Design Decisions**:
- Uses Pydantic for validation and serialization
- `EmailStr` for email validation (via pydantic[email])
- Separate `body` and `html_body` for multi-part messages
- List fields default to empty lists (not None) for convenience
- `extra='allow'` for backend-specific extensions

#### Attachment

```python
class Attachment(BaseModel):
    filename: str                         # Attachment filename
    content: bytes                        # File content
    mimetype: str                         # MIME type (e.g., 'application/pdf')
    content_id: str | None                # Content-ID for inline images (cid: URLs)
```

#### SendStatus

The return value from all send operations, providing detailed status information:

```python
class RecipientStatus(BaseModel):
    message_id: str | None               # ESP message ID for this recipient
    status: SendStatusType | None        # Send status ('sent', 'queued', 'failed', etc.)

class SendStatus(BaseModel):
    message_id: str | set[str] | None    # ESP message ID(s)
    status: set[SendStatusType] | None   # Set of statuses across all recipients
    recipients: dict[str, RecipientStatus]  # Per-recipient details
    esp_response: Any                    # Raw ESP API response
```

**Design Decisions**:
- `message_id` is simplified to `str` when all recipients share same ID (common case)
- `status` is always a `set` to represent multiple possible states
- `recipients` dict provides per-recipient granularity
- Helper methods (`is_success`, `has_failures`) for common checks
- Improved type safety via Pydantic models

**Status Types**:
- `sent` - ESP has accepted and queued the message
- `queued` - ESP will try to send later
- `invalid` - Recipient email address is not valid
- `rejected` - Recipient is blacklisted or rejected by ESP
- `failed` - Send attempt failed for some other reason
- `unknown` - Status could not be determined

### 3. Backend System

#### Base Backend Interface

```python
class BaseEmailBackend(ABC):
    """Abstract base class for all email backends."""

    def __init__(self, fail_silently: bool = False, **kwargs: Any) -> None:
        """
        Initialize backend.

        Args:
            fail_silently: If True, suppress exceptions
            **kwargs: Backend-specific configuration
        """
        self.fail_silently = fail_silently
        self.extra_kwargs = kwargs

    def open(self) -> None:
        """Open persistent connection (optional)."""
        pass

    def close(self) -> None:
        """Close persistent connection (optional)."""
        pass

    @abstractmethod
    def send_messages(self, email_messages: list['EmailMessage']) -> SendStatus:
        """
        Send one or more email messages.

        Returns:
            SendStatus object with message IDs, statuses, and ESP response
        """
        raise NotImplementedError()

    async def asend_messages(self, email_messages: list['EmailMessage']) -> SendStatus:
        """Async version of send_messages."""
        return self.send_messages(email_messages)
```

**Context Manager Support**:
```python
def __enter__(self):
    self.open()
    return self

def __exit__(self, exc_type, exc_value, traceback) -> None:
    self.close()
```

#### Backend Registry

Dynamic backend loading system:

```python
BACKENDS: dict[str, str] = {
    'console': 'amsdal_mail.backends.console.ConsoleBackend',
    'dummy': 'amsdal_mail.backends.dummy.DummyBackend',
    'ses': 'amsdal_mail.backends.ses.SESBackend',
    'smtp': 'amsdal_mail.backends.smtp.SMTPBackend',
}
```

**Backend Resolution**:
1. Check `backend` parameter
2. Fallback to `AMSDAL_EMAIL_BACKEND` env var
3. Default to `'console'` for development

**Import Strategy**:
```python
def import_string(dotted_path: str) -> Any:
    """Import class from dotted module path."""
    module_path, class_name = dotted_path.rsplit('.', 1)
    module = importlib.import_module(module_path)
    return getattr(module, class_name)
```

### 4. Backend Implementations

#### Console Backend

**Purpose**: Development and debugging
**Protocol**: stdout
**Dependencies**: None

```python
class ConsoleBackend(BaseEmailBackend):
    def send_messages(self, email_messages: list) -> SendStatus:
        status = SendStatus()
        recipients = {}

        for message in email_messages:
            self.stream.write(f'Subject: {message.subject}\n')
            self.stream.write(f'From: {message.from_email}\n')
            self.stream.write(f'To: {", ".join(message.to)}\n')
            self.stream.write(f'Body:\n{message.body}\n')
            self.stream.write('-' * 40 + '\n')

            # Track recipients
            for recipient in message.recipients():
                recipients[recipient] = RecipientStatus(
                    message_id=None,
                    status='sent',
                )

        status.set_recipient_status(recipients)
        return status
```

#### SMTP Backend

**Purpose**: Generic SMTP servers (Gmail, Outlook, etc.)
**Protocol**: SMTP/SMTPS/STARTTLS
**Dependencies**: `aiosmtplib` (for async)

**Configuration**:
- `AMSDAL_EMAIL_HOST`: SMTP server hostname
- `AMSDAL_EMAIL_PORT`: Port (25, 587, 465)
- `AMSDAL_EMAIL_USER`: Username
- `AMSDAL_EMAIL_PASSWORD`: Password
- `AMSDAL_EMAIL_USE_TLS`: Use STARTTLS
- `AMSDAL_EMAIL_USE_SSL`: Use SSL/TLS from start
- `AMSDAL_EMAIL_TIMEOUT`: Connection timeout

#### AWS SES Backend

**Purpose**: Amazon Simple Email Service
**Protocol**: AWS SDK (boto3)
**Dependencies**: `boto3`

**Features**:
- Configuration sets
- Message tags
- Template support

#### Dummy Backend

**Purpose**: Testing (no actual sending)
**Protocol**: None

```python
class DummyBackend(BaseEmailBackend):
    def send_messages(self, email_messages: list) -> SendStatus:
        status = SendStatus()
        recipients = {}

        for message in email_messages:
            for recipient in message.recipients():
                recipients[recipient] = RecipientStatus(
                    message_id=None,
                    status='sent',
                )

        status.set_recipient_status(recipients)
        return status  # Pretend all sent
```

### 5. AMSDAL Framework Integration

#### AppConfig

```python
from amsdal.contrib.app_config import AppConfig

class MailAppConfig(AppConfig):
    name = 'amsdal_mail'
    verbose_name = 'AMSDAL Email Integration'

    def on_setup(self) -> None:
        """Called when plugin is loaded."""
        from amsdal_mail.settings import mail_settings

        if mail_settings.WEBHOOK_ENABLED:
            self._setup_webhooks()

    @staticmethod
    def _setup_webhooks() -> None:
        from amsdal_utils.events import EventBus

        from amsdal_mail.webhooks.listener import WebhookRouterListener
        from amsdal_mail.webhooks.registry import WebhookRegistry
        from amsdal_mail.webhooks.ses import SESWebhookParser

        WebhookRegistry.register('ses', SESWebhookParser())
        EventBus.subscribe(RouterSetupEvent, WebhookRouterListener)
```

**Registration**:
```python
# In AMSDAL settings or .env
AMSDAL_CONTRIBS = 'amsdal_mail.app.MailAppConfig'
```

**Loading Process**:
1. AMSDAL framework reads `CONTRIBS` setting
2. Imports `MailAppConfig` class
3. Instantiates and calls `on_setup()`
4. Plugin is now available to application

### 6. Configuration System

#### Environment Variables

| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `AMSDAL_EMAIL_BACKEND` | str | `'console'` | Backend to use |
| `AMSDAL_EMAIL_HOST` | str | `'localhost'` | SMTP hostname |
| `AMSDAL_EMAIL_PORT` | int | `25` | SMTP port |
| `AMSDAL_EMAIL_USER` | str | `None` | SMTP username |
| `AMSDAL_EMAIL_PASSWORD` | str | `None` | SMTP password |
| `AMSDAL_EMAIL_USE_TLS` | bool | `False` | Use STARTTLS |
| `AMSDAL_EMAIL_USE_SSL` | bool | `False` | Use SSL/TLS |
| `AMSDAL_EMAIL_TIMEOUT` | int | `30` | Connection timeout (seconds) |
| `AWS_ACCESS_KEY_ID` | str | - | AWS access key |
| `AWS_SECRET_ACCESS_KEY` | str | - | AWS secret key |
| `AWS_REGION` | str | `'us-east-1'` | AWS region |
| `AMSDAL_MAIL_WEBHOOK_ENABLED` | bool | `False` | Enable webhook endpoints |
| `AMSDAL_MAIL_WEBHOOK_BASE_PATH` | str | `'/webhooks/mail'` | Webhook URL base path |
| `AMSDAL_MAIL_WEBHOOK_SES_SECRET` | str | `None` | Optional SES webhook secret |

#### Programmatic Configuration

```python
# Override via connection parameters
connection = get_connection(
    backend='smtp',
    host='smtp.example.com',
    port=587,
    username='user@example.com',
    password='secret',
    use_tls=True,
)
```

### 7. Error Handling

#### Exception Hierarchy

```python
class EmailError(Exception):
    """Base exception for email-related errors."""
    pass

class ConfigurationError(EmailError):
    """Raised when backend is misconfigured."""
    pass

class EmailConnectionError(EmailError):
    """Raised when connection fails."""
    pass

class SendError(EmailError):
    """Raised when message sending fails."""
    pass
```

#### Fail-Silently Mode

```python
# Suppress exceptions
send_mail(..., fail_silently=True)

# Or at backend level
connection = get_connection(fail_silently=True)
```

**Behavior**:
- `fail_silently=False` (default): Raise exceptions
- `fail_silently=True`: Mark recipients as 'failed' in SendStatus, log errors

### 8. Public API

#### High-Level Functions

```python
def send_mail(
    subject: str,
    message: str,
    from_email: str,
    recipient_list: list[str] | str,
    fail_silently: bool = False,
    html_message: str | None = None,
    connection = None,
    **kwargs,
) -> SendStatus:
    """
    Send a single email message.

    Returns:
        SendStatus object with message IDs, statuses, and ESP response
    """
```

```python
async def asend_mail(...) -> SendStatus:
    """Async version of send_mail."""
```

#### Low-Level API

```python
def get_connection(
    backend: str | None = None,
    fail_silently: bool = False,
    **kwargs,
):
    """
    Get email backend connection.

    Args:
        backend: Backend name or full path
        fail_silently: Suppress errors
        **kwargs: Backend-specific options

    Returns:
        Backend instance
    """
```

### 9. Testing Strategy

#### Test Backends

1. **ConsoleBackend**: Visual inspection of output
2. **DummyBackend**: Unit testing (no side effects)
3. **Mock External APIs**: For SES

#### Test Structure

```python
# tests/test_backends.py
def test_console_backend_output(capsys):
    backend = ConsoleBackend()
    message = EmailMessage(...)
    backend.send_messages([message])
    captured = capsys.readouterr()
    assert 'Subject:' in captured.out

# tests/test_message.py
def test_email_validation():
    with pytest.raises(ValidationError):
        EmailMessage(
            subject='Test',
            body='Body',
            from_email='invalid-email',  # Should fail
            to=['recipient@example.com'],
        )

# tests/test_api.py
def test_send_mail_integration(mocker):
    mock_backend = mocker.patch('amsdal_mail.backends.get_connection')
    send_mail('Subject', 'Message', 'from@example.com', ['to@example.com'])
    mock_backend.return_value.send_messages.assert_called_once()
```

### 10. Extension Points

#### Custom Backend

```python
from amsdal_mail.backends.base import BaseEmailBackend

class CustomBackend(BaseEmailBackend):
    def send_messages(self, email_messages: list) -> SendStatus:
        # Custom implementation
        pass
```

Register:
```python
from amsdal_mail.backends import BACKENDS
BACKENDS['custom'] = 'myapp.backends.CustomBackend'
```

Or use full path:
```python
connection = get_connection('myapp.backends.CustomBackend')
```

#### Custom Message Fields

```python
message = EmailMessage(
    subject='Test',
    body='Content',
    from_email='sender@example.com',
    to=['recipient@example.com'],
    # Extra fields (via extra='allow')
    custom_field='value',
    metadata={'key': 'value'},
)
```

### 11. Performance Considerations

#### Batch Sending

```python
# Efficient: Single connection for multiple messages
with get_connection() as conn:
    messages = [EmailMessage(...) for _ in range(100)]
    conn.send_messages(messages)
```

vs.

```python
# Inefficient: New connection per message
for message_data in data:
    send_mail(...)  # Opens/closes connection each time
```

#### Async Operations

```python
# Concurrent sending with async backend
async def send_bulk():
    tasks = [
        asend_mail(...)
        for _ in range(100)
    ]
    await asyncio.gather(*tasks)
```

### 12. Security Considerations

1. **Credential Storage**: Use environment variables, never hardcode
2. **TLS/SSL**: Enable for production SMTP connections
3. **API Key Rotation**: Support runtime credential updates
4. **Input Validation**: Pydantic validates all email addresses
5. **Header Injection**: Sanitize custom headers

### 13. Roadmap

#### Completed
- ✅ Core architecture and base abstractions
- ✅ Console, SMTP, Dummy backends
- ✅ AWS SES backend
- ✅ Template support (SES)
- ✅ Tags & Metadata (SES)
- ✅ Webhook handlers (SES)
- ✅ Email tracking events via EventBus
- ✅ Settings via pydantic_settings
- ✅ Inline images (CID) support

#### Planned
- SendGrid backend
- Mailgun backend
- Postmark backend
- Scheduled send
- Rate limiting
- Retry logic

## Summary

`amsdal_mail` provides a robust, extensible email integration layer for AMSDAL applications, following established patterns from the AMSDAL ecosystem. The architecture prioritizes simplicity, type safety, and async-first design.