# Connectors — Shared Integration Components

## Overview

Connectors are reusable integration modules for connecting adapters to external systems. They live under `connectors/` as flat modules — one file per integration. Connectors handle protocol details (authentication, pagination, connection management) so handler code stays focused on business logic.

## Concepts

### Connector vs Handler

A **handler** implements business logic for a specific adapter identifier (validate → transform → send → respond). A **connector** implements protocol-level communication with an external system (authenticate, list, download, upload). Handlers consume connectors — connectors are never exposed as endpoints directly.

### Retrieval Connectors

Some connectors implement `RetrievalHandler` directly, meaning they can be injected into a `BaseHandler` as its `retrieval_handler`. These connectors handle the full document retrieval lifecycle (listing and downloading) and are called by Core's scheduler via the `/list_documents` and `/download_document` endpoints.

The package ships with two retrieval connectors:
- **Gmail** (`EntrMailboxRetrievalHandler`) — pulls emails from shared Gmail mailbox using OAuth2 service account credentials, with tenant isolation via Gmail labels
- **Microsoft Exchange** (`MicrosoftEntraRetrievalHandler`) — pulls emails from Exchange Online mailboxes using OAuth2 client credentials, with folder-based navigation

### Utility Connectors

Other connectors provide general-purpose file operations that handlers call directly within `process()`. These are not `RetrievalHandler` subclasses — they're standalone clients.

The package ships with two utility connectors:
- **SFTP** (`SFTPConnector`) — persistent connection with exponential backoff reconnection, asyncio-safe via locking
- **FTP** (`FTPConnector`) — stateless connections using stdlib ftplib, wrapped in asyncio thread pool

## Patterns

### Extras Mechanism

Connectors use Python extras so unused connectors add no install overhead:

```toml
# Install only what you need
"entr-adapter-core[gmail]>=1.0,<2"
"entr-adapter-core[sftp]>=1.0,<2"
"entr-adapter-core[connectors]>=1.0,<2"  # all
```

Available extras: `gmail`, `microsoft`, `sftp`, `connectors` (all).

### Promotion Rule

When a connector is needed by two or more adapters, it should be promoted from adapter-specific code to this package. The promotion flow:

1. **Extract** — Pull the connector out of the adapter into its own module
2. **Generalize** — Remove tenant-specific assumptions, make configuration injectable
3. **Document** — Add docstrings, a `SETUP_*.md`, and a `USERGUIDE_*.md` (see [Documentation Convention](#documentation-convention) below), and update this file
4. **Register extras** — Add connector-specific dependencies as an optional extra in `pyproject.toml`
5. **Test** — Add contract tests to the package test suite
6. **Release** — Publish as a minor version bump

Until a second adapter needs the same connector, keep it in the adapter's own `connectors/` directory. This mirrors the package structure, making future promotion straightforward.

### Version Bounds

Connector-specific dependencies use loose lower bounds (`>=`) to prevent conflicts with adapter-level dependencies. For example, `httpx>=0.24.0` rather than `httpx==0.28.1`.

### Connection Management

- **SFTP connector**: Maintains a persistent connection across requests. Uses asyncio.Lock to prevent thundering herd on reconnection. Auto-reconnects with exponential backoff (up to 3 retries). Call `close()` on shutdown.
- **FTP connector**: Creates a new connection per operation (stateless). No cleanup needed. Reliable with Windows FTP servers that lack modern extensions.
- **Gmail connector**: Stateless HTTP — exchanges refresh token for access token on each batch of requests.
- **Microsoft connector**: Caches access tokens with 5-minute safety margin before expiry. Stateless HTTP otherwise.

### Isolation Models

Retrieval connectors enforce tenant isolation:

- **Gmail**: Tenant name maps to Gmail label (`TENANT/{TENANT_NAME}`). Only messages carrying the tenant's label are visible.
- **Microsoft Exchange**: Isolation is per-mailbox and per-folder. The `mailbox` parameter targets a specific Exchange mailbox; the `folder_path` parameter navigates to a specific folder within it.

## Guidance

### Choosing Between Gmail and Microsoft

| Factor | Gmail | Microsoft Exchange |
|--------|-------|--------------------|
| Auth model | OAuth2 refresh token (granted by mailbox owner) | OAuth2 client credentials (service-to-service) |
| Isolation | Label-based (multiple tenants share one mailbox) | Mailbox-based (one mailbox per tenant) |
| Output format | RFC822 `.eml` files (base64-encoded) | MIME `.eml` files (base64-encoded) |
| Folder navigation | Labels only (no folder hierarchy) | Full folder path traversal |

### Choosing Between SFTP and FTP

| Factor | SFTP | FTP |
|--------|------|-----|
| Security | Encrypted channel (SSH) | Plaintext (use only on trusted networks) |
| Connection | Persistent with auto-reconnect | New connection per operation |
| Concurrency | Lock-protected reconnection | Naturally thread-safe (stateless) |
| Best for | Production file transfers | Legacy systems, Windows FTP servers |

### Adapter-Specific Connectors

For connectors that only one adapter uses, place them in the adapter's own `connectors/` directory:

```
my-adapter/
└── src/
    └── connectors/
        └── sap_rfc.py     # Adapter-specific — stays here until a second adapter needs it
```

This mirrors the package structure, so promoting to the package later is a copy + generalize operation.

## Documentation Convention

Each connector has up to three co-located documentation files:

| File | Purpose | Audience |
|------|---------|----------|
| `SETUP_<name>.md` | Technical setup — env vars, config.py, registration in main.py, verification | Developer implementing the connector in an adapter |
| `USERGUIDE_<name>.md` | Manual human steps required before the connector works | Person (or agent) performing the setup actions |

### SETUP files

Cover the developer-facing integration: what env vars to add, how to wire the connector into `main.py`, how to verify it works. These assume the external system is already configured.

### USERGUIDE files

Cover the **manual actions** that a human must complete in external systems before the connector will function. These are steps that cannot be automated — creating accounts, configuring labels, setting up filters, granting permissions, etc.

User guides are classified as either:
- **Internal** — steps performed by ENTR staff (e.g., Gmail: adding an alias in Google Admin Console, creating labels and filters)
- **External** — instructions to send to the client's IT team (e.g., "send these steps to the person managing your Exchange server")

The classification is stated at the top of each guide so the reader knows whether to act on it themselves or forward it.

### Naming

Files use the pattern `SETUP_<connector>.md` and `USERGUIDE_<connector>.md`, matching the connector module name (e.g., `gmail`, `microsoft`, `ftp`).

### Current documentation

| Connector | SETUP | USERGUIDE |
|-----------|-------|-----------|
| Gmail | `SETUP_gmail.md` | `USERGUIDE_gmail.md` |
| Microsoft Exchange | `SETUP_microsoft.md` | — |
| FTP | `SETUP_ftp.md` | — |
| SFTP | — | — |

## Cross-References

- **API reference**: See docstrings in `gmail.py`, `microsoft.py`, `sftp.py`, `ftp.py` for constructor parameters, method signatures, and return types
- **Retrieval endpoints**: See `app.md` for how `/list_documents` and `/download_document` dispatch to retrieval handlers
- **Handler integration**: See `handlers.md` for how retrieval handlers are injected into `BaseHandler`
- **Extras configuration**: See `pyproject.toml` `[project.optional-dependencies]` for available extras groups
