## ADR-0013: Session-Owned Protocol Liveness, Sequencing, and Multi-Block Reassembly (E27)

Date: 2025-12-27  
Status: Accepted

---

## Context

The Elk E27 Dealer API and observed panel behavior indicate that a long-lived TCP connection may be closed by the panel or network if the client remains idle. In live testing, an otherwise healthy connection ran for a period and then failed with `Connection reset by peer` when no traffic was sent for an interval.

The Dealer API documents a liveness request/response (`system.r_u_alive`) intended to indicate that the control is still present. The E27 comms guidance describes maintaining a 24/7 TCP connection and reconnecting with a new HELLO when the socket is closed.

In addition, the Dealer API defines multiple request/response patterns that return multi-block (paged) responses, where a logical response is delivered across multiple messages using explicit block numbering (`block_id`) and total block count (`block_count`). These appear in APIs such as (but not limited to):

- `area.get_configured`
- `zone.get_configured`
- table and inventory readers

Finally, the E27 protocol uses **two distinct sequence numbers**:

- a JSON message sequence number (`"seq"`) for request/response correlation
- an encrypted presentation-layer envelope sequence number inside framed data

Observed behavior from the Elk API Emulator confirms that these two sequence domains serve different purposes and are not interchangeable.

We must decide where protocol liveness, sequencing, and multi-block response handling belong in the architecture, consistent with DDR-0034:

- `Session` is responsible for transport lifecycle, HELLO, framing receive pump, disconnect/reconnect mechanics, and message dispatch.
- Linking and authentication are explicit operations owned above `Session` and are out of scope for `Session`.
- Feature modules represent panel capabilities (domain semantics), update state, and emit events.
- Home Assistant and sample applications should not be required to implement protocol plumbing.

---

## Decision

### 1. Protocol liveness (keepalive)

Protocol liveness for E27 SHALL be owned by `Session`.

`Session` SHALL implement an internal keepalive loop that periodically sends `system.r_u_alive` while the session is ACTIVE, correlates the response by message sequence number, and uses timeouts/failures as a connection-health signal to drive reconnect behavior.

Keepalive messages and responses SHALL be handled privately by `Session` and SHALL NOT be exposed through feature modules, application-level request registries, or Home Assistant integration surfaces.

---

### 2. Message sequence number handling (JSON `seq`)

The JSON `"seq"` field SHALL be treated as the **application-level message correlation identifier**.

#### Rules

1. Every outbound request sent by the library MUST include a top-level `"seq"` integer.
2. `Session` (or its owning client) SHALL maintain a monotonic JSON sequence generator:
   - incremented by 1 per outbound request
   - wrapping at the signed 32-bit limit (`2,147,483,647`) to a non-zero value
3. `"seq": 0` SHALL be treated as **unsolicited / non-correlatable**:
   - unsolicited events and notifications may use `seq == 0`
   - responses with `seq == 0` MUST NOT be matched to pending requests
4. Pending request correlation SHALL be keyed by the request sequence number, or by an internal transfer identifier that maps back to request sequences.
5. For request formatting, `"seq"` SHALL be placed at the **top level** of the JSON object.
6. For response parsing, only the top-level `"seq"` SHALL be used for correlation; any nested `"seq"` fields are treated as payload data unless explicitly documented otherwise.

---

### 3. Encrypted envelope sequence handling (presentation layer)

The encrypted presentation-layer envelope sequence SHALL be treated as a **transport-level mechanism**, distinct from JSON message sequencing.

#### Rules

1. `Session` SHALL maintain a per-session encrypted-envelope transmit sequence counter.
2. The envelope sequence:
   - increments on every encrypted outbound frame
   - wraps at the signed 32-bit limit to 1
3. The envelope sequence SHALL be included in encrypted frames exactly as required by the presentation-layer format.
4. For inbound encrypted frames:
   - the envelope sequence MAY be parsed and recorded for diagnostics
   - the sequence SHALL NOT be enforced for ordering or validation
5. Envelope sequence counters SHALL be reset when a new session is established (new HELLO / new session key).

The envelope sequence exists solely to satisfy protocol framing requirements and SHALL NOT be used for request/response correlation.

---

### 4. Multi-block (paged) response reassembly

Reassembly of all E27 multi-block / paged responses SHALL be treated as a protocol-level responsibility and SHALL be implemented using a single canonical algorithm.

Multi-block reassembly SHALL occur below feature semantics and SHALL NOT be re-implemented by individual domain handlers, feature modules, Home Assistant, or sample applications.

Partial blocks SHALL NOT be exposed to higher layers.

---

## Rationale

### Keepalive is transport health, not a panel capability

`system.r_u_alive` exists to prevent idle TCP teardown and to detect dead peers. This is a property of maintaining the transport session and does not represent an application feature or panel capability that end users should choose to invoke.

### Message sequencing must be deterministic and layered

The E27 protocol explicitly separates:
- message-level correlation (`seq`)
- transport/presentation sequencing (encrypted envelope sequence)

Treating these as distinct domains avoids coupling application logic to framing details and matches observed emulator behavior.

### Avoid leaking protocol plumbing to higher layers

Implementing keepalive, sequencing, or paging logic in feature modules or applications would:
- expose protocol mechanics unnecessarily,
- require HA/sample programs to implement transport scheduling and correlation,
- blur the separation between transport correctness and domain semantics.

---

### Multi-block responses are deterministic protocol mechanics

Unlike general packet fragmentation, E27 multi-block responses explicitly encode:

- the current block number (`block_id`, 1…N)
- the total number of blocks (`block_count`, N)

This makes reassembly deterministic, order-explicit, easy to validate, and independent of message arrival order.

Because this behavior is defined by the protocol itself, reassembly is not a feature concern and not an application workflow.

---

## Canonical Multi-Block Reassembly Algorithm

For any E27 response that includes both `block_id` and `block_count`, the following algorithm SHALL be used.

### State (per transfer)

- `received_blocks`: mapping of `block_id → data`
- `total_count`: integer, learned from first block
- `started_at`: timestamp for timeout handling

Transfers are keyed by a correlation identifier such as `(session_id, transfer_id, route)`, where `transfer_id` uniquely identifies the paged operation.

### On receiving a block

1. Parse  
   - Extract `block_id`, `block_count`, and payload.

2. Validate  
   - `block_id ≥ 1`  
   - `block_count ≥ 1`  
   - `block_id ≤ block_count`  
   - If `total_count` already known, `block_count` MUST match.

3. Store  
   - If this `block_id` has not been seen, store its payload.  
   - Duplicate blocks are ignored.

4. Completion check  
   - When `len(received_blocks) == total_count`:  
     - Reassemble blocks in numeric order `1..N`.  
     - Deliver the fully assembled result to the handler.  
     - Clear transfer state.

### Failure handling

- If block totals are inconsistent, abort the transfer.
- If no new blocks arrive within the configured timeout, abort the transfer.
- On disconnect, all in-progress transfers are aborted.

### Reassembly semantics

The reassembly mechanism is generic; the merge policy is defined by the endpoint:
- concatenate lists,
- merge keyed dictionaries,
- concatenate byte/string payloads.

---

## Consequences

### Positive

- Home Assistant integration and sample applications remain unaware of keepalive, sequencing, and paging requirements.
- `Session.connect()` delivers a connection that remains stable during idle periods.
- Keepalive response timeouts become a reliable signal to trigger reconnect.
- Message correlation is unambiguous and protocol-correct.
- Multi-block responses are handled consistently across all APIs.

### Negative / Tradeoffs

- `Session` and dispatcher logic become more complex.
- Memory is temporarily required to buffer multi-block responses.
- Timeouts must be chosen carefully to avoid premature aborts on slow panels.

---

## Implementation Notes

### Keepalive

- Keepalive runs only while `SessionState.ACTIVE`.
- Each keepalive uses a distinct JSON message sequence number.
- `Session` logs a warning when a keepalive response is not received within `keepalive_timeout_s`.
- `Session` triggers reconnect after `keepalive_max_missed` consecutive keepalive timeouts (default 2).
- Keepalive responses are consumed internally and not forwarded to dispatcher or features.

### Sequencing

- JSON `seq` is the sole request/response correlation mechanism.
- Encrypted envelope sequence is transport-only and ignored for application logic.
- Multi-block paging MAY use an internal transfer identifier distinct from JSON `seq`.

### Multi-block reassembly

- Reassembly is keyed by transfer correlation, not receive order.
- Partial blocks are never dispatched upward.
- Feature handlers and HA code only see final assembled results.
- Authorization errors (`error_code = 11008`) at the domain root abort reassembly and surface via existing semantic error mechanisms.

---

## Related Decisions

- DDR-0034: Session responsibility boundaries (transport lifecycle, HELLO, framing receive pump, reconnect; no linking/auth orchestration).
- ADR-0011 / ADR-010x: E27 framing and session mechanics (where applicable).
