# DDR-0025: E27 Link-Layer Framing / Deframing Semantics (STARTCHAR, Escape, Length, CRC)

Status: Approved
Date: 2025-12-23
Applies to: elke27_lib (E27 backend only)
Related:
- ADR (planned): E27 framing/padding/word-swap semantics (umbrella ADR in 0100–0199 block)
- DDR-0024: Session / state transitions (deframer used by session receive pump)
- Util: calculate_crc16_checksum (CRC implementation is centralized)

## Context

E27 non-TLS traffic uses a link-layer “data frame” format on top of TCP. The TCP stream may contain:
- partial frames split across reads
- multiple frames concatenated in one read
- arbitrary garbage / desync bytes between frames

The official E27 docs include example Java link-layer code that specifies:
- STARTCHAR-based framing and escaping
- a resync rule that treats STARTCHAR followed by non-zero as a new frame start
- a CRC16 check on the unescaped frame data

We need a deterministic framing + deframing contract that:
- matches the doc’s behavior (especially resync)
- matches our unit tests
- keeps CRC logic centralized (util.py)

## Decision

### 1) Wire format (escaped stream)

STARTCHAR = 0x7E.

Each transmitted frame on the wire is:

    0x7E + escaped_bytes(unescaped_frame)

Where `unescaped_frame` is:

    [ protocol (1 byte) ]
    [ length_le (2 bytes) ]          # little-endian length
    [ payload_bytes (variable) ]
    [ crc_le (2 bytes) ]             # CRC16 appended little-endian

The `length_le` value is the total size of `unescaped_frame` in bytes
(i.e., protocol + length bytes + payload + crc bytes).

### 2) Escaping rules

Escaping is applied AFTER CRC is computed and appended.

- The first byte after STARTCHAR is always the protocol byte and is NOT escape-expanded.
- For every subsequent byte in `unescaped_frame` (starting at index 1):
  - if the byte equals STARTCHAR (0x7E), write: 0x7E 0x00
  - else write the byte as-is

Deframing performs the inverse:
- when 0x7E is seen, set `escaping = True` and consume the next byte:
  - if next byte is 0x00 => literal 0x7E
  - if next byte is non-zero => **resync start** (see below), and that non-zero byte is the protocol byte

### 3) Resync rule (critical)

If the deframer sees:
    STARTCHAR (0x7E) followed by a non-zero byte B

then B MUST be treated as the protocol byte for a brand-new frame, even if we were
mid-frame. Any partially accumulated bytes from the prior frame are discarded.

This is the Node-RED / doc-defined recovery rule.

### 4) Length handling

After capturing the protocol byte, the deframer reads two length bytes (little-endian).
When length is known, it reads until exactly `length` bytes of the *unescaped_frame*
are accumulated (including protocol, the 2 length bytes, payload, and CRC).

Constraints:
- `length` must be within bounds:
  - `length < DATA_BUS_IN_BUFFER_SIZE`
  - `length >= MIN_MESSAGE_SIZE` (at minimum: protocol + length(2) + crc(2) = 5)
- Out-of-range length yields an error result and the deframer returns to WAIT_START.

### 5) CRC16 semantics

CRC computation is delegated to:

    util.calculate_crc16_checksum(w_sum, data_bytes, start, numb)

Validation rule:
- When a full unescaped_frame is collected, compute:

    calculate_crc16_checksum(0, frame_bytes, 0, length)

- A return value of 0 indicates the received CRC matches (doc behavior: compute over full buffer including CRC bytes).

If CRC is bad:
- emit an error result for this frame
- immediately resume scanning for the next STARTCHAR (do not “stick”)

### 6) API surface contract

framing.py provides:

- frame_build(protocol_byte: int, data_frame: bytes) -> bytes
  Builds the wire-format bytes: STARTCHAR + escaped(unescaped_frame),
  using util.calculate_crc16_checksum for CRC.

- deframe_feed(state: DeframeState, chunk: bytes) -> list[DeframeResult]
  Stateful stream parser that can return zero or more results per chunk.

- DeframeState
  A dataclass holding incremental parse state (buffer, indexes, current FSM state).

- DeframeResult
  A dataclass with:
    - ok: bool
    - frame_no_crc: Optional[bytes]
    - error: Optional[str]
  Where `frame_no_crc` is the unescaped bytes excluding the final 2 CRC bytes, but
  INCLUDING protocol + 2-byte length + payload. (This is what presentation layer consumes.)

## Consequences

- Link-layer framing behavior is fully deterministic and testable without hardware.
- The resync rule makes the deframer robust against mid-stream corruption or boundary loss.
- CRC is centralized in util.py; framing.py does not duplicate the polynomial/logic.
- Presentation layer receives a clean “frame_no_crc” suitable for decrypt/decode, and does not
  need to understand escaping or CRC.
