# RNSH Protocol Analysis: Layering SSH over Reticulum RNS

**Analysis Date:** 2026-01-22
**Source:** https://github.com/acehoss/rnsh
**Purpose:** Guide LXMF protocol implementation for Styrene TUI

---

## 1. Protocol Layering Architecture

RNSH uses a **Channel-based message transport** model on top of RNS Links:

### Layer Stack
```
Application Layer (Shell/SSH)
  ↓
rnsh Protocol Messages (NoopMessage, StreamDataMessage, etc.)
  ↓
RNS Channel API (Message delivery with acknowledgments)
  ↓
RNS Link (Authenticated point-to-point)
  ↓
RNS Packet/Transport (Reticulum Network)
```

### Key Architecture Points

- **Magic Number**: `0xac` (MSG_MAGIC) with 8-bit message type encoding
- **Protocol Version**: PROTOCOL_VERSION = 1
- **Transport Model**: One message per packet (stop-and-wait), with plans for sliding window
- **Serialization**: umsgpack binary format for all message payloads
- **Source**: `/tmp/rnsh_repo/rnsh/protocol.py` lines 23-25

---

## 2. Message Framing Strategy

### Message Types (7 defined types)

```python
# From protocol.py lines 31-139
NoopMessage (MSGTYPE: 0xac00)
  - Empty payload, used for keepalive
  - Listener echoes back to initiator

WindowSizeMessage (MSGTYPE: 0xac02)
  - Payload: (rows, cols, hpix, vpix) - packed via umsgpack
  - Used for terminal size synchronization

ExecuteCommandMesssage (MSGTYPE: 0xac03)
  - Payload: (cmdline[], pipe_stdin, pipe_stdout, pipe_stderr,
              tcflags[], term, rows, cols, hpix, vpix)
  - Tuples packed with umsgpack

StreamDataMessage (MSGTYPE: 0xac04)
  - Extends RNS.Buffer.StreamDataMessage
  - Stream IDs: 0=stdin, 1=stdout, 2=stderr
  - Payload: binary data + eof flag + compression_flag

VersionInfoMessage (MSGTYPE: 0xac05)
  - Payload: (sw_version, protocol_version)
  - Protocol compatibility validation

ErrorMessage (MSGTYPE: 0xac06)
  - Payload: (msg, fatal, data_dict)
  - Signals terminal or non-fatal errors

CommandExitedMessage (MSGTYPE: 0xac07)
  - Payload: return_code (int)
  - Signals remote process termination
```

### Framing Example

```python
# From initiator.py line 494
channel.send(protocol.StreamDataMessage(
    protocol.StreamDataMessage.STREAM_ID_STDIN,
    stdin,              # binary data (up to MDU - overhead)
    eof,                # EOF flag
    comp_success        # compression indicator
))
```

### Compression Strategy

Lines 457-487 show adaptive bzip2 compression:
- Compression only if:
  - Compressed size < original size
  - Compressed size < MDU - overhead
  - Data chunk > 32 bytes
- Tries multiple compression ratios (1x, 2x, 4x, etc.)

---

## 3. Multiplexing Approach

**Not traditional SSH multiplexing** - instead uses stream IDs within single channel:

```python
# From protocol.py lines 88-90
STREAM_ID_STDIN  = 0
STREAM_ID_STDOUT = 1
STREAM_ID_STDERR = 2
```

### Design Pattern

- Single RNS Link per session
- Single RNS Channel per session
- Multiple logical streams (stdin/stdout/stderr) multiplexed via stream_id field
- No concurrent SSH channels (unlike SSH protocol)

### Buffer Management

From session.py:
```python
# Lines 98-101
self.stdout_buf = bytearray()      # Accumulates shell output
self.stderr_buf = bytearray()      # Accumulates error output
self.stdout_eof_sent = False       # EOF tracking per stream
self.stderr_eof_sent = False

# Listener pump() method (lines 209-279) sends data in priority order:
# 1. stderr_buf (errors first)
# 2. stdout_buf (standard output second)
# 3. return_code (when process terminates)
```

### Ordering Constraint (Critical Design Choice)

```python
# From session.py line 245
if not self.channel.is_ready_to_send():
    return False  # Wait for ACK before next message
```

**Impact**: Stop-and-wait backpressure - maximum 1 outstanding message per link

---

## 4. Error Handling Patterns

### Listener-side Error States

From session.py, LSState enum lines 34-40:
```python
LSSTATE_WAIT_IDENT    = 1  # Waiting for client identity
LSSTATE_WAIT_VERS     = 2  # Waiting for version info
LSSTATE_WAIT_CMD      = 3  # Waiting for ExecuteCommand
LSSTATE_RUNNING       = 4  # Session active
LSSTATE_ERROR         = 5  # Error detected
LSSTATE_TEARDOWN      = 6  # Cleaning up
```

### Initiator-side Error States

From initiator.py lines 135-141:
```python
IS_INITIAL    = 0  # Just starting
IS_LINKED     = 1  # RNS Link established
IS_WAIT_VERS  = 2  # Awaiting version handshake
IS_RUNNING    = 3  # Session active
IS_TERMINATE  = 4  # Terminating
IS_TEARDOWN   = 5  # Torn down
```

### Error Handling Examples

**1. Protocol Version Mismatch** (session.py lines 355-357):
```python
if message.protocol_version != protocol.PROTOCOL_VERSION:
    self.terminate("Incompatible protocol")
    return
```

**2. Authentication Failure** (session.py lines 188-189):
```python
if not self.allow_all and identity.hash not in allowed_hashes:
    self.terminate("Identity is not allowed.")
```

**3. Command Execution Failure** (session.py line 328):
```python
except Exception as ex:
    self._log.exception(f"Unable to start process", ex)
    self.terminate("Unable to start process")
```

**4. Remote Error Handling** (initiator.py lines 219-225):
```python
async def _handle_error(errmsg: RNS.MessageBase):
    if isinstance(errmsg, protocol.ErrorMessage):
        if _link and _link.status == RNS.Link.ACTIVE:
            _link.teardown()
        raise RemoteExecutionError(f"Remote error: {errmsg.msg}")
```

### Error Message Format

```python
# From protocol.py line 108-121
class ErrorMessage(RNS.MessageBase):
    def __init__(self, msg: str = None, fatal: bool = False, data: dict = None):
        self.msg = msg
        self.fatal = fatal      # If True, initiator exits
        self.data = data        # Optional diagnostic info
```

---

## 5. Timeout Strategies

### Timeout Calculation

From session.py lines 116-122:
```python
def _set_state(self, state: LSState, timeout_factor: float = 10.0):
    # RTT-based timeout with multiplier
    timeout = max(
        self.outlet.rtt * timeout_factor,
        max(self.outlet.rtt * 2, 10)
    ) if timeout_factor is not None else None
```

### Timeout Applications

**1. Per-State Timeouts:**
   - WAIT_IDENT state: 10x RTT (identity must arrive quickly)
   - WAIT_VERS state: 10x RTT (version negotiation)
   - WAIT_CMD state: 10x RTT (command must follow)
   - RUNNING: Return code must arrive within 5x RTT after process termination

**2. Link Establishment** (initiator.py lines 186-188):
```python
if not await _spin(
    until=lambda: RNS.Transport.has_path(destination_hash),
    msg="Requesting path...",
    timeout=timeout,  # Default: RNS.Transport.PATH_REQUEST_TIMEOUT
    quiet=quietness > 0
):
    raise RemoteExecutionError("Path not found")
```

**3. Version Handshake** (initiator.py lines 268-276):
```python
vm = _pq.get(timeout=max(outlet.rtt * 20, 5))  # 20x RTT, minimum 5 seconds
await _handle_error(vm)
if not isinstance(vm, protocol.VersionInfoMessage):
    raise Exception("Invalid message received")
```

**4. Idle Timeout After Process Exit** (session.py line 150):
```python
self._call(self._prune, max(self.outlet.rtt * 3, process.PROCESS_PIPE_TIME+5))
# Cleans up session 3x RTT after last message
```

**5. Window Size Update Rate Limiting** (initiator.py line 499):
```python
if use_tty and winch and time.time() - last_winch > _link.rtt * 25:
    # Only send window updates max every 25x RTT to avoid saturation
```

### Timeout Error Handling

From session.py lines 160-167:
```python
def _check_protocol_timeout(self, fail_condition: Callable[[], bool], name: str):
    timeout = True
    try:
        timeout = self.state != LSState.LSSTATE_TEARDOWN and fail_condition()
    except Exception as ex:
        self._log.exception("Error in protocol timeout", ex)
    if timeout:
        self._protocol_timeout_error(name)  # Terminates session
```

---

## 6. Session Lifecycle State Machine

### Complete State Diagram

```
LISTENER SIDE                          INITIATOR SIDE
────────────────────────────────────────────────────

                INITIAL
                   |
                   | (Establish RNS Link)
                   ↓
            IS_LINKED
            (Link active)
                   |
       [WAIT_IDENT or WAIT_VERS]       IS_WAIT_VERS
             (identity check)          (awaiting version)
                   |                      |
       [Validate identity]      [Receive VersionInfoMessage]
                   |                      |
                   ├──────────────────────┤
                   ↓                      ↓
        LSSTATE_WAIT_VERS    [Version compatible?]
         (protocol version)     - Yes → IS_RUNNING
                   |             - No → error
       [Receive VersionInfo]
              Match?
                   |
       [Send back VersionInfo]
                   |
        LSSTATE_WAIT_CMD        [Send ExecuteCommand]
         (awaiting command)          |
                   |          [Awaiting process startup]
    [Receive ExecuteCommand]        |
              Valid?          [Receive process output]
         - Yes → start          or [Process terminates]
         - No → error           |
                   |                   ↓
                   ↓            IS_RUNNING
        LSSTATE_RUNNING        (session active)
        (process running)            |
             |                       | (Streaming data)
    [Handle messages]:              |
    - StreamData (stdin)        - [User input]
    - WindowSize                - [Receive output]
    - Noop (echo back)          - [Send window updates]
    - Monitor process           |
             |            [Receive CommandExited
    [Process terminates]        or ErrorMessage]
    |                           |
    ↓                           ↓
Send CommandExitedMessage  IS_TERMINATE
    |                      |
    |                      (Client exits with code)
    ↓
LSSTATE_ERROR/TEARDOWN
    |
Cleanup session
```

### Key Handshake Sequence

From README lines 230-254:

1. **Initiator establishes link**
   - Listener enters LSSTATE_WAIT_IDENT or LSSTATE_WAIT_VERS

2. **Initiator identifies** (if not --no-id)
   - Listener validates against allowed list
   - If invalid: send ErrorMessage, start prune timer

3. **Initiator sends VersionInformationMessage**
   - Listener validates protocol version
   - If incompatible: send ErrorMessage

4. **Listener responds with VersionInfoMessage**
   - Listener enters LSSTATE_WAIT_CMD
   - Initiator evaluates version

5. **Initiator sends ExecuteCommandMessage**
   - Initiator enters IS_RUNNING state

6. **Listener evaluates command** against -A/-C options
   - If invalid: send ErrorMessage
   - If valid: start program
   - Listener enters LSSTATE_RUNNING
   - If failure: send CommandExitedMessage

---

## 7. Event Loop & Pump Pattern

### Listener Pump

From session.py lines 209-279:
```python
def pump(self) -> bool:
    # Called repeatedly to process queued data
    # Returns True if any data was sent (for efficiency)

    # Only when state == RUNNING and channel ready
    # Send in priority:
    # 1. stderr (errors first)
    # 2. stdout (normal output)
    # 3. return_code (when process exits)

    # Per-message adaptive compression
    # Stop if channel not ready (backpressure)
```

### Initiator Event Loop

From initiator.py lines 419-517:
```python
while not await _check_finished() and state == IS_RUNNING:
    try:
        # 1. Receive messages from listener
        message = _pq.get(timeout=sleeper.next_sleep_time())

        # 2. Handle output (stdout/stderr)
        if isinstance(message, StreamDataMessage):
            os.write(stream_id, message.data)

        # 3. Handle command exit
        elif isinstance(message, CommandExitedMessage):
            return message.return_code

        # 4. Send pending input
        if channel.is_ready_to_send():
            channel.send(StreamDataMessage(STDIN, data, eof))

        # 5. Send window size updates (rate limited)
        if winch and time.time() - last_winch > link.rtt * 25:
            channel.send(WindowSizeMessage(...))

    except RemoteExecutionError:
        return 255
```

### Queue Pattern

From initiator.py lines 150-152:
```python
_pq = queue.Queue()  # Global message queue

def _client_message_handler(message):
    _pq.put(message)  # Messages enqueued by channel handler
```

---

## 8. Actionable Patterns for LXMF Protocol Implementation

Based on rnsh's design, here are patterns applicable to LXMF protocols:

### 1. Message Type Registration
```python
# Instead of RNS magic numbers, use LXMF-specific prefixes
def register_message_types(channel):
    for message_type in [CustomMessage1, CustomMessage2, ...]:
        channel.register_message_type(message_type)
```

### 2. State Machine + RTT-Based Timeouts
```python
# Calculable, network-aware timeouts
timeout = max(link.rtt * factor, minimum_value)
# Avoids fixed hard-coded timeouts
```

### 3. Compression Strategy
```python
# Adaptive compression per-message
compressed = bz2.compress(data[:chunk_size])
if len(compressed) < len(original) and len(compressed) < mdu:
    send_compressed = True
else:
    send_compressed = False
```

### 4. Backpressure Handling
```python
# Stop-and-wait protocol respects channel readiness
while not channel.is_ready_to_send():
    await asyncio.sleep(0.01)
channel.send(message)
```

### 5. Error State Management
```python
# Define clear state enum for protocol phases
class SessionState(enum.IntEnum):
    INITIALIZING = 1
    HANDSHAKE = 2
    ACTIVE = 3
    ERROR = 4
    TEARDOWN = 5
```

### 6. Graceful Shutdown
```python
# Send error message before closing
send(ErrorMessage(msg, fatal=True))
# Wait before teardown to allow receipt
await asyncio.sleep(rtt * 3)
link.teardown()
```

### 7. Priority Queuing
```python
# In pump(), process in priority order
if stderr_data:
    send(stderr)
elif stdout_data:
    send(stdout)
elif exit_code:
    send(exit_code)
```

### 8. Stream Multiplexing Without Channels
```python
# Use single message field to multiplex
class StreamDataMessage:
    stream_id: int  # 0=stdin, 1=stdout, 2=stderr
    data: bytes
    eof: bool
```

---

## Summary Table

| Aspect | Implementation |
|--------|-------------------|
| **Message Types** | 7 types (Noop, WindowSize, ExecuteCommand, StreamData, VersionInfo, Error, CommandExited) |
| **Serialization** | umsgpack binary format |
| **Framing** | One message per RNS packet |
| **Multiplexing** | Stream IDs (stdin=0, stdout=1, stderr=2) in single logical channel |
| **Backpressure** | Stop-and-wait (max 1 outstanding message) |
| **Compression** | Adaptive per-message bzip2 |
| **Error Handling** | State-driven with fatal/non-fatal error messages |
| **Timeouts** | RTT-based with multipliers (5x-25x RTT) |
| **Session States** | 6 listener states + 6 initiator states |
| **Authentication** | Identity hash validation before protocol start |
| **TTY Support** | Window size synchronization + terminal flags |

---

## Key Takeaways for Styrene Protocol Implementation

1. **Use state machines** with clear phase transitions for protocol sessions
2. **RTT-based timeouts** are more robust than fixed timeouts
3. **Stop-and-wait** provides simple backpressure but limits throughput
4. **Priority queuing** ensures critical messages (errors) arrive first
5. **Adaptive compression** per-message can save bandwidth without adding complexity
6. **Stream multiplexing** via message fields is simpler than multiple channels
7. **Version handshake** before protocol messages prevents compatibility issues
8. **Fatal vs non-fatal errors** allow graceful degradation
