# Phase 12: RPC Bidirectional Communication

**Status**: 🚧 In Progress
**Date Started**: 2026-01-26
**Critical**: Fixes broken styrened imports from styrene (TUI)

---

## Problem Statement

**Current State (Broken)**:
```python
# styrened/daemon.py line 145
from styrene.services.rpc_server import RPCServer  # 🔴 Imports from TUI!
```

styrened declares dependency on `styrene-core` only, but imports from `styrene` (TUI) package. This defeats the entire purpose of the Phase 11 separation.

**Root Cause**: RPC components (client, server, messages) were never migrated to styrene-core during Phases 1-11. They remained TUI-coupled.

---

## Objective

Move RPC implementation to styrene-core, enabling:
1. ✅ **Clean separation** - styrened depends only on styrene-core
2. ✅ **Bidirectional communication** - Both packages can send AND receive RPC
3. ✅ **Prairie dog pattern** - Ephemeral TUI + permanent daemon coordination
4. ✅ **DRY principle** - Single RPC implementation shared by all

---

## Scope

### Files to Migrate (~1,584 lines)

From `styrene/src/styrene/`:
- `services/rpc_client.py` (451 lines) → `styrene_core/rpc/client.py`
- `services/rpc_server.py` (510 lines) → `styrene_core/rpc/server.py`
- `services/rpc_errors.py` (78 lines) → `styrene_core/rpc/errors.py`
- `models/rpc_messages.py` (324 lines) → `styrene_core/rpc/messages.py`
- `models/rpc.py` (221 lines) - Keep in TUI (RPCClient protocol, TUI-specific)

### New styrene-core Structure

```
styrene-core/
└── src/styrene_core/
    └── rpc/
        ├── __init__.py       - Public API
        ├── client.py         - RPCClient (send commands)
        ├── server.py         - RPCServer (receive commands)
        ├── errors.py         - RPC exceptions
        ├── messages.py       - Message models (StatusRequest, etc.)
        └── handlers.py       - Base handler interface (new)
```

---

## Architecture: Bidirectional Communication

### Use Cases

**1. TUI → daemon** (operator manages edge)
```python
# styrene (TUI) sends command
await rpc_client.call_status(edge_device_hash)
# styrened receives and responds
```

**2. daemon → TUI** (daemon reports status)
```python
# styrened pushes alert
await rpc_client.send_alert(operator_hash, "Disk 90% full")
# styrene (TUI) receives notification
```

**3. TUI → TUI** (peer coordination)
```python
# styrene peer A requests config sync
await rpc_client.request_config(peer_b_hash)
# styrene peer B responds
```

**4. daemon → daemon** (hierarchical mesh)
```python
# styrened edge node reports to gateway
await rpc_client.report_status(gateway_hash)
# styrened gateway aggregates
```

---

## Implementation Plan

### Step 1: Create styrene-core RPC Structure (30 min)

```bash
cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrene-core
mkdir -p src/styrene_core/rpc
touch src/styrene_core/rpc/__init__.py
touch src/styrene_core/rpc/client.py
touch src/styrene_core/rpc/server.py
touch src/styrene_core/rpc/errors.py
touch src/styrene_core/rpc/messages.py
touch src/styrene_core/rpc/handlers.py
```

### Step 2: Migrate RPC Errors (15 min)

- Copy `styrene/services/rpc_errors.py` → `styrene_core/rpc/errors.py`
- No dependencies to change (pure exceptions)
- Update `__init__.py` exports

### Step 3: Migrate RPC Messages (30 min)

- Copy `styrene/models/rpc_messages.py` → `styrene_core/rpc/messages.py`
- Update imports (if any)
- Add serialization helpers
- Update `__init__.py` exports

### Step 4: Migrate RPC Client (1 hour)

- Copy `styrene/services/rpc_client.py` → `styrene_core/rpc/client.py`
- Update imports:
  - `from styrene.models.rpc_messages` → `from styrene_core.rpc.messages`
  - `from styrene.services.rpc_errors` → `from styrene_core.rpc.errors`
- Keep LXMFService dependency (already in core)
- Update `__init__.py` exports

### Step 5: Migrate RPC Server (1 hour)

- Copy `styrene/services/rpc_server.py` → `styrene_core/rpc/server.py`
- Update imports:
  - `from styrene.models.rpc_messages` → `from styrene_core.rpc.messages`
- Extract handler interface to `handlers.py`
- Update `__init__.py` exports

### Step 6: Create Handler Interface (30 min)

Create `styrene_core/rpc/handlers.py`:
```python
from abc import ABC, abstractmethod
from typing import Any
from styrene_core.rpc.messages import RPCMessage

class RPCHandler(ABC):
    """Base class for RPC command handlers."""

    @abstractmethod
    async def handle(self, message: RPCMessage, source_hash: str) -> dict[str, Any]:
        """Handle RPC command and return response."""
        pass

    @property
    @abstractmethod
    def command_type(self) -> str:
        """Command type this handler processes."""
        pass
```

### Step 7: Update styrened (1 hour)

**Remove broken imports**:
```python
# Before (BROKEN)
from styrene.services.rpc_server import RPCServer
from styrene.services.reticulum import get_operator_identity_object

# After (FIXED)
from styrene_core.rpc import RPCServer
from styrene_core.services.reticulum import get_operator_identity_object
```

**Add daemon-specific handlers** (keep in styrened):
- `handlers/status.py` - System status handler
- `handlers/exec.py` - Command execution handler
- `handlers/reboot.py` - Reboot handler
- `handlers/config.py` - Config update handler

**Add RPC client** for proactive reporting:
```python
# styrened can now send messages too
from styrene_core.rpc import RPCClient

async def report_alert(self, operator_hash: str, message: str):
    await self.rpc_client.send(operator_hash, AlertMessage(message))
```

### Step 8: Update styrene TUI (1 hour)

**Update imports**:
```bash
# Find all RPC imports in TUI
find src/styrene -name "*.py" -exec grep -l "from styrene.services.rpc\|from styrene.models.rpc_messages" {} \;

# Update to:
from styrene_core.rpc import RPCClient, RPCServer
from styrene_core.rpc.messages import StatusRequest, StatusResponse
from styrene_core.rpc.errors import RPCTimeoutError
```

**Keep TUI-specific RPC handlers**:
- `src/styrene/rpc/handlers/` (new directory)
  - `peer_sync.py` - Config synchronization between TUI peers
  - `provision.py` - Remote provisioning handler
  - `discovery.py` - Device discovery coordination

**Add RPC server** (TUI can receive too):
```python
# In StyreneLifecycle or main app
from styrene_core.rpc import RPCServer

# TUI now listens for incoming RPC (from daemons or peers)
self.rpc_server = RPCServer(lxmf_service)
self.rpc_server.register_handler("alert", AlertHandler())
self.rpc_server.start()
```

### Step 9: Update styrene-bond-rpc (30 min)

**Decision**: Deprecate styrene-bond-rpc or merge into styrene-core

Option A: **Deprecate** - Add deprecation notice, point to `styrene_core.rpc`
Option B: **Merge** - Move remaining unique logic to styrene-core

Recommend: **Option A** - Deprecate and document migration path

### Step 10: Add Tests (2 hours)

**styrene-core tests**:
```python
# tests/test_rpc_client.py
def test_rpc_client_send_request():
    """Test sending RPC request."""

def test_rpc_client_timeout():
    """Test request timeout handling."""

def test_rpc_client_correlation():
    """Test request/response correlation."""

# tests/test_rpc_server.py
def test_rpc_server_handle_request():
    """Test server handles request."""

def test_rpc_server_handler_registration():
    """Test handler registration."""

# tests/test_rpc_messages.py
def test_message_serialization():
    """Test message to_dict/from_dict."""
```

**Integration test**:
```python
# tests/integration/test_rpc_bidirectional.py
async def test_client_server_roundtrip():
    """Test full RPC roundtrip: client -> server -> response."""
```

### Step 11: Update Documentation (1 hour)

**styrene-core README.md**:
```markdown
### RPC Communication

```python
from styrene_core.rpc import RPCClient, RPCServer
from styrene_core.rpc.messages import StatusRequest
from styrene_core.services.lxmf_service import get_lxmf_service

# Client: Send RPC command
lxmf = get_lxmf_service()
client = RPCClient(lxmf)
response = await client.call_status(device_hash)

# Server: Receive RPC commands
server = RPCServer(lxmf)
server.register_handler("status", StatusHandler())
server.start()
```
```

**CHANGELOG.md** (all three packages):
- styrene-core v0.2.0: Add RPC bidirectional communication
- styrene v0.4.0: Use core RPC, add server capabilities
- styrened v0.2.0: Fix broken imports, add client capabilities

### Step 12: Version Bumps and Commits (30 min)

**styrene-core**:
- Bump version: 0.1.0 → 0.2.0
- Commit: "feat: Add RPC bidirectional communication (Phase 12)"

**styrened**:
- Bump version: 0.1.0 → 0.2.0
- Fix imports from styrene-core
- Commit: "fix: Use styrene-core RPC, add client capabilities (Phase 12)"

**styrene**:
- Bump version: 0.3.0 → 0.4.0
- Update imports from styrene-core
- Commit: "feat: Use styrene-core RPC, add server capabilities (Phase 12)"

---

## Testing Strategy

### Unit Tests

- RPC message serialization/deserialization
- Client request correlation and timeout
- Server handler registration and dispatch
- Error propagation

### Integration Tests

- Client → Server roundtrip (same process)
- Bidirectional: Client sends, server responds, client receives
- Timeout handling in full stack
- Error responses from server to client

### Manual Testing

**styrened**:
```bash
cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrened
python -m styrened
# Should start without ImportError
```

**styrene (TUI)**:
```bash
cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrene-tui
python -m styrene
# Navigate to device console, send RPC command
```

---

## Rollback Strategy

**Pre-Phase 12 Tags**:
```bash
# Tag before starting
cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrene-core
git tag pre-phase-12

cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrened
git tag pre-phase-12

cd /Users/cwilson/workspace/vanderlyn/styrene-lab/styrene-tui
git tag pre-phase-12
```

**If Issues Found**:
```bash
git reset --hard pre-phase-12
```

---

## Success Criteria

- [ ] styrened imports only from styrene-core (zero imports from styrene)
- [ ] styrene-core RPC tests pass (>90% coverage)
- [ ] styrened starts without ImportError
- [ ] styrene TUI starts and RPC client works
- [ ] Both styrene and styrened can send AND receive RPC
- [ ] Typecheck passes in all three packages
- [ ] Documentation updated with examples

---

## Time Estimate

| Step | Time | Running Total |
|------|------|---------------|
| 1. Create structure | 30 min | 30 min |
| 2. Migrate errors | 15 min | 45 min |
| 3. Migrate messages | 30 min | 75 min |
| 4. Migrate client | 1 hour | 135 min |
| 5. Migrate server | 1 hour | 195 min |
| 6. Handler interface | 30 min | 225 min |
| 7. Update styrened | 1 hour | 285 min |
| 8. Update styrene | 1 hour | 345 min |
| 9. styrene-bond-rpc | 30 min | 375 min |
| 10. Tests | 2 hours | 495 min |
| 11. Documentation | 1 hour | 555 min |
| 12. Commits | 30 min | 585 min |
| **Total** | **~10 hours** | |

With testing and validation: **10-12 hours**

---

## Next Actions

1. Create git tags (pre-phase-12)
2. Execute Step 1 (create structure)
3. Execute Steps 2-6 (migrate RPC to core)
4. Execute Step 7 (fix styrened)
5. Execute Step 8 (update styrene)
6. Test, document, commit

---

**Phase 12 Status**: 🚧 Ready to Execute
