# Backend Interface Specification

Complete specification for implementing TUI backends.

## Table of Contents

1. [Overview](#overview)
2. [Backend Contract](#backend-contract)
3. [Message Types](#message-types)
4. [Implementing a Backend](#implementing-a-backend)
5. [Built-in Backends](#built-in-backends)
6. [Testing Backends](#testing-backends)

## Overview

The Cleave TUI supports multiple LLM backends through a unified interface. This allows users to choose between:

- **Cloud APIs**: Claude (Anthropic), OpenAI
- **Local inference**: Ollama, vLLM, llama.cpp
- **Custom providers**: Implement your own backend

All backends implement the same `TUIBackend` interface, making them interchangeable.

## Backend Contract

All backends must implement the `TUIBackend` abstract base class.

### Interface Definition

```python
from abc import ABC, abstractmethod
from typing import AsyncIterator, List, Dict, Any

class TUIBackend(ABC):
    """Abstract base class for TUI backends."""

    def __init__(self, config: Dict[str, Any]):
        """
        Initialize backend with configuration.

        Args:
            config: Backend-specific configuration from settings.yaml
        """
        self.config = config

    @abstractmethod
    async def send_message(
        self,
        messages: List[Dict[str, Any]],
        tools: List[Dict[str, Any]] = None,
    ) -> AsyncIterator[Dict[str, Any]]:
        """
        Send message and stream response.

        Args:
            messages: List of messages in Anthropic message format
                [
                    {"role": "user", "content": "Hello"},
                    {"role": "assistant", "content": "Hi there!"},
                    {"role": "user", "content": "How are you?"}
                ]
            tools: Optional list of tool definitions (Claude Agent SDK format)

        Yields:
            Response chunks in standardized format:
                {"type": "text", "content": "partial text"}
                {"type": "tool_use", "id": "...", "name": "...", "input": {...}}
                {"type": "done"}
        """
        pass

    @abstractmethod
    async def close(self):
        """Clean up resources (close connections, etc.)."""
        pass

    @property
    @abstractmethod
    def name(self) -> str:
        """Backend name (e.g., 'claude', 'ollama')."""
        pass

    @property
    @abstractmethod
    def model(self) -> str:
        """Current model identifier."""
        pass
```

## Message Types

### Input Messages

Messages follow the Anthropic message format:

```python
{
    "role": "user" | "assistant",
    "content": "message text" | [content_blocks]
}
```

**User Message**:
```python
{
    "role": "user",
    "content": "Add JWT authentication"
}
```

**Assistant Message**:
```python
{
    "role": "assistant",
    "content": "I'll implement JWT authentication..."
}
```

**Multi-part Content**:
```python
{
    "role": "user",
    "content": [
        {"type": "text", "text": "Analyze this image"},
        {"type": "image", "source": {"type": "base64", "data": "..."}}
    ]
}
```

### Output Chunks

Backends yield chunks during streaming:

#### Text Chunk
```python
{
    "type": "text",
    "content": "partial text"
}
```

#### Tool Use Chunk
```python
{
    "type": "tool_use",
    "id": "tool_use_123",
    "name": "cleave_assess",
    "input": {
        "directive": "Add JWT authentication"
    }
}
```

#### Tool Result Chunk
```python
{
    "type": "tool_result",
    "tool_use_id": "tool_use_123",
    "content": "Assessment complete: complexity 6.0"
}
```

#### Done Chunk
```python
{
    "type": "done"
}
```

Signals end of response.

## Implementing a Backend

### Step 1: Create Backend Class

Create a file in `src/cleave/tui/backends/`:

```python
# src/cleave/tui/backends/my_backend.py

from typing import AsyncIterator, List, Dict, Any
from .base import TUIBackend

class MyBackend(TUIBackend):
    """Backend for My Custom LLM Provider."""

    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        self.base_url = config.get("base_url", "http://localhost:8000")
        self.api_key = config.get("api_key")
        self.default_model = config.get("default_model", "default-model")
        # Initialize your client here

    async def send_message(
        self,
        messages: List[Dict[str, Any]],
        tools: List[Dict[str, Any]] = None,
    ) -> AsyncIterator[Dict[str, Any]]:
        """Send message and stream response."""
        # Convert messages to your provider's format
        provider_messages = self._convert_messages(messages)

        # Make API call (example with streaming)
        async for chunk in self._stream_completion(provider_messages):
            # Convert response chunks to standard format
            yield self._convert_chunk(chunk)

        # Signal completion
        yield {"type": "done"}

    async def close(self):
        """Clean up resources."""
        # Close HTTP clients, cleanup, etc.
        pass

    @property
    def name(self) -> str:
        return "my-backend"

    @property
    def model(self) -> str:
        return self.default_model

    def _convert_messages(self, messages: List[Dict[str, Any]]) -> List[Dict]:
        """Convert Anthropic format to provider format."""
        # Your conversion logic here
        pass

    async def _stream_completion(self, messages: List[Dict]) -> AsyncIterator[Dict]:
        """Stream completion from provider API."""
        # Your API call logic here
        pass

    def _convert_chunk(self, chunk: Dict) -> Dict[str, Any]:
        """Convert provider chunk to standard format."""
        # Your conversion logic here
        pass
```

### Step 2: Register Backend

Add to `src/cleave/tui/backends/__init__.py`:

```python
from .my_backend import MyBackend

BACKENDS = {
    "claude": ClaudeBackend,
    "ollama": OllamaBackend,
    "openai": OpenAICompatBackend,
    "my-backend": MyBackend,  # Add here
}
```

### Step 3: Configure Backend

Add to `~/.cleave/settings.yaml`:

```yaml
backend: my-backend

backends:
  my-backend:
    base_url: http://localhost:8000
    api_key_env: MY_BACKEND_API_KEY
    default_model: my-model-v1
```

### Step 4: Test Backend

```bash
# Set API key (if needed)
export MY_BACKEND_API_KEY=your-key

# Configure backend
cleave config set backend my-backend

# Launch TUI
cleave tui
```

## Built-in Backends

### Claude Backend

**File**: `src/cleave/tui/backends/claude.py`

**Configuration**:
```yaml
backends:
  claude:
    default_model: claude-sonnet-4-20250514
    api_key_env: ANTHROPIC_API_KEY
```

**Features**:
- Full Claude Agent SDK integration
- Tool use support
- Streaming responses

### OpenAI-Compatible Backend

**File**: `src/cleave/tui/backends/openai_compat.py`

**Configuration**:
```yaml
backends:
  ollama:
    base_url: http://localhost:11434/v1
    default_model: qwen2.5-coder:7b

  openai:
    base_url: https://api.openai.com/v1
    api_key_env: OPENAI_API_KEY
    default_model: gpt-4o

  vllm:
    base_url: http://localhost:8000/v1
    default_model: meta-llama/Llama-3.1-70B-Instruct

  llamacpp:
    base_url: http://localhost:8080/v1
    default_model: llama-3.1-70b-instruct
```

**Features**:
- OpenAI API compatibility
- Works with Ollama, vLLM, llama.cpp, OpenAI, and any OpenAI-compatible API
- Streaming support
- Limited tool use (depends on provider)

## Message Type Details

### Text Response

Simple text response from LLM:

```python
# Backend yields
{"type": "text", "content": "I'll help you implement JWT authentication."}
{"type": "text", "content": " First, let me assess the complexity."}
{"type": "done"}
```

**TUI displays**:
```
I'll help you implement JWT authentication. First, let me assess the complexity.
```

### Tool Use

LLM uses a tool:

```python
# Backend yields
{"type": "text", "content": "I'll assess this directive."}
{
    "type": "tool_use",
    "id": "tool_123",
    "name": "cleave_assess",
    "input": {"directive": "Add JWT authentication"}
}
```

**TUI displays**:
```
I'll assess this directive.

[Tool: cleave_assess]
Parameters:
  directive: "Add JWT authentication"
```

### Tool Result

Tool execution result:

```python
# Backend yields (from tool execution)
{
    "type": "tool_result",
    "tool_use_id": "tool_123",
    "content": "Pattern: Authentication System (0.85)\nComplexity: 6.0"
}
```

**TUI displays**:
```
[Output]
Pattern: Authentication System (0.85)
Complexity: 6.0
```

### Error Handling

Errors during streaming:

```python
# Backend yields
{
    "type": "error",
    "error": "Connection timeout",
    "code": "timeout"
}
```

**TUI displays**:
```
Error: Connection timeout
```

## Testing Backends

### Unit Tests

```python
# tests/tui/test_backend.py

import pytest
from cleave.tui.backends.my_backend import MyBackend

@pytest.mark.asyncio
async def test_backend_send_message():
    """Test backend message sending."""
    config = {
        "base_url": "http://localhost:8000",
        "default_model": "test-model"
    }
    backend = MyBackend(config)

    messages = [{"role": "user", "content": "Hello"}]
    chunks = []

    async for chunk in backend.send_message(messages):
        chunks.append(chunk)

    # Verify chunks
    assert len(chunks) > 0
    assert chunks[-1]["type"] == "done"

    await backend.close()
```

### Integration Tests

```python
@pytest.mark.integration
@pytest.mark.asyncio
async def test_backend_tool_use():
    """Test backend tool use support."""
    backend = MyBackend(config)

    messages = [{"role": "user", "content": "Assess this directive"}]
    tools = [
        {
            "name": "cleave_assess",
            "description": "Assess directive complexity",
            "input_schema": {
                "type": "object",
                "properties": {
                    "directive": {"type": "string"}
                }
            }
        }
    ]

    tool_uses = []
    async for chunk in backend.send_message(messages, tools=tools):
        if chunk["type"] == "tool_use":
            tool_uses.append(chunk)

    assert len(tool_uses) > 0
    assert tool_uses[0]["name"] == "cleave_assess"

    await backend.close()
```

### Manual Testing

```bash
# Test with TUI
cleave config set backend my-backend
cleave tui

# In TUI, send test messages
> Hello
> /backend my-backend
> Assess this: Add JWT authentication
```

## Best Practices

### 1. Handle Connection Failures Gracefully

```python
async def send_message(self, messages, tools=None):
    try:
        async for chunk in self._stream_completion(messages):
            yield self._convert_chunk(chunk)
    except Exception as e:
        yield {"type": "error", "error": str(e)}
    finally:
        yield {"type": "done"}
```

### 2. Support Cancellation

```python
async def send_message(self, messages, tools=None):
    try:
        async for chunk in self._stream_completion(messages):
            yield self._convert_chunk(chunk)
    except asyncio.CancelledError:
        # Clean up on cancellation
        await self._cleanup()
        raise
```

### 3. Normalize Message Formats

Different providers have different formats. Normalize to Anthropic format:

```python
def _normalize_message(self, msg: Dict) -> Dict:
    """Normalize provider format to Anthropic format."""
    if "message" in msg:
        return {"role": msg["role"], "content": msg["message"]}
    return msg
```

### 4. Log API Calls (Debug Mode)

```python
async def send_message(self, messages, tools=None):
    if self.config.get("debug"):
        logger.debug(f"Sending messages: {messages}")

    async for chunk in self._stream_completion(messages):
        if self.config.get("debug"):
            logger.debug(f"Received chunk: {chunk}")
        yield self._convert_chunk(chunk)
```

### 5. Implement Retry Logic

```python
async def send_message(self, messages, tools=None):
    max_retries = 3
    for attempt in range(max_retries):
        try:
            async for chunk in self._stream_completion(messages):
                yield self._convert_chunk(chunk)
            break
        except Exception as e:
            if attempt == max_retries - 1:
                yield {"type": "error", "error": str(e)}
            await asyncio.sleep(2 ** attempt)  # Exponential backoff
```

## Advanced Features

### Tool Use Support

If your provider supports function calling:

```python
async def send_message(self, messages, tools=None):
    # Convert tools to provider format
    provider_tools = self._convert_tools(tools) if tools else None

    async for chunk in self._stream_completion(messages, tools=provider_tools):
        if chunk.get("type") == "function_call":
            yield {
                "type": "tool_use",
                "id": chunk["id"],
                "name": chunk["function"]["name"],
                "input": chunk["function"]["arguments"]
            }
        else:
            yield self._convert_chunk(chunk)
```

### Streaming Control

Allow stopping/resuming streams:

```python
async def send_message(self, messages, tools=None):
    self._streaming = True

    async for chunk in self._stream_completion(messages):
        if not self._streaming:
            break
        yield self._convert_chunk(chunk)

async def stop_streaming(self):
    """Stop current stream."""
    self._streaming = False
```

### Custom Parameters

Support provider-specific parameters:

```python
async def send_message(self, messages, tools=None, **kwargs):
    # Extract custom parameters
    temperature = kwargs.get("temperature", 0.7)
    top_p = kwargs.get("top_p", 1.0)

    # Pass to provider
    async for chunk in self._stream_completion(
        messages,
        temperature=temperature,
        top_p=top_p
    ):
        yield self._convert_chunk(chunk)
```

## Next Steps

- **Architecture**: [System overview](overview.md)
- **Contributing**: [How to add backends](../guides/contributing.md)
- **Examples**: [TUI backend config](../../examples/tui/custom-backend-config.yaml)
