# Creating a Cleave Backend Plugin

This directory contains templates and examples for creating custom LLM backend plugins for Cleave.

## Quick Start

1. Copy the template:
   ```bash
   cp docs/plugin-template/custom_backend.py ~/.cleave/backends/my_backend.py
   ```

2. Edit the plugin and implement the required methods:
   - `name` - Display name for your backend
   - `connect()` - Initialize connection to your LLM service
   - `disconnect()` - Clean up resources
   - `query()` - Send prompts and stream responses
   - `check_auth()` - Verify authentication is configured
   - `available_models()` - List available models

3. Set plugin metadata at the bottom of the file:
   ```python
   __plugin_name__ = "my-backend"
   __plugin_version__ = "1.0.0"
   __plugin_author__ = "Your Name"
   __plugin_backend__ = MyBackend
   ```

4. Configure Cleave to use your plugin:
   ```bash
   cleave config set backend my-backend
   cleave tui
   ```

## Plugin Structure

A Cleave plugin is a single Python file that defines:

### Required Components

1. **Backend Class** - Implements the `Backend` interface:
   ```python
   from cleave.tui.backends.base import Backend, BackendConfig, BackendState

   class MyBackend(Backend):
       # Implement required methods
       ...
   ```

2. **Plugin Metadata** - Dunder variables at module level:
   ```python
   __plugin_name__ = "my-backend"        # REQUIRED: Unique identifier
   __plugin_version__ = "1.0.0"          # REQUIRED: Semver version
   __plugin_backend__ = MyBackend        # REQUIRED: Backend class
   __plugin_author__ = "Your Name"       # Optional: Author name
   __plugin_description__ = "..."        # Optional: Brief description
   ```

### Backend Interface

The `Backend` abstract base class defines the contract your plugin must implement:

#### Properties

```python
@property
def name(self) -> str:
    """Human-readable backend name (e.g., 'My Custom LLM')."""
    ...

@property
def state(self) -> BackendState:
    """Current connection state (DISCONNECTED, CONNECTING, CONNECTED, ERROR)."""
    ...

@property
def error(self) -> str | None:
    """Last error message, if any."""
    ...
```

#### Methods

```python
async def connect(self, config: BackendConfig) -> None:
    """Establish connection to the backend.

    Args:
        config: Configuration including model, API keys, timeouts, etc.

    Raises:
        ConnectionError: If connection fails.
    """
    ...

async def disconnect(self) -> None:
    """Close the backend connection. Safe to call multiple times."""
    ...

async def query(self, prompt: str) -> AsyncIterator[Message]:
    """Send a prompt and stream responses.

    Args:
        prompt: User message to send.

    Yields:
        Message objects as they arrive.

    Raises:
        RuntimeError: If not connected.
        ConnectionError: If connection is lost.
    """
    ...

@classmethod
def check_auth(cls) -> tuple[bool, str]:
    """Check if authentication is configured.

    Returns:
        Tuple of (is_authenticated, status_message).
    """
    ...

@classmethod
def available_models(cls, config: BackendConfig | None = None) -> list[str]:
    """List models available from this backend.

    Args:
        config: Optional config for dynamic model discovery.

    Returns:
        List of model identifiers.
    """
    ...

@classmethod
def default_config(cls) -> BackendConfig:
    """Get default configuration for this backend.

    Returns:
        BackendConfig with sensible defaults.
    """
    ...
```

## Streaming Support

For the best user experience, implement streaming in your `query()` method:

```python
async def query(self, prompt: str) -> AsyncIterator[Message]:
    # Signal stream start
    yield Message(
        role="assistant",
        content="",
        message_type=MessageType.STREAM_START,
    )

    # Stream chunks
    full_response = ""
    async for chunk in your_api.stream(prompt):
        full_response += chunk
        yield Message(
            role="assistant",
            content=chunk,
            message_type=MessageType.STREAM_DELTA,
        )

    # Signal stream end with full content
    yield Message(
        role="assistant",
        content=full_response,
        message_type=MessageType.STREAM_END,
    )
```

Non-streaming implementations should yield a single `COMPLETE` message:

```python
async def query(self, prompt: str) -> AsyncIterator[Message]:
    response = await your_api.complete(prompt)
    yield Message(
        role="assistant",
        content=response,
        message_type=MessageType.COMPLETE,
    )
```

## Configuration

### Backend Configuration

The `BackendConfig` dataclass provides standard configuration:

```python
@dataclass
class BackendConfig:
    model: str = ""                      # Model identifier
    base_url: str | None = None          # API endpoint (for custom servers)
    api_key: str | None = None           # API key
    timeout: float = 120.0               # Request timeout (seconds)
    max_turns: int = 50                  # Maximum conversation turns
    system_prompt: str | None = None     # System prompt
    allowed_tools: list[str] | None = None      # Tool filtering
    disallowed_tools: list[str] | None = None
    working_directory: str | None = None        # Working directory for tool calls
```

### User Configuration

Users can configure plugin settings via CLI:

```bash
# Set active backend
cleave config set backend my-backend

# Set backend-specific config
cleave config set backends.my-backend.model "my-model-v2"
cleave config set backends.my-backend.api_key "sk-..."
cleave config set backends.my-backend.base_url "https://api.example.com"

# View current config
cleave config get backend
cleave config get backends.my-backend
```

## Examples

### Example 1: OpenAI-Compatible API

Many local inference servers (Ollama, vLLM, llama.cpp) implement the OpenAI API:

```python
import os
from collections.abc import AsyncIterator
from openai import AsyncOpenAI
from cleave.tui.backends.base import Backend, BackendConfig, BackendState, Message

class LocalLLMBackend(Backend):
    def __init__(self):
        self._state = BackendState.DISCONNECTED
        self._client = None

    @property
    def name(self) -> str:
        return "Local LLM"

    @property
    def state(self) -> BackendState:
        return self._state

    async def connect(self, config: BackendConfig) -> None:
        self._state = BackendState.CONNECTING
        self._client = AsyncOpenAI(
            base_url=config.base_url or "http://localhost:11434/v1",
            api_key=config.api_key or "dummy",  # Some local servers require any key
        )
        self._state = BackendState.CONNECTED

    async def disconnect(self) -> None:
        self._client = None
        self._state = BackendState.DISCONNECTED

    async def query(self, prompt: str) -> AsyncIterator[Message]:
        stream = await self._client.chat.completions.create(
            model=self._config.model or "llama3.2",
            messages=[{"role": "user", "content": prompt}],
            stream=True,
        )

        async for chunk in stream:
            if chunk.choices[0].delta.content:
                yield Message(
                    role="assistant",
                    content=chunk.choices[0].delta.content,
                    message_type=MessageType.STREAM_DELTA,
                )

    @classmethod
    def check_auth(cls) -> tuple[bool, str]:
        return (True, "Local server (no auth required)")

    @classmethod
    def available_models(cls, config: BackendConfig | None = None) -> list[str]:
        return ["llama3.2", "codellama", "mistral"]

__plugin_name__ = "local-llm"
__plugin_version__ = "1.0.0"
__plugin_backend__ = LocalLLMBackend
```

### Example 2: REST API Backend

For simple REST APIs:

```python
import httpx
from cleave.tui.backends.base import Backend, Message

class RestAPIBackend(Backend):
    async def connect(self, config: BackendConfig) -> None:
        self._client = httpx.AsyncClient(
            base_url=config.base_url,
            headers={"Authorization": f"Bearer {config.api_key}"},
        )

    async def query(self, prompt: str) -> AsyncIterator[Message]:
        response = await self._client.post(
            "/v1/completions",
            json={"prompt": prompt, "max_tokens": 1000},
        )
        data = response.json()

        yield Message(
            role="assistant",
            content=data["completion"],
            message_type=MessageType.COMPLETE,
        )
```

## Testing Your Plugin

### Manual Testing

1. Install your plugin:
   ```bash
   cp my_backend.py ~/.cleave/backends/
   ```

2. Verify discovery:
   ```bash
   cleave config get backends  # Should list your plugin
   ```

3. Test in TUI:
   ```bash
   cleave config set backend my-backend
   cleave tui
   ```

### Unit Testing

Create tests for your plugin:

```python
# tests/test_my_backend.py
import pytest
from my_backend import MyBackend
from cleave.tui.backends.base import BackendConfig

@pytest.mark.asyncio
async def test_connect():
    backend = MyBackend()
    config = BackendConfig(api_key="test-key")
    await backend.connect(config)
    assert backend.state == BackendState.CONNECTED

@pytest.mark.asyncio
async def test_query():
    backend = MyBackend()
    await backend.connect(BackendConfig())

    messages = []
    async for msg in backend.query("Hello"):
        messages.append(msg)

    assert len(messages) > 0
    assert messages[-1].message_type == MessageType.STREAM_END
```

## Error Handling

### Connection Errors

Raise `ConnectionError` from `connect()` if initialization fails:

```python
async def connect(self, config: BackendConfig) -> None:
    if not config.api_key:
        raise ConnectionError("API key required")

    try:
        await self._initialize()
    except Exception as e:
        self._state = BackendState.ERROR
        self._error = str(e)
        raise ConnectionError(f"Initialization failed: {e}") from e
```

### Query Errors

Yield error messages instead of raising exceptions:

```python
async def query(self, prompt: str) -> AsyncIterator[Message]:
    try:
        # ... normal processing
        yield Message(role="assistant", content=response)
    except Exception as e:
        # Don't raise - yield error message instead
        yield Message(
            role="assistant",
            content=f"Error: {e}",
            metadata={"error": True},
        )
```

## Security Considerations

1. **API Keys**: Never hardcode API keys. Use environment variables or config:
   ```python
   api_key = config.api_key or os.getenv("MY_API_KEY")
   ```

2. **Input Validation**: Validate user input before sending to external APIs:
   ```python
   if not prompt or len(prompt) > 10000:
       raise ValueError("Invalid prompt")
   ```

3. **Timeouts**: Always use timeouts to prevent hangs:
   ```python
   async with httpx.AsyncClient(timeout=config.timeout) as client:
       # ...
   ```

4. **Error Messages**: Don't expose sensitive information in error messages:
   ```python
   # Bad: yield Message(content=f"API key {api_key} is invalid")
   # Good: yield Message(content="Authentication failed")
   ```

## Plugin Distribution

### Sharing Your Plugin

1. Publish to GitHub:
   ```bash
   # Create repository with your plugin
   git init
   git add my_backend.py README.md
   git commit -m "Initial commit"
   git push
   ```

2. Document installation:
   ```markdown
   ## Installation

   curl -o ~/.cleave/backends/my_backend.py \
     https://raw.githubusercontent.com/you/my-plugin/main/my_backend.py

   cleave config set backend my-backend
   ```

### Publishing to Package Index

For wider distribution, create a Python package:

```
my-cleave-plugin/
├── pyproject.toml
├── README.md
└── src/
    └── my_cleave_plugin/
        └── backend.py
```

Users can then install with:
```bash
pip install my-cleave-plugin
cp $(python -c "import my_cleave_plugin; print(my_cleave_plugin.__file__[:-11] + 'backend.py')") \
   ~/.cleave/backends/
```

## Troubleshooting

### Plugin Not Detected

- Ensure file is in `~/.cleave/backends/`
- Check file has `.py` extension
- Verify `__plugin_name__` and `__plugin_backend__` are set
- Check for syntax errors: `python3 ~/.cleave/backends/my_backend.py`

### Import Errors

- Install required dependencies: `pip install httpx openai`
- Check Python version compatibility (requires Python 3.11+)

### Connection Failures

- Verify API keys are set: `echo $MY_API_KEY`
- Check base URL is correct
- Test API independently: `curl https://api.example.com/health`

### Streaming Issues

- Ensure you yield `STREAM_START`, `STREAM_DELTA`, `STREAM_END` messages
- Check message types are from `MessageType` enum
- Test with non-streaming first, then add streaming

## Additional Resources

- [Cleave Documentation](https://github.com/styrene-lab/cleave)
- [Backend API Reference](../backends.md)
- [Example Plugins](https://github.com/styrene-lab/cleave-plugins)
- [Community Forum](https://github.com/styrene-lab/cleave/discussions)

## Support

- GitHub Issues: https://github.com/styrene-lab/cleave/issues
- Discord: [Join our community]
- Email: support@styrene-lab.dev
