
---
title: LLM Service Architecture
description: Complete LLM service architecture with API communication, tool execution, and conversation management
category: llm-integration
status: review
last_updated: 2025-01-15
version: 1.0.0
tags:
  - level: advanced
  - component: llm
  - status: stable
related:
  - ../architecture/system-architecture.md
  - ../reference/llm-message-flow.md
  - ./tool-execution.md
  - ../../core/llm/
---

# LLM Service Architecture

## Overview

The LLM Service Architecture orchestrates all AI interactions in Kollabor CLI, managing API communication, tool execution, conversation history, profile management, agent orchestration, and MCP integration. It provides a unified interface for LLM requests while handling streaming responses, permission gates, and context management.

## Purpose

This document describes:
- LLMService orchestration and responsibilities
- APICommunicationService for HTTP/streaming
- ToolExecutor for native and MCP tools
- ConversationManager for history and context
- ProfileManager for LLM profiles
- AgentManager for agent discovery and orchestration
- MCPIntegration for MCP server management
- MessageDisplayService for response formatting

## Scope

**Covered**:
- LLMService main coordinator
- APICommunicationService (HTTP, streaming, authentication)
- ToolExecutor (native tools, MCP tools, permissions)
- ConversationManager (history, context window, metadata)
- ProfileManager (profiles, profile switching, environment variables)
- AgentManager (agent discovery, agent orchestration)
- MCPIntegration (MCP servers, tool discovery)
- ResponseParser and ResponseProcessor
- MessageDisplayService

**Not Covered**:
- Input handling (see: [Input Handling System](../core-systems/input-handling.md))
- Message rendering (see: [Terminal Rendering System](../core-systems/terminal-rendering.md))
- Plugin system (see: [Plugin System Architecture](../core-systems/plugin-system.md))

## Prerequisites

- Understanding of LLM API patterns
- Knowledge of streaming responses
- Familiarity with MCP (Model Context Protocol)
- Understanding of tool calling (native and MCP)
- Knowledge of conversation management and context windows

## Table of Contents

- [Architecture Overview](#architecture-overview)
- [LLMService](#llmservice)
- [APICommunicationService](#apicommunicationservice)
- [ToolExecutor](#toolexecutor)
- [ConversationManager](#conversationmanager)
- [ProfileManager](#profilemanager)
- [AgentManager](#agentmanager)
- [MCPIntegration](#mcpintegration)
- [ResponseParser](#responseparser)
- [MessageDisplayService](#messagedisplayservice)
- [Complete Flow](#complete-flow)
- [Examples](#examples)
- [Troubleshooting](#troubleshooting)
- [Related Documents](#related-documents)
- [References](#references)

---

## Architecture Overview

The LLM Service Architecture consists of 7 main components that coordinate AI interactions:

```mermaid
graph TB
    subgraph "LLM Service Architecture"
        LS[LLMService]
        ACS[APICommunicationService]
        TE[ToolExecutor]
        CM[ConversationManager]
        PM[ProfileManager]
        AM[AgentManager]
        MI[MCPIntegration]
        RP[ResponseParser]
        RProc[ResponseProcessor]
        MDS[MessageDisplayService]
    end
    
    subgraph "External"
        LLM[LLM APIs]
        MCP[MCP Servers]
        Agents[Subagents]
        EB[Event Bus]
        TR[Terminal Renderer]
        CL[Conversation Logger]
    end
    
    LS --> ACS
    LS --> TE
    LS --> CM
    LS --> PM
    LS --> AM
    LS --> MI
    
    ACS --> LLM
    TE --> MCP
    TE --> Agents
    TE --> LS
    
    CM --> CL
    CM --> LS
    
    AM --> Agents
    AM --> LS
    
    MI --> MCP
    
    LS --> RP
    RP --> RProc
    RProc --> TE
    
    TE --> MDS
    MDS --> TR
    
    LS --> EB
    EB --> LS
```

### Component Responsibilities

| Component | File | Responsibilities |
|-----------|-------|-----------------|
| **LLMService** | `core/llm/llm_service.py` | Main orchestration, request/response cycle |
| **APICommunicationService** | `core/llm/api_communication_service.py` | HTTP requests, streaming, authentication |
| **ToolExecutor** | `core/llm/tool_executor.py` | Native/MCP tools, permission gates |
| **ConversationManager** | `core/llm/conversation_manager.py` | History, context window, metadata |
| **ProfileManager** | `core/llm/profile_manager.py` | LLM profiles, profile switching |
| **AgentManager** | `core/llm/agent_manager.py` | Agent discovery, orchestration |
| **MCPIntegration** | `core/llm/mcp_integration.py` | MCP servers, tool discovery |
| **ResponseParser** | `core/llm/response_parser.py` | Parse LLM responses |
| **ResponseProcessor** | `core/llm/response_processor.py` | Process tool calls in responses |
| **MessageDisplayService** | `core/llm/message_display_service.py` | Format and display responses |

---

## LLMService

**File**: `core/llm/llm_service.py`

The LLMService is the main orchestrator for all LLM interactions.

### Service Initialization

```python
class LLMService:
    """Orchestrates all LLM interactions."""
    
    def __init__(
        self,
        event_bus: EventBus,
        renderer: TerminalRenderer,
        config: ConfigManager,
        **kwargs
    ):
        """Initialize LLM service.
        
        Args:
            event_bus: Event bus for hook registration
            renderer: Terminal renderer for UI
            config: Configuration manager
            **kwargs: Additional dependencies
        """
        self.event_bus = event_bus
        self.renderer = renderer
        self.config = config
        
        # Initialize specialized components
        self.api_service = APICommunicationService(config)
        self.tool_executor = ToolExecutor(config, event_bus)
        self.conversation_manager = ConversationManager(config)
        self.profile_manager = ProfileManager(config)
        self.agent_manager = AgentManager(config)
        self.mcp_integration = MCPIntegration(config)
        self.response_parser = ResponseParser(config)
        self.response_processor = ResponseProcessor(config)
        self.message_display = MessageDisplayService(config, renderer)
        
        # Request queue and processing
        self.request_queue = asyncio.Queue()
        self.processing = False
```

### User Input Processing

```python
async def _handle_user_input(
    self,
    data: Dict[str, Any],
    event: Event
) -> Dict[str, Any]:
    """Handle USER_INPUT events.
    
    Args:
        data: Event data
        event: Event object
        
    Returns:
        Hook result
    """
    message = data.get('message', '')
    
    # Display user message
    await self.message_display.display_user_message(message)
    
    # Log user message
    await self.conversation_logger.log_user_message(message)
    
    # Enqueue for processing
    await self._enqueue_with_overflow_strategy(message)
    
    return {"queued": True}
```

### Message Queue Management

```python
async def _enqueue_with_overflow_strategy(
    self,
    message: str
) -> None:
    """Enqueue message with overflow handling.
    
    Args:
        message: Message to enqueue
    """
    # Check queue size
    queue_size = self.request_queue.qsize()
    max_size = self.config.get('llm.max_queue_size', 10)
    
    if queue_size >= max_size:
        # Overflow strategy: discard oldest
        self.request_queue.get_nowait()
        logger.warning(f"Queue overflow, discarding oldest request")
    
    # Enqueue message
    await self.request_queue.put(message)
    
    # Start processing if not already
    if not self.processing:
        await self._process_queue()
```

### Request Processing

```python
async def _process_queue(self) -> None:
    """Process message queue."""
    self.processing = True
    
    try:
        while not self.request_queue.empty():
            # Get message
            message = await self.request_queue.get()
            
            # Process message batch
            await self._process_message_batch([message])
    
    finally:
        self.processing = False

async def _process_message_batch(
    self,
    messages: List[str]
) -> None:
    """Process a batch of messages.
    
    Args:
        messages: List of messages to process
    """
    # Add to conversation
    for message in messages:
        self.conversation_manager.add_message(
            role="user",
            content=message
        )
    
    # Start thinking animation
    self.renderer.update_thinking(True, "Processing...")
    thinking_start = time.time()
    
    try:
        # Call LLM API
        response = await self._call_llm()
        
        # Stop thinking animation
        thinking_duration = time.time() - thinking_start
        self.renderer.update_thinking(False)
        
        # Parse response
        parsed = self.response_parser.parse_response(response)
        
        # Execute tools
        tool_results = await self.tool_executor.execute_all_tools(
            parsed.get('tools', [])
        )
        
        # Display complete response
        await self.message_display.display_complete_response(
            thinking_duration=thinking_duration,
            response=parsed.get('text', ''),
            tool_results=tool_results
        )
        
        # Add assistant message to conversation
        self.conversation_manager.add_message(
            role="assistant",
            content=parsed.get('text', ''),
            metadata={
                'tool_calls': parsed.get('tools', []),
                'tool_results': tool_results
            }
        )
        
    except Exception as e:
        # Stop thinking animation
        self.renderer.update_thinking(False)
        
        # Display error
        await self.message_display.display_error(
            f"LLM request failed: {str(e)}"
        )
        
        logger.error(f"LLM request failed: {e}", exc_info=True)
```

---

## APICommunicationService

**File**: `core/llm/api_communication_service.py`

The APICommunicationService handles all HTTP communication with LLM APIs.

### API Request

```python
class APICommunicationService:
    """Handles LLM API communication."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
        self.session = None
        
        # Load API configuration
        self.api_url = config.get('core.llm.api_url')
        self.api_key = config.get('core.llm.api_key') or self._get_env_key()
        self.model = config.get('core.llm.model')
        self.temperature = config.get('core.llm.temperature', 0.7)
        self.max_tokens = config.get('core.llm.max_tokens', 4000)
    
    async def _call_llm(
        self,
        messages: List[Dict[str, str]],
        stream: bool = False
    ) -> Dict[str, Any]:
        """Call LLM API.
        
        Args:
            messages: Message history
            stream: Whether to stream response
            
        Returns:
            API response dict
        """
        # Create session if needed
        if not self.session:
            self.session = aiohttp.ClientSession()
        
        # Prepare request
        request_data = {
            'model': self.model,
            'messages': messages,
            'temperature': self.temperature,
            'max_tokens': self.max_tokens,
            'stream': stream
        }
        
        headers = {
            'Content-Type': 'application/json',
            'Authorization': f'Bearer {self.api_key}'
        }
        
        # Make request
        async with self.session.post(
            self.api_url,
            json=request_data,
            headers=headers
        ) as response:
            if response.status != 200:
                error_text = await response.text()
                raise Exception(f"API error {response.status}: {error_text}")
            
            if stream:
                # Handle streaming response
                return await self._handle_streaming_response(response)
            else:
                # Handle regular response
                return await response.json()
    
    async def _handle_streaming_response(
        self,
        response: aiohttp.ClientResponse
    ) -> Dict[str, Any]:
        """Handle streaming API response.
        
        Args:
            response: Streaming response
            
        Returns:
            Complete response dict
        """
        full_text = ""
        
        async for line in response.content:
            line_text = line.decode('utf-8')
            
            # Parse SSE format
            if line_text.startswith('data: '):
                data_text = line_text[6:]
                
                if data_text == '[DONE]':
                    break
                
                try:
                    data = json.loads(data_text)
                    
                    # Extract content
                    if 'delta' in data:
                        delta = data['delta']
                        if 'content' in delta:
                            content = delta['content']
                            full_text += content
                            
                            # Update thinking display with content
                            self.renderer.update_thinking(
                                True,
                                content[-100:]  # Show last 100 chars
                            )
                
                except json.JSONDecodeError:
                    continue
        
        return {
            'content': full_text,
            'streamed': True
        }
```

### Environment Variable Handling

```python
def _get_env_key(self) -> Optional[str]:
    """Get API key from environment variable.
    
    Returns:
        API key or None
    """
    import os
    
    # Check for various env var names
    env_vars = [
        'KOLLABOR_CLAUDE_TOKEN',
        'ANTHROPIC_API_KEY',
        'OPENAI_API_KEY'
    ]
    
    for var in env_vars:
        value = os.environ.get(var)
        if value:
            logger.info(f"Using API key from {var}")
            return value
    
    return None
```

---

## ToolExecutor

**File**: `core/llm/tool_executor.py`

The ToolExecutor handles execution of both native tools and MCP tools with permission gates.

### Tool Execution Flow

```python
class ToolExecutor:
    """Executes LLM tool calls."""
    
    def __init__(
        self,
        config: ConfigManager,
        event_bus: EventBus
    ):
        self.config = config
        self.event_bus = event_bus
        
        # Native tool handlers
        self.native_tools = {
            'terminal': self._handle_terminal_tool,
            'file_read': self._handle_file_read_tool,
            'file_write': self._handle_file_write_tool,
            'grep': self._handle_grep_tool,
            'file_search': self._handle_file_search_tool
        }
        
        # Permission gates
        self.permission_required = config.get(
            'llm.tool_permission_required',
            False
        )
        
        # Pending approvals
        self.pending_approvals = {}
    
    async def execute_all_tools(
        self,
        tools: List[Dict[str, Any]]
    ) -> List[Dict[str, Any]]:
        """Execute all tools from LLM response.
        
        Args:
            tools: List of tool calls from LLM response
            
        Returns:
            List of tool execution results
        """
        results = []
        
        for tool in tools:
            result = await self._execute_single_tool(tool)
            results.append(result)
        
        return results
    
    async def _execute_single_tool(
        self,
        tool: Dict[str, Any]
    ) -> Dict[str, Any]:
        """Execute a single tool.
        
        Args:
            tool: Tool call dict with 'name' and 'parameters'
            
        Returns:
            Tool execution result
        """
        tool_name = tool.get('name')
        parameters = tool.get('parameters', {})
        
        # Emit tool execution event
        await self.event_bus.emit_with_hooks(
            EventType.TOOL_CALL,
            {
                'tool_name': tool_name,
                'parameters': parameters
            },
            source="tool_executor"
        )
        
        # Check for permission gate
        if self.permission_required:
            approved = await self._request_permission(tool_name, parameters)
            if not approved:
                return {
                    'tool': tool_name,
                    'status': 'cancelled',
                    'result': 'Tool execution cancelled by user'
                }
        
        # Execute tool
        try:
            if tool_name in self.native_tools:
                result = await self.native_tools[tool_name](parameters)
            else:
                # Check for MCP tool
                result = await self._execute_mcp_tool(tool_name, parameters)
            
            return {
                'tool': tool_name,
                'status': 'success',
                'result': result
            }
        
        except Exception as e:
            logger.error(f"Tool {tool_name} failed: {e}", exc_info=True)
            return {
                'tool': tool_name,
                'status': 'error',
                'error': str(e)
            }
```

### Permission Gates

```python
async def _request_permission(
    self,
    tool_name: str,
    parameters: Dict[str, Any]
) -> bool:
    """Request user permission for tool execution.
    
    Args:
        tool_name: Name of tool
        parameters: Tool parameters
        
    Returns:
        True if approved, False if denied
    """
    # Display permission request
    permission_id = str(uuid.uuid4())
    
    await self.message_display.display_permission_request(
        tool_name=tool_name,
        parameters=parameters,
        permission_id=permission_id
    )
    
    # Wait for user response
    self.pending_approvals[permission_id] = None
    
    try:
        # Wait for approval (timeout after 60s)
        await asyncio.wait_for(
            self._wait_for_approval(permission_id),
            timeout=60.0
        )
        
        return self.pending_approvals[permission_id]
    
    except asyncio.TimeoutError:
        # Timeout = deny
        return False

async def _wait_for_approval(
    self,
    permission_id: str
) -> None:
    """Wait for user approval.
    
    Args:
        permission_id: Permission request ID
    """
    # Wait for event that sets approval
    await self.event_bus.wait_for_event(
        EventType.PERMISSION_RESPONSE,
        lambda data: data.get('permission_id') == permission_id
    )
    
    # Get approval from event data
    approval = self.pending_approvals.get(permission_id)
    logger.info(f"Permission {permission_id}: {'approved' if approval else 'denied'}")
```

### Native Tool Handlers

```python
async def _handle_terminal_tool(
    self,
    parameters: Dict[str, Any]
) -> str:
    """Handle terminal tool.
    
    Args:
        parameters: {'command': str}
        
    Returns:
        Command output
    """
    command = parameters.get('command', '')
    
    # Execute command
    result = subprocess.run(
        command,
        shell=True,
        capture_output=True,
        text=True
    )
    
    # Format output
    output = result.stdout
    if result.stderr:
        output += f"\n[stderr]\n{result.stderr}"
    
    output += f"\n[exit code: {result.returncode}]"
    
    return output

async def _handle_file_read_tool(
    self,
    parameters: Dict[str, Any]
) -> str:
    """Handle file read tool.
    
    Args:
        parameters: {'filepath': str}
        
    Returns:
        File content
    """
    filepath = parameters.get('filepath', '')
    
    # Read file
    with open(filepath, 'r') as f:
        content = f.read()
    
    return content

async def _handle_file_write_tool(
    self,
    parameters: Dict[str, Any]
) -> str:
    """Handle file write tool.
    
    Args:
        parameters: {'filepath': str, 'content': str}
        
    Returns:
        Success message
    """
    filepath = parameters.get('filepath', '')
    content = parameters.get('content', '')
    
    # Write file
    with open(filepath, 'w') as f:
        f.write(content)
    
    return f"File written: {filepath}"
```

### MCP Tool Execution

```python
async def _execute_mcp_tool(
    self,
    tool_name: str,
    parameters: Dict[str, Any]
) -> str:
    """Execute MCP tool.
    
    Args:
        tool_name: Name of MCP tool
        parameters: Tool parameters
        
    Returns:
        Tool result
    """
    # Find MCP server for tool
    mcp_servers = self.mcp_integration.get_available_servers()
    
    for server in mcp_servers:
        if tool_name in server.get('tools', []):
            # Execute tool on server
            result = await self.mcp_integration.execute_tool(
                server_name=server['name'],
                tool_name=tool_name,
                parameters=parameters
            )
            
            return result
    
    raise Exception(f"Tool not found in any MCP server: {tool_name}")
```

---

## ConversationManager

**File**: `core/llm/conversation_manager.py`

The ConversationManager manages conversation history, context window, and metadata.

### Conversation History

```python
class ConversationManager:
    """Manages conversation history and context."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
        self.messages = []
        self.max_history = config.get('llm.max_history', 90)
        self.context_window = config.get('llm.context_window', 100000)  # tokens
    
    def add_message(
        self,
        role: str,
        content: str,
        metadata: Dict[str, Any] = None
    ) -> None:
        """Add message to conversation.
        
        Args:
            role: Message role ('user', 'assistant', 'system')
            content: Message content
            metadata: Optional metadata
        """
        message = {
            'role': role,
            'content': content,
            'metadata': metadata or {},
            'timestamp': datetime.now().isoformat()
        }
        
        self.messages.append(message)
        
        # Trim to max history
        if len(self.messages) > self.max_history:
            self.messages = self.messages[-self.max_history:]
    
    def get_messages(
        self,
        include_system: bool = True
    ) -> List[Dict[str, str]]:
        """Get conversation history for API.
        
        Args:
            include_system: Whether to include system prompt
            
        Returns:
            List of message dicts
        """
        messages = []
        
        # Add system prompt if requested
        if include_system:
            system_prompt = self.config.get('llm.system_prompt', '')
            if system_prompt:
                messages.append({
                    'role': 'system',
                    'content': system_prompt
                })
        
        # Add conversation messages
        messages.extend(self.messages)
        
        return messages
```

### Context Window Management

```python
def _trim_to_context_window(self, messages: List[Dict]) -> List[Dict]:
    """Trim messages to fit context window.
    
    Args:
        messages: List of messages
        
    Returns:
            Trimmed list
    """
    # Calculate token count
    total_tokens = self._count_tokens(messages)
    
    if total_tokens <= self.context_window:
        return messages
    
    # Trim from beginning until fits
    trimmed = messages.copy()
    
    while total_tokens > self.context_window and len(trimmed) > 1:
        # Remove oldest non-system message
        for i, msg in enumerate(trimmed):
            if msg['role'] != 'system':
                removed = trimmed.pop(i)
                total_tokens -= self._count_tokens([removed])
                break
    
    logger.warning(f"Trimmed conversation from {len(messages)} to {len(trimmed)} messages")
    
    return trimmed

def _count_tokens(self, messages: List[Dict]) -> int:
    """Estimate token count for messages.
    
    Args:
        messages: List of messages
        
    Returns:
        Estimated token count
    """
    # Simple estimation: ~4 characters per token
    total_chars = sum(len(msg['content']) for msg in messages)
    return total_chars // 4
```

### Metadata Management

```python
def get_conversation_metadata(self) -> Dict[str, Any]:
    """Get conversation metadata.
    
    Returns:
        Dict with metadata
    """
    return {
        'message_count': len(self.messages),
        'user_message_count': len([
            m for m in self.messages
            if m['role'] == 'user'
        ]),
        'assistant_message_count': len([
            m for m in self.messages
            if m['role'] == 'assistant'
        ]),
        'tool_call_count': sum(
            len(m.get('metadata', {}).get('tool_calls', []))
            for m in self.messages
            if m['role'] == 'assistant'
        ),
        'estimated_tokens': self._count_tokens(self.messages)
    }
```

---

## ProfileManager

**File**: `core/llm/profile_manager.py`

The ProfileManager manages LLM profiles for different AI providers.

### Profile Management

```python
class ProfileManager:
    """Manages LLM profiles."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
        self.profiles = {}
        self.active_profile = None
        
        # Load profiles
        self._load_profiles()
    
    def _load_profiles(self) -> None:
        """Load LLM profiles from configuration."""
        self.profiles = self.config.get('llm.profiles', {})
        
        # Set active profile
        active_name = self.config.get('llm.active_profile', 'default')
        self.active_profile = self.profiles.get(active_name, {})
    
    def get_active_profile(self) -> Dict[str, Any]:
        """Get active profile configuration.
        
        Returns:
            Active profile dict
        """
        return self.active_profile
    
    def switch_profile(self, profile_name: str) -> bool:
        """Switch to different profile.
        
        Args:
            profile_name: Name of profile to switch to
            
        Returns:
            True if successful, False otherwise
        """
        if profile_name not in self.profiles:
            logger.error(f"Profile not found: {profile_name}")
            return False
        
        self.active_profile = self.profiles[profile_name]
        
        # Update configuration
        self.config.set('llm.active_profile', profile_name)
        
        # Update API service configuration
        self.api_service.update_config(self.active_profile)
        
        logger.info(f"Switched to profile: {profile_name}")
        return True
```

### Profile Configuration

```python
@staticmethod
def get_default_config() -> Dict[str, Any]:
    """Get default profile configuration."""
    return {
        'llm': {
            'active_profile': 'default',
            'profiles': {
                'default': {
                    'api_url': 'http://localhost:1234/v1/chat/completions',
                    'model': 'qwen/qwen3-4b',
                    'temperature': 0.7,
                    'max_tokens': 4000,
                    'context_window': 100000
                },
                'claude': {
                    'api_url': 'https://api.anthropic.com/v1/messages',
                    'model': 'claude-3-sonnet-20240229',
                    'temperature': 0.7,
                    'max_tokens': 4000,
                    'context_window': 200000
                },
                'gpt4': {
                    'api_url': 'https://api.openai.com/v1/chat/completions',
                    'model': 'gpt-4',
                    'temperature': 0.7,
                    'max_tokens': 4000,
                    'context_window': 8192
                }
            },
            'max_history': 90,
            'system_prompt': 'You are a helpful AI assistant.'
        }
    }
```

### Environment Variable Profiles

```python
def get_env_profile_config(self) -> Optional[Dict[str, Any]]:
    """Get profile from environment variables.
    
    Returns:
            Profile dict from env vars or None
    """
    import os
    
    # Check for LLM API URL
    api_url = os.environ.get('LLM_API_URL')
    if not api_url:
        return None
    
    # Build profile from env vars
    profile = {
        'api_url': api_url,
        'api_key': os.environ.get('LLM_API_KEY'),
        'model': os.environ.get('LLM_MODEL', 'default'),
        'temperature': float(os.environ.get('LLM_TEMPERATURE', '0.7')),
        'max_tokens': int(os.environ.get('LLM_MAX_TOKENS', '4000'))
    }
    
    return profile
```

---

## AgentManager

**File**: `core/llm/agent_manager.py`

The AgentManager manages subagent discovery and orchestration.

### Agent Discovery

```python
class AgentManager:
    """Manages subagent discovery and orchestration."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
        self.agents = {}
        
        # Discover agents
        self._discover_agents()
    
    def _discover_agents(self) -> None:
        """Discover available subagents."""
        agent_configs = self.config.get('llm.agents', {})
        
        for agent_name, agent_config in agent_configs.items():
            self.agents[agent_name] = {
                'name': agent_name,
                'type': agent_config.get('type', 'general'),
                'specialization': agent_config.get('specialization', 'general'),
                'config': agent_config.get('config', {}),
                'enabled': agent_config.get('enabled', True)
            }
        
        logger.info(f"Discovered {len(self.agents)} agents")
    
    def get_available_agents(self) -> List[Dict[str, Any]]:
        """Get list of available agents.
        
        Returns:
            List of agent dicts
        """
        return [
            agent for agent in self.agents.values()
            if agent['enabled']
        ]
```

### Agent Orchestration

```python
async def orchestrate_agent(
    self,
    agent_name: str,
    task: str,
    context: Dict[str, Any] = None
) -> Dict[str, Any]:
    """Orchestrate agent task.
        
    Args:
        agent_name: Name of agent to use
        task: Task description
        context: Additional context
        
    Returns:
        Agent result
    """
    if agent_name not in self.agents:
        raise Exception(f"Agent not found: {agent_name}")
    
    agent = self.agents[agent_name]
    
    # Emit agent start event
    await self.event_bus.emit_with_hooks(
        EventType.AGENT_START,
        {
            'agent_name': agent_name,
            'task': task,
            'context': context
        },
        source="agent_manager"
    )
    
    try:
        # Execute agent task
        result = await self._execute_agent_task(agent, task, context)
        
        # Emit agent complete event
        await self.event_bus.emit_with_hooks(
            EventType.AGENT_COMPLETE,
            {
                'agent_name': agent_name,
                'task': task,
                'result': result
            },
            source="agent_manager"
        )
        
        return result
    
    except Exception as e:
        # Emit agent error event
        await self.event_bus.emit_with_hooks(
            EventType.AGENT_ERROR,
            {
                'agent_name': agent_name,
                'task': task,
                'error': str(e)
            },
            source="agent_manager"
        )
        
        raise

async def _execute_agent_task(
    self,
    agent: Dict[str, Any],
    task: str,
    context: Dict[str, Any]
) -> Dict[str, Any]:
    """Execute agent task.
    
    Args:
        agent: Agent configuration
        task: Task to execute
        context: Task context
        
    Returns:
        Task result
    """
    # Build agent-specific context
    agent_context = {
        'task': task,
        'specialization': agent['specialization'],
        'config': agent['config'],
        **(context or {})
    }
    
    # Call LLM with agent context
    messages = [
        {
            'role': 'system',
            'content': f"You are a specialized agent for {agent['specialization']}."
        },
        {
            'role': 'user',
            'content': task
        }
    ]
    
    response = await self.api_service.call_llm(messages)
    
    return {
        'agent': agent['name'],
        'task': task,
        'response': response
    }
```

---

## MCPIntegration

**File**: `core/llm/mcp_integration.py`

The MCPIntegration manages MCP servers and tool discovery.

### MCP Server Management

```python
class MCPIntegration:
    """Manages MCP (Model Context Protocol) integration."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
        self.servers = {}
        
        # Load MCP servers
        self._load_servers()
    
    def _load_servers(self) -> None:
        """Load MCP server configurations."""
        server_configs = self.config.get('llm.mcp_servers', {})
        
        for server_name, server_config in server_configs.items():
            self.servers[server_name] = {
                'name': server_name,
                'url': server_config.get('url', ''),
                'transport': server_config.get('transport', 'stdio'),
                'enabled': server_config.get('enabled', True),
                'tools': []  # Will be populated by discovery
            }
        
        logger.info(f"Loaded {len(self.servers)} MCP servers")
    
    def get_available_servers(self) -> List[Dict[str, Any]]:
        """Get list of available MCP servers.
        
        Returns:
            List of server dicts
        """
        return [
            server for server in self.servers.values()
            if server['enabled']
        ]
    
    async def discover_tools(self, server_name: str) -> List[Dict[str, Any]]:
        """Discover tools from MCP server.
        
        Args:
            server_name: Name of MCP server
            
        Returns:
            List of tool definitions
        """
        if server_name not in self.servers:
            raise Exception(f"MCP server not found: {server_name}")
        
        server = self.servers[server_name]
        
        # Connect to server
        connection = await self._connect_to_server(server)
        
        if not connection:
            logger.error(f"Failed to connect to MCP server: {server_name}")
            return []
        
        try:
            # Discover tools
            tools = await self._discover_server_tools(connection)
            
            # Cache tools
            server['tools'] = tools
            
            logger.info(f"Discovered {len(tools)} tools from {server_name}")
            return tools
        
        finally:
            # Close connection
            await connection.close()
    
    async def execute_tool(
        self,
        server_name: str,
        tool_name: str,
        parameters: Dict[str, Any]
    ) -> Any:
        """Execute tool on MCP server.
        
        Args:
            server_name: Name of MCP server
            tool_name: Name of tool to execute
            parameters: Tool parameters
            
        Returns:
            Tool result
        """
        if server_name not in self.servers:
            raise Exception(f"MCP server not found: {server_name}")
        
        server = self.servers[server_name]
        
        # Connect to server
        connection = await self._connect_to_server(server)
        
        if not connection:
            raise Exception(f"Failed to connect to MCP server: {server_name}")
        
        try:
            # Execute tool
            result = await self._execute_server_tool(
                connection,
                tool_name,
                parameters
            )
            
            return result
        
        finally:
            # Close connection
            await connection.close()
```

---

## ResponseParser

**File**: `core/llm/response_parser.py`

The ResponseParser parses LLM responses.

### Response Parsing

```python
class ResponseParser:
    """Parses LLM API responses."""
    
    def __init__(self, config: ConfigManager):
        self.config = config
    
    def parse_response(
        self,
        response: Dict[str, Any]
    ) -> Dict[str, Any]:
        """Parse LLM API response.
        
        Args:
            response: Raw API response
            
        Returns:
            Parsed response with text and tools
        """
        parsed = {
            'text': '',
            'tools': []
        }
        
        # Handle different API formats
        if 'content' in response:
            # Standard format
            content = response['content']
            
            # Parse text and tool calls
            parsed['text'], parsed['tools'] = self._parse_content(content)
        
        elif 'choices' in response:
            # OpenAI format
            choice = response['choices'][0]
            content = choice.get('message', {}).get('content', '')
            
            parsed['text'], parsed['tools'] = self._parse_content(content)
        
        elif 'streamed' in response and response['streamed']:
            # Streaming response
            parsed['text'] = response.get('content', '')
        
        return parsed
    
    def _parse_content(
        self,
        content: str
    ) -> Tuple[str, List[Dict[str, Any]]]:
        """Parse content for text and tool calls.
        
        Args:
            content: Content string
            
        Returns:
            Tuple of (text, tool_calls)
        """
        text = ''
        tool_calls = []
        
        # Parse tool calls in format: <tool_name>(params)</tool_name>
        tool_pattern = r'<(\w+)\(([^)]*)\)</\1>'
        
        current_pos = 0
        for match in re.finditer(tool_pattern, content):
            # Text before tool
            text += content[current_pos:match.start()]
            
            # Extract tool call
            tool_name = match.group(1)
            tool_params = match.group(2)
            
            # Parse parameters
            try:
                parameters = json.loads(tool_params)
            except json.JSONDecodeError:
                parameters = {'raw': tool_params}
            
            tool_calls.append({
                'name': tool_name,
                'parameters': parameters
            })
            
            current_pos = match.end()
        
        # Remaining text
        text += content[current_pos:]
        
        return text, tool_calls
```

---

## MessageDisplayService

**File**: `core/llm/message_display_service.py`

The MessageDisplayService formats and displays LLM responses.

### User Message Display

```python
class MessageDisplayService:
    """Formats and displays LLM messages."""
    
    def __init__(
        self,
        config: ConfigManager,
        renderer: TerminalRenderer
    ):
        self.config = config
        self.renderer = renderer
        self.message_coordinator = MessageCoordinator(renderer)
    
    async def display_user_message(self, message: str) -> None:
        """Display user message.
        
        Args:
            message: User message content
        """
        message_sequence = [
            ("user", message, {})
        ]
        
        await self.message_coordinator.display_message_sequence(message_sequence)
```

### Complete Response Display

```python
async def display_complete_response(
    self,
    thinking_duration: float,
    response: str,
    tool_results: List[Dict[str, Any]]
) -> None:
    """Display complete LLM response.
    
    Args:
        thinking_duration: Time spent thinking
        response: Assistant response text
        tool_results: List of tool execution results
    """
        message_sequence = []
        
        # Add thinking message if significant
        if thinking_duration > 0.1:
            thinking_msg = f"Thought for {thinking_duration:.1f} seconds"
            message_sequence.append(("system", thinking_msg, {}))
        
        # Add assistant response
        if response.strip():
            message_sequence.append(("assistant", response, {}))
        
        # Add tool results
        for tool_result in tool_results:
            if tool_result['status'] == 'success':
                output = f"⏺ {tool_result['tool']}(...)\n✓ Done"
            elif tool_result['status'] == 'error':
                output = f"⏺ {tool_result['tool']}(...)\n✗ Error: {tool_result['error']}"
            else:
                output = f"⏺ {tool_result['tool']}(...)\n{tool_result['status']}"
            
            message_sequence.append(("system", output, {}))
        
        # Display all messages atomically
        await self.message_coordinator.display_message_sequence(message_sequence)
```

### Error Display

```python
async def display_error(self, error_message: str) -> None:
    """Display error message.
    
    Args:
        error_message: Error message to display
    """
    message_sequence = [
        ("error", error_message, {})
    ]
    
    await self.message_coordinator.display_message_sequence(message_sequence)
```

---

## Complete Flow

### From User Input to Response Display

```mermaid
sequenceDiagram
    participant U as User
    participant IH as InputHandler
    participant LS as LLMService
    participant TE as ToolExecutor
    participant ACS as APICommunicationService
    participant LLM as LLM API
    participant MDS as MessageDisplayService
    participant MC as MessageCoordinator
    participant TR as Terminal Renderer
    
    U->>IH: Type message
    IH->>LS: USER_INPUT event
    LS->>MDS: display_user_message()
    MDS->>MC: display_message_sequence()
    MC->>TR: print("> message")
    
    LS->>LS: Enqueue request
    LS->>LS: Process queue
    LS->>ACS: call_llm(messages)
    
    Note over TR: Thinking animation...
    
    ACS->>LLM: POST /chat/completions
    LLM-->>ACS: Response
    
    ACS-->>LS: response
    
    Note over TR: Stop thinking
    
    LS->>LS: parse_response()
    LS->>TE: execute_all_tools(tools)
    
    TE->>TE: check permission
    alt Permission required
        TE->>MDS: display_permission_request()
        MDS->>TR: print("Approve tool?")
        U->>TR: Press 'y'
        TR-->>TE: PERMISSION_RESPONSE event
    end
    
    TE->>TE: execute_tool()
    TE-->>LS: tool_results
    
    LS->>MDS: display_complete_response()
    MDS->>MC: display_message_sequence()
    MC->>TR: print("∴ response")
    MC->>TR: print("⏺ tool(...)")
    MC->>TR: print("✓ Done")
```

---

## Examples

### Example 1: Using Native Tools

```python
# LLM response with native tool calls
response = """
Let me check the file contents.

<file_read(filepath="README.md")</file_read>

Now let me search for patterns.

<grep(pattern="TODO", path=".")</grep>
"""

# Parser extracts:
{
    'text': 'Let me check the file contents.\n\nNow let me search for patterns.',
    'tools': [
        {
            'name': 'file_read',
            'parameters': {'filepath': 'README.md'}
        },
        {
            'name': 'grep',
            'parameters': {'pattern': 'TODO', 'path': '.'}
        }
    ]
}

# Tools execute and results display:
⏺ file_read(filepath="README.md")
✓ Done
[file content here...]

⏺ grep(pattern="TODO", path=".")
✓ Done
[grep results here...]
```

### Example 2: Profile Switching

```python
# Switch to Claude profile
profile_manager.switch_profile('claude')
# API configuration updated to Claude API

# Switch to GPT-4 profile
profile_manager.switch_profile('gpt4')
# API configuration updated to OpenAI API
```

### Example 3: Agent Orchestration

```python
# Use code reviewer agent
result = await agent_manager.orchestrate_agent(
    agent_name='code_reviewer',
    task='Review this code for potential bugs',
    context={
        'file': 'app.py',
        'language': 'python'
    }
)

# Agent responds with specialized analysis
```

---

## Troubleshooting

### Issue: API Connection Errors

**Symptoms**: "API error" or "Connection refused"

**Diagnosis**:

```python
# Check API configuration
api_url = config.get('core.llm.api_url')
api_key = config.get('core.llm.api_key') or os.environ.get('LLM_API_KEY')

print(f"API URL: {api_url}")
print(f"API Key set: {bool(api_key)}")

# Test connection
import aiohttp
async with aiohttp.ClientSession() as session:
    async with session.get(api_url) as response:
        print(f"Status: {response.status}")
```

**Solutions**:
1. Verify API URL is correct
2. Check API key is set (env var or config)
3. Test network connectivity
4. Check API service is running

### Issue: Tool Execution Fails

**Symptoms**: Tool returns error or "Tool not found"

**Diagnosis**:

```python
# Check tool is registered
tool_name = 'my_tool'
is_native = tool_name in tool_executor.native_tools

print(f"Native tool: {is_native}")

# Check MCP servers
servers = mcp_integration.get_available_servers()
for server in servers:
    if tool_name in server.get('tools', []):
        print(f"Found in MCP server: {server['name']}")
```

**Solutions**:
1. Verify tool name matches registered tools
2. Check MCP server is connected
3. Check tool parameters are correct
4. Review tool execution logs

### Issue: Context Window Exceeded

**Symptoms**: "Conversation too long" or trimmed messages

**Diagnosis**:

```python
# Check conversation size
metadata = conversation_manager.get_conversation_metadata()
print(f"Message count: {metadata['message_count']}")
print(f"Estimated tokens: {metadata['estimated_tokens']}")

print(f"Max history: {config.get('llm.max_history')}")
print(f"Context window: {config.get('llm.context_window')}")
```

**Solutions**:
1. Increase context window in config
2. Reduce max_history
3. Clear old messages
4. Use summary tools for long conversations

---

## Related Documents

- **[LLM Message Flow](../reference/llm-message-flow.md)** - Complete message display flow
- **[Input Handling System](../core-systems/input-handling.md)** - Input processing
- **[Terminal Rendering System](../core-systems/terminal-rendering.md)** - Message display
- **[Tool Execution System](./tool-execution.md)** - Detailed tool execution

---

## References

**Source Files**:
- `core/llm/llm_service.py` - Main LLM service
- `core/llm/api_communication_service.py` - API communication
- `core/llm/tool_executor.py` - Tool execution
- `core/llm/conversation_manager.py` - Conversation management
- `core/llm/profile_manager.py` - Profile management
- `core/llm/agent_manager.py` - Agent orchestration
- `core/llm/mcp_integration.py` - MCP integration
- `core/llm/response_parser.py` - Response parsing
- `core/llm/message_display_service.py` - Message display

**Related Specifications**:
- `docs/specs/documentation-overhaul-spec.md` - Documentation specification

---

*Last Updated: 2025-01-15*  
*Status: Review - Ready for technical review*  
*Version: 1.0.0*
