# llm_service.py Refactoring Spec

## Goal

Break kollabor/llm/llm_service.py (3149 lines, 61 methods, 1 god class) into
7 focused modules. LLMService becomes a thin coordinator (~1200 lines).

All new modules stay in kollabor/llm/ (NOT extracted to packages).
This is internal decomposition, not package extraction.

## Current State

LLMService.__init__ (lines 89-321) creates 50+ instance variables.
Every method is on self. No separation of concerns.

## Extraction Plan

Each extraction:
1. Create new file in kollabor/llm/
2. Move methods from LLMService to new class
3. New class receives deps via constructor
4. LLMService creates instance in __init__, delegates calls
5. External code accessing self.X on LLMService needs forwarding properties

### Module 1: BackgroundTaskManager (phase A)

File: kollabor/llm/background_task_manager.py
Lines to extract from llm_service.py: 722-1173

Methods to move:
  create_background_task       (L722-892,  171 lines)
  _create_task_with_blocking   (L893-920,   28 lines)
  _safe_task_wrapper           (L921-943,   23 lines)
  _task_done_callback          (L944-1005,  62 lines)
  _handle_task_error           (L1006-1070, 65 lines)
  start_task_monitor           (L1071-1075,  5 lines)
  _monitor_tasks               (L1076-1117, 42 lines)
  get_task_status              (L1118-1137, 20 lines)
  cancel_all_tasks             (L1138-1158, 21 lines)
  wait_for_tasks               (L1159-1173, 15 lines)

Instance variables to move (remove from LLMService.__init__):
  _background_tasks            Set[asyncio.Task]
  _task_metadata               Dict[str, Any]
  _max_concurrent_tasks        int
  _task_error_count            int
  _monitoring_task             Optional[asyncio.Task]
  _circuit_breaker_state       str
  _circuit_breaker_failures    int
  _circuit_breaker_last_failure_time  Optional[float]
  _circuit_breaker_test_task_running  bool
  _queue_metrics (partial)     Dict - the task-related keys
  enable_metrics               bool
  _task_metrics                Dict (conditional)

Constructor signature:
  class BackgroundTaskManager:
      def __init__(self, task_config: LLMTaskConfig, default_timeout=None, enable_metrics=False):

Dependencies (passed to constructor or methods):
  task_config                  LLMTaskConfig (from config)
  default_timeout              Optional[float]
  enable_metrics               bool

Internal deps (these are self-contained, no external calls):
  Only uses asyncio, logging, time, datetime

Wiring in LLMService.__init__:
  self._task_manager = BackgroundTaskManager(
      task_config=self.task_config,
      default_timeout=self.default_timeout,
      enable_metrics=self.enable_metrics,
  )

Delegation in LLMService (add forwarding methods or replace calls):
  self.create_background_task(...)  -> self._task_manager.create_background_task(...)
  self.cancel_all_tasks()           -> self._task_manager.cancel_all_tasks()
  self.wait_for_tasks()             -> self._task_manager.wait_for_tasks()
  self.get_task_status()            -> self._task_manager.get_task_status()
  self.start_task_monitor()         -> self._task_manager.start_task_monitor()

External callers that access these:
  - get_status_line() reads _background_tasks, _task_metadata, etc.
    -> change to self._task_manager._background_tasks or add accessor methods
  - shutdown() calls cancel_all_tasks()
    -> change to self._task_manager.cancel_all_tasks()
  - _process_queue, process_user_input, _handle_add_message, _handle_llm_continue
    all call create_background_task()
    -> change to self._task_manager.create_background_task()

Also move from LLMService.__init__ (lines 293-318):
  The instance variable declarations for task tracking, circuit breaker,
  queue metrics. These become BackgroundTaskManager.__init__ internals.

Note: _queue_metrics is shared between BackgroundTaskManager and
_enqueue_with_overflow_strategy. Either:
  a) Move _queue_metrics to BackgroundTaskManager, pass reference to enqueue method
  b) Keep _queue_metrics on LLMService, pass to both
  Option (a) is cleaner.

### Module 2: SystemPromptBuilder (phase B)

File: kollabor/llm/system_prompt_builder.py
Lines to extract: 1174-1402

Methods to move:
  _get_tree_output                     (L1174-1209, 36 lines)
  _finalize_system_prompt              (L1210-1275, 66 lines)
  set_plugin_instances                 (L1276-1286, 11 lines)
  _get_plugin_system_prompt_additions  (L1287-1316, 30 lines)
  _build_system_prompt                 (L1317-1372, 56 lines)
  rebuild_system_prompt                (L1373-1402, 30 lines)

Instance variables to move:
  _plugin_instances    Optional[Dict[str, Any]]

Constructor:
  class SystemPromptBuilder:
      def __init__(self, config, agent_manager=None):

Dependencies:
  config               ConfigService
  agent_manager        AgentManager (optional)
  _plugin_instances    set via set_plugin_instances() after plugins load

External imports needed:
  subprocess           for _get_tree_output (tree/ls commands)
  pathlib.Path
  kollabor.utils.config_utils  (get_system_prompt_content, initialize_system_prompt,
                             get_system_prompt_path)
  kollabor.utils.prompt_renderer  (render_system_prompt)
  kollabor.utils.shell_alias_detector  (format_aliases_for_prompt)

Wiring in LLMService.__init__:
  self._prompt_builder = SystemPromptBuilder(
      config=self.config,
      agent_manager=self.agent_manager,
  )

Delegation:
  self._build_system_prompt()       -> self._prompt_builder.build()
  self.rebuild_system_prompt()      -> self._prompt_builder.rebuild(self.conversation_history)
  self.set_plugin_instances(p)      -> self._prompt_builder.set_plugin_instances(p)

Note: rebuild_system_prompt() modifies conversation_history[0].
  Pass conversation_history as parameter, not stored on builder.
  def rebuild(self, conversation_history): ...

### Module 3: StreamingHandler (phase B)

File: kollabor/llm/streaming_handler.py
Lines to extract: 2672-2928

Methods to move:
  _stream_thinking_content       (L2672-2679,   8 lines)
  _stream_thinking_sentences     (L2680-2693,  14 lines)
  _stream_thinking_width_based   (L2694-2777,  84 lines)
  _handle_streaming_chunk        (L2778-2842,  65 lines)
  _stream_response_chunk         (L2843-2866,  24 lines)
  _call_llm                      (L2867-2908,  42 lines)
  _cleanup_streaming_state       (L2909-2928,  20 lines)

Instance variables to move:
  _streaming_buffer       str
  _in_thinking            bool
  _thinking_buffer        str
  _response_started       bool
  _last_chunk_position    int

Constructor:
  class StreamingHandler:
      def __init__(self, api_service, message_display_service, renderer):

Dependencies:
  api_service              APICommunicationService
  message_display_service  MessageDisplayService
  renderer                 TerminalRenderer (for update_thinking)

Wiring in LLMService.__init__:
  self._streaming = StreamingHandler(
      api_service=self.api_service,
      message_display_service=self.message_display_service,
      renderer=self.renderer,
  )

Delegation:
  self._call_llm(...)                -> self._streaming.call_llm(...)
  self._handle_streaming_chunk(...)  -> self._streaming.handle_chunk(...)
  self._cleanup_streaming_state()    -> self._streaming.cleanup()

Note: _call_llm needs these from LLMService:
  cancel_processing       pass as callback: lambda: self.cancel_processing
  mcp_discovery_complete  pass as parameter
  conversation_history    pass as parameter
  max_history             pass as parameter
  native_tools            pass as parameter

  def call_llm(self, conversation_history, max_history, native_tools,
               mcp_discovery_complete, is_cancelled_fn):

### Module 4: NativeToolsHandler (phase B)

File: kollabor/llm/native_tools_handler.py
Lines to extract: 1403-1554

Methods to move:
  _background_mcp_discovery      (L1403-1423, 21 lines)
  _load_native_tools             (L1424-1465, 42 lines)
  _execute_native_tool_calls     (L1466-1554, 89 lines)

Instance variables to move:
  native_tools                   Optional[List[Dict]]
  native_tool_calling_enabled    bool
  mcp_discovery_complete         asyncio.Event

Constructor:
  class NativeToolsHandler:
      def __init__(self, mcp_integration, profile_manager, api_service, config):

Dependencies:
  mcp_integration          MCPIntegration
  profile_manager          ProfileManager
  api_service              APICommunicationService (for get_last_tool_calls)
  config                   ConfigService

External imports:
  re                       for malformed tool name fixing
  kollabor_agent.tool_executor  ToolExecutionResult

Wiring in LLMService.__init__:
  self._native_tools = NativeToolsHandler(
      mcp_integration=self.mcp_integration,
      profile_manager=self.profile_manager,
      api_service=self.api_service,
      config=self.config,
  )

Delegation:
  self._background_mcp_discovery()   -> self._native_tools.background_discovery()
  self._load_native_tools()          -> self._native_tools.load_tools()
  self._execute_native_tool_calls()  -> self._native_tools.execute_tool_calls()
  self.native_tools                  -> self._native_tools.tools
  self.mcp_discovery_complete        -> self._native_tools.discovery_complete

IMPORTANT: mcp_discovery_complete is referenced by _call_llm (StreamingHandler).
  Pass self._native_tools.discovery_complete to StreamingHandler.call_llm().

### Module 5: SessionManager (phase C)

File: kollabor/llm/session_manager.py
Lines to extract: 359-586

Methods to move:
  _initialize_conversation       (L359-389,  31 lines)
  restart_session                (L390-475,  86 lines)
  _set_conversation_context      (L545-586,  42 lines)

Constructor:
  class SessionManager:
      def __init__(self, conversation_logger, conversation_manager, config, event_bus):

Dependencies:
  conversation_logger     KollaborConversationLogger
  conversation_manager    ConversationManager
  config                  ConfigService
  event_bus               EventBus (for registry.get_all_hooks in _set_conversation_context)

External imports:
  importlib.metadata.version
  pathlib.Path
  kollabor.utils.session_naming.generate_session_name

Note: restart_session accesses api_service.session_id. Pass api_service too,
  or have restart_session return data and let LLMService update api_service.

  Also needs: _build_system_prompt (from SystemPromptBuilder)
  and _add_conversation_message. These are cross-module calls.

  Best approach: SessionManager receives prompt_builder reference:
    def __init__(self, ..., prompt_builder: SystemPromptBuilder):
  And _add_conversation_message is passed as callback or stays on LLMService.

Wiring:
  self._session = SessionManager(
      conversation_logger=self.conversation_logger,
      conversation_manager=self.conversation_manager,
      config=self.config,
      event_bus=self.event_bus,
      prompt_builder=self._prompt_builder,
  )

### Module 6: QueueProcessor (phase D - hardest)

File: kollabor/llm/queue_processor.py
Lines to extract: 587-721, 2050-2671

Methods to move:
  _enqueue_with_overflow_strategy  (L587-721,    135 lines)
  _process_queue                   (L2050-2107,   58 lines)
  _process_message_batch           (L2108-2405,  298 lines)
  _continue_conversation           (L2406-2671,  266 lines)

CRITICAL: _process_message_batch and _continue_conversation are ~250 lines
of near-identical code. DEDUP FIRST by extracting shared logic into
a private method like _execute_llm_turn(messages, is_continuation=False).

Instance variables to move:
  processing_queue        asyncio.Queue
  max_queue_size          int
  dropped_messages        int
  is_processing           bool
  turn_completed          bool
  cancel_processing       bool
  cancellation_message_shown  bool

Constructor:
  class QueueProcessor:
      def __init__(self, task_config, api_service, tool_executor,
                   response_parser, message_display_service, conversation_logger,
                   event_bus, renderer, config):

Dependencies (this is the kitchen sink - it's the core orchestration):
  task_config              LLMTaskConfig
  api_service              APICommunicationService
  tool_executor            ToolExecutor
  response_parser          ResponseParser
  message_display_service  MessageDisplayService
  conversation_logger      KollaborConversationLogger
  event_bus                EventBus
  renderer                 TerminalRenderer
  config                   ConfigService
  streaming_handler        StreamingHandler (for _call_llm)
  native_tools_handler     NativeToolsHandler (for execute_tool_calls)
  task_manager             BackgroundTaskManager (for create_background_task)

Shared state that QueueProcessor needs from LLMService:
  conversation_history     List[ConversationMessage] - pass by reference
  session_stats            Dict - pass by reference
  current_parent_uuid      str - needs getter/setter or shared object
  pending_tools            List - needs getter/setter
  question_gate_active     bool - needs getter/setter
  question_gate_enabled    bool - config value
  stats                    Dict - pass by reference
  current_processing_tokens  int
  processing_start_time      float
  native_tool_calling_enabled  bool

Approach: Create a shared ConversationState object:
  class ConversationState:
      conversation_history: List[ConversationMessage]
      session_stats: Dict
      stats: Dict
      current_parent_uuid: Optional[str]
      pending_tools: List[Dict]
      question_gate_active: bool
      question_gate_enabled: bool
      current_processing_tokens: int
      processing_start_time: Optional[float]
      native_tool_calling_enabled: bool

  Both LLMService and QueueProcessor hold reference to same state object.
  This avoids passing 10+ individual references.

DEDUP STRATEGY for _process_message_batch vs _continue_conversation:
  Shared logic (in order):
    1. Token estimation + thinking animation start
    2. _call_llm
    3. Token usage tracking + session_stats update
    4. Check for native tool calls -> execute
    5. Parse response -> check question gate
    6. Execute XML-parsed tools
    7. Emit post_response hook
    8. Display response
    9. Log assistant message + tool results
    10. Add to conversation history

  Only difference: _process_message_batch adds user messages first,
  _continue_conversation skips that step.

  Extract to:
    async def _execute_llm_turn(self, state, add_user_message=True, messages=None):
        if add_user_message and messages:
            # add user messages to history
        # ... shared logic ...

## Execution Order

Phase A: BackgroundTaskManager [COMPLETE - commit b3e1797]
  Result: llm_service.py 3149 -> 2712 lines (-437)
  New file: kollabor/llm/background_task_manager.py (493 lines)
  Tests: 15/15 passing, 0 regressions
  Pattern established:
    - New class receives deps via constructor (task_config, queue_metrics ref)
    - LLMService creates instance in __init__ as self._task_manager
    - Thin forwarding methods on LLMService preserve public API
    - get_status_line reads from self._task_manager.* directly
    - shutdown delegates to self._task_manager
    - _queue_metrics stays on LLMService (shared by reference)
  Test fixes applied:
    - @pytest_asyncio.fixture for async fixtures
    - Removed broken initialize() from fixture (not needed for task tests)
    - Pass coroutine instances not functions: sample_task() not sample_task
    - Guard task.exception() with task.done() check
    - Mock async methods (conversation_logger, api_service) for shutdown test
    - Mock tool_executor.get_execution_stats() for get_status_line test
  NOTE: Line numbers in the spec below are from ORIGINAL file (3149 lines).
  Current file is 2712 lines. Use method names to locate code, not line numbers.

Phase B: SystemPromptBuilder + StreamingHandler + NativeToolsHandler (parallel file creation)
  Agent 1: Create kollabor/llm/system_prompt_builder.py
  Agent 2: Create kollabor/llm/streaming_handler.py
  Agent 3: Create kollabor/llm/native_tools_handler.py
  Then sequentially wire each into llm_service.py:
    B1. Wire SystemPromptBuilder, remove methods, verify
    B2. Wire StreamingHandler, remove methods, verify
    B3. Wire NativeToolsHandler, remove methods, verify
  Commit after all 3 are wired.

Phase C: SessionManager
  1. Create kollabor/llm/session_manager.py
  2. Wire into llm_service.py
  3. Verify + commit

Phase D: QueueProcessor
  1. Create ConversationState dataclass (kollabor/llm/conversation_state.py)
  2. Create kollabor/llm/queue_processor.py with DEDUPED _execute_llm_turn
  3. Wire into llm_service.py
  4. Replace conversation_history/session_stats/etc with state object
  5. Verify + commit

## Verification After Each Phase

  uv run pytest tests/unit/llm/test_llm_service.py -x -q
  uv run pytest tests/unit/llm/test_task_management.py -x -q
  uv run pytest tests/integration/test_llm_service_e2e.py -x -q
  uv run kollab --version

## What LLMService Keeps After All Extractions (~1200 lines)

  __init__                     create all sub-components, wire deps
  initialize                   async init sequence
  _initialize_provider         provider setup
  switch_profile               profile switching
  process_user_input           entry point -> delegates to queue_processor
  process_user_input_background  entry point -> delegates
  _add_conversation_message    shared utility (used by queue_processor + hooks)
  _handle_context_injection    hook handler
  _handle_user_input           hook handler
  _handle_cancel_request       hook handler
  _handle_add_message          hook handler (126 lines, biggest)
  _handle_llm_continue         hook handler
  register_hooks               hook registration
  cancel_current_request       delegates to api_service
  reload_config                hot reload
  get_status_line              status reporting (reads from sub-components)
  get_queue_metrics            metrics reporting
  reset_queue_metrics          metrics reset
  shutdown                     cleanup all sub-components

## Import Changes

New imports in llm_service.py:
  from .background_task_manager import BackgroundTaskManager
  from .system_prompt_builder import SystemPromptBuilder
  from .streaming_handler import StreamingHandler
  from .native_tools_handler import NativeToolsHandler
  from .session_manager import SessionManager
  from .queue_processor import QueueProcessor, ConversationState

Removed imports (moved to sub-modules):
  subprocess (moves to system_prompt_builder)
  re (moves to native_tools_handler)
  os (moves to streaming_handler)

## Risk Mitigation

1. Each phase is independently committable and rollbackable
2. No external API changes - LLMService maintains same public interface
3. Forwarding methods preserve backward compat for plugins/commands
4. Shared state via ConversationState object avoids split-brain issues
5. Run targeted tests after each phase, not full suite (too many pre-existing failures)
