# Implementation Guide for Custom Context Window Size

This document outlines the steps to implement the custom context window size feature requested in issue #241.

## Background

Users have reported issues with models that have smaller context windows (like some Llama.cpp and LM Studio models) failing because the default token limit (30,000) exceeds their capabilities.

Issue: [Custom context size #241](https://github.com/LearningCircuit/local-deep-research/issues/241)

## Implementation Steps

### 1. Add New Setting

Add a new setting in the default settings JSON file:

```json
"llm.context_window_size": {
    "category": "llm_parameters",
    "description": "Maximum context window size in tokens for the LLM",
    "editable": true,
    "max_value": 128000.0,
    "min_value": 512.0,
    "name": "Context Window Size",
    "options": null,
    "step": null,
    "type": "LLM",
    "ui_element": "number",
    "value": 32000,
    "visible": true
}
```

### 2. Modify the LLM Configuration

Update the `get_llm` function in `src/local_deep_research/config/llm_config.py`:

1. Add context window size to common parameters:

```python
# Common parameters for all models
common_params = {
    "temperature": temperature,
}

# Get context window size from settings
context_window_size = get_db_setting("llm.context_window_size", 32000)

# Ensure max_tokens doesn't exceed context window size
if get_db_setting("llm.supports_max_tokens", True):
    # Use 80% of context window to leave room for prompts
    max_tokens = min(get_db_setting("llm.max_tokens", 30000), int(context_window_size * 0.8))
    common_params["max_tokens"] = max_tokens
```

2. Add specific handling for models that need context window size parameter:

For example, in the LlamaCpp section:

```python
# For LlamaCpp, also set context size directly if supported
if hasattr(LlamaCpp, "n_ctx"):  # Check if the parameter exists
    llm = LlamaCpp(
        model_path=model_path,
        temperature=temperature,
        max_tokens=max_tokens,
        n_gpu_layers=n_gpu_layers,
        n_batch=n_batch,
        f16_kv=f16_kv,
        n_ctx=context_window_size,  # Set context window size
        verbose=True,
    )
```

### 3. Update UI to Make Context Size Visible

The UI should already display the setting once it's added to the default settings JSON.

### 4. Update Documentation

Add documentation explaining:

1. What the context window size setting does
2. How it affects different model providers
3. Recommended values for common models

## Testing Approach

1. Test with different providers (especially Llama.cpp and LM Studio)
2. Verify models with smaller context windows work correctly
3. Ensure max_tokens is calculated correctly based on context window size
4. Check for any regressions in models that worked previously

## Impact Assessment

This change:
- Allows users to customize context window size for their specific models
- Helps prevent errors when using models with smaller context windows
- Makes the software more adaptable to different LLM capabilities
- Is backwards compatible with existing configurations

## Example Usage

Example user workflow:
1. User sets up a Llama.cpp model with 4K context window size
2. User changes the context window size setting to 4096
3. System automatically adjusts max_tokens to 3277 (80% of 4096)
4. Model works properly without "context too long" errors

## Potential Future Enhancements

1. Auto-detection of model context window size
2. Provider-specific context window settings
3. Dynamic adjustment based on model characteristics
