# Modular System Prompt Architecture

## Overview

The modular system prompt architecture allows breaking large monolithic system prompts into smaller, maintainable sections that are assembled at runtime using `<trender>` include tags.

## Key Features

- **File Inclusion**: Use `<trender type="include" path="sections/file.md" />` to include external files
- **Shell Command Execution**: Use `<trender>command</trender>` to execute shell commands (existing feature)
- **Nested Processing**: Included files can contain their own `<trender>` tags (both includes and commands)
- **Path Resolution**: Relative paths are resolved relative to the agent directory
- **Caching**: Both file contents and command outputs are cached for performance
- **Security**: Built-in protections against including sensitive system files

## Directory Structure

```
agents/
  default/
    system_prompt.md              # Main prompt (uses includes)
    system_prompt_modular.md      # Alternative modular version
    system_prompt_condensed.md    # Monolithic version (backup)
    sections/                     # Modular prompt sections
      00-header.md
      01-session-context.md
      02-tool-workflow.md
      03-response-patterns.md
      04-question-gate.md
      10-key-principles.md
      99-terminal-formatting.md
    agent.json                    # Agent configuration
    *.md                          # Skill files
```

## Include Tag Syntax

### File Inclusion
```markdown
<trender type="include" path="sections/filename.md" />
```

- `type="include"` - specifies file inclusion mode
- `path` - relative or absolute path to file
- Trailing `/` is optional
- Both single and double quotes are supported

### Shell Command (Existing)
```markdown
<trender>command here</trender>
```

## How It Works

1. **Discovery**: Agent manager loads agent directory
2. **Rendering**: When `get_full_system_prompt()` is called:
   - Creates PromptRenderer with agent directory as base_path
   - Processes all `<trender>` tags recursively
   - First processes file includes (allows nested includes)
   - Then processes shell commands
   - Repeats until no tags remain (max 100 iterations to prevent loops)
3. **Assembly**: Renders skills and appends them to the base prompt
4. **Delivery**: Returns complete rendered prompt to LLM

## Benefits of Modular Structure

### Maintainability
- Edit individual sections without loading the entire prompt
- Clear separation of concerns (context, tools, examples, etc.)
- Easier to locate and update specific topics

### Reusability
- Share common sections across multiple agents
- Create section libraries (tool reference, debugging guide, etc.)
- Mix and match sections for different agent personalities

### Organization
- Logical file naming with prefixes (00-header, 01-context, etc.)
- Grouped by functionality
- Easier navigation in version control

### Testing
- Validate sections independently
- Test rendering in isolation
- Compare output before/after changes

### Version Control
- Smaller, focused diffs
- Clear change history per section
- Easier code review
- Merge conflict reduction

## Creating a Modular Prompt

### Step 1: Identify Sections

Break your monolithic prompt into logical sections:
- Header and philosophy
- Session context
- Tool usage patterns
- Response patterns
- Examples
- Reference materials
- Formatting rules

### Step 2: Create Section Files

Create section files in the `sections/` subdirectory:

```bash
mkdir -p agents/default/sections
```

Each section file should contain a cohesive piece of content:

```markdown
<!-- sections/02-tool-workflow.md -->

tool execution:

you have TWO methods for calling tools:

method 1 - xml tags (inline in response):
  write xml tags directly in your response text. they execute as you stream.
  ...
```

### Step 3: Create Main Prompt

Use include tags to assemble sections:

```markdown
<!-- system_prompt.md -->

<trender type="include" path="sections/00-header.md" />
<trender type="include" path="sections/01-session-context.md" />
<trender type="include" path="sections/02-tool-workflow.md" />
...
```

### Step 4: Test Rendering

Verify the rendering works correctly:

```python
from core.utils.prompt_renderer import PromptRenderer
from pathlib import Path

agent_dir = Path("agents/default")
renderer = PromptRenderer(base_path=agent_dir)

prompt = (agent_dir / "system_prompt.md").read_text()
rendered = renderer.render(prompt)

print(f"Original: {len(prompt)} chars")
print(f"Rendered: {len(rendered)} chars")
```

## Advanced Features

### Nested Includes

Included files can contain their own include tags:

```markdown
<!-- sections/advanced-topics.md -->

## Advanced Debugging
<trender type="include" path="sections/debugging/01-memory-leaks.md" />
<trender type="include" path="sections/debugging/02-race-conditions.md" />
```

### Dynamic Content

Combine includes with dynamic shell commands:

```markdown
<!-- sections/session-context.md -->

current time: <trender>date '+%Y-%m-%d %H:%M:%S'</trender>
git branch: <trender>git branch --show-current</trender>
```

### Conditional Content

Use shell commands for conditional includes:

```markdown
<!-- Only include Python section if Python project detected -->
<trender>
if [ -f "pyproject.toml" ]; then
  cat sections/python-specific.md
fi
</trender>
```

## Path Resolution

### Relative Paths
Resolved relative to the agent directory:

```markdown
<trender type="include" path="sections/file.md" />  <!-- agents/default/sections/file.md -->
```

### Absolute Paths
Use absolute paths to include files outside agent directory:

```markdown
<trender type="include" path="/usr/local/share/common-sections/standard-header.md" />
```

### Parent Directory References
Use `../` to reference parent directories:

```markdown
<trender type="include" path="../shared/terminal-formatting.md" />
```

## Security Considerations

The renderer includes built-in protections:

### Blocked Paths
Cannot include sensitive system files:
- `/etc/passwd`, `/etc/shadow`, `/etc/hosts`
- `~/.ssh/`
- `/proc/`, `/sys/`, `/dev/`

### Error Messages
Failed includes return error messages in the output:
```
[trender error: file not found: sections/missing.md]
[trender error: permission denied: /etc/shadow]
```

### Path Traversal Protection
Paths are resolved and canonicalized before access:
- `../../../etc/passwd` is blocked
- Symlinks are resolved to their targets

## Performance

### Caching
- File contents are cached after first read
- Command outputs are cached after execution
- Cache is scoped to the PromptRenderer instance

### Iteration Limit
Maximum 100 rendering iterations to prevent infinite loops from circular includes.

### Monitoring
Check logs for rendering performance:
```
INFO: Rendered 7 includes and 15 commands in 2 iteration(s)
```

## Migration Guide

### Converting Monolithic to Modular

1. **Backup Original**
   ```bash
   cp system_prompt.md system_prompt_backup.md
   ```

2. **Create Sections Directory**
   ```bash
   mkdir -p sections
   ```

3. **Extract Sections**
   Identify natural breakpoints and extract to section files:
   - Headers and intro → `00-header.md`
   - Environment detection → `01-session-context.md`
   - Tool usage → `02-tool-workflow.md`
   - Etc.

4. **Replace with Includes**
   ```markdown
   <!-- Before (monolithic) -->
   kollabor system prompt v0.2
   i am kollabor...
   [hundreds of lines]

   <!-- After (modular) -->
   <trender type="include" path="sections/00-header.md" />
   <trender type="include" path="sections/01-session-context.md" />
   ```

5. **Test Rendering**
   Verify output matches original:
   ```bash
   python3 -m core.llm.agent_manager  # Will show rendering logs
   ```

6. **Validate**
   Compare rendered output with backup:
   ```python
   from core.utils.prompt_renderer import PromptRenderer

   renderer = PromptRenderer()
   original = Path("system_prompt_backup.md").read_text()
   modular = Path("system_prompt.md").read_text()
   rendered = renderer.render(modular)

   # Compare key sections
   assert "INVESTIGATE FIRST" in rendered
   assert len(rendered) > len(modular)  # Should be larger
   ```

## Best Practices

### Section Naming
Use two-digit prefixes for ordering:
- `00-header.md` - Always loaded first
- `01-*.md` - Core sections
- `02-*.md` - Secondary sections
- `99-*.md` - Always loaded last (formatting rules)

### Section Size
Aim for 50-200 lines per section:
- Too small: excessive file count
- Too large: defeats modularity purpose

### Section Independence
Each section should be self-contained:
- Don't rely on previous sections for context
- Include necessary headers in each section
- Avoid cross-references between sections

### Documentation
Add comments at the top of each section:
```markdown
<!-- Tool Workflow Section -->
<!-- Covers: tool execution methods, investigation patterns, best practices -->
<!-- Dependencies: none -->
```

## Troubleshooting

### Issue: "file not found" error
**Solution**: Check path is relative to agent directory
```bash
# Correct (relative)
<trender type="include" path="sections/file.md" />

# Wrong (absolute path to agent dir)
<trender type="include" path="/Users/.../agents/default/sections/file.md" />
```

### Issue: Circular include detected
**Solution**: Break the cycle by extracting common content
```markdown
<!-- Don't do this -->
<!-- a.md includes b.md -->
<!-- b.md includes a.md -->

<!-- Do this instead -->
<!-- a.md includes common.md -->
<!-- b.md includes common.md -->
```

### Issue: Shell commands not executing in includes
**Solution**: This is expected behavior. Commands in included files ARE executed.
If they're not showing output, check:
- Command syntax is correct
- Command produces output (not silent)
- No timeout occurred

### Issue: Output is too large
**Solution**: The renderer caches aggressively. Check what's being cached:
```python
renderer = PromptRenderer()
renderer.render(prompt)

# Inspect cache
print(f"Files cached: {len(renderer._file_cache)}")
print(f"Commands cached: {len(renderer._command_cache)}")
```

## Example: Complete Modular Prompt

See `agents/default/system_prompt_modular.md` for a working example.

## Related Files

- `core/utils/prompt_renderer.py` - Rendering implementation
- `core/llm/agent_manager.py` - Agent loading and rendering
- `agents/*/system_prompt.md` - Various agent prompts
- `agents/*/sections/` - Modular prompt sections
