# MARSYS Framework - AI Agent Instructions

**Version**: 0.1-beta
**Last Updated**: 2026-01-26

> **For AI Agents**: Read this file and the relevant module's `Agents.md` BEFORE modifying any code.
>
> **For System Understanding**: See `docs/architecture/overview.md` for execution flow and module boundaries.

---

## CRITICAL WORKFLOW

```
┌─────────────────────────────────────────────────────────────┐
│ BEFORE modifying any file:                                  │
│   1. Read module's Agents.md (if exists)                    │
│   2. Check Component Criticality Map below                  │
│   3. TRUNK-CRITICAL? → STOP, get human approval             │
│   4. TRUNK-STABLE interface change? → ASK FIRST             │
│   5. Verify design principles are not violated              │
├─────────────────────────────────────────────────────────────┤
│ AFTER modifying:                                            │
│   1. Update module's Agents.md (anti-patterns, lessons)     │
│   2. Update Component Registry if responsibilities changed  │
│   3. Create ADR if architectural decision was made          │
│   4. Update directory structure if files added/removed      │
├─────────────────────────────────────────────────────────────┤
│ NEVER: Modify TRUNK-CRITICAL without approval,              │
│        weaken tests, commit secrets, force push main        │
├─────────────────────────────────────────────────────────────┤
│ BUG FIXING ESCALATION RULE:                                 │
│   When fixing bugs during implementation:                   │
│   - If the bug is in YOUR implementation logic → fix it     │
│   - If the bug reveals an architecture/contract mismatch,   │
│     or requires workarounds, backward-compat hacks, or      │
│     removing/weakening tests → STOP and ASK the user.       │
│   - Explain: root cause, what you tried, 2-3 options.       │
│   - NEVER take shortcuts: no removing tests, no silent      │
│     workarounds, no YOLO fixes to make tests pass.          │
│   - If something fails after 2-3 attempts → escalate.       │
└─────────────────────────────────────────────────────────────┘
```

---

## 0. ENVIRONMENT SETUP

When testing Python code or running Python packages, **always activate the virtual environment first**:

```bash
source .venv/bin/activate && python -c "your test code"
```

Never run `python` or `python3` directly without activating `.venv` first.

---

## 1. GIT COMMIT RULES

- **NEVER** include "Generated with Claude Code" or AI attribution
- **NEVER** include "Co-Authored-By: Claude" or AI co-author footer
- Write clean, conventional commit messages

---

## 2. IMPLEMENTATION GUIDELINES

### 2.1 Deep Investigation Before Changes

- **Read all related source files** - not just files you're modifying:
  - Components that USE the code (callers)
  - Components that code USES (dependencies)
  - Similar implementations for patterns
- **Understand the full call chain** - trace data flow
- **Check for side effects** - ensure changes don't break other parts

### 2.2 Think Hard Before Acting

- **Question assumptions** - verify by reading actual code
- **Identify dead code** - use grep to find if methods are called
- **Understand WHY** code exists before removing
- **Consider edge cases** - think about what could break

### 2.3 Verify Integration Points

When code interacts with other modules, read those modules to understand:
- Expected input/output formats
- Method signatures and parameters
- Error handling patterns
- State management expectations

### 2.4 Incremental Changes

- Make small, focused changes
- Verify each change doesn't break functionality
- Keep related changes in logical commits

### 2.5 Document Decisions

- When removing code, document WHY it's safe to remove
- When adding code, ensure it follows existing patterns
- Update documentation when changing public APIs

---

## 3. DESIGN PRINCIPLES

Cross-cutting invariants. Violating these = **ASK FIRST**.

| ID | Principle | Applies To | Invariant |
|----|-----------|------------|-----------|
| DP-001 | Pure Agent Logic | `agents/agents.py` | `_run()` methods have NO side effects. Memory handled externally. |
| DP-002 | Centralized Validation | `validation/` | ALL response parsing in `ValidationProcessor`. No parsing in agents. |
| DP-003 | Dynamic Branching | `execution/branch_spawner.py` | Branches created at divergence points. Parent waits for children. |
| DP-004 | Branch Isolation | `execution/branch_executor.py` | Each branch has own memory, trace, metadata, status. No sharing. |
| DP-005 | Topology-Driven Routing | `routing/router.py` | Router uses topology graph for ALL decisions. No hardcoded routes. |
| DP-006 | Adapter Pattern | `models/` | All providers abstracted via adapters. Unified interface. |
| DP-007 | Format Pluggability | `formats/` | Response formats are extensible. Each provides prompt building + parsing. |

**Detailed principles**: See `docs/architecture/design-principles.md`

---

## 4. COMPONENT CRITICALITY MAP

### TRUNK-CRITICAL (NEVER modify without approval)

| Component | Path | Why Critical |
|-----------|------|--------------|
| Orchestra | `coordination/orchestra.py` | Central execution coordinator, many dependents |
| BranchExecutor | `coordination/execution/branch_executor.py` | Branch lifecycle, all execution flows through it |
| BranchSpawner | `coordination/execution/branch_spawner.py` | Parallelism, convergence, complex state |
| ValidationProcessor | `coordination/validation/response_validator.py` | Single source of truth for parsing |
| TopologyGraph | `coordination/topology/graph.py` | Runtime routing decisions |

### TRUNK-STABLE (ASK before interface changes)

| Component | Path | Notes |
|-----------|------|-------|
| StepExecutor | `coordination/execution/step_executor.py` | Single step execution |
| Router | `coordination/routing/router.py` | Decision translation |
| BaseAgent | `agents/agents.py` | Agent base class |
| BaseAPIModel | `models/models.py` | Model interface |
| ResponseFormat | `coordination/formats/base.py` | Format interface |
| ExecutionConfig | `coordination/config.py` | Configuration contract |

### BRANCH (Modify with tests)

| Area | Path |
|------|------|
| Specialized Agents | `agents/browser_agent.py`, `agents/file_operation_agent.py`, `agents/web_search_agent.py` |
| Tools | `environment/tools.py`, `environment/search_tools.py`, `environment/file_operations/` |
| Status | `coordination/status/` |
| Communication | `coordination/communication/` |
| Patterns | `coordination/topology/patterns.py` |

### LEAF (Free to modify)

| Area | Path |
|------|------|
| Examples | `examples/` |
| Tests | `tests/` |
| Utilities | `utils/` |

### Protected Paths (NEVER modify)

```
/.env.production  /secrets/**  /credentials/**  /.env*
```

---

## 5. DIRECTORY STRUCTURE

```
src/marsys/
├── __init__.py                    # Package exports
├── agents/                        # Agent system [TRUNK-STABLE]
│   ├── agents.py                  # BaseAgent, Agent classes
│   ├── agent_pool.py              # AgentPool for parallel execution
│   ├── browser_agent.py           # Browser automation agent
│   ├── file_operation_agent.py    # File operations agent
│   ├── web_search_agent.py        # Web search agent
│   ├── learnable_agents.py        # PEFT fine-tunable agents
│   ├── memory.py                  # Message, ConversationMemory
│   ├── memory_strategies.py       # Retention strategies
│   ├── registry.py                # AgentRegistry singleton
│   ├── pool_factory.py            # Agent pool factory
│   ├── exceptions.py              # Agent exceptions
│   ├── utils.py                   # Agent utilities
│   └── planning/                  # Task planning module
│       ├── config.py              # PlanningConfig
│       ├── types.py               # Plan, PlanItem dataclasses
│       ├── state.py               # PlanningState
│       ├── tools.py               # Planning tools
│       └── instructions.py        # Planning instructions
│
├── coordination/                  # Orchestration system [TRUNK-CRITICAL]
│   ├── orchestra.py               # Main Orchestra class [CRITICAL]
│   ├── config.py                  # ExecutionConfig, StatusConfig
│   ├── context_manager.py         # Execution context
│   ├── event_bus.py               # Event system
│   │
│   ├── branches/                  # Branch types
│   │   └── types.py               # ExecutionBranch, BranchType
│   │
│   ├── communication/             # User interaction [BRANCH]
│   │   ├── manager.py             # CommunicationManager
│   │   ├── core.py                # Communication core
│   │   ├── user_node_handler.py   # User node handling
│   │   └── channels/              # Terminal, Web channels
│   │
│   ├── configs/                   # Additional configs
│   │   └── auto_run.py            # Auto-run configuration
│   │
│   ├── execution/                 # Execution engine [CRITICAL]
│   │   ├── branch_executor.py     # Branch lifecycle [CRITICAL]
│   │   ├── step_executor.py       # Single step execution
│   │   ├── branch_spawner.py      # Dynamic branching [CRITICAL]
│   │   └── tool_executor.py       # Tool execution
│   │
│   ├── formats/                   # Response format system
│   │   ├── base.py                # BaseResponseFormat
│   │   ├── builder.py             # SystemPromptBuilder
│   │   ├── context.py             # AgentContext, CoordinationContext
│   │   ├── processors.py          # ResponseProcessor base
│   │   ├── registry.py            # Format registry
│   │   └── json_format/           # JSON format implementation
│   │       ├── format.py          # JSONResponseFormat
│   │       └── processor.py       # StructuredJSONProcessor
│   │
│   ├── routing/                   # Routing system
│   │   ├── router.py              # Routing logic
│   │   └── types.py               # RoutingDecision, ExecutionStep
│   │
│   ├── rules/                     # Rules engine
│   │   ├── rules_engine.py        # RulesEngine
│   │   ├── basic_rules.py         # Built-in rules
│   │   └── rule_factory.py        # Rule factory
│   │
│   ├── state/                     # State persistence
│   │   ├── state_manager.py       # StateManager
│   │   └── checkpoint.py          # Checkpointing
│   │
│   ├── status/                    # Status updates [BRANCH]
│   │   ├── manager.py             # StatusManager
│   │   ├── events.py              # Status events
│   │   └── channels.py            # Status channels
│   │
│   ├── steering/                  # Retry/recovery
│   │   └── manager.py             # SteeringManager
│   │
│   ├── topology/                  # Topology system
│   │   ├── core.py                # Node, Edge, Topology
│   │   ├── graph.py               # TopologyGraph [CRITICAL]
│   │   ├── patterns.py            # PatternConfig
│   │   ├── analyzer.py            # TopologyAnalyzer
│   │   └── converters/            # Format converters
│   │
│   └── validation/                # Response validation [CRITICAL]
│       ├── response_validator.py  # ValidationProcessor [CRITICAL]
│       └── types.py               # ActionType, ValidationResult
│
├── environment/                   # Tools and environment [BRANCH]
│   ├── tools.py                   # Tool utilities
│   ├── utils.py                   # Schema generation
│   ├── web_browser.py             # Browser automation
│   ├── browser_utils.py           # Browser utilities
│   ├── web_tools.py               # Web content tools
│   ├── search_tools.py            # Search tools
│   ├── bash_tools.py              # Bash command tools
│   ├── operator.py                # Environment operator
│   ├── tool_response.py           # Tool response models
│   └── file_operations/           # File handling toolkit
│       ├── core.py                # Core operations
│       ├── config.py              # FileOperationConfig
│       ├── readers.py             # File readers
│       ├── editors.py             # File editors
│       ├── search.py              # File search
│       ├── security.py            # Security features
│       ├── handlers/              # Type handlers
│       └── parsers/               # File parsers
│
├── models/                        # Model system [TRUNK-STABLE]
│   ├── models.py                  # BaseAPIModel, BaseLocalModel
│   ├── response_models.py         # HarmonizedResponse
│   ├── processors.py              # Response processors
│   └── utils.py                   # Model utilities
│
├── learning/                      # Learning utilities [LEAF]
│   └── rl.py                      # Reinforcement learning
│
├── logging/                       # Logging system [BRANCH]
│   └── integrations/              # Logging integrations
│
├── utils/                         # General utilities [LEAF]
│   ├── parsing.py                 # Parsing utilities
│   ├── tokens.py                  # Token utilities
│   ├── monitoring.py              # Monitoring utilities
│   ├── display.py                 # Display utilities
│   └── schema_utils.py            # Schema utilities
│
└── prompts/                       # Prompt templates [BRANCH]
```

---

## 6. CORE COMPONENTS REFERENCE

### 6.1 Orchestra

**Location**: [coordination/orchestra.py](src/marsys/coordination/orchestra.py)
**Criticality**: TRUNK-CRITICAL

Facade for all multi-agent workflows. Hides complexity behind `run()`.

```python
from marsys.coordination import Orchestra
from marsys.coordination.config import ExecutionConfig, StatusConfig

result = await Orchestra.run(
    task="Research quantum computing",
    topology=topology,
    agent_registry=AgentRegistry,
    execution_config=ExecutionConfig(
        user_interaction="terminal",
        convergence_timeout=300.0,
        status=StatusConfig.from_verbosity(1)
    ),
    max_steps=100
)

# OrchestraResult fields
result.success          # bool
result.final_response   # Any
result.branch_results   # List[BranchResult]
result.total_steps      # int
result.total_duration   # float
result.error            # Optional[str]
```

### 6.2 BranchExecutor

**Location**: [coordination/execution/branch_executor.py](src/marsys/coordination/execution/branch_executor.py)
**Criticality**: TRUNK-CRITICAL

Handles branch lifecycle. Delegates step execution to StepExecutor.

```python
class BranchType(Enum):
    SIMPLE = "simple"                # Sequential
    CONVERSATION = "conversation"    # Bidirectional
    NESTED = "nested"                # Hierarchical
    AGGREGATION = "aggregation"      # Waits for branches
    USER_INTERACTION = "user_interaction"

# Branch states: PENDING, RUNNING, PAUSED, WAITING, COMPLETED, FAILED, CANCELLED
```

### 6.3 ValidationProcessor

**Location**: [coordination/validation/response_validator.py](src/marsys/coordination/validation/response_validator.py)
**Criticality**: TRUNK-CRITICAL

Single source of truth for response parsing. No parsing elsewhere.

```python
class ActionType(Enum):
    INVOKE_AGENT = "invoke_agent"
    PARALLEL_INVOKE = "parallel_invoke"
    CALL_TOOL = "call_tool"
    FINAL_RESPONSE = "final_response"
    END_CONVERSATION = "end_conversation"
    WAIT_AND_AGGREGATE = "wait_and_aggregate"
    ERROR_RECOVERY = "error_recovery"

# Response formats:
{"next_action": "invoke_agent", "action_input": "Agent2"}
{"next_action": "parallel_invoke", "agents": ["A", "B"], "agent_requests": {...}}
{"next_action": "call_tool", "tool_calls": [...]}
{"next_action": "final_response", "content": "..."}
```

### 6.4 TopologyGraph

**Location**: [coordination/topology/graph.py](src/marsys/coordination/topology/graph.py)
**Criticality**: TRUNK-CRITICAL

Runtime routing decisions. All transitions validated through this.

```python
class TopologyGraph:
    def can_transition(source: str, target: str) -> bool
    def get_allowed_targets(source: str) -> List[str]
    def identify_divergence_points() -> List[str]
    def identify_convergence_points() -> List[str]
```

---

## 7. MODELS

### 7.1 ModelConfig

```python
from marsys.models import ModelConfig

# API Model
config = ModelConfig(
    type="api",
    name="anthropic/claude-opus-4.6",
    provider="openrouter",  # openai, anthropic, google, openrouter, xai
    max_tokens=12000,
    temperature=0.7,
    thinking_budget=1024,   # Gemini, Anthropic, Alibaba
    reasoning_effort="low", # OpenAI o1/o3
)

# Local Model
config = ModelConfig(
    type="local",
    name="Qwen/Qwen3-4B-Instruct-2507",
    model_class="llm",      # "llm" or "vlm"
    backend="huggingface",  # "huggingface" or "vllm"
    torch_dtype="bfloat16",
    device_map="auto",
)
```

**Environment Variables**:
- `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`
- `OPENROUTER_API_KEY`, `XAI_API_KEY`

---

## 8. AGENTS

### 8.1 BaseAgent

Agents are **stateless executors**. Memory handled externally.

```python
from marsys.agents import Agent

agent = Agent(
    model_config=config,
    name="Researcher",
    goal="Research topics",
    instruction="You are a researcher...",
    tools={"search": search_tool},
    memory_retention="session",  # single_run, session, persistent
)

# Single run
result = await agent.run("Research AI trends")

# Autonomous with tools
result = await agent.auto_run("Research and summarize AI", max_steps=5)

# Cleanup
await agent.cleanup()
```

### 8.2 Specialized Agents

| Agent | Location | Purpose |
|-------|----------|---------|
| BrowserAgent | `agents/browser_agent.py` | Web automation, vision-based |
| FileOperationAgent | `agents/file_operation_agent.py` | File operations, bash |
| WebSearchAgent | `agents/web_search_agent.py` | Multi-source search |

---

## 9. TOPOLOGY

Three ways to define:

**1. String Notation**
```python
topology = {
    "agents": ["Coordinator", "Worker1", "Worker2"],
    "flows": ["Coordinator -> Worker1", "Coordinator -> Worker2"],
    "rules": ["timeout(300)"]
}
```

**2. Object-Based**
```python
from marsys.coordination.topology import Topology, Node, Edge, EdgeType

topology = Topology(
    nodes=[Node("Coordinator"), Node("Worker1")],
    edges=[Edge("Coordinator", "Worker1", edge_type=EdgeType.INVOKE)]
)
```

**3. Patterns**
```python
from marsys.coordination.topology.patterns import PatternConfig

topology = PatternConfig.hub_and_spoke(
    hub="Coordinator",
    spokes=["Worker1", "Worker2"],
    parallel_spokes=True
)
```

**Available Patterns**: HUB_AND_SPOKE, HIERARCHICAL, PIPELINE, MESH, STAR, RING, BROADCAST

---

## 10. CONFIGURATION

```python
from marsys.coordination.config import ExecutionConfig, StatusConfig

config = ExecutionConfig(
    # Timeouts (seconds)
    convergence_timeout=300.0,
    branch_timeout=600.0,
    step_timeout=300.0,

    # Convergence
    dynamic_convergence_enabled=True,
    convergence_policy=1.0,  # or "strict", "majority", "any"

    # Response format
    response_format="json",

    # User interaction
    user_interaction="terminal",  # terminal, none, async

    # Cleanup
    auto_cleanup_agents=True,
)
```

---

## 11. BEST PRACTICES

### Topology Design

**DO**: Start simple, use PatternConfig, keep convergence clear, add timeouts
**DON'T**: Cyclic without max_turns, skip timeouts

### Agent Implementation

**DO**: Keep `_run()` pure, use type hints, test individually, call `cleanup()`
**DON'T**: Manipulate memory in `_run()`, use global state, mix coordination logic

### Error Handling

**DO**: Include User node for recovery, configure retry, set timeouts
**DON'T**: Assume APIs work, use infinite retries

---

## 12. RELATED DOCUMENTATION

### Architecture (Authoritative Sources)

| Document | Purpose | When to Consult |
|----------|---------|-----------------|
| `docs/architecture/overview.md` | System context, execution flow, module boundaries | Understanding how components interact |
| `docs/architecture/design-principles.md` | DP-001 to DP-007 with CORRECT/WRONG examples | Before modifying any TRUNK-* component |

### Architecture Decision Records

| ADR | Decision |
|-----|----------|
| `ADR-001-branch-based-parallel-execution.md` | Why branches instead of task queues |
| `ADR-002-centralized-response-validation.md` | Why all parsing in ValidationProcessor |
| `ADR-003-topology-driven-routing.md` | Why topology graph controls transitions |

### Component Registry (`docs/components/`)

YAML files documenting responsibilities and boundaries for:
- `orchestra.yaml`, `branch-executor.yaml`, `branch-spawner.yaml` (TRUNK-CRITICAL)
- `validation-processor.yaml`, `topology-graph.yaml` (TRUNK-CRITICAL)
- `base-agent.yaml`, `base-api-model.yaml` (TRUNK-STABLE)

### Other

| Document | Purpose |
|----------|---------|
| `project_initialization.md` | Project template and AI contract |
| `sessions/` | Implementation session logs |
| `src/[module]/Agents.md` | Module-specific AI instructions |

---

## QUICK REFERENCE

```
BEFORE modifying: Read Agents.md → Check criticality → ASK if TRUNK-* interface
AFTER modifying:  Update Agents.md → Update registry if needed → Create ADR if decision

CRITICAL paths:   orchestra.py, branch_executor.py, branch_spawner.py,
                  response_validator.py, graph.py

PROTECTED paths:  /.env*, /secrets/**, /credentials/**

Design principles: DP-001 to DP-007 (see docs/architecture/design-principles.md)

Architecture docs: docs/architecture/overview.md (flow), docs/components/*.yaml (boundaries)
```
