# Pipeline Stages

This directory contains the individual stages that make up the flow execution pipeline.

## Overview

Each stage implements the `Stage` protocol and performs a single, well-defined transformation on the execution state. Stages are organized into three categories based on when they execute:

1. **Pre-Execution Stages**: Run before the step is executed (modify context)
2. **Execution Stage**: Executes the actual step
3. **Post-Execution Stages**: Run after the step is executed (modify result)

## Stage Protocol

All stages implement this simple interface:

```python
class Stage(Protocol):
    async def process(self, state: ExecutionState) -> ExecutionState:
        """Process execution state and return new state."""
        ...
```

## Execution Flow

State flows through stages in a linear sequence:

```
Initial State (result=None)
    ↓
[Pre-Execution Stages]
    ↓
[Execution Stage]
    ↓
State (result populated)
    ↓
[Post-Execution Stages]
    ↓
Final State
```

## Stage Implementations

### Pre-Execution Stages

#### PinCheckStage
**Purpose**: Check for pinned results and short-circuit if found
**Can Short-Circuit**: Yes
**Modifies**: Step config (if config override), result (if full pin)

Checks the test manifest for pinned results. If a full pin is found (with `variables_updated`), short-circuits the pipeline and returns the pinned result. If only a config override is found, applies it and continues.

#### InterpolationStage
**Purpose**: Interpolate variables in step configuration
**Modifies**: Step config

Replaces variable references (e.g., `{{variableName}}`) in the step configuration with actual values from the execution context. Preserves nested blocks for composite executors to enable deferred interpolation.

#### CredentialResolutionStage
**Purpose**: Resolve credential references
**Modifies**: Context credentials

Resolves credential references in the step configuration by fetching and decrypting credential bundles from the credential manager. Adds resolved credentials to the execution context.

#### TargetModeStage
**Purpose**: Set target mode flag for composite executors
**Modifies**: Context flags

In test mode, if the current step is both the target and a composite executor, sets the `composite_target_mode` flag. This tells the composite executor to return its own result without executing child blocks.

### Execution Stage

#### ExecutorStage
**Purpose**: Execute the actual step through its executor
**Sets**: Result

Retrieves the appropriate executor from the registry and executes the step. This is the only stage that actually runs the step logic.

### Post-Execution Stages

#### TracingStage
**Purpose**: Add trace entry for executed step
**Modifies**: Context trace log

Appends a trace entry to the execution context's trace log, recording the step execution details for debugging and test verification.

#### TargetMarkingStage
**Purpose**: Mark result as target if step was the target
**Modifies**: Result target fields

In test mode, if this step was the target, marks the result with `is_target_result=True` and populates target metadata fields. For composite executors, also sets `is_composite_target=True`.

#### TargetPropagationStage
**Purpose**: Propagate nested target results to parent
**Modifies**: Result target fields

For composite executors, if any nested result contains a target, propagates the target information to the parent result. This enables target detection to bubble up through nested structures.

#### VariableAssignmentStage
**Purpose**: Assign result to variable if configured
**Modifies**: Result variables_updated

If the step configuration specifies a `resultVariable` or `responseVariable`, assigns the result data to that variable name in `variables_updated`.

## Pipeline Configurations

### Production Pipeline

```python
ExecutionPipeline(stages=[
    InterpolationStage(),
    CredentialResolutionStage(credential_manager),
    ExecutorStage(executor_registry),
    TracingStage(),
    VariableAssignmentStage(),
])
```

### Test Pipeline

```python
ExecutionPipeline(stages=[
    PinCheckStage(test_manifest),
    InterpolationStage(),
    CredentialResolutionStage(credential_manager),
    TargetModeStage(test_manifest),
    ExecutorStage(executor_registry),
    TracingStage(),
    TargetMarkingStage(test_manifest),
    TargetPropagationStage(),
    VariableAssignmentStage(),
])
```

## Writing a New Stage

To add a new stage:

1. Create a file in this directory (e.g., `validation.py`)
2. Implement the `Stage` protocol
3. Decide when the stage should act (pre-execution, post-execution, or both)
4. Return `state` unchanged if the stage doesn't apply
5. Add the stage to the appropriate pipeline configuration

Example:

```python
class ValidationStage:
    """Validates step inputs and outputs."""

    async def process(self, state: ExecutionState) -> ExecutionState:
        # Only run before execution
        if state.is_post_execution:
            return state

        # Validate step config
        self._validate_config(state.step.config)

        return state
```

Then add to pipeline:

```python
ExecutionPipeline(stages=[
    InterpolationStage(),
    ValidationStage(),  # ← Just append to list
    CredentialResolutionStage(credential_manager),
    # ...
])
```

## Benefits of This Architecture

1. **Single Responsibility**: Each stage has exactly one job
2. **Easy to Understand**: Read the pipeline list to see execution order
3. **Easy to Extend**: Add stages by appending to the list
4. **Easy to Test**: Stages are isolated and can be tested independently
5. **Easy to Reorder**: Change list order to reorder execution
6. **No Callbacks**: Simple linear flow, no complex orchestration
7. **Immutable State**: State flows through stages, never mutated
8. **Type Safe**: Strongly-typed state prevents errors

## Implementation Status

- [x] Pipeline infrastructure (Phase 2)
- [x] InterpolationStage (Phase 3.1)
- [x] CredentialResolutionStage (Phase 3.2)
- [x] TracingStage (Phase 3.3)
- [x] TargetModeStage (Phase 3.4)
- [x] TargetMarkingStage (Phase 3.5)
- [x] TargetPropagationStage (Phase 3.6)
- [x] VariableAssignmentStage (Phase 3.7)
- [x] PinCheckStage (Phase 3.8)
- [x] ExecutorStage (Phase 3.9)
