# Recipe Tool (Executor & Creator) Guide

**IMPORTANT**: This document is a work in progress and may contain incomplete or outdated information. It was generated by an AI assistant and may not be accurate. The content of `recipe_excutor` and `recipes/recipe_executor` are the _official_ documentation for the Recipe Executor tool. Please refer to those files for the most accurate and up-to-date information. For a more complete, up-to-date artifact to share with AI assistants, please use the `ai_context/generated/recipe_executor_*.md` files.

## Overview

Recipe Executor is a powerful orchestration system that executes "recipes" - JSON-based definitions of sequential steps to perform tasks like file reading, LLM-based content generation, and file writing. This tool allows you to define complex workflows through simple recipe files, making it ideal for automating repetitive tasks, generating content, and processing information in a reproducible way.

## Core Concepts

### Recipe Format

Recipes are JSON files with a structured format that define a series of steps to be executed sequentially:

```json
{
  "steps": [
    {
      "type": "read_files",
      "config": {
        "path": "input/specifications.md",
        "content_key": "specs"
      }
    },
    {
      "type": "llm_generate",
      "config": {
        "prompt": "Generate content based on: {{specs}}",
        "model": "{{model|default:'openai/o4-mini'}}",
        "output_format": "files",
        "output_key": "generation_result"
      }
    },
    {
      "type": "write_files",
      "config": {
        "root": "./output",
        "files_key": "generation_result"
      }
    }
  ]
}
```

### Key Components

1. **Context**: Shared state container for passing data between steps
2. **Executor**: Central orchestration mechanism that loads and runs recipes
3. **Steps**: Individual operations that make up a recipe (read_files, generate, write_files, etc.)
4. **Utils/Templates**: Dynamic content generation using Liquid templates for variable substitution

## Architecture

Recipe Executor follows a modular design with these main components:

```
recipe_executor/
├── context.py         # Shared state between steps
├── executor.py        # Core recipe execution logic
├── llm.py             # LLM integration for content generation
├── logger.py          # Logging system
├── main.py            # CLI entry point
├── models.py          # Data models for steps and results
├── protocols.py       # Protocols for interface definitions
├── steps/             # Step implementations
│   ├── base.py        # Base class for all steps
│   ├── execute_recipe.py  # Step for running sub-recipes
│   ├── llm_generate.py    # Step for LLM generation
│   ├── parallel.py    # Step for parallel execution
│   ├── read_files.py  # Step for reading files
│   ├── registry.py    # Step type registry
│   └── write_files.py # Step for writing files
└── utils.py           # Utility functions (template rendering)
```

## Available Steps

Recipe Executor comes with several built-in step types:

| Step Type        | Description                                           |
| ---------------- | ----------------------------------------------------- |
| `read_files`     | Reads one or more files and stores content in context |
| `write_files`    | Writes generated files to disk                        |
| `generate`       | Generates content using large language models         |
| `execute_recipe` | Executes a sub-recipe, enabling composition           |
| `parallel`       | Executes multiple sub-steps concurrently              |

## Using Recipe Tool CLI

### Installation

```bash
# Clone the repository
git clone https://github.com/microsoft/recipe-tool.git

# Set up environment and install dependencies
cp .env.example .env
# Edit .env to add your API keys
make
source venv/bin/activate  # Linux/macOS
# or .\venv\Scripts\activate  # Windows
```

### Command Line Usage

Execute a recipe using the command line interface:

```bash
recipe-tool --execute path/to/your/recipe.json
```

With context variables:

```bash
recipe-tool --execute path/to/recipe.json context_key=value context_key2=value2
```

### Creating Your First Recipe JSON File

1. Create a JSON file with a `.json` extension
2. Define the recipe structure with steps
3. Execute the recipe with the command line tool

Example recipe (`hello_world.json`):

```json
{
  "steps": [
    {
      "type": "llm_generate",
      "config": {
        "prompt": "Write a hello world program in Python",
        "model": "openai/o4-mini",
        "output_format": "files",
        "output_key": "hello_world"
      }
    },
    {
      "type": "write_files",
      "config": {
        "root": "./output",
        "files_key": "hello_world"
      }
    }
  ]
}
```

Execute with:

```bash
python -m recipe_executor.main hello_world.json
```

## Advanced Features

### Template Variables

Recipe Executor supports dynamic content using Liquid template syntax:

```json
{
  "steps": [
    {
      "type": "read_files",
      "config": {
        "path": "{{file_path}}",
        "content_key": "content"
      }
    }
  ]
}
```

You can use these variables in paths, prompts, and other configuration values.

### Recipe Composition

Recipes can execute other recipes, allowing for modular composition:

```json
{
  "steps": [
    {
      "type": "execute_recipe",
      "config": {
        "recipe_path": "sub_recipes/prepare_data.json"
      }
    },
    {
      "type": "execute_recipe",
      "config": {
        "recipe_path": "sub_recipes/generate_content.json"
      }
    }
  ]
}
```

### Parallel Execution

The `parallel` step type allows executing multiple steps concurrently:

```json
{
  "steps": [
    {
      "type": "parallel",
      "config": {
        "substeps": [
          {
            "type": "execute_recipe",
            "config": {
              "recipe_path": "task_a.json"
            }
          },
          {
            "type": "execute_recipe",
            "config": {
              "recipe_path": "task_b.json"
            }
          }
        ],
        "max_concurrency": 2
      }
    }
  ]
}
```

### LLM Integration

Recipe Executor supports multiple LLM providers:

- OpenAI
- Anthropic
- Gemini
- Azure OpenAI

Configure the model in the `generate` step:

```json
{
  "type": "llm_generate",
  "config": {
    "prompt": "Generate content about: {{topic}}",
    "model": "{{provider/default:'openai'}}:{{model_name|default:'o4-mini'}}",
    "output_format": "files",
    "output_key": "generated_content"
  }
}
```

## Common Recipe Patterns

### Data Transformation

```json
{
  "steps": [
    {
      "type": "read_files",
      "config": {
        "path": "data/input.csv",
        "content_key": "raw_data"
      }
    },
    {
      "type": "llm_generate",
      "config": {
        "prompt": "Transform this CSV data into JSON: {{raw_data}}",
        "model": "openai/o4-mini",
        "output_format": "files",
        "output_key": "transformed_data"
      }
    },
    {
      "type": "write_files",
      "config": {
        "root": "./output",
        "files_key": "transformed_data"
      }
    }
  ]
}
```

### Multi-Stage Processing

```json
{
  "steps": [
    {
      "type": "read_files",
      "config": {
        "path": "specs/requirements.md",
        "content_key": "requirements"
      }
    },
    {
      "type": "execute_recipe",
      "config": {
        "recipe_path": "stages/parse_requirements.json",
        "context_overrides": {
          "input": "{{requirements}}"
        }
      }
    },
    {
      "type": "execute_recipe",
      "config": {
        "recipe_path": "stages/process_data.json"
      }
    },
    {
      "type": "execute_recipe",
      "config": {
        "recipe_path": "stages/generate_output.json"
      }
    }
  ]
}
```

## Troubleshooting

- Check log files in the `logs` directory
- Use `--log-dir` to specify a different logging location
- Set `LOG_LEVEL=DEBUG` in the `.env` file for detailed logging

## Extending Recipe Executor

You can extend Recipe Executor by creating custom step implementations:

1. Create a new step class that inherits from `BaseStep`
2. Implement the `execute` method
3. Register the step in `STEP_REGISTRY`

Example custom step:

```python
from recipe_executor.steps.base import BaseStep, StepConfig
from recipe_executor.context import Context
from recipe_executor.steps.registry import STEP_REGISTRY

class CustomStepConfig(StepConfig):
    param1: str
    param2: int = 42  # Default value

class CustomStep(BaseStep[CustomStepConfig]):
    def __init__(self, config: dict, logger=None):
        super().__init__(CustomStepConfig(**config), logger)

    def execute(self, context: Context) -> None:
        # Implementation
        value = self.config.param1
        context["result"] = f"Processed {value}"

# Register the step
STEP_REGISTRY["custom_step"] = CustomStep
```

## Best Practices

1. **Modular Recipes**: Break complex workflows into smaller, focused recipes
2. **Context Naming**: Use consistent naming in context artifacts
3. **Error Handling**: Handle missing files and edge cases with optional flags
4. **Documentation**: Document your recipes and custom steps
5. **Testing**: Test recipes with sample data before production use

## Conclusion

Recipe Executor provides a flexible framework for automating complex workflows through simple, declarative recipes. By combining different step types and leveraging template variables, you can create powerful automation tools for a wide range of applications.
