Metadata-Version: 2.4
Name: yaml-workflow
Version: 0.5.0
Summary: A lightweight, powerful, and flexible workflow engine that executes tasks defined in YAML configuration files
Project-URL: Homepage, https://github.com/orieg/yaml-workflow
Project-URL: Issues, https://github.com/orieg/yaml-workflow/issues
Project-URL: Documentation, https://orieg.github.io/yaml-workflow/
Author-email: Nicolas Brousse <nicolas@brousse.info>
License: MIT License
        
        Copyright (c) 2024 YAML Workflow Contributors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE. 
License-File: LICENSE
Keywords: automation,batch-processing,configuration,data-pipeline,data-processing,distributed-tasks,etl,job-scheduler,modular-workflows,orchestration,parallel-processing,pipeline,process-automation,reports,retry-mechanism,state-management,task-orchestration,task-runner,templating,workflow,workflow-automation,workflow-engine,workflow-management,yaml,yaml-configuration
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Systems Administration
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: click<9.0,>=8.0
Requires-Dist: jinja2<4.0,>=3.0
Requires-Dist: pyyaml<7.0,>=6.0
Provides-Extra: dev
Requires-Dist: black==26.1.0; extra == 'dev'
Requires-Dist: build<2.0.0,>=1.0.0; extra == 'dev'
Requires-Dist: isort<9.0,>=5.0; extra == 'dev'
Requires-Dist: twine<7.0.0,>=4.0.0; extra == 'dev'
Provides-Extra: doc
Requires-Dist: docstring-parser<1.0,>=0.16.0; extra == 'doc'
Requires-Dist: griffe>=0.49.0; extra == 'doc'
Requires-Dist: mkdocs-gen-files<1.0,>=0.5.0; extra == 'doc'
Requires-Dist: mkdocs-literate-nav<1.0,>=0.6.0; extra == 'doc'
Requires-Dist: mkdocs-material<10.0,>=9.6.0; extra == 'doc'
Requires-Dist: mkdocs-section-index<1.0,>=0.3.0; extra == 'doc'
Requires-Dist: mkdocs<2.0,>=1.6.0; extra == 'doc'
Requires-Dist: mkdocstrings[python]<2.0,>=0.29.0; extra == 'doc'
Provides-Extra: test
Requires-Dist: mypy<2.0,>=1.0; extra == 'test'
Requires-Dist: pytest-cov<8.0,>=4.0; extra == 'test'
Requires-Dist: pytest-ordering; extra == 'test'
Requires-Dist: pytest<10.0,>=7.0; extra == 'test'
Requires-Dist: types-pyyaml<7.0,>=6.0; extra == 'test'
Description-Content-Type: text/markdown

# YAML Workflow

[![PyPI version](https://img.shields.io/pypi/v/yaml-workflow.svg)](https://pypi.org/project/yaml-workflow/)
[![Python versions](https://img.shields.io/pypi/pyversions/yaml-workflow.svg)](https://pypi.org/project/yaml-workflow/)
[![CI](https://github.com/orieg/yaml-workflow/actions/workflows/ci.yml/badge.svg)](https://github.com/orieg/yaml-workflow/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/orieg/yaml-workflow/graph/badge.svg)](https://codecov.io/gh/orieg/yaml-workflow)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)

A lightweight, powerful, and flexible workflow engine that executes tasks defined in YAML configuration files. Create modular, reusable workflows by connecting tasks through YAML definitions, with support for parallel processing, batch operations, and state management.

## Why yaml-workflow?

Most workflow tools (Airflow, Prefect, Dagster) are designed for distributed cloud infrastructure with complex server setups. **yaml-workflow** takes a different approach:

| | yaml-workflow | Airflow / Prefect / Dagster |
|---|---|---|
| **Setup** | `pip install yaml-workflow` | Server, database, scheduler, workers |
| **Configuration** | Plain YAML files | Python DAGs + infrastructure config |
| **Dependencies** | 3 (PyYAML, Jinja2, Click) | 50+ packages, Docker, PostgreSQL |
| **Use case** | Local automation, scripts, CI/CD, data pipelines | Enterprise orchestration at scale |
| **Learning curve** | Minutes | Hours to days |
| **State** | File-based, resumable | Database-backed |

**Choose yaml-workflow when you need:**
- Simple task automation without infrastructure overhead
- Reproducible pipelines defined in version-controlled YAML
- Batch processing with parallel execution
- State persistence and workflow resume after failures
- A lightweight alternative to shell scripts with better error handling

## Features

- YAML-driven workflow definition with Jinja2 templating
- Multiple task types: shell, Python, file, template, HTTP, batch
- Parallel execution with configurable worker pools
- State persistence and resume capability
- Dry-run mode to preview without executing
- Workflow visualization (ASCII and Mermaid)
- Retry mechanisms with configurable strategies
- Namespaced variables (`args`, `env`, `steps`, `batch`)
- Flow control with custom step sequences and conditions
- Extensible task system via `@register_task` decorator

## Quick Start

```bash
# Install
pip install yaml-workflow

# Initialize example workflows
yaml-workflow init

# Run a workflow with parameters
yaml-workflow run workflows/hello_world.yaml name=Alice
```

**Example workflow** (`hello_world.yaml`):

```yaml
name: Hello World
description: A simple greeting workflow

params:
  name:
    type: string
    default: World

steps:
  - name: create_greeting
    task: template
    inputs:
      template: "Hello, {{ args.name }}!"
      output_file: greeting.txt

  - name: show_greeting
    task: shell
    inputs:
      command: cat greeting.txt
```

### Visualize workflows

```bash
yaml-workflow visualize workflows/data_pipeline.yaml
```

```
  Workflow: Data Pipeline

  ┌─────────────────┐
  │  detect_format   │
  │   python_code    │
  └─────────────────┘
           │
           ▼
  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐
  │ process_json │  │ process_csv  │  │ process_xml  │  │handle_unknown│
  │    shell     │  │    shell     │  │    shell     │  │    shell     │
  └──────────────┘  └──────────────┘  └──────────────┘  └──────────────┘
           │
           ▼
  ┌─────────────────┐
  │ generate_report  │
  │   python_code    │
  └─────────────────┘
```

Adjacent conditional steps are automatically grouped as branches. Use `--format mermaid` to export for docs or GitHub rendering.

### Dry-run mode

Preview what a workflow would do without executing anything:

```bash
yaml-workflow run workflows/hello_world.yaml name=Alice --dry-run
```

```
[DRY-RUN] Workflow: Hello World
[DRY-RUN] Steps: 2 to execute

  [DRY-RUN] Step 'create_greeting' — task: template — WOULD EXECUTE
    template: Hello, Alice!
    output_file: greeting.txt
  [DRY-RUN] Step 'show_greeting' — task: shell — WOULD EXECUTE
    command: cat greeting.txt

[DRY-RUN] Complete. 2 step(s) would execute, 0 would be skipped.
[DRY-RUN] No files were written. No tasks were executed.
```

### More commands

```bash
# List available workflows
yaml-workflow list

# Validate a workflow
yaml-workflow validate workflows/hello_world.yaml

# Resume a failed workflow
yaml-workflow run workflows/hello_world.yaml --resume
```

## Documentation

Full documentation is available at **[orieg.github.io/yaml-workflow](https://orieg.github.io/yaml-workflow/)**.

- [Getting Started](https://orieg.github.io/yaml-workflow/guide/getting-started/) - Installation and first workflow
- [Task Types](https://orieg.github.io/yaml-workflow/guide/tasks/basic-tasks/) - Shell, Python, file, template, and batch tasks
- [Workflow Structure](https://orieg.github.io/yaml-workflow/workflow-structure/) - YAML configuration reference
- [Templating](https://orieg.github.io/yaml-workflow/guide/templating/) - Jinja2 variable substitution
- [State Management](https://orieg.github.io/yaml-workflow/state/) - Persistence and resume
- [Task Development](https://orieg.github.io/yaml-workflow/guide/task-development/) - Creating custom tasks
- [API Reference](https://orieg.github.io/yaml-workflow/reference/) - Full API documentation

## Contributing

Contributions are welcome! See the [Contributing Guide](https://orieg.github.io/yaml-workflow/contributing/development/) for development setup and guidelines.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
