# reflow

> Decorator-based HPC workflow engine with Result-based data wiring, reusable Flows, and an auto-generated CLI.

reflow lets you define HPC workflows as decorated Python functions. Tasks declare their resource requirements (CPUs, memory, walltime) via `@wf.job()` and wire data between each other using `Result` type annotations. The framework handles submission, array job fan-out, Merkle-DAG caching, dependency chaining, and generates a full CLI automatically.

## Supported schedulers

Slurm, PBS Pro / Torque, LSF, SGE / UGE, Flux. Set via `REFLOW_MODE` env var or `mode` in `~/.config/reflow/config.toml`. Users write scheduler-agnostic config (`partition` or `queue`) and reflow translates to the correct flags.

## Quick example

```python
from reflow import Workflow, Param, Result, RunDir
from typing import Annotated

wf = Workflow("pipeline")

@wf.job(cpus=4, time="02:00:00", mem="16G")
def prepare(start: Annotated[str, Param(help="Start date")], run_dir: RunDir = RunDir()) -> list[str]:
    return [str(f) for f in (run_dir / "input").glob("*.nc")]

@wf.job(array=True, cpus=8, time="04:00:00", mem="32G")
def compute(nc_file: Annotated[str, Result(step="prepare")], run_dir: RunDir = RunDir()) -> str:
    return str(run_dir / "output" / Path(nc_file).name)

if __name__ == "__main__":
    wf.cli()
```

## Key APIs

- `Workflow(name)` — main entry point, extends Flow with execution
- `@wf.job(cpus=, time=, mem=, array=, cache=, after=, partition=)` — register a task
- `Result(step="name")` — wire output from upstream task to parameter
- `Param(help=, namespace=)` — expose parameter as CLI flag
- `RunDir` — injected shared working directory (Path)
- `Flow(name)` — reusable task group, composable via `wf.include(flow, prefix=)`
- `wf.submit(run_dir=, executor=, force=, **params)` — submit a run
- `wf.cli()` — auto-generated argparse CLI (submit/status/cancel/retry/dag/describe)
- Executors: `SlurmExecutor`, `PBSExecutor`, `LSFExecutor`, `SGEExecutor`, `FluxExecutor`, `LocalExecutor`

## Links

- Docs: https://reflow.readthedocs.io
- Source: https://github.com/FREVA-CLINT/reflow
- PyPI: https://pypi.org/project/reflow/
