Metadata-Version: 2.4
Name: stagegate
Version: 0.2.0
Summary: A small, explicit scheduler for forward-only pipelines with stage-aware task priority.
Project-URL: Homepage, https://github.com/ttkkmg/stagegate
Project-URL: Repository, https://github.com/ttkkmg/stagegate
Project-URL: Issues, https://github.com/ttkkmg/stagegate/issues
Project-URL: Changelog, https://github.com/ttkkmg/stagegate/releases
Author-email: ttkkmg <ttkkmg@gmail.com>
Maintainer-email: ttkkmg <ttkkmg@gmail.com>
License: MIT
License-File: LICENSE
Keywords: concurrency,pipeline,scheduler,threading,workflow
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# stagegate

`stagegate` is a small, explicit, single-process Python library for running
many forward-only pipelines with:

- FIFO pipeline scheduling
- stage-aware task priority
- strict head-of-line blocking
- abstract resource quotas

It is aimed at local batch workloads that have grown beyond a plain thread pool
but do not justify a full workflow engine.

## Why Stagegate

`stagegate` is built for workloads where:

- many similar pipelines run on one machine
- tasks have heterogeneous CPU / memory / other resource needs
- later-stage work should be preferred over endlessly producing more upstream
  intermediates
- tasks that spend meaningful time in NumPy, SciPy, Pandas, other C-extension code, or external CLI tools

The scheduler stays intentionally small:

- one Python process
- multiple threads
- no DAG engine
- no persistence layer
- no distributed runtime

## Install

```bash
pip install stagegate
```

## Quick Example

```python
import stagegate


class ExamplePipeline(stagegate.Pipeline):
    def run(self) -> int:
        a = self.task(step_a, resources={"cpu": 1}).run()
        b = self.task(step_b, resources={"cpu": 1}).run()

        self.wait([a, b], return_when=stagegate.ALL_COMPLETED)
        result_a = a.result()
        result_b = b.result()

        self.stage_forward()

        c = self.task(
            combine,
            resources={"cpu": 4, "mem": 16},
            args=(result_a, result_b),
        ).run()
        return c.result()


with stagegate.Scheduler(
    resources={"cpu": 8, "mem": 32},
    pipeline_parallelism=2,
    task_parallelism=4,
) as scheduler:
    handle = scheduler.run_pipeline(ExamplePipeline())
    print(handle.result())
```

## Cooperative Terminate

For long-running tasks, `stagegate` supports cooperative terminate:

- `TaskHandle.request_terminate()`
- `stagegate.terminate_requested()`
- `stagegate.run_subprocess(...)`

The scheduler itself does not forcibly stop arbitrary running Python code or
running pipelines. Cooperative terminate is opt-in.

`stagegate.run_subprocess(...)` is currently intended for POSIX platforms such
as Linux, macOS, and BSD. `stagegate` itself may still be usable on Windows if
your tasks do not depend on that helper.

## Documentation

User's guide, use cases and API reference can be found at [http://stagegate.readthedocs.io](http://stagegate.readthedocs.io).
