Metadata-Version: 2.4
Name: pydra
Version: 1.0a3
Summary: Pydra dataflow engine
Project-URL: documentation, https://nipype.github.io/pydra/
Project-URL: homepage, https://nipype.github.io/pydra/
Project-URL: repository, https://github.com/nipype/pydra.git
Author-email: Nipype developers <neuroimaging@python.org>
Maintainer-email: Nipype developers <neuroimaging@python.org>
License: Copyright (c) 2018-2019, Nipype developers
        
        Licensed under the Apache License, Version 2.0 (the "License");
        you may not use this file except in compliance with the License.
        You may obtain a copy of the License at
        
            http://www.apache.org/licenses/LICENSE-2.0
        
        Unless required by applicable law or agreed to in writing, software
        distributed under the License is distributed on an "AS IS" BASIS,
        WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
        See the License for the specific language governing permissions and
        limitations under the License.
License-File: LICENSE
Keywords: brainweb,dataflow,neuroimaging,pydra
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.11
Requires-Dist: attrs>=24.2.0
Requires-Dist: cloudpickle>=3.0.0
Requires-Dist: etelemetry>=0.3.1
Requires-Dist: fileformats>=0.15.0a7
Requires-Dist: filelock>=3.0.0
Requires-Dist: platformdirs>=2
Provides-Extra: all
Requires-Dist: black; extra == 'all'
Requires-Dist: coverage[toml]>=5.2.1; extra == 'all'
Requires-Dist: fileformats-extras>=0.15.0a7; extra == 'all'
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'all'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'all'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'all'
Requires-Dist: furo>=2022.2.14.1; extra == 'all'
Requires-Dist: ipykernel; extra == 'all'
Requires-Dist: ipython; extra == 'all'
Requires-Dist: ipywidgets; extra == 'all'
Requires-Dist: matplotlib; extra == 'all'
Requires-Dist: nbsphinx; extra == 'all'
Requires-Dist: nest-asyncio; extra == 'all'
Requires-Dist: nibabel; extra == 'all'
Requires-Dist: nilearn; extra == 'all'
Requires-Dist: numpy; extra == 'all'
Requires-Dist: numpy>=1.26; extra == 'all'
Requires-Dist: numpydoc>=0.6.0; extra == 'all'
Requires-Dist: openneuro-py; extra == 'all'
Requires-Dist: packaging; extra == 'all'
Requires-Dist: pandas; extra == 'all'
Requires-Dist: pandoc; extra == 'all'
Requires-Dist: pre-commit; extra == 'all'
Requires-Dist: psutil>=5.9.0; extra == 'all'
Requires-Dist: pydra-tasks-mrtrix3>=3.1.0a1; extra == 'all'
Requires-Dist: pyld>=2.0; extra == 'all'
Requires-Dist: pympler>=1.1; extra == 'all'
Requires-Dist: pytest-cov>=6.1; extra == 'all'
Requires-Dist: pytest-env>=1.1; extra == 'all'
Requires-Dist: pytest-rerunfailures>=15.0; extra == 'all'
Requires-Dist: pytest-timeout>=2.3; extra == 'all'
Requires-Dist: pytest-xdist>=3.6; extra == 'all'
Requires-Dist: pytest>=8.3.5; extra == 'all'
Requires-Dist: python-dateutil>=2.8.2; extra == 'all'
Requires-Dist: scipy; extra == 'all'
Requires-Dist: sphinx; extra == 'all'
Requires-Dist: sphinx-argparse; extra == 'all'
Requires-Dist: sphinx-click; extra == 'all'
Requires-Dist: sphinx-rtd-theme; extra == 'all'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'all'
Requires-Dist: sphinxcontrib-versioning; extra == 'all'
Requires-Dist: tornado>=6.1; extra == 'all'
Provides-Extra: dev
Requires-Dist: black; extra == 'dev'
Requires-Dist: coverage[toml]>=5.2.1; extra == 'dev'
Requires-Dist: fileformats-extras>=0.15.0a7; extra == 'dev'
Requires-Dist: matplotlib; extra == 'dev'
Requires-Dist: numpy>=1.26; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: psutil>=5.9.0; extra == 'dev'
Requires-Dist: pyld>=2.0; extra == 'dev'
Requires-Dist: pympler>=1.1; extra == 'dev'
Requires-Dist: pytest-cov>=6.1; extra == 'dev'
Requires-Dist: pytest-env>=1.1; extra == 'dev'
Requires-Dist: pytest-rerunfailures>=15.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
Requires-Dist: pytest>=8.3.5; extra == 'dev'
Requires-Dist: python-dateutil>=2.8.2; extra == 'dev'
Requires-Dist: tornado>=6.1; extra == 'dev'
Provides-Extra: doc
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'doc'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'doc'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'doc'
Requires-Dist: furo>=2022.2.14.1; extra == 'doc'
Requires-Dist: ipykernel; extra == 'doc'
Requires-Dist: ipython; extra == 'doc'
Requires-Dist: ipywidgets; extra == 'doc'
Requires-Dist: matplotlib; extra == 'doc'
Requires-Dist: nbsphinx; extra == 'doc'
Requires-Dist: nest-asyncio; extra == 'doc'
Requires-Dist: nibabel; extra == 'doc'
Requires-Dist: nilearn; extra == 'doc'
Requires-Dist: numpy; extra == 'doc'
Requires-Dist: numpydoc>=0.6.0; extra == 'doc'
Requires-Dist: openneuro-py; extra == 'doc'
Requires-Dist: packaging; extra == 'doc'
Requires-Dist: pandas; extra == 'doc'
Requires-Dist: pandoc; extra == 'doc'
Requires-Dist: pydra-tasks-mrtrix3>=3.1.0a1; extra == 'doc'
Requires-Dist: scipy; extra == 'doc'
Requires-Dist: sphinx; extra == 'doc'
Requires-Dist: sphinx-argparse; extra == 'doc'
Requires-Dist: sphinx-click; extra == 'doc'
Requires-Dist: sphinx-rtd-theme; extra == 'doc'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'doc'
Requires-Dist: sphinxcontrib-versioning; extra == 'doc'
Provides-Extra: docs
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'docs'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'docs'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'docs'
Requires-Dist: furo>=2022.2.14.1; extra == 'docs'
Requires-Dist: ipykernel; extra == 'docs'
Requires-Dist: ipython; extra == 'docs'
Requires-Dist: ipywidgets; extra == 'docs'
Requires-Dist: matplotlib; extra == 'docs'
Requires-Dist: nbsphinx; extra == 'docs'
Requires-Dist: nest-asyncio; extra == 'docs'
Requires-Dist: nibabel; extra == 'docs'
Requires-Dist: nilearn; extra == 'docs'
Requires-Dist: numpy; extra == 'docs'
Requires-Dist: numpydoc>=0.6.0; extra == 'docs'
Requires-Dist: openneuro-py; extra == 'docs'
Requires-Dist: packaging; extra == 'docs'
Requires-Dist: pandas; extra == 'docs'
Requires-Dist: pandoc; extra == 'docs'
Requires-Dist: pydra-tasks-mrtrix3>=3.1.0a1; extra == 'docs'
Requires-Dist: scipy; extra == 'docs'
Requires-Dist: sphinx; extra == 'docs'
Requires-Dist: sphinx-argparse; extra == 'docs'
Requires-Dist: sphinx-click; extra == 'docs'
Requires-Dist: sphinx-rtd-theme; extra == 'docs'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'docs'
Requires-Dist: sphinxcontrib-versioning; extra == 'docs'
Provides-Extra: test
Requires-Dist: coverage[toml]>=5.2.1; extra == 'test'
Requires-Dist: fileformats-extras>=0.15.0a7; extra == 'test'
Requires-Dist: numpy>=1.26; extra == 'test'
Requires-Dist: psutil>=5.9.0; extra == 'test'
Requires-Dist: pyld>=2.0; extra == 'test'
Requires-Dist: pympler>=1.1; extra == 'test'
Requires-Dist: pytest-cov>=6.1; extra == 'test'
Requires-Dist: pytest-env>=1.1; extra == 'test'
Requires-Dist: pytest-rerunfailures>=15.0; extra == 'test'
Requires-Dist: pytest-timeout>=2.3; extra == 'test'
Requires-Dist: pytest-xdist>=3.6; extra == 'test'
Requires-Dist: pytest>=8.3.5; extra == 'test'
Requires-Dist: python-dateutil>=2.8.2; extra == 'test'
Requires-Dist: tornado>=6.1; extra == 'test'
Provides-Extra: tests
Requires-Dist: coverage[toml]>=5.2.1; extra == 'tests'
Requires-Dist: fileformats-extras>=0.15.0a7; extra == 'tests'
Requires-Dist: numpy>=1.26; extra == 'tests'
Requires-Dist: psutil>=5.9.0; extra == 'tests'
Requires-Dist: pyld>=2.0; extra == 'tests'
Requires-Dist: pympler>=1.1; extra == 'tests'
Requires-Dist: pytest-cov>=6.1; extra == 'tests'
Requires-Dist: pytest-env>=1.1; extra == 'tests'
Requires-Dist: pytest-rerunfailures>=15.0; extra == 'tests'
Requires-Dist: pytest-timeout>=2.3; extra == 'tests'
Requires-Dist: pytest-xdist>=3.6; extra == 'tests'
Requires-Dist: pytest>=8.3.5; extra == 'tests'
Requires-Dist: python-dateutil>=2.8.2; extra == 'tests'
Requires-Dist: tornado>=6.1; extra == 'tests'
Provides-Extra: tutorial
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'tutorial'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'tutorial'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'tutorial'
Requires-Dist: jupyter; extra == 'tutorial'
Requires-Dist: jupyter-contrib-nbextensions; extra == 'tutorial'
Requires-Dist: jupyterlab; extra == 'tutorial'
Requires-Dist: jupytext; extra == 'tutorial'
Requires-Dist: matplotlib; extra == 'tutorial'
Requires-Dist: nbformat; extra == 'tutorial'
Requires-Dist: nbval; extra == 'tutorial'
Requires-Dist: nest-asyncio; extra == 'tutorial'
Requires-Dist: nibabel; extra == 'tutorial'
Requires-Dist: nilearn; extra == 'tutorial'
Requires-Dist: numpy; extra == 'tutorial'
Requires-Dist: openneuro-py; extra == 'tutorial'
Requires-Dist: pandas; extra == 'tutorial'
Requires-Dist: psutil; extra == 'tutorial'
Requires-Dist: pydra-tasks-mrtrix3>=3.1.0a1; extra == 'tutorial'
Requires-Dist: scipy; extra == 'tutorial'
Requires-Dist: sh; extra == 'tutorial'
Description-Content-Type: text/x-rst

|CI/CD| |codecov| |PyPI| |Docs|

|Pydralogo|

======================
Pydra: Dataflow Engine
======================

A simple dataflow engine with scalable semantics.

Pydra is a rewrite of the Nipype engine with mapping and joining as
first-class operations. It forms the core of the Nipype 2.0 ecosystem.

The goal of pydra is to provide a lightweight Python dataflow engine for DAG
construction, manipulation, and distributed execution.

Feature list:
=============
1. Python 3.11+ using type annotation and `attrs <https://www.attrs.org/en/stable/>`_
2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
3. `splitter` and `combiner` provides many ways of compressing complex loop semantics
4. Cached execution with support for a global cache across dataflows and users
5. Distributed execution, presently via ConcurrentFutures, SLURM and SGE, with support for PS/IJ and Dask available via plugins

`API Documentation <https://nipype.github.io/pydra/>`_

Learn more about Pydra
======================

* `SciPy 2020 Proceedings <http://conference.scipy.org/proceedings/scipy2020/pydra.html>`_
* `PyCon 2020 Poster <https://docs.google.com/presentation/d/10tS2I34rS0G9qz6v29qVd77OUydjP_FdBklrgAGmYSw/edit?usp=sharing>`_
* `Explore Pydra interactively <https://github.com/nipype/pydra-tutorial>`_ (the tutorial can be also run using Binder service)

|Binder|

.. |Binder| image:: https://mybinder.org/badge_logo.svg
   :alt: Binder


Please note that mybinder times out after an hour.

Installation
============

Pydra can be installed from PyPI using pip, noting that you currently need to specify
the 1.0-alpha version due to a quirk of PyPI version sorting, otherwise you will end up
with the old 0.25 version.

::

   pip install -–upgrade pip
   pip install pydra>=1.0a


If you want to install plugins for psij or dask you can by installing the relevant
plugin packages

::

    pip install pydra-workers-psij
    pip install pydra-workers-dask


Task implementations for various toolkits and workflows are available in task plugins,
which can be installed similarly

::

   pip install pydra-tasks-mrtrix3
   pip install pydra-tasks-fsl


Developer installation
======================

Pydra requires Python 3.11+. To install in developer mode::

    git clone git@github.com:nipype/pydra.git
    cd pydra
    pip install -e ".[dev]"

In order to run pydra's test locally::

    pytest pydra

We use `tox <https://tox.wiki/>`_ to test versions and dependency sets.
For example, to test on the minimum and latest dependencies, run::

    tox -e py311-min -e py313-latest

It is also useful to install pre-commit:

::

    pip install pre-commit
    pre-commit


.. |Pydralogo| image:: https://raw.githubusercontent.com/nipype/pydra/main/docs/source/_static/logo/pydra_logo.jpg
   :width: 200px
   :alt: pydra logo

.. |CI/CD| image:: https://github.com/nipype/pydra/actions/workflows/ci-cd.yml/badge.svg
   :alt: CI/CD
   :target: https://github.com/nipype/pydra/actions/workflows/ci-cd.yml

.. |codecov| image:: https://codecov.io/gh/nipype/pydra/branch/main/graph/badge.svg
   :alt: codecov
   :target: https://codecov.io/gh/nipype/pydra

.. |Python Versions| image:: https://img.shields.io/pypi/pyversions/pydra.svg
   :alt: Supported Python versions
   :target: https://pypi.python.org/pypi/pydra

.. |PyPI| image:: https://img.shields.io/badge/pypi-1.0alpha-orange
   :alt: PyPI
   :target: https://pypi.org/project/pydra/1.0a0/

.. |Docs| image:: https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat
   :alt: Documentation Status
   :target: https://nipype.github.io/pydra
