Metadata-Version: 2.4
Name: pydra
Version: 1.0a0
Summary: Pydra dataflow engine
Project-URL: documentation, https://nipype.github.io/pydra/
Project-URL: homepage, https://nipype.github.io/pydra/
Project-URL: repository, https://github.com/nipype/pydra.git
Author-email: Nipype developers <neuroimaging@python.org>
Maintainer-email: Nipype developers <neuroimaging@python.org>
License: Copyright (c) 2018-2019, Nipype developers
        
        Licensed under the Apache License, Version 2.0 (the "License");
        you may not use this file except in compliance with the License.
        You may obtain a copy of the License at
        
            http://www.apache.org/licenses/LICENSE-2.0
        
        Unless required by applicable law or agreed to in writing, software
        distributed under the License is distributed on an "AS IS" BASIS,
        WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
        See the License for the specific language governing permissions and
        limitations under the License.
License-File: LICENSE
Keywords: brainweb,dataflow,neuroimaging,pydra
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.11
Requires-Dist: attrs>=24.2.0
Requires-Dist: cloudpickle>=2.0.0
Requires-Dist: etelemetry>=0.2.2
Requires-Dist: fileformats>=0.15a4
Requires-Dist: filelock>=3.0.0
Requires-Dist: platformdirs>=2
Provides-Extra: all
Requires-Dist: black; extra == 'all'
Requires-Dist: codecov; extra == 'all'
Requires-Dist: fileformats-extras>=0.15.0a6; extra == 'all'
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'all'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'all'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'all'
Requires-Dist: furo>=2022.2.14.1; extra == 'all'
Requires-Dist: ipykernel; extra == 'all'
Requires-Dist: ipython; extra == 'all'
Requires-Dist: ipywidgets; extra == 'all'
Requires-Dist: matplotlib; extra == 'all'
Requires-Dist: nbsphinx; extra == 'all'
Requires-Dist: nest-asyncio; extra == 'all'
Requires-Dist: nibabel; extra == 'all'
Requires-Dist: nilearn; extra == 'all'
Requires-Dist: numpy; extra == 'all'
Requires-Dist: numpydoc>=0.6.0; extra == 'all'
Requires-Dist: openneuro-py; extra == 'all'
Requires-Dist: packaging; extra == 'all'
Requires-Dist: pandas; extra == 'all'
Requires-Dist: pandoc; extra == 'all'
Requires-Dist: pre-commit; extra == 'all'
Requires-Dist: psutil; extra == 'all'
Requires-Dist: pydra-mrtrix3>=3.0.4a17; extra == 'all'
Requires-Dist: pyld; extra == 'all'
Requires-Dist: pympler; extra == 'all'
Requires-Dist: pytest-cov; extra == 'all'
Requires-Dist: pytest-env; extra == 'all'
Requires-Dist: pytest-rerunfailures; extra == 'all'
Requires-Dist: pytest-timeout; extra == 'all'
Requires-Dist: pytest-xdist<2.0; extra == 'all'
Requires-Dist: pytest>=6.2.5; extra == 'all'
Requires-Dist: python-dateutil; extra == 'all'
Requires-Dist: scipy; extra == 'all'
Requires-Dist: sphinx; extra == 'all'
Requires-Dist: sphinx-argparse; extra == 'all'
Requires-Dist: sphinx-click; extra == 'all'
Requires-Dist: sphinx-rtd-theme; extra == 'all'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'all'
Requires-Dist: sphinxcontrib-versioning; extra == 'all'
Requires-Dist: tornado; extra == 'all'
Provides-Extra: dev
Requires-Dist: black; extra == 'dev'
Requires-Dist: codecov; extra == 'dev'
Requires-Dist: fileformats-extras>=0.15.0a6; extra == 'dev'
Requires-Dist: matplotlib; extra == 'dev'
Requires-Dist: numpy; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: psutil; extra == 'dev'
Requires-Dist: pyld; extra == 'dev'
Requires-Dist: pympler; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest-env; extra == 'dev'
Requires-Dist: pytest-rerunfailures; extra == 'dev'
Requires-Dist: pytest-timeout; extra == 'dev'
Requires-Dist: pytest-xdist<2.0; extra == 'dev'
Requires-Dist: pytest>=6.2.5; extra == 'dev'
Requires-Dist: python-dateutil; extra == 'dev'
Requires-Dist: tornado; extra == 'dev'
Provides-Extra: doc
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'doc'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'doc'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'doc'
Requires-Dist: furo>=2022.2.14.1; extra == 'doc'
Requires-Dist: ipykernel; extra == 'doc'
Requires-Dist: ipython; extra == 'doc'
Requires-Dist: ipywidgets; extra == 'doc'
Requires-Dist: matplotlib; extra == 'doc'
Requires-Dist: nbsphinx; extra == 'doc'
Requires-Dist: nest-asyncio; extra == 'doc'
Requires-Dist: nibabel; extra == 'doc'
Requires-Dist: nilearn; extra == 'doc'
Requires-Dist: numpy; extra == 'doc'
Requires-Dist: numpydoc>=0.6.0; extra == 'doc'
Requires-Dist: openneuro-py; extra == 'doc'
Requires-Dist: packaging; extra == 'doc'
Requires-Dist: pandas; extra == 'doc'
Requires-Dist: pandoc; extra == 'doc'
Requires-Dist: pydra-mrtrix3>=3.0.4a17; extra == 'doc'
Requires-Dist: scipy; extra == 'doc'
Requires-Dist: sphinx; extra == 'doc'
Requires-Dist: sphinx-argparse; extra == 'doc'
Requires-Dist: sphinx-click; extra == 'doc'
Requires-Dist: sphinx-rtd-theme; extra == 'doc'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'doc'
Requires-Dist: sphinxcontrib-versioning; extra == 'doc'
Provides-Extra: docs
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'docs'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'docs'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'docs'
Requires-Dist: furo>=2022.2.14.1; extra == 'docs'
Requires-Dist: ipykernel; extra == 'docs'
Requires-Dist: ipython; extra == 'docs'
Requires-Dist: ipywidgets; extra == 'docs'
Requires-Dist: matplotlib; extra == 'docs'
Requires-Dist: nbsphinx; extra == 'docs'
Requires-Dist: nest-asyncio; extra == 'docs'
Requires-Dist: nibabel; extra == 'docs'
Requires-Dist: nilearn; extra == 'docs'
Requires-Dist: numpy; extra == 'docs'
Requires-Dist: numpydoc>=0.6.0; extra == 'docs'
Requires-Dist: openneuro-py; extra == 'docs'
Requires-Dist: packaging; extra == 'docs'
Requires-Dist: pandas; extra == 'docs'
Requires-Dist: pandoc; extra == 'docs'
Requires-Dist: pydra-mrtrix3>=3.0.4a17; extra == 'docs'
Requires-Dist: scipy; extra == 'docs'
Requires-Dist: sphinx; extra == 'docs'
Requires-Dist: sphinx-argparse; extra == 'docs'
Requires-Dist: sphinx-click; extra == 'docs'
Requires-Dist: sphinx-rtd-theme; extra == 'docs'
Requires-Dist: sphinxcontrib-apidoc~=0.3.0; extra == 'docs'
Requires-Dist: sphinxcontrib-versioning; extra == 'docs'
Provides-Extra: test
Requires-Dist: codecov; extra == 'test'
Requires-Dist: fileformats-extras>=0.15.0a6; extra == 'test'
Requires-Dist: numpy; extra == 'test'
Requires-Dist: psutil; extra == 'test'
Requires-Dist: pyld; extra == 'test'
Requires-Dist: pympler; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Requires-Dist: pytest-env; extra == 'test'
Requires-Dist: pytest-rerunfailures; extra == 'test'
Requires-Dist: pytest-timeout; extra == 'test'
Requires-Dist: pytest-xdist<2.0; extra == 'test'
Requires-Dist: pytest>=6.2.5; extra == 'test'
Requires-Dist: python-dateutil; extra == 'test'
Requires-Dist: tornado; extra == 'test'
Provides-Extra: tests
Requires-Dist: codecov; extra == 'tests'
Requires-Dist: fileformats-extras>=0.15.0a6; extra == 'tests'
Requires-Dist: numpy; extra == 'tests'
Requires-Dist: psutil; extra == 'tests'
Requires-Dist: pyld; extra == 'tests'
Requires-Dist: pympler; extra == 'tests'
Requires-Dist: pytest-cov; extra == 'tests'
Requires-Dist: pytest-env; extra == 'tests'
Requires-Dist: pytest-rerunfailures; extra == 'tests'
Requires-Dist: pytest-timeout; extra == 'tests'
Requires-Dist: pytest-xdist<2.0; extra == 'tests'
Requires-Dist: pytest>=6.2.5; extra == 'tests'
Requires-Dist: python-dateutil; extra == 'tests'
Requires-Dist: tornado; extra == 'tests'
Provides-Extra: tutorial
Requires-Dist: fileformats-extras>=v0.15.0a6; extra == 'tutorial'
Requires-Dist: fileformats-medimage-extras>=v0.10.0a2; extra == 'tutorial'
Requires-Dist: fileformats-medimage>=v0.10.0a2; extra == 'tutorial'
Requires-Dist: jupyter; extra == 'tutorial'
Requires-Dist: jupyter-contrib-nbextensions; extra == 'tutorial'
Requires-Dist: jupyterlab; extra == 'tutorial'
Requires-Dist: jupytext; extra == 'tutorial'
Requires-Dist: matplotlib; extra == 'tutorial'
Requires-Dist: nbformat; extra == 'tutorial'
Requires-Dist: nbval; extra == 'tutorial'
Requires-Dist: nest-asyncio; extra == 'tutorial'
Requires-Dist: nibabel; extra == 'tutorial'
Requires-Dist: nilearn; extra == 'tutorial'
Requires-Dist: numpy; extra == 'tutorial'
Requires-Dist: openneuro-py; extra == 'tutorial'
Requires-Dist: pandas; extra == 'tutorial'
Requires-Dist: psutil; extra == 'tutorial'
Requires-Dist: pydra-mrtrix3>=3.0.4a17; extra == 'tutorial'
Requires-Dist: scipy; extra == 'tutorial'
Requires-Dist: sh; extra == 'tutorial'
Description-Content-Type: text/x-rst

|GHAction| |CircleCI| |codecov|

|Pydralogo|

.. |Pydralogo| image:: https://raw.githubusercontent.com/nipype/pydra/master/docs/logo/pydra_logo.jpg
   :width: 200px
   :alt: pydra logo

.. |GHAction| image:: https://github.com/nipype/pydra/workflows/Pydra/badge.svg
   :alt: GitHub Actions CI
   :target: https://github.com/nipype/Pydra/actions

.. |CircleCI| image:: https://circleci.com/gh/nipype/pydra.svg?style=svg
   :alt: CircleCI

.. |codecov| image:: https://codecov.io/gh/nipype/pydra/branch/master/graph/badge.svg
   :alt: codecov

======================
Pydra: Dataflow Engine
======================

A simple dataflow engine with scalable semantics.

Pydra is a rewrite of the Nipype engine with mapping and joining as
first-class operations. It forms the core of the Nipype 2.0 ecosystem.

The goal of pydra is to provide a lightweight Python dataflow engine for DAG
construction, manipulation, and distributed execution.

Feature list:
=============
1. Python 3.7+ using type annotation and `attrs <https://www.attrs.org/en/stable/>`_
2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
3. `splitter` and `combiner` provides many ways of compressing complex loop semantics
4. Cached execution with support for a global cache across dataflows and users
5. Distributed execution, presently via ConcurrentFutures, SLURM, and Dask (this is an experimental implementation with limited testing)

`API Documentation <https://nipype.github.io/pydra/>`_

Learn more about Pydra
======================

* `SciPy 2020 Proceedings <http://conference.scipy.org/proceedings/scipy2020/pydra.html>`_
* `PyCon 2020 Poster <https://docs.google.com/presentation/d/10tS2I34rS0G9qz6v29qVd77OUydjP_FdBklrgAGmYSw/edit?usp=sharing>`_
* `Explore Pydra interactively <https://github.com/nipype/pydra-tutorial>`_ (the tutorial can be also run using Binder service)

|Binder|

.. |Binder| image:: https://mybinder.org/badge_logo.svg
   :alt: Binder


Please note that mybinder times out after an hour.

Installation
============

::

    pip install pydra


Note that installation fails with older versions of pip on Windows. Upgrade pip before installing:

::

   pip install –upgrade pip
   pip install pydra


Developer installation
======================

Pydra requires Python 3.7+. To install in developer mode:

::

    git clone git@github.com:nipype/pydra.git
    cd pydra
    pip install -e ".[dev]"


In order to run pydra's test locally:

::

    pytest -vs pydra


If you want to test execution with Dask:

::

    git clone git@github.com:nipype/pydra.git
    cd pydra
    pip install -e ".[dask]"



It is also useful to install pre-commit:

::

    pip install pre-commit
    pre-commit
