Metadata-Version: 2.1
Name: pydra
Version: 0.10
Summary: Pydra dataflow engine
Home-page: https://github.com/nipype/pydra
Author: Nipype developers
Author-email: neuroimaging@python.org
Maintainer: Nipype developers
Maintainer-email: neuroimaging@python.org
License: Apache License, 2.0
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Scientific/Engineering
Provides: pydra
Requires-Python: >=3.7
Description-Content-Type: text/x-rst; charset=UTF-8
Requires-Dist: attrs (>=19.1.0)
Requires-Dist: cloudpickle (>=1.2.2)
Requires-Dist: filelock (>=3.0.0)
Requires-Dist: etelemetry (>=0.2.2)
Provides-Extra: all
Requires-Dist: attrs (>=19.1.0) ; extra == 'all'
Requires-Dist: cloudpickle ; extra == 'all'
Requires-Dist: filelock ; extra == 'all'
Requires-Dist: packaging ; extra == 'all'
Requires-Dist: sphinx (>=2.1.2) ; extra == 'all'
Requires-Dist: sphinx-rtd-theme ; extra == 'all'
Requires-Dist: sphinxcontrib-apidoc (~=0.3.0) ; extra == 'all'
Requires-Dist: sphinxcontrib-napoleon ; extra == 'all'
Requires-Dist: sphinxcontrib-versioning ; extra == 'all'
Requires-Dist: pytest (<6.0.0,>=4.4.0) ; extra == 'all'
Requires-Dist: pytest-cov ; extra == 'all'
Requires-Dist: pytest-env ; extra == 'all'
Requires-Dist: pytest-xdist (<2.0) ; extra == 'all'
Requires-Dist: pytest-rerunfailures ; extra == 'all'
Requires-Dist: codecov ; extra == 'all'
Requires-Dist: numpy ; extra == 'all'
Requires-Dist: pyld ; extra == 'all'
Requires-Dist: psutil ; extra == 'all'
Requires-Dist: python-dateutil ; extra == 'all'
Requires-Dist: tornado ; extra == 'all'
Requires-Dist: boutiques ; extra == 'all'
Requires-Dist: black ; extra == 'all'
Requires-Dist: pre-commit ; extra == 'all'
Provides-Extra: dask
Requires-Dist: pytest (<6.0.0,>=4.4.0) ; extra == 'dask'
Requires-Dist: pytest-cov ; extra == 'dask'
Requires-Dist: pytest-env ; extra == 'dask'
Requires-Dist: pytest-xdist (<2.0) ; extra == 'dask'
Requires-Dist: pytest-rerunfailures ; extra == 'dask'
Requires-Dist: codecov ; extra == 'dask'
Requires-Dist: numpy ; extra == 'dask'
Requires-Dist: pyld ; extra == 'dask'
Requires-Dist: psutil ; extra == 'dask'
Requires-Dist: python-dateutil ; extra == 'dask'
Requires-Dist: tornado ; extra == 'dask'
Requires-Dist: boutiques ; extra == 'dask'
Requires-Dist: dask ; extra == 'dask'
Requires-Dist: distributed ; extra == 'dask'
Provides-Extra: dev
Requires-Dist: pytest (<6.0.0,>=4.4.0) ; extra == 'dev'
Requires-Dist: pytest-cov ; extra == 'dev'
Requires-Dist: pytest-env ; extra == 'dev'
Requires-Dist: pytest-xdist (<2.0) ; extra == 'dev'
Requires-Dist: pytest-rerunfailures ; extra == 'dev'
Requires-Dist: codecov ; extra == 'dev'
Requires-Dist: numpy ; extra == 'dev'
Requires-Dist: pyld ; extra == 'dev'
Requires-Dist: psutil ; extra == 'dev'
Requires-Dist: python-dateutil ; extra == 'dev'
Requires-Dist: tornado ; extra == 'dev'
Requires-Dist: boutiques ; extra == 'dev'
Requires-Dist: black ; extra == 'dev'
Requires-Dist: pre-commit ; extra == 'dev'
Provides-Extra: doc
Requires-Dist: attrs (>=19.1.0) ; extra == 'doc'
Requires-Dist: cloudpickle ; extra == 'doc'
Requires-Dist: filelock ; extra == 'doc'
Requires-Dist: packaging ; extra == 'doc'
Requires-Dist: sphinx (>=2.1.2) ; extra == 'doc'
Requires-Dist: sphinx-rtd-theme ; extra == 'doc'
Requires-Dist: sphinxcontrib-apidoc (~=0.3.0) ; extra == 'doc'
Requires-Dist: sphinxcontrib-napoleon ; extra == 'doc'
Requires-Dist: sphinxcontrib-versioning ; extra == 'doc'
Provides-Extra: docs
Requires-Dist: attrs (>=19.1.0) ; extra == 'docs'
Requires-Dist: cloudpickle ; extra == 'docs'
Requires-Dist: filelock ; extra == 'docs'
Requires-Dist: packaging ; extra == 'docs'
Requires-Dist: sphinx (>=2.1.2) ; extra == 'docs'
Requires-Dist: sphinx-rtd-theme ; extra == 'docs'
Requires-Dist: sphinxcontrib-apidoc (~=0.3.0) ; extra == 'docs'
Requires-Dist: sphinxcontrib-napoleon ; extra == 'docs'
Requires-Dist: sphinxcontrib-versioning ; extra == 'docs'
Provides-Extra: test
Requires-Dist: pytest (<6.0.0,>=4.4.0) ; extra == 'test'
Requires-Dist: pytest-cov ; extra == 'test'
Requires-Dist: pytest-env ; extra == 'test'
Requires-Dist: pytest-xdist (<2.0) ; extra == 'test'
Requires-Dist: pytest-rerunfailures ; extra == 'test'
Requires-Dist: codecov ; extra == 'test'
Requires-Dist: numpy ; extra == 'test'
Requires-Dist: pyld ; extra == 'test'
Requires-Dist: psutil ; extra == 'test'
Requires-Dist: python-dateutil ; extra == 'test'
Requires-Dist: tornado ; extra == 'test'
Requires-Dist: boutiques ; extra == 'test'
Provides-Extra: tests
Requires-Dist: pytest (<6.0.0,>=4.4.0) ; extra == 'tests'
Requires-Dist: pytest-cov ; extra == 'tests'
Requires-Dist: pytest-env ; extra == 'tests'
Requires-Dist: pytest-xdist (<2.0) ; extra == 'tests'
Requires-Dist: pytest-rerunfailures ; extra == 'tests'
Requires-Dist: codecov ; extra == 'tests'
Requires-Dist: numpy ; extra == 'tests'
Requires-Dist: pyld ; extra == 'tests'
Requires-Dist: psutil ; extra == 'tests'
Requires-Dist: python-dateutil ; extra == 'tests'
Requires-Dist: tornado ; extra == 'tests'
Requires-Dist: boutiques ; extra == 'tests'

|GHAction| |CircleCI| |codecov|

|Pydralogo|

.. |Pydralogo| image:: https://raw.githubusercontent.com/nipype/pydra/master/docs/logo/pydra_logo.jpg
   :width: 200px
   :alt: pydra logo

.. |GHAction| image:: https://github.com/nipype/pydra/workflows/Pydra/badge.svg
   :alt: GitHub Actions CI
   :target: https://github.com/nipype/Pydra/actions

.. |CircleCI| image:: https://circleci.com/gh/nipype/pydra.svg?style=svg
   :alt: CircleCI

.. |codecov| image:: https://codecov.io/gh/nipype/pydra/branch/master/graph/badge.svg
   :alt: codecov

======================
Pydra: Dataflow Engine
======================

A simple dataflow engine with scalable semantics.

Pydra is a rewrite of the Nipype engine with mapping and joining as
first-class operations. It forms the core of the Nipype 2.0 ecosystem.

The goal of pydra is to provide a lightweight Python dataflow engine for DAG
construction, manipulation, and distributed execution.

Feature list:
=============
1. Python 3.7+ using type annotation and `attrs <https://www.attrs.org/en/stable/>`_
2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
3. `splitter` and `combiner` provides many ways of compressing complex loop semantics
4. Cached execution with support for a global cache across dataflows and users
5. Distributed execution, presently via ConcurrentFutures, SLURM, and Dask (this is an experimental implementation with limited testing)

`API Documentation <https://nipype.github.io/pydra/>`_

Learn more about Pydra
======================

* `SciPy 2020 Proceedings <http://conference.scipy.org/proceedings/scipy2020/pydra.html>`_
* `PyCon 2020 Poster <https://docs.google.com/presentation/d/10tS2I34rS0G9qz6v29qVd77OUydjP_FdBklrgAGmYSw/edit?usp=sharing>`_
* `Explore Pydra interactively <https://github.com/nipype/pydra-tutorial>`_ (the tutorial can be also run using Binder service)

|Binder|

.. |Binder| image:: https://mybinder.org/badge_logo.svg
   :alt: Binder


Please note that mybinder times out after an hour.

Installation
============

::

    pip install pydra


Note that installation fails with older versions of pip on Windows. Upgrade pip before installing:

::

   pip install –upgrade pip
   pip install pydra


Developer installation
======================

Pydra requires Python 3.7+. To install in developer mode:

::

    git clone git@github.com:nipype/pydra.git
    cd pydra
    pip install -e .[dev]


In order to run pydra's test locally:

::

    pytest -vs pydra


If you want to test execution with Dask:

::

    git clone git@github.com:nipype/pydra.git
    cd pydra
    pip install -e .[dask]



It is also useful to install pre-commit:

::

    pip install pre-commit
    pre-commit


