==================================
Pydra task package for niworkflows
==================================

.. image:: https://github.com/nipype/pydra-niworkflows/actions/workflows/ci-cd.yaml/badge.svg
   :target: https://github.com/nipype/pydra-niworkflows/actions/workflows/ci-cd.yaml
.. image:: https://codecov.io/gh/nipype/pydra-niworkflows/branch/main/graph/badge.svg?token=UIS0OGPST7
   :target: https://codecov.io/gh/nipype/pydra-niworkflows
.. image:: https://img.shields.io/pypi/pyversions/pydra-niworkflows.svg
   :target: https://pypi.python.org/pypi/pydra-niworkflows/
   :alt: Supported Python versions
.. image:: https://img.shields.io/pypi/v/pydra-niworkflows.svg
   :target: https://pypi.python.org/pypi/pydra-niworkflows/
   :alt: Latest Version


This package contains a collection of Pydra task interfaces for the niworkflows toolkit.
The basis of this collection has been formed by the semi-automatic conversion of
existing `Nipype <https://github.com/nipy/nipype>`__ interfaces to Pydra using the
`Nipype2Pydra <https://github.com/nipype/nipype2pydra>`__ tool


Automatically-generated vs manually-curated tasks
-------------------------------------------------

Automatically generated tasks can be found in the `pydra.tasks.niworkflows.auto` package.
These packages should be treated with extreme caution as they likely do not pass testing.
Generated tasks that have been edited and pass testing are imported into one or more of the
`pydra.tasks.niworkflows.v*` packages, corresponding to the version of the niworkflows toolkit
they are designed for. 

Tests
-----

This package comes with a battery of automatically generated test modules. To install
the necessary dependencies to run the tests

.. code-block::

   $ pip install -e .[test]

Then the tests, including `doctests` <https://docs.python.org/3/library/doctest.html>`__, can be launched using

.. code-block::

   $ pytest --doctest-modules pydra/tasks/*

By default, the tests are set to time-out after 10s, after which the underlying tool is
assumed to have passed the validation/initialisation phase and we assume that it will
run to completion. To disable this and run the test(s) through to completion run

.. code-block::

   $ pytest --doctest-modules --timeout-pass 0 pydra/tasks/*

Continuous integration
----------------------

This template uses `GitHub Actions <https://docs.github.com/en/actions/>`__` to run tests and
deploy packages to PYPI. New packages are built and uploaded when releases are created on
GitHub, or new releases of Nipype or the Nipype2Pydra conversion tool are released.
Releases triggered by updates to Nipype or Nipype2Pydra are signified by the `postN`
suffix where `N = <nipype-version><nipype2pydra-version>` with the '.'s stripped, e.g.
`v0.2.3post185010` corresponds to the v0.2.3 tag of this repository with auto-generated
packages from Nipype 1.8.5 using Nipype2Pydra 0.1.0.


Contributing to this package
----------------------------

Developer installation
~~~~~~~~~~~~~~~~~~~~~~

Install the `fileformats <https://arcanaframework.github.io/fileformats/>`__ packages
corresponding to AFNI specific file formats


.. code-block::

   $ pip install -e ./related-packages/fileformats[dev]
   $ pip install -e ./related-packages/fileformats-extras[dev]

Install repo in developer mode from the source directory and install pre-commit to
ensure consistent code-style and quality.

.. code-block::

   $ pip install -e .[test,dev]
   $ pre-commit install

Next install the requirements for running the auto-conversion script and generate the
Pydra task interfaces from their Nipype counterparts

.. code-block::

   $ pip install -r nipype-auto-conv/requirements.txt

The run the conversion script to convert Nipype interfaces to Pydra

.. code-block::

   $ nipype-auto-conv/generate

Methodology
~~~~~~~~~~~

The development of this package is expected to have two phases

1. Where the corresponding Nipype interfaces are considered to be the ground truth, and
   the Pydra tasks are generated from them
2. When the Pydra tasks are considered be mature and they are edited by hand

Different tasks will probably mature at different times so there will probably be an
intermediate phase between 1 and 2.

Auto-conversion phase
~~~~~~~~~~~~~~~~~~~~~

The auto-converted Pydra tasks are generated from their corresponding Nipype interface
in combination with "conversion hints" contained in YAML specs
located in `nipype-auto-conv/specs/`. The self-documented conversion specs are
to be edited by hand in order to assist the auto-converter produce valid pydra tasks.
After editing one or more conversion specs the `pydra.tasks.niworkflows.auto` package should
be regenerated by running

.. code-block::

   $ nipype-auto-conv/generate

The tests should be run on the auto-generated tasks to see if they are valid

.. code-block::

   $ pytest --doctest-modules pydra/tasks/niworkflows/auto/tests/test_<the-name-of-the-task-you-edited>.py

If the test passes you should then edit the `pydra/tasks/niworkflows/v<tool-version>/__init__.py` file
to import the now valid task interface to signify that it has been validated and is ready
for use, e.g.

.. code-block::python

   from pydra.tasks.niworkflows.auto import <the-task-you-have-validated>


Typing and sample test data
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The automatically generated tests will attempt to provided the task instance to be tested
with sensible default values based on the type of the field and any constraints it has
on it. However, these will often need to be manually overridden after consulting the
underlying tool's documentation.

For file-based data, automatically generated file-system objects will be created for
selected format types, e.g. Nifti, Dicom. Therefore, it is important to specify the
format of the file using the "mime-like" string corresponding to a
`fileformats <https://github.com/ArcanaFramework/fileformats>`__ class
in the ``inputs > types`` and ``outputs > types`` dicts of the YAML spec.

If the required file-type is not found implemented within fileformats, please see the `fileformats
docs <https://arcanaframework.github.io/fileformats/developer.html>`__ for instructions on how to define
new fileformat types, and see 
`fileformats-medimage-extras <https://github.com/ArcanaFramework/fileformats-medimage-extras/blob/6c2dabe91e95687eebc2639bb6f034cf9595ecfc/fileformats/extras/medimage/nifti.py#L30-L48>`__
for an example on how to implement methods to generate sample data for them.
