.. _masker_objects:

================================================================
From neuroimaging volumes to data matrices: the masker objects
================================================================

This chapter introduces the maskers: objects that go from
neuroimaging volumes, on the disk or in memory, to data matrices, eg of
time series.


The concept of "masker" objects
===============================

In any analysis, the first step is to load the data.
It is often convenient to apply some basic data
transformations and to turn the data in a 2D (samples x features) matrix,
where the samples could be different time points, and the features derived
from different voxels (e.g., restrict analysis to the ventral visual stream),
regions of interest (e.g., extract local signals from spheres/cubes), or
pre-specified networks (e.g., look at data from all voxels of a set of
network nodes). Think of masker objects as swiss-army knives for shaping
the raw neuroimaging data in 3D space into the units of observation
relevant for the research questions at hand.

.. tip::
    Masker objects can transform both 3D and 4D image objects :

    - transforming a 3D image produces a 1D (features,) array,
    - transforming a 4D image produces a 2D (samples, features) array.


.. |niimgs| image:: ../images/niimgs.jpg
    :scale: 50%

.. |arrays| image:: ../images/feature_array.jpg
    :scale: 35%

.. |arrow| raw:: html

   <span style="padding: .5em; font-size: 400%">&rarr;</span>

.. centered:: |niimgs|  |arrow|  |arrays|

"masker" objects (found in module :mod:`nilearn.maskers`)
simplify these "data folding" steps that often precede the
statistical analysis.

Note that the masker objects may not cover all the image transformations
for specific tasks. Users who want to make some specific processing may
have to call :ref:`specific functions <preprocessing_functions>`
(modules :mod:`nilearn.signal`, :mod:`nilearn.masking`).

|

.. topic:: **Advanced: Design philosophy of "Maskers"**

    The design of these classes is similar to `scikit-learn
    <https://scikit-learn.org>`_\ 's transformers. First, objects are
    initialized with some parameters guiding the transformation
    (unrelated to the data). Then the `fit()` method should be called,
    possibly specifying some data-related information (such as number of
    images to process), to perform some initial computation (e.g.,
    fitting a mask based on the data). Finally, `transform()` can be
    called, with the data as argument, to perform some computation on
    data themselves (e.g., extracting time series from images).


.. currentmodule:: nilearn.maskers

.. _nifti_masker:

:class:`NiftiMasker`: applying a mask to load time-series
==========================================================

:class:`NiftiMasker` is a powerful tool to load images and
extract :term:`voxel` signals in the area defined by the mask.
It applies some basic preprocessing
steps with commonly used parameters as defaults.
But it is *very important* to look at your data to see the effects
of the preprocessings and validate them.

.. topic:: **Advanced: scikit-learn Pipelines**

    :class:`NiftiMasker` is a `scikit-learn <https://scikit-learn.org>`_ compliant
    transformer so that you can directly plug it
    into a :sklearn:`scikit-learn pipeline </modules/pipeline.html>`.


Custom data loading: loading only the first 100 time points
------------------------------------------------------------

Suppose we want to restrict a dataset to the first 100 frames. Below, we load
a movie-watching dataset with :func:`fetch_development_fmri()
<nilearn.datasets.fetch_development_fmri>`, restrict it to 100 frames and
build a new niimg object that we can give to the masker. Although
possible, there is no need to save your data to a file to pass it to a
:class:`NiftiMasker`. Simply use :func:`nilearn.image.index_img` to apply a
slice and create a :ref:`Niimg <niimg>` in memory:


.. literalinclude:: ../../examples/06_manipulating_images/plot_mask_computation.py
    :start-after: Load movie watching based brain development fMRI dataset
    :end-before: # To display the background

Controlling how the mask is computed from the data
--------------------------------------------------

In this section, we show how the masker object can compute a mask
automatically for subsequent statistical analysis.
On some datasets, the default algorithm may however perform poorly.
This is why it is very important to
**always look at your data** before and after feature
engineering using masker objects.

.. note::

    The full example described in this section can be found here:
    :doc:`plot_mask_computation.py <../auto_examples/06_manipulating_images/plot_mask_computation>`.
    It is also related to this example:
    :doc:`plot_nifti_simple.py <../auto_examples/06_manipulating_images/plot_nifti_simple>`.


Visualizing the computed mask
..............................

If a mask is not specified as an argument, :class:`NiftiMasker` will try to
compute one from the provided neuroimaging data.
It is *very important* to verify the quality of the generated mask by visualization.
This allows to see whether it is suitable for your data and intended analyses.
Alternatively, the mask computation parameters can still be modified.
See the :class:`NiftiMasker` documentation for a complete list of
mask computation parameters.

The mask can be retrieved and visualized from the ``mask_img_`` attribute
of the masker:

.. literalinclude:: ../../examples/06_manipulating_images/plot_mask_computation.py
    :start-after: # A NiftiMasker with the default strategy
    :end-before: # Plot the generated mask using the .generate_report method

.. figure:: ../auto_examples/06_manipulating_images/images/sphx_glr_plot_mask_computation_002.png
    :target: ../auto_examples/06_manipulating_images/plot_mask_computation.html
    :align: center

Alternatively, the mask can be visualized using the ``generate_report``
method of the masker. The generated report can be viewed in a Jupyter notebook,
opened in a new browser tab using ``report.open_in_browser()``,
or saved as a portable HTML file ``report.save_as_html(output_filepath)``.

.. literalinclude:: ../../examples/06_manipulating_images/plot_mask_computation.py
    :start-after: # We need to specify an 'epi' mask_strategy, as this is raw :term:`EPI` data
    :end-before: # Generate mask with strong opening

.. figure:: /images/niftimasker_report.png
    :target: ../auto_examples/06_manipulating_images/plot_mask_computation.html
    :align: center

Different masking strategies
.............................

The ``mask_strategy`` argument controls how the mask is computed:

* ``background``: detects a continuous background
* ``epi``: suitable for :term:`EPI` images
* ``whole-brain-template``: uses an :term:`MNI` whole-brain template
* ``gm-template``: uses an :term:`MNI` grey-matter template
* ``wm-template``: uses an :term:`MNI` white-matter template

Extra mask parameters: opening, cutoff...
..........................................

The underlying function is :func:`nilearn.masking.compute_epi_mask`
called using the ``mask_args`` argument of the :class:`NiftiMasker`.
Controlling these arguments set the fine aspects of the mask. See the
functions documentation, or :doc:`the NiftiMasker example
<../auto_examples/06_manipulating_images/plot_mask_computation>`.

.. figure:: /images/niftimasker_report_params.png
    :target: ../auto_examples/06_manipulating_images/plot_mask_computation.html
    :align: center

.. _masker_preprocessing_steps:

Common data preparation steps: smoothing, filtering, resampling
----------------------------------------------------------------

:class:`NiftiMasker` comes with many parameters that enable data
preparation::

   >>> import sklearn; sklearn.set_config(print_changed_only=False)
   >>> from nilearn import maskers
   >>> masker = maskers.NiftiMasker()
   >>> masker # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE
   NiftiMasker(clean_args=None, cmap='gray', detrend=False, dtype=None, high_pass=None,
         high_variance_confounds=False, low_pass=None, mask_args=None,
         mask_img=None, mask_strategy='background',
         memory=None, memory_level=1, reports=True,
         runs=None, smoothing_fwhm=None, standardize=False,
         standardize_confounds=True, t_r=None,
         target_affine=None, target_shape=None, verbose=0)

The meaning of each parameter is described in the documentation of
:class:`NiftiMasker` (click on the name :class:`NiftiMasker`), here we
comment on the most important.

.. topic:: **`dtype` argument**

    Forcing your data to have a ``dtype`` of **float32** can help
    save memory and is often a good-enough numerical precision.
    You can force this cast by choosing ``dtype`` to be 'auto'.
    In the future this cast will be the default behavior.


.. seealso::

   If you do not want to use the :class:`NiftiMasker` to perform these
   simple operations on data, note that they can also be manually
   accessed in nilearn such as in
   :ref:`corresponding functions <preprocessing_functions>`.

Smoothing
.........

:class:`NiftiMasker` can apply Gaussian spatial smoothing to the
neuroimaging data, useful to fight noise or for inter-individual
differences in neuroanatomy. It is achieved by specifying the
:term:`full-width half maximum<FWHM>` (:term:`FWHM`; in millimeter
scale) with the ``smoothing_fwhm`` parameter. Anisotropic filtering
is also possible by passing 3 scalars ``(x, y, z)``, the
:term:`FWHM` along the x, y, and z direction.

The underlying function handles properly non-cubic :term:`voxels<voxel>`
by scaling the given widths appropriately.

.. seealso::

   :func:`nilearn.image.smooth_img`

.. _temporal_filtering:

Temporal Filtering and confound removal
........................................

:class:`NiftiMasker` can also improve aspects of temporal data
properties, before conversion to :term:`voxel` signals.

- **Standardization**. Parameter ``standardize``: Signals can be
  standardized (scaled to unit variance).

- **Frequency filtering**. Low-pass and high-pass filters can be used to
  remove artifacts. Parameters: ``high_pass`` and ``low_pass``, specified
  in Hz (note that you must specific the sampling rate in seconds with
  the ``t_r`` parameter: ``loss_pass=.5, t_r=2.1``).

- **Confound removal**. Two ways of removing confounds are provided: simple
  detrending or using prespecified confounds, such as behavioral or movement
  information.

  * Linear trends can be removed by activating the ``detrend`` parameter.
    This accounts for slow (as opposed to abrupt or transient) changes
    in :term:`voxel` values along a series of brain images that are unrelated to the
    signal of interest (e.g., the neural correlates of cognitive tasks).
    It is not activated by default in :class:`NiftiMasker` but is recommended
    in almost all scenarios.

  * More complex confounds, measured during the acquision, can be removed
    by passing them to :meth:`NiftiMasker.transform`. If the dataset
    provides a confounds file, just pass its path to the masker. For
    :term:`fMRIPrep` outputs, one can use
    :func:`~nilearn.interfaces.fmriprep.load_confounds` or
    :func:`~nilearn.interfaces.fmriprep.load_confounds_strategy` to select
    confound variables with some basic sanity check based on
    :term:`fMRIPrep` documentation.

.. topic:: **Exercise**
   :class: green

   You can, more as a training than as an exercise, try to play with
   the parameters in
   :ref:`sphx_glr_auto_examples_00_tutorials_plot_decoding_tutorial.py`.
   Try to enable detrending and run the script:
   does it have a big impact on the result?

.. note::

   Please see the usage example of
   :func:`~nilearn.interfaces.fmriprep.load_confounds` and
   :func:`~nilearn.interfaces.fmriprep.load_confounds_strategy` in
   :doc:`plot_signal_extraction.py <../auto_examples/03_connectivity/plot_signal_extraction>`.

.. seealso::

   :func:`nilearn.signal.clean`


Resampling: resizing and changing resolutions of images
.......................................................

:class:`NiftiMasker` and many similar classes enable resampling
(recasting of images into different resolutions and transformations of
brain :term:`voxel` data). Two parameters control resampling:

* ``target_affine`` to resample (resize, rotate...) images in order to match
  the spatial configuration defined by the new affine (i.e., matrix
  transforming from :term:`voxel` space into world space).

* Additionally, a ``target_shape`` can be used to resize images
  (i.e., cropping or padding with zeros) to match an expected data
  image dimensions (shape composed of x, y, and z).

How to combine these parameter to obtain the specific resampling desired
is explained in details in :ref:`resampling`.

.. seealso::

   :func:`nilearn.image.resample_img`, :func:`nilearn.image.resample_to_img`

.. _unmasking_step:

Inverse transform: unmasking data
---------------------------------

.. note::

  Inverse transform only performs spatial unmasking.
  The data is only brought back into either a 3D or 4D represenetation,
  without inverting any signal processing performed by ``transform``.

Once :term:`voxel` signals have been processed, the result can be visualized as
images after unmasking (masked-reduced data transformed back into
the original whole-brain space). This step is present in many
:ref:`examples <examples-index>` provided in nilearn. Below you will find
an excerpt of :ref:`the example performing Anova-SVM on the Haxby data
<sphx_glr_auto_examples_02_decoding_plot_haxby_anova_svm.py>`:

.. literalinclude:: ../../examples/02_decoding/plot_haxby_anova_svm.py
    :start-after: # Look at the SVC's discriminating weights
    :end-before: # Or we can plot the weights

|

.. tip::
    Masker objects can inverse-transform both 1D and 2D arrays :

    - inverse-transforming a 2D array produces a 4D (X x Y x Z x samples) image,
    - inverse-transforming a 1D array produces a 3D (X x Y x Z) image.

.. topic:: **Examples to better understand the NiftiMasker**

   * :ref:`sphx_glr_auto_examples_06_manipulating_images_plot_nifti_simple.py`

   * :ref:`sphx_glr_auto_examples_06_manipulating_images_plot_mask_computation.py`

|

.. _region:

Extraction of signals from regions:\  :class:`NiftiLabelsMasker`, :class:`NiftiMapsMasker`
==========================================================================================

The purpose of :class:`NiftiLabelsMasker` and :class:`NiftiMapsMasker` is to
compute signals from regions containing many voxels. They make it easy to get
these signals once you have an atlas or a :term:`parcellation` into brain regions.

Regions definition
------------------

Nilearn understands two different ways of defining regions, which are called
labels and maps, handled by :class:`NiftiLabelsMasker` and
:class:`NiftiMapsMasker`, respectively.

- labels: a single region is defined as the set of all the voxels that have a
  common label (e.g., anatomical brain region definitions as integers)
  in the region definition array. The set of
  regions is defined by a single 3D array, containing a voxel-wise
  dictionary of label numbers that denote what
  region a given :term:`voxel` belongs to. This technique has a big advantage: the
  required memory load is independent of the number of regions, allowing
  for a large number of regions. On the other hand, there are
  several disadvantages: regions cannot spatially overlap
  and are represented in a binary present/nonpresent coding (no weighting).

- maps: a single region is defined as the set of all the voxels that have a
  non-zero weight. A set of regions is thus defined by a set of 3D images (or a
  single 4D image), one 3D image per region (as opposed to all regions in a
  single 3D image such as for labels, cf. above).
  While these defined weighted regions can exhibit spatial
  overlap (as opposed to labels), storage cost scales linearly with the
  number of regions. Handling a large number (e.g., thousands)
  of regions will prove difficult with this data transformation of
  whole-brain :term:`voxel` data into weighted region-wise data.

.. note::

   These usage are illustrated in the section :ref:`functional_connectomes`.

:class:`NiftiLabelsMasker` Usage
--------------------------------

Usage of :class:`NiftiLabelsMasker` is similar to that of
:class:`NiftiMapsMasker`. The main difference is that it requires a labels image
instead of a set of maps as input.

The ``background_label`` keyword of :class:`NiftiLabelsMasker` deserves
some explanation. The voxels that correspond to the brain or a region
of interest in an :term:`fMRI` image do not fill the entire image.
Consequently, in the labels image, there must be a label value that corresponds
to "outside" the brain (for which no signal should be extracted).
By default, this label is set to zero in nilearn (referred to as "background").
Should some non-zero value encoding be necessary, it is possible
to change the background value with the ``background_label`` keyword.

.. topic:: **Examples**

    * :ref:`sphx_glr_auto_examples_03_connectivity_plot_signal_extraction.py`

:class:`NiftiMapsMasker` Usage
------------------------------

This atlas defines its regions using maps. The path to the corresponding
file is given in the ``maps_img`` argument.

One important thing that happens transparently during the execution of
:meth:`NiftiMasker.fit_transform` is resampling. Initially, the images
and the atlas do typically not have the same shape nor the same affine.
Casting them into the same format is required for successful signal extraction
The keyword argument ``resampling_target`` specifies which format
(i.e., dimensions and affine) the data should be resampled to.
See the reference documentation for :class:`NiftiMapsMasker` for every
possible option.

.. topic:: **Examples**

   * :ref:`sphx_glr_auto_examples_03_connectivity_plot_probabilistic_atlas_extraction.py`

Extraction of signals from regions for multiple subjects:\  :class:`MultiNiftiMasker`, :class:`MultiNiftiLabelsMasker`, :class:`MultiNiftiMapsMasker`
=====================================================================================================================================================

The purpose of :class:`MultiNiftiMasker`, :class:`MultiNiftiLabelsMasker` and
:class:`MultiNiftiMapsMasker` is to extend the capabilities of
:class:`NiftiMasker`, :class:`NiftiLabelsMasker` and :class:`NiftiMapsMasker`
as to facilitate the computation of :term:`voxel` signals in multi-subjects settings.
While :class:`NiftiMasker`, :class:`NiftiLabelsMasker` and
:class:`NiftiMapsMasker` work with 3D inputs (single brain volume) or 4D inputs
(sequence of brain volumes in time for one subject), :class:`MultiNiftiMasker`,
:class:`MultiNiftiLabelsMasker` and :class:`MultiNiftiMapsMasker`
can also handle list of 3D or 4D image objects.

.. tip::
    MultiMasker objects can transform both 3D, 4D,
    as well as list of 3D or 4D image objects :

    - transforming a 3D image produces a 1D (features,) array,
    - transforming a 4D image produces a 2D (samples, features) array,
    - transforming a list of 3D image produces a list of 1D (features,) array,
    - transforming a list of 4D image produces a list of 2D (samples, features) array.

:class:`MultiNiftiMasker` Usage
-------------------------------

:class:`MultiNiftiMasker` extracts :term:`voxel` signals for each subject in the areas defined by the
masks.

.. topic:: **Examples**

    * :ref:`sphx_glr_auto_examples_02_decoding_plot_miyawaki_reconstruction.py`
    * :ref:`sphx_glr_auto_examples_02_decoding_plot_miyawaki_encoding.py`

:class:`MultiNiftiLabelsMasker` Usage
-------------------------------------

:class:`MultiNiftiLabelsMasker` extracts signals from regions defined by labels
for each subject.

.. topic:: **Examples**

    * :ref:`sphx_glr_auto_examples_03_connectivity_plot_atlas_comparison.py`

:class:`MultiNiftiMapsMasker` Usage
-----------------------------------

:class:`MultiNiftiMapsMasker` extracts signals regions defined by maps
for each subject.

.. topic:: **Examples**

    * :ref:`sphx_glr_auto_examples_03_connectivity_plot_atlas_comparison.py`

Extraction of signals from seeds:\  :class:`NiftiSpheresMasker`
===============================================================

The purpose of :class:`NiftiSpheresMasker` is to compute signals from
seeds containing voxels in spheres. It makes it easy to get these signals once
you have a list of coordinates.
A single seed is a sphere defined by the radius (in millimeters) and the
coordinates (typically :term:`MNI` or TAL) of its center.

Using :class:`NiftiSpheresMasker` needs to define a list of coordinates.
``seeds`` argument takes a list of 3D coordinates (tuples) of the spheres centers,
they should be in the same space as the images.
Seeds can overlap spatially and are represented in a binary present/nonpresent
coding (no weighting).
Below is an example of a coordinates list of four seeds from the default mode network::

  >>> dmn_coords = [(0, -52, 18), (-46, -68, 32), (46, -68, 32), (0, 50, -5)]

``radius`` is an optional argument that takes a real value in millimeters.
If no value is given for the ``radius`` argument, the single :term:`voxel` at the given
seed position is used.

.. topic:: **Examples**

  * :ref:`sphx_glr_auto_examples_03_connectivity_plot_sphere_based_connectome.py`


Extraction of signals from surface images\  :class:`SurfaceMasker`, :class:`SurfaceLabelsMasker`, :class:`SurfaceMapsMasker`
============================================================================================================================

The purpose of :class:`SurfaceMasker`, :class:`SurfaceLabelsMasker`, :class:`SurfaceMapsMasker`
is to mirror the capabilities of
:class:`NiftiMasker`, :class:`NiftiLabelsMasker` and :class:`NiftiMapsMasker`
but to extract data from :class:`~nilearn.surface.SurfaceImage`.

They can perform data extraction from 1D surface data (n_vertices),
2D surface data (n_vertices x samples)
as well as list of 1D or 2D surface data with the same underlying mesh.

.. tip::
    Surface masker objects can transform both 1D, 2D,
    as well as list of 1D or 2D surface image objects.
    Transforming a 1D image produces a 1D (features,) array.
    All other input will produce a 1D (samples, features) array..

    Surface masker objects can inverse-transform both 1D and 2D arrays :

    - inverse-transforming a 1D array produces a 1D (n_vertices,) image,
    - inverse-transforming a 2D array produces a 2D (n_vertices, samples) image.
