=======================
Neo 0.9.0 release notes
=======================

10th November 2020


Group and ChannelView replace Unit and ChannelIndex
---------------------------------------------------

Experience with :class:`ChannelIndex` and :class:`Unit` has shown that these classes are
often confusing and difficult to understand.
In particular, :class:`ChannelIndex` was trying to provide three different functionalities in a
single object:

- providing information about individual traces within :class:`AnalogSignals` like the channel id and the channel name (labelling)
- grouping a subset of traces within an :class:`AnalogSignal` via the ``index`` attribute (masking)
- linking between / grouping :class:`AnalogSignals` (grouping)

while grouping :class:`SpikeTrains` required a different class, :class:`Unit`.
For more pointers to the difficulties this created, and some of the limitations of this approach,
see `this Github issue`_.

With the aim of making the three functionalities of labelling, masking and grouping
both easier to use and more flexible, we have replaced :class:`ChannelIndex` and :class:`Unit`
with:

- array annotations (*labelling*) - already available since Neo 0.8
- :class:`~neo.core.ChannelView` (*masking*) - defines subsets of channels within an `AnalogSignal` using a mask
- :class:`~neo.core.Group`  (*grouping*) - allows any Neo object except :class`Segment` and :class:`Block` to be grouped

For some guidance on migrating from :class:`ChannelIndex`/:class:`Unit`
to :class:`Group` and :class:`ChannelView` see the appendix at the bottom of this page.

Python 3 only
-------------

We have now dropped support for Python 2.7 and Python 3.5, and for versions of NumPy older than 1.13.
In future, we plan to follow NEP29_ + one year, i.e. we will support Python and NumPy versions
for one year longer than recommended in NEP29. This was `discussed here`_.

Change in default behaviour for grouping channels in IO modules
---------------------------------------------------------------

Previously, when reading multiple related signals (same length, same units) from a file,
some IO classes would by default create a separate, single-channel :class:`AnalogSignal` per signal,
others would combine all related signals into one multi-channel :class:`AnalogSignal`.

From Neo 0.9.0, the default for all IO classes is to create a one multi-channel :class:`AnalogSignal`.
To get the "multiple single-channel signals" behaviour, use::

    io.read(signal_group_mode="split-all")

Other new or modified features
------------------------------

* added methods :func:`rectify()`, :func:`downsample` and :func:`resample` to :class:`AnalogSignal`
* :func:`SpikeTrain.merge()` can now merge multiple spiketrains
* the utility function :func:`cut_block_by_epochs()` gives a new :class:`Block` now
  rather than modifying the block in place
* some missing properties such as ``t_start`` were added to :class:`ImageSequence`,
  and ``sampling_period`` was renamed to ``frame_duration``
* :func:`AnalogSignal.time_index()` now accepts arrays of times, not just a scalar.

See all `pull requests`_ included in this release and the `list of closed issues`_.

Bug fixes and improvements in IO modules
----------------------------------------

* NeoMatlabIO (support for signal annotations)
* NeuralynxIO (fix handling of empty .nev files)
* AxonIO (support EDR3 header, fix channel events bug)
* Spike2IO (fix rounding problem, fix for v9 SON files)
* MicromedIO (fix label encoding)


Acknowledgements
----------------

Thanks to Julia Sprenger, Samuel Garcia, Andrew Davison, Alexander Kleinjohann, Hugo van Kemenade,
Achilleas Koutsou, Jeffrey Gill, Corentin Fragnaud, Aitor Morales-Gregorio, Rémi Proville,
Robin Gutzen, Marin Manuel, Simon Danner, Michael Denker, Peter N. Steinmetz, Diziet Asahi and
Lucien Krapp for their contributions to this release.


Appendix: Migrating from ChannelIndex/Unit to ChannelView/Group
===============================================================

While the basic hierarchical :class:`Block` - :class:`Segment` structure of Neo has remained
unchanged since the inception of Neo, the structures used to cross-link objects
(for example to link a signal to the spike trains derived from it) have undergone changes,
in an effort to find an easily understandable and usable approach.

Below we give some examples of how to migrate from :class:`ChannelIndex` and :class:`Unit`,
as used in Neo 0.8, to the new classes :class:`Group` and :class:`ChannelView`
introduced in Neo 0.9.
Note that Neo 0.9 supports the new and old API in parallel, to facilitate migration.
IO classes in Neo 0.9 can read :class:`ChannelIndex` and :class:`Unit` objects,
but do not write them.

:class:`ChannelIndex` and :class:`Unit` will be removed in Neo 0.10.0.

Examples
--------

A simple example with two tetrodes. Here the :class:`ChannelIndex` was not being
used for grouping, simply to associate a name with each channel.

Using :class:`ChannelIndex`::

    import numpy as np
    from quantities import kHz, mV
    from neo import Block, Segment, ChannelIndex, AnalogSignal

    block = Block()
    segment = Segment()
    segment.block = block
    block.segments.append(segment)

    for i in (0, 1):
        signal = AnalogSignal(np.random.rand(1000, 4) * mV,
                              sampling_rate=1 * kHz,)
        segment.analogsignals.append(signal)
        chx = ChannelIndex(name=f"Tetrode #{i + 1}",
                           index=[0, 1, 2, 3],
                           channel_names=["A", "B", "C", "D"])
        chx.analogsignals.append(signal)
        block.channel_indexes.append(chx)

Using array annotations, we annotate the channels of the :class:`AnalogSignal` directly::

    import numpy as np
    from quantities import kHz, mV
    from neo import Block, Segment, AnalogSignal

    block = Block()
    segment = Segment()
    segment.block = block
    block.segments.append(segment)

    for i in (0, 1):
        signal = AnalogSignal(np.random.rand(1000, 4) * mV,
                              sampling_rate=1 * kHz,
                              channel_names=["A", "B", "C", "D"])
        segment.analogsignals.append(signal)


Now a more complex example: a 1x4 silicon probe, with a neuron on channels 0,1,2 and another neuron on channels 1,2,3.
We create a :class:`ChannelIndex` for each neuron to hold the :class:`Unit` object associated with this spike sorting group.
Each :class:`ChannelIndex` also contains the list of channels on which that neuron spiked.

::

    import numpy as np
    from quantities import ms, mV, kHz
    from neo import Block, Segment, ChannelIndex, Unit, SpikeTrain, AnalogSignal

    block = Block(name="probe data")
    segment = Segment()
    segment.block = block
    block.segments.append(segment)

    # create 4-channel AnalogSignal with dummy data
    signal = AnalogSignal(np.random.rand(1000, 4) * mV,
                          sampling_rate=10 * kHz)
    # create spike trains with dummy data
    # we will pretend the spikes have been extracted from the dummy signal
    spiketrains = [
        SpikeTrain(np.arange(5, 100) * ms, t_stop=100 * ms),
        SpikeTrain(np.arange(7, 100) * ms, t_stop=100 * ms)
    ]
    segment.analogsignals.append(signal)
    segment.spiketrains.extend(spiketrains)
    # assign each spiketrain to a neuron (Unit)
    units = []
    for i, spiketrain in enumerate(spiketrains):
        unit = Unit(name=f"Neuron #{i + 1}")
        unit.spiketrains.append(spiketrain)
        units.append(unit)

    # create a ChannelIndex for each unit, to show which channels the spikes come from
    chx0 = ChannelIndex(name="Channel Group 1", index=[0, 1, 2])
    chx0.units.append(units[0])
    chx0.analogsignals.append(signal)
    units[0].channel_index = chx0
    chx1 = ChannelIndex(name="Channel Group 2", index=[1, 2, 3])
    chx1.units.append(units[1])
    chx1.analogsignals.append(signal)
    units[1].channel_index = chx1

    block.channel_indexes.extend((chx0, chx1))


Using :class:`ChannelView` and :class:`Group`::

    import numpy as np
    from quantities import ms, mV, kHz
    from neo import Block, Segment, ChannelView, Group, SpikeTrain, AnalogSignal

    block = Block(name="probe data")
    segment = Segment()
    segment.block = block
    block.segments.append(segment)

    # create 4-channel AnalogSignal with dummy data
    signal = AnalogSignal(np.random.rand(1000, 4) * mV,
                          sampling_rate=10 * kHz)
    # create spike trains with dummy data
    # we will pretend the spikes have been extracted from the dummy signal
    spiketrains = [
        SpikeTrain(np.arange(5, 100) * ms, t_stop=100 * ms),
        SpikeTrain(np.arange(7, 100) * ms, t_stop=100 * ms)
    ]
    segment.analogsignals.append(signal)
    segment.spiketrains.extend(spiketrains)
    # assign each spiketrain to a neuron (now using Group)
    units = []
    for i, spiketrain in enumerate(spiketrains):
        unit = Group([spiketrain], name=f"Neuron #{i + 1}")
        units.append(unit)

    # create a ChannelView of the signal for each unit, to show which channels the spikes come from
    # and add it to the relevant Group
    view0 = ChannelView(signal, index=[0, 1, 2], name="Channel Group 1")
    units[0].add(view0)
    view1 = ChannelView(signal, index=[1, 2, 3], name="Channel Group 2")
    units[1].add(view1)

    block.groups.extend(units)


Now each putative neuron is represented by a :class:`Group` containing the spiketrains of that neuron
and a view of the signal selecting only those channels from which the spikes were obtained.



.. _`list of closed issues`: https://github.com/NeuralEnsemble/python-neo/issues?q=is%3Aissue+milestone%3A0.9.0+is%3Aclosed
.. _`pull requests`: https://github.com/NeuralEnsemble/python-neo/pulls?q=is%3Apr+is%3Aclosed+merged%3A%3E2019-09-30+milestone%3A0.9.0
.. _NEP29: https://numpy.org/neps/nep-0029-deprecation_policy.html
.. _`discussed here`: https://github.com/NeuralEnsemble/python-neo/issues/788
.. _`this Github issue`: https://github.com/NeuralEnsemble/python-neo/issues/456
