.. performance tutorial

.. _performance:

Performance optimisation in pynbody
===================================

``Pynbody`` is built on top of ``numpy``, which means that learning how to optimize ``numpy``
array manipulations is the most important route to writing efficient code; see http://scipy-lectures.github.com/advanced/optimizing/index.html
for an introduction.

However there are a few issues which are specific to ``pynbody``. First, a large
number of ``pynbody``'s most common operations are parallelized: make sure you
have set up these routines to behave in a way that matches your needs by reading
the page on :ref:`threads`.

Other than that, there are some more subtle issues. The most important of these
that we have come across is the overheads incurred by using ``SubSnap``s
which are explained
below.


Overheads of SubSnaps
------------------------


A template for performance-critical code
********************************************

To cut a long story short, if your routine does a lot of array access on an object which might
be a ``SubSnap`` of a certain flavour (explained further below), you will find that wrapping your
code as follows speeds it up.

.. code-block:: python

 def my_expensive_operation(sim_or_subsim) :
     with sim_or_subsim.immediate_mode :
         mass_array = sim_or_subsim['mass']
         other_array = sim_or_subsim['other']
         # ... get other arrays required...

         #
         # perform multiple operations on arrays
         #

         # At end, copy back results if the arrays have
         # changed
         sim_or_subsim['other'] = other_array

The remainder of this document unpacks what this does and why it should be necessary.

What is a SubSnap, really?
****************************

When you construct a ``SubSnap``, the framework records which particles of the underlying
``SimSnap`` are included and which are not. Thereafter, if you access an array from the ``SubSnap``,
it is constructed in one of two ways.

 - If the set of particles indexed by the ``SubSnap`` is expressable a slice, the
   arrays constructed are still ``numpy`` arrays.  This will be the case if you explicitly
   slice the simulation (e.g. ``f[2:100:3]``), or if you ask for a specific particle family (e.g. ``f.dm``).

 - If the set of particles is not expressable in this way, the arrays constructed are
   emulating ``numpy`` arrays and this can become expensive (see below). This will
   be the cae if you ask for a list of particles (e.g. ``f[[2,10,15,22]]``) use a ``numpy``-like
   indexing trick (e.g. ``f[f['x']>10]`` or ``f[numpy.where(f['x']>10)]``) or use a
   :py:mod:`filter <pynbody.filt>`.

In the first case, the optimization is unchanged from the raw ``SimSnap`` case.
In the second case, the situation is different.
To understand how and why, we
need to look at the difference between an indexed and a sliced ``numpy`` array.


Numpy's behaviour
***********************

This section explains why the reason for the slightly awkward design of ``IndexedSubArray``.
The ``pynbody`` framework requires all sub-arrays to continue pointing to the original data
but a simple experiment with numpy shows that it does not enable this behaviour in all
cases that we want to cover.

Here's what happens when you use a slice of an existing ``numpy`` array.

.. ipython::

 In [2]: import numpy as np

 In [3]: a = np.zeros(10)

 In [4]: b = a[3:5]

 In [5]: b[1] = 100

 In [6]: a
 Out[6]: array([   0.,    0.,    0.,    0.,  100.,    0.,    0.,    0.,    0.,    0.])

The ``a`` array has been updated as required, because the ``b`` and ``a`` objects
actually point back to the same part of the computer memory.

On the other hand, when you *index* a ``numpy`` array, the behaviour is different.

.. ipython::

 In [7]: c = a[[4,5,6]]

 In [8]: c[1] = 200

 In [9]: a
 Out[9]: array([   0.,    0.,    0.,    0.,  100.,    0.,    0.,    0.,    0.,    0.])

Here changing ``c`` has not updated ``a``. That's because the construction of ``c`` actually
*copied* the relevant data instead of just pointing back at it.  This is necessitated by
the underlying design of ``numpy`` arrays requiring the data to lie in a predictable
pattern in the memory.

Back to pynbody
******************

The ``IndexedSubArray`` class fixes this problem:

.. ipython ::

 In [10]: import pynbody

 In [12]: d = pynbody.array.IndexedSimArray(a, [4,5,6])

 In [13]: d[1] = 200

 In [14]: a
 Out[14]: array([   0.,    0.,    0.,    0.,  100.,  200.,    0.,    0.,    0.,    0.])

Note that ``a`` has been updated correctly. This is achieved by the ``IndexedSimArray``
*emulating*, rather than *wrapping*, a ``numpy`` array; internally
the syntax ``d[1]=200`` is then translated into ``a[[4,5,6][1]]=200``.

The cost of this is that each time you call a function that requires a ``numpy`` array
as an input, the ``IndexedSimArray`` has to generate a proxy for this purpose. This can become slow.
Have a look at the following timings:

.. ipython ::

 In [22]: %time for i in range(10000) : d+=1
 CPU times: user 0.25 s, sys: 0.03 s, total: 0.28 s
 Wall time: 0.26 s

 In [23]: %time for i in range(10000) : a+=1
 CPU times: user 0.04 s, sys: 0.00 s, total: 0.04 s
 Wall time: 0.04 s

Adding to the subarray is *slower* than adding to the entire array!
This is because of the overheads of continually constructing proxy
``numpy`` arrays to pass to the ``__add__`` method.


How to remove this bottleneck
***************************************

We should emphasize that the example above is quite contrived, since it forces
re-construction of the ``numpy`` proxy 10000 times. In user code,
the number of ``numpy`` proxies that have to be constructed will be vastly smaller,
so the fractional overheads are normally quite small.

Nonetheless, it does sometimes become a problem for performance-critical code.
For that reason, it's possible to avoid constructing ``IndexedSimArray``s altogether
and force only ``numpy`` arrays to be returned. This means you must take responsibility
for understanding which operations copy, as opposed to referencing, data.

This is known as ``immediate mode`` and is activated using python's ``with`` mechanism.
Let's create a test snapshot and a subview into that snapshot to try it out.

.. ipython ::

 In [24]: f = pynbody.new(dm=100)

 In [25]: sub_f = f[[20,21,22]]

Under normal conditions, the type of arrays returned from ``sub_f`` is ``IndexedSimArray``.
Updating one of these arrays will transparently update the main snapshot.

.. ipython ::

 In [36]: sub_mass = sub_f['mass']

 In [30]: type(sub_mass)

 In [37]: sub_mass[:]=3

 In [35]: f['mass'][[20,21,22]]
 [ 3.  3.  3.]


Conversely, in ``immediate mode``, the type of arrays returned from ``sub_f`` is ``SimArray`` (so just
a wrapper round a real ``numpy`` array). But updating that returned ``numpy`` array has *no effect* on the
parent snapshot.

.. ipython ::

 In [32]: with f.immediate_mode :
    ....:     sub_mass = sub_f['mass']
    ....:

 In [30]: type(sub_mass)

 In [30]: sub_mass

 In [64]: sub_mass[:]=5

 In [30]: sub_mass # updated as expected

 In [39]: f['mass'][[20,21,22]] # NOT updated - should still be 3,3,3!
 Out[39]: SimArray([ 3.,  3.,  3.])


So it becomes your responsibility to copy the results back in this case, if required. A template for performance
critical code which might be operating on a ``SubSnap`` follows.




.. note::

 ``with f_sub.immediate_mode``
 is equivalent to ``with f.immediate_mode`` where ``f_sub`` is any ``SubSnap`` of ``f``.

So in summary, the template code at the start of this document advocates:

 - storing a *copy* of the data for the subset of particles
 - working on the copy
 - (if necessary) updating the main snapshot data explicitly before returning


Overheads of raw SimArrays
--------------------------

.. note::

 This information is provided for interest. We have never come across a realistic use case
 where the following is necessary.

In ``pynbody``, arrays are implemented by the class :class:`~pynbody.array.SimArray`. This is a wrapper
around a ``numpy`` array. There is a small extra cost associated with every operation to allow
units to be matched and updated. For long arrays such as those found in typical simulations, this is usually a tiny fraction of the
actual computation time. We have never found it to be a problem, but if you want to disable the
unit tracking you can always do so using ``numpy``'s view mechanism to get a raw ``numpy`` array.
Suppose you have a ``SimSnap`` ``f``; then ``pos = f['pos'].view(numpy.ndarray)`` (for example) will return the position
array without any of the ``SimArray`` wrapping. The new ``pos`` variable can be manipulated without
any unit handling code being called.
