KMC3 Data Processing HOWTO
==========================

This demonstrates how to use the [nx5d](https://gitlab.com/codedump2/nx5d) Python package to load, convert and anlyse X-ray diffraction (XRD) data generated at the BESSY-II KMC3-XPP beamline.

You're likely to be accessing this Jupyer Lab account right now on the [KMC3 beamline control computer](http://kmc3-xpp.basisit.de:8888). You should have received the password from the endstation administration team (Matthias or Florin, as of writing this), or perhaps you're using an already authenticated session. That's alright.

The Jupyter Lab URL ([http://kmc3-xpp.basisit.de:8888](http://kmc3-xpp.basisit.de:8888)) is accessible only inside the BESSY-II network. You can read it from the outside via an [SSH tunnel through Aditum](https://www.helmholtz-berlin.de/bin/extern-access?sprache=en).

Get acquainted with your environment
------------------------------------


Your current working directory is either in the "home directory" of the default user `jovyan`, or in a directory that you've selected in the file explorer on the left-hand side. You can use your favorite IPython command to figure that out:


```python
pwd
```




    '/home/jovyan/code/skycontrol/doc/HOWTOs/KMC3 Data Processing'




```python
ls
```

    [0m[01;32m'KMC3 Data Processing HOWTO.ipynb'[0m*  'KMC3 Data Processing HOWTO.pdf'



Data is available at `~/jovyan/work/...`, where `...` depends on your project.
Typically, there's a year involved (e.g. `.../2023/...`) and a cryptic string containing your proposal number and a shorthand version of your name. For instance:


```python
ls /home/jovyan/work/2023/231-cw7-12082-roessle/
```

    [0m[01;32m231-cw7-12083-roessle.h5[0m*    [01;32mAnalysis.ipynb[0m*             [01;34mpilatus[0m/
    [01;32m231-cw7-12083-roessle.spec[0m*  [01;32mDetectorCalibration.ipynb[0m*


Working with data
-----------------

For the purpose of this example we'll work with the dataset 44 in a file called "231-cw7-12083-roessle.spec" from our 2023 measurements. The `dataPath` tuple below is a combination of the on-disk file path, and of the scan path inside the (HDF5-like) file itself. For convenience, we combine the two in a URL-like scheme `scanUrl` which we'll be using henceforth.


```python
dataPath = ("/home/jovyan/work/2023/231-cw7-12082-roessle/231-cw7-12083-roessle.spec", "44.1")
scanUrl = '#'.join(dataPath)
scanUrl
```




    '/home/jovyan/work/2023/231-cw7-12082-roessle/231-cw7-12083-roessle.spec#44.1'



### Loading and inspecting the "raw" data file

Having located the data on disk, the next step is importing the proper Python libraries for
actually loading that data (you'll need at least `nx5d`) and trying to load the SPEC file
that was generated by the experiment orchestration software.

There are several parts to `nx5d`. The most basic, low-level part, is access to the datafile
itself. It doesn't know anything about data organisation (i.e. "scans" etc), so we'll only
be using the file path part of `dataPath`, i.e. `dataPath[0]`.


```python
import nx5d.xrd.kmc3 as kmc3
h5like = kmc3.SpecTiffH5(dataPath[0])
```

The data object (here called `h5like`) behaves like a
[`h5py.File` object](https://docs.h5py.org/en/stable/high/file.html), which
is probably familiar to you from loading HDF5 files.

At the KMC3-XPP beamline we try to stay compatible with the [ESRF-flavor of the
HDF5/Nexus format](https://bliss.gitlab-pages.esrf.fr/bliss/master/data/data_nexus.html)
for data storage. This means that your data is located somewhere
under a scan name (here "44.1") and a detector name (here "pilatus")
inside the HDF5-like folder named "measurement".


```python
data = h5like["/44.1/measurement/pilatus"]
data.shape
```




    (41, 195, 487)



Basic plotting of raw data works as you've grown to expect from a Jupyter
environment.


```python
from matplotlib import pyplot as plt
import matplotlib.colors as mcol

%matplotlib inline

plt.figure(10)
plt.pcolormesh(data[20]+1, norm=mcol.LogNorm())
plt.show()
```


    
![png](output_15_0.png)
    



    
![png](output_15_1.png)
    



    
![png](output_15_2.png)
    


### XRD interpretation of HDF5-like files

Do do real scientific work, we'll need that to interpret data according to
its scientific context. For XRD, this means that we will use the positioner
(i.e. angles) information we have to transform detector images into "Q-space maps",
a.k.a. "detector intensity images with Q-coordinates as their axes."

This is implemented mostly in the `nx5d.xrd.data` module. The central tool within that
module is the `ScanReader` object, which uses a data URL (`scanUrl` from above) and
an instrument specification.


```python
from nx5d.xrd.data import ScanReader, SetupGenerator
```

#### The experiment specification

The first necessary step towards telling the computer how our experiment looks like.

The experiment specification is just a Python dictionary of which describes our goniometer,
sample and coordinate systems. We're using `xrayutilities` under the hood, so while the
information within an experiment specification dictionary tries to stay fairly general,
it is none the less strongly biased towards being easily fed into an
[HXRD experiment](https://xrayutilities.sourceforge.io/xrayutilities.html#xrayutilities.experiment.HXRD)
class of `xrayutilities.` This is an example of such an experimental setup definition:
```
{
    "goniometerAxes": ('y+', 'z+', 'x+'),
    "detectorTARAxes": (None, "z+", None),
    "imageAxes": ("x+", "z+"),
    
    "imageSize": (1024, 768),
    "imageCenter": (0.5, 0.5),
    "imageChannelSize": (1e-4, 1e-4),
    
    "sampleFaceUp": 'x+',
    "beamDirection": (0, 1, 0),
    "sampleNormal": (0, 0, 1),
    "beamEnergy": 9400.0,
    
    
    "detectorTARAngles": (0, 0, 0),
    "goniometerAngles": ("Theta", "Phi", "Chi"),  
}          
```

For our KMC3 beamline, the experimental setup is dynamically generated on a scan
by scan basis using the `nx5d.xrd.data.SetupGenerartor`.


#### Bonus: generating experiment setup from templates

*NOTE: This deals with the intricacies with templating the experimental setup
definitons and generating them on a scan-by-scan basis, as needed,
to provide information that might change from one scan to anoher. For simplicity
you can safely skip this section and just use the code below.*

If you're going to convert data from an unknown experiment, ultimately you need
to provide all these fields. Very few of the fields are allowed to be undefined
(i.e. `None`) -- in fact, the only such fields are some of the `detectorTARAxes`.
However, some information is not general or "static", but rather changeable on
a scan-by-scan basis (e.g. the `beamEnergy`). For this reason we prefer to work
not with *finished* experiment setup definitions, but with *templates*. These
have the same fields, but instead of directly usable information, they may
contain HDF5 paths of where to get that information on a scan-by-scan basis.
One such template would differ from the example above e.g. like this:
```
{
    "imageSize": ( "@{instrument}/rayonix/y_pixel_size",
                   "@{instrument}/rayonix/x_pixel_size" )
    ...
    "beamEnergy": "@{positioners}/beam_energy"
    ...
}
```

In this template example, for instance, we're requiring that the actual image
size in pixels after loading the data, from the specified variables.
It takes a specific HDF5-like container, containing specific scan data,
to actually generate a usable setup dictionary given a template like the above.
This is done by other `nx5d` components. One such component is, for instance,
the `SetupGenerator` class.

Currently `nx5d` provides experimetnal setup templates for some beamlines,
e.g. for the KMC3-XPP beamline (see `nx5d.xrd.kmc3.ExperimentTemplate`).


```python
# the template
kmc3.ExperimentTemplate
```




    {'goniometerAxes': ('x+', 'y+', 'z+'),
     'detectorTARAxes': ('x+', None, None),
     'imageAxes': ('x-', 'z-'),
     'imageSize': (195, 487),
     'imageCenter': (90, 245),
     'imageChannelSpan': None,
     'imageDistance': '@{positioners}/PilatusYOffset',
     'imageChannelSize': (0.172, 0.172),
     'sampleFaceUp': 'z+',
     'beamDirection': (0, 1, 0),
     'sampleNormal': (0, 0, 1),
     'beamEnergy': '@{positioners}/monoE',
     'goniometerAngles': {'theta': '@{positioners}/Theta',
      'chi': '@{positioners}/Chi',
      'phi': '@{positioners}/Phi'},
     'detectorTARAngles': {'azimuth': '@{positioners}/TwoTheta'}}




```python
# compiling the template into an experiment setup definition based on the current h5like
exampleKmc3SetupGen = SetupGenerator(kmc3.ExperimentTemplate,
                                     paths={ 'positioners': 'instrument/positioners',
                                             'instrument':  'instrument',
                                             'measurement': 'measurement' }
                                    )
exampleKmc3SetupGen(h5like['44.1'])
```




    {'goniometerAxes': ('x+', 'y+', 'z+'),
     'detectorTARAxes': ('x+', None, None),
     'imageAxes': ('x-', 'z-'),
     'imageSize': (195, 487),
     'imageCenter': (90, 245),
     'imageChannelSpan': None,
     'imageChannelSize': (0.172, 0.172),
     'sampleFaceUp': 'z+',
     'beamDirection': (0, 1, 0),
     'sampleNormal': (0, 0, 1),
     'beamEnergy': 9499.994}



### Reading the scan data

#### Defining experiment setup and initializing the `ScanReader`

Since some of our experimental parameters are inside the Spec file, will
only be available upon reading, and will change on a scan by scan basis,
we're using `SetupGenerator` to generate a setup
from the KMC3 template `nx5d.xrd.kmc3.ExperimentTemplate`. However,
sometimes (as in this case) we also want to override some experimental
parameters from the start, simply because they're either not available
in the Spec file at all, or because we want to supply our own values.

Here, we need to overide `"imageSize"` and `"imageChannelSize"` (a.k.a.
the pixel size).

We generate a template with customized entries by copying the KMC3 experiment
setup definition and changing the dictionary as needed.


```python
partialTemplate = kmc3.ExperimentTemplate.copy()
partialTemplate.update({'imageSize': (195, 487),
                        'imageCenter': (90, 220)})
```

Now we have everything we need to define the scan reader object:

  - A scan file name / scan path (`scanUrl`)
  
  - A HDF5-like file reader (`nx5d.xrd.kmc3.SpecTiffH5`)
  
  - A usable experiment setup template containing valid values for everything
    except for the keys which will be filled in when the scan reader object
    will actually... *read* the scan (`partialTemplate`)

The next step is to define the scan reader object.


```python
reader = ScanReader(kmc3.SpecTiffH5, scanUrl, setup=partialTemplate)
```

#### Direct reading of data frames and positioners

The central method of the `ScanReader` object is its `read()` method.
Most of the time you will be not using it directly (you will instead
use a higher-abstraction concepts like `ScanReader.Chunk`, introduced
in the next subsection.) However, it is important to understand how
`ScanReader.read()` works and what are the concepts behind it.

In its most simple form, `read()` expects two arguments:

  - a frame indexing object (`frameSlicer`) as its first parameter unnamed argument
  
  - any number of measurement data key(s) and path(s) as named arguments.

The "frame slicer" is either a number, indicating the index of the data frame (i.e. detector image) we want to retrieve; the data keys/paths the define which datasets of the HDF5 file *actually* make up the data we're interested in.

The `read()` method returns a dictionary which contains the keys we requested, holding the data we're interested in.

In the example below, the key `"image"` of the return data is the same as the
parameter `image=...` of the function call, and the numpy array that is returned
is frame 31 (of 41 in total) that is located at the path `"measurement/pilatus"` within
the HDF5 scan portion.

Of the two path compontents (`"measurement"` and `"pilatus"`) the latter is the
one we specified in the method call: `read(... image="pilatus")`. The former is
the default measurement data path. In accordance with ESRF-style HDF5/Nexus data
storage this is typically the `"measurement"` HDF5 group, but can be changed
when initializing the `ScanReader` object (refer to the documentation for details).

We can access data from all over the HDF5 scan group, for instance angles
from the `"positioner"` group. They will show up in the resulting data dictionary.


```python
scanData = reader.read(31, image="pilatus", twotheta="@{positioners}/TwoTheta", chi="@{positioners}/Chi")
scanData
```




    {'image': array([[0, 0, 0, ..., 0, 0, 0],
            [0, 0, 0, ..., 0, 0, 0],
            [0, 0, 0, ..., 0, 0, 0],
            ...,
            [0, 0, 0, ..., 0, 0, 0],
            [0, 0, 0, ..., 0, 0, 0],
            [0, 0, 0, ..., 0, 0, 0]], dtype=int32),
     'twotheta': 113.94,
     'chi': 0.3}



Note that the data key "chi" in this case is a scalar -- as is the corresponding
dataset in the HDF5-like file (`"instrument/positioners/Chi"`). As such, it
actually wouldn't be indexable by the interger index `31` that we specified
(you can try `h5like["/44.1/instrument/positionres/Chi"][31]` to convince
yourself).

However, donvenient behavior for `ScanData.read()` in cases like these is to
simply reproduce the scalar as it is, disregarding the indexing parameter
passed to `read()`. Keep in mind that this is only done for scalar datasets.
Any other datasets, e.g. with a non-matching shape or length (in our case:
a non-matching datacase, for instance, would be an array with 30 elements or
less) would raise an error. The intention is that all datasets available through
one `ScanReader` interface, if they are arrays, have the same length in their
first dimension: one record per frame.

After successfully retrieving a `scanData` dictionary, we are -- again -- 
able to plot a detector image frame. Only this time we obtained the frame using
the more abstract `ScanReader.read()` interface.


```python
plt.figure(30)
plt.pcolormesh(scanData['image']+1, norm=mcol.LogNorm())
plt.show()
```


    
![png](output_33_0.png)
    


### Streak-based data processing

We have already learned that typical *scan* of experimental data consists of multiple
*frames* (i. e. detector images) -- each for a different set of positions, denoted by
an individual set of positioner variables.
However, *frame* => *scan* is not necessarily the only structure the data has; in many
cases there are other, non-positioning paramteres, which denote a subset of *frames*,
and several such subsets make finally make up a *scan*. We call these subsets *streaks*.

One *streak* differs from another typically by the fact that specific, non-positional
experimental parameters were change, for instance:

  - the time delay between an excitation and measurement (e.g. in pump-probe experiments)  
  - sample temperature (e.g. in measurements that involve temperature series)
  - aging of the sample, ...

#### Scan vs streak -- layout of frames inside a dataset

Streaking is an essential feature of experimental data, which needs to be taken
into account when reading and processing data.

In a series of X-ray diffraction images, an image scan is therefore virtually a series of
streaks (*seies* = [streak0, streak1, streak2, ... streakN]), with each streak being equally 
large and itself a series of frames (*chunk0* = [frame0, frame1, ... frameX],
*chunk1* = [frameX+1, frameX+2, ... frame2X], ...).

So in essence, when chunks are stored one after the other, the the typical scan data series
is a 3D dataset which typically looks like this:

  - [image1, image2, ... imageX, imageX+1, imageX+2, ...image2X, image2X+1, ...]
  
...where each *image* is of size WxH.
There is, however, also the possiblity that chunks are interweaved, making a *series*
look like this:

  - [image1, imageX+1, image2X+1, ...image2, imageX+2, image2X+2, ... imageN, imageX+N, ...]
  
In the latter case, one chunk is made up of every X-th image, while in the former, each
chunk is made up of equally large segments of X consecutive images.

#### Reading scans, one streak at a time

The method `streaks()` of `ScanReader` generates a series of *streaks* corresponding to
either the first type of layout (`streaks(length=...)`), or to the second type
(`streaks(delta=...)`). It also needs a list of HDF5 or measurement keys to actually read,
so the most simple call, e.g. for reading 4 "pieces" of 10 frames each, looks like the
following.



```python
reader.streaks(length=10, image="pilatus")
```




    <generator object ScanReader.streaks at 0x7f523033b1b0>



However, in our current dataset, there is no physically meaningful parameter other that the
positioners. We will therefore all the data at once, in one giant streak, corresponding
to the length of our scan. This is done by supplying neither `length` nor `delta` as parameter.


```python
chks = [c for c in reader.streaks(image="pilatus")]
chk  = chks[0]
```

The resulting object is a dictionary containing the requested data (image) aswell as 
the data that we have passed as a parameter to the initialization of the `ScanReader`
object, the experimental setup.


```python
chk
```




    {'setup': {'goniometerAxes': ('x+', 'y+', 'z+'),
      'detectorTARAxes': ('x+', None, None),
      'imageAxes': ('x-', 'z-'),
      'imageSize': (195, 487),
      'imageCenter': (90, 220),
      'imageChannelSpan': None,
      'imageDistance': 155.0,
      'imageChannelSize': (0.172, 0.172),
      'sampleFaceUp': 'z+',
      'beamDirection': (0, 1, 0),
      'sampleNormal': (0, 0, 1),
      'beamEnergy': 9499.994,
      'goniometerAngles': {'theta': array([26.351093, 26.371094, 26.391094, 26.411095, 26.431093, 26.451094,
              26.471094, 26.491095, 26.511093, 26.531094, 26.551094, 26.571095,
              26.591093, 26.611094, 26.631094, 26.651094, 26.671095, 26.691093,
              26.711094, 26.731094, 26.751095, 26.771093, 26.791094, 26.811094,
              26.831095, 26.851093, 26.871094, 26.891094, 26.911095, 26.931093,
              26.951094, 26.971094, 26.991095, 27.011093, 27.031094, 27.051094,
              27.071095, 27.091093, 27.111094, 27.131094, 27.151094],
             dtype=float32),
       'chi': 0.3,
       'phi': 0.0},
      'detectorTARAngles': {'azimuth': array([112.7 , 112.74, 112.78, 112.82, 112.86, 112.9 , 112.94, 112.98,
              113.02, 113.06, 113.1 , 113.14, 113.18, 113.22, 113.26, 113.3 ,
              113.34, 113.38, 113.42, 113.46, 113.5 , 113.54, 113.58, 113.62,
              113.66, 113.7 , 113.74, 113.78, 113.82, 113.86, 113.9 , 113.94,
              113.98, 114.02, 114.06, 114.1 , 114.14, 114.18, 114.22, 114.26,
              114.3 ], dtype=float32)}},
     'image': array([[[0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]],
     
            [[0, 0, 0, ..., 0, 0, 1],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]],
     
            [[0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]],
     
            ...,
     
            [[0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]],
     
            [[0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]],
     
            [[0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             ...,
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0],
             [0, 0, 0, ..., 0, 0, 0]]], dtype=int32)}




```python
chk['image'].shape
```




    (41, 195, 487)



This is how we access one  particular image of our data.


```python
chk['image'][31]
```




    array([[0, 0, 0, ..., 0, 0, 0],
           [0, 0, 0, ..., 0, 0, 0],
           [0, 0, 0, ..., 0, 0, 0],
           ...,
           [0, 0, 0, ..., 0, 0, 0],
           [0, 0, 0, ..., 0, 0, 0],
           [0, 0, 0, ..., 0, 0, 0]], dtype=int32)



#### Angular-to-Q conversion

Further processing of data is performed by different modules within `nx5d`,
for instance `nx5d.xrd.signal`. There we have the `LazyQMap` class, which
takes a list of data sets and one experimental setup (as they are
contained within `chk`) and offer a angular-to-Q conversion based on
`xrayutilities`.



```python
from nx5d.xrd.signal import LazyQMap
```


```python
qtrans = LazyQMap(**chk)
qimg = qtrans("image")
```

This is all it takes to do the angular-to-Q conversion of our data.

The result dataset is an `xarray.DataArray` data set, which mostly can be used
like a `numpy` array, but has the benefit of holding intrinsical scaling
information. In this case, it contains the Q coordinate axes along the width
and the height of the image (sometimes also called $q_x$ and $q_z$, or $q_{||}$
and $q_{\perp}$ -- which is not always a correct description, which is why
we stick to "qw" and "qh").

(Note: if you are not already familiar with the
[xarray.DataArray](https://docs.xarray.dev/en/stable/user-guide/data-structures.html)
and its API, now is a good time to get acquainted.)


```python
qimg
```




<div><svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>
<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.
 *
 */

:root {
  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));
  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));
  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));
  --xr-border-color: var(--jp-border-color2, #e0e0e0);
  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);
  --xr-background-color: var(--jp-layout-color0, white);
  --xr-background-color-row-even: var(--jp-layout-color1, white);
  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);
}

html[theme=dark],
body[data-theme=dark],
body.vscode-dark {
  --xr-font-color0: rgba(255, 255, 255, 1);
  --xr-font-color2: rgba(255, 255, 255, 0.54);
  --xr-font-color3: rgba(255, 255, 255, 0.38);
  --xr-border-color: #1F1F1F;
  --xr-disabled-color: #515151;
  --xr-background-color: #111111;
  --xr-background-color-row-even: #111111;
  --xr-background-color-row-odd: #313131;
}

.xr-wrap {
  display: block !important;
  min-width: 300px;
  max-width: 700px;
}

.xr-text-repr-fallback {
  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */
  display: none;
}

.xr-header {
  padding-top: 6px;
  padding-bottom: 6px;
  margin-bottom: 4px;
  border-bottom: solid 1px var(--xr-border-color);
}

.xr-header > div,
.xr-header > ul {
  display: inline;
  margin-top: 0;
  margin-bottom: 0;
}

.xr-obj-type,
.xr-array-name {
  margin-left: 2px;
  margin-right: 10px;
}

.xr-obj-type {
  color: var(--xr-font-color2);
}

.xr-sections {
  padding-left: 0 !important;
  display: grid;
  grid-template-columns: 150px auto auto 1fr 20px 20px;
}

.xr-section-item {
  display: contents;
}

.xr-section-item input {
  display: none;
}

.xr-section-item input + label {
  color: var(--xr-disabled-color);
}

.xr-section-item input:enabled + label {
  cursor: pointer;
  color: var(--xr-font-color2);
}

.xr-section-item input:enabled + label:hover {
  color: var(--xr-font-color0);
}

.xr-section-summary {
  grid-column: 1;
  color: var(--xr-font-color2);
  font-weight: 500;
}

.xr-section-summary > span {
  display: inline-block;
  padding-left: 0.5em;
}

.xr-section-summary-in:disabled + label {
  color: var(--xr-font-color2);
}

.xr-section-summary-in + label:before {
  display: inline-block;
  content: '►';
  font-size: 11px;
  width: 15px;
  text-align: center;
}

.xr-section-summary-in:disabled + label:before {
  color: var(--xr-disabled-color);
}

.xr-section-summary-in:checked + label:before {
  content: '▼';
}

.xr-section-summary-in:checked + label > span {
  display: none;
}

.xr-section-summary,
.xr-section-inline-details {
  padding-top: 4px;
  padding-bottom: 4px;
}

.xr-section-inline-details {
  grid-column: 2 / -1;
}

.xr-section-details {
  display: none;
  grid-column: 1 / -1;
  margin-bottom: 5px;
}

.xr-section-summary-in:checked ~ .xr-section-details {
  display: contents;
}

.xr-array-wrap {
  grid-column: 1 / -1;
  display: grid;
  grid-template-columns: 20px auto;
}

.xr-array-wrap > label {
  grid-column: 1;
  vertical-align: top;
}

.xr-preview {
  color: var(--xr-font-color3);
}

.xr-array-preview,
.xr-array-data {
  padding: 0 5px !important;
  grid-column: 2;
}

.xr-array-data,
.xr-array-in:checked ~ .xr-array-preview {
  display: none;
}

.xr-array-in:checked ~ .xr-array-data,
.xr-array-preview {
  display: inline-block;
}

.xr-dim-list {
  display: inline-block !important;
  list-style: none;
  padding: 0 !important;
  margin: 0;
}

.xr-dim-list li {
  display: inline-block;
  padding: 0;
  margin: 0;
}

.xr-dim-list:before {
  content: '(';
}

.xr-dim-list:after {
  content: ')';
}

.xr-dim-list li:not(:last-child):after {
  content: ',';
  padding-right: 5px;
}

.xr-has-index {
  font-weight: bold;
}

.xr-var-list,
.xr-var-item {
  display: contents;
}

.xr-var-item > div,
.xr-var-item label,
.xr-var-item > .xr-var-name span {
  background-color: var(--xr-background-color-row-even);
  margin-bottom: 0;
}

.xr-var-item > .xr-var-name:hover span {
  padding-right: 5px;
}

.xr-var-list > li:nth-child(odd) > div,
.xr-var-list > li:nth-child(odd) > label,
.xr-var-list > li:nth-child(odd) > .xr-var-name span {
  background-color: var(--xr-background-color-row-odd);
}

.xr-var-name {
  grid-column: 1;
}

.xr-var-dims {
  grid-column: 2;
}

.xr-var-dtype {
  grid-column: 3;
  text-align: right;
  color: var(--xr-font-color2);
}

.xr-var-preview {
  grid-column: 4;
}

.xr-index-preview {
  grid-column: 2 / 5;
  color: var(--xr-font-color2);
}

.xr-var-name,
.xr-var-dims,
.xr-var-dtype,
.xr-preview,
.xr-attrs dt {
  white-space: nowrap;
  overflow: hidden;
  text-overflow: ellipsis;
  padding-right: 10px;
}

.xr-var-name:hover,
.xr-var-dims:hover,
.xr-var-dtype:hover,
.xr-attrs dt:hover {
  overflow: visible;
  width: auto;
  z-index: 1;
}

.xr-var-attrs,
.xr-var-data,
.xr-index-data {
  display: none;
  background-color: var(--xr-background-color) !important;
  padding-bottom: 5px !important;
}

.xr-var-attrs-in:checked ~ .xr-var-attrs,
.xr-var-data-in:checked ~ .xr-var-data,
.xr-index-data-in:checked ~ .xr-index-data {
  display: block;
}

.xr-var-data > table {
  float: right;
}

.xr-var-name span,
.xr-var-data,
.xr-index-name div,
.xr-index-data,
.xr-attrs {
  padding-left: 25px !important;
}

.xr-attrs,
.xr-var-attrs,
.xr-var-data,
.xr-index-data {
  grid-column: 1 / -1;
}

dl.xr-attrs {
  padding: 0;
  margin: 0;
  display: grid;
  grid-template-columns: 125px auto;
}

.xr-attrs dt,
.xr-attrs dd {
  padding: 0;
  margin: 0;
  float: left;
  padding-right: 10px;
  width: auto;
}

.xr-attrs dt {
  font-weight: normal;
  grid-column: 1;
}

.xr-attrs dt:hover span {
  display: inline-block;
  background: var(--xr-background-color);
  padding-right: 10px;
}

.xr-attrs dd {
  grid-column: 2;
  white-space: pre-wrap;
  word-break: break-all;
}

.xr-icon-database,
.xr-icon-file-text2,
.xr-no-icon {
  display: inline-block;
  vertical-align: middle;
  width: 1em;
  height: 1.5em !important;
  stroke-width: 0;
  stroke: currentColor;
  fill: currentColor;
}
</style><pre class='xr-text-repr-fallback'>&lt;xarray.DataArray (qx: 41, qy: 195, qz: 487)&gt;
array([[[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
...
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]]])
Coordinates:
  * qx       (qx) float64 -0.5885 -0.5627 -0.5369 ... 0.3907 0.4164 0.4422
  * qy       (qy) float64 -5.192 -5.179 -5.165 -5.152 ... -2.686 -2.673 -2.66
  * qz       (qz) float64 6.628 6.629 6.629 6.63 6.631 ... 7.009 7.01 7.01 7.011</pre><div class='xr-wrap' style='display:none'><div class='xr-header'><div class='xr-obj-type'>xarray.DataArray</div><div class='xr-array-name'></div><ul class='xr-dim-list'><li><span class='xr-has-index'>qx</span>: 41</li><li><span class='xr-has-index'>qy</span>: 195</li><li><span class='xr-has-index'>qz</span>: 487</li></ul></div><ul class='xr-sections'><li class='xr-section-item'><div class='xr-array-wrap'><input id='section-3ee14c45-1dc9-4df7-b055-24798d10c7eb' class='xr-array-in' type='checkbox' checked><label for='section-3ee14c45-1dc9-4df7-b055-24798d10c7eb' title='Show/hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-array-preview xr-preview'><span>0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0</span></div><div class='xr-array-data'><pre>array([[[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
...
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],

       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]]])</pre></div></div></li><li class='xr-section-item'><input id='section-495e2d0b-9df0-48f3-b168-fa828f1ca14f' class='xr-section-summary-in' type='checkbox'  checked><label for='section-495e2d0b-9df0-48f3-b168-fa828f1ca14f' class='xr-section-summary' >Coordinates: <span>(3)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>qx</span></div><div class='xr-var-dims'>(qx)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>-0.5885 -0.5627 ... 0.4164 0.4422</div><input id='attrs-7b239896-fa80-4e67-b9f1-b1bcecf2830a' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-7b239896-fa80-4e67-b9f1-b1bcecf2830a' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-8f7344ff-5be8-45cf-8958-01f33ba4ebb5' class='xr-var-data-in' type='checkbox'><label for='data-8f7344ff-5be8-45cf-8958-01f33ba4ebb5' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([-0.58845 , -0.562684, -0.536917, -0.511151, -0.485384, -0.459618,
       -0.433851, -0.408085, -0.382319, -0.356552, -0.330786, -0.305019,
       -0.279253, -0.253486, -0.22772 , -0.201953, -0.176187, -0.15042 ,
       -0.124654, -0.098887, -0.073121, -0.047355, -0.021588,  0.004178,
        0.029945,  0.055711,  0.081478,  0.107244,  0.133011,  0.158777,
        0.184544,  0.21031 ,  0.236076,  0.261843,  0.287609,  0.313376,
        0.339142,  0.364909,  0.390675,  0.416442,  0.442208])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>qy</span></div><div class='xr-var-dims'>(qy)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>-5.192 -5.179 ... -2.673 -2.66</div><input id='attrs-63705b47-6878-44b8-b5ac-f00aa983b6cd' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-63705b47-6878-44b8-b5ac-f00aa983b6cd' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-4be555a6-de44-40b7-b09c-74f2443740bb' class='xr-var-data-in' type='checkbox'><label for='data-4be555a6-de44-40b7-b09c-74f2443740bb' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([-5.191571, -5.178521, -5.165471, -5.152421, -5.139371, -5.126321,
       -5.113271, -5.100221, -5.087171, -5.074121, -5.061071, -5.048021,
       -5.034971, -5.021921, -5.008871, -4.995821, -4.982771, -4.969721,
       -4.956672, -4.943622, -4.930572, -4.917522, -4.904472, -4.891422,
       -4.878372, -4.865322, -4.852272, -4.839222, -4.826172, -4.813122,
       -4.800072, -4.787022, -4.773972, -4.760922, -4.747872, -4.734822,
       -4.721773, -4.708723, -4.695673, -4.682623, -4.669573, -4.656523,
       -4.643473, -4.630423, -4.617373, -4.604323, -4.591273, -4.578223,
       -4.565173, -4.552123, -4.539073, -4.526023, -4.512973, -4.499923,
       -4.486874, -4.473824, -4.460774, -4.447724, -4.434674, -4.421624,
       -4.408574, -4.395524, -4.382474, -4.369424, -4.356374, -4.343324,
       -4.330274, -4.317224, -4.304174, -4.291124, -4.278074, -4.265024,
       -4.251975, -4.238925, -4.225875, -4.212825, -4.199775, -4.186725,
       -4.173675, -4.160625, -4.147575, -4.134525, -4.121475, -4.108425,
       -4.095375, -4.082325, -4.069275, -4.056225, -4.043175, -4.030125,
       -4.017076, -4.004026, -3.990976, -3.977926, -3.964876, -3.951826,
       -3.938776, -3.925726, -3.912676, -3.899626, -3.886576, -3.873526,
       -3.860476, -3.847426, -3.834376, -3.821326, -3.808276, -3.795227,
       -3.782177, -3.769127, -3.756077, -3.743027, -3.729977, -3.716927,
       -3.703877, -3.690827, -3.677777, -3.664727, -3.651677, -3.638627,
       -3.625577, -3.612527, -3.599477, -3.586427, -3.573377, -3.560328,
       -3.547278, -3.534228, -3.521178, -3.508128, -3.495078, -3.482028,
       -3.468978, -3.455928, -3.442878, -3.429828, -3.416778, -3.403728,
       -3.390678, -3.377628, -3.364578, -3.351528, -3.338478, -3.325429,
       -3.312379, -3.299329, -3.286279, -3.273229, -3.260179, -3.247129,
       -3.234079, -3.221029, -3.207979, -3.194929, -3.181879, -3.168829,
       -3.155779, -3.142729, -3.129679, -3.116629, -3.103579, -3.09053 ,
       -3.07748 , -3.06443 , -3.05138 , -3.03833 , -3.02528 , -3.01223 ,
       -2.99918 , -2.98613 , -2.97308 , -2.96003 , -2.94698 , -2.93393 ,
       -2.92088 , -2.90783 , -2.89478 , -2.88173 , -2.86868 , -2.855631,
       -2.842581, -2.829531, -2.816481, -2.803431, -2.790381, -2.777331,
       -2.764281, -2.751231, -2.738181, -2.725131, -2.712081, -2.699031,
       -2.685981, -2.672931, -2.659881])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>qz</span></div><div class='xr-var-dims'>(qz)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>6.628 6.629 6.629 ... 7.01 7.011</div><input id='attrs-5b98b4f8-21f1-4bba-ae85-53c777eb37da' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-5b98b4f8-21f1-4bba-ae85-53c777eb37da' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-a0c4f708-36f8-4e4d-9a28-4a7fc9da4686' class='xr-var-data-in' type='checkbox'><label for='data-a0c4f708-36f8-4e4d-9a28-4a7fc9da4686' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([6.627862, 6.628651, 6.62944 , ..., 7.009703, 7.010492, 7.011281])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-17ec461d-00a0-4c4e-8de6-1f004abd18a3' class='xr-section-summary-in' type='checkbox'  ><label for='section-17ec461d-00a0-4c4e-8de6-1f004abd18a3' class='xr-section-summary' >Indexes: <span>(3)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-index-name'><div>qx</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-3eca1493-fd39-4e5c-a7c9-83e6fcb0baee' class='xr-index-data-in' type='checkbox'/><label for='index-3eca1493-fd39-4e5c-a7c9-83e6fcb0baee' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Float64Index([   -0.588450190010869,   -0.5626837320892625,
                -0.5369172741676559,   -0.5111508162460494,
                -0.4853843583244428,  -0.45961790040283634,
                -0.4338514424812298,   -0.4080849845596233,
               -0.38231852663801674,   -0.3565520687164102,
                -0.3307856107948037,  -0.30501915287319714,
               -0.27925269495159066,  -0.25348623702998413,
                -0.2277197791083776,  -0.20195332118677106,
               -0.17618686326516453,    -0.150420405343558,
               -0.12465394742195146,  -0.09888748950034493,
                -0.0731210315787384, -0.047354573657131915,
              -0.021588115735525326,  0.004178342186081152,
                0.02994480010768763,   0.05571125802929422,
                 0.0814777159509007,   0.10724417387250729,
                0.13301063179411377,   0.15877708971572035,
                0.18454354763732683,   0.21031000555893342,
                 0.2360764634805399,    0.2618429214021464,
                0.28760937932375297,   0.31337583724535945,
                0.33914229516696603,    0.3649087530885725,
                 0.3906752110101791,    0.4164416689317856,
                0.44220812685339217],
             dtype=&#x27;float64&#x27;, name=&#x27;qx&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>qy</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-a7a60275-de26-4ed1-a0b2-726374998959' class='xr-index-data-in' type='checkbox'/><label for='index-a7a60275-de26-4ed1-a0b2-726374998959' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Float64Index([ -5.191570532976708,  -5.178520588775616, -5.1654706445745235,
              -5.1524207003734315,   -5.13937075617234,  -5.126320811971248,
               -5.113270867770156,  -5.100220923569064,  -5.087170979367972,
                -5.07412103516688,
              ...
                 -2.7773308557747, -2.7642809115736084, -2.7512309673725164,
              -2.7381810231714243, -2.7251310789703327, -2.7120811347692406,
              -2.6990311905681486, -2.6859812463670565,  -2.672931302165965,
               -2.659881357964873],
             dtype=&#x27;float64&#x27;, name=&#x27;qy&#x27;, length=195))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>qz</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-55d19b60-07fc-4efa-8d09-585f30014587' class='xr-index-data-in' type='checkbox'/><label for='index-55d19b60-07fc-4efa-8d09-585f30014587' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Float64Index([ 6.627862181916936,   6.62865110854383,  6.629440035170725,
                6.63022896179762,  6.631017888424514,  6.631806815051409,
               6.632595741678303,  6.633384668305198,  6.634173594932093,
               6.634962521558987,
              ...
              7.0041801829456265,  7.004969109572521,  7.005758036199415,
                7.00654696282631,  7.007335889453205,  7.008124816080099,
               7.008913742706993,  7.009702669333889,  7.010491595960783,
               7.011280522587677],
             dtype=&#x27;float64&#x27;, name=&#x27;qz&#x27;, length=487))</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-371a93d4-ff31-432e-ae52-4005aed46567' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-371a93d4-ff31-432e-ae52-4005aed46567' class='xr-section-summary'  title='Expand/collapse section'>Attributes: <span>(0)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'></dl></div></li></ul></div></div>



#### Leveraging `xarray`

With intrinsical scaling "all there", all other operations -- like slicing, interpolating,
integrating, and in particular plotting -- become a lot easier. Note how the following
plot already has the proper axis labeling in place.


```python
qimg.sum('qx').plot()
```




    <matplotlib.collections.QuadMesh at 0x7f521c698e20>




    
![png](output_52_1.png)
    

