# qutil / qutech_util
Long term goal is to gather utility functions here.
The original name was `qutil` but we included the additional alias `qutech_util` so you can install it via [pipy](https://pypi.org/project/qutech-util).
It is not meant as a lightweight package but some heavy dependencies like qcodes are feature gated.
If you don't have a reason for a lightweight install you should install all features i.e. `qutil[complete]`.
If you just want to use it you can install the latest "released" version via
```sh
python -m pip install qutech-util[complete]
```
However, this package profits from everybody's work and the releases are infrequent. Please make a development install
and contribute your changes. You can do this via
```sh
python -m pip install -e git+https://git.rwth-aachen.de/qutech/qutil.git#egg=qutech-util[complete]
```
This will download the source code (i.e. clone the git repository) into a subdirectory of the `./src` argument and link the files into your environment instead of copying them. If you are on windows you can use [SourceTree](https://www.sourcetreeapp.com/) which is a nice GUI for git.
You can specify the source code directory with the `--src` argument (which needs to be BEFORE `-e`):
```sh
python -m pip install --src some_directory/my_python_source -e git+https://git.rwth-aachen.de/qutech/qutil.git#egg=qutech-util[complete]
```
If you have already downloaded/cloned the package yourself you can use `python -m pip install .[complete] -e`.

Please file an issue if any of these instructions does not work.

### Tests
There is no plan for writing extensive tests for the code in this package but please try to write proper docstrings for
your functions and include examples in them which can be checked via [`doctest`](https://docs.python.org/3/library/doctest.html).
Follow the link for an example for an example :)

You can run the tests either via
```sh
python -m pytest --doctest-modules
```
or to check if everything works for a clean install (requires hatch to be installed)
```sh
python -m hatch run test:run
```

## Documentation
The auto-generated documentation can be found at [the Gitlab Pages](https://qutech.pages.rwth-aachen.de/qutil/index.html).

To build the documentation locally, navigate to `doc/` and run either

```bat
.\make.bat html
```

(on Windows),

```sh
make html
```

(on Unix), or

```bat
sphinx-build -b html source build
```

Make sure the dependencies are installed via

```sh
python -m pip install -e .[doc]
```

in the top-level directory.

## qutil.plotting
This module contains useful classes and functions surrounding `maptlotlib` plots.

`cycle_plots` helps you cycling through many plots with the arrow keys (there are probably much better functions for this out there)
`plot_2d_dataframe` helps you plot 2d data frames with numeric indices
`BlitManager` can be used to significantly speed up plotting when certain parts of the plot, such as the artists on the canvas (lines, annotations, etc.) change but others do not (axes, labels, etc.) does not.
`CoordClicker` and `LineClicker` allow some interactive selection of data.
`get_rwth_color_cycle` and the predefined `rwth_color_cycle` are cycler instances with the official RWTH corporate design colors:

![cycler example](./doc/source/_static/cycles.png)

## qutil.matlab
In this module there are functions that are helpful for reading `.mat` files, especially those created with special measure.
If you simply want to open a random `.mat` file you can use `hdf5storage.loadmat`.

Loading matlab files with "newer" MATLAB classes like `table` requires connecting (and starting) MATLAB instance.
The function `load_special_measure_with_matlab_engine` can load most special measure scans by utilizing the MATLAB engine interface. To use it you require a "sufficiently new" version of MATLAB and then navigate to `C:\Program Files\MATLAB\$VERSION\extern\engines\python` and call `python setup.py install`.

Recommended: There are dataclasses like `SimpleScan` or `VirtualScan` that are a python representation of certain common scan
types and have a convenience `to_xarray` method. Use `load_simple_scan_with_matlab_engine` or `load_virtual_scan_with_matlab_engine` to load them.

There are the dataclasses `FigureData`, `AxesData` and `PlotData` that represent matlab figure data. They help inspecting saved matlab figures with the help of a matlab engine.

## qutil.const
This module defines all the constants you could wish for as well as functions to convert temperatures (`convert_temperature`) or between wavelengths and frequencies (`lambda2nu`, `nu2lambda`). For an overview, see the module docstring.

## qutil.linalg
This module provides several handy linear algebra functions. While some are implemented elsewhere, the implementation here is typically speedier for large arrays. For example, `pauli_expm` exploits the fact that a matrix exponential of Pauli matrices can be written as a cosine times the identity matrix plus a sine times the Paulis to speed up the calculation.

For an overview of the included functions, see the module docstring.

## qutil.ui
This module collects UI helpers, such as a progress bar for loops that can be used like so:
```python
for i in qutil.ui.progressbar(range(n)):
    do_something()
```

## qutil.qi
In this module there are some quantities and functions related to quantum information, like the Pauli matrices in different data types.

## qutil.random
Here we collect functions for random numbers like `random_hermitian` to generate random Hermitian matrices.

## qutil.itertools
This module contains everything from `itertools`, `more_itertools` and custom functions.

## qutil.caching
Here you find decorators, functions and classes that help you implement caching like `file_cache` and `lru_cache`. This is helpful if you need to call computationally expensive functions with the same arguments repeatedly.

## qutil.io
User input related functions like `query_yes_no` or a `CsvLogger` interface (for reading use pandas.read_csv).

`to_global_path` resolves all network drive mappings (such as `Z:\`) as well as domain names 
(such as `\\janeway`) to their global address (`\\janeway.physik.rwth-aachen.de` in this case).

## qutil.parallel
Functions and classes related to parallel execution i.e. multi-threading, multi-processing and asyncio.
There is a class for periodic callbacks from another thread `ThreadedPeriodicCallback`.

## qutil.hardware
This package contains little scripts to talk to various hardware devices. For example reading the leak tester via serial interface.

## qutil.electronics
### lumped_elements
Exposes the contents of [fastz](https://github.com/whileman133/fastZ), a package for simple lumped-elements calculations. Overloads `+` and `//` to implement series and parallel connections, respectively.

See the `fastz` documentation for more information.

## qutil.qcodes
Functions to convert from and to qcodes data sets. Currently only
from `pandas.DataFrame` to `qcodes.data.data_set.DataSet`

## qutil.measurement
This package contains measurement-related functionality.
### spectrometer
Defines a `Spectrometer` class that handles data acquisition, processing, storage, and
visualization. Several processing steps can be applied to the raw timeseries data, for instance
to convert from a voltage signal to a current signal given a known impedance.

Some example code using the Keysight DMM qcodes driver for data acquisition:
```python
from qutil.measurement.spectrometer import Spectrometer, daq
from qcodes.instrument_drivers.Keysight.Keysight_34465A_submodules import Keysight_34465A
dmm = Keysight_34465A('dmm', 'some_tcpip_address')

# Pre-defined functions that set up and execute a measurement using a DMM
setup, acquire = daq.qcodes.keysight_344xxA(dmm)
spect = Spectrometer(setup, acquire, procfn=lambda V: V/100,  # 100 Ohms
                     raw_unit='V', processed_unit='A')
settings = {'f_min': 0.1, 'f_max': 1000}
spect.take('a comment', n_avg=5, **settings)
spect.hide(0)
spect.show('a comment')  # same as spect.show(0)
# Save and recall functionality
spect.serialize_to_disk('./foo')
spect_loaded = Spectrometer.recall_from_disk('./foo', setup, acquire)
spect_loaded.show_keys()
# (0, 'a comment')
```

You can also play around with simulated noise (requires `qopt`):
```python
from qutil.measurement.spectrometer import Spectrometer, daq
spect = Spectrometer(*daq.simulator.qopt_colored_noise(lambda f, A, **_: A/f))
spect.take('foobar', n_avg=10, n_seg=5, A=42)
```

## qutil.typecheck
Functions and decorators to  help with runtime typechecking. Notably the `@check_literals` decorator to ensure that arguments match an annotated literal.
Imports the [`typeguard`](https://typeguard.readthedocs.io/en/latest/userguide.html) which provides the powerful `@typechecked` decorator.
```python
from typing import Literal, Sequence
from qutil.typecheck import check_literals
@check_literals
def my_function(a: Sequence[int], b: Literal['forward', 'backward']):
    pass # do something

# works
my_function([1, 2, 3], 'backward')

# works because the first arguement is not checked at runtime
my_function({'no': 'sequence'}, 'backward')

# runtime error because of typo in 'backward'
my_function('wrong', 'backwardd')
```

## qutil.pandas_tools
Pandas utility functions for common code patterns. `consecutive_groupby` is like `pandas.DataFrame.groupby` but only
groups consecutive rows.

## qutil.image
Image and video processing tools. `convert_tiff` converts a multipage `.tif` image to a video with a format of choice using `moviepy`.
