User's Guide
============

By default, the user interface, shown in figure :numref:`fig:overview`, is structured into five main sections.
The tabs on the left contain all controls for :ref:`sec:confman`
including :ref:`sec:settings`, :ref:`sec:explorer`, :ref:`sec:devices`, :ref:`sec:scans`, and other controls.
During and after the measurement, results are displayed using various
:ref:`sec:displays` on the right. The most important signals of
any deposition experiment, including the real-time ion-currents, are always
visible in a dedicated :ref:`sec:live_displays` on the top.
The :ref:`sec:device_manager` is a central interface that allows to control all devices simultaneously.
Finally, there is a hidden :ref:`sec:console` for status messages
and advanced features that can be opened from :ref:`sec:settings`. 
Users are free to rearrange the user interface by dragging and tearing components.
All devices will be disconnected and turned off to leave them in a save
state when the software is closed. All numeric inputs can be
conveniently adjusted using the arrow keys. All plugins can be detached
and rearranged using the float icon or by dragging them inside or out of
the main window. All plugins provide basic documentation via the
question mark in their toolbar.

.. _`fig:overview`:
.. figure:: 2023-10_ESIBD_GUI.png

   **ESIBD Explorer user interface.** Colored boxes highlight the five main sections. :blue:`Live displays` showing
   data from several devices. Central :darkorange:`device manager`.
   :red:`Configuration and management` section including settings, file explorer, as well devices, scans and other
   controls. :orange:`Displays` of supported files
   including saved and running scans. :green:`Console` for
   debugging and development.

.. _sec:confman:

Configuration and management
----------------------------

This is the main control section used to define and access most plugin
settings, device configurations, and scans.

.. _`sec:settings`:

Settings
~~~~~~~~

The settings plugin allows to edit, save, and load all general program
and hardware settings. Settings can be edited either directly or using
the context menu that opens on right click. Settings are stored in an
.ini file which can be edited directly with any text editor if needed. The
settings file that is used on startup is automatically generated if it
does not exist. Likewise, default values are used for any missing
parameters. Setting files can be exported or imported from the user
interface. A change log will show which settings have changed after importing.
In addition, the plugin manager and console can be opened
from here.

General settings
^^^^^^^^^^^^^^^^

The most important parameters are:

.. _`data_path`:

Data path
    Defines where measurement files generated by scans or other plugins will be saved.
    
.. _`config_path`:

Config path
    Defines where all settings and plugin configurations are saved to restore them when
    restarting the software. 

.. _`plugin_path`:

Plugin path
    Defines where the software will look for custom plugins to load while starting. 
    
These parameters are saved in the registry and will be restored independent of the .ini file.

Further settings in this section allow to change the resolution used for
graphs, change between dark and light mode, and enable the test mode.
All device plugins have access to the test mode and should simulate
communication if it is active. This is used for development on machines
that are not connected to the hardware.

.. _`sec:session_settings`:

Session settings
^^^^^^^^^^^^^^^^

All results are automatically saved to the *Session path* (inside *Data
path*), based on the date, time, substrate, ion, session type, and
automatically incremented measurement number. This ensures a consistent
file structure that makes it easy to find related sessions. The session
path is updated automatically when any of its elements is changed and
maintained if the program is restarted. If the software is extended for
other types of experiments, the components of the session path can be
replaced by more appropriate ones.

Acquisition settings
^^^^^^^^^^^^^^^^^^^^

This section allows to reduce the number of displayed data points to
increase performance. It also allows to chose if data should be restored
after a restart.

Other settings
^^^^^^^^^^^^^^

Any plugin can define additional settings and add them here in a
dedicated section (e.g., see :ref:`sec:device_settings`). Alternatively, plugins can
use the same built-in functions to manage settings internally (e.g., see :ref:`sec:scan_settings`).

.. _`sec:explorer`:

Explorer
~~~~~~~~

The integrated file explorer is used to navigate all results and
complementary data. All files can be accessed independently using the operating system
file explorer, e.g., when working on a computer where *ESIBD Explorer*
is not installed. However, the integrated explorer connects dedicated :ref:`sec:displays`
to all files that were created with or are supported by *ESIBD
Explorer*. All supported files are preceded with an icon that indicates
which plugin will be used to display them. The :ref:`data path<data_path>`, current
:ref:`session path<sec:session_settings>`, and a search bar are accessible directly from here. File system
links or shortcuts are supported as well.

These displays were made to simplify data analysis and documentation.
They use dedicated and customizable views that allow saving images as
files or sending them to the clipboard for sharing or documentation in a
lab book. Right clicking supported files opens a context menu that allows
to load settings and configurations directly. For example, a scan file
does not only contain the scan data, but also allows to inspect and
restore all experimental settings used to record it. Note that the
context menu only allows to load device values, but the files contain
the entire device configuration. To restore the device configuration
based on a scan file, import the file from the device toolbar. A double
click will open the file in the external default program.

The explorer may also be useful for other applications beyond managing
experimental data. For example, if you organize the documentation of the
experimental setup in folders following the hierarchy of components and sub
components, it allows you to quickly find the corresponding manuals and
order numbers. In combination with the :ref:`sec:notes` plugin, you can add comments to
each component that will be displayed automatically as soon as you
enter the corresponding folder.

.. _`sec:devices`:

Devices
~~~~~~~

Device plugins are used to handle communication with one or more
devices, provide controls to configure the device and display live or
previously recorded data. There are *input devices* (sending input from
the user to hardware) and *output devices* (reading outputs from
hardware). Note that some *input devices* may also read back data from
hardware to confirm that the user defined values are applied correctly.
The main interface consists of a list of :ref:`sec:channels`. By
default only the physically relevant information is shown. By entering
the advanced mode, additional channel parameters can be configured. The
configuration can be exported and imported, though once all channels
have been setup it is sufficient to only load values which can be done
using a file dialog or from the context menu of an appropriate file in
the :ref:`sec:explorer`. After loading the configurations or values, a change log will be
available in the :ref:`sec:text` plugin to quickly identify what has changed. Each
device also comes with a :ref:`display<sec:displays>` and a :ref:`live display<sec:live_displays>`. The
values can also be plotted to get a quick overview and identify any
unusual values.

.. _`sec:device_settings`:

Device settings
^^^^^^^^^^^^^^^

Each device adds a section with the following settings to the :ref:`sec:settings`.

.. _`acquisition_interval`:

Interval
   Data acquisition interval in ms.

Interval (measured)
   Measured plot interval in ms. If this deviates multiple times in a
   row, the number of display points will be reduced and eventually
   acquisition will be stopped to ensure the application remains
   responsive.

Max storage
   Maximum amount of storage used to store history in MB.

Max data points
   Maximum number of data points saved per channel, based on max
   storage. If this is reached, older data will be thinned to allow to
   keep longer history.

Other
   Additional device specific settings, e.g. COM ports or IP addresses
   for communication.

.. _`sec:channels`:

Channels
~~~~~~~~

Channels provide a consistent and structured interface to inputs and
outputs. In the advanced mode, channels can be duplicated, moved, or
deleted. You may also edit channels directly in the corresponding .ini
file in the :ref:`config path<config_path>`. The following defines basic channel
proterties.

Select
   In the advanced mode, this is used to select a channel to be
   duplicated, moved, or deleted.

Enabled
   If enabled, the channel will communicate with the device.

Name
   Each channel should have a unique name that will be used to link to
   the channel from other plugins. The same name may be used for an
   input and an output channel. Warnings will be displayed if a channel
   name is repeated.

Value
   The value to be applied to or read from the physical channel.

Monitor
   The read back used to confirm a value has been applied correctly.

Min
   Minimal limit for value. Values will be coerced to limits.

Max
   Maximal limit for value. Values will be coerced to limits.

Optimize
   If selected, the value will be included in optimization using the :ref:`sec:genetic algorithm<sec:genetic_algorithm>`.

.. _`equation`:

Equation
   The equation is used to determine the value if the channel is not active.
   For input devices, this can be used simplify the setup by reducing
   the number of independent parameters. Any input channel from any
   device can be linked. For output devices this could be used, e.g., to
   record differences between two channels. Any input or output channel
   form any device can be linked. There are no special requirements for
   the format of the equation as long as it evaluates to a valid python
   expression after replacing the names with the value from the
   corresponding channel. Take care to avoid circular definitions and
   non unique names!

Background
   For output channels, background signals can be defined manually or
   using the *set current value as background* option in the
   corresponding :ref:`live display<sec:live_displays>`. Each value will be saved with a
   corresponding background. Backgrounds can be optionally subtracted.

Display
   If selected, channel will be shown in the corresponding :ref:`live display<sec:live_displays>`.

Active
   See :ref:`equation`. If channels are not active, they will only be visible
   in the advanced mode.

Real
   Real channels correspond to a real physical input or output. A
   channel that is not real is a virtual channel. It can be used to in
   combination with equations to create a more intuitive set of
   parameters for the user. For example, a deflector lens with four
   elements can be controlled by virtual *up-down*, and *left-right*
   parameters.

Linewidth
   The linewith is used to highlight the relative importance of channels in
   the :ref:`live display<sec:live_displays>`.

Color
   The channel color can be used to visually group related channels
   allowing for easier navigation in a long list of channels. The
   channel color will also be used in the corresponding 
   :ref:`display<sec:displays>` and :ref:`live display<sec:live_displays>`.

Other
   Additional device specific channel parameters like *ID*, *COM*,
   *Module*, ect.

.. _`sec:scans`:

Scans
~~~~~

Scans are all sort of measurements that record any number of outputs as a
function of any number of inputs. The main interface consists of the
:ref:`sec:scan settings<sec:scan_settings>`. Each scan comes with a tailored display
optimized for its specific data format. Scan settings can be imported
and exported from the scan toolbar, though in most cases it will be
sufficient to import them from the context menu of a previously saved
scan file in the :ref:`sec:explorer`. When all settings are defined and all relevant channels are
communicating the scan can be started. A scan can be stopped at any
time. At the end of a scan the corresponding file will be saved to the
:ref:`sec:session path<sec:session_settings>`. The filename is displayed inside the corresponding graph to
allow to find the file later based on exported figures. Scan files are
saved in the widely used HDF5 file format that allows to keep data and
metadata together in a structured binary file. External viewers, such as
HDFView, or minimal python scripts based on the h5py package can be used
if files need to be accessed outside of the *ESIBD Explorer*. Use the
context menu of a scan file to create a template plot file using h5py
and adjust it to your needs.


.. _`sec:scan_settings`:

Scan settings
^^^^^^^^^^^^^

Notes
   Add specific notes and metadata to the current scan. Will be reset
   after scan is saved.

Display
   List of output channels. All defined display channels are recorded
   but only the selected channel is displayed. A corresponding control in
   the display allows to change which channel is displayed.

Wait
   Wait time between small steps in ms.

Wait long
   Wait time between steps larger than :ref:`large step<large_step>` in ms.

.. _`large_step`:

Large step
   Threshold step size to use longer wait time. This can be useful if
   the effect of changing the input is delayed or there is a pickup that
   needs to decay before measuring.

Average
   Time used for averaging in ms.

Scan time
   Estimated scan time based on the other settings.

Channel
   Input channel to be scanned.

From
   Scan will start at this value.

To
   Scan will stop at this value.

Step
   Scan will use this step size.

Other
   Additional scan specific parameters.

.. _`sec:genetic_algorithm`:

Genetic algorithm
~~~~~~~~~~~~~~~~~

This plugin allows to integrate an independently developed genetic
algorithm (GA) for automated optimization of
signals.\ :cite:`esser_cryogenic_2019`
Input channels can be selected to be included in the optimization. Make sure to choose safe
limits for optimized channels and choose appropriate wait and average
values to get valid feedback. The performance and reliability of the
optimization depends on the stability and reproducibility of the
selected output channel. At the end of the optimization the changed
parameters will be shown in the plugin. The initial parameters can
always be restored in case the optimization fails.

Omni scan
~~~~~~~~~

.. automodule:: esibd.plugins_internal.scans.Scans.Omni
   :noindex: 

Custom extensions
~~~~~~~~~~~~~~~~~

The minimal code in *examples/Custom.py* demonstrates how to integrate your own
custom elements to the *ESIBD Explorer*. This should be sufficient as
long as your code does not requires interaction with any other elements
of the *ESIBD Explorer*. See :ref:`sec:plugin_system` for more information.

Custom device extensions
~~~~~~~~~~~~~~~~~~~~~~~~

The minimal code in *examples/CustomDevice.py* is an example of how to integrate a custom device.
See :ref:`sec:plugin_system` for more information.

.. _sec:displays:

Displays
--------

A variety of displays are implemented and selected automatically by the
corresponding plugin or based on the file type. This includes common
file types like .txt, .tex, .png, .jpg, .bmp, .gif, .svg, .wav, .mp3,
.ogg, .pdf, .h5, .hdf5, .npy, .star, .pdb1, .css, .py, .js, .html, .ini,
.bat, and more. :ref:`sec:devices` and :ref:`scans<sec:scans>` will also come with internal
display plugins that are tailored to visualize their specific data.
Being able to get all required information form these files without
switching to external software avoids clutter and allows to keep an eye
on the live signals of the experiment while working with complementary
information. Graphs are made with matplotlib, pyqtgraph, or both,
allowing the user to choose between complementary ways of data
representations and interaction.

All build in display plugins may serve as examples for the development
of similar plugins for other purposes. See :ref:`sec:plugin_system` for more information.

.. _`sec:browser`:

Browser
~~~~~~~

.. automodule:: esibd.plugins.Browser
   :noindex:

.. _`sec:notes`:

Notes
~~~~~

.. automodule:: esibd.plugins.Notes
   :noindex:

.. _`sec:text`:

Text
~~~~

.. automodule:: esibd.plugins.Text
   :noindex:
   
Tree
~~~~

.. automodule:: esibd.plugins.Tree
   :noindex:

MS
~~

.. automodule:: esibd.plugins_internal.displays.Displays.MS
   :noindex:

In most cases you will need to create your own version of this plugin
that is inheriting from the built-in version and redefines how data is
loaded for your specific data format. See :ref:`sec:plugin_system` for more information.

Line
~~~~

.. automodule:: esibd.plugins_internal.displays.Displays.LINE
   :noindex:

In most cases you will need to create your own version of this plugin
that is inheriting from the build in version and redefines how data is
loaded for your specific data format. See :ref:`sec:plugin_system` for more information.

PDB
~~~

.. automodule:: esibd.plugins_internal.displays.Displays.PDB
   :noindex:

Holo
~~~~

The *Holo* plugin was designed to display 3D NumPy arrays such as
holograms from low energy electron holography.\ :cite:`longchamp_imaging_2017,ochner_low-energy_2021,ochner_electrospray_2023` Interactive 3D surface
plots with density thresholds allow for efficient visualization of very
large files.

.. _`sec:live_displays`:

Live displays
-------------

*Live displays* show the history of measured data over time. The toolbar
provides icons to initialize, start, pause, stop acquisition, optionally
subtract backgrounds, or export displayed data to the current session.
The length of the displayed history is determined by the display time
control in the tool bar.

Frequently updating those plots is typically the computationally most
expensive action of the *ESIBD Explorer*. Thus you might want to reduce
the number of displayed data points in the :ref:`sec:settings`. This will make sure that
the graphs are updated less frequently and select a smaller but
consistent subset of data points for a smooth visualization. While
PyQtGraph provides its own algorithms for down sampling data (accessible
via the context menu), they tend to cause a flicker when updating data.

.. _`sec:device_manager`:

Device manager
--------------

The *device manager*, by default located below the :ref:`live displays<sec:live_displays>`, bundles
functionality of devices and thus allows to initialize, start, and stop
data acquisition from all devices with a single click. Ideally, plugins
that control potentially dangerous hardware like power supplies, cryo
coolers, or vacuum valves should add a status icon to the instrument
manager, so that their status is visible at all times and they can be
shut down quickly, even when the corresponding plugin tab is is not
selected. Internally, the *device manager* also serves as a
central interface to all data channels, independent of the devices they
belong to, making it easy to setup collection of any number of output
signals as a function of any number of input signals.

.. _`sec:console`:

Console
-------

.. automodule:: esibd.plugins.Console
   :noindex:

ESIBD specific plugins
----------------------

The following plugins are specific for ESIBD experiments but may be very
similar to what is needed for other applications. If you do ESIBD
experiments, make sure to get familiar to those plugins, otherwise
consider using them as templates for your own custom plugins. See :ref:`sec:plugin_system` for
more information.

.. _`sec:ISEG`:

ISEG
~~~~

.. automodule:: esibd.plugins_internal.devices.ISEG.Voltage
   :noindex:

RBD
~~~

.. automodule:: esibd.plugins_internal.devices.RBD.Current
   :noindex:

Pressure
~~~~~~~~

.. automodule:: esibd.plugins_internal.devices.Pressure.Pressure
   :noindex:
   
.. _`sec:Temperature`:

Temperature
~~~~~~~~~~~

.. automodule:: esibd.plugins_internal.devices.Temperature.Temperature
   :noindex:

Beam
~~~~

.. automodule:: esibd.plugins_internal.scans.Scans.Beam
   :noindex:

Energy
~~~~~~

.. automodule:: esibd.plugins_internal.scans.Scans.Energy
   :noindex:
   
Depo
~~~~

.. automodule:: esibd.plugins_internal.scans.Scans.Depo
   :noindex:
   

