:tocdepth: 3

*******************
Conceptual Overview
*******************

The Relay is built on top of the `Realtime API <https://docs.pupil-labs.com/invisible/how-tos/integrate-with-the-real-time-api/introduction/>`_.
It searches and displays Pupil Companion Device instances in the network. Once the user selected one of them, it creates
LSL outlets for each data stream, reads data samples from the connected device, adapts them to an LSL-readable
format, and pushes the data samples into their respective outlets.

The Relay pushes data samples with explicit timestamps. Timestamps are taken at the
companion device, and corrected by the offset between the lsl clock and the current time. We
provide a detailed overview of timestamps and potential caveats in the `Timestamps`_ section.

Device Selection
================
After starting the lsl relay via the command line, the relay searches for Pupil Companion device instances in the network.
Available instances will be displayed in a list, with the IP address and name of the Companion device. Select
the instance you want to connect with via the displayed index.

Troubleshooting
***************
If your Pupil Companion device does not appear in the device selection, please check if both the PC running the relay
and the Companion device are connected to the same network. Also, make sure that your Pupil Companion app is at
at least version v1.4.14 or higher.


LSL Outlets
===========
The Relay creates two outlets - one streaming gaze data, the other streaming event data. Both outlets can be
subscribed to independently, either through the LabRecorder App or with custom written LSL inlets.

Gaze Data Outlet
****************
The default name of the gaze data stream is **pupil_labs_Gaze**. This name will also show in the LabRecorder.
The info of the data stream follows the recommendations of the `xdf Gaze Meta Data format <https://github.com/sccn/xdf/wiki/Gaze-Meta-Data>`_.

The time series of the gaze data stream consists of two channels - one streaming the x- and the other streaming
the y-coordinate of the gaze position in pixel, estimated based on both eyes. The gaze data is streamed at a
frequency of ~66 Hz. If you want to use the 200 Hz gaze data from pupil cloud, you will need to start a
recording with your companion device simultaneously with the LSL recording, and use the ``lsl.time_sync.*``
events generated by the relay to align you data streams post-hoc.

.. Important::
   If you want to do the post-hoc alignment of LSL data and cloud data, you must also subscribe to the LSL
   event stream and make sure that at least two events are contained in your recording.

Event Data Outlet
*****************
The default name of the event stream is **pupil_labs_Event**.
The event stream contains the name of each generated event as a string.

By default, the event stream will contain an ``lsl.time_sync.*`` event every 60 seconds. These events can be used to align
the gaze and event data from pupil cloud to the lsl xdf file, or vice versa. If you want to change the rate at which the
time_sync event is generated, you can use the ``--time_sync_interval`` argument to set the interval to a value of your choice.
If you want to remove the lsl.time_sync events, you can set the argument to 0.

.. _timestamp_docs:

Timestamps
==========
The recording of gaze data at the companion device, and its transfer to LSL involves several steps of taking
and aligning timestamps. This section gives you an overview, of how the timestamps you'll get out of LSL will
be generated.

#. The original timestamp for both Gaze samples and Events is generated at the Pupil Labs Companion Device. The timestamps are generated in nanoseconds since epoch (01.01.1970).

#. The Gaze samples and Events are transferred to the Relay via the realtime API. Gaze timestamps are converted to seconds during the transfer, Event timestamps are converted to seconds in the Relay.

#. Upon the arrival of each sample, the Relay computes the difference between the lsl local clock (in seconds) and the momentary time since epoch (converted to seconds). This assumes that the time since epoch measured on the device running the relay and on the companion device are equivalent.

#. The offset between the lsl local clock and the time since epoch in seconds is subtracted from the sample timestamp in seconds to find the corresponding lsl time. This corrected timestamp is explicitly pushed to LSL together with the Gaze samples and the Events.

.. caution::
   A misalignment between the time since epoch measured at the companion device and at the time since epoch measured
   at the device running the Relay might lead to a distortion of the time series.
