Metadata-Version: 2.4
Name: isis_streaming_data_types
Version: 0.1.1
Summary: Python utilities for handling ISIS streamed data
Author-email: ISIS Experiment Controls <ISISExperimentControls@stfc.ac.uk>
Maintainer-email: ISIS Experiment Controls <ISISExperimentControls@stfc.ac.uk>
Project-URL: Homepage, https://github.com/isiscomputinggroup/streaming_data_types
Project-URL: Bug Reports, https://github.com/isiscomputinggroup/streaming_data_types/issues
Project-URL: Source, https://github.com/isiscomputinggroup/streaming_data_types
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.12
Description-Content-Type: text/markdown
Requires-Dist: flatbuffers
Requires-Dist: numpy>2
Provides-Extra: dev
Requires-Dist: ruff>=0.8; extra == "dev"
Requires-Dist: pyright; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"

# Streaming Data Types
[![Crates.io](https://img.shields.io/crates/v/isis_streaming_data_types.svg)](https://crates.io/crates/isis_streaming_data_types) [![PyPI](https://img.shields.io/pypi/v/isis_streaming_data_types.svg)](https://pypi.org/project/isis_streaming_data_types/)

FlatBuffers is the format chosen for the ISIS data streaming system, derived from the
[ESS messaging system](https://github.com/ess-dmsc/streaming-data-types).

## Schema ids
| ID   | File name                        | Description                                                                                   |
|------|----------------------------------|-----------------------------------------------------------------------------------------------|
| f144 | `f144_logdata.fbs              ` | Controls related log data, typically from EPICS or NICOS. Note: not to be used for array data |
| ev44 | `ev44_events.fbs               ` | Multi-institution neutron event data for both single and multiple pulses                      |
| df12 | `df12_det_spec_map.fbs         ` | Detector-spectrum map for Mantid                                                              |
| se00 | `se00_data.fbs                 ` | Used for storing arrays with optional timestamps, for example waveform data. Replaces _senv_. | 
| ad00 | `ad00_area_detector_array.fbs  ` | EPICS area detector array data                                                                |
| hs01 | `hs01_event_histogram.fbs      ` | Event histogram stored in n dim array                                                         |
| ep01 | `ep01_epics_connection.fbs  `    | Status or event of EPICS connection. Replaces _ep00_                                          |
| json | `json_json.fbs                 ` | Carries a JSON payload                                                                        |
| pl72 | `pl72_run_start.fbs            ` | File writing, run start message for file writer and Mantid                                    |
| 6s4t | `6s4t_run_stop.fbs             ` | File writing, run stop message for file writer and Mantid                                     |
| answ | `answ_action_response.fbs      ` | Holds the result of a command to the filewriter                                               |
| wrdn | `wrdn_finished_writing.fbs     ` | Message from the filewriter when it is done writing a file                                    |
| x5f2 | `x5f2_status.fbs               ` | Status update and heartbeat message for any software                                          |
| fc00 | `fc00_forwarder_config.fbs     ` | Configuration update for Forwarder                                                            |
| al00 | `al00_alarm.fbs                ` | Generic alarm schema for EPICS, NICOS, etc.                                                   |
| da00 | `da00_dataarray.fbs            ` | Pseudo-scipp DataArray with time-dependent and constant Variables                             |
| un00 | `un00_units.fbs            `     | Engineering units update                                                                      |


## Adding new schemas

### Add `.fbs` file to `schemas directory

Check `ess-streaming-data-types` first; attempt not to diverge without reason. If a new schema is really needed, then
add a new `.fbs` schema in the `schemas` directory.

Note: to generate code from `.fbs` schemas, you will need the `flatc` tool installed. It can be acquired from
https://github.com/google/flatbuffers/releases .

### Adding Python bindings

Python bindings have low-level code (autogenerated by `flatc`) in the `fbschemas` directory, but **also**
manually-written convenience serializers and deserializers in the top-level of the python module.

When adding or modifying a schema:
- The low-level code must be generated by manually calling `flatc --python schemas\some_schema.fbs` and adding
  the resulting generated code to `python/src/streaming_data_types/fbschemas`.
- Manually-written serializers & deserializers will need to be updated, and added to the lists in `__init__.py`.

### Rust bindings

Rust bindings have low-level code in `flatbuffers_generated`, and a small high-level wrapper
to deserialize any message.

When adding or modifying a schema:
- The low-level code must be generated by calling `generate_rust_bindings.py`
- The wrapper (defined in `lib.rs`) will need to be updated with the new schema.
