Metadata-Version: 2.4
Name: zynamon
Version: 0.3.0
Summary: Generalized & powerful time-series class and related functions
Author-email: "Dr. Marcus Zeller" <dsp4444@gmail.com>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/zynt3c/zynamon
Project-URL: Package, https://pypi.org/project/zynamon/
Keywords: timeseries,analysis,monitoring,prediction,visualisation
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: h5py~=3.15.1
Requires-Dist: matplotlib~=3.10.8
Requires-Dist: numpy~=2.4.2
Requires-Dist: pandas~=3.0.1
Requires-Dist: plotly~=6.5.2
Requires-Dist: psutil~=7.2.2
Requires-Dist: pyarrow~=23.0.0
Requires-Dist: scipy~=1.17.0
Requires-Dist: streamlit~=1.54.0
Requires-Dist: typer~=0.21.1
Requires-Dist: zdev
Dynamic: license-file

# zynamon

Generalized & powerful time-series class and related functions.

## Synopsis

This library defines a powerful, generic **time-series representation based on the popular ```pandas``` DataFrames**. However, it adds meta-data as well as conveniently built-in filtering & alignment routines for both value and time dimensions. Moreover, time-series objects can be **constructed from nearly any 1D data** such that handling of various sources **in a standardised way** is made possible (e.g. regularly-sampled sensor data, event logs, audio streams etc).

## Package Contents

The package is structured into the following modules:

**```zeit```**: Main class ```TimeSeries``` containing time-, value- and samples-related methods and algorithmic operators (+, -, *, /)

**```zeitspec```**: Class ```TimeSpec``` and conversion routine for a unified treatment of various timestamp representations as settings or from data (types and strings)

**```imex```**: Routines for the automatic bulk import, conversion & aggregation of data from (a lot of) CSV-like files, e.g. proper extraction of all individual time-series hidden in lengthy event logs

**```utils```**: Utilities working on ```TimeSeries``` objects such as helpers for general analysis (e.g. outliers, predictions, history etc) and operators on two arbitrary objects (incl. routines to make objects "time-coherent" by aggregation / re-sampling)

**```xutils```**: Special conversion utilities for CSV-files containing DFT spectra as generated by the "CMS X-Tools" real-time multi-channel analysis software (Siemens AG).

## CLI Application

Besides the library core, the **```cli```** module provides convenient **script routines for an automatic processing** (e.g. bulk import and/or conversion from - a loooot of! - CSV-like files). After installation of the package, these routines can either be invoked via the Python module ```zynamon.cli``` or even simpler as standalone executables from the terminal:

    zynamon ROUTINE

In total, the following tasks of ```ROUTINE``` are offered by the CLI application:

+ **importer**
+ **compresser**
+ **merger**

and also a generic **summarizer** that will collate readable information on existing time-series collections within a given root folder. *Note: By default this step is already performed by the above routines.*

All of the routines come with a reasonable default behaviour such that *only few arguments are actually required* while missing details are "guessed in a smart way". Using the ```--help``` option will generate self-explanatory information for an intuitive usage on all routines. For more details refer to the background section (e.g. expected folder structures).

## GUI Application

A special routine of the above interface is finally given by

    zynamon explorer

as this provides a web-based GUI experience powererd by the popular ```streamlit``` rapid prototyping package. This enables to quickly visualise multiple time-series from previously stored collection files while additional cropping, filtering and aggregation options can be applied on-the-fly (i.e. in a non-destructive way).

<img src="docs/GUI_screen.png" width="800" height="275">

## Background

This section provides additional information for users.

### Collection & summary files

```TimeSeries``` objects can either be stored individually or in collection files, housing one or more objects at a time. For the latter, both ```dict``` or ```list``` arrangements are available in Python, allowing to choose the more suitable layout.

In general, the following file formats are supported:

+ ***.pk** => Python's builtin pickle
+ ***.json** => JSON / readable dict-type
+ ***.h5** => HDF5 hierachical storage format
+ ***.parquet** => Apache parquet file (implemented by ```pyarrow``` package, only for single files!)

Since **collections** may include a large number of different time-series, it may be desirable to have means for a quick overview. To this end, so called **summary files** with extension ***.tsinfo** can be generated, offering details in readable text as follows:

+ collection totals: # time-series, min/max lengths, common time interval (if any)
+ per time-series details: name, number of samples, time interval covered

### Folder structures for projects

For "smart guess" defaults to work, a certain folder hierarchy is expected and must be present below each project's root location. This **expected structure** is shown in the following:  

+ ---Project---
  + --Asset A1--
    + -Period P1-
      + subfolders "log-enum" (if any)
      + subfolders "log-real" (if any)
      + subfolders "streams" (if any)
      + subfolders "streams-xt" (if any)
    + -Period P2-
      + subfolders "log-enum" (if any)
      + subfolders "log-real" (if any)
      + subfolders "streams" (if any)
      + subfolders "streams-xt" (if any)
    + ...
  + --Asset A2--
    + -Period P1-
      + ...
    + -Period P2-
      + ...
  + ...

Note that all of the subfolder categories within a period are optional and, thus, zero or more items may be given. However, *at least one item* must be given *for any category* in every period, as otherwise no data will be processed!

### Typical example calls

a) Importing all files acc. to a project configuration and saving all time-series separately:

    zynamon importer . --cfg-file D:/MyData/cfg_project.json --save-series

b) Compressing collection files for given assets & time period sub-folders acc. desired settings:

    zynamon compresser D:/MyProject [Asset1,Asset2] 2025-07 --agg-time [10,60,300,3600] --agg-mode [avg,min,max]

c) Traverse given folder and create summary files for all collections (even if different extensions may be found) and existing summary files may be overwritten:

    zynamon summarizer D:/MyProject --ignore-ext --overwrite --verbosity 1
    
*Note: This use case may occur if collection files are created at different points in time (after new data has become available) and/or with different storage formats and proper summarizing needs to be repeated.*

### Template for configuration files

To create a JSON configuration for a specific project (folder) layout, the following schema can be used. Obviously, not all of these fields are mandatory, but rather dependent on the actual contents of the CSV information. To keep the text short, a **simplified schema documenting style** is used here, where the expected types for values are denoted by ```/schema-type/``` and ```*required``` or ```*required-if-category-used``` is appended if the field *is* essential:

```json
{
    "name": /str/,
    "info": /str/,

    "root": /str/, *required*
    "path": /str/, *required*

    "assets":  /list[str]/,
    "periods": /list[str]/,

    "sub_logs_enum":  /list[str]/,
    "sub_logs_real":  /list[str]/,
    "sub_streams":    /list[str]/,
    "sub_streams_xt": /list[str]/,
    
    "def_logs_enum": /dict/ *required-if-category-used*
    {
        "name": /str/ | /list[str]/, *required*
        "time": /str/, *required*
        "data": /str/  *required*
    },
        
    "def_logs_real": /dict/ *required-if-category-used* 
    {
        "name": /str/ | /list[str]/, *required*
        "time": /str/, *required*
        "data": /str/  *required*
    },

   "def_streams": /dict/ *required-if-category-used*  
   {
        "time": /str/, *required*
        "data": /list[str]/ *required*
    },

    "def_streams_xt": /dict/ *required-if-category-used*  
    {
        "time": /str/, *required*
        "data": /str/ *required*
    },

    "_comment": /str/    
}
```

[ Dr. Marcus Zeller | <dsp4444@gmail.com> | Erlangen, Germany | 2022-2026 ]
