Metadata-Version: 2.1
Name: histgrinder
Version: 0.1.6
Summary: Generic system to perform streaming transformations on histograms
Home-page: https://github.com/ponyisi/histogram_postprocessing
Author: Peter Onyisi
Author-email: ponyisi@utexas.edu
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML (>=5)

# histgrinder
Generic system to perform streaming transformations on histograms

Features:
* core does not depend on any histogram library or input/output format; plugins can be written (and these need not live in this package). Probably most useful if histograms live in a hierarchical namespace but this is not necessary. Example implementation to read/write ROOT files provided.
* intended for streaming, e.g. for online environments where histograms are updated asynchronously.
* pattern matching makes it easy to apply the same transformation to multiple histograms
* no code needed to configure

This is still very much early-release software, you can test it as follows (e.g. should work on lxplus, if you have a CERN account):
* set up ROOT and Python (>=3.7) in a way that you like. For ATLAS people you can set up a master nightly. (The code may run on Python 3.6 but we no longer test it there.)
* install (not needed if you are on ATLAS and using master,2020-10-20T2101 or later): `python3 -m pip install -U --user histgrinder==0.1.6`
* prepare a sample ROOT file: `python3 -m histgrinder.make_sample_file`
* download an example YAML configuration from https://raw.githubusercontent.com/ponyisi/histogram_postprocessing/master/resources/example.yaml 
* run. The following will postprocess `example.root`, created above, according to the `example.yaml` configuration, ignoring the top-level path "prefix", then add the outputs to `example.root`: 

`python3 -m histgrinder.engine example.root example.root -c example.yaml --prefix prefix`
* the transformation above will perform a number of operations on the histograms of the input file. For example, 20 different histogram divisions are configured with the first config block.

Command line arguments:
| Argument | Description |
|--|--|
| `-c`, `--configfile CONFIGFILE [CONFIGFILE ...]` | one or more YAML configuration file(s) |
| `--inmodule` | Python class which implements an input module (default: `histgrinder.io.root.ROOTInputModule`) |
| `--outmodule` | Python class which implements an output module (default: `histgrinder.io.root.ROOTOutputModule`) |
| `--prefix` | Path prefix to ignore in histogram locations in input (will also be prepended to output locations) |
| `--loglevel` | Set the logging level (choices: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`; default: `INFO`) |
| `--defer` |  If specified, defer processing of histograms until all input histograms are read. Major speedups possible if some transformations take a lot of histograms as input. Not for streaming-type jobs. |
| `--delaywrite` | If specified, write histograms at once at end of job. Can speed up tasks if I/O is a bottleneck. Not for streaming-type jobs. |

This work was supported by the US Department of Energy, Office of Science, Office of High Energy Physics, under Award Number DE-SC0007890.


