# Calculated data documents construction module

This module is intended to simplify the construction of calculated data documents.

This documentation starts with a general explication of the module and then each of the components is explained in detail.

## General structure

The module is composed of 5 parts.

```
  ├── README.md        # documentation file
1 ├── extractor.py     # generic logic to transform parser data into elements
2 ├── config.py        # classes to specify the configuration of a construction
3 ├── view.py          # generic logic to split elements into a tree like structure
4 ├── constructor.py   # main logic of module
  ├── docs/            # directory for diagrams
5 ├── <parser_dir N>/  # directory for specific parser code
  │ ├── extractor.py   # specific parser implementation of generic extractor
  │ └── views.py       # specific parser implementation of views
```

### Interaction of parts

![general_struct](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/general_struct.png?raw=true)

1. The objects containing the information of a specific parser is passed to the extractor.
<br> *This objects usually are dataclasses. But they can be any type of python object
2. The specific parser instance of the extractor takes the objects and transform them into generic elements.
<br> *Elements are just a dictionary like representation of the information contained in the original objects
3. The configuration is used to organize the elements into a tree like structure using the inner views defined.
4. The constructor takes the instantiated configuration to create calculated data documents
5. The specific parser mapper takes the calculated data documents and generates the ASM json output.

## Extractor

![extractor_dataclass](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/extractor_dataclass.png?raw=true)

The first part of the module is the extractor. The idea of this part is to transform objects of type T into elements.
An element is a dataclass with a single dictionary as attribute and methods to get the information it contains.

The extractor needs to be implemented for each of the parsers to match the structure of the objects it handles. 

## Config

![config_dataclass](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/config_dataclass.png?raw=true)

The second part of the module is the configuration structure. It consists of two possible dataclasses, Calculated data config or measurement config. The first one represent nodes and the second one leafs. Each calculated data config node needs a view, which is a way to structure the elements needed to construct the desired output.

An example of the configuration follows

![config_struct](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/config_struct.png?raw=true)

In this diagram Calc doc represents a Calculated data config object, Meas represents a Measurement config object and the cylinders represent the views. 

The final structure of the calculated data elements generated by the module is defined by the structure of Calculated data config and Measurement config elements. The number of children or data sources that a certain calculated data node will have is defined by the view attached to it. 

## Views

![views_dataclass](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_dataclass.png?raw=true)

Inside the config there are special elements called views, they are a way to split the elements into a tree like structure. The way this can be achieved is by nesting the view instances. This type of object also allows the implementation of filters and result in a convenient way of data representation called **ViewData**, which are an object to iterate the elements and the different instances that compose the structure.

The views need to be implemented for each of the parsers to match the desired structure of the objects that is needed. 

### Examples

1. Example of a single level view.

![views_generic_simple](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_generic_simple.png?raw=true)

In this example there are 8 elements that are splitted into two groups (one of 5 elements and the other of 3 elements) when a view that use sample id as attribute is used.

2. Example of a multi level view.

![views_generic_composed](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_generic_composed.png?raw=true)

### Quantstudio design and analysis examples

In this section a real example from quantstudio adapter is shown. 
The file used in this example is **appbio_quantstudio_designandanalysis_QS7_Standard_Curve_example06.xlsx** 

1. Example of view with sample id and target dna as levels.

![views_quantstudio1](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_quantstudio1.png?raw=true)

In this example the input of the view are 96 elements. Those elements are first splitted by all values of sample id founded in the 96 samples, in this case **5K, D1, D2, D3, D4, 1250, 2500, 5000, 10000, 20000, 10K**. Then each of those splits is further splitted using the target dna attribute (In this case each group only contains the **RNase P** value) resulting in a single sub group. Finally, at the end of the three there are the elements represented by their **UUIDs**. As can be seen in the diagram each leaf node is composed of different amounts of elements (some contains just 1 element like **D1**, others 4 elements like **2500** and the rest 36 elements like **5K**).

2.  Example of view with sample id, target dna and uuid as levels.

![views_quantstudio2](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_quantstudio2.png?raw=true)

In this example the input of the view are 96 elements. Those elements are first splitted by all values of sample id founded in the 96 samples, in this case **5K, D1, D2, D3, D4, 1250, 2500, 5000, 10000, 20000, 10K**. Then each of those splits is further splitted using the target dna attribute (In this case each group only contains the **RNase P** value) resulting in a single sub group at this level. Finally, at the end of the three there is the split generated by the **UUID**, which splits the elements by an unique attribute resulting in leafs of only one element. As can be seen in the diagram the last level of the three composed of different amounts of nodes, each of them composed by only one element (some finish with just 1 node like **D1**, others with 4 nodes like **2500** and the rest with 36 nodes like **5K**).

3. Example of view with only target dna as level but containing a filter condition.

![views_quantstudio3](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/views_quantstudio3.png?raw=true)

In this example the input of the view are 96 elements. Those elements are first filtered by a condition imposed on the target dna attribute, which results in only 20 elements as part of the split produced by this view. The only value of target dna that is present for those elements is **RNase P** resulting in a single group at this level.

## Constructor

![constructor_dataclass](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/constructor_dataclass.png?raw=true)

Finally, there is the constructor, which is the main component of the module. This class is responsible of the construction of the calculated data documents based on the configuration structure that is provided. The output it provides is defined as follows

![results_dataclass](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/results_dataclass.png?raw=true)

It is composed of two classes, CalculatedDocument representing an operation performed over a list of sources and a DataSource that represent the elements used in a specific calculation. 

The CalculatedDocument dataclass contains an inter_struct method used to transform the tree structure into a list. This method use the iterated attribute to avoid duplication in the final result.

 ### Quantstudio design and analysis example output

![constructor_output1](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/constructor_output1.png?raw=true)

This diagram is a partial representation of the actual output produced by the module when executed with the file **appbio_quantstudio_designandanalysis_QS7_Standard_Curve_example06.xlsx** and the configuration described previously in the config section.

In the diagram a single quantity mean calculated data is displayed (representing the combination of sample id **5k** and target dna **RNase P**. This node contains 36 quantity nodes (representing the combination from **5K, RNase P, UUID 0** to **5K, RNase P, UUID 35**. In this visualization only the childs of the first quantity node are shown, in this case a single measurement **cycle threshold result** corresponding to the element with **UUID 0** and two other calculated data **y-intercept** and **slope** each containing 20 measurements **cycle threshold result** corresponding to elements from **UUID 40** to **UUID 59**.

### Execution

An execution example is shown in the following diagram. In there each box correspond to a function invocation connected by an arrow, the dotted line represents the specification of a computation and the elements outside the box correspond to descriptions or clarifications.

![constructor_execution](https://github.com/Benchling-Open-Source/allotropy/blob/calcdocs-documentation/src/allotropy/calcdocs/docs/constructor_execution.png?raw=true)
