Core elements
*************

Ocelot workflow
===============

Running Ocelot usually involves the following steps:

* Get the path of an undefined database in ecospold2 format on your local computer.
* Decide on a system model configuration to transform the undefined datasets to a linked database. This configuration could be a list of Python functions, or could be the default Ocelot system model.
* Call the ``system_model`` function, either directly or through the command line application. ``system_model`` takes the directory path from step one and the configuration from step two as inputs.
* Look through the HTML report generated by the system model function, and either accept the given linked database, or make changes in your configuration definitions or transformation functions.

.. _configuration:

Configuration & ``system_model``
================================

An Ocelot system model configuration is essentially just a list of transformation functions which, when applied in order, produce a realization of a linked database. Configurations are currently specified in Python code, but in the future will also be able to be defined in other formats such as Excel.

We are actively exploring various ways of defining these configurations. The built-in configurations will be provided as a list of transformation functions already in Ocelot, perhaps wrapped in a configuration object. Another simple configuration format would be a text file, where each line was the name of a transformation function that could be imported from ``ocelot.transformations``. However, this doesn't work well for user-defined functions, nor if you need to prepare functions by e.g. `currying <https://en.wikipedia.org/wiki/Currying>`__ them. We are also looking at several configuration libraries, but haven't found anything that seems to fit our mental models or use cases well:

* `ConfigParser <https://docs.python.org/3/library/configparser.html>`__ (In Python standard library)
* `configure <http://configure.readthedocs.io/en/latest/>`__
* `PyStaticConfiguration <https://github.com/dnephin/PyStaticConfiguration>`__
* `pymlconf <https://pythonhosted.org/pymlconf/>`__

So far no final decisions have been made, and things here will evolve along with the Ocelot codebase.

Running Ocelot without specifying a configuration will use the default configuration, which is the cutoff system model.

A typical system model may have many transformation functions, as each function should do exactly one specific change. To make configurations more readable, you can use a ``Collection`` object to group transformation functions that are commonly used together, or that form one unit of work.

.. autoclass:: ocelot.Collection

The ``system_model`` function is actually quite simple:

.. _systemmodel:

.. automethod:: ocelot.model.system_model

Transformation functions
========================

Transform functions are the heart of Ocelot - each one performs one distinct change to the collection of datasets. Transform functions can be any callable, bit are mostly commonly functions.

The report generator will use information about each transform function when creating the report. Specifically, the report generator will look at the function name, its `docstring <https://www.python.org/dev/peps/pep-0257/>`__ (a text description of what the function does, included in the function code), and a new object attribute that you have to specify: ``__table__``.

Most of the time you will want to provide logging data that will be turned into tables in the run report. To do this, define the transformation function attribute ``__table__`` as follows:

.. code-block:: python

    def some_transformation(data):
        return data

    some_transformation.__table__ = {
        'title': 'Name of title to put in report',
        'columns': ["names", "of", "columns"]
    }

Functions take two input arguments: The input ``data``, and the ``logger``. Functions should return the transformed data. Log messages are written using ``logger.log(message)``. Log messages should be a dictionary, with at least the key ``type`` defined. Log messages that provide table data should look like this:

.. code-block:: python

    logger.log({
        'type': 'table element',
        'data': [data in same order as columns]
    })

If you need to initialize functions using `functools.partial <https://docs.python.org/3.5/library/functools.html#functools.partial>`__, the report generator will still get the correct function metadata. Other forms of currying are not supported.

.. _logger:

Logger
======

Ocelot uses a very simple ``Logger`` class that writes messages in JSON. Log messages are written when a run is started or finished, when transformation functions are started or finished, and whenever the transformation function wants to log something. The log message format is documented in :ref:`logging-format`.

.. autoclass:: ocelot.Logger

.. _report:

Reports
=======

In the last step in the workflow, the model run log data is formatted into an HTML report.

.. autoclass:: ocelot.HTMLReport
