.. _datamodels:

About datamodels
================

The purpose of the data model is to abstract away the peculiarities of
the underlying file format.  The same data model may be used for data
created from scratch in memory, or loaded from
`ASDF <https://asdf-standard.readthedocs.io/>`_ files or some future file format.

The detailed datamodel structure and specifics are contained in the
documentation included with the `roman_datamodels` package found `here <https://roman-datamodels.readthedocs.io/en/latest/>`_.

Each model instance is created to contain a variety of attributes and data that
are needed for analysis or to propagate information about the file and the
contents of the file. For example, the `ImageModel` class has the following
arrays associated with it:

    - `data`: The science data
    - `dq`: The data quality array
    - `err`: The error array

Along with data arrays the datamodel also contains information about the
observation that can include the observation program, exposure information,
pointing information and processing steps.

Working with models
===================

Reading a data model
--------------------

If you have an existing data file it is straightforward to access the file
using python.

.. testsetup::
    >>> from roman_datamodels import datamodels as rdm
    >>> from roman_datamodels.maker_utils import mk_datamodel
    >>> fn = 'r0019106003005004023_03203_0034_WFI01_cal.asdf'
    >>> image_model = mk_datamodel(rdm.ImageModel)

    # set some metadata for the below tests
    >>> image_model.meta.filename = fn


.. code-block:: python

    >>> from roman_datamodels import datamodels as rdm
    >>> fn = 'r0019106003005004023_03203_0034_WFI01_cal.asdf'
    >>> image_model = rdm.open(fn)  # doctest: +SKIP
    >>> type(image_model)
    <class 'roman_datamodels.datamodels._datamodels.ImageModel'>

Where the output of the type command tells you that you have imported an
ImageModel from roman_datamodels,

Creating a data model from scratch
----------------------------------

To create a new `ImageModel`, you can just

.. code-block:: python

    >>> from roman_datamodels import datamodels as rdm
    >>> from roman_datamodels.maker_utils import mk_datamodel

    >>> new_model = mk_datamodel(rdm.ImageModel)
    >>> type(new_model)
    <class 'roman_datamodels.datamodels._datamodels.ImageModel'>

.. warning::

    The values in the file generated by mk_datamodel are intended to be
    clearly incorrect and should be replaced if the file is intended to be used
    for anything besides a demonstration.

Saving a data model to a file
-----------------------------

Simply call the `save` method on the model instance:

.. code-block:: python

    >>> image_model.save("myimage.asdf")
    PosixPath('myimage.asdf')

.. note::

   This `save` always clobbers the output file. For now the only format
   supported is ASDF.

Creating a data model from a file
---------------------------------

The `roman_datamodels.open` function is a convenient way to create a
model from a file on disk.  It may be passed any of the following:

    - a path to an ASDF file
    - a readable file-like object

The file will be opened, and based on the nature of the data in the
file, the correct data model class will be returned.  For example, if
the file contains 2-dimensional data, an `ImageModel` instance will be
returned.  You will generally want to instantiate a model using a
`with` statement so that the file will be closed automatically when
exiting the `with` block.

.. code-block:: python

    >>> with rdm.open("myimage.asdf") as im:
    ...    assert isinstance(im, rdm.ImageModel)

If you know the type of data stored in the file, or you want to ensure
that what is being loaded is of a particular type, use the constructor
of the desired concrete class.  For example, if you want to ensure
that the file being opened contains 2-dimensional image data

.. code-block:: python

    >>> with rdm.ImageModel("myimage.asdf") as im:
    ...     pass  # raises exception if myimage.asdf is not an image file

This will raise an exception if the file contains data of the wrong
type.

Copying a model
---------------

To create a new model based on another model, simply use its `copy`
method.  This will perform a deep-copy: that is, no changes to the
original model will propagate to the new model

.. code-block:: python

    >>> new_model = image_model.copy()

Looking at the contents of a model
----------------------------------

You can examine the contents of your model from within python using

.. code-block:: python

    >>> print("\n".join("{: >20}\t{}".format(k, v) for k, v in image_model.items()), "\n") # doctest: +ELLIPSIS
    meta.calibration_software_name	RomanCAL
    meta.calibration_software_version	9.9.0
       meta.product_type	l2
           meta.filename	r0019106003005004023_03203_0034_WFI01_cal.asdf
          meta.file_date	2020-01-01T00:00:00.000
         meta.model_type	ImageModel
             meta.origin	STSCI/SOC
        meta.prd_version	8.8.8
    meta.sdf_software_version	7.7.7
          meta.telescope	ROMAN
    meta.coordinates.reference_frame	ICRS
    ...

or you can print specifics

.. code-block:: python

    >>> print("\n".join("{: >20}\t{}".format(k, v) for k, v in image_model.meta.instrument.items()))
                    name        WFI
                detector        WFI01
         optical_element        F158

.. note::

    These will be incorporated as methods in the data models in a future release.
