Generative Metadata Format (GMF)
================================

At the core of ``metasyn`` lies its ability to :doc:`generate </usage/generating_metaframes>`, :doc:`export</usage/exporting_metaframes>` and :doc:`import</usage/exporting_metaframes>` statistical metadata for a given dataset, which can then be used to :doc:`generate synthetic datasets </usage/generating_synthetic_data>`. To achieve this, ``metasyn`` uses the Generative Metadata Format (GMF), an open source format (available on `GitHub <https://github.com/sodascience/generative_metadata_format>`_) designed to store statistical metadata for tabular datasets. The GMF standard is designed to be modular and extensible,  with more distributions and privacy-enhancing mechanisms. Due to its open nature, GMF can be used by other software too.



On a base level, the generative metadata generated by ``metasyn`` contains the following information:

* ``n_rows``: Number of rows
* ``n_columns``: Number of columns
* ``provenance``: Details on how the file was created.
* ``vars``: Description of all variables (columns) of the dataset. 

Each variable in ``vars`` contains at least the following attributes:

* ``name``: Name of the variable (column name).
* ``type``: Type of variable
* ``dtype``: This is the polars dtype that the variable has to be converted back to.
* ``prop_missing``: The proportion of values that are missing (NA).
* ``distribution``: The distribution that describes the variable. This distribution has at least two attributes:

    * ``name``: Name of the distribution.
    * ``parameters``: Dictionary containing the parameters of the distribution.

The full metadata schema used in ``metasyn`` can be found on the `metasyn GitHub repository <https://github.com/sodascience/metasyn/blob/main/metasyn/schema/generative_metadata_format.json>`_.

