# Featurisation

The production featurisation metadata is built on top of the `molflux.features` module standard config format to ensure
that the featurisation process performed at model training time, can be replicated across all downstream environments -
for example, when performing real-time model serving through a REST API or when querying models for local batch inference.


## Featurisation Metadata

The production featurisation metadata schema is a higher-level wrapper around the `molflux.features` featurisation configs.

Crucially, it defines a schema `version` (the only valid value at present is `1`, but having
this key allows the schema to evolve while still preserving backwards compatibility), and supplements
the familiar `molflux.features` configs listed under the `config` key with the `as` field which lets you assign
custom output names to the features generated by a given representation:

```json
{
    "version": 1,
    "config": [
        {
            "column": "smiles",
            "representations": [
                {
                    "name": "morgan",
                    "as": "ecfp6"
                },
                {
                    "name": "character_count",
                    "as": "{feature_name}",
                    "config": {},
                    "presets": {}
                }
            ]
        }
    ]
}
```

## Featurisation API

The productionising `feastusiation` API helps you handle dataset featurisation when training your machine learning models.
Featurisation happens on your local machine. This means that you are in charge of making sure that your environment
is able to support all of the featurisation methods required by your model.

To featurise your dataset from featurisation metadata:

```python
from molflux.core import featurise_dataset

# featurisation_metadata = <your-featurisation-metadata>
# dataset = <dataset-to-featurise>

featurised_dataset = featurise_dataset(dataset, featurisation_metadata=featurisation_metadata)
```

You can also pass any `map_kwargs` that the HuggingFace `map` method takes in the `featurise_dataset` function, for example:

```python
featurised_dataset = featurise_dataset(dataset, featurisation_metadata=featurisation_metadata, num_proc=4, batch_size=100)
```


```{tip}
When saving a model, make sure to pass along your featurisation metadata to ensure that the featurisation process
can be replayed downstream.

See [model saving](models.md) for more info.
```

As a user, you can then automatically replay the featurisation process for any given model that has been saved:

```python
from molflux.core import replay_dataset_featurisation

# samples = <my-arrow-like-dataset>
# model_path = <path-to-the-model-of-interest>

featurised_samples = replay_dataset_featurisation(samples, model_path=model_path)
```

You can pass any `map_kwargs` here as well!
