kedro.io.MemoryDataSet

class kedro.io.MemoryDataSet(data=None, max_loads=None)

MemoryDataSet loads and saves data from/to an in-memory Python object.

Example:

from kedro.io import MemoryDataSet
import pandas as pd

data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5],
                     'col3': [5, 6]})
data_set = MemoryDataSet(data=data)

loaded_data = data_set.load()
assert loaded_data.equals(data)

new_data = pd.DataFrame({'col1': [1, 2], 'col2': [4, 5]})
data_set.save(new_data)
reloaded_data = data_set.load()
assert reloaded_data.equals(new_data)

__init__(data=None, max_loads=None)

Creates a new instance of MemoryDataSet pointing to the provided Python object.

Parameters:

• data (Optional[Any]) – Python object containing the data.
• max_loads (Optional[int]) – Maximum number of times load method can be invoked. MemoryDataSet data is reset after this number of calls is made. Any number of calls is allowed if the argument is not set. max_loads counter is reset after every save method call.

Methods

__init__([data, max_loads])	Creates a new instance of MemoryDataSet pointing to the provided Python object.
exists()	Checks whether a data set’s output already exists by calling the provided _exists() method.
from_config(name, config[, load_version, …])	Create a data set instance using the configuration provided.
load()	Loads data by delegation to the provided load method.
save(data)	Saves data by delegation to the provided save method.