kedro.io — Kedro 0.17.6 documentation

# kedro.io

Description

`kedro.io` provides functionality to read and write to a number of data sets. At core of the library is `AbstractDataSet` which allows implementation of various `AbstractDataSet`s.

Classes

`kedro.io.AbstractDataSet`() | `AbstractDataSet` is the base class for all data set implementations. All data set implementations should extend this abstract class and implement the methods marked as abstract. If a specific dataset implementation cannot be used in conjunction with the `ParallelRunner`, such user-defined dataset should have the attribute _SINGLE_PROCESS = True. Example: ::.
`kedro.io.AbstractVersionedDataSet`(filepath, …) | `AbstractVersionedDataSet` is the base class for all versioned data set implementations.
`kedro.io.AbstractTransformer`() | Transformers will be deprecated in Kedro 0.18.0 in favour of the Dataset Hooks.
`kedro.io.DataCatalog`([data_sets, feed_dict, …]) | `DataCatalog` stores instances of `AbstractDataSet` implementations to provide `load` and `save` capabilities from anywhere in the program.
`kedro.io.LambdaDataSet`(load, save[, exists, …]) | `LambdaDataSet` loads and saves data to a data set.
`kedro.io.MemoryDataSet`([data, copy_mode]) | `MemoryDataSet` loads and saves data from/to an in-memory Python object.
`kedro.io.PartitionedDataSet`(path, dataset[, …]) | `PartitionedDataSet` loads and saves partitioned file-like data using the underlying dataset definition.
`kedro.io.IncrementalDataSet`(path, dataset[, …]) | `IncrementalDataSet` inherits from `PartitionedDataSet`, which loads and saves partitioned file-like data using the underlying dataset definition.
`kedro.io.CachedDataSet`(dataset[, version, …]) | `CachedDataSet` is a dataset wrapper which caches in memory the data saved, so that the user avoids io operations with slow storage media.
`kedro.io.DataCatalogWithDefault`([data_sets, …]) | A `DataCatalog` with a default `DataSet` implementation for any data set which is not registered in the catalog.
`kedro.io.Version`(load, save) | This namedtuple is used to provide load and save versions for versioned data sets.

Exceptions

`kedro.io.DataSetAlreadyExistsError` | `DataSetAlreadyExistsError` raised by `DataCatalog` class in case of trying to add a data set which already exists in the `DataCatalog`.
`kedro.io.DataSetError` | `DataSetError` raised by `AbstractDataSet` implementations in case of failure of input/output methods.
`kedro.io.DataSetNotFoundError` | `DataSetNotFoundError` raised by `DataCatalog` class in case of trying to use a non-existing data set.