kedro.config.ConfigLoader — Kedro 0.17.6 documentation

# kedro.config.ConfigLoader

class `kedro.config.` `ConfigLoader`(conf_paths)

  Recursively scan the directories specified in `conf_paths` for configuration files with a `yaml`, `yml`, `json`, `ini`, `pickle`, `xml` or `properties` extension, load them, and return them in the form of a config dictionary.

  When the same top-level key appears in any 2 config files located in the same `conf_path` (sub)directory, a `ValueError` is raised.

  When the same key appears in any 2 config files located in different `conf_path` directories, the last processed config path takes precedence and overrides this key.

  For example, if your `conf_path` looks like this:

      .
      `-- conf
          |-- README.md
          |-- base
          |   |-- catalog.yml
          |   |-- logging.yml
          |   `-- experiment1
          |       `-- parameters.yml
          `-- local
              |-- catalog.yml
              |-- db.ini
              |-- experiment1
              |   |-- parameters.yml
              |   `-- model_parameters.yml
              `-- experiment2
                  `-- parameters.yml

  You can access the different configurations as follows:

      import logging.config
      from kedro.config import ConfigLoader

      conf_paths = ['conf/base', 'conf/local']
      conf_loader = ConfigLoader(conf_paths)

      conf_logging = conf_loader.get('logging*')
      logging.config.dictConfig(conf_logging)  # set logging conf

      conf_catalog = conf_loader.get('catalog*', 'catalog*/**')
      conf_params = conf_loader.get('**/parameters.yml')

  Methods

  `get`(*patterns) | Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.

  `__init__`(conf_paths)

    Instantiate a ConfigLoader.

    Parameters

      conf_paths (`Union`[`str`, `Iterable`[`str`]]) – Non-empty path or list of paths to configuration directories.

    Raises

      ValueError – If `conf_paths` is empty.

  `get`(*patterns)

    Recursively scan for configuration files, load and merge them, and return them in the form of a config dictionary.

    Parameters

      *patterns – Glob patterns to match. Files, which names match any of the specified patterns, will be processed.

    Raises

      - ValueError – If 2 or more configuration files inside the same config path (or its subdirectories) contain the same top-level key.

      - MissingConfigException – If no configuration files exist within a specified config path.

      - BadConfigException – If configuration is poorly formatted and cannot be loaded.

    Returns

      A Python dictionary with the combined

        configuration from all configuration files. Note: any keys that start with _ will be ignored.

    Return type

      Dict[str, Any]