kedro.pipeline.node.Node — Kedro 0.17.6 documentation

# kedro.pipeline.node.Node

class `kedro.pipeline.node.` `Node`(func, inputs, outputs, *, name=None, tags=None, decorators=None, confirms=None, namespace=None)

  `Node` is an auxiliary class facilitating the operations required to run user-provided functions as part of Kedro pipelines.

  Attributes

  `confirms` | Return dataset names to confirm as a list.
  `func` | Exposes the underlying function of the node.
  `inputs` | Return node inputs as a list, in the order required to bind them properly to the node’s function.
  `name` | Node’s name.
  `namespace` | Node’s namespace.
  `outputs` | Return node outputs as a list preserving the original order
  `short_name` | Node’s name.
  `tags` | Return the tags assigned to the node.

  Methods

  `decorate`(*decorators) | Create a new `Node` by applying the provided decorators to the underlying function.
  `run`([inputs]) | Run this node using the provided inputs and return its results in a dictionary.
  `tag`(tags) | Create a new `Node` which is an exact copy of the current one,

  `__init__`(func, inputs, outputs, *, name=None, tags=None, decorators=None, confirms=None, namespace=None)

    Create a node in the pipeline by providing a function to be called along with variable names for inputs and/or outputs.

    Parameters

      - func (`Callable`) – A function that corresponds to the node logic. The function should have at least one input or output.

      - inputs (`Union`[`None`, `str`, `List`[`str`], `Dict`[`str`, `str`]]) – The name or the list of the names of variables used as inputs to the function. The number of names should match the number of arguments in the definition of the provided function. When Dict[str, str] is provided, variable names will be mapped to function argument names.

      - outputs (`Union`[`None`, `str`, `List`[`str`], `Dict`[`str`, `str`]]) – The name or the list of the names of variables used as outputs to the function. The number of names should match the number of outputs returned by the provided function. When Dict[str, str] is provided, variable names will be mapped to the named outputs the function returns.

      - name (`Optional`[`str`]) – Optional node name to be used when displaying the node in logs or any other visualisations.

      - tags (`Union`[`str`, `Iterable`[`str`], `None`]) – Optional set of tags to be applied to the node.

      - decorators (`Optional`[`Iterable`[`Callable`]]) – Optional list of decorators to be applied to the node.

      - confirms (`Union`[`str`, `List`[`str`], `None`]) – Optional name or the list of the names of the datasets that should be confirmed. This will result in calling `confirm()` method of the corresponding data set instance. Specified dataset names do not necessarily need to be present in the node `inputs` or `outputs`.

      - namespace (`Optional`[`str`]) – Optional node namespace.

    Raises

      ValueError – Raised in the following cases: a) When the provided arguments do not conform to the format suggested by the type hint of the argument. b) When the node produces multiple outputs with the same name. c) When an input has the same name as an output. d) When the given node name violates the requirements: it must contain only letters, digits, hyphens, underscores and/or fullstops.

  property `confirms`

    Return dataset names to confirm as a list.

    Return type

      `List`[`str`]

    Returns

      Dataset names to confirm as a list.

  `decorate`(*decorators)

    Create a new `Node` by applying the provided decorators to the underlying function. If no decorators are passed, it will return a new `Node` object, but with no changes to the function.

    Parameters

      *decorators – Decorators to be applied on the node function. Decorators will be applied from right to left.

    Return type

      `Node`

    Returns

      A new `Node` object with the decorators applied to the function.

    Example:

        from functools import wraps

        def apply_f(func: Callable) -> Callable:
            @wraps(func)
            def with_f(*args, **kwargs):
                args = ["f({})".format(a) for a in args]
                return func(*args, **kwargs)
            return with_f

        def apply_g(func: Callable) -> Callable:
            @wraps(func)
            def with_g(*args, **kwargs):
                args = ["g({})".format(a) for a in args]
                return func(*args, **kwargs)
            return with_g

        def apply_h(func: Callable) -> Callable:
            @wraps(func)
            def with_h(*args, **kwargs):
                args = ["h({})".format(a) for a in args]
                return func(*args, **kwargs)
            return with_h

        def apply_fg(func: Callable) -> Callable:
            @wraps(func)
            def with_fg(*args, **kwargs):
                args = ["fg({})".format(a) for a in args]
                return func(*args, **kwargs)
            return with_fg

        def identity(value):
            return value

        # using it as a regular python decorator
        @apply_f
        def decorated_identity(value):
            return value

        # wrapping the node function
        old_node = node(apply_g(decorated_identity), 'input', 'output',
                        name='node')
        # using the .decorate() method to apply multiple decorators
        new_node = old_node.decorate(apply_h, apply_fg)
        result = new_node.run(dict(input=1))

        assert old_node.name == new_node.name
        assert "output" in result
        assert result['output'] == "f(g(fg(h(1))))"

  property `func`

    Exposes the underlying function of the node.

    Return type

      `Callable`

    Returns

      Return the underlying function of the node.

  property `inputs`

    Return node inputs as a list, in the order required to bind them properly to the node’s function.

    Return type

      `List`[`str`]

    Returns

      Node input names as a list.

  property `name`

    Node’s name.

    Return type

      `str`

    Returns

      Node’s name if provided or the name of its function.

  property `namespace`

    Node’s namespace.

    Return type

      `Optional`[`str`]

    Returns

      String representing node’s namespace, typically from outer to inner scopes.

  property `outputs`
    Return node outputs as a list preserving the original order

      if possible.

    Return type

      `List`[`str`]

    Returns

      Node output names as a list.

  `run`(inputs=None)

    Run this node using the provided inputs and return its results in a dictionary.

    Parameters

      inputs (`Optional`[`Dict`[`str`, `Any`]]) – Dictionary of inputs as specified at the creation of the node.

    Raises

      - ValueError – In the following cases: a) The node function inputs are incompatible with the node input definition. Example 1: node definition input is a list of 2 DataFrames, whereas only 1 was provided or 2 different ones were provided. b) The node function outputs are incompatible with the node output definition. Example 1: node function definition is a dictionary, whereas function returns a list. Example 2: node definition output is a list of 5 strings, whereas the function returns a list of 4 objects.

      - Exception – Any exception thrown during execution of the node.

    Return type

      `Dict`[`str`, `Any`]

    Returns

      All produced node outputs are returned in a dictionary, where the keys are defined by the node outputs.

  property `short_name`

    Node’s name.

    Return type

      `str`

    Returns

      Returns a short, user-friendly name that is not guaranteed to be unique. The namespace is stripped out of the node name.

  `tag`(tags)
    Create a new `Node` which is an exact copy of the current one,

      but with more tags added to it.

    Parameters

      tags (`Union`[`str`, `Iterable`[`str`]]) – The tags to be added to the new node.

    Return type

      `Node`

    Returns

      A copy of the current `Node` object with the tags added.

  property `tags`

    Return the tags assigned to the node.

    Return type

      `Set`[`str`]

    Returns

      Return the set of all assigned tags to the node.