Util - Nautobot Design Builder Documentation

Skip to content

# Util

## `nautobot_design_builder.util`

Main design builder app module, contains DesignJob and base methods and functions.

### `conditional_load_job(designs, module_path, module_name, class_name)`

Populate the `designs` dictionary if the loaded design works with the current version of Nautobot.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`designs` | `dict` | Dictionary of design classes | required
`module_path` | `str` | path to module containing the design class | required
`module_name` | `str` | name of the module to load from the package. | required
`class_name` | `str` | name of the class to load from the module. | required

Source code in `nautobot_design_builder/util.py`

    238
    239
    240
    241
    242
    243
    244
    245
    246
    247
    248
    249
    250
    251
    252
    def conditional_load_job(designs, module_path, module_name, class_name):
        """Populate the `designs` dictionary if the loaded design works with the current version of Nautobot.

        Args:
            designs (dict): Dictionary of design classes
            module_path (str): path to module containing the design class
            module_name (str): name of the module to load from the package.
            class_name (str): name of the class to load from the module.
        """
        design_class = get_design_class(module_path, module_name, class_name)
        required_version = getattr(design_class.Meta, "nautobot_version", None)
        if required_version is not None:
            required_version = Specifier(required_version)
        if required_version is None or nautobot_version.version in required_version:
            designs[class_name] = design_class

### `designs_in_directory(path, package_name, local_logger=logger, module_name=None, reload_modules=False)`

Find all the designs in a directory.

Walk the available Python modules in the given directory, and for each module, walk its DesignJob class members.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`path` | `str` | Directory to import modules from, outside of sys.path | required
`package_name` | `str` | The package to which discovered modules will belong. | required
`local_logger` | `Logger` | The logging instance to use. This is especially useful when a logger includes a JobResult. | `logger`
`module_name` | `str` | Specific module name to select; if unspecified, all modules will be inspected | `None`
`reload_modules` | `bool` | Whether to force reloading of modules even if previously loaded into Python. | `False`

Yields:

Type | Description
--- | ---
`str` | ("package_name.module_name", "DesignJobClassName")

Source code in `nautobot_design_builder/util.py`

    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    154
    155
    156
    157
    158
    159
    160
    161
    162
    163
    164
    165
    166
    167
    168
    169
    170
    171
    172
    173
    174
    175
    176
    177
    178
    179
    180
    181
    182
    183
    184
    185
    186
    187
    188
    189
    190
    def designs_in_directory(
        path,
        package_name: str,
        local_logger=logger,
        module_name: str = None,
        reload_modules=False,
    ) -> Iterator[Tuple[str, Type["DesignJob"]]]:
        """
        Find all the designs in a directory.

        Walk the available Python modules in the given directory, and for each module,
        walk its DesignJob class members.

        Args:
            path (str): Directory to import modules from, outside of sys.path

            package_name (str): The package to which discovered modules will belong.

            local_logger (logging.Logger): The logging instance to use. This is especially useful when a
                logger includes a JobResult.

            module_name (str): Specific module name to select; if unspecified, all modules will be inspected

            reload_modules (bool): Whether to force reloading of modules even if previously loaded into Python.

        Yields:
            ("package_name.module_name", "DesignJobClassName")
        """
        # this prevents a circular import
        from nautobot_design_builder.design_job import DesignJob  # pylint: disable=import-outside-toplevel

        def is_design(obj):
            try:
                return issubclass(obj, DesignJob) and obj is not DesignJob and not inspect.isabstract(obj)
            except TypeError:
                return False

        if reload_modules:
            for key in list(sys.modules.keys()):
                if key.startswith(package_name):
                    del sys.modules[key]
        for _, discovered_module_name, _ in pkgutil.iter_modules([path]):
            if module_name and discovered_module_name != module_name:
                continue
            try:
                module = load_design_module(path, package_name, discovered_module_name)
            except Exception as ex:  # pylint:disable=broad-except
                local_logger.error(f"Unable to load module {discovered_module_name} from {path}: {ex}")
                continue
            # Get all members of the module that are DesignJob subclasses
            for design_class_name, _ in inspect.getmembers(module, is_design):
                yield f"{package_name}.{discovered_module_name}", design_class_name

### `designs_in_repository(repo, local_logger=logger, module_name=None, reload_modules=False)`

Iterate over the designs in a given git repository.

Returns:

Name | Type | Description
--- | --- | ---
`Iterator` | `Iterator[Tuple[str, Type[DesignJob]]]` | an iterator that will return tuples of package names and class names.

For the following directory structure

repo-base-dir/designs/design1.py # containing class Design1(DesignJob) repo-base-dir/designs/design2.py # containing class Design2(DesignJob)

The returned values will be yielded

("repo_base_dir.design1", "Design1") ("repo_base_dir.design2", "Design2")

Note that the package name is the full package name where the base package is a normalized version of the GitRepository slug. The name is normalized by replacing dashes and spaces with underscores and converting to lower case.

The reason designs are loaded with a base package of the git repo slug is to prevent collisions of class names and modules between design repos and also to allow for designs spanning multiple files (breaking out contexts and so forth into other files).

Source code in `nautobot_design_builder/util.py`

    205
    206
    207
    208
    209
    210
    211
    212
    213
    214
    215
    216
    217
    218
    219
    220
    221
    222
    223
    224
    225
    226
    227
    228
    229
    230
    231
    232
    233
    234
    235
    def designs_in_repository(
        repo: GitRepository, local_logger=logger, module_name: str = None, reload_modules=False
    ) -> Iterator[Tuple[str, Type["DesignJob"]]]:
        """Iterate over the designs in a given git repository.

        Returns:
            Iterator: an iterator that will return tuples of package names and class names.

        For the following directory structure:
            repo-base-dir/designs/design1.py # containing class Design1(DesignJob)
            repo-base-dir/designs/design2.py # containing class Design2(DesignJob)

        The returned values will be yielded:
            ("repo_base_dir.design1", "Design1")
            ("repo_base_dir.design2", "Design2")

        Note that the package name is the full package name where the base package is
        a normalized version of the GitRepository slug. The name is normalized by replacing
        dashes and spaces with underscores and converting to lower case.

        The reason designs are loaded with a base package of the git repo slug is to
        prevent collisions of class names and modules between design repos and also
        to allow for designs spanning multiple files (breaking out contexts and so
        forth into other files).
        """
        package_name = package_name_for_repo(repo)
        path = os.path.join(repo.filesystem_path, "designs")
        for discovered_name, class_name in designs_in_directory(
            path, package_name, local_logger=local_logger, module_name=module_name, reload_modules=reload_modules
        ):
            yield discovered_name, class_name, path

### `get_class_dir(cls)`

Function to return the directory where a given path is stored.

Returns:

Name | Type | Description
--- | --- | ---
`str` | `str` | A path to a directory

Source code in `nautobot_design_builder/util.py`

    33
    34
    35
    36
    37
    38
    39
    def get_class_dir(cls) -> str:
        """Function to return the directory where a given path is stored.

        Returns:
            str: A path to a directory
        """
        return os.path.dirname(inspect.getfile(cls))

### `get_created_and_last_updated_usernames_for_model(instance)`

Get the user who created and last updated an instance.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`instance` | `Model` | A model class instance | required

Returns:

Name | Type | Description
--- | --- | ---
`created_by` | `str` | Username of the user that created the instance
`last_updated_by` | `str` | Username of the user that last modified the instance

Source code in `nautobot_design_builder/util.py`

    337
    338
    339
    340
    341
    342
    343
    344
    345
    346
    347
    348
    349
    350
    351
    352
    353
    354
    355
    356
    357
    358
    359
    360
    361
    362
    363
    def get_created_and_last_updated_usernames_for_model(instance):
        """Get the user who created and last updated an instance.

        Args:
            instance (Model): A model class instance

        Returns:
            created_by (str): Username of the user that created the instance
            last_updated_by (str): Username of the user that last modified the instance
        """
        from nautobot.extras.choices import ObjectChangeActionChoices
        from nautobot.extras.models import ObjectChange

        object_change_records = get_changes_for_model(instance)
        created_by = None
        last_updated_by = None
        try:
            created_by_record = object_change_records.get(action=ObjectChangeActionChoices.ACTION_CREATE)
            created_by = created_by_record.user_name
        except ObjectChange.DoesNotExist:
            pass

        last_updated_by_record = object_change_records.order_by("time").last()
        if last_updated_by_record:
            last_updated_by = last_updated_by_record.user_name

        return created_by, last_updated_by

### `get_design_class(path, module_name, class_name)`

Retrieve the Python class using a filesystem path, module name and class name.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`path` | `str` | filesystem path to the package containing the module. | required
`module_name` | `str` | name of the module to load from the package. | required
`class_name` | `str` | name of the class to load from the module. | required

Returns:

Type | Description
--- | ---
`Type[DesignJob]` | Type[DesignJob]: The class.

Source code in `nautobot_design_builder/util.py`

    317
    318
    319
    320
    321
    322
    323
    324
    325
    326
    327
    328
    329
    330
    331
    332
    333
    334
    def get_design_class(path: str, module_name: str, class_name: str) -> Type["DesignJob"]:
        """Retrieve the Python class using a filesystem path, module name and class name.

        Args:
            path (str): filesystem path to the package containing the module.
            module_name (str): name of the module to load from the package.
            class_name (str): name of the class to load from the module.

        Returns:
            Type[DesignJob]: The class.
        """
        package_name, module_name = module_name.rsplit(".", 1)
        if package_name in sys.modules:
            del sys.modules[package_name]

        path = os.path.join(path)
        module = load_design_module(path, package_name, module_name)
        return getattr(module, class_name)

### `load_design_file(cls, resource)`

Reads data from a file and returns it as string.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`cls` | `type` | The class to use to determine the path to find the resource. | required
`resource` | `str` | name of the YAML design file without the path | required

Returns:

Name | Type | Description
--- | --- | ---
`str` | `str` | contents of design file as string

Source code in `nautobot_design_builder/util.py`

    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    def load_design_file(cls, resource) -> str:
        """Reads data from a file and returns it as string.

        Args:
            cls (type): The class to use to determine the path to find the resource.

            resource (str): name of the YAML design file without the path

        Returns:
            str: contents of design file as string
        """
        with open(os.path.join(get_class_dir(cls), resource), encoding="UTF-8") as file:
            return file.read()

### `load_design_module(path, package_name, module_name)`

Load module_name from the path and set the parent package to package_name.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`path` | `str` | Path to directory containing the module. | required
`package_name` | `str` | Name to give the loaded package. | required
`module_name` | `str` | Name of the module to load from the path. | required

Raises:

Type | Description
--- | ---
`ModuleNotFoundError` | If the package cannot be found (missing init.py) or the module cannot be found.

Returns:

Type | Description
--- | ---
`Type[ModuleType]` | Type[ModuleType]: The loaded module.

Source code in `nautobot_design_builder/util.py`

    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    134
    135
    136
    def load_design_module(path: str, package_name: str, module_name: str) -> Type[ModuleType]:
        """Load module_name from the path and set the parent package to package_name.

        Args:
            path (str): Path to directory containing the module.
            package_name (str): Name to give the loaded package.
            module_name (str): Name of the module to load from the path.

        Raises:
            ModuleNotFoundError: If the package cannot be found (missing __init__.py)
                or the module cannot be found.

        Returns:
            Type[ModuleType]: The loaded module.
        """
        if package_name not in sys.modules:
            load_design_package(path, package_name)

        package_path = f"{package_name}.{module_name}"

        importer = pkgutil.get_importer(path)
        spec = importer.find_spec(package_path)
        if spec:
            module = importlib.util.module_from_spec(spec)
            if spec.submodule_search_locations:
                module.__package__ = package_path
            else:
                module.__package__ = package_name
            sys.modules[package_path] = module
            spec.loader.exec_module(module)
            return module

        raise ModuleNotFoundError(f"no module named '{module_name}' at {path}")

### `load_design_package(path, package_name)`

Load the package (init.py) from the path and assign it package_name.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`path` | `str` | description | required
`package_name` | `str` | description | required

Raises:

Type | Description
--- | ---
`ModuleNotFoundError` | description

Returns:

Name | Type | Description
--- | --- | ---
`_type_` | `Type[ModuleType]` | _description_import

Source code in `nautobot_design_builder/util.py`

     71
     72
     73
     74
     75
     76
     77
     78
     79
     80
     81
     82
     83
     84
     85
     86
     87
     88
     89
     90
     91
     92
     93
     94
     95
     96
     97
     98
     99
    100
    101
    def load_design_package(path: str, package_name: str) -> Type[ModuleType]:
        """Load the package (__init__.py) from the path and assign it package_name.

        Args:
            path (str): _description_
            package_name (str): _description_

        Raises:
            ModuleNotFoundError: _description_

        Returns:
            _type_: _description_import
        """
        if DESIGN_BUILDER_ROOT_MODULE not in sys.modules:
            package_spec = ModuleSpec(name=DESIGN_BUILDER_ROOT_MODULE, loader=None)
            package = importlib.util.module_from_spec(package_spec)
            sys.modules[DESIGN_BUILDER_ROOT_MODULE] = package

        if package_name in sys.modules:
            return sys.modules[package_name]

        init_path = os.path.join(path, "__init__.py")
        try:
            package_spec = importlib.util.spec_from_file_location(package_name, init_path)
            package = importlib.util.module_from_spec(package_spec)
            sys.modules[package_name] = package
            package_spec.loader.exec_module(package)
            return package
        except FileNotFoundError:
            # pylint: disable=raise-missing-from
            raise ModuleNotFoundError(f"no module named '{package_name}' at {path}")

### `load_design_yaml(cls, resource)`

Loads data from a YAML design file.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`cls` | `type` | The class to use to determine the path to find the resource. | required
`resource` | `str` | name of the YAML design file without the path | required

Returns:

Type | Description
--- | ---
`List | Dict` | list or dict: list or dictionary containing data from YAML design files

Source code in `nautobot_design_builder/util.py`

    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    def load_design_yaml(cls, resource) -> "List | Dict":
        """Loads data from a YAML design file.

        Args:
            cls (type): The class to use to determine the path to find the resource.

            resource (str): name of the YAML design file without the path

        Returns:
            list or dict: list or dictionary containing data from YAML design files
        """
        return yaml.safe_load(load_design_file(cls, resource))

### `load_jobs(module_name=None)`

Expose designs to the Nautobot Jobs framework.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`module_name` | `str` | Module name to limit design discovery. Defaults to None. | `None`

This method is used inside a jobs module to expose designs to Nautobot. Due to a limitation with the way jobs are loaded, it is not possible for jobs to be organized in multiple files. The `load_jobs` method essentially overcomes this limitation by discovering designs and creating dynamic classes within a jobs module.

To use this method, create a jobs module like so:

    # jobs.py
    from nautobot_design_builder.util import load_jobs

    load_jobs()

If designs are stored in different modules and that module separation is desired, then a module name can be supplied to the method:

    # jobs/tenant1.py
    from nautobot_design_builder.util import load_jobs

    load_jobs(module_name="tenant1")

Source code in `nautobot_design_builder/util.py`

    255
    256
    257
    258
    259
    260
    261
    262
    263
    264
    265
    266
    267
    268
    269
    270
    271
    272
    273
    274
    275
    276
    277
    278
    279
    280
    281
    282
    283
    284
    285
    286
    287
    288
    289
    290
    291
    292
    293
    294
    295
    296
    297
    298
    299
    300
    301
    302
    303
    304
    305
    306
    307
    308
    309
    310
    311
    312
    313
    314
    def load_jobs(module_name=None):
        """Expose designs to the Nautobot Jobs framework.

        Args:
            module_name (str, optional): Module name to limit design discovery. Defaults to None.

        This method is used inside a jobs module to expose designs to Nautobot. Due
        to a limitation with the way jobs are loaded, it is not possible for jobs
        to be organized in multiple files. The `load_jobs` method essentially overcomes
        this limitation by discovering designs and creating dynamic classes within
        a jobs module.

        To use this method, create a jobs module like so:

            # jobs.py
            from nautobot_design_builder.util import load_jobs

            load_jobs()

        If designs are stored in different modules and that module separation
        is desired, then a module name can be supplied to the method:

            # jobs/tenant1.py
            from nautobot_design_builder.util import load_jobs

            load_jobs(module_name="tenant1")
        """
        frame = sys._getframe(1)  # pylint:disable=protected-access
        filename = frame.f_globals["__file__"]
        dirname = os.path.abspath(os.path.join(os.path.dirname(filename)))
        designs = {}
        is_local = dirname == settings.JOBS_ROOT
        if is_local:
            dirname = os.path.join(dirname, "..", "designs")
            package_name = f"{DESIGN_BUILDER_ROOT_MODULE}.local_designs"
            for discovered_name, class_name in designs_in_directory(
                dirname, package_name, module_name=module_name, reload_modules=True
            ):
                conditional_load_job(designs, dirname, discovered_name, class_name)
        else:
            try:
                repo_slug = os.path.basename(os.path.abspath(os.path.join(dirname, "..")))
                repo = GitRepository.objects.get(slug=repo_slug)
                for discovered_name, class_name, module_path in designs_in_repository(
                    repo, module_name=module_name, reload_modules=True
                ):
                    conditional_load_job(designs, module_path, discovered_name, class_name)
            except GitRepository.DoesNotExist:
                return

        frame.f_globals["jobs"] = []
        from nautobot.apps.jobs import register_jobs

        for class_name, cls in designs.items():
            new_cls = type(class_name, (cls,), {})
            new_cls.__module__ = frame.f_globals["__name__"]
            frame.f_globals[class_name] = new_cls
            frame.f_globals["jobs"].append(new_cls)

            register_jobs(*frame.f_globals["jobs"])

### `package_name_for_repo(repo)`

Generate the package name for a git repository.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`repo` | `GitRepository` | Git Repository containing some design builder designs | required

Returns:

Name | Type | Description
--- | --- | ---
`str` | `str` | package name, such as design_builder_designs.repo_slug

Source code in `nautobot_design_builder/util.py`

    193
    194
    195
    196
    197
    198
    199
    200
    201
    202
    def package_name_for_repo(repo: GitRepository) -> str:
        """Generate the package name for a git repository.

        Args:
            repo (GitRepository): Git Repository containing some design builder designs

        Returns:
            str: package name, such as design_builder_designs.repo_slug
        """
        return f"{DESIGN_BUILDER_ROOT_MODULE}.{repo.slug.replace('.', '_').replace('-', '_')}"