Migrating from PostgreSQL to MySQL - Nautobot Documentation

Skip to content

# Migrating from PostgreSQL to MySQL

This document explains how to migrate the contents of an existing Nautobot PostgreSQL database to a new MySQL database.

## Export data from PostgreSQL

In your existing installation of Nautobot with PostgreSQL, run the following command to generate a JSON dump of the database contents. This may take several minutes to complete depending on the size of your database. From the Postgres host `(nautobot-postgres) $`:

    nautobot-server dumpdata \
        --exclude auth.permission \
        --format json \
        --indent 2 \
        --traceback \
        > nautobot_dump.json

Changed in version 1.5.23

- We do not recommend at this time using `--natural-primary` as this can result in inconsistent or incorrect data for data models that use GenericForeignKeys, such as `Cable`, `Note`, `ObjectChange`, and `Tag`.
- We also do not recommend at this time using `--natural-foreign` as it can potentially result in errors if any data models incorrectly implement their `natural_key()` and/or `get_by_natural_key()` API methods.
- `contenttypes` must not be excluded from the dump (it could be excluded previously due to the use of `--natural-foreign`).

Warning

Because of the different SQL dialects used by PostgreSQL and MySQL, Django's JSON database dump format is being used as the go-between for migrating your database contents from the one system to the other. This is a different case than general database backup and recovery; for best practices there, please refer to Database Backup.

This will result in a file named `nautobot_dump.json`.

## Create the MySQL database

Create the MySQL database for Nautobot, ensuring that it is utilizing the default character set (`utf8mb4`) and default collation (`utf8mb4_0900_ai_ci`) settings for case-insensitivity. It is required that MySQL will be case-insensitive. Because these encodings are the defaults, if your MySQL installation has not been modified, there will be nothing for you to do other than make sure.

In very rare cases, there may be problems when importing your data from the case-sensitive PostgreSQL database dump that will need to be handled on a case-by-case basis. Please refer to the instructions as necessary if you are unsure how to set up MySQL and create the Nautobot database.

### Confirming database encoding

To confirm that your MySQL database has the correct encoding, you may start up a database shell using `nautobot-server dbshell` and run the following command with the prompt `(nautobot-mysql) $`

    nautobot-server dbshell
    mysql> SELECT @@character_set_database, @@collation_database;
    +--------------------------+----------------------+
    | @@character_set_database | @@collation_database |
    +--------------------------+----------------------+
    | utf8mb4                  | utf8mb4_0900_ai_ci   |
    +--------------------------+----------------------+
    1 row in set (0.00 sec)

## Apply database migrations to the MySQL database

With Nautobot pointing to the MySQL database (we recommend creating a new Nautobot installation for this purpose), run `nautobot-server migrate` to create all of Nautobot's tables in the MySQL database `(nautobot-mysql) $`:

    nautobot-server migrate

## Remove auto-populated records from the MySQL database

A side effect of the `nautobot-server migrate` command is that it will populate the `ContentType`, `Job`, and `Status` tables with a number of predefined records. Depending on what Nautobot App(s) you have installed, the app(s) may also have auto-created database records of their own, such as `CustomField` or `Tag` records, in response to `nautobot-server migrate`. This is normally useful for getting started quickly with Nautobot, but since we're going to be importing data from our other database, these records will likely conflict with the records to be imported. Therefore we need to remove them, using the `nautobot-server nbshell` command in our MySQL instance of Nautobot (`(nautobot-mysql) $` shell prompt):

    nautobot-server nbshell

Enter the following Python commands into the shell:

    from django.apps import apps
    for model in apps.get_models():
        if model._meta.managed and model.objects.exists():
            print(f"Deleting objects of {model}")
            print(model.objects.all().delete())

Example output:

    ...
    # Django version 3.2.16
    # Nautobot version 2.0.0a0
    # Example Nautobot App version 1.0.0
    Python 3.8.16 (default, Mar 23 2023, 04:48:11)
    [GCC 10.2.1 20210110] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    (InteractiveConsole)
    >>> from django.apps import apps
    >>> for model in apps.get_models():
    ...     if model._meta.managed and model.objects.exists():
    ...         print(f"Deleting objects of {model}")
    ...         print(model.objects.all().delete())
    ...
    Deleting objects of <class 'django.contrib.auth.models.Permission'>
    (465, {'auth.Permission': 465})
    Deleting objects of <class 'django.contrib.contenttypes.models.ContentType'>
    (186, {'extras.CustomField_content_types': 1, 'extras.Status_content_types': 68, 'contenttypes.ContentType': 117})
    Deleting objects of <class 'nautobot.extras.models.customfields.CustomField'>
    (1, {'extras.CustomField': 1})
    Deleting objects of <class 'nautobot.extras.models.statuses.Status'>
    (20, {'extras.Status': 20})
    Deleting objects of <class 'nautobot.extras.models.jobs.Job'>
    (6, {'extras.Job': 6})

Press Control-D to exit the `nbshell` when you are finished.

## Import the database dump into MySQL

Use the `nautobot-server loaddata` command to import the database dump that you previously created. This may take several minutes to complete depending on the size of your database. This is from the MySQL host with the prompt (`(nautobot-mysql) $`):

    nautobot-server loaddata --traceback nautobot_dump.json

Assuming that the command ran to completion with no errors, you should now have a fully populated clone of your original database in MySQL.

## Rebuild cached cable path traces

Because cable path traces contain cached data which includes denormalized references to other database objects, it's possible that this cached data will be inaccurate after doing a `loaddata`. Fortunately there is a `nautobot-server` command to force rebuilding of these caches, and we recommend doing so after the import is completed:

    nautobot-server trace_paths --force --no-input