# Fluctuate

[![pipeline status](https://gitlab.com/munipal-oss/fluctuate/badges/master/pipeline.svg)](https://gitlab.com/munipal-oss/fluctuate/-/commits/master)
[![coverage report](https://gitlab.com/munipal-oss/fluctuate/badges/master/coverage.svg)](https://gitlab.com/munipal-oss/fluctuate/-/commits/master)
[![Latest Release](https://gitlab.com/munipal-oss/fluctuate/-/badges/release.svg)](https://gitlab.com/munipal-oss/fluctuate/-/releases)

Fluctuate is a simple migration utility for [Fauna DB] written in [Python]. It doesn't
aim to be as full featured as something like [fauna-schema-migrate], but aims to provide
a foundation to set up your Fauna dabatases as code.

# Table of contents

[[_TOC_]]

# Installation

Install with the usual [Python] package manager. Here's an example with [pip]:

```bash
pip install fluctuate
```

Ensure it installed correctly by running:

```bash
fluctuate --version
```

If you would like to use the [AWS SecretsManager] integration for retrieval of the fauna
secret key, you will need to install the `aws_secrets_manager` extra. Here's an example
with [pip]:

```bash
pip install fluctuate[aws_secrets_manager]
```

# Creating migrations

Fluctuate will automatically search for migrations when invoked. It looks for a Python
module named "fluctuate_migrations" that has a module level attribute called
"migrations" somewhere on or under the current working directory.

To define your projects migrations, create a file named `fluctuate_migrations.py` in a
subfolder of your project along with an `__init__.py` within that subfolder. Within
`fluctuate_migrations.py` you need to create a tuple or list of `Migration`s named
"migrations" that defines your migrations. An example from our test suite is below:

```python
# faunadb_migrations.py
from faunadb import query

from fluctuate.migrations import Migration

migrations = (
    Migration(
        name="test_migration_1",
        namespace="cli_test",
        migration=query.create_collection(
            {"name": "test_collection_1", "history_days": 0}
        ),
        reverse_migration=query.delete(query.collection("test_collection_1")),
    ),
    Migration(
        name="test_migration_2",
        namespace="cli_test",
        migration=query.create_collection(
            {"name": "test_collection_2", "history_days": 0}
        ),
        reverse_migration=query.delete(query.collection("test_collection_2")),
    ),
    Migration(
        name="test_migration_3",
        namespace="cli_test",
        migration=query.create_database({"name": "test_database_1"}),
        reverse_migration=query.delete(query.database("test_database_1")),
    ),
    Migration(
        name="test_migration_4",
        namespace="cli_test",
        migration=query.create_collection(
            {"name": "test_collection_3", "history_days": 0}
        ),
        reverse_migration=query.delete(query.collection("test_collection_3")),
        child_database="test_database_1",
    ),
    Migration(
        name="test_migration_5",
        namespace="cli_test",
        migration=query.create_database({"name": "test_database_2"}),
        reverse_migration=query.delete(query.database("test_database_2")),
        child_database="test_database_1",
    ),
    Migration(
        name="test_migration_6",
        namespace="cli_test",
        migration=query.create_collection(
            {"name": "test_collection_4", "history_days": 0}
        ),
        reverse_migration=query.delete(query.collection("test_collection_4")),
        child_database="test_database_1/test_database_2",
    ),
)
```

Let's walk through this example. First a bit about migrations. Each `Migration` has 5
fields, the first 4 of which are required:

1. name: This is the migration name, and needs to be unique within the migration's
   namespace
2. namespace: This is the namespace under which the migration resides. You should
   uniquely namespace your application's migrations so as not to clash with any other
   application's migrations that may also be applied to your database.
3. migration: This is the FQL query used to create your schema documents, be it
   collections, indexes, UDFs, or otherwise.
4. reverse_migration: This is the FQL query that will be used to undo whatever the FQL
   in the migration did, so it should be the opposite operation.
5. (OPTIONAL) child_database: This defines the child database to which this migration
   should be applied. Fluctuate will attempt to log into this database and apply the
   migrations there. Nested child databases should be specified in the same format used
   for [scoped keys]. If no child database is defined, the migrations apply to the
   database the current key in use provides access to.

In the example above, there are a total of 6 migrations to be preformed. The first 3
migrations apply to the top level database, which is the database the key used to run
the migration has access to. Then the next 2 migrations are applied to "test_database_1"
which is created by migration number 3. Finally, the 6th and final migration is applied
to "test_database_2" which was created in the 5th migration.

In this example we applied each schema document in a separate migration, but as the
`migration` field can be any arbitrary FQL, all schema documents could be created in one
migration. This tool does not impose an opinion on how to lay out your migrations.

Each migration is applied in a single transaction using [Do] to write both the migration
and the record that keeps track of the applied migrations. You need to be aware of the
[caveats of schema documents] when referring to them within the same migration. There is
also the constraint of not being able to use any schema documents created within the
same transaction.

# Applying migrations

The `fluctuate migrate` command is used to apply all unapplied migrations. As migrations
are applied to your databases, their application is recorded in a separate collection.
This is referred to when applying migrations to ensure the same migration is not applied
twice. Migrations are applied in the order they are defined in the migrations tuple or
list.

Each migration, and the recording of the application of that migration, are applied in a
single transaction using [Do]. If a migration fails, the transaction rolls back and no
changes from that migration will be applied. Any migrations that were already
successfully applied will remain applied. A unique index ensures that the same migration
in the same namespace can only be applied once.

This tool will attempt to create a collection named "fluctuate_migrations" and an index
named "fluctuate_migrations_unique_name_and_namespace" on all databases this is used on.
If a collection and/or index with these names already exist, running this tool will
fail.

# Unapplying migrations

The `fluctuate unmigrate` command is used to unapply migrations. This is a destructive
operation so it should be run with care. A target migration to unapply up to can be
provided, otherwise all applied migrations will be unapplied. Migrations are unapplied
in the reverse order from how they are defined in the migrations tuple or list.

Using the example above, if the following is run:

```bash
fluctuate unmigrate --key some_key --target test_cli.test_migration_5
```

Then the reverse_migration for test_cli.test_migration_6 is run, removing
test_collection_4 from test_database_2 and then the reverse migration for
test_cli.test_migration_5 is run deleting test_database_2 from test_database_1.

Unapplying a migration, and the recording of unapplying that migration, is done in a
single transaction using [Do]. If a reverse migration fails, the transaction rolls back
and no changes from that migration will be reversed. Any migrations that were already
successfully unapplied will remain unapplied.

# Specifying Fauna DB credentials

Both commands above require access to a FaunaDB database in order to operate. A key can
be specified directly when invoking the command using `--key` or the key can be fetched
from [AWS SecretsManager] by specifying `--secret-arn` with the [ARN] of the secret to
read. Usage of `--secret-arn` requires installation of the `aws_secrets_manager` extra
and configuration of AWS credentials. See the command help message for more details.

Both of these arguments can be specified with the environment variables FLUCTUATE_KEY
and FLUCTUATE_SECRET_ARN, respectively.

Fluctuate supports the usage of [scoped keys] as credentials, but the scoped key must
have sufficient access to perform the migration operations or migrating will fail.

When applying migrations to a child database, Fluctuate creates a scoped key from the
key provided to the command. By default this scoped key uses the [admin role]. However,
if the key provided to the command is itself a scoped key, the role of that scoped key
will be used instead.

[Fauna DB]: https://docs.fauna.com/fauna/current/
[Python]: https://www.python.org/
[fauna-schema-migrate]: https://github.com/fauna-labs/fauna-schema-migrate
[pip]: https://pip.pypa.io/en/stable/
[scoped keys]: https://docs.fauna.com/fauna/current/security/keys?lang=python#scoped-keys
[Do]: https://docs.fauna.com/fauna/current/api/fql/functions/do?lang=python
[caveats of schema documents]: https://forums.fauna.com/t/do-and-creation-of-schema-documents/3418/12
[AWS SecretsManager]: https://docs.aws.amazon.com/secretsmanager/
[ARN]: https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
[admin role]: https://docs.fauna.com/fauna/current/security/keys?lang=python#admin-role
