.. _audit-trail:

Audit trail
===========

For every built artifact Bob records all involved sources and build steps that
lead to a particular artifact. The recorded information contains at least, but
not limited to, the following records:

* state of the recipes,
* recipe name, package path,
* build host/time,
* environment,
* dependencies (with their respective audit trail),
* state of SCMs (e.g. commit id, dirty, ...).

The information in the audit trail records may be extended with additional
information in the future. An application that parses the audit trail should
ignore unknown fields for future compatibility.

Storage format
--------------

Audit trails are stored as gzip compressed JSON documents. For local builds the
audit trail is stored as ``audit.json.gz`` file next to the workspace. Jenkins
builds only rely on binary artifacts where the same file is stored in the
compressed tar file in a ``meta/`` directory. The general structure of an audit
trail looks like the following::

    {
        "artifact" : {
            // audit record
        },
        "references" : [
            {
                // audit record
            },
            ...
        ]
    }

The audit information about the involved artifact is stored under the
``artifact`` key. Any audit records about the dependencies that were used to
create the artifact are included in the list under the ``references`` key. This
includes all transitive records too. A correct audit trail must include the
full transitive information to be accepted by Bob.

Records
-------

The following sections describe the various keys and their semantics that can
be found in an audit record.

Example of a single audit record::

    {
      "artifact-id" : "c1cd9616caa783fcd8ef9b170cd968ccb306a727",
      "variant-id" : "f5aa3695a2d6f0e70af2ffaf43bb461a428e6fba",
      "build-id" : "a95ce8e3e30b7535751cefea942941c31a8ad1aa",
      "result-hash" : "7710368d8165f1c780fb9f33b34415ab76a618c0",
      "env" : "declare -- BASH=\"/bin/bash\"\ndeclare -r BASHOPTS=...",
      "metaEnv" : {
         "VERSION" : "1.2.3",
         "LICENSE" : "GPLv2"
      },
      "scms" : [],
      "dependencies" : {
         "args" : [
            "c5b2a8231156f43728af34f3a2dcb731ade2f76a"
         ]
      },
      "meta" : {
         "language" : "bash",
         "recipe" : "root",
         "step" : "dist",
         "bob" : "0.12.1",
         "package" : "root"
      },
      "build" : {
         "date" : "2019-12-02T13:19:34.193136+00:00",
         "machine" : "x86_64",
         "nodename" : "kloetzke",
         "os-release" : "PRETTY_NAME=\"Debian GNU/Linux 10 (buster)\"\nNAME=...",
         "release" : "4.19.0-6-amd64",
         "sysname" : "Linux",
         "version" : "#1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11)"
      }
    }

Basic information
~~~~~~~~~~~~~~~~~

``artifact-id``
    Hexadecimal number that identifies a particular artifact. This is also the
    primary key for audit records.

``variant-id``
    The Variant-Id as described in :ref:`concepts-implicit-versioning`.

``build-id``
    The Build-Id as described in :ref:`concepts-implicit-versioning`.

``result-hash``
    A hash sum across the content of the workspace after the artifact was
    built.

``env``
    Dump of the bash environment as created by ``declare -p``. See
    `bash declare`_. For PowerShell recipes it is a JSON string that contain
    all internal variables and environment variables as dictionaries. Use
    the ``meta.language`` key to determine the used scripting language.

``metaEnv``
    This is a dictionary of all :ref:`configuration-recipes-metaenv` variables
    of the package. They are included in the audit trail regardless of their
    actual usage.

.. _bash declare: https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#index-declare

Recipes
~~~~~~~

If Bob recognizes that the recipes are managed in a supported SCM (currently
git or svn) there will be a ``recipes`` key in the audit record. The format of
the object under this key is described in :ref:`audit-trail-scms`.


Dependencies
~~~~~~~~~~~~

Each step can have any number of dependencies. They will be recorded under a
``dependencies`` key. The other step is referenced by the Artifact-Id and their
audit record will be found in the ``references`` list of the audit trail. There
are three types of dependencies to other steps that each have their different
representation in audit record:

``arguments``
    Ordered list of all dependencies whose result was input to this step. They
    correspond to the ``$1`` to ``$n`` arguments of the script that was
    executed.

``tools``
    Object that maps all available tools by their name to the Artifact-Id.

``sandbox``
    Used sandbox during execution.

Example::

    "dependencies" : {
        "args" : [
            "b0a6632c6e7677220e46e4ae9c528efb949137c6"
        ],
        "tools" : {
            "toolchain" : "0b1c5e3489bed347ccf8e0e1e12dc70c92b09472"
        },
        "sandbox" : "3473b28df3891046618420428b530418ce006ad9"
    }

.. _audit-trail-scms:

SCMs
~~~~

All SCMs are recorded after the checkout step was run. The audit record will
contain a list of objects under the ``scms`` key. Each object has at least a
``type`` key that identifies the kind ob SCM and a ``dir`` key for the relative
directory (or file) that was managed by the SCM in the workspace.

See the following list for the additional information that each SCM adds to the
record:

git
    The git SCM records all remotes, the current commit that HEAD points to and
    if the tree is dirty. The output of ``git describe`` is also recorded.

    Example::

        {
            "commit": "6e986014563b70ecd867fb6a6e1adeb408f63dd6",
            "description": "v0.11.0-59-g6e98601-dirty",
            "dir": ".",
            "dirty": true
            "remotes": {
                "origin": "git@github.com:BobBuildTool/bob.git"
            },
            "type": "git",
        }

svn
    Example::

        {
            "dir" : ".",
            "dirty" : false,
            "repository" : {
                "root" : "http://svn.haiku-os.org/oldhaiku",
                "uuid" : "a95241bf-73f2-0310-859d-f6bbb57e9c96",
            },
            "revision" : 43238,
            "type" : "svn",
            "url" : "http://svn.haiku-os.org/oldhaiku/haiku/",
        }

url
    Example::

        {
            "digest" : {
                "algorithm" : "sha1",
                "value" : "697b7c87c73eb53bf80e19b65a4ac245214d530c" 
            },
            "dir" : "author.txt",
            "type" : "url",
            "url" : "https://example.test/author.txt",
        }


Meta data
~~~~~~~~~

There can be any number of key-value meta data pairs. They will be contained
under the ``meta`` key and typically hold at least the following information:

``bob``
    Bob version string.

``language``
   The scripting language that was used to create the artifact. Can be ``bash``
   or ``PowerShell``. If missing it must be interpreted as ``bash``. Use this to
   correctly parse the ``env`` string.

``package``
    Package path of the artifact that was built. Note that there might be
    multiple packages that produce the same result. Only one will be built by
    Bob without recording all possible package paths here.

``recipe``
    Name of the recipe that declared the package.

``step``
    The executed step for this audit record. Can be ``src``, ``build`` or
    ``dist``.

Example::

    "meta" : {
        "bob" : "0.11.0-56-g9b3d2c6-dirty",
        "package" : "root/lib"
        "recipe" : "lib",
        "step" : "src",
    },

Build data
~~~~~~~~~~

The build data describes when and where the artifact has been built. It can be
found under the ``build`` key and contains the following fields:

``date``
    The date and time of the build. This is stored as UTC time and formatted in
    ISO 8601 format with full precision.

``machine``
    The hardware identifier as returned by the uname system call. This is
    typically the processor architecture of the host.

``nodename``
    The host name.

``os-release``
    This optional field holds the content of ``/etc/os-release``, if existing.
    If the file does not exist or cannot be read then this field will not be
    present.

``release``
    The operating system release.

``sysname``
    The operating system name (e.g. "Linux").

``version``
    The operating system version.

.. attention::
   The information of the ``machine``, ``release``, ``sysname``, ``version``
   and possibly ``nodename`` fields show the host in case of container builds,
   e.g. when running in a docker container. Be careful when relying on this
   information. The ``os-release`` field, if present, is more reliable in this
   case.

