.. _dependency_monkey:

Dependency Monkey
-----------------

This document discusses Dependency Monkey design in project Thoth. Dependency
Monkey is a tool which aims to automatically verify software stacks and
aggregate relevant observations into Thoth’s knowledge database.

Dependency Monkey use cases
===========================

Dependency Monkey was designed to automate the evaluation of certain aspects of
a software stack, such as code quality or performance. Two main aspects will be
derived from software stacks as observations. Observations are currently
consumed by Thoth for its recommendations/advises:

#. Does the given software build on a given operating system on the targeted
   hardware (if any hardware requirements such as specific CPU are provided for
   builds)?

#. What are runtime characteristics of the given software stack on targeted
   hardware (if requested a specific one, any hardware available is used to
   test the software stack). An example of runtime characteristics could be
   performance indicators or indicators such as accuracy in case of math
   libraries.

Dependency Monkey architecture design and overview
==================================================

Dependency Monkey is made out of multiple parts used in the project Thoth:

* `Management API <https://github.com/thoth-station/management-api>`_
  (optional)

* Dependency Monkey job (see `this repo sources
  <https://github.com/thoth-station/adviser>`_)

* `Amun API <https://github.com/thoth-station/amun-api>`_

* Inspection builds and inspection jobs (generated by Amun API)

Example Use of Dependency Monkey
================================

To illustrate Dependency Monkey use, let’s create a fictional use case. A first
interaction point can be ``/dependency-monkey/python`` endpoint exposed on
Thoth’s Management API service (there is currently supported only Python
ecosystem).  This endpoint accepts input from a user which is in the form of a
software stack that should be validated. The software stack input is compatible
with ``Pipfile`` as produced by `Pipenv <https://pipenv.readthedocs.io>`_ that
can assign different package indexes to packages so dependency monkey can
verify Python packages across different Python package indexes.

Besides software stack, the mentioned dependency-monkey endpoint accepts a base
image that should be used for software stack validation and verification.
Currently, supported distributions span across those using ``dnf``, ``yum`` and
``apt`` for native packages. There can be optionally passed a list of native
packages that should be installed into the given base image on build time (for
example ``Pipenv`` for testing ``Pipenv`` provided in Fedora as an rpm).
Optionally a URL or a script can be passed  and will be used to validate the
given software stack (scripts output a JSON - see `this example
<https://github.com/fridex/tf-benchmark-mock/blob/master/benchmark.py>`_\ ).
See OpenAPI specification of `Management API
<https://github.com/thoth-station/management-api>`_ to see all the possible
options.

Once the dependency-monkey endpoint is triggered with the requested input, a
“Dependency Monkey” job is created  in one of the `Thoth’s namespaces
<https://github.com/thoth-station/core#architecture-overview>`_ reserved for
Dependency Monkey runs (pre-allocated pool of resources).

Amun is a service that is capable of building software stacks considering the
base container image and native packages that are requested to be installed.
This procedure is done in the build inspection jobs triggered by Amun API in
inspection namespace with resources dedicated for inspection runs. OpenShift
builds and image streams were reused for building resulting container images.

If a user supplies a test script, an inspection job is created. The job's goal
is to verify the given software stack. The very first step done by the
inspection job is gathering hardware information about runtime hardware present
(and other node-specific characteristics, such as kernel version).  The
inspection job after that executes user provided script (e.g. a Python script)
in the given environment and gathers all the relevant information in a form of
a JSON object (e.g. standard error, standard output, script’s status code) -
these information are called inspection job reports.

These reports are gathered by a job triggered by an operator and placed onto
Ceph as JSON documents. Besides that, results are gathered, parsed and
observations are feeded into the graph database where these observations are
used by Thoth’s recommendation engine to recommend high quality software stacks
based on gathered observations.  If a user wants to perform an inspection on a
pinned down software stack without running the Dependency Monkey job to
generate software stacks, the thoth-adviser  CLI can be utilized.
`Thoth-adviser <https://github.com/thoth-station/adviser>`_ provides
submit-amun subcommand to trigger Amun API to run the inspection job and build
(or use REST exposed by OpenAPI UI to trigger inspection run directly from a
web browser).


.. image:: _static/dm.png
   :target: _static/dm.png
   :alt: Dependency Monkey
