Metadata-Version: 2.1
Name: alpy
Version: 0.23.0
Summary: Library for testing network virtual appliances using Docker
Home-page: https://gitlab.com/abogdanenko/alpy
Author: Alexey Bogdanenko
Author-email: alexey@bogdanenko.com
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Classifier: Operating System :: POSIX :: Linux
Description-Content-Type: text/x-rst
Requires-Dist: docker
Requires-Dist: pexpect
Requires-Dist: qmp

####
Alpy
####

*Test network virtual appliance using Docker containers*

.. contents::
   :backlinks: none

General information
===================

Description
-----------

This project is a Python library for testing network virtual appliances.

Author
------

Alexey Bogdanenko

License
-------

Alpy is licensed under ``SPDX-License-Identifier: GPL-3.0-or-later``. See
``COPYING`` for more details.

Features
========

The simplest docker to QEMU networking connection
-------------------------------------------------

Nothing in the middle. No bridges, no veth pairs, no NAT etc.

Each layer 2 frame emitted is delivered unmodified, reliably.

Reliable packet capture
-----------------------

Each frame is captured reliably thanks to the QEMU *filter-dump* feature.

First-class Docker container support
------------------------------------

Alpy follows and encourages single process per container design.

Logging
-------

Test logs are easy to configure and customize. Alpy consistently uses Python
*logging* module.

Alpy collects serial console log in binary as well as text (escaped) form.

No trash left behind
--------------------

Alpy cleans up after itself:

- processes stopped with error codes and logs collected,
- files, directories unmounted,
- temporary files removed,
- sockets closed,
- interfaces removed...

... reliably.

No root required
----------------

Run as a regular user.

Documentation
=============

General
-------

Alpy manages containers via Docker Python API.

Alpy interacts with QEMU using Python API of the QEMU Monitor Protocol (QMP).
QMP is a JSON-based protocol that allows applications to communicate with a QEMU
instance. For information about QMP, see `qmp package page on PyPI`__.

__ https://pypi.org/project/qmp/

Alpy gives user Pexpect object to interact with a serial console. The Pexpect
object is configured to log console input and output via the standard *logging*
module.

API documentation
-----------------

The documentation is published on GitLab Pages of your GitLab project (if GitLab
Pages is enabled on your GitLab instance). For example, upstream project
documentation lives at https://abogdanenko.gitlab.io/alpy.

Alpy API documentation is generated using Sphinx__. To generate HTML API
documentation locally, install `Sphinx package`__ and run the following
command::

   PYTHONPATH=. sphinx-build docs public

To view the generated documentation, open ``public/index.html`` in a browser.

__ https://www.sphinx-doc.org/
__ https://pypi.org/project/Sphinx/

FAQ
---

How do I watch serial console?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Use *tail*::

   tail --follow name --retry console.log

The same command, but shorter::

   tail -F console.log

How do I watch traffic on an interface?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Use tcpdump::

   tail --bytes +0 --follow name --retry link0.pcap | tcpdump -n -r -

The same command, but shorter::

   tail -Fc +0 link0.pcap | tcpdump -nr-

Can I use Wireshark to watch traffic on an interface?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Yes, you can::

   tail --bytes +0 --follow name --retry link0.pcap | wireshark -k -i -

The same command, but shorter::

   tail -Fc +0 link0.pcap | wireshark -ki-

How do I debug my program?
^^^^^^^^^^^^^^^^^^^^^^^^^^

Use `The Python Debugger <https://docs.python.org/3/library/pdb.html>`_.

How do I enter node network namespace?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

#. Get node pid::

      docker inspect --format '{{.State.Pid}}' node0

#. Jump into node namespace using that pid::

      nsenter --net --target "$pid"

One-liner::

   nsenter --net --target "$(docker inspect --format '{{.State.Pid}}' node0)"

Network design
--------------

The appliance being tested is referred to as a *device under test* or *DUT*.

The DUT communicates with containers attached to each of its network links.

Guest network adapters are connected to the host via tap devices (Figure 1)::

   +-----QEMU hypervisor------+
   |                          |   +-------------+
   | +-----Guest OS-----+     |   |             |
   | |                  |     |   |  docker     |
   | | +--------------+ |     |   |  container  |
   | | |              | |     |   |  network    |
   | | |  NIC driver  | |     |   |  namespace  |
   | | |              | |     |   |             |
   | +------------------+     |   |   +-----+   |
   |   |              |       |   |   |     |   |
   |   | NIC hardware +---+-----------+ tap |   |
   |   |              |   |   |   |   |     |   |
   |   +--------------+   |   |   |   +-----+   |
   |                      |   |   |             |
   +--------------------------+   +-------------+
                          |
                          |
                          v
                    +-----------+
                    |           |
                    | pcap file |
                    |           |
                    +-----------+

*Figure 1. Network link between QEMU guest and a docker container.*

Each tap device lives in its network namespace. This namespace belongs to a
dedicated container - a *node*. The node's purpose is to keep the namespace
alive during the lifetime of a test.

For an application to be able to communicate with the DUT the application is
containerized. The application container must be created in a special way: it
must share network namespace with one of the nodes.

Figure 2 shows an example where application containers *app0* and *app1* share
network namespace with node container *node0*. Application container *app2*
shares another network namespace with *node2*.

This sharing is supported by Docker. All we have to do is to create the
application container with the ``--network=container:NODE_NAME`` Docker option.
For example, if we want to send traffic to the DUT via its first link, we create
a traffic generator container with Docker option ``--network=container:node0``.

::

   +----QEMU---+   +------shared network namespace-----+
   |           |   |                                   |
   |           |   |    eth0                           |
   |   +---+   |   |   +---+   +-----+ +----+ +----+   |
   |   |NIC+-----------+tap|   |node0| |app0| |app1|   |
   |   +---+   |   |   +---+   +-----+ +----+ +----+   |
   |           |   |                                   |
   |           |   +-----------------------------------+
   |           |
   |           |
   |           |
   |           |   +------shared network namespace-----+
   |           |   |                                   |
   |           |   |    eth0                           |
   |   +---+   |   |   +---+   +-----+                 |
   |   |NIC+-----------+tap|   |node1|                 |
   |   +---+   |   |   +---+   +-----+                 |
   |           |   |                                   |
   |           |   +-----------------------------------+
   |           |
   |           |
   |           |
   |           |   +------shared network namespace-----+
   |           |   |                                   |
   |           |   |    eth0                           |
   |   +---+   |   |   +---+   +-----+ +----+          |
   |   |NIC+-----------+tap|   |node2| |app2|          |
   |   +---+   |   |   +---+   +-----+ +----+          |
   |           |   |                                   |
   +-----------+   +-----------------------------------+

*Figure 2. Application containers attached to the DUT links.*

Rabbit
------

The alpy library repository includes scripts and modules to build a simple
appliance called Rabbit. Rabbit is Alpine Linux with a few packages
pre-installed. Having this simple DUT allows to demonstrate the library
features.

The tests for the Rabbit device share a lot of code so the code is organized as
a library. The library is called *carrot*.

Testing the alpy library
========================

A note about GitLab Container Registry
--------------------------------------

Many CI jobs use one of the custom images built on the "build-docker-images"
stage. The images are stored in the GitLab Container Registry.

The images are pulled from locations specified by GitLab variables. By default,
the variables point to the registry of the current GitLab project.

If you forked this project and GitLab Container Registry is disabled in your
project, override the variables on a project level so that the images are pulled
from some other registry.

For example, set ``IMAGE_ALPINE=registry.gitlab.com/abogdanenko/alpy/alpine``.

Related projects
================

- `Containernet <https://containernet.github.io/>`_
- `Kathará <http://www.kathara.org/>`_
- `Netkit <http://wiki.netkit.org/index.php/Main_Page>`_
- `GNS3 <https://www.gns3.com/>`_
- `Eve-NG <https://www.eve-ng.net/>`_


