Using tox with Jenkins — tox 2.9.0rc2.dev63 documentation

# Using tox with Jenkins

## Using Jenkins multi-configuration jobs

The Jenkins continuous integration server allows to define “jobs” with “build steps” which can be test invocations. install `tox` on your default Python installation on each Jenkins slave, you can easily create a Jenkins multi-configuration job that will drive your tox runs from the CI-server side, using these steps:

- install the Python plugin for Jenkins under “manage jenkins”

- create a “multi-configuration” job, give it a name of your choice

- configure your repository so that Jenkins can pull it

- (optional) configure multiple nodes so that tox-runs are performed on multiple hosts

- configure `axes` by using TOXENV as an axis name and as values provide space-separated test environment names you want Jenkins/tox to execute.

- add a Python-build step with this content (see also next example):

  >     import tox

  >     os.chdir(os.getenv('WORKSPACE'))
  >     tox.cmdline() # environment is selected by ``TOXENV`` env variable

- check `Publish JUnit test result report` and enter `**/junit-*.xml` as the pattern so that Jenkins collects test results in the JUnit XML format.

The last point requires that your test command creates JunitXML files, for example with `pytest` it is done like this:

    commands = pytest --junitxml=junit-{envname}.xml

## Integrating “sphinx” documentation checks in a Jenkins job

If you are using a multi-configuration Jenkins job which collects JUnit Test results you will run into problems using the previous method of running the sphinx-build command because it will not generate JUnit results. To accommodate this issue one solution is to have `pytest` wrap the sphinx-checks and create a JUnit result file which wraps the result of calling sphinx-build. Here is an example:

1. create a `docs` environment in your `tox.ini` file like this:

>     [testenv:docs]
>     basepython = python
>     changedir = doc # or wherever you keep your sphinx-docs
>     deps = sphinx
>            py
>     commands = pytest --tb=line -v --junitxml=junit-{envname}.xml check_sphinx.py

2. create a `doc/check_sphinx.py` file like this:

>     import py
>     import subprocess

>     def test_linkcheck(tmpdir):
>         doctrees = tmpdir.join("doctrees")
>         htmldir = tmpdir.join("html")
>         subprocess.check_call(
>             ["sphinx-build", "-W", "-blinkcheck",
>               "-d", str(doctrees), ".", str(htmldir)])

>     def test_build_docs(tmpdir):
>         doctrees = tmpdir.join("doctrees")
>         htmldir = tmpdir.join("html")
>         subprocess.check_call([
>             "sphinx-build", "-W", "-bhtml",
>               "-d", str(doctrees), ".", str(htmldir)])

3. run `tox -e docs` and then you may integrate this environment along with your other environments into Jenkins.

Note that `pytest` is only installed into the docs environment and does not need to be in use or installed with any other environment.

## Access package artifacts between Jenkins jobs

In an extension to Share package artifacts between testruns you can also configure Jenkins jobs to access each others artifacts. `tox` uses the `distshare` directory to access artifacts and in a Jenkins context (detected via existence of the environment variable `HUDSON_URL`); it defaults to to `{toxworkdir}/distshare`.

This means that each workspace will have its own `distshare` directory and we need to configure Jenkins to perform artifact copying. The recommend way to do this is to install the Jenkins Copy Artifact plugin and for each job which “receives” artifacts you add a Copy artifacts from another project build step using roughly this configuration:

>     Project-name: name of the other (tox-managed) job you want the artifact from
>     Artifacts to copy: .tox/dist/*.zip   # where tox jobs create artifacts
>     Target directory: .tox/distshare     # where we want it to appear for us
>     Flatten Directories: CHECK           # create no subdir-structure

You also need to configure the “other” job to archive artifacts; This is done by checking `Archive the artifacts` and entering:

>     Files to archive: .tox/dist/*.zip

So our “other” job will create an sdist-package artifact and the “copy-artifacts” plugin will copy it to our `distshare` area. Now everything proceeds as Share package artifacts between testruns shows it.

So if you are using defaults you can re-use and debug exactly the same `tox.ini` file and make use of automatic sharing of your artifacts between runs or Jenkins jobs.

## Avoiding the “path too long” error with long shebang lines

If you are using Jenkins builds you might run into the issue that tox can not call `pip` because the so called “shebang” line is too long. There is a limit of 127 chars on some systems. Probably the best way to fix the problem is to use the new `--workdir` option which tells tox to use a specific directory for its virtualenvironments. Set it to some unique enough short path. If somebody is interested to do a PR you could add a new option to tox which uses a random directory for storing its workdir results and removes it after the tox run finishes. This could be used from CI environments where you probably anyway want to recreate everything on new runs.