vdist(1)

NAME
vdist - Create OS packages from Python projects using Docker containers

SYNOPSYS
vdist batch CONFIGURATION FILE

DESCRIPTION
vdist is a tool that lets you create OS packages from your Python applications in a clean and self contained manner. It uses virtualenv, Docker and fpm under the hood, and it uses Jinja2 to render its templates (shell scripts) for each individual target OS.

To use vdist you only need to create a configuration file to put all parameter for all packages you want to generate.

A typical vdist configuration file could be like this:

    [DEFAULT]
    app = geolocate
    version = 1.3.0
    source_git = https://github.com/dante-signal31/${app}, master
    fpm_args = --maintainer dante.signal31@gmail.com -a native --url
        https://github.com/dante-signal31/${app} --description
        "This program accepts any text and searchs inside every IP
        address. With each of those IP addresses,
        ${app} queries
        Maxmind GeoIP database to look for the city and
        country where
        IP address or URL is located. Geolocate is designed to be
        used in console with pipes and redirections along with
        applications like traceroute, nslookup, etc."
        --license BSD-3 --category net
    requirements_path = /REQUIREMENTS.txt
    runtime_deps = libssl1.0.0, dummy1.0.0
    compile_python = True
    python_version = 3.4.4
    output_folder = ./vdist
    after_install = packaging/postinst.sh
    after_remove = packaging/postuninst.sh
    [Ubuntu-package]
    profile = ubuntu-trusty
    [Centos7-package]
    profile = centos7

Running vdist with that configuration file will place generated packages at path you set in output_folder variable. If you don't use absolute path but a relative one, then reference folder is the one where you are when vdist command is called. Always set output_folder variable.

There can be many sections in every configuration file but all of them should have a [DEFAULT] section. You can name each section as you want but [DEFAULT] should always exists in your configurations because, as its name suggest, it contains default values that will apply to all of your packages unless one of the sections overrides any of the default values. Write a section for every package you want to create.

You can use tags in your configurations. In previous example ${app} tag pastes the value you set in app variable. You can cross-reference values from specific sections using tags with format S{section:variable}, but if you you don't specify a section variable is fetched from current section and probably from default one.

You can put your configuration file whatever name and extension you want.

Once you have a configuration file you can launch vdist from your console using batch mode:

    $ vdist batch configuration_file

Batch mode is the usual mode your are going to use through console but vdist offers a manual mode too. That mode does not use a configuration file but allows you to set parameters as command arguments:

    $ vdist manual --app geolocate --version 1.3.0 --source_git https://github.com/dante-signal31/geolocate,master
    --profile ubuntu-trusty --compile_python --python_version 3.4.4 --fpm_args '--maintainer dante.signal31@gmail.com
    -a native --url https://github.com/dante-signal31/geolocate --description "This program accepts any text and searchs
    inside every IP address." --license BSD-3 --category net' --requirements_path /REQUIREMENTS.txt --runtime_deps
    libssl1.0.0 dummy1.0.0 --output_folder ./dist --after_install = packaging/postinst.sh
    --after_remove = packaging/postuninst.sh

Pay attention to the point that --fpm_args argument is enclosed in single quotes.

Manual mode may be useful to dinamically set parameters through console scripts.

Whatever mode you use you can call console help anytime:
    $ vdist --help
    [...]
    $ vdist batch --help
    [...]
    $ vdist manual --help
    [...]

At "Required arguments" and "Optional arguments"sections, below in this very text, you can find a list of parameters you can set in your configuration file or through console manual parameters.

REQUIRED ARGUMENTS

First flag is for configuration files and second one for manual parameters:

app, --app
    The name of the application to build; this should also equal the project name in Git, and is used as the prefix for the filename of the resulting package.

version, --version
    The version of the application; this is used when building the OS package both in the name and in its meta information.

profile, --profile
    The name of the profile to use for this specific build; its value should be one of two things:

    •   A vdist built-in profile (for instance centos or ubuntu-trusty).

    •   A custom profile that you create yourself; see "How to customize" for instructions.

source, --source
    The argument that specifies how to get the source code to build from; the available source types are:

    •   git(uri=uri, branch=branch): this source type attempts to git clone by using the supplied arguments.

    •   directory(path=path): this source type uses a local directory to build the project from, and uses no versioning data.

    •   git_directory(path=path, branch=branch): this source type uses a git checkout in a local directory to build the project from; it checks out the supplied branch before building.

OPTIONAL ARGUMENTS

First flag is for configuration files and second one for manual parameters:

name, --name
    The name of the build; this does not do anything in the build process itself, but is used in e.g. logs; when omitted, the build name is a sanitized combination of the app, version and profile arguments.

build_deps, --build_deps
    A list of build time dependencies; these are the names of the OS packages that need to be present on the build machine before setting up and building the project.

runtime_deps, --runtime_deps
    a list of run time dependencies; these names are given to the resulting OS package as dependencies, so that they act as prerequisites when installing the final OS package.

custom_filename, --custom_filename
    Specifies a custom filename to use when generating the OS package; within this filename, references to environment variables may be used when put in between curly braces (e.g. foo-{ENV_VAR_ONE}-bar-{ENV_VAR_TWO}.deb); this is useful when for example your CI system passes values such as the build number and so on.

fpm_args, --fpm_args
    Any extra arguments that are given to fpm when the actual package is being built.

pip_args, --pip_args
    Any extra arguments that are given to pip when your pip requirements are being installed (a custom index url pointing to your private PyPI repository for example).

package_install_root, --package_install_root
    Base directory were this package is going to be installed in target system (defaults to python_basedir).

package_tmp_root, --package_tmp_root
    Temporal folder used in docker container to build your package (defaults to '/tmp').

working_dir, --working_dir
    A subdirectory under your source tree that is to be regarded as the base directory; if set, only this directory is packaged, and the pip requirements are tried to be found here. This makes sense when you have a source repository with multiple projects under it.

python_basedir, --python_basedir
    specifies one of two things: 1) where Python can be found (your company might have a prepackaged Python already installed on your custom docker container) 2) where vdist should install the compiled Python distribution on your docker container. Read vdist's various use cases to understand the nuance. Defaults to '/opt'.

compile_python, --compile_python
    Indicates whether Python should be fetched from python.org, compiled and shipped for you; defaults to True. If not True then python_basedir should point to a python distribution already installed in docker container.

python_version, --python_version
    Specifies two things: 1) if 'compile_python' is True then it means the exact python version that should be downloaded and compiled. 2) if 'compile_python' is False then only mayor version number is considered (currently 2 or 3) and latest available python distribution of that mayor version is searched (in given 'python_basedir' of your docker container) to be used. Defaults to '2.7.9'.

requirements_path, --requirements_path
    The path to your pip requirements file, relative to your project root; this defaults to /requirements.txt.

after_install, --after_install
    A script to include inside package to be run after package installation. Given path should be given relative to the root of your source project folder.

before_install, --before_install
    A script to include inside package to be run before package installation. Given path should be given relative to the root of your source project folder.

after_remove, --after_remove
    A script to include inside package to be run after package removal. Given path should be given relative to the root of your source project folder.

before_remove, --before_remove
    A script to include inside package to be run before package removal. Given path should be given relative to the root of your source project folder.

after_upgrade, --after_upgrade
    A script to include inside package to be run after package upgrade. Given path should be given relative to the root of your source project folder.

before_upgrade, --before_upgrade
    A script to include inside package to be run before package upgrade. Given path should be given relative to the root of your source project folder.

EXAMPLES

Here's a quickstart to give you an idea of how to use vdist, once you're set up. First create a configuration file like this:

    [DEFAULT]
    app = yourapp
    version = 1.0
    source_git = https://github.com/you/%(app)s, master
    compile_python = True
    python_version = 3.4.4
    requirements_path = ./requirements.txt
    build_deps = package1, package2
    runtime_deps = package3, package4
    output_folder = ./generated_packages_folder
    after_install = packaging/postinst.sh
    after_remove = packaging/postuninst.sh
    [Ubuntu-package]
    profile = ubuntu-trusty
    [Centos7-package]
    profile = centos7

Let guess that configuration file is called yourapp_vdist.cnf. Give that file to vdist launcher running next command:

    $ vdist batch yourapp_vdist.cnf

Running the above would do this:

    Set up a Docker container running Ubuntu Trusty Tahr.

    Install the OS packages listed in build_deps.

    Download and compile a python interpreter framework.

    Git clone the repository at https://github.com/you/yourapp

    checkout the branch 'master'.

    Install your application's dependencies from requirements.txt if found in the checked out branch into compiled python framework.

    If your application includes a setup.py then it is installed in compiled python framework.

    Wrap the compiled python framework and your application files in a package called yourapp-1.0.deb which includes a dependency on the OS packages listed in runtime_deps.

    Repeat sequence setting up a Docker container running Centos 7.

    In the end you'll find generated packages for ubuntu and centos 7 in generated_packages_folder.

AUTHOR
Written by Dante Signal31 <dante.signal31@gmail.com>.

REPORTING BUGS
vdist is currently in beta stage, but it should work just fine. If you find any issues, please report issues or submit pull requests via Github <https://github.com/dante-signal31/coko/issues>

COPYRIGHT
Copyright 2018 Dante Signal31 <dante.signal31@gmail.com>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

SEE ALSO
Website: <https://github.com/dante-signal31/vdist>