rdopkg(1)
=========
:doctype: manpage
:toc2:


NAME
----
rdopkg - RDO packaging tool


SYNOPSIS
--------

`rdopkg` <action> <arg>...

`rdopkg` `-c`

`rdopkg` `-h`


DESCRIPTION
-----------
**rdopkg** is a tool for automating RDO/RHOSP packaging tasks, such as
introducing patches, updating packages to new versions and submitting
packages to RDO.

Run `rdopkg -h` to get available actions.

persistence
~~~~~~~~~~~

`rdopkg` provides multi-step actions where each step is (should be) idempotent
so if something fails along the way or human interaction is required, rdopkg
drops to shell, lets you fix the problem, and then continue by running
`rdopkg --continue` (`rdopkg -c`).

The state is stored in a file named `.rdopkg.json` in the current directory. The
last stored action can be inspected (`rdopkg status`), continued
(`rdopkg --continue`) or aborted (`rdopkg --abort`). `rdopkg` will refuse to
perform a new multi-step action if a state file is present.


ACTIONS
-------

Important actions diagram
~~~~~~~~~~~~~~~~~~~~~~~~~

&nbsp;

                          +---------------------+
                         /  WHAT DO YOU NEED?  /
                        +-----+----------+----+
                                   |
                                   |
                                   |
                            build a new package
                                   |
                                   v
                          +---------------------+
                         /  what is changing?  /
                        +----+-----+-----+----+
                             |     |     |
                      +------+     |     +--------+
                      |            |              |
                 just update   introduce    update to new
                   .spec       patch(es)   upstream version
                      |            |              |
                      |            |              |
                      v            v              v
                   +-----+     +-------+    +-------------+
                   | fix |     | patch |    | new-version |
                   +-----+     +-------+    +-------------+

ACTION: fix
~~~~~~~~~~~

Apply changes to the `.spec` file.

*Flow:*

  * Bump Release, prepare a new %changelog entry header.
  * Drop to shell, let user edit the .spec file.
  * After running `rdopkg`, ensure description was added to %changelog
    and commit changes in a new commit.

*Example:*

    cd python-novaclient
    rdopkg fix
    vim python-novaclient.spec
    rdopkg -c

More use cases can be found in
link:rdpokg-feature-fix.7.html[rdopkg-feature-fix(7)].


ACTION: patch
~~~~~~~~~~~~~

Introduce new patches to the package.

This action works with the `patches` branch
(see <<patches-branch,AUTOMAGIC -> patches branch>>).

By default, `rdopkg` resets the local patches branch to the remote patches branch.
You can skip this with `-l`/`--local-patches` to directly use the local
patches branch as is.

Don't forget to `git fetch` before running the action.

After running `rdopkg patch`, the new commit will contain the changes if there are
any.

You can use `-C`/`--changelog` option to select how `rdopkg` detects new/old
patches and changelog message:

  * `detect`: detect new/old patches using commit hash/subject (default)
  * `count`: count old and new patches (doesn't work for simultaneous
     removal & addition of patches and rewriting -patches history)
  * `plain`: just use generic "- Update patches" message

Or you can use `-B`/`--no-bump` to skip Release bump and changelog generation
and only update patch files and their references in the .spec file. This is
useful when you only need to align distgit with the patches branch.

Use `--amend` to amend previous git commit with the changes and autogenerated
commit message instead of creating a new one. This is very convenient when
modifying distgit commits.

Please see <<automagic,AUTOMAGIC>> for additional information about using
magic patches_base and patches_ignore comments in your .spec file to
influcence patches generation.

*Flow:*

  * Unless `-l`/`--local-patches` was used, reset the local patches branch to
    the remote patches branch.
  * Update patch files from local `patches` branch using `git format-patch`.
  * Update .spec file with correct patch files references.
  * Unless `-B`/`--no-bump` was used, update .spec file: bump Release, create
    new %changelog entry with new patches' titles depending on
    `-C`/`--changelog` option.
  * If a `%global commit asdf1234` macro declaration is present, rewrite
    it with the current sha1 of the patches branch. (This makes the sha1
    value available during your package's build process. You can use this
    to build your program so that "mycoolprogram --version" could display
    the sha1 to users.)
  * Create new commit (or amend previous one with `-a`/`--amend`)
    with the changes using %changelog to generate commit message if available.
  * Display the diff.

*Example:*

    rdopkg patch
    rdopkg patch -lBa

More use cases can be found in
link:rdopkg-feature-patch.7.html[rdopkg-feature-patch(7)].


ACTION: update-patches
~~~~~~~~~~~~~~~~~~~~~~

An alias for:

    rdopkg patch --local-patches --no-bump

in the spirit of the ancient `update-patches.sh` script.

See **ACTION: PATCH** above.


ACTION: new-version
~~~~~~~~~~~~~~~~~~~

Update package to new upstream version.

This action works with the `patches` branch
(see <<patches-branch,AUTOMAGIC -> patches branch>>).
After a successful rebase, `rdopkg` will offer to push the
rebased patches branch.

Required `new-version` argument is a new version to rebase on, presumably
a git version tag.

Don't forget to `git fetch --all` before running the action.

You can use the `-N`/`--new-sources` or `-n`/`--no-new-sources` options to
control whether `new-version` will run `fedpkg new-sources`
(`rhpkg new-sources` on Red Hat downstream products). By default, `rdopkg`
will automatically enable this step in following scenarios:

* Fedora distgit detected from `origin` git remote (`pkgs.fedoraproject.org`)
* RH distgit detected from git branch (`rhos-*`, `rh-*`, `ceph-*`, `rhscon-*`)

Otherwise, `fedpkg new-sources` is disabled (`-n`).

After running `rdopkg new-version`, a new commit will contain the changes.

To only update `.spec` without touching patches branch, `-b`/`--bump-only` can
be used. Along with `-n`/`--no-new-sources` this enables local-only operations,
much like `rpmdev-bumpspec`:

    rdopkg -bn

To note particular bugs in the changelog, use the `-B`/`--bug` option. rdopkg
will append the supplied string to changelog in brackets. For example:

    rdopkg new-version --bug rhbz#1234,rhbz#5678

will result in following %changelog line:

    Update to 1.1.1 (rhbz#1234,rhbz#5678)

and corresponding `Resolves:` lines in commit message.


*Flow:*

  * Show changes between the previous version and the current one, especially
    modifications to `requirements.txt`.
  * Reset the local patches branch to the remote patches branch
  * Rebase the local patches branch on `$NEW_VERSION` tag.
  * Update `.spec` file: set `Version`, `Release` and `patches_base` to
    appropriate values and create a new %changelog entry.
  * Download source tarball.
  * Run `fedpkg new-sources` (`rhpkg new-sources`).
  * Update patches from the local patches branch.
  * Display the diff.

*Example:*

    cd python-novaclient
    git fetch --all
    rdopkg new-version 2.15.0
    # rebase failed, manually fix using git
    rdopkg -c

More use cases can be found in
link:rdopkg-feature-new-version.7.html[rdopkg-feature-new-version(7)].


ACTION: lint
~~~~~~~~~~~~

Run checks for errors in current distgit.

Available checks selectable with `--lint-checks`:

* `sanity`: internal rdopkg sanity checks on the .spec
* `rpmlint`: run `rpmlint` tool on the .spec
* `all`: run all available checks (default)

Available error levels selectable with `--error-level` affect the exit code:

* `E`: exit with code 23 when linting error is found (default)
* `W`: exit with code 23 when linttng error or warning is found
* `-`: only print errors/warnings, always returns 0

*Most of the time you probably want just:*

    rdopkg lint

*Example of only running rpmlint with `W` error level:*

    rdopkg lint --lint-checks rpmlint --error-level W


ACTION: clone
~~~~~~~~~~~~~

Clone an RDO package distgit and setup remotes.

`clone` uses <<rdoinfo,rdoinfo>> metadata to clone the specified RDO package
distgit and also setup relevant remotes to get you packaging quickly.

If your github username differs from your `$USER`, use `-u`/`--review-user`.

*Example:*

    rdopkg clone -u github-user python-novaclient
    cd python-novaclient
    git remote -v


ACTION: query
~~~~~~~~~~~~~

Query RDO/distro repos for available package versions.

See link:rdopkg-adv-requirements.7.html[rdopkg-adv-requirements(7)] for
complete example of `query` and other requirements management actions.

This action uses `repoquery` to discover the latest package versions available
from RDO and other repos available on a supported distibution.

See output of `rdopkg info` for supported releases and distros.

*Query specific RELEASE/DIST:*

     rdopkg query kilo/el7 openstack-nova

*Query all dists of a release and show what's happening:*

     rdopkg query -v kilo openstack-nova


ACTION: reqquery
~~~~~~~~~~~~~~~~

Query RDO/distro repos for versions defined in requirements.txt.

See link:rdopkg-adv-requirements.7.html[rdopkg-adv-requirements(7)] for a
complete example of `reqquery` and other requirements management actions.

This action essentially runs `rdopkg query` on every module/package defined in
`requirements.txt` and prints a colorful report to quickly find unmet
dependencies. It accepts the same RELEAESE/DIST filter as `rdopkg query`.

Python module names listed in `requirements.txt` are mapped to package names
using the `rdopkg.actionmods.pymod2pkg` module.

*Query `requirements.txt` from `2015.1` tag:*

    rdopkg reqquery -R 2015.1 kilo/el7

*Query `requirements.txt` file:*

    rdopkg reqquery -r path/to/requirements.txt kilo/f21

*Query `.spec` Requires (experimental):*

    rdopkg reqquery -s

*Verbosely dump query results to a file and view them:*

    rdopkg reqquery -v -d
    rdopkg reqquery -l


ACTION: reqcheck
~~~~~~~~~~~~~~~~

Inspect `requirements.txt` vs `.spec` Requires.

See link:rdopkg-adv-requirements.7.html[rdopkg-adv-requirements(7)] for
complete example of `reqcheck` and other requirements management actions.

This action parses the current `requirements.txt` from git and checks whether
they're met in the `.spec` file. A simple report is produced.

Python module names listed in `requirements.txt` are mapped to package names
using `rdopkg.actionmods.pymod2pkg` module.

Use `--spec`/`-s` option to output Requires: suitable for pasting into .spec
files. Version comparisons are hidden, whitespace is detected from .spec.

*Example:*

    rdopkg reqcheck
    rdopkg reqcheck -s


ACTION: reqdiff
~~~~~~~~~~~~~~~

Show pretty diff of `requirements.txt`.

See link:rdopkg-adv-requirements.7.html[rdopkg-adv-requirements(7)] for a
complete example of `reqdiff` and other requirements management actions.

Use this to see how requirements changed between versions.

*See diff between current and latest upstream version (automagic):*

    rdopkg reqdiff

*See diff between current and specified version:*

    rdopkg reqdiff 2015.1

*See diff between two supplied versions:*

    rdopkg reqdiff 2015.1 2015.2


ACTION: kojibuild
~~~~~~~~~~~~~~~~~

Build the package in `koji`.

*Flow:*

 * Run equivalent of `fedpkg build` using disgusting `fedpkg` python module.
 * Watch the build.

*Example:*

    rdopkg kojibuild


ACTION: amend
~~~~~~~~~~~~~

Amend last git commit with current dist-git changes and (re)generate the commit
message from %changelog.

This simple action is equivalent to running

    git commit -a --amend -m "$AUTOMAGIC_COMMIT_MESSAGE"

See <<commit-message,AUTOMAGIC -> commit message>>
for more information about the generated commit message.


ACTION: squash
~~~~~~~~~~~~~~

Squash last git commit into previous one. The commit message of the previous
commit is used.

This simple action is a shortcut for

    git reset --soft HEAD~
    git commit --amend --no-edit

This is useful for squashing commits created by lower level actions such as
`update-patches`.


ACTION: get-sources
~~~~~~~~~~~~~~~~~~~

Download package source archive.

Currently, `Source0` from `.spec` file is downloaded.


ACTION: info
~~~~~~~~~~~~

Show information about RDO packaging.

Use this command to find out about:

 * currently supported RDO OpenStack releases
 * which distros are supported for each release
 * what branch to build from
 * what build system to build in
 * supported packages
 * various repositories tied to a package
 * package maintainers

This command is a human interface to <<rdoinfo,rdoinfo>>.

*Releases/dists/branches overview:*

    rdopkg info

*Detailed information about a package:*

    rdopkg info novaclient

*Filter packages by maintainers:*

    rdopkg info maintainers:jruzicka


ACTION: info-tags-diff
~~~~~~~~~~~~~~~~~~~~~~

Show rdoinfo tag changes.

    rdopkg info-tags-diff RDOINFODIR

will show per-package new/changed tags in rdoinfo between `HEAD~..HEAD`.

For an existing or new package, a list of changed tags is returned

Example:

    $ rdopkg info-tags-diff ~/.rdopkg/rdoinfo
    openstack-changed ['newton-uc', 'newton']
    openstack-new-pkg ['under-review']

This is an interface to rdopkg.actionmods.rdoinfo:tags_diff().


ACTION: findpkg
~~~~~~~~~~~~~~~

Find and show the single best matching package in <<rdoinfo,rdoinfo>>.

This command produces the same output as `rdopkg info` but

 * smart search is performed on package name, project name and upstream URL
 * only a single matching package is shown

Use `-s`/`--strict` to disable magic substring search and only match whole
fields.

This command is a human interface to
`distroinfo.query.find_package()` function.

*Examples of usage:*

    rdopkg findpkg nova
    rdopkg findpkg -s openstack-nova
    rdopkg findpkg git://git.openstack.org/openstack/nova
    rdopkg findpkg openstack/nova
    rdopkg findpkg novacli


ACTION: conf
~~~~~~~~~~~~

Display rdopkg's local configuration.

This command prints the default configuration that ships with rdopkg out of the
box. You can override the individual settings here by using `.py` files in the
configuration directories.

Store your per-user configuration in `~/.rdopkg/conf.d/*.py`, or store
system-wide configuration in `/etc/rdopkg.d/*.py`.


ACTION: tag-patches
~~~~~~~~~~~~~~~~~~~

Tag the local -patches branch with the package's Name-Version-Release.

Since the -patches branch can change over time, including rebases, rewrites,
etc, we need a mechanism to keep historical records of what the -patches branch
looked like over time. Tagging the -patches branch for each new NVR will
maintain Git references to each snapshot of the particular patches that went
into each build.

To look at the -patches branch for an old build, you can simply "git checkout
name-version-release" for that build and get an exact representation of the Git
tree for that build.

If a previous tag exists with this name, rdopkg will exit with an error unless
you use the `--force` option to overwrite the existing tag with this name.

You can automatically push the new tag with the `--push` option. It's a good
idea to create and push the tag after every successful build.


[[automagic]]
AUTOMAGIC
---------

Instead of requiring project config files or endless lists of command line
arguments, `rdopkg` tries to guess all the necessary variables.


[[patches-branch]]
patches branch
~~~~~~~~~~~~~~

`update-patches` is a core lower level action for updating the dist-git `.spec`
file with patches from associated patches branch. `rdopkg` tries hard to
detect the patches branch automagically, it's usually `$BRANCH-patches` for
`$BRANCH` dist-git but one patches branch per multiple dist-gits is also
supported.

Best illustrated by example, the following are all valid patches branches for
`rhos-5.0-rhel-7` dist-git and they're searched in that order:

 * rhos-5.0-rhel-7-patches
 * rhos-5.0-rhel-patches
 * **rhos-5.0-patches <--- preferred for RHOSP**
 * rhos-patches

Use `rdopkg pkgenv` to check detected patches branch.

You can specify remote patches branch by `-p`/`--patches-branch` action
parameter for actions that use it, such as `patch` and `new-version`.

You may explicitly set the name of your patches remote and patches
branch in your git configuration using the
`rdopkg.<branch>.patches-remote` and `rdopkg.<branch>.patches-branch`
options.  For example, if you are working on a `dist-git` branch named
`rhel-7.4` and you want to use `rhel-7-patches` for your patches
branch, you would run:

    git config rdopkg.rhel-7.4.patches-branch rhel-7-patches


patches base
~~~~~~~~~~~~

`rdopkg` calculates the git tag on which you are applying patches from
the `Version` tag in your `.spec` file.  If your `.spec` file contains
a macro named `milestone`, the value of this macro will be appended to
the version.  That is, if your spec file has:

    Version: 2014.2.3

Then `rdopkg` will use `2014.2.3` as the base.  If instead your
`.spec` file has:

    %global milestone rc2

    Version: 2014.2.3

Then `rdopkg` will use `2014.2.3rc2` as the base.

In older versions of `rdopkg`, it was necessary to explicitly set
the patch base using a special `patches_base` comment in your spec
file.  This is now **optional** behavior (the patches base is
calculated automatically), but you can use this if you need to
override the automatic behavior.

The most common use of `patches_base` is to specify number of patches
on top of patches base (which defaults to spec Version) to skip:

    # patches_base=+2

You can set an arbitrary git revision as a patches base:

    # patches_base=1.2.3+2

You shouldn't need to modify this by hand (other than perhaps the
number of skipped patches) as `rdopkg` manages `patches_base` as
needed.


patches_ignore
~~~~~~~~~~~~~~

`update-patches` also supports filtering out patches based on matching a
regex provided by a magic #patches_ignore comment in the spec file. This is
useful, for example, in case the `patches` branch contains changes that are
related to the CI/code review infra, that are useful to keep around but don't
need to end up in the RPM.

For example, if you add the following comments in your package's .spec file:

    # patches_base=10.2.5
    # patches_ignore=DROP-IN-RPM

then rdopkg will not create .patch files for any commits that have
"DROP-IN-RPM" in the Git commit log's subject line.

Note: these lines should be directly above any Patch000X lines in your .spec
file.


[[release-bumping]]
Release bumping
~~~~~~~~~~~~~~~

`rdopkg fix` and `rdopkg patch` bump the `Release` tag in `.spec` file.

By default, last numeric only part of `Release` is bumped:

    1.1.1.a.b.c -> 1.1.2.a.b.c

You can override this using `-R`/`--release-bump-index` argument which
expects `MAJOR`/`MINOR`/`PATCH` or integer index of release part to bump,
starting at 1 from the left:

    -R 1 / -R MAJOR: 1.1.1 -> 2.1.1
    -R 2 / -R MINOR: 1.1.1 -> 1.2.1
    -R 3 / -R PATCH: 1.1.1 -> 1.1.2
    -R 4: 1.1.1.1 -> 1.1.1.2
    -R 5: 1.1.1.1.1 -> 1.1.1.1.2
    ...

DLRN `0.date.hash` and `0.1.date.hash` formats are detected and default
to bumping 2nd Release part (`-R 2`).


[[commit-message]]
commit message
~~~~~~~~~~~~~~

Commit messages created by `rdopkg` are generated from `.spec` file `Name`,
`Version` and `Release` (NVR) as well as last `%changelog` entry.

All `rdopkg` actions that modify distgit use following format:

    package-name-1.2.3-4

    Changelog:
    - Doom the World (rhbz#111111)
    - Fix Impending Doom support
    - Save the World (rhbz#222222)

    Resolves: rhbz#111111
    Resolves: rhbz#222222
    Change-Id: deadbeedeadbeedeadbeedeadbeedeadbeedeadbee

For each `(rhbz#XYZ)` mentioned in latest `%changelog` entry,
`Resolves: rhbz#XYZ` line is appended to commit message as required by RHOSP
workflow.

**protip:** To (re)generate nice commit message after modifying `.spec` file,
use `rdopkg amend` (see **ACTION: amend** above).

For example, following `%changelog` entry:

    %changelog
    * Tue Feb 11 2014 Jakub Ruzicka <jruzicka@redhat.com> 0.5.0-1
    - Update to upstream 0.5.0
    - Fix evil Bug of Doom (rhbz#123456)

will generate following commit message:

    package-name-0.5.0-1

    Changelog:
    - Update to upstream 0.5.0
    - Fix evil Bug of Doom (rhbz#123456)

    Resolves: rhbz#123456


[[rdoinfo]]
rdoinfo
~~~~~~~

`rdoinfo` is a special utility repository with RDO metadata:

https://github.com/redhat-openstack/rdoinfo

`rdopkg` uses `rdoinfo` to

 * detect release/dist from branch name
 * check valid RDO updates
 * query packages from RDO/distribution repos

and more.

You can view the `rdoinfo` metada using `rdopkg info`.


SEE ALSO
--------

link:rdopkg-adv-new-version.7.html[rdopkg-adv-new-version(7)],
link:rdopkg-adv-requirements.7.html[rdopkg-adv-requirements(7)],
link:rdopkg-feature-pkgenv.7.html[rdopkg-feature-pkgenv(7)],
link:rdopkg-feature-fix.7.html[rdopkg-feature-fix(7)],
link:rdopkg-feature-patch.7.html[rdopkg-feature-patch(7)],
link:rdopkg-feature-new-version.7.html[rdopkg-feature-new-version(7)],
link:rdopkg-feature-actions.7.html[rdopkg-feature-actions(7)]

CONTACT
-------

`rdopkg` is maintained by Jakub Ruzicka <jruzicka@redhat.com>.

Bugs are tracked as github Issues:

https://github.com/softwarefactory-project/rdopkg/issues

To report a new bug:

https://github.com/softwarefactory-project/rdopkg/issues/new
