.. _release-guide:

***********************************
A guide to making a nipy release
***********************************

A guide for developers who are doing a nipy release

.. _release-tools:

Release tools
=============

There are some release utilities that come with nibabel_.  nibabel should
install these as the ``nisext`` package, and the testing stuff is understandably
in the ``testers`` module of that package.  nipy has Makefile targets for their
use.  The relevant targets are::

    make check-version-info

This installs the code from a git archive, from the repository, and for
in-place use, and runs the ``get_info()`` function to confirm that installation
is working and information parameters are set correctly.

.. _release-checklist:

Release checklist
=================

* Review the open list of `nipy issues`_.  Check whether there are
  outstanding issues that can be closed, and whether there are any issues that
  should delay the release.  Label them !

* Review and update the release notes.  Review and update the :file:`Changelog`
  file.  Get a partial list of contributors with something like::

      git log 0.2.0.. | grep '^Author' | cut -d' ' -f 2- | sort | uniq

  where ``0.2.0`` was the last release tag name.

  Then manually go over ``git shortlog 0.2.0..`` to make sure the release notes
  are as complete as possible and that every contributor was recognized.

* Use the opportunity to update the ``.mailmap`` file if there are any duplicate
  authors listed from ``git shortlog -ns``.

* Add any new authors to the ``AUTHORS`` file.  Add any new entries to the
  ``THANKS`` file.

* Check the copyright years in ``doc/conf.py`` and ``LICENSE``

* Refresh the ``README.rst`` text from the ``LONG_DESCRIPTION`` in ``info.py``
  by running ``make refresh-readme``.

  Check the output of::

    rst2html.py README.rst > ~/tmp/readme.html

  because this will be the output used by PyPI_

* Check the dependencies listed in ``nipy/info.py`` (e.g.
  ``NUMPY_MIN_VERSION``) and in ``doc/users/installation.rst``.  They should
  at least match. Do they still hold?  Make sure ``.travis.yml`` is testing
  these minimum dependencies specifically.

* Check the examples in python 2 and python 3, by running something like::

    cd ..
    ./nipy/tools/run_log_examples.py nipy/examples --log-path=~/tmp/eg_logs

  in a python 2 and python 3 virtualenv.  Review the output in (e.g.)
  ``~/tmp/eg_logs``. The output file ``summary.txt`` will have the pass file
  printout that the ``run_log_examples.py`` script puts onto stdout while
  running.  You can run the examples via the buildbot by triggering builds
  for:

  * https://nipy.bic.berkeley.edu/builders/nipy-examples-2.7
  * https://nipy.bic.berkeley.edu/builders/nipy-examples-3.5

  The matching outputs appear at:

  * https://nipy.bic.berkeley.edu/nipy-examples-2.7
  * https://nipy.bic.berkeley.edu/nipy-examples-3.5

  I use the following commands to get the output to my laptop and review with
  vim::

    PY_VER=2.7

  Then::

    BB_SSH=buildbot@nipy.bic.berkeley.edu
    scp -r ${BB_SSH}:nibotmi/public_html/nipy-examples-${PY_VER} .
    cd nipy-examples-${PY_VER}
    # Delete empty files
    find . -size 0 -exec rm {} \;
    # Review stderr; :bd to close buffer once reviewed
    vim *.stderr

  Then::

    # Review stdout
    vim *.stdout

  Finally::

    cd ..

  Likewise for::

    PY_VER=3.5

  I also did a by-eye comparison between the 2.7 and 3.4 files with::

    diff -r nipy-examples-2.7 nipy-examples-3.5 | less

* Do a final check on the `nipy buildbot`_

* If you have travis-ci_ building set up on your own fork of nipy you might
  want to push the code in its current state to a branch that will build,
  e.g::

    git branch -D pre-release-test # in case branch already exists
    git co -b pre-release-test
    git push your-github-user pre-release-test

* Make sure all the ``.c`` generated files are up to date with Cython sources
  with::

    ./tools/nicythize

Release checking - buildbots
============================

* Check all the buildbots pass
* Run the builder and review the possibly green output from
  http://nipy.bic.berkeley.edu/builders/nipy-release-checks

  This runs all of::

    make distclean
    python -m compileall .
    make sdist-tests
    make check-version-info
    make check-files

* You need to review the outputs for errors; at the moment this buildbot builder
  does not check whether these tests passed or failed.
* ``make check-version-info`` checks how the commit hash is stored in the
  installed files.  You should see something like this::

    {'sys_version': '2.6.6 (r266:84374, Aug 31 2010, 11:00:51) \n[GCC 4.0.1 (Apple Inc. build 5493)]', 'commit_source': 'archive substitution', 'np_version': '1.5.0', 'commit_hash': '25b4125', 'pkg_path': '/var/folders/jg/jgfZ12ZXHwGSFKD85xLpLk+++TI/-Tmp-/tmpGPiD3E/pylib/nipy', 'sys_executable': '/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python', 'sys_platform': 'darwin'}
    /var/folders/jg/jgfZ12ZXHwGSFKD85xLpLk+++TI/-Tmp-/tmpGPiD3E/pylib/nipy/__init__.pyc
    {'sys_version': '2.6.6 (r266:84374, Aug 31 2010, 11:00:51) \n[GCC 4.0.1 (Apple Inc. build 5493)]', 'commit_source': 'installation', 'np_version': '1.5.0', 'commit_hash': '25b4125', 'pkg_path': '/var/folders/jg/jgfZ12ZXHwGSFKD85xLpLk+++TI/-Tmp-/tmpGPiD3E/pylib/nipy', 'sys_executable': '/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python', 'sys_platform': 'darwin'}
    /Users/mb312/dev_trees/nipy/nipy/__init__.pyc
    {'sys_version': '2.6.6 (r266:84374, Aug 31 2010, 11:00:51) \n[GCC 4.0.1 (Apple Inc. build 5493)]', 'commit_source': 'repository', 'np_version': '1.5.0', 'commit_hash': '25b4125', 'pkg_path': '/Users/mb312/dev_trees/nipy/nipy', 'sys_executable': '/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python', 'sys_platform': 'darwin'}

* ``make check-files`` checks if the source distribution is picking up all the
  library and script files.  Look for output at the end about missed files, such
  as::

    Missed script files:  /Users/mb312/dev_trees/nipy/bin/nib-dicomfs, /Users/mb312/dev_trees/nipy/bin/nifti1_diagnose.py

  Fix ``setup.py`` to carry across any files that should be in the distribution.
* Check the documentation doctests pass from
  http://nipy.bic.berkeley.edu/builders/nipy-doc-builder

* Build and test the Nipy wheels.  See the `wheel builder README
  <https://github.com/MacPython/nipy-wheels>`_ for instructions.  In summary,
  clone the wheel-building repo, edit the ``.travis.yml`` and ``appveyor.yml``
  text files (if present) with the branch or commit for the release, commit
  and then push back up to github.  This will trigger a wheel build and test
  on OSX, Linux and Windows. Check the build has passed on on the Travis-CI
  interface at https://travis-ci.org/MacPython/nipy-wheels.  You'll need commit
  privileges to the ``dipy-wheels`` repo; ask Matthew Brett or on the mailing
  list if you do not have them.

* The release should now be ready.

Doing the release
=================

* The release should now be ready.

* Edit :file:`nipy/info.py` to set ``_version_extra`` to ``''``; commit.
  Then::

    make source-release

* For the wheel build / upload, follow the `wheel builder README`_
  instructions again.  Edit the ``.travis.yml`` and ``appveyor.yml`` files (if
  present) to give the release tag to build.  Check the build has passed on
  the Travis-CI interface at https://travis-ci.org/MacPython/nipy-wheels.  Now
  follow the instructions in the page above to download the built wheels to a
  local machine and upload to PyPI.

* Once everything looks good, you are ready to upload the source release to
  PyPI.  See `setuptools intro`_.  Make sure you have a file ``\$HOME/.pypirc``,
  of form::

    [distutils]
    index-servers =
        pypi

    [pypi]
    username:your.pypi.username
    password:your-password

* Sign and upload the source release to PyPI using Twine_::

    gpg --detach-sign -a dist/nipy*.tar.gz
    twine upload dist/nipy*.tar.gz*

* Tag the release with tag of form ``0.5.0``. `-s` below makes a signed tag::

    git tag -s 'Second main release' 0.5.0

* Now the version number is OK, push the docs to github pages with::

    make upload-html

* Set up maintenance / development branches

  If this is this is a full release you need to set up two branches, one for
  further substantial development (often called 'trunk') and another for
  maintenance releases.

  * Branch to maintenance::

      git co -b maint/0.5.x

    Set ``_version_extra`` back to ``.dev`` and bump ``_version_micro`` by 1.
    Thus the maintenance series will have version numbers like - say - '0.5.1.dev'
    until the next maintenance release - say '0.5.1'.  Commit. Don't forget to
    push upstream with something like::

      git push upstream maint/0.2.x --set-upstream

  * Start next development series::

      git co main-master

    then restore ``.dev`` to ``_version_extra``, and bump ``_version_minor`` by 1.
    Thus the development series ('trunk') will have a version number here of
    '0.3.0.dev' and the next full release will be '0.3.0'.

  * Merge ``-s ours`` the version number changes from the maint release, e.g::

      git merge -s ours maint/0.3.x

    This marks the version number changes commit as merged, so we can merge any
    changes we need from the maintenance branch without merge conflicts.

  If this is just a maintenance release from ``maint/0.2.x`` or similar, just
  tag and set the version number to - say - ``0.2.1.dev``.

* Push tags::

    git push --tags

* Announce to the mailing lists.

.. _setuptools intro: http://packages.python.org/an_example_pypi_project/setuptools.html
.. _twine: https://pypi.python.org/pypi/twine
.. _travis-ci: http://travis-ci.org

.. include:: ../../links_names.txt
