Basic functionality
===================

The big-picture purpose of astropy-helpers is to provide customization to Python's
packaging infrastructure process in ways that the Astropy Project has found to
help simplifying the developing and releasing packages. This is primarily
built around ``setup.py`` commands, as outlined below, as well as code to help
manage version numbers and better control the build process of larger packages.

Custom setup.py commands
------------------------

The main part of astropy-helpers is to provide customized setuptools commands.
For example, in a package that uses astropy-helpers, the following command
will be available::

    python setup.py build_docs

and this command is implemented in astropy-helpers. To use the custom commands
described here, you can either use the simplified method of opting in to
astropy-helpers described in :ref:`setup_all`, or if you want more control, use
the :func:`~astropy_helpers.setup_helpers.register_commands` function by
adding::

    from astropy_helpers.setup_helpers import register_commands

to your ``setup.py`` file, then doing::

    # Create a dictionary with setup command overrides. Note that this gets
    # information about the package (name and version) from the setup.cfg file.
    cmdclassd = register_commands()

This function requires that the package name and full version are set in the
``setup.cfg`` file in the ``[metadata]`` section, e.g.::

    [metadata]
    name = mypackage
    version = 0.4.dev

Then, pass ``cmdclassd`` to the ``setup`` function in ``setup.py``::

     setup(...,
           cmdclass=cmdclassd)

The commands we provide or customize are:

python setup.py test
^^^^^^^^^^^^^^^^^^^^

This command will automatically build the package, install it to a temporary
directory, and run the tests using `pytest <http://pytest.org/>`_ on this
installed version. Note that the bulk of this command is actually defined
in ``astropy.tests.command.AstropyTest``, because that allows the test
machinery to operate outside a setuptools command. This, here we
simply define the custom
setuptools command.

python setup.py sdist
^^^^^^^^^^^^^^^^^^^^^

We redefine ``sdist`` to use the version from distutils rather than from
setuptools, as the setuptools version requires duplication of information
in ``MANIFEST.in``.

python setup.py build_docs
^^^^^^^^^^^^^^^^^^^^^^^^^^

This command will automatically build the package, then run sphinx to build
the documentation. This makes development much easier because it ensures
sphinx extensions that use the package's  code to make documentation are
actually using the in-development version of the code. Sphinx itself
provides a custom setuptools command, which we
expand with the following options:

* ``-w``: set the return code to 1 if there are any warnings during the build
  process.

* ``-l``: completely clean previous builds, including files generated by
  the sphinx-automodapi package (which creates API pages for different
  functions/classes).

* ``-n``: disable the intersphinx option.

* ``-o``: open the documentation in a browser if a build finishes successfully.

In addition, ``build_docs`` will automatically download and temporarily install
sphinx-astropy (which is a meta-package that
provides standardized configuration and documentation dependencies for astropy
packages) if it isn't already installed. Temporary installation means that the
package will be installed into an ``.eggs`` directory in the current working
directory, and it will only be available for the duration of the call to
``build_docs``.

python setup.py build_ext
^^^^^^^^^^^^^^^^^^^^^^^^^

This is also used when running ``build`` or ``install``. We add several features
compared to the default ``build_ext`` command:

* For packages with C/Cython extensions, we create a
  ``packagename.compiler_version`` submodule and a
  ``packagename.cython_version`` submodule that contain information about the
  compilers used.

* Packages that need to build C extensions using the Numpy C API, we allow
  those packages to define the include path as ``'numpy'`` as opposed to having
  to import Numpy and call ``get_include``. The goal is to solve the issue that
  if one has to import Numpy to define extensions, then Numpy has to be
  installed/available before the package is installed, which means that one
  needs to install Numpy in a separate installation step.

* We detect broken compilers and replace them with other compilers on-the-fly
  unless the compiler is explicitly specified with the ``CC`` environment
  variable.

* If Cython is not installed, then we automatically check for generated C files
  (which are normally present in the stable releases) and give a nice error
  if these are not found.

Version helpers
---------------

Another piece of functionality we provide in astropy-helpers is the ability
to generate a ``packagename.version`` module that includes functions that
automatically set the version string for developer versions, to e.g.
``3.2.dev22213`` so that each developer version has a unique number (although
note that branches an equal number of commits away from the master branch will
share the same version number).

In addition, this module contains variables such as ``major``, ``minor``, and
``bugfix``, as well as ``version_info`` (a tuple of the previous three values),
a ``release`` flag that indicates whether we are using a stable release, and
several other complementary variables.  To use the version helpers, you can
either use the simplified method of opting in to astropy-helpers described in
:ref:`setup_all`, or if you want more control, use the
:func:`~astropy_helpers.version_helpers.generate_version_py`, import::

    from astropy_helpers.version_helpers import generate_version_py

in your ``setup.py`` file, and call::

    # Freeze build information in version.py. Note that this gets information
    # about the package (name and version) from the setup.cfg file.
    version = generate_version_py()

The ``version`` variable will be set to the version number of your package
including any developer suffix. Note that this requires that the package name
and version are set in the ``setup.cfg`` file in the ``[metadata]`` section,
e.g.::

    [metadata]
    name = mypackage
    version = 0.4.dev

Then, pass ``version`` to the ``setup`` function in ``setup.py``::

     setup(...,
           version=version)

Note that if you want to be able to generate developer versions such as
``3.2.dev22213`` without having to use the ``generate_version_py`` machinery,
you can instead just import :func:`~astropy_helpers.git_helpers.get_git_devstr`::

    from astropy_helpers.git_helpers import get_git_devstr

and you will then be able to use e.g.::

    version += get_git_devstr()

to add the developer suffix to the version string.

Collecting package information
------------------------------

The ``setup`` function from setuptools can take a number of options that indicate
for example what extensions to build, and what package data to include. However,
for large packages this can become cumbersome. We therefore provide a mechanism
for defining extensions and package data inside individual sub-packages. To do
this, you can create ``setup_package.py`` files anywhere in your package, and
these files can include one or more of the following functions:

* ``get_package_data``:
    This function, if defined, should return a dictionary mapping the name of
    the subpackage(s) that need package data to a list of data file paths
    (possibly including wildcards) relative to the path of the package's source
    code.  e.g. if the source distribution has a needed data file
    ``astropy/wcs/tests/data/3d_cd.hdr``, this function should return
    ``{'astropy.wcs.tests':['data/3d_cd.hdr']}``. See the ``package_data``
    option of the  :func:`distutils.core.setup` function.

    It is recommended that all such data be in a directory named ``data`` inside
    the package within which it is supposed to be used.  This package data
    should be accessed via the ``astropy.utils.data.get_pkg_data_filename`` and
    ``astropy.utils.data.get_pkg_data_fileobj`` functions.

* ``get_extensions``:
    This provides information for building C or Cython extensions. If defined,
    it should return a list of ``distutils.core.Extension`` objects.

* ``get_build_options``:
    This function allows a package to add extra build options.  It
    should return a list of tuples, where each element has:

    - *name*: The name of the option as it would appear on the
      commandline or in the ``setup.cfg`` file.

    - *doc*: A short doc string for the option, displayed by
      ``setup.py build --help``.

    - *is_bool* (optional): When `True`, the option is a boolean
      option and doesn't have an associated value.

    Once an option has been added, its value can be looked up using
    ``astropy_helpers.setup_helpers.get_distutils_build_option``.

* ``get_external_libraries``:
    This function declares that the package uses libraries that are
    included in the astropy distribution that may also be distributed
    elsewhere on the users system.  It should return a list of library
    names.  For each library, a new build option is created,
    ``'--use-system-X'`` which allows the user to request to use the
    system's copy of the library.  The package would typically call
    ``astropy_helpers.setup_helpers.use_system_library`` from its
    ``get_extensions`` function to determine if the package should use
    the system library or the included one.

With these files in place, you can either use the simplified method of opting in
to astropy-helpers described in :ref:`setup_all`, or if you want more control,
use theyou can then make use of the
:func:`~astropy_helpers.setup_helpers.get_package_info`  function in your
``setup.py`` file with::

    from astropy_helpers.setup_helpers import get_package_info

    ...

    package_info = get_package_info()

    ...

    setup(..., **package_info)


.. _setup_all:

Opting in to all basic functionality
------------------------------------

If you are happy to opt in to all the functionality described on this page, you
can make use of the :func:`~astropy_helpers.setup_helpers.setup` function from
:mod:`astropy_helpers.setup_helpers` which wraps the function of the same name
from setuptools and automatically runs
:func:`~astropy_helpers.setup_helpers.register_commands`,
:func:`~astropy_helpers.version_helpers.generate_version_py`, and
:func:`~astropy_helpers.setup_helpers.get_package_info`. If you want to do this,
make sure the package name and version number are defined in ``setup.cfg``::

    [metadata]
    name = mypackage
    version = 0.4.dev

then use the :func:`~astropy_helpers.setup_helpers.setup` function from
astropy-helpers in your ``setup.py`` file as follows::

    import ah_bootstrap
    from astropy_helpers.setup_helpers import setup
    setup()

We recommend that you also include a comment along the following lines in your
``setup.py`` file::

    # The configuration for the package, including the name, version, and other
    # information are set in the setup.cfg file.
