.. _dev:

Developer Guidelines
=======================

Everyone contributing code to TOAST should aim to follow these guidelines in order to
keep the code consistent.  Whenever you make changes and before opening a pull request,
run the script ``src/format_source.sh`` to apply the standard formatting rules to the
Python and C++ code.  If you use an editor with some other automatic code formatting,
you should disable it unless you can configure it identically to the action of this
script.


Python Code
-------------------

We aim to follow PEP8 style guidelines whenever possible.  There are some reasonable
exceptions to this.  In particular, import statements might not always be placed at the
top of the code if:

- The import is an optional feature that has large performance impacts and which is only used infrequently.  For example matplotlib, astropy, etc.

- Some initialization code is needed prior to the import statement.  For example, setting the matplotlib backend.

Other style choices:

- Double quotes for strings unless the string contains double quotes, resulting in excessive backslash escaping.  This should be handled automatically by the code formatter.

We use the "black" command line tool to format our source.  This needs to be installed
on your system before running the ``format_source.sh`` script.


Compiled Code
-------------------

For consistency with python, class names follow python CamelCase convention.  Function
names follow python_underscore_convention.  Formatting is set by uncrustify with a
custom config file and this is run by the ``format_source.sh`` script.

All code that is exposed through pybind11 is in a single toast namespace.  Nested
namespaces may be used for code that is internal to the C++ code.

The "using" statement is allowed for aliasing a specific class or type::

    using ShapeContainer = py::detail::any_container<ssize_t>;

But should **not** be used to import an entire namespace::

    using std;

Header files included with "#include" should use angle brackets ("<>") for the header
file name.  If this fails for some reason then that indicates a problem with the build
system and its header file search paths.  Using double quotes for "#include" statements
is OK when including raw source files in the same directory (for example, when including
raw \*.cpp contents generated by external scripts).

When including C standard header files in C++, use the form::

    #include <cstdio>

Rather than::

    #include <stdio.h>

Pointer / reference declarations:  this allows reading from right to left as "a pointer
to a constant double" or "a reference to a constant double"::

    double const * data
    double const & data

Not::

    const double * data
    const double & data

When indexing the size of an STL container, the index variable should be either of the
size type declared in the container class or size_t.

When describing time domain sample indices or intervals, we using int64_t everywhere for
consistency.  This allows passing, e.g. "-1" to communicate unspecified intervals or
sample indices.

Single line conditional statements::

    if (x > 0) y = x;

Are permitted if they fit onto a single line.  Otherwise, insert braces.

Internal toast source files should not include the main "toast.hpp".  Instead
they should include the specific headers they need.  For example::

    #include <toast/sys_utils.hpp>
    #include <toast/math_lapack.hpp>
    #include <toast/math_qarray.hpp>

If attempting to vectorize code with OpenMP simd constructs, be sure to check that any
data array used in the simd region are aligned (see toast::is_aligned).  Otherwise this
can result in silent data corruption.

Documentation:  sphinx is used.  All python code should have docstrings.  All C++ code
exposed through pybind11 should also have docstrings defined in the bindings.  C++ code
that is not exposed to python is considered internal, expert-level code that does not
require formal documentation.  However, such code should have sufficient comments to
describe the algorithm and design choices.

Testing Workflow
--------------------------

We are using a github workflow to pull docker containers with our dependencies and run
our unit tests.  Those docker containers are re-generated whenever a new tag is made on
the `cmbenv git repository <https://github.com/hpc4cmb/cmbenv>`_.  So if there are
dependencies that need to be updated, open a PR against cmbenv which updates the version
or build in the package file.  After merging and tagging a new cmbenv release the
updated docker images will be available in an hour or two and be used automatically.

Release Process
---------------------------

There are some github workflows that only run when a new tag is created.  Unless you are
sure everything works, create a "release candidate" tag first.  Before making a tag,
ensure that the `docs/changes.rst` file contains all pull requests that have been merged
since the last tag.  Also edit the `src/toast/RELEASE` file and set the version to a
`PEP-440 compatible string <https://www.python.org/dev/peps/pep-0440/>`_.  Next go onto
the github releases page and create a new release from the master branch.  Briefly, the
format for a stable release, a release candidate or alpha version is:

    2.6.9
    2.6.7rc2
    2.6.8a1

After tagging the release, verify that the github workflow to deploy pip wheels runs and
uploads these to PyPI.  The conda-forge bots will automatically detect the new tag and
open a PR to update the feedstock.  Also, after the release is complete, update the
`RELEASE` file to be at the "alpha" of the next release.  For example, after tagging
version 3.4.5, set the version in the RELEASE file to 3.4.6a1.  This version is **only**
used when building pip wheels.  Anyone installing locally with setup.py or with pip
running on the local source tree will get a version constructed from the number of
commits since the last git tag.
