# Contributing

We welcome all contributions to decomon.

You can help by:

- fixing bugs (see [issues](https://github.com/airbus/decomon/issues) with label "bug"),
- improving the documentation,
- adding and improving educational notebooks in `tutorials/`.

This is not exhaustive.

The project is hosted on [https://github.com/airbus/decomon](https://github.com/airbus/decomon).
Contributions to the repository are made by submitting pull requests.


This guide is organized as follows:

```{contents}
---
depth: 2
local: true
---
```


## Setting up your development environment

### Prerequisites

#### Python 3.8+ environment

The use of a virtual environment is recommended, and you will need to ensure that the environment use a Python version
greater than 3.8.
This can be achieved for instance either by using [conda](https://docs.conda.io/en/latest/) or by using [pyenv](https://github.com/pyenv/pyenv) (or [pyenv-win](https://github.com/pyenv-win/pyenv-win) on windows)
and [venv](https://docs.python.org/fr/3/library/venv.html) module.

The following examples show how to create a virtual environment with Python version 3.8.13 with the mentioned methods.

##### With conda (all platforms)

```shell
conda create -n do-env python=3.8.13
conda activate do-env
```

##### With pyenv + venv (Linux/MacOS)

```shell
pyenv install 3.8.13
pyenv shell 3.8.13
python -m venv do-venv
source do-venv/bin/activate
```

##### With pyenv-win + venv (Windows)

```shell
pyenv install 3.8.13
pyenv shell 3.8.13
python -m venv do-venv
do-venv\Scripts\activate
```

#### Tensorflow 2.6+

A version of tensorflow greater than 2.6 is required. If no package tensorflow is detected when installing decomon, a package tensorflow will be installed.
Howver it is better to do it beforehand to ensure having a version that is adapted to your GPU/CPU. See [tensorflow documentation](https://www.tensorflow.org/install/pip)
to see how to install it properly.

### Installing from source in developer mode

We use the option `--editable` (or `-e`) of `pip install`.

> **Note**: you need a version of pip >=21.3 to be able to install in editable mode a project using a `pyproject.toml` file.

```shell
git clone https://github.com/airbus/decomon.git
cd decomon
pip install -U pip
pip install -e .
```


### Building the docs locally

The documentation is using [Sphinx](https://www.sphinx-doc.org/en/master/index.html) to generate
the html pages, and in particular the autogenerated API doc from in-code docstrings.


#### Install the documentation dependencies

See [above](#installing-from-source-in-developer-mode) to install decomon in developer mode, and then

```shell
pip install -r docs/requirements.txt
```

#### Define environment variables for notebook github links

In order to define appropriate github and colab links for notebooks,
we need several environment variables:

- AUTODOC_NOTEBOOKS_REPO_URL: url of the content repository for the notebooks
- AUTODOC_NOTEBOOKS_BRANCH: branch containing the notebooks

For instance:
```shell
current_repo_url_withdotgit=$(git remote get-url origin)
export AUTODOC_NOTEBOOKS_REPO_URL=${current_repo_url_withdotgit/.git/}
export AUTODOC_NOTEBOOKS_BRANCH=$(git branch --show-current)
```

#### Build the docs

On Linux or Mac, or with [git-bash](https://gitforwindows.org/) on windows, make the documentation with

```shell
cd docs
# generate api doc source files
rm source/api/decomon*.rst
sphinx-apidoc -o source/api -f -T ../src/decomon
# generate available notebooks list
python generate_nb_index.py
# remove previous build
rm -rf build
# build doc html pages
sphinx-build -M html source build
```

The index of the built documentation is then available at `build/html/index.html`
from the `docs/` repository. You can for instance browse the documentation by running
```shell
python -m http.server -d build/html
```
and go to [http://localhost:8000/](http://localhost:8000/). Doing this, rather than just opening `index.html` directly in you browser,
make javascript work properly.

On windows with standard command prompt, the same lines work except that you have to use `rm` equivalent command to remove files and directories.


```{admonition} Notes
- `sphinx-apidoc` is used to generate the source files for api doc. It is necessary to launch it
each time a new subpackage/module is added to the code.
- The line before is useful to avoid having source file corresponding to previous subpackage/module
that have been removed.
- `generate_nb_index.py` update the list of notebooks in the generated file notebooks.md that is
the source file of notebooks page. The command has to be launched when a new notebook is added.
- The last command actually build the html outputs and is required each time a doc page has a change
(either beacause a source file has changed or the docstrings in the code have been modified).
- To update the doc faster when testing changes, launch only the last command
without the previous `rm [...]`.
```


### Running unit tests

The unit tests are gathered in `tests/` folder and can can be launched with [tox](https://tox.wiki/).

Tox manage the creation of its own virtual environments with different versions of python,
so do not use the virtual environment where you installed the library.

Install tox (for instance in a separate virtual environment):
```shell
python -m venv tox_venv
. tox_venv/bin/activate
pip install -U pip
pip install tox
```

Launch the test suite:
```shell
tox
```

If all python versions are not available or if you want to test only on a given version,
specify the tox environment you want to test on. For instance:
```shell
tox -e py39
```

Under the table, tox is actually calling [pytest](https://docs.pytest.org/) to perform the tests. You could also directly
call it (after installing it in your development environment) with:
```shell
pytest tests -v
```

### Running notebooks as tests

One can test programmatically that notebooks are not broken thanks to [nbmake](https://github.com/treebeardtech/nbmake) extension for pytest.

```shell
pip install nbmake
pytest --nbmake tutorials -v
```

## Guidelines to follow when preparing a contribution

### Coding style and code linting

To help maintaining the same coding style across the project, some code linters are used via [pre-commit](https://pre-commit.com/).

It is used by CI to run checks at each push, but can also be used locally.

Once installed, you can run it on all files with
```shell
pre-commit run --all-files
```
Beware that doing so, you are actually modifying the files.

You can also use it when committing:

- stage your changes: `git add your_files`,
- run pre-commit on the staged files: `pre-commit run`,
- check the changes made,
- accept them by adding modified files: `git add -u`,
- commit: `git commit`.

This can also be done automatically at each commit if you add pre-commit to git hooks with `pre-commit install`.
Beware that when doing so,
- the changes will be refused if pre-commit actually modifies the files,
- you can then accept the modifications with `git add -u`,
- you can still force a commit that violates pre-commit checks with `git commit -n` or `git commit --no-verify`.

If you prefer run pre-commit manually, you can remove the hooks with `pre-commit uninstall`.

### Annotations

The code is now fully annotated, and [mypy](https://mypy-lang.org/) is used to statically type checking it during CI workflow.
So when you write new code, you should add the appropriate annotations to your functions and methods.

You can then test it directly with mypy
```shell
pip install mypy
mypy
```

This check is also part of the tox test suite, via the tox environment "type":
```shell
tox -e type
```
See the [section about unit tests](#running-unit-tests) for more information on tox.


### Notebooks

We try to give some introductory examples via notebooks available in the corresponding `tutorials/` directory.

#### Title and description extraction
The list of these notebooks is automatically inserted in the documentation with a title and a description.
These are actually extracted from the first cell. To enable that, each notebook should

- start with a markdown cell,
- its first line being the title starting with one number sign ("# "),
- the remaining lines being used as the description.

For instance:
```markdown
# Great notebook title

A quick description of the main features of the notebook.
Can be on several lines.

Can include a nice thumbnail.
![Notebook_thumbnail](https://upload.wikimedia.org/wikipedia/commons/2/27/MnistExamples.png)
```

#### Colab specificities
In order to make it work on colab, you should also adding a cell that, after testing that the notebook is run on colab,
install decomon and other necessary dependencies:
```python
# On Colab: install the library
on_colab = "google.colab" in str(get_ipython())
if on_colab:
    import sys  # noqa: avoid having this import removed by pycln

    # install dev version for dev doc, or release version for release doc
    !{sys.executable} -m pip install -U pip
    !{sys.executable} -m pip install git+https://github.com/airbus/decomon@main#egg=decomon
```
When deployed in the release doc, the pip spec for decomon will be automatically updated to "decomon==x.y.z",
"x.y.z" being the appropriate version.


### Adding unit tests

- Whenever adding some code, think to add some tests to the `tests/` folder.
- Whenever fixing a bug, think to add a test that crashes before fixing the bug and does not afterwards.

Follow [above instructions](#running-unit-tests) to run them with pytest.

### Writing the documentation

#### In-code docstrings

The API is self-documented thanks to in-code docstrings and annotations. Whenever adding/fixing some code, you should add/update
doctrings and annotations.

In order to generate properly the API doc, some guidelines should be followed:

- Docstrings should follow [Google style](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) docstrings
(see this [example](https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html#example-google)),
parsed thanks to [napoleon extension](https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html?highlight=napoleon#module-sphinx.ext.napoleon).

- As we use type annotations in the code, types hints should not be added to the docstrings in order to avoid duplicates,
and potentially inconsistencies.

- You should use [annotations](https://peps.python.org/pep-0484/) to explicit the types of public variables and of the inputs, outputs of public methods/functions.

- Docstrings should also follow [PEP-257](https://peps.python.org/pep-0257/) which can be checked
thanks to [pydocstyle](https://www.pydocstyle.org/en/stable/) tool.

    ```shell
    pip install pydocstyle
    pydocstyle decomon
    ```

#### Doc pages

Natively, sphinx is meant to parse [reStructuredText files](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html).

To be able to directly reuse materials already written in markdown format, we use here the [myST extension](https://myst-parser.readthedocs.io/en/latest/intro.html),
that allows to write [makdown files](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html).

In markdown files, we can still write sphinx directives and roles as explained in [myST documentation](https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html).
For instance a sphinx table of contents tree can be inserted in a markodown file with a code block like:

    ```{toctree}
    ---
    maxdepth: 2
    caption: Contents
    ---
    install
    getting_started
    notebooks
    api/modules
    contribute
    ```


## Submitting pull requests

When you think you are ready to merge your modifications into the main repository, you will have to open a pull request (PR).
We can summarize the process as follows:

- Fork the repository on github.
- Clone your fork on your computer.
- Make your changes and push them to your fork.
- Do the necessary checks (see [below](#prior-checks)).
- Reorganize your commits (see [below](#reorganizing-commits)).
- Submit your pull request (see [github documentation](https://help.github.com/articles/creating-a-pull-request-from-a-fork/)).
- See if all CI checks passed on your PR.
- Wait for a review.
- Take the comments and required changes into account.

Note that a PR needs at least one review by a core developer to be merged.


You may want to add a reference to the main repository to fetch from it and (re)base your changes on it:
```shell
git remote add upstream https://github.com/airbus/decomon
```

This [post](https://medium.com/google-developer-experts/how-to-pull-request-d75ac81449a5) points out good practices to follow
to submit great pull requests and review them efficiently.


### Prior checks

Before submitting your pull request, think to
- [run the unit tests](#running-unit-tests),
- [check the documentation locally](#building-the-docs-locally) if you modified it,
- check you respect the coding styles by [running linters](#coding-style-and-code-linting),
- check that mypy is happy with [your annotations](#annotations).

If you do not, you will still be able to see the status of your PR as CI will do these checks for you.

### Reorganizing commits

On your way to implement your contribution, you will probably have lots of commits,
some modifying other ones from the same PR, or only modifying the code style.

At the end of your work, consider reorganizing them by
- squashing them into one or only a few logical commits,
- having a separate commit to reformat previous existing code if necessary,
- rewritting commit messages so that it explains the changes made and why, the "how" part being explained by the code itself
  (see this [post](https://chris.beams.io/posts/git-commit/) about what a commit message should and should not contain),
- rebasing on upstream repository main branch if it diverged too much by the time you finished.

You can use `git rebase -i` to do that, as explained in [git documentation](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
