# Developer documentation

Welcome to the developer guidelines! This document is split into two parts:

1.  The [repository setup](#setting-up-the-repository). This section is relevant primarily for the repository maintainer and shows how to connect
    continuous integration services and documents initial set-up of the repository.
2.  The [contributor guide](#contributing-guide). It contains information relevant to all developers who want to make a contribution.

## Setting up the repository

### Documentation on _readthedocs_

We recommend using [readthedocs.org][] (RTD) to build and host the documentation for your project.
To enable readthedocs, head over to [their webiste][readthedocs.org] and sign in with your GitHub account.
On the RTD dashboard choose "Import a Project" and follow the instructions to add your repository.

-   Make sure to choose the correct name of the default branch. On GitHub, the default name of the default branch has
    recently changed from `master` to `main`.
-   We recommend to enable documentation builds for pull requests (PRs). This ensures that a PR doesn't introduce changes
    that break the documentation. To do so, got to `Admin -> Advanced Settings`, check the
    `Build pull requests for this projects` option, and click `Save`. For more information, please refer to
    the [official RTD documentation](https://docs.readthedocs.io/en/stable/pull-requests.html).

### Coverage tests with _Codecov_

Coverage tells what fraction of the code is "covered" by unit tests, thereby encouraging contributors to
[write tests](#writing-tests).
To enable coverage checks, head over to [codecov][] and sign in with your GitHub account.
You'll find more information in "getting started" section of the [codecov docs][].

In brief, you need to:

1. Generate a Codecov Token by clicking _setup repo_ in the codecov dashboard.
2. Go to the _Settings_ of your newly created repository on GitHub.
3. Go to _Security > Secrets > Actions_.
4. Create new repository secret with name `CODECOV_TOKEN` and paste the token generated by codecov
5. Go back to Github Actions page an re-run previously failed jobs.

### Pre-commit checks

[Pre-commit][] checks are fast programs that
check code for errors, inconsistencies and code styles, before the code
is committed.

We recommend setting up [pre-commit.ci][] to enforce consistency checks on every commit
and pull-request.

To do so, head over to [pre-commit.ci][] and click "Sign In With GitHub". Follow
the instructions to enable pre-commit.ci for your account or your organization. You
may choose to enable the service for an entire organization or on a per-repository basis.

Once authorized, pre-commit.ci should automatically be activated.

#### Overview of pre-commit hooks used by the template

The following pre-commit checks are for code style and format:

-   [black](https://black.readthedocs.io/en/stable/): standard code
    formatter in Python.
-   [isort](https://pycqa.github.io/isort/): sort module imports into
    sections and types.
-   [prettier](https://prettier.io/docs/en/index.html): standard code
    formatter for non-Python files (e.g. YAML).
-   [blacken-docs](https://github.com/asottile/blacken-docs): black on
    python code in docs.

The following pre-commit checks are for errors and inconsistencies:

-   [flake8](https://flake8.pycqa.org/en/latest/): standard check for errors in Python files.
    -   [flake8-tidy-imports](https://github.com/adamchainz/flake8-tidy-imports):
        tidy module imports.
    -   [flake8-docstrings](https://github.com/PyCQA/flake8-docstrings):
        pydocstyle extension of flake8.
    -   [flake8-rst-docstrings](https://github.com/peterjc/e8-rst-docstrings):
        extension of `flake8-docstrings` for `rst` docs.
    -   [flake8-comprehensions](https://github.com/adamchainz/e8-comprehensions):
        write better list/set/dict comprehensions.
    -   [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear):
        find possible bugs and design issues in program.
    -   [flake8-blind-except](https://github.com/elijahandrews/flake8-blind-except):
        checks for blind, catch-all `except` statements.
-   [yesqa](https://github.com/asottile/yesqa):
    remove unneccesary `# noqa` comments, follows additional dependencies listed above.
-   [autoflake](https://github.com/PyCQA/autoflake):
    remove unused imports and variables.
-   [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks): generic pre-commit hooks.
    -   **detect-private-key**: checks for the existence of private keys.
    -   **check-ast**: check whether files parse as valid python.
    -   **end-of-file-fixer**:check files end in a newline and only a newline.
    -   **mixed-line-ending**: checks mixed line ending.
    -   **trailing-whitespace**: trims trailing whitespace.
    -   **check-case-conflict**: check files that would conflict with case-insensitive file systems.
-   [pyupgrade](https://github.com/asottile/pyupgrade):
    upgrade syntax for newer versions of the language.

#### Notes on pre-commit checks

-   To ignore lint warnigs from **flake8**, see [Ignore certain lint warnings](#ignore-certain-lint-warnings).
-   You can add or remove pre-commit checks by simply deleting relevant lines in the `.pre-commit-config.yaml` file.
    Some pre-commit checks have additional options that can be specified either in the `pyproject.toml` or tool-specific
    config files, such as `.prettierrc.yml` for **prettier** and `.flake8` for **flake8**.

### API design

Scverse ecosystem packages should operate on [AnnData][] and/or [MuData][] datastructures and typically use an API
as originally [introduced by scanpy][scanpy-api] with the following submodules:

-   `pp` for preprocessing
-   `tl` for tools (that, compared to `pp` generate interpretable output, often associated with a corresponding plotting
    function)
-   `pl` for plotting functions

You may add additional submodules as appropriate. While we encourage to follow a scanpy-like API for ecosystem packages,
there may also be good reasons to choose a different approach, e.g. using an object-oriented API.

[scanpy-api]: https://scanpy.readthedocs.io/en/stable/usage-principles.html

### Ignore certain lint warnings

The [pre-commit checks](#pre-commit-checks) include [flake8](https://flake8.pycqa.org/en/latest/) which checks
for errors in Python files, including stylistic errors.

In some cases it might overshoot and you may have good reasons to ignore certain warnings.

To ignore an specific error on a per-case basis, you can add a comment `# noqa` to the offending line. You can also
specify the error ID to ignore, with e.g. `# noqa: E731`. Check the [flake8 guide][] for reference.

Alternatively, you can disable certain error messages for the entire project. To do so, edit the `.flake8`
file in the root of the repository. Add one line per linting code you wish to ignore and don't forget to add a comment.

```toml
...
# line break before a binary operator -> black does not adhere to PEP8
W503
# line break occured after a binary operator -> black does not adhere to PEP8
W504
...
```

[flake8 guide]: https://flake8.pycqa.org/en/3.1.1/user/ignoring-errors.html

### Using VCS-based versioning

By default, the template uses hard-coded version numbers that are set in `pyproject.toml` and [managed with
bump2version](#making-a-release). If you prefer to have your project automatically infer version numbers from git
tags, it is straightforward to switch to vcs-based versioning using [hatch-vcs][].

In `pyproject.toml` add the following changes, and you are good to go!

```diff
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,11 +1,11 @@
 [build-system]
 build-backend = "hatchling.build"
-requires = ["hatchling"]
+requires = ["hatchling", "hatch-vcs"]


 [project]
 name = "scib-metrics"
-version = "0.3.1dev"
+dynamic = ["version"]

@@ -60,6 +60,9 @@
+[tool.hatch.version]
+source = "vcs"
+
 [tool.coverage.run]
 source = ["scib-metrics"]
 omit = [
```

Don't forget to update the [Making a release section](#making-a-release) in this document accordingly, after you are done!

[hatch-vcs]: https://pypi.org/project/hatch-vcs/

## Contributing guide

Scanpy provides extensive [developer documentation][scanpy developer guide], most of which applies to this repo, too.
This document will not reproduce the entire content from there. Instead, it aims at summarizing the most important
information to get you started on contributing.

We assume that you are already familiar with git and with making pull requests on GitHub. If not, please refer
to the [scanpy developer guide][].

### Installing dev dependencies

In addition to the packages needed to _use_ this package, you need additional python packages to _run tests_ and _build
the documentation_. It's easy to install them using `pip`:

```bash
pip install "scib-metrics[dev,test,doc]"
```

### Code-style

This template uses [pre-commit][] to enforce consistent code-styles. On every commit, pre-commit checks will either
automatically fix issues with the code, or raise an error message. See [pre-commit checks](#pre-commit-checks) for
a full list of checks enabled for this repository.

To enable pre-commit locally, simply run

```bash
pre-commit install
```

in the root of the repository. Pre-commit will automatically download all dependencies when it is run for the first time.

Alternatively, you can rely on the [pre-commit.ci][] service enabled on GitHub. If you didn't run `pre-commit` before
pushing changes to GitHub it will automatically commit fixes to your pull request, or show an error message.

If pre-commit.ci added a commit on a branch you still have been working on locally, simply use

```bash
git pull --rebase
```

to integrate the changes into yours.

Finally, most editors have an _autoformat on save_ feature. Consider enabling this option for [black][black-editors]
and [prettier][prettier-editors].

[black-editors]: https://black.readthedocs.io/en/stable/integrations/editors.html
[prettier-editors]: https://prettier.io/docs/en/editors.html

### Writing tests

This package uses the [pytest][] for automated testing. Please [write tests][scanpy-test-docs] for every function added
to the package.

Most IDEs integrate with pytest and provide a GUI to run tests. Alternatively, you can run all tests from the
command line by executing

```bash
pytest
```

in the root of the repository. Continuous integration will automatically run the tests on all pull requests.

[scanpy-test-docs]: https://scanpy.readthedocs.io/en/latest/dev/testing.html#writing-tests

### Making a release

#### Updating the version number

Before making a release, you need to update the version number. Please adhere to [Semantic Versioning][semver], in brief

> Given a version number MAJOR.MINOR.PATCH, increment the:
>
> 1.  MAJOR version when you make incompatible API changes,
> 2.  MINOR version when you add functionality in a backwards compatible manner, and
> 3.  PATCH version when you make backwards compatible bug fixes.
>
> Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

We use [bump2version][] to automatically update the version number in all places and automatically create a git tag.
Run one of the following commands in the root of the repository

```bash
bump2version patch
bump2version minor
bump2version major
```

Once you are done, run

```
git push --tags
```

to publish the created tag on GitHub.

[bump2version]: https://github.com/c4urself/bump2version

#### Upload on PyPI

Please follow the [Python packaging tutorial][].

It is possible to automate this with GitHub actions, see also [this feature request][pypi-feature-request]
in the cookiecutter-scverse template.

[python packaging tutorial]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives
[pypi-feature-request]: https://github.com/scverse/cookiecutter-scverse/issues/88

### Writing documentation

Please write documentation for your package. This project uses [sphinx][] with the following features:

-   the [myst][] extension allows to write documentation in markdown/Markedly Structured Text
-   [Numpy-style docstrings][numpydoc] (through the [napoloen][numpydoc-napoleon] extension).
-   Jupyter notebooks as tutorials through [nbsphinx][] (See [Tutorials with nbsphinx](#tutorials-with-nbsphinx-and-jupyter-notebooks))
-   [Sphinx autodoc typehints][], to automatically reference annotated input and output types

See the [scanpy developer docs](https://scanpy.readthedocs.io/en/latest/dev/documentation.html) for more information
on how to write documentation.

### Tutorials with nbsphinx and jupyter notebooks

The documentation is set-up to render jupyter notebooks stored in the `docs/notebooks` directory using [nbsphinx][].
Currently, only notebooks in `.ipynb` format are supported that will be included with both their input and output cells.
It is your reponsibility to update and re-run the notebook whenever necessary.

If you are interested in automatically running notebooks as part of the continuous integration, please check
out [this feature request](https://github.com/scverse/cookiecutter-scverse/issues/40) in the `cookiecutter-scverse`
repository.

[nbsphinx]: https://github.com/spatialaudio/nbsphinx

#### Hints

-   If you refer to objects from other packages, please add an entry to `intersphinx_mapping` in `docs/conf.py`. Only
    if you do so can sphinx automatically create a link to the external documentation.
-   If building the documentation fails because of a missing link that is outside your control, you can add an entry to
    the `nitpick_ignore` list in `docs/conf.py`

#### Building the docs locally

```bash
cd docs
make html
open _build/html/index.html
```

<!-- Links -->

[scanpy developer guide]: https://scanpy.readthedocs.io/en/latest/dev/index.html
[codecov]: https://about.codecov.io/sign-up/
[codecov docs]: https://docs.codecov.com/docs
[pre-commit.ci]: https://pre-commit.ci/
[readthedocs.org]: https://readthedocs.org/
[nbshpinx]: https://github.com/spatialaudio/nbsphinx
[jupytext]: https://jupytext.readthedocs.io/en/latest/
[pre-commit]: https://pre-commit.com/
[anndata]: https://github.com/scverse/anndata
[mudata]: https://github.com/scverse/mudata
[pytest]: https://docs.pytest.org/
[semver]: https://semver.org/
[sphinx]: https://www.sphinx-doc.org/en/master/
[myst]: https://myst-parser.readthedocs.io/en/latest/intro.html
[numpydoc-napoleon]: https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html
[numpydoc]: https://numpydoc.readthedocs.io/en/latest/format.html
[sphinx autodoc typehints]: https://github.com/tox-dev/sphinx-autodoc-typehints
