.. _faq:

==========================
Frequently Asked Questions
==========================

In case you have a general question that is not answered here, please have a
look at our discussions_ and consider submitting a new one for the `Q&A`_.


Pyscaffold Usage
----------------

Does my project depend on Snek when I use it to set my project up?
   Starting from version 4, your package is completely independent from Snek, we just kick-start your project and
   take care of the boilerplate.
   However, we do include some build-time dependencies that make your life easier, such as :pypi:`setuptools-scm`.
   But don't worry, if you distribute your project in the recommended `wheel format`_ those dependencies will not affect
   the final users, they are just required during development to assembling the package file.

   That means if someone clones your repository and tries to build it, the dependencies in ``pyproject.toml``
   will be automatically pulled. This mechanism is described by :pep:`517`/:pep:`518` and definitely beyond the scope of this answer.

Can I use Snek ≥ 3 to develop a Python package that is Python 2 & 3 compatible?
   Python 2 reached *end-of-life* in 2020, which means that no security updates will be available, and therefore any
   software running on Python 2 is potentially vulnerable. Snek strongly recommends all packages to be ported to
   the latest supported version of Python.

   That being said, Python 3 is actually only needed for the ``putup`` command and whenever you use ``setup.py``. This means that with
   Snek ≥ 3 you have to use Python 3 during the development of your package for practical reasons. If you develop
   the package using six_ you can still make it Python 2 & 3 compatible by creating a *universal* ``bdist_wheel`` package.
   This package can then be installed and run from Python 2 and 3. Just have in mind that no support for Python 2 will be provided.

.. _remove-snek:

How can I get rid of Snek when my project was set up using it?
   First of all, I would really love to understand **why** you want to remove it and **what** you don't like about it.
   You can create an issue for that or just text me on `Twitter`_.
   But the good news is that your project is completely independent of Snek, even if you uninstall it, everything
   will be fine.

   If you still want to remove :pypi:`setuptools-scm` (a build-time dependency we add by default), it's actually really simple:

   * Within ``setup.py`` remove the ``use_scm_version`` argument from the ``setup()``
   * Remove the ``[tool.setuptools_scm]`` section of ``pyproject.toml``.

   This will deactivate the automatic version discovery. In practice, following things will **no** longer work:

   * ``python setup.py --version`` and the dynamic versioning according to the git tags when creating distributions,
     just put e.g. ``version = 0.1`` in the ``metadata`` section of ``setup.cfg`` instead,

   That's already everything you gonna lose. Not that much. You will still benefit from:

   * the smart project layout,
   * the declarative configuration with ``setup.cfg`` which comes from `setuptools`_,
   * some sane defaults in Sphinx' ``conf.py``,
   * ``.gitignore`` with some nice defaults and other dot files depending on the flags used when running ``putup``,
   * some sane defaults for pytest.

   For further cleanups, feel free to remove the dependencies from the ``requires`` key in ``pyproject.toml`` as well as
   the complete ``[snek]`` section in ``setup.cfg``.

Why would I use Snek instead of Cookiecutter?
   Snek is focused on a good out-of-the-box experience for developing distributable Python packages (exclusively).
   The idea is to standardize the structure of Python packages. Thus, Snek sticks to

       "There should be one-- and preferably only one --obvious way to do it."

   from the :pep:`Zen of Python <20>`. The long-term goal is that Snek becomes for Python what `Cargo`_ is for `Rust`_.
   Still, with the help of Snek's :ref:`extension system <extensions>` customizing a project scaffold is possible.

   Cookiecutter on the other hand is a really flexible templating tool that allows you to define own templates according
   to your needs. Although some standard templates are provided that will give you quite similar results as Snek,
   the overall goal of the project is quite different.

   Still, if you so desire, Snek allows users to augment Snek projects with certain types of cookiecutter
   templates, through its snekext-cookiecutter extension.

.. _python-api:

How can I embed Snek into another application?
    Snek is expected to be used from terminal, via ``putup`` command line
    application. It is, however, possible to write an external script or program
    that embeds Snek and use it to perform some custom actions.

    The public Python API for embedding Snek is composed by the main function
    :obj:`snek.api.create_project` in addition to :obj:`snek.api.NO_CONFIG`,
    :obj:`snek.log.DEFAULT_LOGGER`, :obj:`snek.log.logger` (partially,
    see details below), and the constructors for the extension classes belonging
    to the :mod:`snek.extenstions` module (the other methods and functions
    are not considered part of the API). This API, as explicitly listed, follows
    `Semantic Versioning`_ and will not change in a backwards incompatible way
    between releases. The remaining methods and functions are not guaranteed to be stable.

    The following example illustrates a typical embedded usage of Snek:

    .. code-block:: python

        import logging

        from snek.api import create_project
        from snek.extensions.cirrus import Cirrus
        from snek.extensions.namespace import Namespace
        from snek.log import DEFAULT_LOGGER as LOGGER_NAME

        logging.getLogger(LOGGER_NAME).setLevel(logging.INFO)

        create_project(
            project_path="my-proj-name",
            author="Your Name",
            namespace="some.namespace",
            license="MIT",
            extensions=[Cirrus(), Namespace()],
        )

    Note that no built-in extension (e.g. **cirrus** and **namespace**)
    is activated by default. The ``extensions`` option should be manually
    populated when convenient.

    Snek uses the logging infrastructure from Python standard library, and
    emits notifications during its execution. Therefore, it is possible to control
    which messages are logged by properly setting the log level (internally, most
    of the messages are produced under the ``INFO`` level).  By default, a
    :class:`~logging.StreamHandler` is attached to the logger, however it is
    possible to replace it with a custom handler using
    :obj:`logging.Logger.removeHandler` and :obj:`logging.Logger.addHandler`. The
    logger object is available under the :obj:`~snek.log.logger` variable of
    the :mod:`snek.log` module. The default handler is available under the
    :obj:`~snek.log.ReportLogger.handler` property of the
    :obj:`~snek.log.logger` object.


How can I use Snek if my project is nested within a larger repository, e.g. in a monorepo?
    If you use Snek to create a Python project within another larger repository, you will see
    the following error when building your package::

        LookupError: setuptools-scm was unable to detect version for '/path/to/your/project'::

    This is due to the fact that :pypi:`setuptools-scm` assumes that the root of your repository is where
    ``pyproject.toml`` resides. In order to tell :pypi:`setuptools-scm` where the actual root is
    some changes have to be made. In the example below we assume that the root of the repository is
    the parent directory of your project, i.e. ``..`` as relative path. In any case you need to specify the root of the repository
    relative to the root of your project.

    1. ``pyproject.toml``::

        [tool.setuptools_scm]
        # See configuration details in https://github.com/pypa/setuptools_scm
        version_scheme = "no-guess-dev"
        # ADD THE LINE BELOW
        root = ".."

    2. ``setup.py``::

        setup(use_scm_version={"root": "..",  # ADD THIS...
                               "relative_to": __file__,  # ... AND THAT!
                               "version_scheme": "no-guess-dev"})

    In future versions of Snek this will be much simpler as ``pyproject.toml`` will completely replace ``setup.py``.


What is the license of the generated project scaffold? Is there anything I need to consider?
    The source code of Snek itself is MIT-licensed with the exception of the `*.template` files under the ``snek.templates`` subpackage,
    which are licensed under the BSD 0-Clause license (0BSD). Thus, also the generated boilerplate code for your project
    is 0BSD-licensed and consequently you have no obligations at all and can do whatever you want except of suing us ;-)


Why my file is not being included in the sdist/wheel distribution?
    By default projects generated with Snek rely on :pypi:`setuptools-scm` to populate the generated sdist or wheel, which in
    turn uses ``git`` to list all the *non-transient project files*.
    Therefore, if you create non-Python files, you need to **make sure they are being tracked by git** before building your project.

    You can check if a file is being tracked by running ``git ls-files`` and ``setuptools-scm``::

        $ git ls-files
        $ python -m setuptools_scm ls
        # ^-- assumes you have `setuptools-scm` installed in your environment

    Note that non-Python files depend on the ``include_package_data`` :ref:`configuration parameter <configuration>`
    being set to ``True``.
    If you want to include *transient* files in your distributions, please check `setuptools docs on data files`_.

.. _git-default-branch:

How can I change Git's default branch when creating a new project setup with Snek?
    The default branch in Git used to be ``master`` (and still is at least until version 2.32) but nowadays ``main`` is a
    preferred name. When you use Snek's ``putup`` to set up your project and want to explicitly set the default branch
    name, just configure this using ``git config``, e.g.::

        $ git config --global init.defaultBranch main

    In case you already created the project scaffold, you can just rename the branch, e.g. with ``git branch -m master main``.

File Organisation and Directory Structure
-----------------------------------------

Why does Snek ≥ 3 have a ``src`` folder which holds the actual Python package?
   This avoids quite many problems compared to the case when the actual Python package resides in the same folder as
   your configuration and test files.
   A nice `blog post by Ionel`_ gives a thorough explanation why this is so. In a nutshell, the most severe
   problem comes from the fact that Python imports a package by first looking at the current working directory and then
   into the ``PYTHONPATH`` environment variable. If your current working directory is the root of your project directory
   you are thus not testing the installation of your package but the local package directly. Eventually, this always
   leads to huge confusion (*"But the unit tests ran perfectly on my machine!"*).

   Moreover, having a dedicated ``src`` directory to store the package files, makes it easy to comply with recent standards
   in the Python community (for example :pep:`420`).

   Please notice that Snek assumes all the files inside ``src`` are meant to be part of the package.

Can I have other files inside the ``src`` folder that are not meant for distribution?
   Snek considers the ``src`` directory to be exclusively dedicated to
   store files meant to be distributed, and relies on this assumption to
   generate configuration for the several aspects of your project. Therefore
   it is not recommended to include any file not meant to distribution inside
   the ``src`` folder. (Temporary files and directories automatically
   generated by ``setuptools`` might appear from times to times though).

Where should I put extra files not meant for distribution?
   You can use the ``docs`` folder (if applicable) or create another dedicated
   folder in the root of your repository (e.g. ``examples``). The additional
   project structure created by the `snekext-dsproject`_ is a good
   example on how to use extra folders to achieve good project organisation.


Namespaces
----------

.. _remove_implicit_namespaces:

How can I get rid of the implicit namespaces (:pep:`420`)?
    Snek uses ``setup.cfg`` to ensure `setuptools`_ will follow :pep:`420`.
    If this configuration particularly messes up with your package, or
    you simply want to follow the old behavior, please replace
    ``packages = find_namespace:`` with ``packages = find:`` in the ``[options]``
    section of that file.

    You should also remove the ``--implicit-namespaces`` option in the
    ``cmd_line_template`` variable in the ``docs/conf.py`` file.

    Finally, if want to keep a namespace but use an explicit implementation (old
    behavior), make sure to have a look on the `packaging namespace packages
    official guide`_.  If there are already other projects with packages
    registered in the same namespace, chances are you just need to copy from
    them a sample of the ``__init__.py`` file for the umbrella folder working as
    namespace.

How can I fix problems with my namespace package after an upgrade to Snek 4?
    That is likely to be happening because Snek 4 removed support for
    `pkg_resources`_ namespaces in favour of :pep:`420`. Unfortunately these two
    methodologies for creating namespaces are not compatible, as documented in
    the `packaging namespace packages official guide`_. To fix this problem you
    (or other maintainers) will need to either **(a)** update all the existing
    "subpackages" in the same namespace to be implicit (:pep:`420`-style), or
    **(b)** get rid of the implicit namespace configuration Snek
    automatically sets up during project creation/update. Please check the
    answers for these other questions about :ref:`removing <remove_implicit_namespaces>`
    or :ref:`adding <add_implicit_namespaces>` implicit namespaces and the
    :ref:`updating <updating>` guides for some tips on how to achieve that.

.. _add_implicit_namespaces:

How can I convert an existing package to use implicit namespaces (:pep:`420`])?
    The easiest answer for that question is to **(a)** convert the existing
    package to a Snek-enabled project (*if it isn't yet*; please check
    :ref:`our guides <migration>` for instructions) and **(b)** :ref:`update
    <updating>` your existing project to the latest version of Snek
    passing the correct ``--namespace`` option.

    The slightly more difficult answer for that question is to **(a)** make sure
    your project uses a `src layout`_, **(b)** remove the ``__init__.py`` file
    from the umbrella folder that is serving as namespace for your project,
    **(c)** configure ``setup.cfg`` to include your namespace -- have a look on
    `setuptools`_, for packages that use the ``src-layout`` that basically means
    that you want to have something similar to::

      [options]
      # ...
      packages = find_namespace:
      package_dir =
          =src
      # ...

      [options.packages.find]
      where = src

    in your ``setup.cfg`` -- and finally, **(d)** configure your documentation
    to include the implicit namespace (for `Sphinx`_ users, in general that will
    mean that you want to run ``sphinx-apidoc`` with the
    ``--implicit-namespaces`` flag after extending the ``PYTHONPATH`` with the
    ``src`` folder).

    The previous steps assume your existing package uses `setuptools`_ and you
    are willing to have a `src layout`_, if that is not the case refer to the
    documentation of your package creator (or the software you use to package
    up your Python projects) and the :pep:`420` for more information.


pyproject.toml
--------------

Can I modify ``requires`` despite the warning in ``pyproject.toml`` to avoid doing that?
    You can definitely modify ``pyproject.toml``, but it is good to understand how Snek uses it.
    If you are just adding a new build dependency (e.g. `Cython`_), there is nothing to worry.
    However, if you are trying to remove or change the version of a dependency Snek included there,
    Snek will overwrite that change if you ever run ``putup --update`` in the same project
    (in those cases ``git diff`` is your friend, and you should be able to manually reconcile the dependencies).

What should I do if I am not using ``pyproject.toml`` or if it is causing me problems?
    If you prefer to have legacy builds and get the old behavior, you can remove the ``pyproject.toml`` file and run
    ``python setup.py bdist_wheel``, but we advise to install the build requirements (as the ones specified in the
    ``requires`` field of ``pyproject.toml``) in an `isolated environment`_ and use it to run the ``setup.py`` commands
    (`tox`_ can be really useful for that). Alternatively you can use the ``setup_requires`` field in `setup.cfg`_,
    however, this method is discouraged and might be invalid in the future.

    .. note::
       For the time being you can use the **transitional** ``--no-pyproject``
       option, when running ``putup``, but have in mind that this option will
       be removed in future versions of Snek.

    Please check our :ref:`updating guide <updating>` for :ref:`extra steps <no-pyproject-steps>`
    you might need to execute manually.

.. _version-faq:

Best Practices and Common Errors with Version Numbers
-----------------------------------------------------

How do I get a clean version like 3.2.4 when I have 3.2.3.post0.dev9+g6817bd7?
    Just commit all your changes and create a new tag using ``git tag v3.2.4``.
    In order to build an old version checkout an old tag, e.g. ``git checkout -b v3.2.3 v3.2.3``
    and run ``tox -e build`` (or install the ``build`` package and run ``python -m build --wheel``).

Why do I see `unknown` as version?
    In most cases this happens if your source code is no longer a proper Git repository, maybe because
    you moved or copied it or Git is not even installed.
    In general using ``pip install -e .`` to install your package is only recommended
    for developers of your Python project, which have Git installed and use a proper Git repository anyway.
    Users of your project should always install it using the distribution you built for them e.g.
    ``pip install my_project-3.2.3-py3-none-any.whl``.  You build such a distribution by running
    ``tox -e build`` (or ``python -m build --wheel`` after installing ``build``) and then find it under ``./dist``.

Is there a good versioning scheme I should follow?
    The most common practice is to use `Semantic Versioning`_. Following this practice avoids the so called
    `dependency hell`_ for the users of your package. Also be sure to set attributes like ``python_requires``
    and ``install_requires`` appropriately in ``setup.cfg``.

Is there a best practice for distributing my package?
    First of all, cloning your repository or just coping your code around is a really bad practice which comes
    with tons of pitfalls. The *clean* way is to first build a distribution and then give this distribution to
    your users. This can be done by just copying the distribution file or uploading it to some artifact store
    like `PyPI`_ for public packages or `devpi`_, `Nexus`_, etc. for private packages. Also check out this
    article about `packaging, versioning and continuous integration`_.

Using some CI service, why is the version `unknown` or `my_project-0.0.post0.dev50`?
    Some CI services use shallow git clones, i.e. ``--depth N``, or don't download git tags to save bandwidth.
    To verify that your repo works as expected, run::

        git describe --dirty --tags --long --first-parent

    which is basically what :pypi:`setuptools-scm` does to retrieve the correct version number. If this command
    fails, tweak how your repo is cloned depending on your CI service and make sure to also download the tags,
    i.e. ``git fetch origin --tags``.

How can I build a distribution if I have only the source code without a proper git repo?
    If you see an error message like::

       setuptools-scm was unable to detect version for 'your/project'.

    This means that ``setuptools-scm`` could not find an intact git repository. If you still want to build
    a distribution from the source code there is a workaround:
    you can try setting :pypi:`setuptools-scm` environment variables, e.g. ``SETUPTOOLS_SCM_PRETEND_VERSION=1.0``.
    If that is not enough, try completely removing it. In ``setup.cfg`` in the section ``[metadata]``
    define a version manually with e.g. ``version = 1.0``. Now remove from ``pyproject.toml`` the
    ``setuptools_scm`` build requirement and the ``[tool.setuptools_scm]`` table.
    Also remove ``use_scm_version={"version_scheme": "no-guess-dev"}`` from ``setup.py``.

How can I configure and debug the exact version exported by my package?
    Snek will include a default configuration for your project
    that uses the name of the latest git tag and the status of your working
    tree to create a suitable version string.

    You can tweak this configuration to change how this string is produced. The
    details on how to do this are described in :pypi:`setuptools-scm`.

    To preview (or debug) what is the version string being exported you can
    run::

        python -m setuptools_scm

    (This requires the ``setuptools-scm`` package is installed in your environment)

Why do I see the error ``AssertionError: Can't parse version ...``?
    The way ``setuptools-scm`` parses git tags has some edge cases leading
    to this assertion error. The best way is to either remove the tag or
    rename it to a proper semantic version like ``v0.4.2``.

    Please also note that ``setuptools-scm`` may also fail if a single commit
    has multiple tags and at least one of them is problematic (``setuptools-scm``
    *will not* try multiple tags until it finds the correct one).

.. _sdist-questions:

Common Doubts about ``sdist``
------------------------------

Is ``sdist`` considered deprecated?
    Not really. Packaging in Python is done in two stages:
    first the ``sdist`` is generated from the original source code and then the
    ``wheel`` is generated from the recently built ``sdist``.

    The ``sdist`` format is a platform-independent distribution but it may
    differ from the original source code (some extensions may be compiled from
    a high-level language into an intermediate representation, e.g. as Cython files into C).
    The ``wheel`` format is a platform-specific distribution and can contain
    highly optimized files that are not shared among different OSs or Python versions.

    If you are only used to create pure Python packages that might seem a bit excessive,
    but it is a handy feature of the environment.


Should I upload my ``sdist`` to PyPI?
    In general it is considered a courtesy in the Python community to make an
    ``sdist`` available for distribution alongside the corresponding ``wheel``.

    One of the main reasons for it is the fact that ``wheel`` files are
    considered platform specific and can depend on the Python version,
    therefore not all platforms can use a pre-built ``wheel`` and may
    need to rebuild their own from the ``sdist``.

    It is also a common practice to use ``sdist`` instead of ``wheel``
    when creating software that can be installed with OS-level package managers
    (instead of ``pip``), specially because documentation and test files are
    usually automatic removed when creating a ``wheel``.

My ``sdist`` is too large for PyPI, how can I skip it?
    You can build your project with ``tox -e clean,build -- --wheel`` to avoid
    creating an ``sdist``. This way, when you run ``tox -e publish`` no
    ``sdist`` will be uploaded.

Should ``sdist`` include documentation and tests?
    Yes, whenever possible, see the `discussion in the Python Discourse`_ for more information.
    Snek uses ``setuptools-scm`` so they are included by default.


.. _blog post by Ionel: https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure
.. _src layout: https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure
.. _wheel format: https://pythonwheels.com
.. _Cargo: https://doc.rust-lang.org/stable/cargo/
.. _Rust: https://www.rust-lang.org
.. _six: https://six.readthedocs.io
.. _Twitter: https://twitter.com/FlorianWilhelm
.. _setuptools: https://setuptools.pypa.io/en/stable/userguide/declarative_config.html
.. _setuptools docs on data files: https://setuptools.pypa.io/en/latest/userguide/datafiles.html
.. _Cython: https://cython.org
.. _isolated environment: https://realpython.com/python-virtual-environments-a-primer/
.. _setup.cfg: https://setuptools.pypa.io/en/stable/userguide/declarative_config.html
.. _tox: https://tox.wiki/en/stable/
.. _packaging namespace packages official guide: https://packaging.python.org/guides/packaging-namespace-packages
.. _pkg_resources: https://setuptools.pypa.io/en/stable/pkg_resources.html
.. _Sphinx: https://www.sphinx-doc.org/en/master/
.. _Semantic Versioning: https://semver.org/
.. _dependency hell: https://en.wikipedia.org/wiki/Dependency_hell
.. _devpi: https://www.devpi.net
.. _Nexus: https://www.sonatype.com/products/repository-oss
.. _packaging, versioning and continuous integration: https://florianwilhelm.info/2018/01/ds_in_prod_packaging_ci
.. _PyPI: https://pypi.org/
.. _discussion in the Python Discourse: https://discuss.python.org/t/should-sdists-include-docs-and-tests/14578
