|jazzband| |pypi| |pyversions| |buildstatus-travis| |buildstatus-appveyor| |codecov|

==================================
pip-tools = pip-compile + pip-sync
==================================

A set of command line tools to help you keep your ``pip``-based packages fresh,
even when you've pinned them.  `You do pin them, right?`_

.. image:: https://github.com/jazzband/pip-tools/raw/master/img/pip-tools-overview.png
   :alt: pip-tools overview for phase II

.. |buildstatus-travis| image:: https://img.shields.io/travis/jazzband/pip-tools/master.svg?logo=travis
   :alt: Travis CI build status
   :target: https://travis-ci.org/jazzband/pip-tools
.. |buildstatus-appveyor| image:: https://img.shields.io/appveyor/ci/jazzband/pip-tools/master.svg?logo=appveyor
   :alt: AppVeyor build status
   :target: https://ci.appveyor.com/project/jazzband/pip-tools
.. |codecov| image:: https://codecov.io/gh/jazzband/pip-tools/branch/master/graph/badge.svg
   :alt: Coverage
   :target: https://codecov.io/gh/jazzband/pip-tools
.. |jazzband| image:: https://jazzband.co/static/img/badge.svg
   :alt: Jazzband
   :target: https://jazzband.co/
.. |pypi| image:: https://img.shields.io/pypi/v/pip-tools.svg
   :alt: PyPI version
   :target: https://pypi.org/project/pip-tools/
.. |pyversions| image:: https://img.shields.io/pypi/pyversions/pip-tools.svg
   :alt: Supported Python versions
   :target: https://pypi.org/project/pip-tools/
.. _You do pin them, right?: http://nvie.com/posts/pin-your-packages/


Installation
============

As part of a Python project's environment tooling (similar to ``pip``), it's
recommended to install ``pip-tools`` in each project's `virtual environment`_:

.. code-block:: bash

    $ source /path/to/venv/bin/activate
    (venv)$ pip install pip-tools

**Note**: all of the remaining example commands assume you've activated your
project's virtual environment.

.. _virtual environment: https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments

Example usage for ``pip-compile``
=================================

The ``pip-compile`` command lets you compile a ``requirements.txt`` file from
your dependencies, specified in either ``setup.py`` or ``requirements.in``.

Run it with ``pip-compile`` or  ``python -m piptools compile``. If you use
multiple Python versions, you can run ``pip-compile`` as ``py -X.Y -m piptools
compile`` on Windows and ``pythonX.Y -m piptools compile`` on other systems.

**Note**: ensure you don't have ``requirements.txt`` if you compile
``setup.py`` or ``requirements.in`` from scratch, otherwise, it might
interfere.

Requirements from ``setup.py``
------------------------------

Suppose you have a Flask project, and want to pin it for production.
If you have a ``setup.py`` with ``install_requires=['Flask']``, then run
``pip-compile`` without any arguments:

.. code-block:: bash

    $ pip-compile
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    pip-compile --output-file requirements.txt setup.py
    #
    click==6.7                # via flask
    flask==0.12.2
    itsdangerous==0.24        # via flask
    jinja2==2.9.6             # via flask
    markupsafe==1.0           # via jinja2
    werkzeug==0.12.2          # via flask

``pip-compile`` will produce your ``requirements.txt``, with all the Flask
dependencies (and all underlying dependencies) pinned.  You should put
``requirements.txt`` under version control.

Without ``setup.py``
--------------------

If you don't use ``setup.py`` (`it's easy to write one`_), you can create a
``requirements.in`` file to declare the Flask dependency:

.. code-block:: ini

    # requirements.in
    Flask

Now, run ``pip-compile requirements.in``:

.. code-block:: bash

    $ pip-compile requirements.in
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    pip-compile --output-file requirements.txt requirements.in
    #
    click==6.7                # via flask
    flask==0.12.2
    itsdangerous==0.24        # via flask
    jinja2==2.9.6             # via flask
    markupsafe==1.0           # via jinja2
    werkzeug==0.12.2          # via flask

And it will produce your ``requirements.txt``, with all the Flask dependencies
(and all underlying dependencies) pinned.  You should put both
``requirements.in`` and ``requirements.txt`` under version control.

.. _it's easy to write one: https://packaging.python.org/guides/distributing-packages-using-setuptools/#configuring-your-project

Using hashes
------------

If you would like to use *Hash-Checking Mode* available in ``pip`` since
version 8.0, ``pip-compile`` offers ``--generate-hashes`` flag:

.. code-block:: bash

    $ pip-compile --generate-hashes requirements.in
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    pip-compile --generate-hashes --output-file requirements.txt requirements.in
    #
    click==6.7 \
        --hash=sha256:29f99fc6125fbc931b758dc053b3114e55c77a6e4c6c3a2674a2dc986016381d \
        --hash=sha256:f15516df478d5a56180fbf80e68f206010e6d160fc39fa508b65e035fd75130b \
        # via flask
    flask==0.12.2 \
        --hash=sha256:0749df235e3ff61ac108f69ac178c9770caeaccad2509cb762ce1f65570a8856 \
        --hash=sha256:49f44461237b69ecd901cc7ce66feea0319b9158743dd27a2899962ab214dac1
    itsdangerous==0.24 \
        --hash=sha256:cbb3fcf8d3e33df861709ecaf89d9e6629cff0a217bc2848f1b41cd30d360519 \
        # via flask
    jinja2==2.9.6 \
        --hash=sha256:2231bace0dfd8d2bf1e5d7e41239c06c9e0ded46e70cc1094a0aa64b0afeb054 \
        --hash=sha256:ddaa01a212cd6d641401cb01b605f4a4d9f37bfc93043d7f760ec70fb99ff9ff \
        # via flask
    markupsafe==1.0 \
        --hash=sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665 \
        # via jinja2
    werkzeug==0.12.2 \
        --hash=sha256:903a7b87b74635244548b30d30db4c8947fe64c5198f58899ddcd3a13c23bb26 \
        --hash=sha256:e8549c143af3ce6559699a01e26fa4174f4c591dbee0a499f3cd4c3781cdec3d \
        # via flask

Updating requirements
---------------------

To update all packages, periodically re-run ``pip-compile --upgrade``.

To update a specific package to the latest or a specific version use the
``--upgrade-package`` or ``-P`` flag:

.. code-block:: bash

    # only update the flask package
    $ pip-compile --upgrade-package flask

    # update both the flask and requests packages
    $ pip-compile --upgrade-package flask --upgrade-package requests

    # update the flask package to the latest, and requests to v2.0.0
    $ pip-compile --upgrade-package flask --upgrade-package requests==2.0.0

You can combine ``--upgrade`` and ``--upgrade-package`` in one command, to
provide constraints on the allowed upgrades. For example to upgrade all
packages whilst constraining requests to the latest version less than 3.0:

.. code-block:: bash

    $ pip-compile --upgrade --upgrade-package 'requests<3.0'

Output File
-----------

To output the pinned requirements in a filename other than
``requirements.txt``, use ``--output-file``. This might be useful for compiling
multiple files, for example with different constraints on flask to test a
library with both versions using `tox <https://tox.readthedocs.io/en/latest/>`__:

.. code-block:: bash

    $ pip-compile --upgrade-package 'flask<1.0' --output-file requirements-flask0x.txt
    $ pip-compile --upgrade-package 'flask<2.0' --output-file requirements-flask1x.txt

Or to output to standard output, use ``--output-file=-``:

.. code-block:: bash

    $ pip-compile --output-file=- > requirements.txt
    $ pip-compile - --output-file=- < requirements.in > requirements.txt

Configuration
-------------

You might be wrapping the ``pip-compile`` command in another script. To avoid
confusing consumers of your custom script you can override the update command
generated at the top of requirements files by setting the
``CUSTOM_COMPILE_COMMAND`` environment variable.

.. code-block:: bash

    $ CUSTOM_COMPILE_COMMAND="./pipcompilewrapper" pip-compile requirements.in
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    ./pipcompilewrapper
    #
    flask==0.10.1
    itsdangerous==0.24        # via flask
    jinja2==2.7.3             # via flask
    markupsafe==0.23          # via jinja2
    werkzeug==0.10.4          # via flask

Workflow for layered requirements
---------------------------------

If you have different environments that you need to install different but
compatible packages for, then you can create layered requirements files and use
one layer to constrain the other.

For example, if you have a Django project where you want the newest ``2.1``
release in production and when developing you want to use the Django debug
toolbar, then you can create two ``*.in`` files, one for each layer:

.. code-block:: ini

    # requirements.in
    django<2.2

At the top of the development requirements ``dev-requirements.in`` you use ``-c
requirements.txt`` to constrain the dev requirements to packages already
selected for production in ``requirements.txt``.

.. code-block:: ini

    # dev-requirements.in
    -c requirements.txt
    django-debug-toolbar

First, compile ``requirements.txt`` as usual:

.. code-block:: bash

    $ pip-compile
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    pip-compile
    #
    django==2.1.12
    pytz==2019.2              # via django

Now compile the dev requirements and the ``requirements.txt`` file is used as
a constraint:

.. code-block:: bash

    $ pip-compile dev-requirements.in
    #
    # This file is autogenerated by pip-compile
    # To update, run:
    #
    #    pip-compile dev-requirements.in
    #
    django-debug-toolbar==2.0
    django==2.1.12            # via django-debug-toolbar
    sqlparse==0.3.0           # via django-debug-toolbar

As you can see above, even though a ``2.2`` release of Django is available, the
dev requirements only include a ``2.1`` version of Django because they were
constrained. Now both compiled requirements files can be installed safely in
the dev environment.

Example usage for ``pip-sync``
==============================

Now that you have a ``requirements.txt``, you can use ``pip-sync`` to update
your virtual environment to reflect exactly what's in there. This will
install/upgrade/uninstall everything necessary to match the
``requirements.txt`` contents.

Run it with ``pip-sync`` or ``python -m piptools sync``. If you use multiple
Python versions, you can also run ``py -X.Y -m piptools sync`` on Windows and
``pythonX.Y -m piptools sync`` on other systems.

**Be careful**: ``pip-sync`` is meant to be used only with a
``requirements.txt`` generated by ``pip-compile``.

.. code-block:: bash

    $ pip-sync
    Uninstalling flake8-2.4.1:
      Successfully uninstalled flake8-2.4.1
    Collecting click==4.1
      Downloading click-4.1-py2.py3-none-any.whl (62kB)
        100% |................................| 65kB 1.8MB/s
      Found existing installation: click 4.0
        Uninstalling click-4.0:
          Successfully uninstalled click-4.0
    Successfully installed click-4.1

To sync multiple ``*.txt`` dependency lists, just pass them in via command
line arguments, e.g.

.. code-block:: bash

    $ pip-sync dev-requirements.txt requirements.txt

Passing in empty arguments would cause it to default to ``requirements.txt``.

If you use multiple Python versions, you can run ``pip-sync`` as
``py -X.Y -m piptools sync ...`` on Windows and
``pythonX.Y -m piptools sync ...`` on other systems.

**Note**: ``pip-sync`` will not upgrade or uninstall packaging tools like
``setuptools``, ``pip``, or ``pip-tools`` itself. Use ``pip install --upgrade``
to upgrade those packages.

Other useful tools
==================

- `pipdeptree`_ to print the dependency tree of the installed packages.
- ``requirements.in``/``requirements.txt`` syntax highlighting:

  * `requirements.txt.vim`_ for Vim.
  * `Python extension for VS Code`_ for VS Code.

.. _pipdeptree: https://github.com/naiquevin/pipdeptree
.. _requirements.txt.vim: https://github.com/raimon49/requirements.txt.vim
.. _Python extension for VS Code: https://marketplace.visualstudio.com/items?itemName=ms-python.python
