.. _basic:

Basic Usage of Pipenv
=====================

.. image:: https://farm4.staticflickr.com/3931/33173826122_b7ee8f1a26_k_d.jpg

This document covers some of Pipenv's more basic features.

☤ Example Pipfile & Pipfile.lock
--------------------------------

Pipfiles contain information for the dependencies of the project, and supersedes
the requirements.txt file used in most Python projects. You should add a Pipfile in the
Git repository. The only thing required for users who clone the repository would be
installing Pipenv in the machine and typing ``pipenv install``. Pipenv is a reference
implementation for using Pipfile.

.. _example_files:

Here is a simple example of a ``Pipfile`` and the resulting ``Pipfile.lock``.

Example Pipfile
///////////////

::

    [[source]]
    url = "https://pypi.python.org/simple"
    verify_ssl = true
    name = "pypi"

    [packages]
    requests = "*"


    [dev-packages]
    pytest = "*"


Example Pipfile.lock
////////////////////

::

    {
        "_meta": {
            "hash": {
                "sha256": "8d14434df45e0ef884d6c3f6e8048ba72335637a8631cc44792f52fd20b6f97a"
            },
            "host-environment-markers": {
                "implementation_name": "cpython",
                "implementation_version": "3.6.1",
                "os_name": "posix",
                "platform_machine": "x86_64",
                "platform_python_implementation": "CPython",
                "platform_release": "16.7.0",
                "platform_system": "Darwin",
                "platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
                "python_full_version": "3.6.1",
                "python_version": "3.6",
                "sys_platform": "darwin"
            },
            "pipfile-spec": 5,
            "requires": {},
            "sources": [
                {
                    "name": "pypi",
                    "url": "https://pypi.python.org/simple",
                    "verify_ssl": true
                }
            ]
        },
        "default": {
            "certifi": {
                "hashes": [
                    "sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704",
                    "sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5"
                ],
                "version": "==2017.7.27.1"
            },
            "chardet": {
                "hashes": [
                    "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691",
                    "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae"
                ],
                "version": "==3.0.4"
            },
            "idna": {
                "hashes": [
                    "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4",
                    "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f"
                ],
                "version": "==2.6"
            },
            "requests": {
                "hashes": [
                    "sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b",
                    "sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e"
                ],
                "version": "==2.18.4"
            },
            "urllib3": {
                "hashes": [
                    "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b",
                    "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f"
                ],
                "version": "==1.22"
            }
        },
        "develop": {
            "py": {
                "hashes": [
                    "sha256:2ccb79b01769d99115aa600d7eed99f524bf752bba8f041dc1c184853514655a",
                    "sha256:0f2d585d22050e90c7d293b6451c83db097df77871974d90efd5a30dc12fcde3"
                ],
                "version": "==1.4.34"
            },
            "pytest": {
                "hashes": [
                    "sha256:b84f554f8ddc23add65c411bf112b2d88e2489fd45f753b1cae5936358bdf314",
                    "sha256:f46e49e0340a532764991c498244a60e3a37d7424a532b3ff1a6a7653f1a403a"
                ],
                "version": "==3.2.2"
            }
        }
    }

☤ General Recommendations & Version Control
-------------------------------------------

- Generally, keep both ``Pipfile`` and ``Pipfile.lock`` in version control.
- Do not keep ``Pipfile.lock`` in version control if multiple versions of Python are being targeted.
- Specify your target Python version in your ``Pipfile``'s ``[requires]`` section. Ideally, you should only have one target Python version, as this is a deployment tool. ``python_version`` should be in the format ``X.Y`` (or ``X``) and ``python_full_version`` should be in ``X.Y.Z`` format.
- ``pipenv install`` is fully compatible with ``pip install`` syntax, for which the full documentation can be found `here <https://pip.pypa.io/en/stable/user_guide/#installing-packages>`__.
- Note that the ``Pipfile`` uses the `TOML Spec <https://github.com/toml-lang/toml#user-content-spec>`_.



☤ Example Pipenv Workflow
-------------------------

Clone / create project repository::

    $ cd myproject

Install from Pipfile, if there is one::

    $ pipenv install

Or, add a package to your new project::

    $ pipenv install <package>

This will create a ``Pipfile`` if one doesn't exist. If one does exist, it will automatically be edited with the new package you provided.

Next, activate the Pipenv shell::

    $ pipenv shell
    $ python --version

This will spawn a new shell subprocess, which can be deactivated by using ``exit``.

.. _initialization:

☤ Example Pipenv Upgrade Workflow
---------------------------------

- Find out what's changed upstream: ``$ pipenv update --outdated``.
- Upgrade packages, two options:
    a. Want to upgrade everything? Just do ``$ pipenv update``.
    b. Want to upgrade packages one-at-a-time? ``$ pipenv update <pkg>`` for each outdated package.

☤ Importing from requirements.txt
---------------------------------

If you only have a ``requirements.txt`` file available when running ``pipenv install``,
pipenv will automatically import the contents of this file and create a ``Pipfile`` for you.

You can also specify ``$ pipenv install -r path/to/requirements.txt`` to import a requirements file.

If your requirements file has version numbers pinned, you'll likely want to edit the new ``Pipfile``
to remove those, and let ``pipenv`` keep track of pinning.  If you want to keep the pinned versions
in your ``Pipfile.lock`` for now, run ``pipenv lock --keep-outdated``.  Make sure to
`upgrade <#initialization>`_ soon!

.. _specifying_versions:

☤ Specifying Versions of a Package
----------------------------------

You can specify versions of a package using the `Semantic Versioning scheme <https://semver.org/>`_
(i.e. ``major.minor.micro``).

For example, to install requests you can use: ::

    $ pipenv install requests~=1.2

Pipenv will install version ``1.2`` and any minor update, but not ``2.0``.

This will update your ``Pipfile`` to reflect this requirement, automatically.

In general, Pipenv uses the same specifier format as pip. However, note that according to `PEP 440`_ , you can't use versions containing a hyphen or a plus sign.

.. _`PEP 440`: https://www.python.org/dev/peps/pep-0440/

To make inclusive or exclusive version comparisons you can use: ::

    $ pipenv install "requests>=1.4"   # will install a version equal or larger than 1.4.0
    $ pipenv install "requests<=2.13"  # will install a version equal or lower than 2.13.0
    $ pipenv install "requests>2.19"   # will install 2.19.1 but not 2.19.0

.. note:: The use of double quotes around the package and version specification (i.e. ``"requests>2.19"``) is highly recommended
    to avoid issues with `Input and output redirection <https://robots.thoughtbot.com/input-output-redirection-in-the-shell>`_
    in Unix-based operating systems.

The use of ``~=`` is preferred over the ``==`` identifier as the latter prevents pipenv from updating the packages:  ::

    $ pipenv install "requests~=2.2"  # locks the major version of the package (this is equivalent to using >=2.2, ==2.*)

To avoid installing a specific version you can use the ``!=`` identifier.

For an in depth explanation of the valid identifiers and more complex use cases check `the relevant section of PEP-440`_.

.. _`the relevant section of PEP-440`: https://www.python.org/dev/peps/pep-0440/#version-specifiers

☤ Specifying Versions of Python
-------------------------------

To create a new virtualenv, using a specific version of Python you have installed (and
on your ``PATH``), use the ``--python VERSION`` flag, like so:

Use Python 3::

   $ pipenv --python 3

Use Python3.6::

   $ pipenv --python 3.6

Use Python 2.7.14::

    $ pipenv --python 2.7.14

When given a Python version, like this, Pipenv will automatically scan your system for a Python that matches that given version.

If a ``Pipfile`` hasn't been created yet, one will be created for you, that looks like this::

    [[source]]
    url = "https://pypi.python.org/simple"
    verify_ssl = true

    [dev-packages]

    [packages]

    [requires]
    python_version = "3.6"

.. note:: The inclusion of ``[requires] python_version = "3.6"`` specifies that your application requires this version
          of Python, and will be used automatically when running ``pipenv install`` against this ``Pipfile`` in the future
          (e.g. on other machines). If this is not true, feel free to simply remove this section.

If you don't specify a Python version on the command–line, either the ``[requires]`` ``python_full_version`` or ``python_version`` will be selected
automatically, falling back to whatever your system's default ``python`` installation is, at time of execution.


☤ Editable Dependencies (e.g. ``-e .`` )
----------------------------------------

You can tell Pipenv to install a path as editable — often this is useful for
the current working directory when working on packages::

    $ pipenv install --dev -e .

    $ cat Pipfile
    ...
    [dev-packages]
    "e1839a8" = {path = ".", editable = true}
    ...

.. note:: All sub-dependencies will get added to the ``Pipfile.lock`` as well. Sub-dependencies are **not** added to the
          ``Pipfile.lock`` if you leave the ``-e`` option out.


.. _environment_management:

☤ Environment Management with Pipenv
------------------------------------

The three primary commands you'll use in managing your pipenv environment are
``$ pipenv install``, ``$ pipenv uninstall``, and ``$ pipenv lock``.

.. _pipenv_install:

$ pipenv install
////////////////

``$ pipenv install`` is used for installing packages into the pipenv virtual environment
and updating your Pipfile.

Along with the basic install command, which takes the form::

    $ pipenv install [package names]

The user can provide these additional parameters:

    - ``--three`` — Performs the installation in a virtualenv using the system ``python3`` link.
    - ``--python`` — Performs the installation in a virtualenv using the provided Python interpreter.

    .. warning:: None of the above commands should be used together. They are also
                 **destructive** and will delete your current virtualenv before replacing
                 it with an appropriately versioned one.

    - ``--dev`` — Install both ``develop`` and ``default`` packages from ``Pipfile``.
    - ``--system`` — Use the system ``pip`` command rather than the one from your virtualenv.
    - ``--deploy`` — Make sure the packages are properly locked in Pipfile.lock, and abort if the lock file is out-of-date.
    - ``--ignore-pipfile`` — Ignore the ``Pipfile`` and install from the ``Pipfile.lock``.
    - ``--skip-lock`` — Ignore the ``Pipfile.lock`` and install from the ``Pipfile``. In addition, do not write out a ``Pipfile.lock`` reflecting changes to the ``Pipfile``.

.. _pipenv_uninstall:

$ pipenv uninstall
//////////////////

``$ pipenv uninstall`` supports all of the parameters in `pipenv install <#pipenv-install>`_,
as well as two additional options, ``--all`` and ``--all-dev``.

    - ``--all`` — This parameter will purge all files from the virtual environment,
      but leave the Pipfile untouched.

    - ``--all-dev`` — This parameter will remove all of the development packages from
      the virtual environment, and remove them from the Pipfile.


.. _pipenv_lock:

$ pipenv lock
/////////////

``$ pipenv lock`` is used to create a ``Pipfile.lock``, which declares **all** dependencies (and sub-dependencies) of your project, their latest available versions, and the current hashes for the downloaded files. This ensures repeatable, and most importantly *deterministic*, builds.

☤ About Shell Configuration
---------------------------

Shells are typically misconfigured for subshell use, so ``$ pipenv shell --fancy`` may produce unexpected results. If this is the case, try ``$ pipenv shell``, which uses "compatibility mode", and will attempt to spawn a subshell despite misconfiguration.

A proper shell configuration only sets environment variables like ``PATH`` during a login session, not during every subshell spawn (as they are typically configured to do). In fish, this looks like this::

    if status --is-login
        set -gx PATH /usr/local/bin $PATH
    end

You should do this for your shell too, in your ``~/.profile`` or ``~/.bashrc`` or wherever appropriate.

.. note:: The shell launched in interactive mode. This means that if your shell reads its configuration from a specific file for interactive mode (e.g. bash by default looks for a ``~/.bashrc`` configuration file for interactive mode), then you'll need to modify (or create) this file.

If you experience issues with ``$ pipenv shell``, just check the ``PIPENV_SHELL`` environment variable, which ``$ pipenv shell`` will use if available. For detail, see :ref:`configuration-with-environment-variables`.

☤ A Note about VCS Dependencies
-------------------------------

You can install packages with pipenv from git and other version control systems using URLs formatted according to the following rule::

    <vcs_type>+<scheme>://<location>/<user_or_organization>/<repository>@<branch_or_tag>#egg=<package_name>

The only optional section is the ``@<branch_or_tag>`` section.  When using git over SSH, you may use the shorthand vcs and scheme alias ``git+git@<location>:<user_or_organization>/<repository>@<branch_or_tag>#egg=<package_name>``. Note that this is translated to ``git+ssh://git@<location>`` when parsed.

Note that it is **strongly recommended** that you install any version-controlled dependencies in editable mode, using ``pipenv install -e``, in order to ensure that dependency resolution can be performed with an up-to-date copy of the repository each time it is performed, and that it includes all known dependencies.

Below is an example usage which installs the git repository located at ``https://github.com/requests/requests.git`` from tag ``v2.20.1`` as package name ``requests``::

    $ pipenv install -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests
    Creating a Pipfile for this project...
    Installing -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests...
    [...snipped...]
    Adding -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests to Pipfile's [packages]...
    [...]

    $ cat Pipfile
    [packages]
    requests = {git = "https://github.com/requests/requests.git", editable = true, ref = "v2.20.1"}

Valid values for ``<vcs_type>`` include ``git``, ``bzr``, ``svn``, and ``hg``.  Valid values for ``<scheme>`` include ``http``, ``https``, ``ssh``, and ``file``.  In specific cases you also have access to other schemes: ``svn`` may be combined with ``svn`` as a scheme, and ``bzr`` can be combined with ``sftp`` and ``lp``.

You can read more about pip's implementation of VCS support `here <https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support>`__. For more information about other options available when specifying VCS dependencies, please check the `Pipfile spec <https://github.com/pypa/pipfile>`_.


☤ Pipfile.lock Security Features
--------------------------------

``Pipfile.lock`` takes advantage of some great new security improvements in ``pip``.
By default, the ``Pipfile.lock`` will be generated with the sha256 hashes of each downloaded
package. This will allow ``pip`` to guarantee you're installing what you intend to when
on a compromised network, or downloading dependencies from an untrusted PyPI endpoint.

We highly recommend approaching deployments with promoting projects from a development
environment into production. You can use ``pipenv lock`` to compile your dependencies on
your development environment and deploy the compiled ``Pipfile.lock`` to all of your
production environments for reproducible builds.

.. note::

    If you'd like a ``requirements.txt`` output of the lockfile, run ``$ pipenv requirements``.


☤ Pipenv and Docker Containers
------------------------------

In general, you should not have Pipenv inside a linux container image, since
it is a build tool. If you want to use it to build, and install the run time
dependencies for your application, you can use a multistage build for creating
a virtual environment with your dependencies. In this approach,
Pipenv in installed in the base layer, it is then used to create the virtual
environment. In a later stage, in a ``runtime`` layer the virtual environment
is copied from the base layer, the layer containing pipenv and other build
dependencies is discarded.
This results in a smaller image, which can still run your application.
Here is an example ``Dockerfile``, which you can use as a starting point for
doing a multistage build for your application::

  FROM docker.io/python:3.9 AS builder

  RUN pip install --user pipenv

  # Tell pipenv to create venv in the current directory
  ENV PIPENV_VENV_IN_PROJECT=1

  # Pipfile contains requests
  ADD Pipfile.lock Pipfile /usr/src/

  WORKDIR /usr/src

  # NOTE: If you install binary packages required for a python module, you need
  # to install them again in the runtime. For example, if you need to install pycurl
  # you need to have pycurl build dependencies libcurl4-gnutls-dev and libcurl3-gnutls
  # In the runtime container you need only libcurl3-gnutls

  # RUN apt install -y libcurl3-gnutls libcurl4-gnutls-dev

  RUN /root/.local/bin/pipenv sync

  RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)"

  FROM docker.io/python:3.9 AS runtime

  RUN mkdir -v /usr/src/.venv

  COPY --from=builder /usr/src/.venv/ /usr/src/.venv/

  RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)"

  # HERE GOES ANY CODE YOU NEED TO ADD TO CREATE YOUR APPLICATION'S IMAGE
  # For example
  # RUN apt install -y libcurl3-gnutls
  # RUN adduser --uid 123123 coolio
  # ADD run.py /usr/src/

  WORKDIR /usr/src/

  USER coolio

  CMD ["./.venv/bin/python", "-m", "run.py"]

.. Note::

   Pipenv is not meant to run as root. However, in the multistage build above
   it is done nevertheless. A calculated risk, since the intermediate image
   is discarded.
   The runtime image later shows that you should create a user and user it to
   run your application.
   **Once again, you should not run pipenv as root (or Admin on Windows) normally.
   This could lead to breakage of your Python installation, or even your complete
   OS.**

When you build an image with this example (assuming requests is found in Pipfile), you
will see that ``requests`` is installed in the ``runtime`` image::

  $ sudo docker build --no-cache -t oz/123:0.1 .
  Sending build context to Docker daemon  1.122MB
  Step 1/12 : FROM docker.io/python:3.9 AS builder
   ---> 81f391f1a7d7
  Step 2/12 : RUN pip install --user pipenv
   ---> Running in b83ed3c28448
   ... trimmed ...
   ---> 848743eb8c65
  Step 4/12 : ENV PIPENV_VENV_IN_PROJECT=1
   ---> Running in 814e6f5fec5b
  Removing intermediate container 814e6f5fec5b
   ---> 20167b4a13e1
  Step 5/12 : ADD Pipfile.lock Pipfile /usr/src/
   ---> c7632cb3d5bd
  Step 6/12 : WORKDIR /usr/src
   ---> Running in 1d75c6cfce10
  Removing intermediate container 1d75c6cfce10
   ---> 2dcae54cc2e5
  Step 7/12 : RUN /root/.local/bin/pipenv sync
   ---> Running in 1a00b326b1ee
  Creating a virtualenv for this project...
  ... trimmed ...
  ✔ Successfully created virtual environment!
  Virtualenv location: /usr/src/.venv
  Installing dependencies from Pipfile.lock (fe5a22)...
  ... trimmed ...
  Step 8/12 : RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)"
   ---> Running in 3a66e3ce4a11
  2.27.1
  Removing intermediate container 3a66e3ce4a11
   ---> 1db657d0ac17
  Step 9/12 : FROM docker.io/python:3.9 AS runtime
  ... trimmed ...
  Step 12/12 : RUN /usr/src/venv/bin/python -c "import requests; print(requests.__version__)"
   ---> Running in fa39ba4080c5
  2.27.1
  Removing intermediate container fa39ba4080c5
   ---> 2b1c90fd414e
  Successfully built 2b1c90fd414e
  Successfully tagged oz/123:0.1
