PEP: 599 Title: The manylinux2014 Platform Tag Version: \$Revision\$
Last-Modified: \$Date\$ Author: Dustin Ingram \<<di@python.org>\>
Sponsor: Paul Moore \<<p.f.moore@gmail.com>\> BDFL-Delegate: Paul Moore
\<<p.f.moore@gmail.com>\> Discussions-To:
<https://discuss.python.org/t/the-next-manylinux-specification/> Status:
Final Type: Informational Content-Type: text/x-rst Created:
29-April-2019 Post-History: 29-April-2019 Resolution:
<https://discuss.python.org/t/the-next-manylinux-specification/1043/199>

Abstract
========

This PEP proposes the creation of a `manylinux2014` platform tag to
succeed the `manylinux2010` tag introduced by `513`{.interpreted-text
role="pep"}. It also proposes that PyPI and `pip` both be updated to
support uploading, downloading, and installing `manylinux2014`
distributions on compatible platforms.

Rationale
=========

CentOS 6 is now the oldest supported CentOS release, and will receive
maintenance updates through November 30th, 2020,[^1] at which point it
will reach end-of-life, and no further updates such as security patches
will be made available. All wheels built under the `manylinux2010`
images will remain at obsolete versions after that point.

Therefore, we propose the continuation of the existing manylinux
standard, and that a new PEP 425-style[^2] platform tag called
`manylinux2014` be derived from CentOS 7 and that the `manylinux`
toolchain, PyPI, and `pip` be updated to support it.

Similar to how `571`{.interpreted-text role="pep"} and
`513`{.interpreted-text role="pep"} drew allowed shared libraries and
their symbol versions from CentOS 5.11 and CentOS 6, respectively, a
`manylinux2014` platform tag will draw its libraries and symbol versions
from CentOS 7, which will reach end-of-life on June 30th, 2024.[^3]

The `manylinuxYYYY` pattern has a number of advantages that motivate
continuing with the current status quo:

-   Well-defined Docker images with clearly specified compatible
    libraries;
-   No need to survey for compatibility issues across multiple releases;
-   A single build image and `auditwheel` profile per architecture.

There are also some disadvantages:

-   Requires drafting a new PEP for every new standard;
-   Requires adding the new platform tag to installers (e.g., `pip`);
-   Installers are unable to install a platform tag which predates a
    given release.

There are also challenges which would exist for any proposal, including
the time and effort it takes to define, prepare and release the Docker
images and corresponding `auditwheel` profiles. These challenges were
experienced in the long rollout period for `manylinux2010`, which took
approximately 1 year from PEP acceptance to compatible build environment
published.[^4]

However, if this PEP can be an indicator, the process is now
well-defined and easily repeatable, which should increase the timeline
for rollout of a newer, updated platform tag.

The `manylinux2014` policy
==========================

The following criteria determine a `linux` wheel\'s eligibility for the
`manylinux2014` tag:

1.  The wheel may only contain binary executables and shared objects
    compiled for one of the following architectures supported by CentOS
    7, or a CentOS 7 compatible base image (such as ubi7):[^5] :

        x86_64
        i686
        aarch64
        armv7l
        ppc64
        ppc64le
        s390x

    This list adds support for ARMv7 (armv7l), ARMv8 (aarch64) and
    PowerPC (ppc64, ppc64le) architectures supported by the CentOS
    Alternative Architecture Special Interest Group, as well as the IBM
    Z (s390x) architecture.[^6]

2.  The wheel\'s binary executables or shared objects may not link
    against externally-provided libraries except those in the following
    list: :

        libgcc_s.so.1
        libstdc++.so.6
        libm.so.6
        libdl.so.2
        librt.so.1
        libc.so.6
        libnsl.so.1
        libutil.so.1
        libpthread.so.0
        libresolv.so.2
        libX11.so.6
        libXext.so.6
        libXrender.so.1
        libICE.so.6
        libSM.so.6
        libGL.so.1
        libgobject-2.0.so.0
        libgthread-2.0.so.0
        libglib-2.0.so.0

    This list is identical to the externally-provided libraries
    originally allowed for `manylinux2010`, with one exception:
    `libcrypt.so.1` was removed due to being deprecated in Fedora 30.
    `libpythonX.Y` remains ineligible for inclusion for the same reasons
    outlined in `513`{.interpreted-text role="pep"}.

    On Debian-based systems, these libraries are provided by the
    packages:

      Package        Libraries
      -------------- ----------------------------------------------------------------------------------------------------------
      libc6          libdl.so.2, libresolv.so.2, librt.so.1, libc.so.6, libpthread.so.0, libm.so.6, libutil.so.1, libnsl.so.1
      libgcc1        libgcc\_s.so.1
      libgl1         libGL.so.1
      libglib2.0-0   libgobject-2.0.so.0, libgthread-2.0.so.0, libglib-2.0.so.0
      libice6        libICE.so.6
      libsm6         libSM.so.6
      libstdc++6     libstdc++.so.6
      libx11-6       libX11.so.6
      libxext6       libXext.so.6
      libxrender1    libXrender.so.1

    On RPM-based systems, they are provided by these packages:

      Package      Libraries
      ------------ ----------------------------------------------------------------------------------------------------------
      glib2        libglib-2.0.so.0, libgthread-2.0.so.0, libgobject-2.0.so.0
      glibc        libresolv.so.2, libutil.so.1, libnsl.so.1, librt.so.1, libpthread.so.0, libdl.so.2, libm.so.6, libc.so.6
      libICE       libICE.so.6
      libX11       libX11.so.6
      libXext:     libXext.so.6
      libXrender   libXrender.so.1
      libgcc:      libgcc\_s.so.1
      libstdc++    libstdc++.so.6
      mesa         libGL.so.1

3.  If the wheel contains binary executables or shared objects linked
    against any allowed libraries that also export versioned symbols,
    they may only depend on the following maximum versions:

        GLIBC_2.17
        CXXABI_1.3.7, CXXABI_TM_1 is also allowed
        GLIBCXX_3.4.19
        GCC_4.8.0

    As an example, `manylinux2014` wheels may include binary artifacts
    that require `glibc` symbols at version `GLIBC_2.12`, because this
    an earlier version than the maximum of `GLIBC_2.17`.

4.  If a wheel is built for any version of CPython 2 or CPython versions
    3.0 up to and including 3.2, it *must* include a CPython ABI tag
    indicating its Unicode ABI. A `manylinux2014` wheel built against
    Python 2, then, must include either the `cpy27mu` tag indicating it
    was built against an interpreter with the UCS-4 ABI or the `cpy27m`
    tag indicating an interpreter with the UCS-2 ABI.[^7][^8]
5.  A wheel *must not* require the `PyFPE_jbuf` symbol. This is achieved
    by building it against a Python compiled *without* the
    `--with-fpectl` `configure` flag.

Compilation of Compliant Wheels
===============================

Like `manylinux1`, the `auditwheel` tool adds `manylinux2014` platform
tags to `linux` wheels built by `pip wheel` or `bdist_wheel` in a
`manylinux2014` Docker container.

Docker Images
-------------

A `manylinux2014` Docker image based on CentOS 7 x86\_64 should be
provided for building binary `linux` wheels that can reliably be
converted to `manylinux2014` wheels. This image will come with a full
compiler suite installed (`gcc`, `g++`, and `gfortran` 4.8.5) as well as
the latest releases of Python and `pip`.

Auditwheel
----------

The `auditwheel` tool will also be updated to produce `manylinux2014`
wheels.[^9] Its behavior and purpose will be otherwise unchanged from
`513`{.interpreted-text role="pep"}.

Platform Detection for Installers
=================================

Platforms may define a `manylinux2014_compatible` boolean attribute on
the `_manylinux` module described in `513`{.interpreted-text
role="pep"}. A platform is considered incompatible with `manylinux2014`
if the attribute is `False`.

If the `_manylinux` module is not found, or it does not have the
attribute `manylinux2014_compatible`, tools may fall back to checking
for glibc. If the platform has glibc 2.17 or newer, it is assumed to be
compatible unless the `_manylinux` module says otherwise.

Specifically, the algorithm we propose is:

    def is_manylinux2014_compatible():
        # Only Linux, and only supported architectures
        from distutils.util import get_platform

        if get_platform() not in [
            "linux-x86_64",
            "linux-i686",
            "linux-aarch64",
            "linux-armv7l",
            "linux-ppc64",
            "linux-ppc64le",
            "linux-s390x",
        ]:
            return False

        # Check for presence of _manylinux module
        try:
            import _manylinux

            return bool(_manylinux.manylinux2014_compatible)
        except (ImportError, AttributeError):
            # Fall through to heuristic check below
            pass

        # Check glibc version. CentOS 7 uses glibc 2.17.
        # PEP 513 contains an implementation of this function.
        return have_compatible_glibc(2, 17)

Backwards compatibility with `manylinux2010` wheels
===================================================

As explained in `513`{.interpreted-text role="pep"}, the specified
symbol versions for `manylinux1` allowed libraries constitute an *upper
bound*. The same is true for the symbol versions defined for
`manylinux2014` in this PEP. As a result, `manylinux1` and
`manylinux2010` wheels are considered `manylinux2014` wheels. A `pip`
that recognizes the `manylinux2014` platform tag will thus install
`manylinux2010` wheels for `manylinux2014` platforms \-- even when
explicitly set \--when no `manylinux2014` wheels are available.

PyPI Support
============

PyPI should permit wheels containing the `manylinux2014` platform tag to
be uploaded in the same way that it permits `manylinux2010`.

If technically feasible, PyPI should attempt to verify the compatibility
of `manylinux2014` wheels, but that capability is not a requirement for
adoption of this PEP.

Package authors should not upload non-compliant `manylinux2014` wheels
to PyPI, and should be aware that PyPI may begin blocking non-compliant
wheels from being uploaded.

References
==========

Acceptance
==========

PEP 599 was [accepted by Paul Moore on July 31,
2019](https://discuss.python.org/t/the-next-manylinux-specification/1043/199).

Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

[^1]: CentOS Product Specifications
    (<https://wiki.centos.org/About/Product>)

[^2]: PEP 425: Compatibility Tags for Built Distributions
    (<https://www.python.org/dev/peps/pep-0425/>)

[^3]: CentOS Product Specifications
    (<https://wiki.centos.org/About/Product>)

[^4]: Tracking issue for manylinux2010 rollout
    (<https://github.com/pypa/manylinux/issues/179>)

[^5]: Red Hat Universal Base Image 7
    (<https://access.redhat.com/containers/?tab=overview#/registry.access.redhat.com/ubi7>)

[^6]: The CentOS Alternative Architecture Special Interest Group
    (<https://wiki.centos.org/SpecialInterestGroup/AltArch>)

[^7]: PEP 3149: ABI version tagged .so files
    (<https://www.python.org/dev/peps/pep-3149/>)

[^8]: SOABI support for Python 2.X and PyPy
    (<https://github.com/pypa/pip/pull/3075>)

[^9]: auditwheel (<https://github.com/pypa/auditwheel/>)
