PEP: 425 Title: Compatibility Tags for Built Distributions Version:
$Revision$ Last-Modified: 07-Aug-2012 Author: Daniel Holth
<dholth@gmail.com> BDFL-Delegate: Nick Coghlan <ncoghlan@gmail.com>
Status: Final Type: Standards Track Content-Type: text/x-rst Created:
27-Jul-2012 Python-Version: 3.4 Post-History: 8-Aug-2012, 18-Oct-2012,
15-Feb-2013 Resolution:
https://mail.python.org/pipermail/python-dev/2013-February/124116.html

Abstract
========

This PEP specifies a tagging system to indicate with which versions of
Python a built or binary distribution is compatible. A set of three tags
indicate which Python implementation and language version, ABI, and
platform a built distribution requires. The tags are terse because they
will be included in filenames.

PEP Acceptance
==============

This PEP was accepted by Nick Coghlan on 17th February, 2013.

Rationale
=========

Today "python setup.py bdist" generates the same filename on PyPy and
CPython, but an incompatible archive, making it inconvenient to share
built distributions in the same folder or index. Instead, built
distributions should have a file naming convention that includes enough
information to decide whether or not a particular archive is compatible
with a particular implementation.

Previous efforts come from a time where CPython was the only important
implementation and the ABI was the same as the Python language release.
This specification improves upon the older schemes by including the
Python implementation, language version, ABI, and platform as a set of
tags.

By comparing the tags it supports with the tags listed by the
distribution, an installer can make an educated decision about whether
to download a particular built distribution without having to read its
full metadata.

Overview
========

The tag format is {python tag}-{abi tag}-{platform tag}

python tag 'py27', 'cp33' abi tag 'cp32dmu', 'none' platform tag
'linux\_x86\_64', 'any'

For example, the tag py27-none-any indicates compatible with Python 2.7
(any Python 2.7 implementation) with no abi requirement, on any
platform.

Use
===

The `wheel` built package format includes these tags in its filenames,
of the form
`{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl`.
Other package formats may have their own conventions.

Details
=======

Python Tag
----------

The Python tag indicates the implementation and version required by a
distribution. Major implementations have abbreviated codes, initially:

-   py: Generic Python (does not require implementation-specific
    features)
-   cp: CPython
-   ip: IronPython
-   pp: PyPy
-   jy: Jython

Other Python implementations should use `sys.implementation.name`.

The version is `py_version_nodot`. CPython gets away with no dot, but if
one is needed the underscore `_` is used instead. PyPy should probably
use its own versions here `pp18`, `pp19`.

The version can be just the major version `2` or `3` `py2`, `py3` for
many pure-Python distributions.

Importantly, major-version-only tags like `py2` and `py3` are not
shorthand for `py20` and `py30`. Instead, these tags mean the packager
intentionally released a cross-version-compatible distribution.

A single-source Python 2/3 compatible distribution can use the compound
tag `py2.py3`. See `Compressed Tag Sets`, below.

ABI Tag
-------

The ABI tag indicates which Python ABI is required by any included
extension modules. For implementation-specific ABIs, the implementation
is abbreviated in the same way as the Python Tag, e.g. `cp33d` would be
the CPython 3.3 ABI with debugging.

The CPython stable ABI is `abi3` as in the shared library suffix.

Implementations with a very unstable ABI may use the first 6 bytes (as 8
base64-encoded characters) of the SHA-256 hash of ther source code
revision and compiler flags, etc, but will probably not have a great
need to distribute binary distributions. Each implementation's community
may decide how to best use the ABI tag.

Platform Tag
------------

The platform tag is simply `distutils.util.get_platform()` with all
hyphens `-` and periods `.` replaced with underscore `_`.

-   win32
-   linux\_i386
-   linux\_x86\_64

Use
===

The tags are used by installers to decide which built distribution (if
any) to download from a list of potential built distributions. The
installer maintains a list of (pyver, abi, arch) tuples that it will
support. If the built distribution's tag is `in` the list, then it can
be installed.

It is recommended that installers try to choose the most feature
complete built distribution available (the one most specific to the
installation environment) by default before falling back to pure Python
versions published for older Python releases. Installers are also
recommended to provide a way to configure and re-order the list of
allowed compatibility tags; for example, a user might accept only the
`*-none-any` tags to only download built packages that advertise
themselves as being pure Python.

Another desirable installer feature might be to include "re-compile from
source if possible" as more preferable than some of the compatible but
legacy pre-built options.

This example list is for an installer running under CPython 3.3 on a
linux\_x86\_64 system. It is in order from most-preferred (a
distribution with a compiled extension module, built for the current
version of Python) to least-preferred (a pure-Python distribution built
with an older version of Python):

1.  cp33-cp33m-linux\_x86\_64
2.  cp33-abi3-linux\_x86\_64
3.  cp3-abi3-linux\_x86\_64
4.  cp33-none-linux\_x86\_64\*
5.  cp3-none-linux\_x86\_64\*
6.  py33-none-linux\_x86\_64\*
7.  py3-none-linux\_x86\_64\*
8.  cp33-none-any
9.  cp3-none-any
10. py33-none-any
11. py3-none-any
12. py32-none-any
13. py31-none-any
14. py30-none-any

-   Built distributions may be platform specific for reasons other than
    C extensions, such as by including a native executable invoked as a
    subprocess.

Sometimes there will be more than one supported built distribution for a
particular version of a package. For example, a packager could release a
package tagged `cp33-abi3-linux_x86_64` that contains an optional C
extension and the same distribution tagged `py3-none-any` that does not.
The index of the tag in the supported tags list breaks the tie, and the
package with the C extension is installed in preference to the package
without because that tag appears first in the list.

Compressed Tag Sets
===================

To allow for compact filenames of bdists that work with more than one
compatibility tag triple, each tag in a filename can instead be a
'.'-separated, sorted, set of tags. For example, pip, a pure-Python
package that is written to run under Python 2 and 3 with the same source
code, could distribute a bdist with the tag `py2.py3-none-any`. The full
list of simple tags is::

    for x in pytag.split('.'):
        for y in abitag.split('.'):
            for z in archtag.split('.'):
                yield '-'.join((x, y, z))

A bdist format that implements this scheme should include the expanded
tags in bdist-specific metadata. This compression scheme can generate
large numbers of unsupported tags and "impossible" tags that are
supported by no Python implementation e.g. "cp33-cp31u-win64", so use it
sparingly.

FAQ
===

What tags are used by default? Tools should use the most-preferred
architecture dependent tag e.g. `cp33-cp33m-win32` or the most-preferred
pure python tag e.g. `py33-none-any` by default. If the packager
overrides the default it indicates that they intended to provide
cross-Python compatibility.

What tag do I use if my distribution uses a feature exclusive to the
newest version of Python? Compatibility tags aid installers in selecting
the *most compatible* build of a *single version* of a distribution. For
example, when there is no Python 3.3 compatible build of
`beaglevote-1.2.0` (it uses a Python 3.4 exclusive feature) it may still
use the `py3-none-any` tag instead of the `py34-none-any` tag. A Python
3.3 user must combine other qualifiers, such as a requirement for the
older release `beaglevote-1.1.0` that does not use the new feature, to
get a compatible build.

Why isn't there a `.` in the Python version number? CPython has lasted
20+ years without a 3-digit major release. This should continue for some
time. Other implementations may use \_ as a delimeter, since both - and
. delimit the surrounding filename.

Why normalise hyphens and other non-alphanumeric characters to
underscores? To avoid conflicting with the "." and "-" characters that
separate components of the filename, and for better compatibility with
the widest range of filesystem limitations for filenames (including
being usable in URL paths without quoting).

Why not use special character <X> rather than "." or "-"? Either because
that character is inconvenient or potentially confusing in some contexts
(for example, "+" must be quoted in URLs, "\~" is used to denote the
user's home directory in POSIX), or because the advantages weren't
sufficiently compelling to justify changing the existing reference
implementation for the wheel format defined in PEP 427 (for example,
using "," rather than "." to separate components in a compressed tag).

Who will maintain the registry of abbreviated implementations? New
two-letter abbreviations can be requested on the python-dev mailing
list. As a rule of thumb, abbreviations are reserved for the current 4
most prominent implementations.

Does the compatibility tag go into METADATA or PKG-INFO? No. The
compatibility tag is part of the built distribution's metadata. METADATA
/ PKG-INFO should be valid for an entire distribution, not a single
build of that distribution.

Why didn't you mention my favorite Python implementation? The
abbreviated tags facilitate sharing compiled Python code in a public
index. Your Python implementation can use this specification too, but
with longer tags. Recall that all "pure Python" built distributions just
use 'py'.

Why is the ABI tag (the second tag) sometimes "none" in the reference
implementation? Since Python 2 does not have an easy way to get to the
SOABI (the concept comes from newer versions of Python 3) the reference
implementation at the time of writing guesses "none". Ideally it would
detect "py27(d\|m\|u)" analogous to newer versions of Python, but in the
meantime "none" is a good enough way to say "don't know".

References
==========

.. \[1\] Egg Filename-Embedded Metadata
(http://peak.telecommunity.com/DevCenter/EggFormats\#filename-embedded-metadata)

.. \[2\] Creating Built Distributions
(http://docs.python.org/distutils/builtdist.html)

.. \[3\] PEP 3147 -- PYC Repository Directories
(http://www.python.org/dev/peps/pep-3147/)

Acknowledgements
================

The author thanks Paul Moore, Nick Coghlan, Marc Abramowitz, and
Mr. Michele Lacchia for their valuable help and advice.

Copyright
=========

This document has been placed in the public domain.

.. Local Variables: mode: indented-text indent-tabs-mode: nil
sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
