PYBUILD(7)

NAME
pybuild - Portmod Packaging Guide

BASIC PACKAGING

  Package Names and Versions

NOTE: Valid characters are displayed as a regular expression, so that they can be concisely specified while still being exhaustive. Brackets ([]) are part of the regex and are not considered valid characters in any context.

  Category Names

A category name may contain any of the characters [A-Za-z0-9+_.-]. It must not begin with a hyphen, a dot or a plus sign.

  Package Names

A package name may contain any of the characters [A-Za-z0-9+_-]. It must not begin with a hyphen or a plus sign, and must not end in a hyphen followed by anything matching the Version Syntax.

NOTE: A package name does not include the category. The term qualified package name is used where a category/package pair is meant.

  USE flag names

A USE flag name may contain any of the characters [A-Za-z0-9+_-]. It must begin with an alphanumeric character.

Underscores should be considered reserved for USE_EXPAND, as described in the USE Expand variables section of the USE flag guide.

  Repository Names

A repository name may contain any of the characters [A-Za-z0-9_-]. It must not begin with a hyphen. In addition, every repository name must also be a valid package name.

  License Names

A license name may contain any of the characters [A-Za-z0-9+_.-]. It must not begin with a hyphen, a dot or a plus sign.

  Keyword names

A keyword name may contain any of the characters [A-Za-z0-9_.-]. It must begin with an alphanumeric character. In contexts where it makes sense to do so, a keyword name may be prefixed by a tilde or a hyphen. In pybuild.Pybuild2.KEYWORDS, -* is also acceptable as a keyword.

Changed in version 2.6: Keywords were not validated in the past, and any value was accepted and used, whether or not it conformed to the above specification. Now, invalid keywords will cause errors.

Additionally, the . character was added to the above specification, to allow for keywords which include version numbers. This is for legacy purposes only, and it is recommended that you use the new keyword versioning system also introduced in Portmod 2.6.

  Version Syntax

A version may optionally begin with an epoch, in the form e[0-9]+ (an e, then one or more digits). This epoch is a packaging-only component used to indicate that a package has changed to a new versioning system which is incompatible with the old versioning system. If the epoch is omitted, the version is considered to be epoch e0. The epoch, when present, is separated from the rest of the version by a hyphen.

A version always must contain a number part, which is in the form [0-9]+(\.[0-9]+)* (an unsigned integer, followed by zero or more dot-prefixed unsigned integers). E.g. 1.2.3

This may optionally be followed by one of [a-z] (a lowercase letter).

This may be followed by zero or more of the suffixes _alpha, _beta, _pre, _rc or _p (in order of priority), each of which may optionally be followed by an unsigned integer. Suffix and integer count as separate version components.

This may optionally be followed by the suffix -r followed immediately by an unsigned integer (the “revision number”, used to indicate changes to the package which require rebuilding, when there are no changes to the upstream project). If this suffix is not present, it is assumed to be -r0.

E.g. Using all the components: e2-1.2.3a_alpha12-r3.

  External Versions

New in version 2.6.

A variant on the version syntax is used for "external" versions, that is, versions where packaging concepts like revisions and epochs aren't meaningful. These versions are identical to regular versions except that they cannot include revisions, epochs or the _p suffix.

E.g. 1.2.3a_alpha12.

  Version Comparison and Ordering

Generally speaking, versions are compared component by component, from left to right, with the left-most components being the most significant when differences are found between the versions.

This means that epoch differences always outweigh everything else, and revisions are always the last to have an effect.

Numeric version components are compared as individual integers if they have no leading zeroes. If one of the components being compared has a leading zero, lexicographical comparison is done instead (comparing each digit one by one).

E.g. 1.1 < 1.2 < 1.10 < 1.11

However 1.01 < 1.09 < 1.1 even in the second comparison, because there is a leading 0 and 0 < 1 (the 9 is not compared because the second version only has one digit in that component).

For the exact version comparison algorithm, see Section 3.3 of the Package Manager Specification. Note that epochs are not included in that algorithm, but are checked first and the package with the larger epoch is always the greater version.

  Version Specifier

New in version 2.6.

Version specifiers should consist of a list of versions with operators, and separated by commas. The specifier matches versions which match all conditions in the specifier.

E.g. >=1.0,<3.0 matches versions between 1.0 and 3.0, including 1.0, but not 3.0.

Available operators are listed below. Note that all operators except globstar should be written before the version they apply to, while globstar folows the version.

•   >: Greater than -- matches versions greater than the specified version.

•   <: Less than

•   >=: Greater than or equal to

•   <=: Less than or equal to

•   ==: Equals -- Matches versions which are exactly equal to the specified version.

•   !=: Not Equals -- Matches versions which are not exactly equal to the specified version.

•   *: Globstar -- Matches all versions starting with the specified version. Note that the version preceeding the star operator must still be valid. E.g. 1.0* is valid, but 1.0.* is not.

    Version specifiers are currently only used in KEYWORDS/ACCEPT_KEYWORDS for versioned architectures. They cannot be used in DEPEND or similar package variables, where a different syntax is used.

  External Resources

Ignoring slots and epochs, Portmod's version specifiers are identical to Portage's version specifiers. The following Gentoo resources may be helpful.

•   https://wiki.gentoo.org/wiki/Version_specifier

•   https://devmanual.gentoo.org/ebuild-writing/file-format/index.html#file-naming-rules.

    Portmod's package names and versions generally follow Section 3 of the Package Manager Specification, with some exceptions. Excerpts of this section have been copied verbatim and are licensed CC-BY-SA-3.0.

  Manifest

In the tree, every package has a Manifest file. The Manifest file contains various hashes and file size data for every external source that is to be fetched. This is used primarily to verify the integrity of external files.

  Generating the Manifest

To generate the Manifest, use inquisitor manifest foo.pybuild. When new sources are added or removed, the Manifest must be regenerated.

  Structure

Manifest files are plain text files with the following format:

<type> <filename> <size> <hash-type> <hash> [<hash-type> <hash> ...]

  Members
type | The type of the file. Supported types are DIST (for remote package sources) and LINK and MISC (used internally for CONTENTS files in the package DB).
filename | The name of the file the manifest entry references.
size | The size of the file as a decimal number, in bytes
hash-type | One of the supported hash types listed below
hash | The hash of the file as a hexadecimal number, matching the preceeding hash type.

  Supported Hash Types

The hashes currently supported by portmod are:

•   BLAKE2B

•   BLAKE3 (recommended for performance)

•   MD5

•   SHA256 (SHA-2)

•   SHA512 (SHA-2)

  External Resources

https://wiki.gentoo.org/wiki/Repository_format/package/Manifest

  Dependencies

One of portmod's most useful features is automated dependency resolution. You can mark packages as requiring or conflicting with other packages, as well as having these relationthips be conditional on both the configuration of the package and the packages being depended on.

All dependency fields support both Use-conditional Expressions and the || (at-least-one-of) operator.

E.g. || ( cat/foo cat/bar ) will require at at least one of cat/foo and cat/bar must be installed, but both may also be installed simultaneously.

Packages can also depend on specific versions and use-flag configurations of other packages. E.g. >=cat/foo-1.0[bar] requires a version greater than 1.0, and that the bar use flag is set on the other package.

Version requirements are written by prepending the package name with an operator, and appending a version (separated from the name with a hyphen). Valid operators for version dependencies are the following:

•   =: Only matches packages with a version, including package revision exactly equal to the specified version (note that packages are implicitly r0 if a revision is not specified, so =cat/foo-1.0-r0 is equivalent to =cat/foo-1.0.

•   ~: Only matches packages with a version exactly equal to the specified version, ignoring package revisions.

•   >=: Only matches packages with a version that is greater than or equal to the specified version.

•   <=: Only matches packages with a version that is less than or equal to the specified version.

•   >: Only matches packages with a version that is greater than the specified version (but not equal to).

•   <: Only matches packages with a version that is less than the specified version (but not equal to).

    See pybuild.Pybuild2.DEPEND for further details about the format used by both RDEPEND and DEPEND.

  Runtime Dependencies

Most mod dependencies are runtime dependencies, that is, they are dependencies that must be satisfied eventually (so that the game can run), but may not need to be satisfied for package installation.

Runtime dependencies should be specified in pybuild.Pybuild2.RDEPEND.

  Build Dependencies

Unlike software package managers, build dependencies are less frequently used by mods, as they are usually packaged so that they can be installed without changes to their files, however build dependencies are still useful if mods require tools to patch them prior to installation.

Build dependencies differ from runtime dependencies in that they ensure that packages will have these dependencies satisfied before the package is installed. In addition, any runtime dependencies of a package's build dependencies will be satisfied before package installation begins.

Build dependencies should be specified in pybuild.Pybuild2.DEPEND, which also includes format details.

  External Resources

Portmod's dependency specification is heavily based on Gentoo/Portage's specification, documented both on the Gentoo Developer Manual and the Package Manager Specification Section 8.

  Use Flag Guide

Use flags can be used to control optional dependencies and optional configurations which a user may want to choose between.

Use flags should be declared in IUSE and described using the use top level field in the mod’s metadata.yaml.

E.g.

Build File

...
class Package(Pybuild1):
    IUSE = "+foo bar"
    RDEPEND = """
        foo? ( !!cat/bar cat/foo )
        bar? ( !!cat/foo cat/bar )
    """
    REQUIRED_USE = "?? ( foo bar )"
...

metadata.yaml

use:
    foo: description of foo
    bar: description of bar

  IUSE

In the above example, we declare flags foo (enabled by default by the + prefix) and bar (disabled by default).

See pybuild.Pybuild2.IUSE for more details.

  Use-conditional Expressions

Use flags can be used in use-conditional expressions, that is, either flag? or !flag?, followed by brackets to indicate the scope of the conditional. DEPEND, RDEPEND and SRC_URI all support including use-conditional expressions.

E.g. foo? ( !!cat/bar cat/foo ) in the above example is a use-conditional expression that indicates that if the foo flag is enabled, the !!cat/bar package is blocked and cannot be installed, and the cat/foo package must be present at runtime. On the other hand, if the user enables bar, the second use-conditional bar? ( !!cat/foo cat/bar ) makes the reverse occur.

See pybuild.Pybuild2.DEPEND (same format as RDEPEND) for more details about the specific syntax used in DEPEND/RDEPEND.

  REQUIRED_USE

Use flags can also be used in the pybuild.Pybuild2.REQUIRED_USE field. REQUIRED_USE, in addition to use-conditional expressions, allows use-flags to be included on the right side of a use-conditional, and supports the ^^ (exactly-one-of) and ?? (at-most-one-of) operators, in addition to the || (at-least-one-of) operator supported by DEPEND/RDEPEND (see Dependencies).

RDEPEND we see that if the user enables flag foo, the mod will pull in cat/foo as a dependency, and block cat/bar from being installed.

We then declare in REQUIRED_USE, that at most one of foo and bar can be set at once (?? allows zero or one of the flags in the set to be enabled). You might see a configuration like this for a mod that has two mutually exclusive optional dependencies.

Note that the pybuild.InstallDir object as well as the pybuild.File object, also have REQUIRED_USE fields. In their cases, the InstallDir, or File will only be included if REQUIRED_USE is satisfied. You could, for example, disable an InstallDir when a use flag is not enabled by including that use flag in its REQUIRED_USE field.

  Global and Local Use Flags

Global use flags are described in the /profiles/use.yaml file and should not be included in a mod’s specific metadata.yaml file. They do however need to be included in IUSE. If a use flag is to be made global, it should be used by multiple mods for more or less the same purpose.

All use flags that are not declared in /profiles/use.yaml, and are not use expand flags (see below), are considered to be local use flags.

  Use Flag Defaults

You can add a + before a flag in pybuild.Pybuild2.IUSE to enable it by default. This affects both local and global flags, though only for this package in particular.

Defaults can also be configured for each profile by modifying the USE variable in the profile (for global flags), or setting the flag for specific packages in package.use.

  Use Flag Aliases

Use flags can be set up to default to enabled when a particular package is installed, or disabled when the package is not installed (or going to be installed).

This can be useful for configuring patches for mods which are only necessary to allow it to work with another mod.

To create a use flag alias, add the flag and the package to profiles/use.alias.yaml (also see Package Repositories).

E.g. profiles/use.alias.yaml

tr: landmasses/tamriel-rebuilt

Use flag aliases are always global use flags, and do not need to be added to use.yaml.

NOTE: This is a loose relationship. Users can still override aliased flags, the feature just removes the need to update package.use when a package is installed or removed. This means that you must still add the relevant dependencies for the use flags, which usually come in one of two forms:

Mandatory patches: when a flag is used to enable a patch which otherwise causes a conflict. In this case it is necessary to also add a blocker to the dependency string. E.g.

RDEPEND = """
   tr? ( >=landmasses/tamriel-rebuilt-8 )
   !tr? ( !!landmasses/tamriel-rebuilt )
"""

Optional patches: when a flag is used to enable a patch which adds behaviour to match or extend another package, but would not cause issues if it is not installed. E.g.

RDEPEND = """
   tr? ( landmasses/tamriel-rebuilt )
"""

  USE Expand variables

USE Expand variables are a certain class of automatically generated use flags set by the user’s global configuration. Every variable specified in the USE_EXPAND profile variable causes the values of the profile variable of the same name to generate use flags of the form {use-expand-name}_{value}.

Supported flags should then be described in /profiles/desc/{use}.yaml (where {use} is the lowercased use flag group name).

E.g. profiles/base/defaults.conf (path varies depending on profile setup)

USE_EXPAND = "SCREEN_ASPECT"

E.g. profiles/desc/screen_aspect.yaml

4x3: Use 4x3 screen aspect ratio
16x9: Use 16x9 screen aspect ratio
16x10: Use 16x10 screen aspect ratio

This produces the flags screen_aspect_4x3, screen_aspect_16x9 and screen_aspect_16x10, which can be enabled by the user declaring something such as the following in their config file:

SCREEN_ASPECT = "16x9"

  TEXTURE_SIZES

The TEXTURE_SIZE variable is a special variable which behaves similarly to use-expand variables, but has two special properties

1.  Values for a package are declared in TEXTURE_SIZES, instead of IUSE, and are implicitly added to REQUIRED_USE as part of a ^^ (exactly-one-of) relationship.

2.  The flag enabled is determined using the user's TEXTURE_SIZE configuration variable.

    They can be used in use-conditionals and REQUIRED_USE just like other use flags.

  Use Flag Priority

Use flags values can be defined in several different places. The priority for the different ways of specifying a value is as follows, from lowest priority (always overridden), to highest priority (never overridden).

•   pybuild.Pybuild2.IUSE Default (see Use Flag Defaults).

•   Global profile flags set in defaults.conf. This includes USE as well as use-expand variables.

•   Global user flags set in portmod.conf. This includes USE as well as use-expand variables.

•   Local (i.e. package-specific) profile flags set in the profile via package.use.

•   Local user flags set in the user's package.use.

•   Flags set via environment variables, including both USE and use-expand variables.

•   Forced flags set in use.force and package.use.force.

    Generally speaking, the IUSE default is only useful for flags which are specific to a certain package. For use flags which are shared between many packages with a consistent meaning, the default should be set globally in the profile.

    The profile package.use should only need to be used for flags which are set globally in the profile, but need to be overridden for a particular package (since the IUSE default cannot be used in such cases).

  External Resources

https://wiki.gentoo.org/wiki/USE_flag https://devmanual.gentoo.org/general-concepts/use-flags/index.html

  Package Archives

  Pybuild2.unpack

pybuild.Pybuild2.unpack() supports unpacking archives via shutil.unpack_archive().

This includes support for the following formats:

•   zip: ZIP file.

•   tar: uncompressed tar file.

•   gztar: gzip’ed tar-file

•   bztar: bzip2’ed tar-file

•   xztar: xz’ed tar-file

  Pybuild1.unpack

pybuild.Pybuild1.unpack() supports unpacking archives via patool

Primary archive formats:

•   zip

•   tar, tar.gz

•   7z (requires 7zip, or p7zip, to be installed)

    Secondary Archive Formats:

•   rar (proprietary format which is only supported through the proprietary rar/unrar programs, as well as a non-free extension to 7zip. [1]).

•   tar.bz2 (has issues on windows due to the builtin windows tar command not properly supporting it).

•   tar.xz (has had some issues related to detecting archives, since fixed in the latest releases of patool).

[1] Free implementations exist in libarchive, and unar, but these are not supported by patool. libarchive's support is also incomplete (some archives fail to extract with the message "Parsing filters is unsupported").

  pybuild package

  Submodules
pybuild.info | Importing values from this module fills them with information about the pybuild file from which they were imported
pybuild.winreg | Helper functions for interacting with the Windows registry

  pybuild.info module

Importing values from this module fills them with information about the pybuild file from which they were imported

Data:
CATEGORY | The package's category.
P | The package name and version.
PF | The package name with version and revision.
PN | The package name without version.
PR | The package's revision
PV | The package's version without revision
PVR | The package's version and revision

pybuild.info.CATEGORY: str = ''
    The package's category.

    E.g. base

pybuild.info.P: str = ''
    The package name and version.

    E.g.: example-suite-1.0

pybuild.info.PF: str = ''
    The package name with version and revision.

    E.g.: example-suite-1.0-r1

pybuild.info.PN: str = ''
    The package name without version.

    E.g.: example-suite

pybuild.info.PR: str | None = None
    The package's revision

    E.g. r1

pybuild.info.PV: str = ''
    The package's version without revision

    E.g. 1.0

pybuild.info.PVR: str = ''
    The package's version and revision

    E.g. 1.0-r1

  pybuild.winreg module

Helper functions for interacting with the Windows registry

Also provides the following constants from the winreg builtin module:

HKEY_CLASSES_ROOT: int
HKEY_CURRENT_CONFIG: int
HKEY_CURRENT_USER: int
HKEY_LOCAL_MACHINE: int
HKEY_USERS: int

Functions:
read_reg(key, subkey[, entry]) | Reads the given registry key/subkey

pybuild.winreg.read_reg(key, subkey, entry=None)
    Reads the given registry key/subkey

Parameters

•   key (str) -- Registry key to read from

•   subkey (str) -- Registry subkey to read from

•   entry (Optional[str]) -- Optional name in the dictionary stored at the given key/subkey to use.

Return type
    Union[Any, Dict[str, Any]]

Returns
    Key data, type is usually a string. If the key contains subkeys and the entry is not provided, returns a dictionary mapping subkey names to their values

  Module contents

The module accessible within pybuilds

Note that this module should not be imported outside of pybuild files

Classes:
File(NAME[, REQUIRED_USE, OVERRIDES]) | Represents important installed files and their metadata
InstallDir(PATH[, REQUIRED_USE, PATCHDIR, ...]) | Represents a directory in the Virtual File System
Pybuild1() | Legacy class.
Pybuild2() | The class all Pybuilds should derive.

Functions:
apply_patch(patch) | Applies git patch using Git apply
find_file(name) | Locates the path of a file within the OpenMW virtual file system
get_masters(file) | Detects masters for the given file
list_dir(name) | Locates all path of files matching the given pattern within the OpenMW virtual file system
patch_dir(src, dst, *[, overwrite, ignore, ...]) | Copies src ontop of dst
use_reduce(depstr[, uselist, masklist, ...]) | Takes a dep string and reduces the use? conditionals out, leaving an array with subarrays.
version_gt(version1, version2) | Version comparision function

class pybuild.File(NAME, REQUIRED_USE='', OVERRIDES=[], **kwargs)
    Bases: object

    Represents important installed files and their metadata

    Deprecated since version 2.4: It will be removed in Portmod 3.0

Parameters

•   NAME (str) --

•   REQUIRED_USE (str) --

•   OVERRIDES (Union[str, List[str]]) --

    Attributes:
    NAME | Name of the file relative to the root of the InstallDir
    OVERRIDES | A list of files which this overrides when sorting (if applicable).
    REQUIRED_USE | Requirements for installing this file

NAME: str
    Name of the file relative to the root of the InstallDir

OVERRIDES: Union[str, List[str]]
    A list of files which this overrides when sorting (if applicable).

    Can either be in the form of a string containing use-conditionals (note that this does not support files that contain spaces) or a list of files to override. Note that these overridden files are not considered masters and do not need to be present.

    For archives it determines the order in which the fallback archives will be searched during VFS lookups.

REQUIRED_USE: str
    Requirements for installing this file

    The default empty string is always satisfied. See Pybuild2.REQUIRED_USE for details on the syntax.

class pybuild.InstallDir(PATH, REQUIRED_USE='', PATCHDIR='.', S=None, WHITELIST=None, BLACKLIST=None, RENAME=None, DATA_OVERRIDES='', ARCHIVES=(), VFS=None, DOC=(), **kwargs)
    Bases: object

    Represents a directory in the Virtual File System

    Note that arbitrary arguments can be passed to the constructor, as repositories may make use of custom information. See the repository-level documentation for such information.

    Deprecated since version 2.4: It will be removed in Portmod 3.0

Parameters

•   PATH (str) --

•   REQUIRED_USE (str) --

•   PATCHDIR (str) --

•   S (Optional[str]) --

•   WHITELIST (Optional[List[str]]) --

•   BLACKLIST (Optional[List[str]]) --

•   RENAME (Optional[str]) --

•   DATA_OVERRIDES (str) --

•   ARCHIVES (Iterable[File]) --

•   VFS (Optional[bool]) --

•   DOC (Iterable[str]) --

    Attributes:
    ARCHIVES | A list of File objects representing VFS archives.
    BLACKLIST | If present, does not install files matching the patterns in this list.
    DATA_OVERRIDES | A list of packages that this InstallDir should override in the VFS
    DOC | A list of patterns matching documentation files within the package
    PATCHDIR | The destination path of the InstallDir within the package's directory.
    PATH | The path to the data directory that this InstallDir represents relative to the root of the archive it is contained within.
    RENAME | Destination path of this directory within the final directory.
    REQUIRED_USE | A list of use flags with the same format as the package's REQUIRED_USE variable which enable the InstallDir if satisfied.
    S | The source directory corresponding to this InstallDir.
    VFS | Whether or not this InstallDir gets added to the VFS
    WHITELIST | If present, only installs files matching the patterns in this list.

    Methods:
    get_files() | Generator function yielding file subattributes of the installdir

ARCHIVES: List[File]
    A list of File objects representing VFS archives.

    These will be searched, in order, during VFS file lookups if the file is not present in the package directories.

BLACKLIST: Optional[List[str]]
    If present, does not install files matching the patterns in this list. fnmatch-style globbing patterns (e.g. * and [a-z]) can be used

DATA_OVERRIDES: str
    A list of packages that this InstallDir should override in the VFS

    This only has a different effect from Pybuild1.DATA_OVERRIDES if multiple PATCHDIRs are set, as it can define overrides for individual PATCHDIRS, while Pybuild1.DATA_OVERRIDES affects all PATCHDIRs. See Pybuild1.DATA_OVERRIDES for details of the syntax.

DOC: List[str]
    A list of patterns matching documentation files within the package

    This documentation will be installed separately fnmatch-style globbing patterns (e.g. * and [a-z]) can be used.

PATCHDIR: str
    The destination path of the InstallDir within the package's directory.

    Defaults to ".", i.e. the root of the mod directory. If multiple InstallDirs share the same PATCHDIR they will be installed into the same directory in the order that they are defined in the INSTALL_DIRS list. Each unique PATCHDIR has its own entry in the VFS, and its own sorting rules

PATH: str
    The path to the data directory that this InstallDir represents relative to the root of the archive it is contained within.

RENAME: Optional[str]
    Destination path of this directory within the final directory.

    E.g.:

    InstallDir("foo/bar", PATCHDIR=".", RENAME="bar")

    Will install the contents of foo/bar (in the source) into the directory bar inside the package's installation directory (and also the VFS).

REQUIRED_USE: str
    A list of use flags with the same format as the package's REQUIRED_USE variable which enable the InstallDir if satisfied. Defaults to an empty string that is always satisfied.

S: Optional[str]
    The source directory corresponding to this InstallDir.

    Similar function to S for the entire pybuild, this determines which directory contains this InstallDir, and generally corresponds to the name of the source archive, minus extensions. This is required for packages that contain more than one source, but is automatically detected for those with only one source if it is not specified, and will first take the value of Pybuild2.S, then the source's file name without extension if the former was not defined.

VFS: Optional[bool]
    Whether or not this InstallDir gets added to the VFS

    Defaults to the value of the VFS variable in the profile configuration

WHITELIST: Optional[List[str]]
    If present, only installs files matching the patterns in this list. fnmatch-style globbing patterns (e.g. * and [a-z]) can be used

get_files()
    Generator function yielding file subattributes of the installdir

class pybuild.Pybuild1
    Bases: Pybuild2

    Legacy class. Superseded by Pybuild2

    Deprecated since version 2.4: Pybuild2 should be used instead. This will be removed in Portmod 3.0

    Attributes:
    DATA_OVERRIDES | A use-reduce-able list of atoms indicating packages whose data directories should come before the data directories of this package when sorting data directories.
    INSTALL_DIRS | The INSTALL_DIRS variable consists of a python list of InstallDir objects.
    REBUILD_FILES | Files in the VFS which, if changed, should cause this package to be rebuilt
    TIER | The Tier of a package represents the position of its data directories and plugins in the virtual file system.

    Methods:
    get_files(typ) | Returns all enabled files and their directories
    src_install() | The src_install function installs the package’s content to a directory specified in Pybuild2.D.
    unpack(archives) | Unpacks the given archive into the workdir

A: List[Source]
    The list of enabled sources [1]

    Scope: All except __init__

ATOM: FQAtom

CATEGORY: str
    The package's category. [1]

    E.g. base

CP: QualifiedAtom

CPN: QualifiedAtom

D: str
    The full path of the directory where the package is to be installed. [1]

    Note that this is a temporary directory and not the final install location.

    Scope: src_install

DATA_OVERRIDES = ''
    A use-reduce-able list of atoms indicating packages whose data directories should come before the data directories of this package when sorting data directories.

    They do not need to be dependencies. Blockers (atoms beginning with !!) can be used to specify underrides, and use dependencies (e.g. the [bar] in foo[bar]) can be used to conditionally override based on the target atom's flag configuration.

    Not included in PMS

FILE: str

FILESDIR: str
    Path of the directory containing additional repository files

    Scope: src_*

INSTALL_DIRS: List[InstallDir] = []
    The INSTALL_DIRS variable consists of a python list of InstallDir objects.

    E.g.:

    INSTALL_DIRS=[
        InstallDir(
            'Morrowind/Data Files',
            REQUIRED_USE='use use ...',
            DESTPATH='.',
            PLUGINS=[File('Plugin Name',
                REQUIRED_USE='use use ...', satisfied
            )],
            ARCHIVES=[File('Archive Name')],
            S='Source Name Without Extension',
        )
    ]

    Not included in PMS

P: Atom
    The package name and version. [1]

    E.g.: example-suite-1.0

PF: Atom
    The package name with version and revision. [1]

    E.g.: example-suite-1.0-r1

PN: Atom
    The package name without version. [1]

    E.g.: example-suite

PR: str
    The package's revision [1]

    E.g. r1

    Is equal to r0 is no revision is specified

PV: str
    The package's version without revision [1]

    E.g. 1.0

PVR: str
    The package's version and revision [1]

    E.g. 1.0-r1

REBUILD_FILES: List[str] = []
    Files in the VFS which, if changed, should cause this package to be rebuilt

    Can include glob-style patterns using the , ? and [] operators. See https://docs.python.org/3/library/fnmatch.html. Unlike normal fnmatch parsing, wildcards () will not match accross path separators.

    This field can be modified during installation, and will only be used after the package has been installed.

REPO: str

REPO_PATH: Optional[str]

ROOT: str
    The full path of the prefix root where packages will be installed

    Note: This functions as both ROOT and SYSROOT (as defined by PMS section 11.1).

    Scope: src_*, pkg_*

T: str
    Path to a temporary directory which may be used during packaging [1]

    Scope: All except __init__

TIER = 'a'
    The Tier of a package represents the position of its data directories and plugins in the virtual file system.

    This is used to group packages in such a way to avoid having to individually specify overrides whenever possible.

    The value is either in the range [0-9] or [a-z].

    Default value: 'a'

    Tier 0 represents top-level mods such as morrowind Tier 1 is for mods that replace or modify top-level mods. E.g. texture and mesh replacers. Tier 2 is for large mods that are designed to be built on top of by other mods, such as Tamriel Data Tier a is for all other mods. Tier z is for mods that should be installed or loaded last. E.g. omwllf The remaining tiers are reserved in case the tier system needs to be expanded

    Not included in PMS

UNFETCHED: List[Source]
    The list of sources which need to be fetched [1]

    Scope: pkg_nofetch

USE: Set[str]
    Enabled use flags [1]

    Scope: All except __init__

WORKDIR: str
    The directory where packaging takes place [1]

    Scope: src_*

get_files(typ)
    Returns all enabled files and their directories

Parameters
    typ (str) --

Return type
    Generator[Tuple[InstallDir, File], None, None]

src_install()
    The src_install function installs the package’s content to a directory specified in Pybuild2.D.

    The initial working directory is Pybuild2.S, falling back to Pybuild.WORKDIR if the directory does not exist.

    The default implementation used when the package lacks the src_install function moves each InstallDir in Pybuild1.INSTALL_DIRS which is not hidden due to an unsatisfied REQUIRED_USE into Pybuild2.D.

unpack(archives)
    Unpacks the given archive into the workdir

    Uses patool as its backend.

Parameters
    archives (Union[str, Iterable[Union[Source, str]]]) -- The list of archives to be unpacked

version: Version

class pybuild.Pybuild2
    Bases: FullPybuild

    The class all Pybuilds should derive.

    The name and path of a pybuild declares the package name, category, version and (optionally) revision:

    {CATEGORY}/{PKG_NAME}-{VER}(-r{REV}).pybuild

    Categories and package names may contain lower case letters, numbers and hyphens. Versions may contain numbers and dots. Revisions may only contain numbers (following the -r prefix). (See the PMS for the complete naming scheme).

    Note that revisions refer to revisions of the pybuild itself, not the package, and are used to indicate that the way the mod is configured has changed in a way that will impact the installed version. For changes, such as the source files moving, that would not impact a mod that is already installed, you do not need to update the revision.

    There are certain fields which are defined automatically and may only be available in some scopes:
    Variable | Scope
    Pybuild2.P | All scopes
    Pybuild2.PF | All scopes
    Pybuild2.PN | All scopes
    Pybuild2.CATEGORY | All scopes
    Pybuild2.PV | All scopes
    Pybuild2.PVR | All scopes
    Pybuild2.USE | All scopes except __init__
    Pybuild2.WORKDIR | src_*
    Pybuild2.T | All scopes except __init__
    Pybuild2.D | Pybuild2.src_install()
    Pybuild2.FILESDIR | src_*
    Pybuild2.ROOT | src_*, pkg_*
    Pybuild2.A | src_*, pkg_nofetch
    Pybuild2.UNFETCHED | pkg_nofetch
    Pybuild2.S | src_*

[1] Fields which are set automatically and should not be defined in the package file. Described in the table above.

    Attributes:
    A | The list of enabled sources [1]
    CATEGORY | The package's category.
    D | The full path of the directory where the package is to be installed.
    DEPEND | Build dependencies.
    DESC | A short description of the package.
    DOCS | A list of documentation patterns for the default src_install to install using Pybuild2.dodoc
    FILESDIR | Path of the directory containing additional repository files
    HOMEPAGE | The URL of the package's homepage(s).
    IUSE | A field containing a space-separated list of the use flags used by the package.
    KEYWORDS | Keywords indicating compatibility.
    LICENSE | One or more licenses used by the package.
    NAME | Descriptive package name.
    P | The package name and version.
    PATCHES | A list of patch files stored within the package's files directory in the repository
    PF | The package name with version and revision.
    PN | The package name without version.
    PR | The package's revision [1]
    PROPERTIES | A white-space-delimited list of additional properties of the given pybuild to enable special behaviour.
    PV | The package's version without revision [1]
    PVR | The package's version and revision [1]
    RDEPEND | Runtime dependencies.
    REQUIRED_USE | An expression indicating valid combinations of use flags.
    RESTRICT | Lists features which should be disabled for this package
    ROOT | The full path of the prefix root where packages will be installed
    S | Specifies the default working directory for src_* functions.
    SRC_URI | A List of sources to be fetched.
    T | Path to a temporary directory which may be used during packaging [1]
    TEXTURE_SIZES | A field declaring the texture size options that the package supports.
    UNFETCHED | The list of sources which need to be fetched [1]
    USE | Enabled use flags [1]
    WORKDIR | The directory where packaging takes place [1]

    Methods:
    can_update_live() | Indicates whether or not a live package can be updated.
    dodoc(pattern) | Installs documentation matching the given pattern into the image directory (Self.D)
    execute(command[, pipe_output, pipe_error]) | Allows execution of arbitrary commands at runtime.
    get_installed_env() | Returns a dictionary containing installed object values
    info(string) | Displays info message both immediately, and in the summary after all transactions have been completed
    pkg_postinst() | Function called immediately after package installation
    pkg_prerm() | Function called immediately before package removal
    pkg_pretend() | May be used to carry out sanity checks early on in the install process
    src_install() | The src_install function installs the package’s content to a directory specified in Pybuild2.D.
    src_prepare() | The src_prepare function can be used for post-unpack source preparation.
    src_unpack() | The src_unpack function extracts all of the package’s sources.
    unpack(archives) | Unpacks the given archive into the workdir
    validate() | (Since Portmod 2.4) inquisitor will call this function when scanning repositories.
    warn(string) | Displays warning message both immediately, and in the summary after all transactions have been completed

A: List[Source]
    The list of enabled sources [1]

    Scope: All except __init__

ATOM: FQAtom

CATEGORY: str
    The package's category. [1]

    E.g. base

CP: QualifiedAtom

CPN: QualifiedAtom

D: str
    The full path of the directory where the package is to be installed. [1]

    Note that this is a temporary directory and not the final install location.

    Scope: src_install

DEPEND: str = ''
    Build dependencies.

    The DEPEND field is used to specify packages which need to be installed in order for this package to install correctly.

    Most mods do not have build dependencies, however mods that require patching using tools external to portmod, or packages that generate content from other sources, will need to include their masters, or the other sources, as build dependencies, to ensure that they are installed prior to the package being installed.

    Format (both DEPEND and RDEPEND): A list of dependencies in the form of package atoms. All dependencies should include both category and package name. Versions should also be included if the package depends on a specific version of another mod. It is recommended not to include a version number in the dependency unless it is known that the package will not work with other versions.

    Ranges of versions can be indicated by prefixing >,<,<=,>= to the atoms. E.g. >=cat/foo-1.0

    Specific versions can be indicated by prefixing = (matches version and revision exactly) or ~ (matches version, but allows any revision) to the atoms. E.g. =cat/foo-1.0

    Use flag dependencies can be specified in the following manner:

•   cat/foo[flag] - Indicates that flag must be enabled

•   cat/foo[flag,flag2] - Indicates that both flag and flag2 must be enabled

•   cat/foo[-flag] - Indicates that flag must be disabled

•   cat/foo[flag?] - Indicates that flag must be enabled if it is enabled for this package

•   cat/foo[!flag?] - Indicates that flag must be disabled if it is enabled for this package

    Atoms can be surrounded by use-conditionals if they are only dependencies when that use flag is enabled/disabled.

    E.g. flag? ( cat/foo )

    Atoms can be grouped and prefixed by a || operator to indicate that any of the given packages will satisfy the dependency.

    E.g. || ( cat/foo cat/bar cat/baz )

    Note that it is required that the parentheses ( ) are separated from the atoms by whitespace.

    Packages which cannot be installed at the same time can be marked as blocks using the !! operator. I.e. !!cat/foo indicates that cat/foo cannot be installed at the same time as the current package.

DESC: str = ''
    A short description of the package.

    Is may (depending on options provided) be used in searches. Note that a longer description can be provided in metadata.yaml.

DOCS: List[str] = ['README*', 'readme*', 'ReadMe*', 'ChangeLog', 'CHANGELOG*', 'AUTHORS*', 'NEWS*', 'TODO*', 'CHANGES*', 'THANKS*', 'BUGS*', 'FAQ*', 'CREDITS*', 'Doc/*', 'doc/*', 'docs/*', 'Docs/*']
    A list of documentation patterns for the default src_install to install using Pybuild2.dodoc

FILE: str

FILESDIR: str
    Path of the directory containing additional repository files

    Scope: src_*

HOMEPAGE: str = ''
    The URL of the package's homepage(s).

    Used for descriptive purposes and included in search results.

IUSE: Union[Set[str], str] = ''
    A field containing a space-separated list of the use flags used by the package.

    IUSE should contain all regular use flags used by this package, both local and global. Prefixing the use flags with a + means that the option is enabled by default. Otherwise use flags are disabled by default.

    Note that you do not need to include TEXTURE_SIZES type flags in IUSE, but USE_EXPAND variables should be included in IUSE.

KEYWORDS: str = ''
    Keywords indicating compatibility. Existence of the keyword indicates that the mod is stable on that platform.

    a ~ in front of the keyword indicates that the mod is unstable on that platform no keyword indicates that the mod is untested on that platform a - in front of the keyword indicates that the mod is known to not work on that platform

    E.g. A package that works on OpenMW but does not on tes3mp:

    KEYWORDS='openmw -tes3mp'

    Keywords can optionally be followed by a Version Specifier surrounded by {}. This requires setting up version detection so that the ARCH_VERSION variable is available. See Architecture Versioning for details.

    E.g.

    KEYWORDS="openmw{>=0.48.0}"

LICENSE: str = ''
    One or more licenses used by the package.

    A list of licenses can be found in the licenses directory of the repository.

NAME: str = ''
    Descriptive package name.

    The package name used for identification is the name used in the filename, however this name is included when searching for packages.

P: Atom
    The package name and version. [1]

    E.g.: example-suite-1.0

PATCHES: str = ''
    A list of patch files stored within the package's files directory in the repository

    Note that unlike as specified in the PMS, their paths must be relative to the files directory.

    See apply_patch() for details on the supported patch format.

PF: Atom
    The package name with version and revision. [1]

    E.g.: example-suite-1.0-r1

PN: Atom
    The package name without version. [1]

    E.g.: example-suite

PR: str
    The package's revision [1]

    E.g. r1

    Is equal to r0 is no revision is specified

PROPERTIES: str = ''
    A white-space-delimited list of additional properties of the given pybuild to enable special behaviour.

    Possible values are given below:

•

live: Indicates that the pybuild doesn't have a specific version (e.g. if
    installing from a git repository branch but not using a specific commit). Live pybuilds should have an empty KEYWORDS list, as stability testing is not meaningful if the upstream source is changing. Live packages must override Pybuild2.can_update_live().

•   local: Only used internally to refer to Local mods with generated metadata

PV: str
    The package's version without revision [1]

    E.g. 1.0

PVR: str
    The package's version and revision [1]

    E.g. 1.0-r1

RDEPEND: str = ''
    Runtime dependencies.

    Is used to specify packages which are required at runtime for this package to function. The format is the same as for DEPEND

REPO: str

REPO_PATH: Optional[str]

REQUIRED_USE: str = ''
    An expression indicating valid combinations of use flags.

    Consists of a string containing sub-expressions of the form given below. Note that the brackets can contain arbitrary nested expressions of this form, and are not limited to what is shown in the examples below.
    Behaviour | Expression
    flag must be enabled | flag
    flag must not be enabled | !flag
    If flag1 enabled then flag2 enabled | flag1? ( flag2 )
    If flag1 disabled then flag2 enabled | !flag1? ( flag2 )
    If flag1 disabled then flag2 disabled | !flag1? ( !flag2 )
    Must enable any one or more (inclusive or) | || ( flag1 flag2 flag3 )
    Must enable exactly one but not more (exclusive or) | ^^ ( flag1 flag2 flag3 )
    May enable at most one | ?? ( flag1 flag2 flag3 )

RESTRICT: str = ''
    Lists features which should be disabled for this package

    The following two options are supported:

•   mirror: The package's SRC_URI entries should not be mirrored, and mirrors should not be checked when fetching.

•   fetch: The packages's SRC_URI entries should not be fetched automatically, and the pkg_nofetch function should be invoked if a source cannot be found. This option implies mirror.

    Note that portmod also supports determining these automatically based on source URIs and licenses, so it is no longer necessary to set them explicitly. mirror is restricted for licenses which are not in the REDISTRIBUTABLE license group (see license_groups.yaml), and fetch is restricted for files which are not redistributable (according to license) and do not have a scheme in their SRC_URI (i.e. just a filename, no https://domain.tld etc.).

ROOT: str
    The full path of the prefix root where packages will be installed

    Note: This functions as both ROOT and SYSROOT (as defined by PMS section 11.1).

    Scope: src_*, pkg_*

S: Optional[str] = None
    Specifies the default working directory for src_* functions.

    The default value (if S is None) is the name (minus extension) of the first source in SRC_URI (after use-conditionals have been evaluated). If this path does not exist, the working directory falls back to WORKDIR.

    This is also used to determine the base source path used for installing a InstallDir in the default src_install if S is not defined on the InstallDir.

SRC_URI: str = ''
    A List of sources to be fetched.

    If source files should be renamed, this can be done with the arrow operator as shown in the example below.

    Sources can be wrapped in use-conditional expressions to prevent certain sources from being downloaded unless certain use flags are set or unset.

    E.g.:

    SRC_URI="""
        http://mw.modhistory.com/file.php?id=9321 -> FileName-1.0.zip
        flag? ( https://cdn.bethsoft.com/elderscrolls/morrowind/other/masterindex.zip )
    """

    Note that if you are renaming files, they should correspond to the original filename as best possible, but should also contain version information of some sort to prevent conflicts with other sources from the same package. That is, if the package is updated, we do not want the updated source name to be the same as a previous source name, even if the source name did not change upstream.

T: str
    Path to a temporary directory which may be used during packaging [1]

    Scope: All except __init__

TEXTURE_SIZES: str = ''
    A field declaring the texture size options that the package supports.

    If only one texture size option is available, this field need not be included. Texture sizes should be numbers representing the size of the texture in pixels. Given that textures are usually two-dimensional, the convention is to use: \sqrt{ l \cdot w}

    E.g.:

    TEXTURE_SIZES = "1024 2048"

    This is a special type of USE_EXPAND variable, as use flags are created for its values in the form texture_size_SIZE (in the above example texture_size_1024 and texture_size_2048).

    These use flags can (and should) be used in the pybuild to enable sources and InstallDirs conditionally depending on whether or not the texture size was selected. Exactly one of these use flags will be enabled when the mod is installed depending on the value of the TEXTURE_SIZE variable in the user's portmod.cfg.

    Not included in the PMS

UNFETCHED: List[Source]
    The list of sources which need to be fetched [1]

    Scope: pkg_nofetch

USE: Set[str]
    Enabled use flags [1]

    Scope: All except __init__

WORKDIR: str
    The directory where packaging takes place [1]

    Scope: src_*

can_update_live()
    Indicates whether or not a live package can be updated.

    The default implementation just returns False. If the package has live in its `PROPERTIES, it must implement this method.

Return type
    bool

Returns
    If the package has PROPERTIES="live" and can be updated, should return True Otherwise, should return False

dodoc(pattern)
    Installs documentation matching the given pattern into the image directory (Self.D)

Parameters
    pattern (str) -- A pattern which can include glob-style wildcards as implemented by glob.

static execute(command, pipe_output=False, pipe_error=False)
    Allows execution of arbitrary commands at runtime. Command is sandboxed with filesystem and network access depending on the context in which it is called

Parameters

•   command (str) -- Command to be executed

•   pipe_output (bool) -- If true, returns the output of the command

•   pipe_error (bool) --

Return type
    Optional[str]

get_installed_env()
    Returns a dictionary containing installed object values

info(string)
    Displays info message both immediately, and in the summary after all transactions have been completed

Parameters
    string (str) -- String to display

pkg_postinst()
    Function called immediately after package installation

    In Pybuild1, this function has full write permissions to ROOT. In Pybuild2 it only has read permissions.

    Note that the default does nothing, and it will not even be executed unless defined.

pkg_prerm()
    Function called immediately before package removal

    In Pybuild1, this function has full write permissions to ROOT. In Pybuild2 it only has read permissions.

    Note that the default does nothing, and it will not even be executed unless defined.

pkg_pretend()
    May be used to carry out sanity checks early on in the install process

    Note that the default does nothing, and it will not even be executed unless defined.

    pkg_pretend is run separately from the main phase function sequence, and does not participate in any kind of environment saving. There is no guarantee that any of an package’s dependencies will be met at this stage, and no guarantee that the system state will not have changed substantially before the next phase is executed.

    pkg_pretend must not write to the filesystem and the initial working directory should not be expected to be consistent.

src_install()
    The src_install function installs the package’s content to a directory specified in Pybuild2.D.

    The initial working directory is Pybuild2.S, falling back to Pybuild.WORKDIR if the directory does not exist.

    The default implementation used when the package lacks the src_install function shall behave as:

    for pattern in self.DOCS:
        self.dodoc(pattern)

src_prepare()
    The src_prepare function can be used for post-unpack source preparation.

    The initial working directory is Pybuild2.S, falling back to Pybuild2.WORKDIR if the directory does not exist.

    The default implementation used when the package lacks the src_prepare function shall behave as:

    if self.PATCHES:
        for patch in use_reduce(self.PATCHES, self.USE, flat=True):
            path = os.path.join(self.FILESDIR, patch)
            apply_patch(path)

src_unpack()
    The src_unpack function extracts all of the package’s sources.

    The initial working directory must be self.WORKDIR, and the default implementation used when the package lacks the src_unpack function shall behave as:

    self.unpack(self.A)

unpack(archives)
    Unpacks the given archive into the workdir

    Uses shutil.unpack_archive() as its backend, supporting the following archive formats: - .zip - .tar - .tar.xz / .txz - .tar.bz2 / .tbz2 - .tar.xz / .txz

Parameters
    archives (Union[str, Iterable[Union[Source, str]]]) -- The list of archives to be unpacked

validate()
    (Since Portmod 2.4) inquisitor will call this function when scanning repositories.

    Only code that would be valid in the package global scope (see Sandbox) may be used.

    This is designed to allow basic structural checks without hindering package loading.

    It differs from Pybuild2.pkg_pretend() in that it is meant for static checks, not runtime checks.

version: Version

warn(string)
    Displays warning message both immediately, and in the summary after all transactions have been completed

Parameters
    string (str) -- String to display

pybuild.apply_patch(patch)
    Applies git patch using Git apply

    Patch files must be in a format that can be applied via git apply. Such patches can be produced with git diff --no-index ORIG NEW. The --binary option can be used to produce binary diffs for non-text files.

    Patches must be self-applying. I.e. they should not rely on paths being passed to git apply, and must apply from the default working directory in src_prepare.

    It is recommended that a comment header is included to describe what the patch does, where it's from etc.

Parameters
    patch (str) -- Path to the patch to be applied

pybuild.find_file(name)
    Locates the path of a file within the OpenMW virtual file system

    Deprecated since version 2.4: It will be removed in Portmod 3.0

Parameters
    name (str) -- The relative path within the VFS to search for

Return type
    str

Returns
    The absolute path of the file

pybuild.get_masters(file)
    Detects masters for the given file

    Deprecated since version 2.4: it will be removed in Portmod 3.0

Parameters
    file (str) -- File to be examined

Return type
    Set[str]

Returns
    A set of all the master names

pybuild.list_dir(name)
    Locates all path of files matching the given pattern within the OpenMW virtual file system

    Deprecated since version 2.4: It will be removed in Portmod 3.0

Parameters
    name (str) -- The relative path of the directory within the VFS

Return type
    List[str]

Returns
    A list of files contained within the directory

pybuild.patch_dir(src, dst, *, overwrite=True, ignore=None, case_sensitive=True, move_function=<function _move2>)
    Copies src ontop of dst

Parameters

•   src (Union[str, DirEntry]) -- Source directory to copy from

•   dst (str) -- Destination directory to copy to

•   overwrite (bool) -- If true, overwrite existing files.

•   ignore (Optional[Callable[[str, List[str]], Set[str]]]) -- A callable which, given a directory and its contents, should return a set of files to ignore

•   case_sensitive (bool) -- If False, treat file and directory names as case insensitive

•   move_function (Callable[[DirEntry, str], None]) -- The function to use to transfer individual files. Default is shutil.move (modified to accept a DirEntry). The signature should match shutil.copy2.

Raises
    FileExistsError --

Return type
    str

Returns
    Returns dst

pybuild.use_reduce(depstr, uselist={}, masklist={}, matchall=False, excludeall={}, is_src_uri=False, opconvert=False, flat=False, is_valid_flag=None, token_class=None, matchnone=False)
    Takes a dep string and reduces the use? conditionals out, leaving an array with subarrays. All redundant brackets are removed. Adapted from portage's use_reduce

Parameters

•   depstr (str) -- depstring

•   uselist (AbstractSet[str]) -- List of use enabled flags

•   masklist (AbstractSet[str]) -- List of masked flags (always treated as disabled)

•   matchall (bool) -- Treat all conditionals as active. Used by inquisitor.

•   excludeall (AbstractSet[str]) -- List of flags for which negated conditionals are always treated as inactive.

•   is_src_uri (bool) -- Indicates if depstr represents a SRC_URI

•   opconvert (bool) -- Put every operator as first element into it's argument list

•   flat (bool) -- Create a flat list of all tokens

•   is_valid_flag (Optional[Callable[[str], bool]]) -- Function that decides if a given use flag might be used in use conditionals

•   token_class (Optional[Type]) -- Convert all non operator tokens into this class

•   matchnone (bool) -- Treat all conditionals as inactive. Used by digestgen().

•   matchall --

Return type
    List

Returns
    The use reduced depend array

pybuild.version_gt(version1, version2)
    Version comparision function

Parameters

•   version1 (str) -- A version string

•   version2 (str) -- Another version string

Return type
    bool

Returns
    True if and only if version1 is greater than version2

ADVANCED PACKAGING

  Common Packages

Packages in the common category are a special type of package which can be imported within other packages without needing to be installed.

E.g. Usage

from common.git import Git
class Package(Git):
   ...

  Properties

These packages are implicitly included in pybuild.Pybuild2.DEPEND for packages importing them. When installed, this implicit dependency is pinned to the version, but not the revision, meaning that version changes to common packages will result in any installed package which uses the common package being rebuilt. This means that version bumps should only be used for changes which result in the package installing differently.

If you don't want to trigger rebuilds, for example, when fixing bugs which cause a fatal error when they occur, you can bump the revision instead.

common dependencies are not versioned by default, so if the interface for a common package changes in a way which is not backwards-compatible, it should be moved to a new package (e.g. common/git2).

The consequence of this is that common packages should only include patch versions [1] and revisions, while major/minor versions should be part of the package name.

[1] See https://semver.org/ for a description of patch vs major/minor version components.

  Creating common packages

common packages are themselves regular packages and need to be installed, which means that they must also include a Package class with basic information about the common package itself.

E.g. the common/git package:

common/git/git-1.0.pybuild

from pybuild import Pybuild2
class Package(Pybuild2):
   NAME = "Git"
   DESC = "Pybuild Class that directly fetches from Git repos"
   # This package relies on no repository or profile infrastructure,
   # so it is safe to consider stable on all architectures once stabilized
   KEYWORDS = "**"
class Git(Pybuild2):
   # A description is useful to provide instructions to anyone who wants
   # to use the package
   # Currently there is no sphinx APIDoc created for common packages, so it is
   # recommended to create a wiki page duplicating this description.
   """
   Pybuild Class that directly fetches from Git repos
   Subclasses should specify GIT_SRC_URI, containing a use-reduce-able
   list of remote git repositories
   Optionally, GIT_BRANCH, GIT_COMMIT and GIT_COMMIT_DATE can be used to specify
   what branch and commit should be used.
   """
   GIT_SRC_URI: str
   GIT_BRANCH: Optional[str] = None
   GIT_COMMIT: Optional[str] = None
   GIT_COMMIT_DATE: Optional[str] = None
   def __init__(self):
      # You can use __init__ to modify the package's attributes so that they are
      # not overridden as they would be by attributes defined in the class scope.
      if not self.GIT_BRANCH and not self.GIT_COMMIT and not self.GIT_COMMIT_DATE:
           self.PROPERTIES = self.PROPERTIES + " live"
   def src_unpack(self):
      # You may want to call the super version of the method, so that any other
      # classes in the package's class heirarchy get invoked.
      # If you specifically want to ignore the default behaviour, you can
      # omit this, but this means that packages with multiple superclasses will
      # need to call their parent class phase functions manually.
      super().src_unpack()
      # The remainder of the code has been omitted for brevity
      ...

  Modules

Modules can be used to apply global updates, or other configuration which needs to be done more frequently than just at package installation. This commonly takes the form of updating configuration files.

Modules can also provide interfaces for the user to manually configure their system.

Modules are python files, similar to pybuilds, and contain functions of the form do_{name} or describe_{name}, where name is the name of the operation. Do variants should execute the function, while describe should return a string which describes the operation.

They use the .pmodule extension.

  Installation

Modules must be installed into the MODULEPATH for them to be activated.

  Special Functions

The update operation is invoked by default (if present) after installation or removal of mods. It is recommended that update not produce any output unless changes are necessary. Other helper functions can be included, but will not have any special meaning.

The prerm operation is invoked before the module is removed from the system. Any changes made should be reverted if possible.

Functions of the form describe_{name}, describe_{name}_options and describe_{name}_parameters can also be used to describe functions and their parameters. describe_{name} should return a single string describing the function (prior to portmod 2.5 this was specified in the doc string), and the others should each return a list of strings, one for each parameter, with describe_{name}_options listing argument names, and describe_{name}_parameters listing the corresponding argument descriptions. Unlike update and the do_* functions, describe_* functions have the same restrictions as global scope code (i.e. no using imported objects, file i/o, etc. See Sandbox).

  Module Description

Module files which contain operations other than update must also contain a docstring, which is used to describe the module when invoking portmod <prefix> select.

  Function arguments

Module functions take up to two arguments: state, which is an object with a number of constants that describe the module’s environment, and args, which contains the arguments passed by the user (if any), in the form of an object.

•   do_update only gets passed the state and is not provided with args.

•   All other do_ functions are provided both state and args.

    Note that functions expecting too many or too few arguments will case runtime exceptions.

  Module state

New constants may be introduced, however they will not be removed between major versions of Portmod.

Valid variables include:
Name | Value
ROOT | The root of the module’s installed tree. This location can be used to store custom files which are designed to be used at runtime (e.g. patches)
CACHE | A directory that can be used to cache temporary files between runs. It is not guaranteed to persist, however its contents are not automatically deleted
TEMP | A directory which can be used to store temporary files during module execution. This directory and all files will be removed after the module has finished executing
VERSION | The currently installed version number of the module. Useful for determining if information needs to be regenerated due to an update to the module itself

  Sandbox

Module files execute within the Sandbox.

File access is only permitted within the ROOT, CACHE and TEMP directories passed in the state objects (see above).

The python module portmod.module_util provides access to two helper functions:

1.  execute(command): Permits execution of binaries. The interface is the same as the pybuild.Pybuild2.execute function.

2.  create_file(path): Permits access to arbitrary files on the filesystem via redirection. This returns a path to a file within a temporary directory, and the user will be shown the difference between the existing file and the contents of this new file and prompted to allow the changes to proceed after the module is done executing.

  Example module

"""Shopping list module"""
import os
def do_update(state):
    """This function isn't actually necessary here. We're not updating anything"""
def describe_list():
    return "Displays shopping list"
def do_list(state, args):
    """Displays shopping list"""
    path = os.path.join(state.ROOT, "shopping.txt")
        if os.path.exists(path):
            with open(path, "r") as file:
                for line in file.readlines():
                    print(line)
    # Essentials!
    print("Eggs")
    print("Milk")
    print("Carrots")
    print("Marmite")
    print("Hackle-lo leaves")
def describe_add():
    return "Add to list"
def do_add(state, args):
    """Add to list"""
    with open(os.path.join(state.ROOT, "shopping.txt"), "a") as file:
        print(args.item, file)
def describe_add_options():
    return ["item"]
def describe_add_parameters():
    return ["item to add to the list"]

Note: while it may be possible to use Portmod for shopping lists, this is outside the scope of the project and is not something which is officially supported. The above is provided as an example of the format only.

  Notes

Like Pybuilds, global scope code is not permitted! Modules must be sourced to get information such as descriptions when running portmod <prefix> select, and global code running at this point would be undesirable.

  Architecture Versioning

The file .get_arch_version.py, if installed into a prefix, can be used to detect the version of an architecture. The file should, when run in a python interpreter, print the version to stdout (in portmod's external version format, i.e. without revisions or epochs. See External Versions). If the script produces a non-zero exit code, any output will be ignored.

It is recommended that this be written as efficiently as possible, as it is run at the beginning of almost every single portmod command since it's necessary to set up the profile.

The script has full read access in the Sandbox, but no write access outside of a temporary directory which will be specified through the TMP or TMPDIR environment variables (such as can be accessed by the python tempfile module; see Temporary Directories for details). It also has no network access.

The output of the script, assuming it parses as a valid version, gets set in the sandbox as the ARCH_VERSION variable (see defaults.conf). Otherwise, ARCH_VERSION is None.

E.g. .get_arch_version.py

"""Finds openmw version on linux"""
version_file_path = "/usr/share/openmw/resources/version"
with open(version_file_path, encoding="utf-8") as file:
    print(file.read().strip(), end="")

NOTE: stderr will not be redirected, and it is recommended that any commands which produce stderr should be redirected or suppressed. Successful version detection should write nothing to stderr.

However, stderr can be used in the case of failure to indicate to the user things such as environment variables which can be set to aid version detection (e.g. the path of the game data directory).

  Usage

Keywords in pybuild.Pybuild2.KEYWORDS can optionally be followed by a Version Specifier surrounded by {}, making it possible to indicate that a package is stable, testing or masked on a specific version or range of versions.

Note that multiple levels of support are available.

At minimum, by installing the version detection script so that ARCH_VERSION gets set while using non-version-specific ACCEPT_KEYWORDS you immediately will get support for masking packages when they are broken on a particular version. That is, packages with version-specific masks (e.g. -openmw{<0.48}) will get masked, while without ARCH_VERSION only generic masks such as -openmw that apply to all versions will have an effect.

By using a version-specific ACCEPT_KEYWORDS (e.g. ACCEPT_KEYWORDS = "openmw{==0.47.0}") then version-specific stable and testing keywords will also be respected, and when your ARCH_VERSION falls into the range of versions specified, the package will have the stability provided by the keyword.

If the .get_arch_version.py file is being installed by a package, users will not have ARCH_VERSION set when portmod first runs. To handle this, it is generally recommended that ACCEPT_KEYWORDS be set in the following manner (to accept stable by default, minor modifications would be necessary for testing by default):

if ARCH_VERSION is None:
    ACCEPT_KEYWORDS = ARCH
else:
    ACCEPT_KEYWORDS = f"{ARCH}{{=={ARCH_VERSION}}}"

This also means that packages in the @system set (notably including the package that installs the version detection script) should not use version-specific keywords, as they will be ignored on the first run.

SETUP

To be able to create packages that you can use you will need to fork the repository you want to contribute to, and set up a clone of your fork (this is described in Contributing to Portmod):

You will also need to adjust repos.cfg to reference your fork instead of the original package repository. This will allow you to install packages you have added or modified via the merge prefix subcommand, which is necessary for testing the package before you submit it.

Example repos.cfg

[<repo>]
location = /path/to/your/cloned/fork
# You may want to disable auto_sync, as it may not work properly on your fork
auto_sync = False
...

When you have finished working with the repository, you may want to revert your changes to repos.cfg to make sure the repository is kept up to date when you run portmod sync. Otherwise you will need to manually make sure your fork is up to date.

  Packaging Mods

See Package Directories for the basics of the files which go into the package directory.

For details more specific to the game and repository you are packaging for, refer to the guides on the wiki. The package repositories for each supported game have different packaging conventions to match the installation requirements of the engines.

You may also want to consult the base pybuild documentation, which package repositories build on top of.

  Custom Repositories

Alternatively, you can create your own package repository as described in repos.cfg. Full details of repository metadata are described in Package Repositories.

If you publish your custom repository other people can

This guide does not currently document the necessary requirements to set up a package repository for a completely new game. If you wish to do so, contact the authors for assistance (contact details are provided in Portmod's README).

AUTHOR
Benjamin Winger, Roma Tentser, PopeRigby, Hristos N. Triantafillou, lumbo7332, Pope_Rigby, marius david, marius851000, Basile Thullen, Joel, Nords Morrow, Anonymous, Paul Infield-Harm, Qazaaq, Tenchi, Weblate, gbtb, louisabraham, ultramink

COPYRIGHT
2019-2025, Portmod Authors