PEP: 587 Title: Python Initialization Configuration Author: Victor
Stinner \<<vstinner@python.org>\>, Nick Coghlan \<<ncoghlan@gmail.com>\>
BDFL-Delegate: Thomas Wouters \<<thomas@python.org>\> Discussions-To:
<python-dev@python.org> Status: Final Type: Standards Track
Content-Type: text/x-rst Created: 27-Mar-2019 Python-Version: 3.8

Abstract
========

Add a new C API to configure the Python Initialization providing finer
control on the whole configuration and better error reporting.

It becomes possible to read the configuration and then override some
computed parameters before it is applied. It also becomes possible to
completely override how Python computes the module search paths
(`sys.path`).

The new [Isolated Configuration](#isolated-configuration) provides sane
default values to isolate Python from the system. For example, to embed
Python into an application. Using the environment are now opt-in
options, rather than an opt-out options. For example, environment
variables, command line arguments and global configuration variables are
ignored by default.

Building a customized Python which behaves as the regular Python becomes
easier using the new `Py_RunMain()` function. Moreover, using the
[Python Configuration](#python-configuration), `PyConfig.argv` arguments
are now parsed the same way the regular Python parses command line
arguments, and `PyConfig.xoptions` are handled as `-X opt` command line
options.

This extracts a subset of the API design from the PEP 432 development
and refactoring work that is now considered sufficiently stable to make
public (allowing 3rd party embedding applications access to the same
configuration APIs that the native CPython CLI is now using).

Rationale
=========

Python is highly configurable but its configuration evolved organically.
The initialization configuration is scattered all around the code using
different ways to set them: global configuration variables (ex:
`Py_IsolatedFlag`), environment variables (ex: `PYTHONPATH`), command
line arguments (ex: `-b`), configuration files (ex: `pyvenv.cfg`),
function calls (ex: `Py_SetProgramName()`). A straightforward and
reliable way to configure Python is needed.

Some configuration parameters are not accessible from the C API, or not
easily. For example, there is no API to override the default values of
`sys.executable`.

Some options like `PYTHONPATH` can only be set using an environment
variable which has a side effect on Python child processes if not unset
properly.

Some options also depends on other options: see [Priority and
Rules](#priority-and-rules). Python 3.7 API does not provide a
consistent view of the overall configuration.

The C API of Python 3.7 Initialization takes `wchar_t*` strings as input
whereas the Python filesystem encoding is set during the initialization
which can lead to mojibake.

Python 3.7 APIs like `Py_Initialize()` aborts the process on memory
allocation failure which is not convenient when Python is embedded.
Moreover, `Py_Main()` could exit directly the process rather than
returning an exit code. Proposed new API reports the error or exit code
to the caller which can decide how to handle it.

Implementing the PEP 540 (UTF-8 Mode) and the new `-X dev` correctly was
almost impossible in Python 3.6. The code base has been deeply reworked
in Python 3.7 and then in Python 3.8 to read the configuration into a
structure with no side effect. It becomes possible to clear the
configuration (release memory) and read again the configuration if the
encoding changed . It is required to implement properly the UTF-8 which
changes the encoding using `-X utf8` command line option. Internally,
bytes `argv` strings are decoded from the filesystem encoding. The
`-X dev` changes the memory allocator (behaves as `PYTHONMALLOC=debug`),
whereas it was not possible to change the memory allocation *while*
parsing the command line arguments. The new design of the internal
implementation not only allowed to implement properly `-X utf8` and
`-X dev`, it also allows to change the Python behavior way more easily,
especially for corner cases like that, and ensure that the configuration
remains consistent: see [Priority and Rules](#priority-and-rules).

This PEP is a partial implementation of PEP 432 which is the overall
design. New fields can be added later to `PyConfig` structure to finish
the implementation of the PEP 432 (e.g. by adding a new partial
initialization API which allows to configure Python using Python objects
to finish the full initialization). However, those features are omitted
from this PEP as even the native CPython CLI doesn\'t work that way -
the public API proposal in this PEP is limited to features which have
already been implemented and adopted as private APIs for us in the
native CPython CLI.

Python Initialization C API
===========================

This PEP proposes to add the following new structures and functions.

New structures:

-   `PyConfig`
-   `PyPreConfig`
-   `PyStatus`
-   `PyWideStringList`

New functions:

-   `PyConfig_Clear(config)`
-   `PyConfig_InitIsolatedConfig()`
-   `PyConfig_InitPythonConfig()`
-   `PyConfig_Read(config)`
-   `PyConfig_SetArgv(config, argc, argv)`
-   `PyConfig_SetBytesArgv(config, argc, argv)`
-   `PyConfig_SetBytesString(config, config_str, str)`
-   `PyConfig_SetString(config, config_str, str)`
-   `PyPreConfig_InitIsolatedConfig(preconfig)`
-   `PyPreConfig_InitPythonConfig(preconfig)`
-   `PyStatus_Error(err_msg)`
-   `PyStatus_Exception(status)`
-   `PyStatus_Exit(exitcode)`
-   `PyStatus_IsError(status)`
-   `PyStatus_IsExit(status)`
-   `PyStatus_NoMemory()`
-   `PyStatus_Ok()`
-   `PyWideStringList_Append(list, item)`
-   `PyWideStringList_Insert(list, index, item)`
-   `Py_BytesMain(argc, argv)`
-   `Py_ExitStatusException(status)`
-   `Py_InitializeFromConfig(config)`
-   `Py_PreInitialize(preconfig)`
-   `Py_PreInitializeFromArgs(preconfig, argc, argv)`
-   `Py_PreInitializeFromBytesArgs(preconfig, argc, argv)`
-   `Py_RunMain()`

This PEP also adds `_PyRuntimeState.preconfig` (`PyPreConfig` type) and
`PyInterpreterState.config` (`PyConfig` type) fields to these internal
structures. `PyInterpreterState.config` becomes the new reference
configuration, replacing global configuration variables and other
private variables.

PyWideStringList
----------------

`PyWideStringList` is a list of `wchar_t*` strings.

`PyWideStringList` structure fields:

-   `length` (`Py_ssize_t`)
-   `items` (`wchar_t**`)

Methods:

-   `PyStatus PyWideStringList_Append(PyWideStringList *list, const wchar_t *item)`:
    Append *item* to *list*.
-   `PyStatus PyWideStringList_Insert(PyWideStringList *list, Py_ssize_t index, const wchar_t *item)`:
    Insert *item* into *list* at *index*. If *index* is greater than
    *list* length, just append *item* to *list*.

If *length* is non-zero, *items* must be non-NULL and all strings must
be non-NULL.

PyStatus
--------

`PyStatus` is a structure to store the status of an initialization
function: success, error or exit. For an error, it can store the C
function name which created the error.

Example:

    PyStatus alloc(void **ptr, size_t size)
    {
        *ptr = PyMem_RawMalloc(size);
        if (*ptr == NULL) {
            return PyStatus_NoMemory();
        }
        return PyStatus_Ok();
    }

    int main(int argc, char **argv)
    {
        void *ptr;
        PyStatus status = alloc(&ptr, 16);
        if (PyStatus_Exception(status)) {
            Py_ExitStatusException(status);
        }
        PyMem_Free(ptr);
        return 0;
    }

`PyStatus` fields:

-   `exitcode` (`int`): Argument passed to `exit()`.
-   `err_msg` (`const char*`): Error message.
-   `func` (`const char *`): Name of the function which created an
    error, can be `NULL`.
-   private `_type` field: for internal usage only.

Functions to create a status:

-   `PyStatus_Ok()`: Success.
-   `PyStatus_Error(err_msg)`: Initialization error with a message.
-   `PyStatus_NoMemory()`: Memory allocation failure (out of memory).
-   `PyStatus_Exit(exitcode)`: Exit Python with the specified exit code.

Functions to handle a status:

-   `PyStatus_Exception(status)`: Is the result an error or an exit? If
    true, the exception must be handled; by calling
    `Py_ExitStatusException(status)` for example.
-   `PyStatus_IsError(status)`: Is the result an error?
-   `PyStatus_IsExit(status)`: Is the result an exit?
-   `Py_ExitStatusException(status)`: Call `exit(exitcode)` if *status*
    is an exit. Print the error messageand exit with a non-zero exit
    code if *status* is an error. Must only be called if
    `PyStatus_Exception(status)` is true.

::: {.note}
::: {.admonition-title}
Note
:::

Internally, Python uses macros which set `PyStatus.func`, whereas
functions to create a status set `func` to `NULL`.
:::

Preinitialization with PyPreConfig
----------------------------------

The `PyPreConfig` structure is used to preinitialize Python:

-   Set the Python memory allocator
-   Configure the LC\_CTYPE locale
-   Set the UTF-8 mode

Example using the preinitialization to enable the UTF-8 Mode:

    PyStatus status;
    PyPreConfig preconfig;

    PyPreConfig_InitPythonConfig(&preconfig);

    preconfig.utf8_mode = 1;

    status = Py_PreInitialize(&preconfig);
    if (PyStatus_Exception(status)) {
        Py_ExitStatusException(status);
    }

    /* at this point, Python will speak UTF-8 */

    Py_Initialize();
    /* ... use Python API here ... */
    Py_Finalize();

Function to initialize a preconfiguration:

-   `PyStatus PyPreConfig_InitIsolatedConfig(PyPreConfig *preconfig)`
-   `PyStatus PyPreConfig_InitPythonConfig(PyPreConfig *preconfig)`

Functions to preinitialize Python:

-   `PyStatus Py_PreInitialize(const PyPreConfig *preconfig)`
-   `PyStatus Py_PreInitializeFromBytesArgs(const PyPreConfig *preconfig, int argc, char * const *argv)`
-   `PyStatus Py_PreInitializeFromArgs(const PyPreConfig *preconfig, int argc, wchar_t * const * argv)`

The caller is responsible to handle exceptions (error or exit) using
`PyStatus_Exception()` and `Py_ExitStatusException()`.

For [Python Configuration](#python-configuration)
(`PyPreConfig_InitPythonConfig()`), if Python is initialized with
command line arguments, the command line arguments must also be passed
to preinitialize Python, since they have an effect on the
pre-configuration like encodings. For example, the `-X utf8` command
line option enables the UTF-8 Mode.

`PyPreConfig` fields:

-   `allocator` (`int`): Name of the memory allocator (ex:
    `PYMEM_ALLOCATOR_MALLOC`). Valid values:
    -   `PYMEM_ALLOCATOR_NOT_SET` (`0`): don\'t change memory allocators
        (use defaults)
    -   `PYMEM_ALLOCATOR_DEFAULT` (`1`): default memory allocators
    -   `PYMEM_ALLOCATOR_DEBUG` (`2`): default memory allocators with
        debug hooks
    -   `PYMEM_ALLOCATOR_MALLOC` (`3`): force usage of `malloc()`
    -   `PYMEM_ALLOCATOR_MALLOC_DEBUG` (`4`): force usage of `malloc()`
        with debug hooks
    -   `PYMEM_ALLOCATOR_PYMALLOC` (`5`): Python \"pymalloc\" allocator
    -   `PYMEM_ALLOCATOR_PYMALLOC_DEBUG` (`6`): Python \"pymalloc\"
        allocator with debug hooks
    -   Note: `PYMEM_ALLOCATOR_PYMALLOC` and
        `PYMEM_ALLOCATOR_PYMALLOC_DEBUG` are not supported if Python is
        configured using `--without-pymalloc`
-   `configure_locale` (`int`): Set the LC\_CTYPE locale to the user
    preferred locale? If equals to 0, set `coerce_c_locale` and
    `coerce_c_locale_warn` to 0.
-   `coerce_c_locale` (`int`): If equals to 2, coerce the C locale; if
    equals to 1, read the LC\_CTYPE locale to decide if it should be
    coerced.
-   `coerce_c_locale_warn` (`int`): If non-zero, emit a warning if the C
    locale is coerced.
-   `dev_mode` (`int`): See `PyConfig.dev_mode`.
-   `isolated` (`int`): See `PyConfig.isolated`.
-   `legacy_windows_fs_encoding` (`int`, Windows only): If non-zero,
    disable UTF-8 Mode, set the Python filesystem encoding to `mbcs`,
    set the filesystem error handler to `replace`.
-   `parse_argv` (`int`): If non-zero, `Py_PreInitializeFromArgs()` and
    `Py_PreInitializeFromBytesArgs()` parse their `argv` argument the
    same way the regular Python parses command line arguments: see
    [Command Line Arguments](#command-line-arguments).
-   `use_environment` (`int`): See `PyConfig.use_environment`.
-   `utf8_mode` (`int`): If non-zero, enable the UTF-8 mode.

The `legacy_windows_fs_encoding` field is only available on Windows.
`#ifdef MS_WINDOWS` macro can be used for Windows specific code.

`PyPreConfig` private fields, for internal use only:

-   `_config_init` (`int`): Function used to initialize `PyConfig`, used
    for preinitialization.

`PyMem_SetAllocator()` can be called after `Py_PreInitialize()` and
before `Py_InitializeFromConfig()` to install a custom memory allocator.
It can be called before `Py_PreInitialize()` if `allocator` is set to
`PYMEM_ALLOCATOR_NOT_SET` (default value).

Python memory allocation functions like `PyMem_RawMalloc()` must not be
used before Python preinitialization, whereas calling directly
`malloc()` and `free()` is always safe. `Py_DecodeLocale()` must not be
called before the preinitialization.

Initialization with PyConfig
----------------------------

The `PyConfig` structure contains most parameters to configure Python.

Example setting the program name:

    void init_python(void)
    {
        PyStatus status;

        PyConfig config;
        PyConfig_InitPythonConfig(&config);

        /* Set the program name. Implicitly preinitialize Python. */
        status = PyConfig_SetString(&config, &config.program_name,
                                    L"/path/to/my_program");
        if (PyStatus_Exception(status)) {
            goto fail;
        }

        status = Py_InitializeFromConfig(&config);
        if (PyStatus_Exception(status)) {
            goto fail;
        }
        PyConfig_Clear(&config);
        return;

    fail:
        PyConfig_Clear(&config);
        Py_ExitStatusException(status);
    }

`PyConfig` methods:

-   `void PyConfig_InitPythonConfig(PyConfig *config)` Initialize
    configuration with [Python Configuration](#python-configuration).
-   `void PyConfig_InitIsolatedConfig(PyConfig *config)`: Initialize
    configuration with [Isolated
    Configuration](#isolated-configuration).
-   `PyStatus PyConfig_SetString(PyConfig *config, wchar_t * const *config_str, const wchar_t *str)`:
    Copy the wide character string *str* into `*config_str`.
    Preinitialize Python if needed.
-   `PyStatus PyConfig_SetBytesString(PyConfig *config, wchar_t * const *config_str, const char *str)`:
    Decode *str* using `Py_DecodeLocale()` and set the result into
    `*config_str`. Preinitialize Python if needed.
-   `PyStatus PyConfig_SetArgv(PyConfig *config, int argc, wchar_t * const *argv)`:
    Set command line arguments from wide character strings.
    Preinitialize Python if needed.
-   `PyStatus PyConfig_SetBytesArgv(PyConfig *config, int argc, char * const *argv)`:
    Set command line arguments: decode bytes using `Py_DecodeLocale()`.
    Preinitialize Python if needed.
-   `PyStatus PyConfig_Read(PyConfig *config)`: Read all Python
    configuration. Fields which are already initialized are left
    unchanged. Preinitialize Python if needed.
-   `void PyConfig_Clear(PyConfig *config)`: Release configuration
    memory.

Most `PyConfig` methods preinitialize Python if needed. In that case,
the Python preinitialization configuration in based on the `PyConfig`.
If configuration fields which are in common with `PyPreConfig` are
tuned, they must be set before calling a `PyConfig` method:

-   `dev_mode`
-   `isolated`
-   `parse_argv`
-   `use_environment`

Moreover, if `PyConfig_SetArgv()` or `PyConfig_SetBytesArgv()` is used,
this method must be called first, before other methods, since the
preinitialization configuration depends on command line arguments (if
`parse_argv` is non-zero).

Functions to initialize Python:

-   `PyStatus Py_InitializeFromConfig(const PyConfig *config)`:
    Initialize Python from *config* configuration.

The caller of these methods and functions is responsible to handle
exceptions (error or exit) using `PyStatus_Exception()` and
`Py_ExitStatusException()`.

`PyConfig` fields:

-   `argv` (`PyWideStringList`): Command line arguments, `sys.argv`. See
    `parse_argv` to parse `argv` the same way the regular Python parses
    Python command line arguments. If `argv` is empty, an empty string
    is added to ensure that `sys.argv` always exists and is never empty.
-   `base_exec_prefix` (`wchar_t*`): `sys.base_exec_prefix`.
-   `base_prefix` (`wchar_t*`): `sys.base_prefix`.
-   `buffered_stdio` (`int`): If equals to 0, enable unbuffered mode,
    making the stdout and stderr streams unbuffered.
-   `bytes_warning` (`int`): If equals to 1, issue a warning when
    comparing `bytes` or `bytearray` with `str`, or comparing `bytes`
    with `int`. If equal or greater to 2, raise a `BytesWarning`
    exception.
-   `check_hash_pycs_mode` (`wchar_t*`): `--check-hash-based-pycs`
    command line option value (see PEP 552). Valid values: `always`,
    `never` and `default`. The default value is `default`.
-   `configure_c_stdio` (`int`): If non-zero, configure C standard
    streams (`stdio`, `stdout`, `stdout`). For example, set their mode
    to `O_BINARY` on Windows.
-   `dev_mode` (`int`): Development mode
-   `dump_refs` (`int`): If non-zero, dump all objects which are still
    alive at exit. Require a special Python build with `Py_REF_DEBUG`
    macro defined.
-   `exec_prefix` (`wchar_t*`): `sys.exec_prefix`.
-   `executable` (`wchar_t*`): `sys.executable`.
-   `faulthandler` (`int`): If non-zero, call `faulthandler.enable()`.
-   `filesystem_encoding` (`wchar_t*`): Filesystem encoding,
    `sys.getfilesystemencoding()`.
-   `filesystem_errors` (`wchar_t*`): Filesystem encoding errors,
    `sys.getfilesystemencodeerrors()`.
-   `use_hash_seed` (`int`), `hash_seed` (`unsigned long`): Randomized
    hash function seed.
-   `home` (`wchar_t*`): Python home directory.
-   `import_time` (`int`): If non-zero, profile import time.
-   `inspect` (`int`): Enter interactive mode after executing a script
    or a command.
-   `install_signal_handlers` (`int`): Install signal handlers?
-   `interactive` (`int`): Interactive mode.
-   `isolated` (`int`): If greater than 0, enable isolated mode:
    -   `sys.path` contains neither the script\'s directory (computed
        from `argv[0]` or the current directory) nor the user\'s
        site-packages directory.
    -   Python REPL doesn\'t import `readline` nor enable default
        readline configuration on interactive prompts.
    -   Set `use_environment` and `user_site_directory` to 0.
-   `legacy_windows_stdio` (`int`, Windows only): If non-zero, use
    `io.FileIO` instead of `WindowsConsoleIO` for `sys.stdin`,
    `sys.stdout` and `sys.stderr`.
-   `malloc_stats` (`int`): If non-zero, dump statistics on `pymalloc`
    memory allocator at exit. The option is ignored if Python is built
    using `--without-pymalloc`.
-   `pythonpath_env` (`wchar_t*`): Module search paths as a string
    separated by DELIM (usually `:` character). Initialized from
    `PYTHONPATH` environment variable value by default.
-   `module_search_paths_set` (`int`), `module_search_paths`
    (`PyWideStringList`): `sys.path`. If `module_search_paths_set` is
    equal to 0, the `module_search_paths` is overridden by the function
    computing the [Path Configuration](#path-configuration).
-   `optimization_level` (`int`): Compilation optimization level:
    -   0: Peephole optimizer (and `__debug__` is set to `True`)
    -   1: Remove assertions, set `__debug__` to `False`
    -   2: Strip docstrings
-   `parse_argv` (`int`): If non-zero, parse `argv` the same way the
    regular Python command line arguments, and strip Python arguments
    from `argv`: see [Command Line Arguments](#command-line-arguments).
-   `parser_debug` (`int`): If non-zero, turn on parser debugging output
    (for expert only, depending on compilation options).
-   `pathconfig_warnings` (`int`): If equal to 0, suppress warnings when
    computing the path configuration (Unix only, Windows does not log
    any warning). Otherwise, warnings are written into stderr.
-   `prefix` (`wchar_t*`): `sys.prefix`.
-   `program_name` (`wchar_t*`): Program name.
-   `pycache_prefix` (`wchar_t*`): `.pyc` cache prefix.
-   `quiet` (`int`): Quiet mode. For example, don\'t display the
    copyright and version messages even in interactive mode.
-   `run_command` (`wchar_t*`): `python3 -c COMMAND` argument.
-   `run_filename` (`wchar_t*`): `python3 FILENAME` argument.
-   `run_module` (`wchar_t*`): `python3 -m MODULE` argument.
-   `show_alloc_count` (`int`): Show allocation counts at exit? Need a
    special Python build with `COUNT_ALLOCS` macro defined.
-   `show_ref_count` (`int`): Show total reference count at exit? Need a
    debug build of Python (`Py_REF_DEBUG` macro should be defined).
-   `site_import` (`int`): Import the `site` module at startup?
-   `skip_source_first_line` (`int`): Skip the first line of the source?
-   `stdio_encoding` (`wchar_t*`), `stdio_errors` (`wchar_t*`): Encoding
    and encoding errors of `sys.stdin`, `sys.stdout` and `sys.stderr`.
-   `tracemalloc` (`int`): If non-zero, call `tracemalloc.start(value)`.
-   `user_site_directory` (`int`): If non-zero, add user site directory
    to `sys.path`.
-   `verbose` (`int`): If non-zero, enable verbose mode.
-   `warnoptions` (`PyWideStringList`): Options of the `warnings` module
    to build warnings filters.
-   `write_bytecode` (`int`): If non-zero, write `.pyc` files.
-   `xoptions` (`PyWideStringList`): `sys._xoptions`.

The `legacy_windows_stdio` field is only available on Windows.
`#ifdef MS_WINDOWS` macro can be used for Windows specific code.

If `parse_argv` is non-zero, `argv` arguments are parsed the same way
the regular Python parses command line arguments, and Python arguments
are stripped from `argv`: see [Command Line
Arguments](#command-line-arguments).

The `xoptions` options are parsed to set other options: see [-X
Options](#x-options).

`PyConfig` private fields, for internal use only:

-   `_config_init` (`int`): Function used to initialize `PyConfig`, used
    for preinitialization.
-   `_install_importlib` (`int`): Install importlib?
-   `_init_main` (`int`): If equal to 0, stop Python initialization
    before the \"main\" phase (see PEP 432).

More complete example modifying the default configuration, read the
configuration, and then override some parameters:

    PyStatus init_python(const char *program_name)
    {
        PyStatus status;

        PyConfig config;
        PyConfig_InitPythonConfig(&config);

        /* Set the program name before reading the configuration
           (decode byte string from the locale encoding).

           Implicitly preinitialize Python. */
        status = PyConfig_SetBytesString(&config, &config.program_name,
                                      program_name);
        if (PyStatus_Exception(status)) {
            goto done;
        }

        /* Read all configuration at once */
        status = PyConfig_Read(&config);
        if (PyStatus_Exception(status)) {
            goto done;
        }

        /* Append our custom search path to sys.path */
        status = PyWideStringList_Append(&config.module_search_paths,
                                      L"/path/to/more/modules");
        if (PyStatus_Exception(status)) {
            goto done;
        }

        /* Override executable computed by PyConfig_Read() */
        status = PyConfig_SetString(&config, &config.executable,
                                    L"/path/to/my_executable");
        if (PyStatus_Exception(status)) {
            goto done;
        }

        status = Py_InitializeFromConfig(&config);

    done:
        PyConfig_Clear(&config);
        return status;
    }

::: {.note}
::: {.admonition-title}
Note
:::

`PyImport_FrozenModules`, `PyImport_AppendInittab()` and
`PyImport_ExtendInittab()` functions are still relevant and continue to
work as previously. They should be set or called after Python
preinitialization and before the Python initialization.
:::

Isolated Configuration
----------------------

`PyPreConfig_InitIsolatedConfig()` and `PyConfig_InitIsolatedConfig()`
functions create a configuration to isolate Python from the system. For
example, to embed Python into an application.

This configuration ignores global configuration variables, environments
variables and command line arguments (`argv` is not parsed). The C
standard streams (ex: `stdout`) and the LC\_CTYPE locale are left
unchanged by default.

Configuration files are still used with this configuration. Set the
[Path Configuration](#path-configuration) (\"output fields\") to ignore
these configuration files and avoid the function computing the default
path configuration.

Python Configuration
--------------------

`PyPreConfig_InitPythonConfig()` and `PyConfig_InitPythonConfig()`
functions create a configuration to build a customized Python which
behaves as the regular Python.

Environments variables and command line arguments are used to configure
Python, whereas global configuration variables are ignored.

This function enables C locale coercion (PEP 538) and UTF-8 Mode (PEP
540) depending on the LC\_CTYPE locale, `PYTHONUTF8` and
`PYTHONCOERCECLOCALE` environment variables.

Example of customized Python always running in isolated mode:

    int main(int argc, char **argv)
    {
        PyStatus status;

        PyConfig config;
        PyConfig_InitPythonConfig(&config);

        config.isolated = 1;

        /* Decode command line arguments.
           Implicitly preinitialize Python (in isolated mode). */
        status = PyConfig_SetBytesArgv(&config, argc, argv);
        if (PyStatus_Exception(status)) {
            goto fail;
        }

        status = Py_InitializeFromConfig(&config);
        if (PyStatus_Exception(status)) {
            goto fail;
        }
        PyConfig_Clear(&config);

        return Py_RunMain();

    fail:
        PyConfig_Clear(&config);
        if (PyStatus_IsExit(status)) {
            return status.exitcode;
        }
        /* Display the error message and exit the process with
           non-zero exit code */
        Py_ExitStatusException(status);
    }

This example is a basic implementation of the \"System Python
Executable\" discussed in PEP 432.

Path Configuration
------------------

`PyConfig` contains multiple fields for the path configuration:

-   Path configuration input fields:
    -   `home`
    -   `pythonpath_env`
    -   `pathconfig_warnings`
-   Path configuration output fields:
    -   `exec_prefix`
    -   `executable`
    -   `prefix`
    -   `module_search_paths_set`, `module_search_paths`

If at least one \"output field\" is not set, Python computes the path
configuration to fill unset fields. If `module_search_paths_set` is
equal to 0, `module_search_paths` is overridden and
`module_search_paths_set` is set to 1.

It is possible to completely ignore the function computing the default
path configuration by setting explicitly all path configuration output
fields listed above. A string is considered as set even if it is
non-empty. `module_search_paths` is considered as set if
`module_search_paths_set` is set to 1. In this case, path configuration
input fields are ignored as well.

Set `pathconfig_warnings` to 0 to suppress warnings when computing the
path configuration (Unix only, Windows does not log any warning).

If `base_prefix` or `base_exec_prefix` fields are not set, they inherit
their value from `prefix` and `exec_prefix` respectively.

`Py_RunMain()` and `Py_Main()` modify `sys.path`:

-   If `run_filename` is set and is a directory which contains a
    `__main__.py` script, prepend `run_filename` to `sys.path`.
-   If `isolated` is zero:
    -   If `run_module` is set, prepend the current directory to
        `sys.path`. Do nothing if the current directory cannot be read.
    -   If `run_filename` is set, prepends the directory of the filename
        to `sys.path`.
    -   Otherwise, prepends an empty string to `sys.path`.

If `site_import` is non-zero, `sys.path` can be modified by the `site`
module. If `user_site_directory` is non-zero and the user\'s
site-package directory exists, the `site` module appends the user\'s
site-package directory to `sys.path`.

See also [Configuration Files](#configuration-files) used by the path
configuration.

Py\_BytesMain()
---------------

Python 3.7 provides a high-level `Py_Main()` function which requires to
pass command line arguments as `wchar_t*` strings. It is non-trivial to
use the correct encoding to decode bytes. Python has its own set of
issues with C locale coercion and UTF-8 Mode.

This PEP adds a new `Py_BytesMain()` function which takes command line
arguments as bytes:

    int Py_BytesMain(int argc, char **argv)

Py\_RunMain()
-------------

The new `Py_RunMain()` function executes the command
(`PyConfig.run_command`), the script (`PyConfig.run_filename`) or the
module (`PyConfig.run_module`) specified on the command line or in the
configuration, and then finalizes Python. It returns an exit status that
can be passed to the `exit()` function. :

    int Py_RunMain(void);

See [Python Configuration](#python-configuration) for an example of
customized Python always running in isolated mode using `Py_RunMain()`.

Multi-Phase Initialization Private Provisional API
--------------------------------------------------

This section is a private provisional API introducing multi-phase
initialization, the core feature of the PEP 432:

-   \"Core\" initialization phase, \"bare minimum Python\":
    -   Builtin types;
    -   Builtin exceptions;
    -   Builtin and frozen modules;
    -   The `sys` module is only partially initialized (ex: `sys.path`
        doesn\'t exist yet);
-   \"Main\" initialization phase, Python is fully initialized:
    -   Install and configure `importlib`;
    -   Apply the [Path Configuration](#path-configuration);
    -   Install signal handlers;
    -   Finish `sys` module initialization (ex: create `sys.stdout` and
        `sys.path`);
    -   Enable optional features like `faulthandler` and `tracemalloc`;
    -   Import the `site` module;
    -   etc.

Private provisional API:

-   `PyConfig._init_main`: if set to 0, `Py_InitializeFromConfig()`
    stops at the \"Core\" initialization phase.
-   `PyStatus _Py_InitializeMain(void)`: move to the \"Main\"
    initialization phase, finish the Python initialization.

No module is imported during the \"Core\" phase and the `importlib`
module is not configured: the [Path Configuration](#path-configuration)
is only applied during the \"Main\" phase. It may allow to customize
Python in Python to override or tune the [Path
Configuration](#path-configuration), maybe install a custom
sys.meta\_path importer or an import hook, etc.

It may become possible to compute the [Path
Configuration](#path-configuration) in Python, after the Core phase and
before the Main phase, which is one of the PEP 432 motivation.

The \"Core\" phase is not properly defined: what should be and what
should not be available at this phase is not specified yet. The API is
marked as private and provisional: the API can be modified or even be
removed anytime until a proper public API is designed.

Example running Python code between \"Core\" and \"Main\" initialization
phases:

    void init_python(void)
    {
        PyStatus status;

        PyConfig config;
        PyConfig_InitPythonConfig(&config);

        config._init_main = 0;

        /* ... customize 'config' configuration ... */

        status = Py_InitializeFromConfig(&config);
        PyConfig_Clear(&config);
        if (PyStatus_Exception(status)) {
            Py_ExitStatusException(status);
        }

        /* Use sys.stderr because sys.stdout is only created
           by _Py_InitializeMain() */
        int res = PyRun_SimpleString(
            "import sys; "
            "print('Run Python code before _Py_InitializeMain', "
                   "file=sys.stderr)");
        if (res < 0) {
            exit(1);
        }

        /* ... put more configuration code here ... */

        status = _Py_InitializeMain();
        if (PyStatus_Exception(status)) {
            Py_ExitStatusException(status);
        }
    }

Backwards Compatibility
=======================

This PEP only adds a new API: it leaves the existing API unchanged and
has no impact on the backwards compatibility.

The Python 3.7 `Py_Initialize()` function now disable the C locale
coercion (PEP 538) and the UTF-8 Mode (PEP 540) by default to prevent
mojibake. The new API using the [Python
Configuration](#python-configuration) is needed to enable them
automatically.

Annexes
=======

Comparison of Python and Isolated Configurations
------------------------------------------------

Differences between `PyPreConfig_InitPythonConfig()` and
`PyPreConfig_InitIsolatedConfig()`:

+------------------------------+---------+----------+
| PyPreConfig                  | Python  | Isolated |
+==============================+=========+==========+
| `coerce_c_locale_warn`       | > -1    | > 0      |
+------------------------------+---------+----------+
| `coerce_c_locale`            | > -1    | > 0      |
+------------------------------+---------+----------+
| `configure_locale`           | > **1** | > 0      |
+------------------------------+---------+----------+
| `dev_mode`                   | > -1    | > 0      |
+------------------------------+---------+----------+
| `isolated`                   | > 0     | > **1**  |
+------------------------------+---------+----------+
| `legacy_windows_fs_encoding` | > -1    | > 0      |
+------------------------------+---------+----------+
| `use_environment`            | > 0     | > 0      |
+------------------------------+---------+----------+
| `parse_argv`                 | > **1** | > 0      |
+------------------------------+---------+----------+
| `utf8_mode`                  | > -1    | > 0      |
+------------------------------+---------+----------+

Differences between `PyConfig_InitPythonConfig()` and
`PyConfig_InitIsolatedConfig()`:

+---------------------------+---------+----------+
| PyConfig                  | Python  | Isolated |
+===========================+=========+==========+
| `configure_c_stdio`       | > **1** | > 0      |
+---------------------------+---------+----------+
| `install_signal_handlers` | > **1** | > 0      |
+---------------------------+---------+----------+
| `isolated`                | > 0     | > **1**  |
+---------------------------+---------+----------+
| `parse_argv`              | > **1** | > 0      |
+---------------------------+---------+----------+
| `pathconfig_warnings`     | > **1** | > 0      |
+---------------------------+---------+----------+
| `use_environment`         | > **1** | > 0      |
+---------------------------+---------+----------+
| `user_site_directory`     | > **1** | > 0      |
+---------------------------+---------+----------+

Priority and Rules
------------------

Priority of configuration parameters, highest to lowest:

-   `PyConfig`
-   `PyPreConfig`
-   Configuration files
-   Command line options
-   Environment variables
-   Global configuration variables

Priority of warning options, highest to lowest:

-   `PyConfig.warnoptions`
-   `PySys_AddWarnOption()`
-   `PyConfig.bytes_warning` (add `"error::BytesWarning"` filter if
    greater than 1, add `"default::BytesWarning` filter if equals to 1)
-   `-W opt` command line argument
-   `PYTHONWARNINGS` environment variable
-   `PyConfig.dev_mode` (add `"default"` filter)

Rules on `PyConfig` parameters:

-   If `isolated` is non-zero, `use_environment` and
    `user_site_directory` are set to 0.
-   If `dev_mode` is non-zero, `allocator` is set to `"debug"`,
    `faulthandler` is set to 1, and `"default"` filter is added to
    `warnoptions`. But the `PYTHONMALLOC` environment variable has the
    priority over `dev_mode` to set the memory allocator.
-   If `base_prefix` is not set, it inherits `prefix` value.
-   If `base_exec_prefix` is not set, it inherits `exec_prefix` value.
-   If the `python._pth` configuration file is present, `isolated` is
    set to 1 and `site_import` is set to 0; but `site_import` is set to
    1 if `python._pth` contains `import site`.

Rules on `PyConfig` and `PyPreConfig` parameters:

-   If `PyPreConfig.legacy_windows_fs_encoding` is non-zero, set
    `PyPreConfig.utf8_mode` to 0, set `PyConfig.filesystem_encoding` to
    `mbcs`, and set `PyConfig.filesystem_errors` to `replace`.

Configuration Files
-------------------

Python configuration files used by the [Path
Configuration](#path-configuration):

-   `pyvenv.cfg`
-   `python._pth` (Windows only)
-   `pybuilddir.txt` (Unix only)

Global Configuration Variables
------------------------------

Global configuration variables mapped to `PyPreConfig` fields:

  Variable                           Field
  ---------------------------------- ------------------------------
  `Py_IgnoreEnvironmentFlag`         `use_environment` (NOT)
  `Py_IsolatedFlag`                  `isolated`
  `Py_LegacyWindowsFSEncodingFlag`   `legacy_windows_fs_encoding`
  `Py_UTF8Mode`                      `utf8_mode`

(NOT) means that the `PyPreConfig` value is the opposite of the global
configuration variable value. `Py_LegacyWindowsFSEncodingFlag` is only
available on Windows.

Global configuration variables mapped to `PyConfig` fields:

  Variable                                 Field
  ---------------------------------------- ------------------------------
  `Py_BytesWarningFlag`                    `bytes_warning`
  `Py_DebugFlag`                           `parser_debug`
  `Py_DontWriteBytecodeFlag`               `write_bytecode` (NOT)
  `Py_FileSystemDefaultEncodeErrors`       `filesystem_errors`
  `Py_FileSystemDefaultEncoding`           `filesystem_encoding`
  `Py_FrozenFlag`                          `pathconfig_warnings` (NOT)
  `Py_HasFileSystemDefaultEncoding`        `filesystem_encoding`
  `Py_HashRandomizationFlag`               `use_hash_seed`, `hash_seed`
  `Py_IgnoreEnvironmentFlag`               `use_environment` (NOT)
  `Py_InspectFlag`                         `inspect`
  `Py_InteractiveFlag`                     `interactive`
  `Py_IsolatedFlag`                        `isolated`
  `Py_LegacyWindowsStdioFlag`              `legacy_windows_stdio`
  `Py_NoSiteFlag`                          `site_import` (NOT)
  `Py_NoUserSiteDirectory`                 `user_site_directory` (NOT)
  `Py_OptimizeFlag`                        `optimization_level`
  `Py_QuietFlag`                           `quiet`
  `Py_UnbufferedStdioFlag`                 `buffered_stdio` (NOT)
  `Py_VerboseFlag`                         `verbose`
  `_Py_HasFileSystemDefaultEncodeErrors`   `filesystem_errors`

(NOT) means that the `PyConfig` value is the opposite of the global
configuration variable value. `Py_LegacyWindowsStdioFlag` is only
available on Windows.

Command Line Arguments
----------------------

Usage:

    python3 [options]
    python3 [options] -c COMMAND
    python3 [options] -m MODULE
    python3 [options] SCRIPT

Command line options mapped to pseudo-action on `PyPreConfig` fields:

  Option            `PyConfig` field
  ----------------- -----------------------
  `-E`              `use_environment = 0`
  `-I`              `isolated = 1`
  `-X dev`          `dev_mode = 1`
  `-X utf8`         `utf8_mode = 1`
  `-X utf8=VALUE`   `utf8_mode = VALUE`

Command line options mapped to pseudo-action on `PyConfig` fields:

  Option                           `PyConfig` field
  -------------------------------- --------------------------------------------
  `-b`                             `bytes_warning++`
  `-B`                             `write_bytecode = 0`
  `-c COMMAND`                     `run_command = COMMAND`
  `--check-hash-based-pycs=MODE`   `check_hash_pycs_mode = MODE`
  `-d`                             `parser_debug++`
  `-E`                             `use_environment = 0`
  `-i`                             `inspect++` and `interactive++`
  `-I`                             `isolated = 1`
  `-m MODULE`                      `run_module = MODULE`
  `-O`                             `optimization_level++`
  `-q`                             `quiet++`
  `-R`                             `use_hash_seed = 0`
  `-s`                             `user_site_directory = 0`
  `-S`                             `site_import`
  `-t`                             ignored (kept for backwards compatibility)
  `-u`                             `buffered_stdio = 0`
  `-v`                             `verbose++`
  `-W WARNING`                     add `WARNING` to `warnoptions`
  `-x`                             `skip_source_first_line = 1`
  `-X OPTION`                      add `OPTION` to `xoptions`

`-h`, `-?` and `-V` options are handled without `PyConfig`.

-X Options
----------

-X options mapped to pseudo-action on `PyConfig` fields:

  Option                       `PyConfig` field
  ---------------------------- ---------------------------
  `-X dev`                     `dev_mode = 1`
  `-X faulthandler`            `faulthandler = 1`
  `-X importtime`              `import_time = 1`
  `-X pycache_prefix=PREFIX`   `pycache_prefix = PREFIX`
  `-X showalloccount`          `show_alloc_count = 1`
  `-X showrefcount`            `show_ref_count = 1`
  `-X tracemalloc=N`           `tracemalloc = N`

Environment Variables
---------------------

Environment variables mapped to `PyPreConfig` fields:

  Variable                          `PyPreConfig` field
  --------------------------------- -------------------------------------------
  `PYTHONCOERCECLOCALE`             `coerce_c_locale`, `coerce_c_locale_warn`
  `PYTHONDEVMODE`                   `dev_mode`
  `PYTHONLEGACYWINDOWSFSENCODING`   `legacy_windows_fs_encoding`
  `PYTHONMALLOC`                    `allocator`
  `PYTHONUTF8`                      `utf8_mode`

Environment variables mapped to `PyConfig` fields:

  Variable                     `PyConfig` field
  ---------------------------- ----------------------------------
  `PYTHONDEBUG`                `parser_debug`
  `PYTHONDEVMODE`              `dev_mode`
  `PYTHONDONTWRITEBYTECODE`    `write_bytecode`
  `PYTHONDUMPREFS`             `dump_refs`
  `PYTHONEXECUTABLE`           `program_name`
  `PYTHONFAULTHANDLER`         `faulthandler`
  `PYTHONHASHSEED`             `use_hash_seed`, `hash_seed`
  `PYTHONHOME`                 `home`
  `PYTHONINSPECT`              `inspect`
  `PYTHONIOENCODING`           `stdio_encoding`, `stdio_errors`
  `PYTHONLEGACYWINDOWSSTDIO`   `legacy_windows_stdio`
  `PYTHONMALLOCSTATS`          `malloc_stats`
  `PYTHONNOUSERSITE`           `user_site_directory`
  `PYTHONOPTIMIZE`             `optimization_level`
  `PYTHONPATH`                 `pythonpath_env`
  `PYTHONPROFILEIMPORTTIME`    `import_time`
  `PYTHONPYCACHEPREFIX,`       `pycache_prefix`
  `PYTHONTRACEMALLOC`          `tracemalloc`
  `PYTHONUNBUFFERED`           `buffered_stdio`
  `PYTHONVERBOSE`              `verbose`
  `PYTHONWARNINGS`             `warnoptions`

`PYTHONLEGACYWINDOWSFSENCODING` and `PYTHONLEGACYWINDOWSSTDIO` are
specific to Windows.

Default Python Configuration
----------------------------

`PyPreConfig_InitPythonConfig()`:

-   `allocator` = `PYMEM_ALLOCATOR_NOT_SET`
-   `coerce_c_locale_warn` = -1
-   `coerce_c_locale` = -1
-   `configure_locale` = 1
-   `dev_mode` = -1
-   `isolated` = 0
-   `legacy_windows_fs_encoding` = -1
-   `use_environment` = 1
-   `utf8_mode` = -1

`PyConfig_InitPythonConfig()`:

-   `argv` = \[\]
-   `base_exec_prefix` = `NULL`
-   `base_prefix` = `NULL`
-   `buffered_stdio` = 1
-   `bytes_warning` = 0
-   `check_hash_pycs_mode` = `NULL`
-   `configure_c_stdio` = 1
-   `dev_mode` = 0
-   `dump_refs` = 0
-   `exec_prefix` = `NULL`
-   `executable` = `NULL`
-   `faulthandler` = 0
-   `filesystem_encoding` = `NULL`
-   `filesystem_errors` = `NULL`
-   `hash_seed` = 0
-   `home` = `NULL`
-   `import_time` = 0
-   `inspect` = 0
-   `install_signal_handlers` = 1
-   `interactive` = 0
-   `isolated` = 0
-   `malloc_stats` = 0
-   `module_search_path_env` = `NULL`
-   `module_search_paths` = \[\]
-   `optimization_level` = 0
-   `parse_argv` = 1
-   `parser_debug` = 0
-   `pathconfig_warnings` = 1
-   `prefix` = `NULL`
-   `program_name` = `NULL`
-   `pycache_prefix` = `NULL`
-   `quiet` = 0
-   `run_command` = `NULL`
-   `run_filename` = `NULL`
-   `run_module` = `NULL`
-   `show_alloc_count` = 0
-   `show_ref_count` = 0
-   `site_import` = 1
-   `skip_source_first_line` = 0
-   `stdio_encoding` = `NULL`
-   `stdio_errors` = `NULL`
-   `tracemalloc` = 0
-   `use_environment` = 1
-   `use_hash_seed` = 0
-   `user_site_directory` = 1
-   `verbose` = 0
-   `warnoptions` = \[\]
-   `write_bytecode` = 1
-   `xoptions` = \[\]
-   `_init_main` = 1
-   `_install_importlib` = 1

Default Isolated Configuration
------------------------------

`PyPreConfig_InitIsolatedConfig()`:

-   `allocator` = `PYMEM_ALLOCATOR_NOT_SET`
-   `coerce_c_locale_warn` = 0
-   `coerce_c_locale` = 0
-   `configure_locale` = 0
-   `dev_mode` = 0
-   `isolated` = 1
-   `legacy_windows_fs_encoding` = 0
-   `use_environment` = 0
-   `utf8_mode` = 0

`PyConfig_InitIsolatedConfig()`:

-   `argv` = \[\]
-   `base_exec_prefix` = `NULL`
-   `base_prefix` = `NULL`
-   `buffered_stdio` = 1
-   `bytes_warning` = 0
-   `check_hash_pycs_mode` = `NULL`
-   `configure_c_stdio` = 0
-   `dev_mode` = 0
-   `dump_refs` = 0
-   `exec_prefix` = `NULL`
-   `executable` = `NULL`
-   `faulthandler` = 0
-   `filesystem_encoding` = `NULL`
-   `filesystem_errors` = `NULL`
-   `hash_seed` = 0
-   `home` = `NULL`
-   `import_time` = 0
-   `inspect` = 0
-   `install_signal_handlers` = 0
-   `interactive` = 0
-   `isolated` = 1
-   `malloc_stats` = 0
-   `module_search_path_env` = `NULL`
-   `module_search_paths` = \[\]
-   `optimization_level` = 0
-   `parse_argv` = 0
-   `parser_debug` = 0
-   `pathconfig_warnings` = 0
-   `prefix` = `NULL`
-   `program_name` = `NULL`
-   `pycache_prefix` = `NULL`
-   `quiet` = 0
-   `run_command` = `NULL`
-   `run_filename` = `NULL`
-   `run_module` = `NULL`
-   `show_alloc_count` = 0
-   `show_ref_count` = 0
-   `site_import` = 1
-   `skip_source_first_line` = 0
-   `stdio_encoding` = `NULL`
-   `stdio_errors` = `NULL`
-   `tracemalloc` = 0
-   `use_environment` = 0
-   `use_hash_seed` = 0
-   `user_site_directory` = 0
-   `verbose` = 0
-   `warnoptions` = \[\]
-   `write_bytecode` = 1
-   `xoptions` = \[\]
-   `_init_main` = 1
-   `_install_importlib` = 1

Python 3.7 API
--------------

Python 3.7 has 4 functions in its C API to initialize and finalize
Python:

-   `Py_Initialize()`, `Py_InitializeEx()`: initialize Python
-   `Py_Finalize()`, `Py_FinalizeEx()`: finalize Python

Python 3.7 can be configured using [Global Configuration
Variables](#global-configuration-variables), [Environment
Variables](#environment-variables), and the following functions:

-   `PyImport_AppendInittab()`
-   `PyImport_ExtendInittab()`
-   `PyMem_SetAllocator()`
-   `PyMem_SetupDebugHooks()`
-   `PyObject_SetArenaAllocator()`
-   `Py_SetPath()`
-   `Py_SetProgramName()`
-   `Py_SetPythonHome()`
-   `Py_SetStandardStreamEncoding()`
-   `PySys_AddWarnOption()`
-   `PySys_AddXOption()`
-   `PySys_ResetWarnOptions()`

There is also a high-level `Py_Main()` function and
`PyImport_FrozenModules` variable which can be overridden.

See [Initialization, Finalization, and
Threads](https://docs.python.org/dev/c-api/init.html) documentation.

Python Issues
=============

Issues that will be fixed by this PEP, directly or indirectly:

-   [bpo-1195571](https://bugs.python.org/issue1195571): \"simple
    callback system for Py\_FatalError\"
-   [bpo-11320](https://bugs.python.org/issue11320): \"Usage of API
    method Py\_SetPath causes errors in Py\_Initialize() (Posix ony)\"
-   [bpo-13533](https://bugs.python.org/issue13533): \"Would like
    Py\_Initialize to play friendly with host app\"
-   [bpo-14956](https://bugs.python.org/issue14956): \"custom PYTHONPATH
    may break apps embedding Python\"
-   [bpo-19983](https://bugs.python.org/issue19983): \"When interrupted
    during startup, Python should not call abort() but exit()\"
-   [bpo-22213](https://bugs.python.org/issue22213): \"Make pyvenv style
    virtual environments easier to configure when embedding Python\".
-   [bpo-29778](https://bugs.python.org/issue29778):
    \"\_Py\_CheckPython3 uses uninitialized dllpath when embedder sets
    module path with Py\_SetPath\"
-   [bpo-30560](https://bugs.python.org/issue30560): \"Add
    Py\_SetFatalErrorAbortFunc: Allow embedding program to handle fatal
    errors\".
-   [bpo-31745](https://bugs.python.org/issue31745): \"Overloading
    \"Py\_GetPath\" does not work\"
-   [bpo-32573](https://bugs.python.org/issue32573): \"All sys
    attributes (.argv, \...) should exist in embedded environments\".
-   [bpo-33135](https://bugs.python.org/issue33135): \"Define field
    prefixes for the various config structs\". The PEP now defines well
    how warnings options are handled.
-   [bpo-34725](https://bugs.python.org/issue34725):
    \"Py\_GetProgramFullPath() odd behaviour in Windows\"
-   [bpo-36204](https://bugs.python.org/issue36204): \"Deprecate calling
    Py\_Main() after Py\_Initialize()? Add Py\_InitializeFromArgv()?\"

Issues of the PEP implementation:

-   [bpo-16961](https://bugs.python.org/issue16961): \"No regression
    tests for -E and individual environment vars\"
-   [bpo-20361](https://bugs.python.org/issue20361): \"-W command line
    options and PYTHONWARNINGS environmental variable should not
    override -b / -bb command line options\"
-   [bpo-26122](https://bugs.python.org/issue26122): \"Isolated mode
    doesn\'t ignore PYTHONHASHSEED\"
-   [bpo-29818](https://bugs.python.org/issue29818):
    \"Py\_SetStandardStreamEncoding leads to a memory error in debug
    mode\"
-   [bpo-31845](https://bugs.python.org/issue31845):
    \"PYTHONDONTWRITEBYTECODE and PYTHONOPTIMIZE have no effect\"
-   [bpo-32030](https://bugs.python.org/issue32030): \"PEP 432: Rewrite
    Py\_Main()\"
-   [bpo-32124](https://bugs.python.org/issue32124): \"Document
    functions safe to be called before Py\_Initialize()\"
-   [bpo-33042](https://bugs.python.org/issue33042): \"New 3.7 startup
    sequence crashes PyInstaller\"
-   [bpo-33932](https://bugs.python.org/issue33932): \"Calling
    Py\_Initialize() twice now triggers a fatal error (Python 3.7)\"
-   [bpo-34008](https://bugs.python.org/issue34008): \"Do we support
    calling Py\_Main() after Py\_Initialize()?\"
-   [bpo-34170](https://bugs.python.org/issue34170): \"Py\_Initialize():
    computing path configuration must not have side effect (PEP 432)\"
-   [bpo-34589](https://bugs.python.org/issue34589): \"Py\_Initialize()
    and Py\_Main() should not enable C locale coercion\"
-   [bpo-34639](https://bugs.python.org/issue34639):
    \"PYTHONCOERCECLOCALE is ignored when using -E or -I option\"
-   [bpo-36142](https://bugs.python.org/issue36142): \"Add a new
    \_PyPreConfig step to Python initialization to setup memory
    allocator and encodings\"
-   [bpo-36202](https://bugs.python.org/issue36202): \"Calling
    Py\_DecodeLocale() before \_PyPreConfig\_Write() can produce
    mojibake\"
-   [bpo-36301](https://bugs.python.org/issue36301): \"Add
    \_Py\_PreInitialize() function\"
-   [bpo-36443](https://bugs.python.org/issue36443): \"Disable
    coerce\_c\_locale and utf8\_mode by default in \_PyPreConfig?\"
-   [bpo-36444](https://bugs.python.org/issue36444): \"Python
    initialization: remove \_PyMainInterpreterConfig\"
-   [bpo-36471](https://bugs.python.org/issue36471): \"PEP 432, PEP 587:
    Add \_Py\_RunMain()\"
-   [bpo-36763](https://bugs.python.org/issue36763): \"PEP 587: Rework
    initialization API to prepare second version of the PEP\"
-   [bpo-36775](https://bugs.python.org/issue36775): \"Rework filesystem
    codec implementation\"
-   [bpo-36900](https://bugs.python.org/issue36900): \"Use
    \_PyCoreConfig rather than global configuration variables\"

Issues related to this PEP:

-   [bpo-12598](https://bugs.python.org/issue12598): \"Move sys variable
    initialization from import.c to sysmodule.c\"
-   [bpo-15577](https://bugs.python.org/issue15577): \"Real argc and
    argv in embedded interpreter\"
-   [bpo-16202](https://bugs.python.org/issue16202): \"sys.path\[0\]
    security issues\"
-   [bpo-18309](https://bugs.python.org/issue18309): \"Make python
    slightly more relocatable\"
-   [bpo-22257](https://bugs.python.org/issue22257): \"PEP 432: Redesign
    the interpreter startup sequence\"
-   [bpo-25631](https://bugs.python.org/issue25631): \"Segmentation
    fault with invalid Unicode command-line arguments in embedded
    Python\"
-   [bpo-26007](https://bugs.python.org/issue26007): \"Support embedding
    the standard library in an executable\"
-   [bpo-31210](https://bugs.python.org/issue31210): \"Can not import
    modules if sys.prefix contains DELIM\".
-   [bpo-31349](https://bugs.python.org/issue31349): \"Embedded
    initialization ignores Py\_SetProgramName()\"
-   [bpo-33919](https://bugs.python.org/issue33919): \"Expose
    \_PyCoreConfig structure to Python\"
-   [bpo-35173](https://bugs.python.org/issue35173): \"Re-use already
    existing functionality to allow Python 2.7.x (both embedded and
    standalone) to locate the module path according to the shared
    library\"

Discussions
===========

-   May 2019:
    -   [\[Python-Dev\] PEP 587 \"Python Initialization Configuration\"
        version
        4](https://mail.python.org/pipermail/python-dev/2019-May/157492.html)
    -   [\[Python-Dev\] RFC: PEP 587 \"Python Initialization
        Configuration\": 3rd
        version](https://mail.python.org/pipermail/python-dev/2019-May/157435.html)
    -   [Study on applications embedding
        Python](https://mail.python.org/pipermail/python-dev/2019-May/157385.html)
    -   [\[Python-Dev\] RFC: PEP 587 \"Python Initialization
        Configuration\": 2nd
        version](https://mail.python.org/pipermail/python-dev/2019-May/157290.html)
-   March 2019:
    -   [\[Python-Dev\] PEP 587: Python Initialization
        Configuration](https://mail.python.org/pipermail/python-dev/2019-March/156892.html)
    -   [\[Python-Dev\] New Python Initialization
        API](https://mail.python.org/pipermail/python-dev/2019-March/156884.html)
-   February 2019:
    -   [Adding char\* based APIs for
        Unix](https://discuss.python.org/t/adding-char-based-apis-for-unix/916)
-   July-August 2018:
    -   July: [\[Python-Dev\] New \_Py\_InitializeFromConfig() function
        (PEP 432)](https://mail.python.org/pipermail/python-dev/2018-July/154882.html)
    -   August: [\[Python-Dev\] New \_Py\_InitializeFromConfig()
        function
        (PEP 432)](https://mail.python.org/pipermail/python-dev/2018-August/154896.html)

Version History
===============

-   Version 5:
    -   Rename `PyInitError` to `PyStatus`
    -   Rename `PyInitError_Failed()` to `PyStatus_Exception()`
    -   Rename `Py_ExitInitError()` to `Py_ExitStatusException()`
    -   Add `PyPreConfig._config_init` private field.
    -   Fix Python Configuration default values: isolated=0 and
        use\_environment=1, instead of -1.
    -   Add \"Multi-Phase Initialization Private Provisional API\" and
        \"Discussions\" sections
-   Version 4:
    -   Introduce \"Python Configuration\" and \"Isolated
        Configuration\" which are well better defined. Replace all
        macros with functions.
    -   Replace `PyPreConfig_INIT` and `PyConfig_INIT` macros with
        functions:
        -   `PyPreConfig_InitIsolatedConfig()`,
            `PyConfig_InitIsolatedConfig()`
        -   `PyPreConfig_InitPythonConfig()`,
            `PyConfig_InitPythonConfig()`
    -   `PyPreConfig` no longer uses dynamic memory, the `allocator`
        field type becomes an int, add `configure_locale` and
        `parse_argv` field.
    -   `PyConfig`: rename `module_search_path_env` to `pythonpath_env`,
        rename `use_module_search_paths` to `module_search_paths_set`,
        remove `program` and `dll_path`.
    -   Replace `Py_INIT_xxx()` macros with `PyInitError_xxx()`
        functions.
    -   Remove the \"Constant PyConfig\" section. Remove
        `Py_InitializeFromArgs()` and `Py_InitializeFromBytesArgs()`
        functions.
-   Version 3:
    -   `PyConfig`: Add `configure_c_stdio` and `parse_argv`; rename
        `_frozen` to `pathconfig_warnings`.
    -   Rename functions using bytes strings and wide character strings.
        For example, `Py_PreInitializeFromWideArgs()` becomes
        `Py_PreInitializeFromArgs()`, and `PyConfig_SetArgv()` becomes
        `PyConfig_SetBytesArgv()`.
    -   Add `PyWideStringList_Insert()` function.
    -   New \"Path configuration\", \"Isolate Python\", \"Python
        Issues\" and \"Version History\" sections.
    -   `PyConfig_SetString()` and `PyConfig_SetBytesString()` now
        requires the configuration as the first argument.
    -   Rename `Py_UnixMain()` to `Py_BytesMain()`
-   Version 2: Add `PyConfig` methods (ex: `PyConfig_Read()`), add
    `PyWideStringList_Append()`, rename `PyWideCharList` to
    `PyWideStringList`.
-   Version 1: Initial version.

Acceptance
==========

PEP 587 was [accepted by Thomas Wouters on May 26,
2019](https://mail.python.org/pipermail/python-dev/2019-May/157721.html).

Copyright
=========

This document has been placed in the public domain.
