betty.multiprocessing module - Betty documentation

Hide navigation sidebar
Hide table of contents sidebar
Skip to content
Back to top
View this page
Toggle table of contents sidebar

# betty.multiprocessing module

Multiprocessing functionality.

class betty.multiprocessing.ProcessPoolExecutor

  Bases: `ProcessPoolExecutor`

  Like `concurrent.futures.ProcessPoolExecutor`, but with error handling and low memory consumption.

  This - uses the `spawn` method to create new processes, which is the Python 3.14 default. - ignores SIGINT/`KeyboardInterrupt` delivered to it by the parent process to prevent unhelpful

  > additional `KeyboardInterrupt` being raised from within the process pool.

  __init__(max_workers: int | None = None, *, max_tasks_per_child: int | None = None)

    Initializes a new ProcessPoolExecutor instance.

    Args:
      max_workers: The maximum number of processes that can be used to

        execute the given calls. If None or not given then as many worker processes will be created as the machine has processors.

      mp_context: A multiprocessing context to launch the workers created

        using the multiprocessing.get_context(‘start method’) API. This object should provide SimpleQueue, Queue and Process.

      initializer: A callable used to initialize worker processes. initargs: A tuple of arguments to pass to the initializer. max_tasks_per_child: The maximum number of tasks a worker process

      > can complete before it will exit and be replaced with a fresh worker process. The default of None means worker process will live as long as the executor. Requires a non-‘fork’ mp_context start method. When given, we default to using ‘spawn’ if no mp_context is supplied.