lenstronomy.Sampling.Pool package — lenstronomy 1.8.2 documentation

# lenstronomy.Sampling.Pool package

## Submodules

## lenstronomy.Sampling.Pool.multiprocessing module

this file is taken from schwimmbad (https://github.com/adrn/schwimmbad) and an explicit fork by Aymeric Galan to replace the multiprocessing with the multiprocess dependence as for multi-threading, multiprocessing is not supporting dill (only pickle) which is required.

The class also extends with a `is_master()` definition

class `lenstronomy.Sampling.Pool.multiprocessing.` `MultiPool`(processes=None, initializer=None, initargs=(), **kwargs)

  Bases: `multiprocess.pool.Pool`

  A modified version of `multiprocessing.pool.Pool` that has better behavior with regard to `KeyboardInterrupts` in the `map()` method. (Original author: Peter K. G. Williams)

  processes : int, optional
    The number of worker processes to use; defaults to the number of CPUs.
  initializer : callable, optional
    If specified, a callable that will be invoked by each worker process when it starts.
  initargs : iterable, optional
    Arguments for `initializer`; it will be called as `initializer(*initargs)`.
  kwargs:
    Extra arguments passed to the `multiprocessing.pool.Pool` superclass.
  static `enabled`()
  `is_master`()
  `is_worker`()
  `map`(func, iterable, chunksize=None, callback=None)

    Equivalent to the built-in `map()` function and `multiprocessing.pool.Pool.map()`, without catching `KeyboardInterrupt`.

    func : callable
      A function or callable object that is executed on each element of the specified `tasks` iterable. This object must be picklable (i.e. it can’t be a function scoped within a function or a `lambda` function). This should accept a single positional argument and return a single object.
    iterable : iterable
      A list or iterable of tasks. Each task can be itself an iterable (e.g., tuple) of values or data to pass in to the worker function.
    callback : callable, optional
      An optional callback function (or callable) that is called with the result from each worker run and is executed on the master process. This is useful for, e.g., saving results to a file, since the callback is only called on the master thread.
    results : list
      A list of results from the output of each `worker()` call.
  `wait_timeout` = 3600

## lenstronomy.Sampling.Pool.pool module

this file is taken from schwimmbad (https://github.com/adrn/schwimmbad) and an explicit fork by Aymeric Galan to replace the multiprocessing with the multiprocess dependence as for multi-threading, multiprocessing is not supporting dill (only pickle) which is required.

Tests show that the MPI mode works with Python 3.7.2 but not with Python 3.7.0 on a specific system due to mpi4py dependencies and configurations.

Contributions by: - Peter K. G. Williams - Júlio Hoffimann Mendes - Dan Foreman-Mackey - Aymeric Galan - Simon Birrer

Implementations of four different types of processing pools:

> - MPIPool: An MPI pool.
> - MultiPool: A multiprocessing for local parallelization.
> - SerialPool: A serial pool, which uses the built-in map function

`lenstronomy.Sampling.Pool.pool.` `choose_pool`(mpi=False, processes=1, **kwargs)

  Extends the capabilities of the schwimmbad.choose_pool method.

  It handles the use_dill parameters in kwargs, that would otherwise raise an error when processes > 1. Any thread in the returned multiprocessing pool (e.g. processes > 1) also default

  The requirement of schwimmbad relies on the master branch (as specified in requirements.txt). The ‘use_dill’ functionality can raise if not following the requirement specified.

  Choose between the different pools given options from, e.g., argparse.

  mpi : bool, optional
    Use the MPI processing pool, `MPIPool`. By default, `False`, will use the `SerialPool`.
  processes : int, optional
    Use the multiprocessing pool, `MultiPool`, with this number of processes. By default, `processes=1`, will use them:class:~schwimmbad.serial.SerialPool.

  Any additional kwargs are passed in to the pool class initializer selected by the arguments.

## Module contents