joblib.Memory — joblib 1.3.0.dev0 documentation

# joblib.Memory

class joblib.Memory(location=None, backend='local', mmap_mode=None, compress=False, verbose=1, bytes_limit=None, backend_options=None)

  A context object for caching a function’s return value each time it is called with the same input arguments.

  All values are cached on the filesystem, in a deep directory structure.

  Read more in the User Guide.

  Parameters:
    location: str, pathlib.Path or None

      The path of the base directory to use as a data store or None. If None is given, no caching is done and the Memory object is completely transparent. This option replaces cachedir since version 0.12.

    backend: str, optional

      Type of store backend for reading/writing cache files. Default: ‘local’. The ‘local’ backend is using regular filesystem operations to manipulate data (open, mv, etc) in the backend.

    mmap_mode: {None, ‘r+’, ‘r’, ‘w+’, ‘c’}, optional

      The memmapping mode used when loading from cache numpy arrays. See numpy.load for the meaning of the arguments.

    compress: boolean, or integer, optional

      Whether to zip the stored data on disk. If an integer is given, it should be between 1 and 9, and sets the amount of compression. Note that compressed arrays cannot be read by memmapping.

    verbose: int, optional

      Verbosity flag, controls the debug messages that are issued as functions are evaluated.

    bytes_limit: int | str, optional

      Limit in bytes of the size of the cache. By default, the size of the cache is unlimited. When reducing the size of the cache, `joblib` keeps the most recently accessed items first. If a str is passed, it is converted to a number of bytes using units { K | M | G} for kilo, mega, giga.

      Note: You need to call `joblib.Memory.reduce_size()` to actually reduce the cache size to be less than `bytes_limit`.

      Note: This argument has been deprecated. One should give the value of `bytes_limit` directly in `joblib.Memory.reduce_size()`.

    backend_options: dict, optional

      Contains a dictionary of named parameters used to configure the store backend.

  cache(func=None, ignore=None, verbose=None, mmap_mode=False, cache_validation_callback=None)

    Decorates the given function func to only compute its return value for input arguments not cached on disk.

    Parameters:
      func: callable, optional

        The function to be decorated

      ignore: list of strings

        A list of arguments name to ignore in the hashing

      verbose: integer, optional

        The verbosity mode of the function. By default that of the memory object is used.

      mmap_mode: {None, ‘r+’, ‘r’, ‘w+’, ‘c’}, optional

        The memmapping mode used when loading from cache numpy arrays. See numpy.load for the meaning of the arguments. By default that of the memory object is used.

      cache_validation_callback: callable, optional

        Callable to validate whether or not the cache is valid. When the cached function is called with arguments for which a cache exists, this callable is called with the metadata of the cached result as its sole argument. If it returns True, then the cached result is returned, else the cache for these arguments is cleared and recomputed.

    Returns:
      decorated_func: MemorizedFunc object

        The returned object is a MemorizedFunc object, that is callable (behaves like a function), but offers extra methods for cache lookup and management. See the documentation for `joblib.memory.MemorizedFunc`.

  clear(warn=True)

    Erase the complete cache directory.

  eval(func, *args, **kwargs)

    Eval function func with arguments *args and **kwargs, in the context of the memory.

    This method works similarly to the builtin apply, except that the function is called only if the cache is not up to date.

  reduce_size(bytes_limit=None, items_limit=None, age_limit=None)

    Remove cache elements to make the cache fit its limits.

    The limitation can impose that the cache size fits in `bytes_limit`, that the number of cache items is no more than `items_limit`, and that all files in cache are not older than `age_limit`.

    Parameters:
      bytes_limit: int | str, optional

        Limit in bytes of the size of the cache. By default, the size of the cache is unlimited. When reducing the size of the cache, `joblib` keeps the most recently accessed items first. If a str is passed, it is converted to a number of bytes using units { K | M | G} for kilo, mega, giga.

      items_limit: int, optional

        Number of items to limit the cache to. By default, the number of items in the cache is unlimited. When reducing the size of the cache, `joblib` keeps the most recently accessed items first.

      age_limit: datetime.timedelta, optional

        Maximum age of items to limit the cache to. When reducing the size of the cache, any items last accessed more than the given length of time ago are deleted.

## Examples using `joblib.Memory`

Checkpoint using joblib.Memory and joblib.Parallel

Checkpoint using joblib.Memory and joblib.Parallel

How to use joblib.Memory

How to use joblib.Memory