pyfscache — pyfscache 0.9.9 documentation

### Navigation

- index
- modules |
- pyfscache 0.9.9 documentation »

# pyfscache

A simple filesystem cache for python.

## Home Page, Documentation, & Repository

- Home Page: http://pyfscache.bravais.net
- Online Documentation: http://pythonhosted.org/pyfscache/
- Code Repository: https://github.com/jcstroud/pyfscache/

## Introduction

Pyfscache (python filesystem cache) is a filesystem cache that is easy to use. The principal class is `pyfscache.FSCache`, instances of which may be used as decorators to create cached functions with very little coding overhead:

    import pyfscache
    cache_it = pyfscache.FSCache('some/cache/directory',
                                 days=13, hours=4, minutes=2.5)
    @cache_it
    def cached_doit(a, b, c):
      return [a, b, c]

It’s that simple!

Now, every time the function `cached_doit()` is called with a particular set of arguments, the cache `cache_it` is inspected to see if an identical call has been made before. If it has, then the return value is retrieved from the `cache_it` cache. If not, the return value is calculated with `cached_doit()`, stored in the cache, and then returned.

## Expiration

In the code above, the expiration for `cache_it` is set to 1,137,750 seconds (13 days, 4 hours, and 2.5 minutes), which means that every item created by `cache_it` has a lifetime of 1,137,750 seconds, beginning when the item is made (i.e not beginning when `cache_it` is made). Values specifying lifetime may be provided with the keywords `years`, `months`, `weeks`, `days`, `hours`, `minutes`, and `seconds`. The lifetime is the total for all keywords.

If these optional keyword arguments are not included, then items added by the `pyfscache.FSCache` object never expire:

    no_expiry_cache = pyfscache.FSCache('some/cache/directory')

Note

Several instances of `pyfscache.FSCache` objects can use the same cache directory. Each will honor the expirations of the items therein. Thus, it is possible to have a cache mixed with objects of many differening lifetimes, made by many instances of `pyfscache.FSCache`.

## Works Like a Map

Instances of `phyles.FSCache` work like mapping objects, supporting item getting and setting:

    >>> cache_it[('some', ['key'])] = {'some': 'value'}
    >>> cache_it[('some', ['key'])]
    {'some': 'value}

However, deletion with the `del` statement only works on memory. To erase an item in the cache directory, use `phyles.FSCache.expire()`:

    >>> cache_it.get_loaded()
    ['LIlWpBZL68MBJaXouRjFBL3fzScyxh5q56hqSZ3DBK']
    >>> del cache_it[('some', ['key'])]
    >>> cache_it.get_loaded()
    []
    >>> ('some', ['key']) in cache_it
    True
    >>> cache_it[('some', ['key'])]
    {'some': 'value}
    >>> cache_it.expire(('some', ['key']))
    >>> ('some', ['key']) in cache_it
    False

## Decorators

What if you didn’t write the function you want to cache? Although their convenience is manifest in the example above, it is not necessary to use decorators:

    import pyfscache
    cache = pyfscache.FSCache('some/cache/directory',
                              days=13, hours=4, minutes=2.5)

    def uncached_doit(a, b, c):
      return [a, b, c]

    cached_doit = cache(uncached_doit)

## Versatility

`phyles.FSCache` objects should work on the vast majority of python “callables”, including instance methods and even built-ins:

    # a cached built-in
    cached_list = cache_it(list)

    # a cached instance method
    def AClass(object):
      @cahe_it
      def some_cached_instance_method(self, a, r, g, s):
        return (a + r) / (g * s)

Note

The rule of thumb is that if python’s `cPickle` module can handle the expected arguments to the cached function, then so can pyfscache.

# API

pyfscache: A file system cache for python. Copyright (c) 2013, James C. Stroud; All rights reserved.

class `pyfscache.` `FSCache`(path, **kwargs)

  A class that manages a filesystem cache. Works like a dictionary and can decorate functions to make them cached.

  A `pyfscache.FSCache` object is instantiated with a path and optional lifetime keyword arguments:

      >>> c = FSCache('cache/dir', days=7)

  This command creates a new FSCache instance at the given path (`cache/dir`). Each item added by this cache has a lifetime of 7 days, starting when the item (not the cache) is created. If the path doesn’t exist, one is made. New items added to the cache are given a lifetime expressed by the keyword arguments with potential keys of `years`, `months`, `weeks`, `days`, `hours`, `minutes`, `seconds` (see `to_seconds()`). If no keyword arguments are given, then the items added by the cache do not expire automatically.

  Creating an `pyfscache.FSCache` object does not purge the cache in path if the cache already exists. Instead, the `pyfscache.FSCache` object will begin to use the cache, loading items and storing items as necessary.

      >>> import os
      >>> import shutil
      >>> from pyfscache import *
      >>> if os.path.exists('cache/dir'):
      ...   shutil.rmtree('cache/dir')
      ...
      >>> c = FSCache('cache/dir', days=7)
      >>> c['some_key'] = "some_value"
      >>> c['some_key']
      'some_value'
      >>> os.listdir('cache/dir')
      ['PXBZzwEy3XnbOweuMtoPj9j=PwkfAsTXexmW2v05JD']
      >>> c.expire('some_key')
      >>> os.listdir('cache/dir')
      []
      >>> c['some_key'] = "some_value"
      >>> @c
      ... def doit(avalue):
      ...   print "had to call me!"
      ...   return "some other value"
      ...
      >>> doit('some input')
      had to call me!
      'some other value'
      >>> doit('some input')
      'some other value'
      >>> shutil.rmtree('cache/dir')

  `clear`()

    Unloads all loaded cache items from memory. All cache items remain on the disk, however.

  `expire`(k)

    Use with care. This permanently removes the object keyed by k from the cache, both in the memory and in the filesystem.

  `expiry`()

    Returns an expiry for the cache in seconds as if the start of the expiration period were the moment at which this the method is called.

        >>> import time
        >>> c = FSCache('cache/dir', seconds=60)
        >>> round(c.expiry() - time.time(), 3)
        60.0

  `get_lifetime`()

    Returns the lifetime, in seconds, of new items in the cache. If new items do not expire, then `None` is returned.

  `get_loaded`()

    Returns a list of keys for all objects that are loaded.

  `get_names`()

    Returns the names of the files in the cache on the filesystem. These are not keys but one-way hashes (or “digests”) of the keys created by `make_digest()`.

  `get_path`()

    Returns the absolute path to the file system cache represented by the instance.

  `is_loaded`(k)

    Returns `True` if the item keyed by k has been loaded, `False` if not.

  `lifetime`

    Returns the lifetime, in seconds, of new items in the cache. If new items do not expire, then `None` is returned.

  `load`(k)

    Causes the object keyed by k to be loaded from the file system and returned. It therefore causes this object to reside in memory.

  `path`

    Returns the absolute path to the file system cache represented by the instance.

  `purge`()

    Be careful, this empties the cache from both the filesystem and memory!

  `unload`(k)

    Removes the object keyed by k from memory but not from the filesystem. To remove it from both memory and permanently from the filesystem, use expire.

  `update_item`(k, v)

    Use with care. Updates, both in memory and on the filesystem, the object for key k with the object v. If the key k already exists with a stored object, it will be replaced.

`pyfscache.` `make_digest`(k)

  Creates a digest suitable for use within an `phyles.FSCache` object from the key object k.

      >>> adict = {'a' : {'b':1}, 'f': []}
      >>> make_digest(adict)
      'a2VKynHgDrUIm17r6BQ5QcA5XVmqpNBmiKbZ9kTu0A'

`pyfscache.` `auto_cache_function`(f, cache)

  Creates a cached function from function f. The cache can be any mapping object, such as FSCache objects.

  The function arguments are expected to be well-behaved for python’s `cPickle`. Or, in other words, the expected values for the parameters (the arguments) should be instances new-style classes (i.e. inheriting from `object`) or implement `__getstate__()` with well-behaved results.

  If the arguments to f are not expected to be well-behaved, it is best to use cache_function instead and create a custom keyer.

`pyfscache.` `cache_function`(f, keyer, cache)

  Takes any function f and a function that creates a key, keyer and caches the result in cache.

  The keys created by keyer should be well behaved for python’s `cPickle`. See the documentation for `auto_cache_funtion()` for details.

  It is best to have a unique keyer for every function.

`pyfscache.` `to_seconds`(**kwargs)

  Converts keyword arguments to seconds.

  The the keyword arguments can have the following keys:

  > - `years` (31,556,900 seconds per year)
  > - `months` (2,629,740 seconds per month)
  > - `weeks` (604,800 seconds per week)
  > - `days` (86,400 seconds per day)
  > - `hours` (3600 seconds per hour)
  > - `minutes` (60 seconds per minute)
  > - `seconds`

      >>> to_seconds(seconds=15, minutes=20)
      1215.0
      >>> to_seconds(seconds=15.42, hours=10, minutes=18, years=2)
      63150895.42

# Indices and tables

- Index
- Module Index
- Search Page

### Navigation

- index
- modules |
- pyfscache 0.9.9 documentation »