Caching — Mako 1.0.4 Documentation

Hyperfast and lightweight templating for the Python platform.

# Mako 1.0.4 Documentation

Search:
Release: 1.0.4

# Caching

Any template or component can be cached using the `cache` argument to the `<%page>`, `<%def>` or `<%block>` directives:

    <%page cached="True"/>

    template text

The above template, after being executed the first time, will store its content within a cache that by default is scoped within memory. Subsequent calls to the template’s `render()` method will return content directly from the cache. When the `Template` object itself falls out of scope, its corresponding cache is garbage collected along with the template.

The caching system requires that a cache backend be installed; this includes either the Beaker package or the dogpile.cache, as well as any other third-party caching libraries that feature Mako integration.

By default, caching will attempt to make use of Beaker. To use dogpile.cache, the `cache_impl` argument must be set; see this argument in the section Cache Arguments.

In addition to being available on the `<%page>` tag, the caching flag and all its options can be used with the `<%def>` tag as well:

    <%def name="mycomp" cached="True" cache_timeout="60">
        other text
    </%def>

... and equivalently with the `<%block>` tag, anonymous or named:

    <%block cached="True" cache_timeout="60">
        other text
    </%block>

## Cache Arguments

Mako has two cache arguments available on tags that are available in all cases. The rest of the arguments available are specific to a backend.

The two generic tags arguments are:

- `cached="True"` - enable caching for this `<%page>`, `<%def>`, or `<%block>`.

- `cache_key` - the “key” used to uniquely identify this content in the cache. Usually, this key is chosen automatically based on the name of the rendering callable (i.e. `body` when used in `<%page>`, the name of the def when using `<%def>`, the explicit or internally-generated name when using `<%block>`). Using the `cache_key` parameter, the key can be overridden using a fixed or programmatically generated value.

  For example, here’s a page that caches any page which inherits from it, based on the filename of the calling template:

      <%page cached="True" cache_key="${self.filename}"/>

      ${next.body()}

      ## rest of template

On a `Template` or `TemplateLookup`, the caching can be configured using these arguments:

- `cache_enabled` - Setting this to `False` will disable all caching functionality when the template renders. Defaults to `True`. e.g.:

      lookup = TemplateLookup(
                      directories='/path/to/templates',
                      cache_enabled = False
                      )

- `cache_impl` - The string name of the cache backend to use. This defaults to `'beaker'`, indicating that the ‘beaker’ backend will be used.

- `cache_args` - A dictionary of cache parameters that will be consumed by the cache backend. See Using the Beaker Cache Backend and Using the dogpile.cache Backend for examples.

### Backend-Specific Cache Arguments

The `<%page>`, `<%def>`, and `<%block>` tags accept any named argument that starts with the prefix `"cache_"`. Those arguments are then packaged up and passed along to the underlying caching implementation, minus the `"cache_"` prefix.

The actual arguments understood are determined by the backend.

- Using the Beaker Cache Backend - Includes arguments understood by Beaker.
- Using the dogpile.cache Backend - Includes arguments understood by dogpile.cache.

### Using the Beaker Cache Backend

When using Beaker, new implementations will want to make usage of cache regions so that cache configurations can be maintained externally to templates. These configurations live under named “regions” that can be referred to within templates themselves.

New in version 0.6.0: Support for Beaker cache regions.

For example, suppose we would like two regions. One is a “short term” region that will store content in a memory-based dictionary, expiring after 60 seconds. The other is a Memcached region, where values should expire in five minutes. To configure our `TemplateLookup`, first we get a handle to a `beaker.cache.CacheManager`:

    from beaker.cache import CacheManager

    manager = CacheManager(cache_regions={
        'short_term':{
            'type': 'memory',
            'expire': 60
        },
        'long_term':{
            'type': 'ext:memcached',
            'url': '127.0.0.1:11211',
            'expire': 300
        }
    })

    lookup = TemplateLookup(
                    directories=['/path/to/templates'],
                    module_directory='/path/to/modules',
                    cache_impl='beaker',
                    cache_args={
                        'manager':manager
                    }
            )

Our templates can then opt to cache data in one of either region, using the `cache_region` argument. Such as using `short_term` at the `<%page>` level:

    <%page cached="True" cache_region="short_term">

    ## ...

Or, `long_term` at the `<%block>` level:

    <%block name="header" cached="True" cache_region="long_term">
        other text
    </%block>

The Beaker backend also works without regions. There are a variety of arguments that can be passed to the `cache_args` dictionary, which are also allowable in templates via the `<%page>`, `<%block>`, and `<%def>` tags specific to those sections. The values given override those specified at the `TemplateLookup` or `Template` level.

With the possible exception of `cache_timeout`, these arguments are probably better off staying at the template configuration level. Each argument specified as `cache_XYZ` in a template tag is specified without the `cache_` prefix in the `cache_args` dictionary:

- `cache_timeout` - number of seconds in which to invalidate the cached data. After this timeout, the content is re-generated on the next call. Available as `timeout` in the `cache_args` dictionary.
- `cache_type` - type of caching. `'memory'`, `'file'`, `'dbm'`, or `'ext:memcached'` (note that the string `memcached` is also accepted by the dogpile.cache Mako plugin, though not by Beaker itself). Available as `type` in the `cache_args` dictionary.
- `cache_url` - (only used for `memcached` but required) a single IP address or a semi-colon separated list of IP address of memcache servers to use. Available as `url` in the `cache_args` dictionary.
- `cache_dir` - in the case of the `'file'` and `'dbm'` cache types, this is the filesystem directory with which to store data files. If this option is not present, the value of `module_directory` is used (i.e. the directory where compiled template modules are stored). If neither option is available an exception is thrown. Available as `dir` in the `cache_args` dictionary.

### Using the dogpile.cache Backend

dogpile.cache is a new replacement for Beaker. It provides a modernized, slimmed down interface and is generally easier to use than Beaker. As of this writing it has not yet been released. dogpile.cache includes its own Mako cache plugin – see `dogpile.cache.plugins.mako_cache` in the dogpile.cache documentation.

## Programmatic Cache Access

The `Template`, as well as any template-derived `Namespace`, has an accessor called `cache` which returns the `Cache` object for that template. This object is a facade on top of the underlying `CacheImpl` object, and provides some very rudimental capabilities, such as the ability to get and put arbitrary values:

    <%
        local.cache.set("somekey", type="memory", "somevalue")
    %>

Above, the cache associated with the `local` namespace is accessed and a key is placed within a memory cache.

More commonly, the `cache` object is used to invalidate cached sections programmatically:

    template = lookup.get_template('/sometemplate.html')

    # invalidate the "body" of the template
    template.cache.invalidate_body()

    # invalidate an individual def
    template.cache.invalidate_def('somedef')

    # invalidate an arbitrary key
    template.cache.invalidate('somekey')

You can access any special method or attribute of the `CacheImpl` itself using the `impl` attribute:

    template.cache.impl.do_something_special()

Note that using implementation-specific methods will mean you can’t swap in a different kind of `CacheImpl` implementation at a later time.

## Cache Plugins

The mechanism used by caching can be plugged in using a `CacheImpl` subclass. This class implements the rudimental methods Mako needs to implement the caching API. Mako includes the `BeakerCacheImpl` class to provide the default implementation. A `CacheImpl` class is acquired by Mako using a `pkg_resources` entrypoint, using the name given as the `cache_impl` argument to `Template` or `TemplateLookup`. This entry point can be installed via the standard setuptools/`setup()` procedure, underneath the EntryPoint group named `"mako.cache"`. It can also be installed at runtime via a convenience installer `register_plugin()` which accomplishes essentially the same task.

An example plugin that implements a local dictionary cache:

    from mako.cache import Cacheimpl, register_plugin

    class SimpleCacheImpl(CacheImpl):
        def __init__(self, cache):
            super(SimpleCacheImpl, self).__init__(cache)
            self._cache = {}

        def get_or_create(self, key, creation_function, **kw):
            if key in self._cache:
                return self._cache[key]
            else:
                self._cache[key] = value = creation_function()
                return value

        def set(self, key, value, **kwargs):
            self._cache[key] = value

        def get(self, key, **kwargs):
            return self._cache.get(key)

        def invalidate(self, key, **kwargs):
            self._cache.pop(key, None)

    # optional - register the class locally
    register_plugin("simple", __name__, "SimpleCacheImpl")

Enabling the above plugin in a template would look like:

    t = Template("mytemplate",
                 file="mytemplate.html",
                 cache_impl='simple')

### Guidelines for Writing Cache Plugins

- The `CacheImpl` is created on a per-`Template` basis. The class should ensure that only data for the parent `Template` is persisted or returned by the cache methods. The actual `Template` is available via the `self.cache.template` attribute. The `self.cache.id` attribute, which is essentially the unique modulename of the template, is a good value to use in order to represent a unique namespace of keys specific to the template.
- Templates only use the `CacheImpl.get_or_create()` method in an implicit fashion. The `CacheImpl.set()`, `CacheImpl.get()`, and `CacheImpl.invalidate()` methods are only used in response to direct programmatic access to the corresponding methods on the `Cache` object.
- `CacheImpl` will be accessed in a multithreaded fashion if the `Template` itself is used multithreaded. Care should be taken to ensure caching implementations are threadsafe.
- A library like Dogpile, which is a minimal locking system derived from Beaker, can be used to help implement the `CacheImpl.get_or_create()` method in a threadsafe way that can maximize effectiveness across multiple threads as well as processes. `CacheImpl.get_or_create()` is the key method used by templates.
- All arguments passed to `**kw` come directly from the parameters inside the `<%def>`, `<%block>`, or `<%page>` tags directly, minus the `"cache_"` prefix, as strings, with the exception of the argument `cache_timeout`, which is passed to the plugin as the name `timeout` with the value converted to an integer. Arguments present in `cache_args` on `Template` or `TemplateLookup` are passed directly, but are superseded by those present in the most specific template tag.
- The directory where `Template` places module files can be acquired using the accessor `self.cache.template.module_directory`. This directory can be a good place to throw cache-related work files, underneath a prefix like `_my_cache_work` so that name conflicts with generated modules don’t occur.

## API Reference

class `mako.cache.` `Cache`(template, *args)

  Bases: `object`

  Represents a data content cache made available to the module space of a specific `Template` object.

  New in version 0.6: `Cache` by itself is mostly a container for a `CacheImpl` object, which implements a fixed API to provide caching services; specific subclasses exist to implement different caching strategies. Mako includes a backend that works with the Beaker caching system. Beaker itself then supports a number of backends (i.e. file, memory, memcached, etc.)

  The construction of a `Cache` is part of the mechanics of a `Template`, and programmatic access to this cache is typically via the `Template.cache` attribute.

  `get`(key, **kw)

    Retrieve a value from the cache.

    Parameters: | key – the value’s key. **kw – cache configuration arguments. The backend is configured using these arguments upon first request. Subsequent requests that use the same series of configuration values will use that same backend.

  `get_or_create`(key, creation_function, **kw)

    Retrieve a value from the cache, using the given creation function to generate a new value.

  `id` = None

    Return the ‘id’ that identifies this cache.

    This is a value that should be globally unique to the `Template` associated with this cache, and can be used by a caching system to name a local container for data specific to this template.

  `impl` = None

    Provide the `CacheImpl` in use by this `Cache`.

    This accessor allows a `CacheImpl` with additional methods beyond that of `Cache` to be used programmatically.

  `invalidate`(key, **kw)

    Invalidate a value in the cache.

    Parameters: | key – the value’s key. **kw – cache configuration arguments. The backend is configured using these arguments upon first request. Subsequent requests that use the same series of configuration values will use that same backend.

  `invalidate_body`()

    Invalidate the cached content of the “body” method for this template.

  `invalidate_closure`(name)

    Invalidate a nested `<%def>` within this template.

    Caching of nested defs is a blunt tool as there is no management of scope – nested defs that use cache tags need to have names unique of all other nested defs in the template, else their content will be overwritten by each other.

  `invalidate_def`(name)

    Invalidate the cached content of a particular `<%def>` within this template.

  `put`(key, value, **kw)

    A synonym for `Cache.set()`.

    This is here for backwards compatibility.

  `set`(key, value, **kw)

    Place a value in the cache.

    Parameters: | key – the value’s key. value – the value. **kw – cache configuration arguments.

  `starttime` = None

    Epochal time value for when the owning `Template` was first compiled.

    A cache implementation may wish to invalidate data earlier than this timestamp; this has the effect of the cache for a specific `Template` starting clean any time the `Template` is recompiled, such as when the original template file changed on the filesystem.

class `mako.cache.` `CacheImpl`(cache)

  Bases: `object`

  Provide a cache implementation for use by `Cache`.

  `get`(key, **kw)

    Retrieve a value from the cache.

    Parameters: | key – the value’s key. **kw – cache configuration arguments.

  `get_or_create`(key, creation_function, **kw)

    Retrieve a value from the cache, using the given creation function to generate a new value.

    This function must return a value, either from the cache, or via the given creation function. If the creation function is called, the newly created value should be populated into the cache under the given key before being returned.

    Parameters: | key – the value’s key. creation_function – function that when called generates a new value. **kw – cache configuration arguments.

  `invalidate`(key, **kw)

    Invalidate a value in the cache.

    Parameters: | key – the value’s key. **kw – cache configuration arguments.

  `pass_context` = False

    If `True`, the `Context` will be passed to `get_or_create` as the name `'context'`.

  `set`(key, value, **kw)

    Place a value in the cache.

    Parameters: | key – the value’s key. value – the value. **kw – cache configuration arguments.

`mako.cache.` `register_plugin`(self, name, modulepath, objname)

class `mako.ext.beaker_cache.` `BeakerCacheImpl`(cache)

  Bases: `mako.cache.CacheImpl`

  A `CacheImpl` provided for the Beaker caching system.

  This plugin is used by default, based on the default value of `'beaker'` for the `cache_impl` parameter of the `Template` or `TemplateLookup` classes.

Website content copyright © by Michael Bayer. All rights reserved. Mako and its documentation are licensed under the MIT license. mike(&)zzzcomputing.com