Template Loaders — Jinja Documentation

# Jinja

## Template Loaders

This part of the documentation explains how to use and write a template loader.

## Builtin Loaders

This list contains the builtin loaders you can use without further modification:

ChoiceLoader

  A loader that tries multiple loaders in the order they are given to the ChoiceLoader:

      from jinja import ChoiceLoader, FileSystemLoader
      loader1 = FileSystemLoader("templates1")
      loader2 = FileSystemLoader("templates2")
      loader = ChoiceLoader([loader1, loader2])

DictLoader

  Load templates from a given dict:

      from jinja import Environment, DictLoader
      e = Environment(loader=DictLoader(dict(
          layout='...',
          index='{% extends 'layout' %}...'
      )))

  This loader does not have any caching capabilities.

FileSystemLoader

  Loads templates from the filesystem:

      from jinja import Environment, FileSystemLoader
      e = Environment(loader=FileSystemLoader('templates/'))

  You can pass the following keyword arguments to the loader on initialization:

  `searchpath` | String with the path to the templates on the filesystem.
  `use_memcache` | Set this to `True` to enable memory caching. This is usually a good idea in production mode, but disable it during development since it won't reload template changes automatically. This only works in persistent environments like FastCGI.
  `memcache_size` | Number of template instance you want to cache. Defaults to `40`.
  `cache_folder` | Set this to an existing directory to enable caching of templates on the file system. Note that this only affects templates transformed into python code. Default is `None` which means that caching is disabled.
  `auto_reload` | Set this to False for a slightly better performance. In that case Jinja won't check for template changes on the filesystem.
  `cache_salt` | Optional unique number to not confuse the caching system when caching more than one template loader in the same folder. Defaults to the searchpath. New in Jinja 1.1

FunctionLoader

  Loads templates by calling a function which has to return a string or None if an error occoured.

      from jinja import Environment, FunctionLoader

      def my_load_func(template_name):
          if template_name == 'foo':
              return '...'

      e = Environment(loader=FunctionLoader(my_load_func))

  Because the interface is limited there is no way to cache such templates. Usually you should try to use a loader with a more solid backend.

  You can pass the following keyword arguments to the loader on initialization:

  `loader_func` | Function that takes the name of the template to load. If it returns a string or unicode object it's used to load a template. If the return value is None it's considered missing.
  `getmtime_func` | Function used to check if templates requires reloading. Has to return the UNIX timestamp of the last template change or `-1` if this template does not exist or requires updates at any cost.
  `use_memcache` | Set this to `True` to enable memory caching. This is usually a good idea in production mode, but disable it during development since it won't reload template changes automatically. This only works in persistent environments like FastCGI.
  `memcache_size` | Number of template instance you want to cache. Defaults to `40`.
  `cache_folder` | Set this to an existing directory to enable caching of templates on the file system. Note that this only affects templates transformed into python code. Default is `None` which means that caching is disabled.
  `auto_reload` | Set this to False for a slightly better performance. In that case of getmtime_func not being provided this won't have an effect.
  `cache_salt` | Optional unique number to not confuse the caching system when caching more than one template loader in the same folder.

MemcachedFileSystemLoader

  Loads templates from the filesystem and caches them on a memcached server.

      from jinja import Environment, MemcachedFileSystemLoader
      e = Environment(loader=MemcachedFileSystemLoader('templates/',
          memcache_host=['192.168.2.250:11211']
      ))

  You can pass the following keyword arguments to the loader on initialization:

  `searchpath` | String with the path to the templates on the filesystem.
  `use_memcache` | Set this to `True` to enable memcached caching. In that case it behaves like a normal FileSystemLoader with disabled caching.
  `memcache_time` | The expire time of a template in the cache.
  `memcache_host` | a list of memcached servers.
  `item_prefix` | The prefix for the items on the server. Defaults to `'template/'`.

PackageLoader

  Loads templates from python packages using setuptools.

      from jinja import Environment, PackageLoader
      e = Environment(loader=PackageLoader('yourapp', 'template/path'))

  You can pass the following keyword arguments to the loader on initialization:

  `package_name` | Name of the package containing the templates.
  `package_path` | Path of the templates inside the package.
  `use_memcache` | Set this to `True` to enable memory caching. This is usually a good idea in production mode, but disable it during development since it won't reload template changes automatically. This only works in persistent environments like FastCGI.
  `memcache_size` | Number of template instance you want to cache. Defaults to `40`.
  `cache_folder` | Set this to an existing directory to enable caching of templates on the file system. Note that this only affects templates transformed into python code. Default is `None` which means that caching is disabled.
  `auto_reload` | Set this to False for a slightly better performance. In that case Jinja won't check for template changes on the filesystem. If the templates are inside of an egg file this won't have an effect.
  `cache_salt` | Optional unique number to not confuse the caching system when caching more than one template loader in the same folder. Defaults to `package_name + '/' + package_path`. New in Jinja 1.1

  Important note: If you're using an application that is inside of an egg never set auto_reload to True. The egg resource manager will automatically export files to the file system and touch them so that you not only end up with additional temporary files but also an automatic reload each time you load a template.

## Loader Baseclasses

With Jinja 1.1 onwards all the loaders have (except of the uncached) baseclasses. You can use them to mix your own caching layer in. This technique is described below. The BaseLoader itself is also a loader baseclass but because it's the baseclass of all loaders it's covered in the "Developing Loaders" section.

BaseFileSystemLoader

  Baseclass for the file system loader that does not do any caching. It exists to avoid redundant code, just don't use it without subclassing.

  How subclassing can work:

      from jinja.loaders import BaseFileSystemLoader

      class MyFileSystemLoader(BaseFileSystemLoader):
          def __init__(self):
              BaseFileSystemLoader.__init__(self, '/path/to/templates')

  The base file system loader only takes one parameter beside self which is the path to the templates.

BaseFunctionLoader

  Baseclass for the function loader that doesn't do any caching.

  It just accepts one parameter which is the function which is called with the name of the requested template. If the return value is None the loader will raise a TemplateNotFound error.

      from jinja.loaders import BaseFunctionLoader

      templates = {...}

      class MyFunctionLoader(BaseFunctionLoader):
          def __init__(self):
              BaseFunctionLoader(templates.get)

BasePackageLoader

  Baseclass for the package loader that does not do any caching.

  It accepts two parameters: The name of the package and the path relative to the package:

      from jinja.loaders import BasePackageLoader

      class MyPackageLoader(BasePackageLoader):
          def __init__(self):
              BasePackageLoader.__init__(self, 'my_package', 'shared/templates')

  The relative path must use slashes as path delimiters, even on Mac OS and Microsoft Windows.

  It uses the pkg_resources libraries distributed with setuptools for retrieving the data from the packages. This works for eggs too so you don't have to mark your egg as non zip safe. If pkg_resources is not available it just falls back to path joining relative to the package.

## Developing Loaders

Template loaders are just normal Python classes that have to provide some functions used to load and translate templates. Because some of the tasks a loader has to do are redundant there are some classes that make loader development easier.

Here the implementation of a simple loader based on the BaseLoader from jinja.loaders:

    import codecs
    from os.path import join
    from jinja.loaders import BaseLoader
    from jinja.exceptions import TemplateNotFound

    class SimpleLoader(BaseLoader):

        def __init__(self, path):
            self.path = path

        def get_source(self, environment, name, parent):
            filename = join(self.path, name)
            if not path.exists(filename):
                raise TemplateNotFound(name)
            f = codecs.open(filename, 'r', environment.template_charset)
            try:
                return f.read()
            finally:
                f.close()

The functions load and parse which are a requirement for a loader are added automatically by the BaseLoader. Instead of the normal BaseLoader you can use one of the other base loaders that already come with a proper get_source method for further modification. Those loaders however are new in Jinja 1.1.

### CachedLoaderMixin

Additionally to the BaseLoader there is a mixin class called CachedLoaderMixin that implements memory and disk caching of templates. Note that you have to give it a higher priority in the MRO than the BaseLoader which means that's the first base class when inheriting from it:

    import codecs
    from os.path import join, getmtime, exists
    from jinja.loaders import BaseLoaderCachedLoaderMixin
    from jinja.exceptions import TemplateNotFound

    class CachedLoader(CachedLoaderMixin, BaseLoader):

        def __init__(self, path):
            self.path = path
            CachedLoaderMixin.__init__(self,
                True,       # use memory caching
                40,         # for up to 40 templates
                '/tmp',     # additionally save the compiled templates in /tmp
                True,       # and reload cached templates automatically if changed
                'foo'       # optional salt used to keep templates with the same
                            # name in the same cache folder, but from different
                            # loaders. New in Jinja 1.1 and can be omitted.
            )

        def get_source(self, environment, name, parent):
            filename = join(self.path, name)
            if not path.exists(filename):
                raise TemplateNotFound(name)
            f = codecs.open(filename, 'r', environment.template_charset)
            try:
                return f.read()
            finally:
                f.close()

        def check_source_changed(self, environment, name):
            fn = join(self.path, name)
            if exists(fn):
                return getmtime(fn)
            return -1

You don't have to provide the check_source_changed method. If it doesn't exist the option auto_reload won't have an effect. Also note that the check_source_changed method must not raise an exception if the template does not exist but return `-1`. The return value `-1` is considered "always reload" whereas `0` means "do not reload". The default return value for not existing templates should be `-1`.

For the default base classes that come with Jinja 1.1 onwards there exist also concrete implementations that support caching. The implementation just mixes in the CachedLoaderMixin.

### MemcachedLoaderMixin

New in Jinja 1.1

The MemcachedLoaderMixin class adds support for memcached caching. There is only one builtin loader that mixes it in: The MemcachedFileSystemLoader. If you need other loaders with this mixin you can easily subclass one of the existing base loaders. Here an example for the FunctionLoader:

    from jinja.loaders import FunctionLoader, MemcachedLoaderMixin

    class MemcachedFunctionLoader(MemcachedLoaderMixin, FunctionLoader):

        def __init__(self, loader_func):
            BaseFunctionLoader.__init__(self, loader_func)
            MemcachedLoaderMixin.__init__(self,
                True,                   # use memcached
                60 * 60 * 24 * 7,       # 7 days expiration
                ['127.0.0.1:11211'],    # the memcached hosts
                'template/'             # string prefix for the cache keys
            )

This mixin requires the python-memcached library.

## How Mixin Classes Work

The idea of the cached loader mixins is that you override the load method of the other base class so that it's only called to get the data from the loader and put it into a cache and then bypass the original load.

This works because mixin classes, as well as the loaders are so called "new style classes" with a MRO (method resolution order). So it's possible to access the parent without actually knowing the name of it.

Here as small mixin class that stores everything after loading in a dict:

    class SimpleCacheMixin(object):

        def __init__(self):
            self.__cache = {}

        def load(self, environment, name, translator):
            if name in self.__cache:
                return self.__cache[name]
            tmpl = super(SimpleCacheMixin, self).load(environment, name,
                                                      translator)
            self.__cache[name] = tmpl
            return tmpl

You can then mix the class in like any other mixin class. Note that all non public attributes must be prefixed with two underscores to enable the name mangling. Otherwise the mixin class could break the internal structure of the loader.

The `super(SimpleCacheMixin, self)` call returns an object that looks up all the attributes you request in all the parent classes. The SimpleCacheMixin just has the object parent which makes it a new style class, but as soon as a loader is mixed in it will call the load method of the loader that is the other parent of the resulting class. Here a full example.

## Combining Everything

Here a full example with a custom cache mixin and a custom base loader:

    import codecs
    from os.path import join
    from jinja.loaders import BaseLoader
    from jinja.exceptions import TemplateNotFound

    class SimpleBaseLoader(BaseLoader):

        def __init__(self, path):
            self.path = path

        def get_source(self, environment, name, parent):
            filename = join(self.path, name)
            if not path.exists(filename):
                raise TemplateNotFound(name)
            f = codecs.open(filename, 'r', environment.template_charset)
            try:
                return f.read()
            finally:
                f.close()

    class SimpleCacheMixin(object):

        def __init__(self):
            self.__cache = {}

        def load(self, environment, name, translator):
            if name in self.__cache:
                return self.__cache[name]
            tmpl = super(SimpleCacheMixin, self).load(environment, name,
                                                      translator)
            self.__cache[name] = tmpl
            return tmpl

    class SimpleLoader(SimpleBaseLoader, SimpleCacheMixin):

        def __init__(self, path):
            SimpleBaseLoader.__init__(self, path)
            SimpleCacheMixin.__init__()

You can of course put all the functionallity into the SimpleLoader but then you cannot exchange parts of it without rewriting much code. In the example above replacing the SimpleCacheMixin with a MemcachedLoaderMixin is a matter of 20 seconds.