.. _ecosystem:

Ecosystem
#########

This section gathers information about extending *Cliquet*, and third-party packages.

Packages
========

* :github:`mozilla-services/cliquet-fxa`:
  Add support of :term:`Firefox Accounts` OAuth2 authentication in *Cliquet*


.. note::

    If you build a package that you would like to see listed here, just
    get in touch with us!


Extending Cliquet
=================

Pluggable components
--------------------

:term:`Pluggable` components can be substituted from configuration files,
as long as the replacement follows the original component API.

.. code-block:: ini

    # myproject.ini
    cliquet.logging_renderer = cliquet_fluent.FluentRenderer

This is the simplest way to extend *Cliquet*, but will be limited to its
existing components (cache, storage, log renderer, ...).

In order to add extra features, including external packages is the way to go!


Include external packages
-------------------------

Appart from usual python «*import and use*», *Pyramid* can include external
packages, which can bring views, event listeners etc.

.. code-block:: python
    :emphasize-lines: 11

    import cliquet
    from pyramid.config import Configurator


    def main(global_config, **settings):
        config = Configurator(settings=settings)

        cliquet.initialize(config, '0.0.1')
        config.scan("myproject.views")

        config.include('cliquet_elasticsearch')

        return config.make_wsgi_app()


Alternatively, packages can also be included via configuration:

.. code-block:: ini

    # myproject.ini
    pyramid.includes = cliquet_elasticsearch
                       pyramid_debugtoolbar


There are `many available packages`_, and it is straightforward to build one.

.. _curated list: https://github.com/ITCase/awesome-pyramid


Include me
----------

In order to be included, a package must define an ``includeme(config)`` function.

For example, in :file:`cliquet_elasticsearch/init.py`:

.. code-block:: python

    def includeme(config):
        settings = config.get_settings()

        config.add_view(...)


Configuration
-------------

In order to ease the management of settings, *Cliquet* provides a helper that
reads values from :ref:`environment variables <configuration-environment>`
and uses default application values.

.. code-block:: python
    :emphasize-lines: 1,2,5-7,11,14,15

    import cliquet
    from pyramid.settings import asbool


    DEFAULT_SETTINGS = {
        'cliquet_elasticsearch.refresh_enabled': False
    }


    def includeme(config):
        cliquet.load_default_settings(config, DEFAULT_SETTINGS)
        settings = config.get_settings()

        refresh_enabled = settings['cliquet_elasticsearch.refresh_enabled']
        if asbool(refresh_enabled):
            ...

        config.add_view(...)


In this example, if the environment variable ``CLIQUET_ELASTICSEARCH_REFRESH_ENABLED``
is set to ``true``, the value present in configuration file is ignored.


Custom backend
==============

As a simple example, let's add add another kind of cache backend to *Cliquet*.

:file:`cliquet_riak/cache.py`:

.. code-block:: python

    from cliquet.cache import CacheBase
    from riak import RiakClient


    class Riak(CacheBase):
        def __init__(self, **kwargs):
            self._client = RiakClient(**kwargs)
            self._bucket = self._client.bucket('cache')

        def set(self, key, value, ttl=None):
            key = self._bucket.new(key, data=value)
            key.store()
            if ttl is not None:
                # ...

        def get(self, key):
            fetched = self._bucked.get(key)
            return fetched.data

        #
        # ...see cache documentation for a complete API description.
        #


    def load_from_config(config):
        settings = config.get_settings()
        uri = settings['cliquet.cache_url']
        uri = urlparse.urlparse(uri)

        return Riak(pb_port=uri.port or 8087)


Once its package installed and available in Python path, this new backend type
can be specified in application configuration:

.. code-block:: ini

    # myproject.ini
    cliquet.cache_backend = cliquet_riak.cache


Adding features
===============

Another use-case would be to add extra-features, like indexing for example.

* Initialize an indexer on startup;
* Add a ``/search/{collection}/`` end-point;
* Index records manipulated by resources.


Inclusion and startup in :file:`cliquet_indexing/__init__.py`:

.. code-block:: python

    DEFAULT_BACKEND = 'cliquet_indexing.elasticsearch'

    def includeme(config):
        settings = config.get_settings()
        backend = settings.get('cliquet.indexing_backend', DEFAULT_BACKEND)
        indexer = config.maybe_dotted(backend)

        # Store indexer instance in registry.
        config.registry.indexer = indexer.load_from_config(config)

        # Activate end-points.
        config.scan('cliquet_indexing.views')


End-point definitions in :file:`cliquet_indexing/views.py`:

.. code-block:: python

    from cornice import Service

    search = Service(name="search",
                     path='/search/{collection_id}/',
                     description="Search")

    @search.post()
    def get_search(request):
        collection_id = request.matchdict['collection_id']
        query = request.body

        # Access indexer from views using registry.
        indexer = request.registry.indexer
        results = indexer.search(collection_id, query)

        return results


Example indexer class in :file:`cliquet_indexing/elasticsearch.py`:

.. code-block:: python

    class Indexer(...):
        def __init__(self, hosts):
            self.client = elasticsearch.Elasticsearch(hosts)

        def search(self, collection_id, query, **kwargs):
            try:
                return self.client.search(index=collection_id,
                                          doc_type=collection_id,
                                          body=query,
                                          **kwargs)
            except ElasticsearchException as e:
                logger.error(e)
                raise

        def index_record(self, collection_id, record, id_field):
            record_id = record[id_field]
            try:
                index = self.client.index(index=collection_id,
                                          doc_type=collection_id,
                                          id=record_id,
                                          body=record,
                                          refresh=True)
                return index
            except ElasticsearchException as e:
                logger.error(e)
                raise


Indexed resource in :file:`cliquet_indexing/resource.py`:

.. code-block:: python

    class IndexedCollection(cliquet.resource.Collection):
        def create_record(self, record):
            r = super(IndexedCollection, self).create_record(self, record)

            self.indexer.index_record(self, record)

            return r

    class IndexedResource(cliquet.resource.BaseResource):
        def __init__(self, request):
            super(IndexedResource, self).__init__(request)
            self.collection.indexer = request.registry.indexer

.. note::

    In this example, ``IndexedResource`` must be used explicitly as a
    base resource class in applications.
    A nicer pattern would be to trigger *Pyramid* events in *Cliquet* and
    let packages like this one plug listeners. If you're interested,
    `we started to discuss it <https://github.com/mozilla-services/cliquet/issues/32>`_!


JavaScript client
=================

One of the main goal of *Cliquet* is to ease the development of REST
microservices, most likely to be used in a JavaScript environment.

A client could look like this:

.. code-block:: javascript

    var client = new cliquet.Client({
        server: 'https://api.server.com',
        store: localforage
    });

    var articles = client.resource('/articles');

    articles.create({title: "Hello world"})
      .then(function (result) {
        // success!
      });

    articles.get('id-1234')
      .then(function (record) {
        // Read from local if offline.
      });

    articles.filter({
        title: {'$eq': 'Hello'}
      })
      .then(function (results) {
        // List of records.
      });

    articles.sync()
      .then(function (result) {
        // Synchronize offline store with server.
      })
      .catch(function (err) {
        // Error happened.
        console.error(err);
      });
