Optimization and Performance — Thorn 1.3.0 documentation

# Optimization and Performance

Table of Contents:

- Celery

## Celery

### Eventlet/Gevent

By far the best way to deploy Thorn for optimal web request performance is to use the Celery eventlet/gevent pools. Which one you choose does not matter much, but some will prefer one over the other.

To start a Celery worker with the eventlet/gevent pool set the `-P` option:

    $ celery -A proj worker -l info -P eventlet -c 1000

The `-c 1000` tells the worker to use up to one thousand green-threads for task execution.

Note that this will only start one OS process, so to take advantage of multiple CPUs or CPU-cores you need to start multiple processes.

This can be achived by using the `CELERYD_NODES` option to the Celery generic-init.d script, or by manually starting celery multi, for example if you have four CPU-cores you may want to start four worker instances, with a thousand green-threads each:

    $ celery multi start 4 -A proj -P eventlet -c 1000
    $ celery multi restart 4 -A proj -P eventlet -c 1000
    $ celery multi stop 4 -A proj -P eventlet -c 1000

#### Eventlet: Asynchronous DNS lookups

To use the Celery eventlet pool you should make sure to install the dnspython library, to enable asynchronous DNS lookups:

    $ pip install dnspython

#### Task retry settings

### Prefetch multiplier