Deploying Nagare applications in a production environment
=========================================================

Serving a Nagare application in a development setup is easy. You just have to launch
the following command:

.. code-block:: sh

    nagare-admin serve [options] <app_name>

It starts a standalone threaded server which stores the sessions data in memory.

This standalone server is very convenient when developing Nagare applications, especially when
using the ``--reload`` and ``--debug`` options that respectively restart the server when source
files are changed and show the exceptions stacktraces as Web pages instead of in the console
shell.

However, this standalone mode is not suitable in a production environment, because it is not
efficient nor fail-safe:

- Since it's a threaded program, it does not fully utilize the CPU cores due to the GIL (see `this
  discussion <http://stackoverflow.com/questions/4496680/python-threads-all-executing-on-a-single-core>`_).
  To solve this problem, we need to use multiprocessing.
- Static files are served by the standalone server and are not cached. They would be better served
  by a faster dedicated Web server such as Apache, Nginx, Lighttpd ...
- The server processes are not monitored, and they don't cope with failures very well. For example, in a
  production setup, it's important to detect zombie processes and restart them to prevent attacks or to limit
  the impact of unnoticed programmation errors in the application. Of course, you should also use logging
  to trace the origin of the problems in order to solve them.

So, in a production environment, the recommended way to serve Nagare applications
is through *FastCGI* associated to a frontend Web server such as Apache, Nginx or
Lighttpd.  We are now going to describe how to do that.


Serving an application through FastCGI
--------------------------------------

In a FastCGI setup, a Web server such as `Apache`_, `Nginx`_ or `Lighttpd`_ is needed. It will serve
the static contents of the application, such as images, CSS and javascript files, and pass the other requests
(i.e. those with a "dynamic" nature) to the Nagare application through the `FastCGI`_ protocol.

FastCGI is a variation of the CGI protocol, which defines an interface layer between a Web server and external
applications (such as shell or python scripts). The main difference between FastCGI and CGI is that FastCGI
doesn't create a new worker process at the start of each request but use a pool of processes that are created
when the FastCGI service starts, and uses sockets for exchanging informations (i.e. request and response data)
between the Web server and the external applications.

This setup is particularly efficient when serving Nagare applications because
the pool of processes fully utilize the CPU cores. Furthermore, Web servers are
far more efficient than the standalone threaded server when serving the static
contents of the application since they cache the data and they are mostly written
in C.


The publisher configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to serve a Nagare application through FastCGI, we must first tell the ``nagare-admin serve`` utility
that we don't want to use the standalone server but FastCGI instead. This is done by creating a publisher
configuration file (see :doc:`publisher_file`) and passing it to the ``nagare-admin serve`` command
with the ``-c`` option.

Here is a typical publisher configuration file to serve an application through FastCGI:

.. code-block:: ini

    [publisher]
    type = fastcgi
    host = 127.0.0.1
    port = 9000

    [sessions]
    type = memcache
    host = 127.0.0.1
    port = 11211

We use the ``fastcgi`` backend for the publisher (with the default settings) and the ``memcache`` backend for
the sessions management.

Then, the FastCGI server can be launched by running this command:

.. code-block:: sh

    nagare-admin serve -c publisher.conf <app_name>

By default, the FastCGI server forks 5 worker processes at startup to handle the incoming requests received on
port 9000. However, the number of worker processes can grow or shrink depending on the workload.

.. warning::

  When an application is served through FastCGI, you must use a ``memcache``
  backend for storing the sessions data. The default ``standalone`` session
  backend will not work with FastCGI because the session data is stored
  in memory, and since the processes spawned by the FastCGI server don't
  share memory (by definition), the sessions data stored in one process would
  not be available in another process.

  That's why a kind of distributed "shared memory" session backend should be
  used with the FastCGI publisher. So don't forget to launch your ``memcached``
  server.

As the sessions are shared through the memcached server, the same FastCGI
application can be launched on several machines thus easily creating a scalable
applicative cluster.


Configuring the Web server
~~~~~~~~~~~~~~~~~~~~~~~~~~

The next step is to configure the Web server so that it serves the static files and pass the other requests to the
FastCGI processes we just launched. Of course, the procedure depend on the Web server used, so we are going to
show you how to achieve that with `Apache`_, `Nginx`_ and `Lighttpd`_. However, it should be easy to use another Web
server supporting FastCGI if you want to.


Apache
++++++

On Apache, the FastCGI support is provided by the ``mod_fastcgi`` module which is not installed by default.
You should download and install it by yourself, as described `here <http://www.fastcgi.com/mod_fastcgi/INSTALL>`_.
Then, the ``mod_fastcgi`` module should be activated by including it in the main apache configuration file as shown
below:

.. code-block:: apache

    LoadModule fastcgi_module /path/to/mod_fastcgi.so


The rewrite rules serving the static files of your application can be generated with this command:

.. code-block:: sh

    nagare-admin create-rules --apache <application> > rewrite_rules.apache


You should obtain something like this:

.. code-block:: apache

    RewriteEngine On
    RewriteRule ^/static/nagare/(.*)$ /path/to/python/site-packages/nagare-0.5.0-py2.7.egg/static/$1 [L]
    RewriteRule ^/static/myapp/(.*)$ /path/to/python/site-packages/myapp-0.0.1-py2.7.egg/static/$1 [L]

Then, you must create a ``VirtualHost`` for your application and include the rewrite rules file
into the configuration, as shown below:

.. code-block:: apache

    # virtualhost configuration for http://www.myapp.com
    <VirtualHost *:80>
        ServerName www.myapp.com

        FastCGIExternalServer /path/to/python/site-packages/fcgi -host 127.0.0.1:9000

        Include /path/to/rewrite_rules.apache
        RewriteRule ^/(.*)$ /fcgi/$1 [QSA,L]
    </VirtualHost>

The ``FastCGIExternalServer`` directive instructs Apache to forward the requests directed to
``/path/to/python/site-packages/fcgi`` to the FastCGI server listening at ``127.0.0.1`` on port
``9000`` which has been launched externally, and the last ``RewriteRule`` directive redirects
the requests not handled by the other rewrite rules to ``/path/to/python/site-packages/fcgi``,
thus forwarding those requests to the FastCGI processes.

Finally, you should ``include`` the ``VirtualHost`` configuration file into the main
apache configuration file, or put it in the ``conf.d`` directory of your apache installation,
and restart Apache.


Nginx
+++++

Nginx has builtin support for FastCGI thanks to the `Nginx's HttpFastcgiModule`_, so no
manual installation is necessary.

The rewrite rules serving the static files can be generated with this command:

.. code-block:: sh

    nagare-admin create-rules --nginx <application> > rewrite_rules.nginx


You should obtain something like this:

.. code-block:: nginx

    location /static/nagare/ {
      alias /path/to/python/site-packages/nagare-0.5.0-py2.7.egg/static/;
    }

    location /static/myapp/ {
      alias /path/to/python/site-packages/myapp-0.0.1-py2.7.egg/static/;
    }

Then, you must create a server configuration, as shown below:

.. code-block:: nginx

    server {
        listen 80;
        server_name www.myapp.com;
        access_log /var/log/nginx/myapp.access.log;
        error_log /var/log/nginx/myapp.error.log;

        # serve the static files
        include /path/to/rewrite_rules.nginx;

        # serve the application
        location / {
            include /etc/nginx/fastcgi_params;
            # Nagare applications need a properly set PATH_INFO variable
            # either pass the request URI or use fastcgi_split_path_info to split the URI
            fastcgi_param PATH_INFO $fastcgi_script_name;
            fastcgi_pass 127.0.0.1:9000;
        }
    }

Note that, in addition to the default FastCGI parameters specified in ``/etc/nginx/fastcgi_params``,
we must also set the ``PATH_INFO`` parameter which is required by Nagare.

Finally, you must copy the configuration file in ``/etc/nginx/conf.d/``, or create a symbolic link
there pointing to your configuration file, and restart Nginx.


Lighttpd
++++++++

Lighttpd has also builtin support for FastCGI thanks to the `Lighttpd's mod_fastcgi`_ module.

The rewrite rules serving the static files can be generated with this command:

.. code-block:: sh

    nagare-admin create-rules --lighttpd <application> > rewrite_rules.lighttpd

However, it's easier the generate the rewrite rules in an ``include_shell`` directive.
An example of Lighttpd server configuration is show below:

.. code-block:: lighttpd

    server.modules = ( "mod_rewrite", "mod_fastcgi" )
    server.errorlog = "/tmp/lighttpd.log"
    server.indexfiles = ( "index.html" )

    #debug.log-request-handling = "enable"

    server.port = 8080
    server.bind = "0.0.0.0"
    #server.event-handler = "linux-sysepoll"

    fastcgi.server = (
                     "/fcgi" =>
                     (
                         (
                             "host" => "127.0.0.1",
                             "port" => 9000,
                             "check-local" => "disable",
                         )
                     )
                 )

    # Here, change the path to the ``nagare-admin`` command
    include_shell "path/to/nagare-admin create-rules --lighttpd"
    url.rewrite += (
        "^(.*)" => "/fcgi/$1"
    )

    # ---------------------------------------------------------------------------------

    mimetype.assign = (
      ".pdf"          =>      "application/pdf",
      ".sig"          =>      "application/pgp-signature",
      ".spl"          =>      "application/futuresplash",
      ".class"        =>      "application/octet-stream",
      ".ps"           =>      "application/postscript",
      ".torrent"      =>      "application/x-bittorrent",
      ".dvi"          =>      "application/x-dvi",
      ".gz"           =>      "application/x-gzip",
      ".pac"          =>      "application/x-ns-proxy-autoconfig",
      ".swf"          =>      "application/x-shockwave-flash",
      ".tar.gz"       =>      "application/x-tgz",
      ".tgz"          =>      "application/x-tgz",
      ".tar"          =>      "application/x-tar",
      ".zip"          =>      "application/zip",
      ".mp3"          =>      "audio/mpeg",
      ".m3u"          =>      "audio/x-mpegurl",
      ".wma"          =>      "audio/x-ms-wma",
      ".wax"          =>      "audio/x-ms-wax",
      ".ogg"          =>      "application/ogg",
      ".wav"          =>      "audio/x-wav",
      ".gif"          =>      "image/gif",
      ".jpg"          =>      "image/jpeg",
      ".jpeg"         =>      "image/jpeg",
      ".png"          =>      "image/png",
      ".xbm"          =>      "image/x-xbitmap",
      ".xpm"          =>      "image/x-xpixmap",
      ".xwd"          =>      "image/x-xwindowdump",
      ".css"          =>      "text/css",
      ".html"         =>      "text/html",
      ".htm"          =>      "text/html",
      ".js"           =>      "text/javascript",
      ".asc"          =>      "text/plain",
      ".c"            =>      "text/plain",
      ".conf"         =>      "text/plain",
      ".text"         =>      "text/plain",
      ".txt"          =>      "text/plain",
      ".dtd"          =>      "text/xml",
      ".xml"          =>      "text/xml",
      ".mpeg"         =>      "video/mpeg",
      ".mpg"          =>      "video/mpeg",
      ".mov"          =>      "video/quicktime",
      ".qt"           =>      "video/quicktime",
      ".avi"          =>      "video/x-msvideo",
      ".asf"          =>      "video/x-ms-asf",
      ".asx"          =>      "video/x-ms-asf",
      ".wmv"          =>      "video/x-ms-wmv",
      ".bz2"          =>      "application/x-bzip",
      ".tbz"          =>      "application/x-bzip-compressed-tar",
      ".tar.bz2"      =>      "application/x-bzip-compressed-tar"
     )

Then, place this config file in ``/etc/lighttpd/lighttpd.conf`` and restart Lighttpd.


Handling the FastCGI processes
------------------------------

Since the FastCGI processes are launched externally from the Web server, you can use any monitoring
tool to handle the processes, such as `Supervisor`_. Here is a typical configuration file that starts
a Nagare application as a daemon with Supervisor:

.. code-block:: ini

    [unix_http_server]
    file=/path/to/supervisord/supervisor.sock

    [supervisord]
    logfile=/path/to/supervisord/supervisord.log
    pidfile=/path/to/supervisord/supervisord.pid
    directory=/path/to/supervisord

    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

    [supervisorctl]
    serverurl=unix:///path/to/supervisord/supervisor.sock

    [program:myapp]
    command=/path/to/nagare-admin serve /path/to/application/conf/myapp.conf -c /path/to/application/conf/fastcgi.conf
    process_name=myapp
    autostart=true
    autorestart=true
    stdout_logfile=/path/to/application/logs/myapp.log
    redirect_stderr=true

Then, you can start the application by running this command:

.. code-block:: sh

    /path/to/supervisord -c /path/to/supervisord.conf

The ``supervisord`` process daemonizes itself and starts the FastCGI processes through
the ``nagare-admin serve`` command. Furthermore, they are restarted automatically when something goes
wrong thanks to the ``autorestart`` option.


.. _`FastCGI`: https://fastcgi-archives.github.io
.. _`Apache`: http://httpd.apache.org/
.. _`Nginx`: http://nginx.org/
.. _`Supervisor`: http://supervisord.org/
.. _`Lighttpd`: http://www.lighttpd.net/
.. _`Nginx's HttpFastcgiModule`: http://wiki.nginx.org/HttpFastcgiModule
.. _`Lighttpd's mod_fastcgi`: http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs:ModFastCGI

