Metadata-Version: 1.1
Name: shelter
Version: 2.1.2
Summary: Simple Python's Tornado wrapper which provides helpers for creationa new project, writing management commands, service processes, ...
Home-page: https://github.com/seznam/shelter
Author: Jan Seifert (Seznam.cz, a.s.)
Author-email: jan.seifert@firma.seznam.cz
License: BSD
Description: Shelter
        =======
        
        Shelter is a *Python's Tornado* wrapper which provides classes and helpers
        for creation new application skeleton, writing management commands (like a
        *Django*), service processes and request handlers. It was tested with
        *Python 2.7* and *Python 3.4* or higher and *Tornado 3.2* or higher.
        
        Instalation
        -----------
        
        ::
        
            cd shelter/
            python setup.py install
        
        After instalation **shelter-admin** command is available. For help type:
        
        ::
        
            shelter-admin -h
        
        The most important argument is ``-s/--settings``, which joins Shelter library
        and your application. Format is Python's module path, eg. `myapp.settings`.
        Second option how to handle `settings` module is ``SHELTER_SETTINGS_MODULE``
        environment variable. If both are handled, command line argument has higher
        priority than environment variable.
        
        ::
        
            shelter-admin -s myapp.settings
        
        or
        
        ::
        
            SHELTER_SETTINGS_MODULE=myapp.settings shelter-admin
        
        Usage
        ------
        
        ::
        
            shelter-admin startproject myproject
        
        Skeleton of the new application will be created in current working directory.
        Project name has the same rules as Python's module name. Entry point into new
        application is a script **manage.py**.
        
        ::
        
            cd myproject/
            ./manage.py -h
            ./manage.py devserver
        
        `settings.py` is included in the new skeleton. See comments in file how to
        define interfaces, management commands, service processes, ...
        
        List of the management commands which provides Shelter library:
        
        + **runserver** runs production HTTP, multi-process server. Number of the
          processes are detected according to ``INTERFACES`` setting in the
          ``settings`` module. Service processes are run in separated processes.
          Parent process checks child processes and when child process exits (due to
          crash or signal), it is run again. Crashes are counted, maximum amount of
          the crashes are 100, after it whole application will be exited. If child
          stops with exit code 0, crash counter is not incremented, so you can exit
          the worker deliberately and it will be run again.
        + **devserver** runs development HTTP server, which autoreloads application
          when source files are changes. Server is run only in one process.
        + **shell** runs interactive Python's shell. First it tries to run *IPython*,
          then standard *Python* shell.
        + **showconfig** shows effective configuration.
        + **startproject** will generate new apllication skeleton.
        
        Pay attention, service processes are run only in **runserver** command!
        
        Config class
        ------------
        
        Library provides base configuration class ``shelter.core.config.Config``
        which holds all configuration. Public attributes are **settings** which
        is ``settings`` module of the application and **args_parser** which is
        instance of the ``argparse.ArgumentParser`` from *Python's standard library*.
        
        You can override this class in the `settings` module::
        
            CONFIG_CLASS = 'myapp.core.config.AppConfig'
        
        Your own `AppConfig` class can contain additional *properties* with
        application's settings, e.g. database connection arguments. Way how the value
        is found is only on you - either only in **settings** or **args_parser** or
        in both. You can define additional arguments of the command line.
        
        ::
        
            import time
        
            from shelter.core.config import Config, argument
        
            class AppConfig(Config):
        
                arguments = (
                    argument(
                        '-k', '--secret-key',
                        dest='secret_key', action='store',
                        type=str, default='',
                        help='configuration file'
                    ),
                )
        
                Database = collections.namedtuple('Database', ['host', 'db'])
        
                def initialize(self):
                    # initialize() is called after instance is created. If you want
                    # add some instance attributes, use this method instead of
                    # override __init__().
                    self._server_started_time = time.time()
        
                def get_config_items(self):
                    # Override get_config_items() method if you want to add
                    # your options into showconfig management command.
                    base_items = super(AppConfig, self).get_config_items()
                    app_items = (
                        ('secret_key', self.secret_key),
                        ('database', self.database),
                    )
                    return base_items + app_items
        
                @property
                def secret_key(self):
                    # If command-line argument -k/--secret-key exists, it will
                    # override settings.SECRET_KEY value.
                    return self.args_parser.secret_key or self.settings.SECRET_KEY
        
                @property
                def database(self):
                    return self.Database(
                        db=self.settings.DATABASE_NAME,
                        host=self.settings.DATABASE_HOST,
                        passwd=getattr(self.settings, DATABASE_PASSWORD, '')
                    )
        
        Context class
        -------------
        
        In all handlers, management commands and service processes is available
        instance of the ``shelter.core.context.Context`` which holds resources for
        your appllication. Bundled class ``Context`` contains only one property
        **config** with ``Config`` instance (see previous chapter).
        
        You can define own class in ``settings`` module::
        
            CONTEXT_CLASS = 'myapp.core.context.Context'
        
        Overrided ``Context`` can contain additional *properties*, e.g. database
        connection pool.
        
        **It is necesary to initialize shared resources (sockets, open files, ...)
        lazy!** The reason is that subprocesses (Tornado HTTP workers, service
        processes) have to get uninitialized ``Context``, because forked resources
        can cause a lot of nights without dreams... **Also it is necessary to known
        that Context is shared among coroutines!** So you are responsible for
        locking shared resources (be careful, it is blocking operation) or use
        another mechanism, e.g. database connection pool.
        
        ``Context`` class contains two methods, ``initialize()`` and
        ``initialize_child()``.
        
        ``initialize()`` is called from constructor during instance is initialized.
        So it is the best place where you can initialize attributes which can be
        shared among processes.
        
        ``initialize_child()`` is called when service processes or Tornado workers
        are initialized. So it is the best place where you can safely initialize
        shared resources like a database connection. *process_type* argument contains
        type of the child – **shelter.core.constants.SERVICE_PROCESS** or
        **shelter.core.constants.TORNADO_WORKER**. *kwargs* contains additional data
        according to *process_type*:
        
        + for **SERVICE_PROCESS** contains *process* key which is instance of the
          service process.
        + for **TORNADO_WORKER** contains *process* key which is instance of the
          HTTP worker.
        
        ::
        
            class Context(shelter.core.context.Context):
        
                def initialize(self):
                    self._database = None
        
                def initialize_child(self, process_type, **kwargs):
                    # Initialize database in the subprocesses when child is created
                    self._init_database(max_connections=10)
        
                def _init_database(self, max_connections):
                    self._database = ConnectionPool(
                        self.config.database.host,
                        self.config.database.db,
                        max_connections=max_connections,
                        connect_on_init=True)
        
                @property
                def database(self):
                    # Lazy property if you need database connection in
                    # the main process (e.g. management command)
                    if self._database is None:
                        self._init_database(max_connections=1)
                    return self._database
        
        Hooks
        -----
        
        You can define several hooks in the ``settings`` module - when application
        is launched, on **SIGUSR1** and **SIGUSR2** signals and when instance of the
        Tornado application is created.
        
        ::
        
            APP_SETTINGS_HANDLER = 'myapp.core.app.get_app_settings'
            INIT_HANDLER = 'myapp.core.app.init_handler'
            SIGUSR1_HANDLER = 'myapp.core.app.sigusr1_handler'
            SIGUSR2_HANDLER = 'myapp.core.app.sigusr2_handler'
        
        ``INIT_HANDLER`` is allowed to contain multiple values.
        
        ::
        
            INIT_HANDLER = [
                'myapp.core.app.init_handler1', 'myapp.core.app.init_handler2']
        
        Handler is common *Python's* function which takes only one argument
        *context* with ``Context`` instance (see previous chapter).
        
        ::
        
            def init_handler(context):
                do_something(context.config)
        
        + **INIT_HANDLER** is called during the application starts, before workers
          or service processes are run.
        + **SIGUSR1_HANDLER** is called on **SIGUSR1** signal. When signal receives
          worker/child process, it is processed only in this process. When signal
          receives main/parent process, signal is propagated into all workers.
        + **SIGUSR2_HANDLER** is called on **SIGUSR2** signal. Signal is processed
          only in process which received signal. It is not propagated anywhere.
        + **APP_SETTINGS_HANDLER** is called on when instance of the Tornado
          application is created. Function have to return *dict*, which is passed as
          *\*\*settings* argument into ``tornado.web.Application`` constructor. Do not
          pass *debug*, *context* and *interface* keys.
        
        Service processes
        -----------------
        
        Service process are tasks which are repeatedly launched in adjusted interval,
        e.g. warms cache data before they expire. Library provides base class
        ``shelter.core.process.BaseProcess``. For new service process
        you must inherit ``BaseProcess``, adjust ``interval`` attribute and override
        ``loop()`` method.
        
        ::
        
            from shelter.core.processes import BaseProcess
        
            class WarmCache(BaseProcess)
        
                interval = 30.0
        
                def initialize(self):
                    self.db_conn = self.context.db.conn_pool
                    self.cache = self.context.cache
        
                def loop(self):
                    self.logger.info("Warn cached data")
                    with self.db_conn.get() as db:
                        self.cache.set('key', db.get_data(), timeout=60)
        
        + **interval** is a time in seconds. After this time ``loop()`` method is
          repeatedly called.
        
        Service process has to be registered in the ``settings`` module.
        
        ::
        
            SERVICE_PROCESSES = (
                ('myapp.processes.WarmCache', True, 15.0),
            )
        
        Each service process definition is list/tuple in format
        ``('path.to.ClassName', wait_unless_ready, timeout)``. If *wait_unless_ready*
        is ``True``, wait maximum *timeout* seconds unless process is successfully
        started, otherwise raise ``shelter.core.exceptions.ProcessError`` exception.
        
        Management commands
        -------------------
        
        Class ``shelter.core.commands.BaseCommand`` is an ancestor for user
        defined managemend commands, e.g. export/import database data. For new
        management command you must inherit ``BaseCommand`` and override ``command()``
        method and/or ``initialize()`` method.
        
        ::
        
            import sys
        
            from gettext import gettext as _
        
            from shelter.core.commands import BaseCommand, argument
        
            class Export(BaseCommand)
        
                name = 'export'
                help = 'export data from database'
                arguments = (
                    argument(
                        '-o', dest=output_file, type=str, default='-',
                        help=_('output filename')),
                )
        
                def initialize(self):
                    filename = self.conntext.config.args_parser.output_file
                    if filename == '-':
                        self.output_file = sys.stdout
                    else:
                        self.output_file = open(filename, 'wt')
        
                def command(self):
                    self.logger.info("Exporting data")
                    with self.context.db.get_connection_from_pool() as db:
                        data = db.get_data()
                    self.output_file.write(data)
                    self.output_file.flush()
        
        + **name** is a name of the management command. This name is used from command
          line, e.g. ``./manage.py export``.
        + **help** is a short description of the management command. This help is
          printed onto console when you type ``./manage.py command -h``.
        + **arguments** are arguments of the command line parser. ``argument()``
          function has the same meaning as ``ArgumentParser.add_argument()``
          from *Python's standard library*.
        + **settings_required** If it is ``False``, `settings` module will not be
          required for command. However, only internal ``shelter.core.config.Config``
          and ``shelter.core.context.Context`` will be available, not your own defined
          in settings. For example, internal **startprocest** command sets this flag
          to ``False``. **It is not public API, do not use this attribute unless you
          really know what you are doing**!
        
        Management command has to be registered in the ``settings`` module.
        
        ::
        
            MANAGEMENT_COMMANDS = (
                'myapp.commands.Export',
            )
        
        Interfaces
        ----------
        
        *Tornado's HTTP server* can be run in multiple instances. Interface are
        defined in ``settings`` module. Interfaces can be set as either TCP/IP sockets
        (``LISTEN`` directive) or unix sockets (``UNIX_SOCKET`` directive) or both.
        
        ::
        
            INTERFACES = {
                'default': {
                    # IP/hostname (not required) and port where the interface
                    # listens to requests
                    'LISTEN': ':8000',
        
                    # Path to desired unix socket
                    'UNIX_SOCKET': '/run/myapp.sock',
        
                    # Amount of the server processes if application is run
                    # using runserver command. Positive integer, 0 will
                    # detect amount of the CPUs
                    'PROCESSES': 0,
        
                    # Path in format 'path.to.module.variable_name' where
                    # urls patterns are defined
                    'URLS': 'myapp.urls.default_urls',
                },
            }
        
        URL path to HTTP handler routing
        --------------------------------
        
        It is the same as in *Python's Tornado* application.
        
        ::
        
            from tornado.web import URLSpec
        
            from myapp.handlers import HomepageHandler, AboutHandler
        
            urls_default = (
                URLSpec(r'/', HomepageHandler),
                URLSpec(r'/about/', AboutHandler),
            )
        
        Tuple/list **urls_default** is handled into relevant interface in the
        ``settings`` module, see previous chapter.
        
        HTTP handler is a subclass of the ``shelter.core.web.BaseRequestHandler``
        which enhances ``tornado.web.RequestHandler``. Provides additional instance
        attributes/properties **logger**, **context** and **interface**.
        
        + **logger** is an instance of the ``logging.Logger`` from *Python's standard
          library*. Logger name is derived from handlers's name, e.g
          ``myapp.handlers.HomepageHandler``.
        + **context** is an instance of the ``Context``, see *Context* paragraph.
        + **interface** is a namedtuple with informations about current interface.
          Named attributes are **name**, **host**, **port**, **processes** and
          **urls**.
        
        ::
        
            from shelter.core.web import BaseRequestHandler
        
            class DummyHandler(BaseRequestHandler):
        
                def get(self):
                    self.write(
                        "Interface '%s' works!\n" % self.interface.name)
                    self.set_header(
                        "Content-Type", 'text/plain; charset=UTF-8')
        
        Logging
        -------
        
        Standard *Python's logging* is used. ``Config.configure_logging()`` method
        is responsible for setting the logging. Default ``Config`` class reads
        logging's configuration from ``settings`` module::
        
            LOGGING = {
                'version': 1,
                'disable_existing_loggers': False,
                'handlers': {
                    'console': {
                        'class': 'logging.StreamHandler',
                        'level': 'INFO',
                        'formatter': 'default',
                    },
                },
                'root': {
                    'handlers': ['console'],
                    'level': 'INFO',
                },
            }
        
        Contrib
        -------
        
        shelter.contrib.config.iniconfig.IniConfig
        ``````````````````````````````````````````
        
        Descendant of the ``shelter.core.config.Config``, provides **INI** files
        configuration. Adds additional public attribute **config_parser** which is
        instance of the ``RawConfigParser`` from *Python's standard library*.
        Interfaces and application's name can be overrided in configuration file,
        *Python's logging* must be defined.
        
        Configuration file is specified either by ``SHELTER_CONFIG_FILENAME``
        environment variable or ``-f/--config-file`` command line argument. First,
        main configuration file is read. Then all configuration files from
        ``file.conf.d`` subdirectory are read in alphabetical order. E.g. if
        ``-f conf/myapp.conf`` is handled, first ``conf/myapp.conf`` file is read
        and then all ``conf/myapp.conf.d/*.conf`` files. Value in later
        configuration file overrides previous defined value.
        
        ::
        
            [application]
            name = MyApp
        
            [interface_http]
            Listen=:4444
            Processes=8
            Urls=tests.urls1.urls_http
        
            [formatters]
            keys=default
        
            [formatter_default]
            class=logging.Formatter
            format=%(asctime)s %(name)s %(levelname)s: %(message)s
        
            [handlers]
            keys=console
        
            [handler_console]
            class=logging.StreamHandler
            args=()
            level=NOTSET
        
            [loggers]
            keys=root
        
            [logger_root]
            level=INFO
            handlers=console
        
        License
        -------
        
        3-clause BSD
        
Platform: any
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
