Metadata-Version: 1.0
Name: gocept.runner
Version: 0.5.1
Summary: Create stand alone programs with full ZCA
Home-page: UNKNOWN
Author: gocept gmbh & co. kg
Author-email: mail@gocept.com
License: ZPL 2.1
Description: Creating runners
        ================
        
        The ``gocept.runner`` package allows it to *easily* create small, long running
        scripts which interact with the ZODB. The scripts have the full component
        architecture set up when they're run.
        
        .. contents::
        
        Runners are defined with the appmain decorator:
        
        >>> import logging
        >>> import gocept.runner
        >>> work_count = 0
        >>> @gocept.runner.appmain(ticks=0.1)
        ... def worker():
        ...     import zope.app.appsetup.product
        ...     log = logging.getLogger('test')
        ...     log.info("Working")
        ...     log.info(sorted(
        ...         zope.app.appsetup.product.getProductConfiguration('test').items()))
        ...     global work_count
        ...     work_count += 1
        ...     if work_count >= 3:
        ...         return gocept.runner.Exit
        
        
        The decorated worker takes two arguments now:
        
        1. The name of an object in the root which will be set as site or None for the
        root.
        2. The path to a configuration file (zope.conf)
        
        Create a simple zope.conf:
        
        >>> import os.path
        >>> import tempfile
        >>> zodb_path = tempfile.mkdtemp()
        >>> site_zcml = os.path.join(
        ...     os.path.dirname(__file__), 'ftesting.zcml')
        >>> fd, zope_conf = tempfile.mkstemp()
        >>> zope_conf_file = os.fdopen(fd, 'w')
        >>> zope_conf_file.write('''\
        ... site-definition %s
        ... <zodb>
        ...   <filestorage>
        ...     path %s/Data.fs
        ...   </filestorage>
        ... </zodb>
        ... <product-config test>
        ...     foo bar
        ...     test-principal zope.mgr
        ... </product-config>
        ... <accesslog>
        ...   <logfile>
        ...     path STDOUT
        ...   </logfile>
        ... </accesslog>
        ... <eventlog>
        ...   <logfile>
        ...     formatter zope.exceptions.log.Formatter
        ...     path STDOUT
        ...   </logfile>
        ... </eventlog>
        ... ''' % (site_zcml, zodb_path))
        >>> zope_conf_file.close()
        
        
        So call the worker:
        
        >>> worker(None, zope_conf)
        ------
        ... INFO test Working
        ------
        ... INFO test [('foo', 'bar'), ('test-principal', 'zope.mgr')]
        ------
        ... INFO test Working
        ------
        ... INFO test [('foo', 'bar'), ('test-principal', 'zope.mgr')]
        ------
        ... INFO test Working
        ------
        ... INFO test [('foo', 'bar'), ('test-principal', 'zope.mgr')]
        
        
        
        Signals
        +++++++
        
        The worker-procss can be terminated by SIGTERM and SIGHUP in a sane way. Write
        a script to a temporary file:
        
        >>> import sys
        >>> runner_path = os.path.abspath(
        ...     os.path.join(os.path.dirname(__file__), '..', '..'))
        >>> fd, script_name = tempfile.mkstemp(suffix='.py')
        >>> exchange_fd, exchange_file_name = tempfile.mkstemp()
        >>> script = os.fdopen(fd, 'w')
        >>> script.write("""\
        ... import sys
        ... sys.path[0:0] = %s
        ... sys.path.insert(0, '%s')
        ... import gocept.runner
        ...
        ... f = open('%s', 'w')
        ...
        ... @gocept.runner.appmain(ticks=0.1)
        ... def worker():
        ...     f.write("Working.\\n")
        ...     f.flush()
        ...
        ... worker(None, '%s')
        ... """ % (sys.path, runner_path, exchange_file_name, zope_conf))
        >>> script.close()
        
        
        Call the script and wait for it to produce some output:
        
        >>> import signal
        >>> import subprocess
        >>> import time
        >>> exchange = os.fdopen(exchange_fd, 'r+')
        >>> proc = subprocess.Popen(
        ...     [sys.executable, script_name],
        ...     stdout=subprocess.PIPE)
        >>> while not exchange.read():
        ...     time.sleep(0.1)
        ...     exchange.seek(0, 0)
        >>> exchange.seek(0, 0)
        >>> print exchange.read(),
        Working.
        
        Okay, now kill it:
        
        >>> os.kill(proc.pid, signal.SIGTERM)
        
        Wait for the process to really finish and get the output. The runner logs that
        it was terminated:
        
        >>> stdout, stderr = proc.communicate()
        >>> print stdout,
        ------
        ... INFO gocept.runner.runner Received signal 15, terminating.
        
        
        This also works with SIGHUP:
        
        >>> exchange.truncate(0)
        >>> proc = subprocess.Popen(
        ...     [sys.executable, script_name],
        ...     stdout=subprocess.PIPE)
        >>> while not exchange.read():
        ...     time.sleep(0.1)
        ...     exchange.seek(0, 0)
        >>> exchange.seek(0, 0)
        >>> print exchange.read(),
        Working.
        
        Okay, now kill it:
        
        >>> os.kill(proc.pid, signal.SIGHUP)
        >>> stdout, stderr = proc.communicate()
        >>> print stdout,
        ------
        ... INFO gocept.runner.runner Received signal 1, terminating.
        
        
        Clean up:
        
        >>> os.remove(script_name)
        >>> os.remove(exchange_file_name)
        
        
        Setting the principal
        +++++++++++++++++++++
        
        It is also prossible to create a main loop which runs in an interaction:
        
        >>> def get_principal():
        ...     return 'zope.mgr'
        
        >>> import zope.security.management
        >>> work_count = 0
        >>> def interaction_worker():
        ...     global work_count
        ...     work_count += 1
        ...     if work_count >= 3:
        ...         raise SystemExit(1)
        ...     log = logging.getLogger('test')
        ...     interaction = zope.security.management.getInteraction()
        ...     principal = interaction.participations[0].principal
        ...     log.info("Working as %s" % principal.id)
        >>> worker = gocept.runner.appmain(ticks=0.1, principal=get_principal)(
        ...     interaction_worker)
        
        Call the worker now:
        
        >>> worker(None, zope_conf)
        ------
        ... INFO test Working as zope.mgr
        ------
        ... INFO test Working as zope.mgr
        
        
        After the worker is run there is no interaction:
        
        >>> zope.security.management.queryInteraction() is None
        True
        
        It's quite common to read the principal from zope.conf. Therefore there is a
        helper which makes this task easier:
        
        >>> work_count = 0
        >>> worker = gocept.runner.appmain(
        ...     ticks=0.1,
        ...     principal=gocept.runner.from_config('test', 'test-principal'))(
        ...     interaction_worker)
        >>> worker(None, zope_conf)
        ------
        ... INFO test Working as zope.mgr
        ------
        ... INFO test Working as zope.mgr
        
        
        .. [#legacy-principal] The principal used to be set directly in the decorator.
        Since the principal is configuration this behaviour is deprecated.
        
        >>> work_count = 0
        >>> worker = gocept.runner.appmain(ticks=0.1, principal='zope.mgr')(
        ...     interaction_worker)
        
        
        Subsites
        ++++++++
        
        It is possible to directly work on sites inside the root. The site must already
        exist of course, otherwise there will be an error:
        
        >>> worker('a-site', zope_conf)
        Traceback (most recent call last):
        ...
        KeyError: 'a-site'
        
        
        Clean up:
        
        >>> import shutil
        >>> shutil.rmtree(zodb_path)
        >>> os.remove(zope_conf)
        
        
        Run once commands
        =================
        
        It is often needed to run a command once (or via cron) with the full component
        architecture. Usually ``zopectl run`` is used for this.
        
        >>> import logging
        >>> import gocept.runner
        >>> import zope.app.component.hooks
        >>> @gocept.runner.once()
        ... def worker():
        ...     log = logging.getLogger('test')
        ...     log.info("Working.")
        ...     site = zope.app.component.hooks.getSite()
        ...     if hasattr(site, 'store'):
        ...         log.info("Having attribute store.")
        ...     site.store = True
        
        
        Define a Zope environment:
        
        >>> import os.path
        >>> import tempfile
        >>> zodb_path = tempfile.mkdtemp()
        >>> site_zcml = os.path.join(
        ...     os.path.dirname(__file__), 'ftesting.zcml')
        >>> fd, zope_conf = tempfile.mkstemp()
        >>> zope_conf_file = os.fdopen(fd, 'w')
        >>> zope_conf_file.write('''\
        ... site-definition %s
        ... <zodb>
        ...   <filestorage>
        ...     path %s/Data.fs
        ...   </filestorage>
        ... </zodb>
        ... <product-config test>
        ...     foo bar
        ... </product-config>
        ... <accesslog>
        ...   <logfile>
        ...     path STDOUT
        ...   </logfile>
        ... </accesslog>
        ... <eventlog>
        ...   <logfile>
        ...     formatter zope.exceptions.log.Formatter
        ...     path STDOUT
        ...   </logfile>
        ... </eventlog>
        ... ''' % (site_zcml, zodb_path))
        >>> zope_conf_file.close()
        
        
        So call the worker for the first time. It will be terminated after one call:
        
        >>> worker(None, zope_conf)
        ------
        ... INFO test Working.
        
        
        Calling it a second time indicates that a property was changed in the first
        run:
        
        >>> worker(None, zope_conf)
        ------
        ... INFO test Working.
        ------
        ... INFO test Having attribute store.
        ...
        
        
        Runner details
        ==============
        
        Main loop
        +++++++++
        
        The main loop loops until it encounters a KeyboardInterrupt or a SystemExit
        exception and calls the worker once a second.
        
        Define a worker function which exits when it is called the 3rd time:
        
        >>> work_count = 0
        >>> def worker():
        ...     print "Working"
        ...     global work_count
        ...     work_count += 1
        ...     if work_count >= 3:
        ...         raise SystemExit(1)
        
        
        Call the main loop:
        
        >>> import gocept.runner.runner
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.1, worker)()
        Working
        Working
        Working
        >>> work_count
        3
        
        
        During the loop the site is set:
        
        >>> import zope.app.component.hooks
        >>> zope.app.component.hooks.getSite() is None
        True
        >>> def worker():
        ...     print zope.app.component.hooks.getSite()
        ...     raise SystemExit(1)
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.1, worker)()
        <zope.site.folder.Folder object at 0x...>
        
        
        
        After the loop, no site is set again:
        
        >>> zope.app.component.hooks.getSite() is None
        True
        
        
        When the worker passes without error a transaction is commited:
        
        >>> work_count = 0
        >>> def worker():
        ...     print "Working"
        ...     global work_count
        ...     work_count += 1
        ...     if work_count >= 2:
        ...         raise SystemExit(1)
        ...     site = zope.app.component.hooks.getSite()
        ...     site.worker_done = 1
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.1, worker)()
        Working
        Working
        
        We have set the attribute ``worker_done`` now:
        
        >>> getRootFolder().worker_done
        1
        
        
        When the worker produces an error, the transaction is aborted:
        
        >>> work_count = 0
        >>> def worker():
        ...     global work_count
        ...     work_count += 1
        ...     print "Working"
        ...     site = zope.app.component.hooks.getSite()
        ...     site.worker_done += 1
        ...     if work_count < 3:
        ...         raise ValueError('hurz')
        ...     raise SystemExit(1)
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.1, worker)()
        Working
        Working
        Working
        
        
        We still have the attribute ``worker_done`` set to 1:b
        
        >>> getRootFolder().worker_done
        1
        
        
        Controlling sleep time
        ++++++++++++++++++++++
        
        The worker function can control the sleep time[#log-handler]_
        
        >>> work_count = 0
        >>> def worker():
        ...     global work_count
        ...     work_count += 1
        ...     new_sleep = work_count * 0.1
        ...     if work_count == 3:
        ...         print "Will sleep default"
        ...         return None
        ...     if work_count > 3:
        ...         raise SystemExit(1)
        ...     print "Will sleep", new_sleep
        ...     return new_sleep
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.15, worker)()
        Will sleep 0.1
        Will sleep 0.2
        Will sleep default
        
        The real sleep values are in the log:
        
        >>> print log.getvalue(),
        new transaction
        commit
        Sleeping 0.1 seconds
        new transaction
        commit
        Sleeping 0.2 seconds
        new transaction
        commit
        Sleeping 0.15 seconds
        new transaction
        abort
        
        [#error-uses-default-sleep-time]_
        
        Restore old log handler:
        
        >>> logging.root.removeHandler(log_handler)
        >>> logging.root.setLevel(old_log_level)
        
        
        .. [#log-handler] Register a log handler
        
        >>> import logging
        >>> import StringIO
        >>> log = StringIO.StringIO()
        >>> log_handler = logging.StreamHandler(log)
        >>> logging.root.addHandler(log_handler)
        >>> old_log_level = logging.root.level
        >>> logging.root.setLevel(logging.DEBUG)
        
        
        .. [#error-uses-default-sleep-time] When an error occours within the worker,
        the default sleep time will be used:
        
        >>> log.seek(0)
        >>> log.truncate()
        >>> work_count = 0
        >>> def worker():
        ...     global work_count
        ...     work_count += 1
        ...     if work_count == 1:
        ...         new_sleep = 0.1
        ...     elif work_count == 2:
        ...         print "Failing"
        ...         raise Exception("Fail!")
        ...     elif work_count == 3:
        ...         print "Will sleep default"
        ...         return None
        ...     elif work_count > 3:
        ...         return gocept.runner.Exit
        ...     print "Will sleep", new_sleep
        ...     return new_sleep
        >>> gocept.runner.runner.MainLoop(getRootFolder(), 0.15, worker)()
        Will sleep 0.1
        Failing
        Will sleep default
        
        The real sleep values are in the log:
        
        >>> print log.getvalue(),
        new transaction
        commit
        Sleeping 0.1 seconds
        new transaction
        Fail!
        Traceback (most recent call last):
        ...
        Exception: Fail!
        abort
        Sleeping 0.15 seconds
        new transaction
        commit
        Sleeping 0.15 seconds
        new transaction
        commit
        
        
        
        Changes
        =======
        
        0.5.1 (2009-10-13)
        ++++++++++++++++
        
        - Declared dependencies correctly.
        
        
        0.5 (2009-09-21)
        ++++++++++++++++
        
        - Does no longer use ``zope.app.twisted`` but ``zope.app.server`` instead.
        
        
        0.4 (2009-09-03)
        ++++++++++++++++
        
        - The principal set by appmain/once can be computed by a function now.
        
        0.3.2 (2009-05-21)
        ++++++++++++++++++
        
        - Fixed handling of subsites in appmain.
        
        0.3.1 (2009-05-21)
        ++++++++++++++++++
        
        - Declared namespace package.
        
        0.3 (2009-04-15)
        ++++++++++++++++
        
        - When a worker fails the default sleep time (instead of the last one) will be
        used.
        
        0.2 (2009-04-09)
        ++++++++++++++++
        
        - Added a clean way to exit the runner (by returning gocept.runner.Exit).
        
        0.1 (2009-04-07)
        ++++++++++++++++
        
        - first public release
        
Keywords: zope3 mainloop
Platform: UNKNOWN
Classifier: Topic :: Software Development
Classifier: Framework :: Zope3
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
