Metadata-Version: 1.1
Name: os-m3-engine
Version: 0.1.1
Summary: 3 stage multithread engine.
Home-page: https://github.com/cfhamlet/os-m3-engine
Author: Ozzy
Author-email: cfhamlet@gmail.com
License: MIT License
Description: # os-m3-engine
        
        [![Build Status](https://www.travis-ci.org/cfhamlet/os-m3-engine.svg?branch=master)](https://www.travis-ci.org/cfhamlet/os-m3-engine)
        [![codecov](https://codecov.io/gh/cfhamlet/os-m3-engine/branch/master/graph/badge.svg)](https://codecov.io/gh/cfhamlet/os-m3-engine)
        [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/os-m3-engine.svg)](https://pypi.python.org/pypi/os-m3-engine)
        [![PyPI](https://img.shields.io/pypi/v/os-m3-engine.svg)](https://pypi.python.org/pypi/os-m3-engine)
        
        Multi-thread engine for 3(or 2) stages job.
        
        
        
        Although multi-thread of python is buggy and slow, it is still necessary to write multi-thread program. Typically, producer-consumer is the most common model(so called 2-stage job), further more a transporter is needed between them(3-stage job). This library is used to simplify creating 3(or 2) stages program. 
        
        
        
        The 3 stages are:
        
        * Frontend: think as producer, usually used to create or receive data.
        * Transport: receive data from frontend stage, transform and send the transformed data to backend.
        * Backend: think as consumer, process the data received from transport.
        
        
        
        Something else need to know:
        
        * Each of the stage can be multi-thread.
        * Frontend is required, transport or backend can be omitted.
        
        
        
        
        # Install
        
        `pip install os-m3-engine`
        
        # API
        
        * Create default engine, a typical 3-stage job:
        
            - frontend: ``os_m3_engine.ootb.StdinFrontend``, read from stdin, send to transport stage
            - transport: ``os_m3_engine.ootb.LogTransport``, log data received from fronted, send to backend stage
            - backend: ``os_m3_engine.ootb.LogBackend``, log data received from transport
        
            ```
            from os_m3_engine.launcher import create
            
            engine = create()
            ```
        
        * Create engine with custom defined stage:
        
            ```
            from os_m3_engine.launcher import create
            
            engine = create(transport_cls='transport_class_path_or_class')
            ```
        
        * Create engine with custom engine config:
        
            ```
            from os_m3_engine.launcher import create
            
            config = WhateverOjbectYouWant
            config.thread_num = 10
            engine = create(transport_cls='your_transport_cls', engine_transport_config=config) 
            ```
        
        * Start the engine:
        
            ``start`` will block the current thread until all of the stage threads stopped
            
            ```
            engine.start()
            ```
        
        * The regular practice of stopping the engine
        
            ```
            from signal
            from os_m3_engine.launcher import create
            
            engine = create()
            
            def stop(signum, frame):
                engine.stop()
            
            signal.signal(signal.SIGINT, stop)
            
            engine.start()
            ```
            
        * Custom frontend class:
        
            - inherit from ``os_m3_engine.core.frontend.Frontend``
            - define ``produce`` method as generator
        
            ```
            from os_m3_engine.core.frontend import Frontend
            
            class CustomFrontend(Frontend):
                def produce(self):
                    yield 'Hello world!'
            ```
            
        * Custom transport class:
        
            - inherit from ``os_m3_engine.core.transport.Transport``
            - define ``transport`` method, the only parameter is the data received, the return value will be sent to backend
        
            ```
            from os_m3_engine.core.transport import Transport
            
            class CustomTransport(Transport):
                def transport(self, data):
                    return data
            ```
        
        * Custom backend class:
        
            - inherit from ``os_m3_engine.core.backend.Backend``
            - define ``process`` method, the only parameter is the data received, no need return
        
            ```
            from os_m3_engine.core.backend import Backend
            
            class CustomBackend(Backend):
                def process(self, data):
                    print(data)
            ```
        
        * Passing parameters
        
            - create engine with custom config object
            - use ``self.config`` to get the config in stage class
        
            ```
            from os_m3_engine.launcher import create
            
            config = WhateverOjbectYouWant
            engine = create(app_config=config)
            ```
        
        
        * Custom ``setup``, ``cleanup`` behavior
          
            - each stage class can define ``setup``, ``cleanup`` methods
            - these will be called at each thread start/stop
            
            
            ```
            from os_m3_engine.core.backend import Backend
            
            class CustomBackend(Backend):
            
                def setup(self):
                    print('Setup')
                    
                def cleanup(self):
                    print('Cleanup')
                
                def process(self, data):
                    print(data)
            ```
        
        
        
        # Unit Tests
        
        `$ tox`
        
        # License
        
        MIT licensed.
        
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
