Metadata-Version: 2.1
Name: hueyx
Version: 1.0.3.dev2
Summary: Django huey extension which supports multiple huey queues.
Home-page: https://github.com/APGSGA/hueyx
Author: Severin Bühler
Author-email: severin.buehler@apgsga.ch
License: MIT License
Description: # hueyx
        
        [![PyPI version](https://badge.fury.io/py/hueyx.svg)](https://badge.fury.io/py/hueyx)
        
        A django extension to run huey with multiple queues.
        Multiple queues allow tasks to not block each other and to scale independently.
        Only the redis storage is supported.
        
        ---
        ##### Important
        
        - If you use `huey 1.x` then install `hueyx 0.1.2`. Checkout the git tag [huey1.x](https://github.com/APGSGA/hueyx/tree/huey1.x).
        - If you use `huey 2.x` then install `hueyx  >= 1.0`.
        
        ---
        
        ### Usage
        
        Install it with
        ```bash
        pip install hueyx
        ```
        
        Add hueyx in your installed apps.
        
        ```python
        INSTALLED_APPS = [
            'django.contrib.admin',
            'django.contrib.auth',
            'django.contrib.contenttypes',
            'django.contrib.sessions',
            'django.contrib.messages',
            'django.contrib.staticfiles',
            'hueyx',
        ]
        ```
        
        
        
        ##### settings.py
        
        Compared to djhuey, hueyx allows several queues to be defined in the settings.py. 
        
        ```python
        HUEYX = {
            'queue_name1': {
                'connection': {
                    'host': 'localhost',
                    'port': 6379,
                    'db': 0,
                },
                'consumer': {
                    'workers': 1,
                    'worker_type': 'process',
                }
            },
            'queue_name2': {
                'connection': {
                    'connection_pool': ConnectionPool(host='localhost', port=6379, db=1)
                },
                'consumer': {
                    'multiple_scheduler_locking': True,
                    'prometheus_metrics': True,
                    'workers': 2,
                    'worker_type': 'thread',
                }
            },
            'priority_queue_name3': {
                'huey_class': 'huey.PriorityRedisHuey',
                'connection': {
                    'connection_pool': ConnectionPool(host='localhost', port=6379, db=1)
                },
                'consumer': {
                    'multiple_scheduler_locking': True,
                    'prometheus_metrics': True,
                    'workers': 2,
                    'worker_type': 'thread',
                }
            },
        }
        ```
        
        The settings are almost the same as in djhuey.
        Have a look at the [huey documentation](https://huey.readthedocs.io/en/latest/contrib.html#setting-things-up) 
        to see the exact parameter usage.
        
        Exceptions:
        - You can only configure redis as storage engine by configure `huey_class` to `huey.RedisHuey`, `huey.PriorityRedisHuey`, `huey.RedisExpireHuey` or `huey.PriorityRedisExpireHuey`.
        - The `name` and `backend_class` parameters are not supported.
        - The options `multiple_scheduler_locking` and `prometheus_metrics_enabled` have been added. See below.
        - The parameters `heartbeat_timeout` for `db_task` has been added. See below.
        
        ##### tasks.py
        
        ```python
        from hueyx.queues import hueyx
        
        """
        Define which queue you want to use.
        They are predefined in settings.py.
        """
        HUEY_Q1 = hueyx('queue_name1')
        HUEY_Q2 = hueyx('queue_name2')
        
        
        @HUEY_Q1.task()
        def my_task1():
            print('my_task1 called')
            
        @HUEY_Q1.db_task()
        def my_db_task1():
            print('my_db_task1 called')
            
        @HUEY_Q2.task()
        def my_task2():
            print('my_task2 called')
        
        @HUEY_Q2.periodic_task(crontab(minute='0', hour='3'))
        def my_periodic_task2():
            print('my_periodic_task2 called')
            return 1
            
        @HUEY_Q2.db_task(heartbeat_timeout=120)
        def my_heartbeat_task(heartbeat: Heartbeat):
            with heartbeat.long_running_operation():
                print('This operation can take a while -> don\'t check for heartbeats')
            print('Now we check for heartbeats -> call heartbeat() periodically')
            heartbeat()
        ```
        
        ##### Push task to queue
        ```python
        from example.tasks import my_task1, my_db_task1, my_task2
        
        
        my_task1()  # Task for queue_name1
        my_db_task1()  # Task for queue_name1
        my_task2()  # Task for queue_name2
        ```
        
        ##### Run consumer
        Consumers are started with the queue_name.
        ```bash
        ./manage.py run_hueyx queue_name1
        ```
        
        ##### Heartbeat tasks
        Heartbeat tasks are tasks with the parameter `heartbeat_timeout`. It defines the timeout in seconds. 
        They get a Heartbeat object which needs to be called in order to send a heartbeat to redis. 
        If no heartbeat occurs in set timeout the task is presumed to be dead and will automatically get restarted. 
        `heartbeat_timeout` needs to be at least 120 seconds. It does not work together with the parameter `include_task`.
        
        ### Additional settings
        
        ##### multiple_scheduler_locking
        `multiple_scheduler_locking` has been added to support multiple huey schedulers.
        If you run huey in a cloud environment, you will end up running multiple huey instances which each will
        schedule the periodic task.
        `multiple_scheduler_locking` prevents periodic tasks to be scheduled multiple times. It is false by default.
        
        
        ### Huey signals
        
        Optionally hueyx pushes all huey signals to the redis pubsub `hueyx.huey2.signaling` if enabled.
        ```python
        HUEYX_SIGNALS = {
            'enabled': True,
            'environment': 'your environment'
        }
        ```
        
        The format of the message is
        ```python
        {
            'environment': settings.HUEYX_SIGNALS['environment'],
            'queue': queue,
            'pid': pid,
            'signal': signal_name,
            'task': task_name
        }
        ``` 
        The environment parameter is a optional variable.
        
        
        ##### Prometheus
        The [huey-exporter](https://github.com/APGSGA/huey-exporter) project takes the signals und reports it to prometheus.
        
        
        ### Collaborators
        
        - [Update hueyx on PyPi](documentation/update_version.md)
        
Platform: UNKNOWN
Classifier: Environment :: Web Environment
Classifier: Framework :: Django
Classifier: Framework :: Django :: 2.2
Classifier: Framework :: Django :: 3.0
Classifier: Framework :: Django :: 3.1
Classifier: Framework :: Django :: 3.2
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Description-Content-Type: text/markdown
