Metadata-Version: 2.1
Name: themule
Version: 0.2.2
Summary: The Mule - run arbitrary python function on AWS Batch
Home-page: http://github.com/wlatanowicz/themule
Author: Wiktor Latanowicz
Author-email: wiktor@latanowicz.com
License: MIT
Classifier: Programming Language :: Python :: 3.9
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: click >=8.1.3
Requires-Dist: django-environ >=0.9.0
Provides-Extra: aws_batch
Requires-Dist: boto3 >=1.23.5 ; extra == 'aws_batch'
Provides-Extra: docker
Requires-Dist: docker >=6.1.3 ; extra == 'docker'

The Mule
===

[![tests](https://github.com/wlatanowicz/themule/actions/workflows/tests.yml/badge.svg)](https://github.com/wlatanowicz/themule/actions/workflows/tests.yml)

TheMule is an async task runner for python.

About
---

The purpose of this library is to run long running, heavy tasks in the background using cloud-native solutions. It's modular design allows you to use different backends for individual background tasks.


TheMule vs. Celery
---

TheMule is not intended to replace Celery. They play well together: Celery used to run small and fast tasks, while TheMule is used to run heavy and slow task.


Design principal     | TheMule                         | Celery
---------------------|---------------------------------|--------
Worker lifecycle     | One docker container per task   | Long living worker process
Application codebase | Main application and worker can run from different docker images, but the syntax is simpler when they share the codebase | Main application and worker share codebase
Task assignment      | Tasks can run on different backends; Tasks assignment is done in backend   | Tasks can be assigned to workers with routing
Task queueing        | Backend-specific      | External queueing service (Redis, RabbitMQ)


Available Backends
===

TheMule comes with pre-made backends and you can write your own.


AWS Batch
---

Installation: `pip install themule[aws_batch]`

Class path: `themule.backends.AwsBatchBackend`

Configuration:

Job parameter | Env variable | Required | Default | Description
---|---|---|--|--
aws_batch_queue_name | THEMULE_AWS_BATCH_QUEUE_NAME | Yes | - | The name of AWS Batch queue
aws_batch_job_definition | THEMULE_AWS_BATCH_JOB_DEFINITION | Yes | - | The name of AWS Batch job definition


Local Docker
---

Installation: `pip install themule[docker]`

Class path: `themule.backends.LocalDockerBackend`

Configuration:

Job parameter | Env variable | Required | Default | Description
---|---|---|--|--
docker_image | THEMULE_DOCKER_IMAGE | Yes | - | The name of the Docker image
docker_entrypoint | THEMULE_DOCKER_ENTRYPOINT | No | None | Override of the container's entrypoint
docker_pass_environment | THEMULE_DOCKER_PASS_ENVIRONMENT | No | True | Passes application's env to worker container if true
docker_environment | THEMULE_DOCKER_ENVIRONMENT | No | None | Additional container environment variables
docker_auto_remove | THEMULE_DOCKER_AUTO_REMOVE | No | True | Removes the container after worker exit if true
run_options | THEMULE_DOCKER_RUN_OPTIONS | No | - | Allows to set docker run options
