Metadata-Version: 2.1
Name: kueue
Version: 0.1.2
Summary: Distributed Task Queue - backed by Kafka
Home-page: https://github.com/jarojasm95/kueue
License: MIT
Keywords: python,kafka,distributed,task-queue
Author: Jose Rojas
Author-email: jose.rojas95@mail.com
Requires-Python: >=3.8,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Dist: confluent-kafka (>=1.5.0,<2.0.0)
Requires-Dist: pydantic (>=1.7.3,<2.0.0)
Requires-Dist: wrapt (>=1.12.1,<2.0.0)
Project-URL: Repository, https://github.com/jarojasm95/kueue
Description-Content-Type: text/markdown

## Kueue
![Build](https://github.com/jarojasm95/kueue/workflows/Build/badge.svg)

Python Distributed Task Queue - backed by Kafka


## Motivation
Kueue was born out of the lack of a library that leverages [Kafka](https://kafka.apache.org/) for distributed task processing

## Features
- simple
- intuitive api
- extensible

## Code Example
```python
import time
from kueue import task, TaskExecutorConsumer, KueueConfig

KueueConfig(
    kafka_bootstrap='localhost:9092'
)

@task(topic="my-topic")
def sleepy_task(sleep: int):
    time.sleep(sleep)
    print("done sleeping", sleep)
    return sleep

sleepy_task.enqueue(args=(15,))

consumer = TaskExecutorConsumer(["my-topic"])
consumer.start()
# prints "done sleeping, 15"
```

## Installation
```
pip install kueue
```

## Development

Install [poetry](https://python-poetry.org/) and run `poetry install` at the root of the repository. This should create a virtual environment with all the necessary python dependencies.

## Tests
The test framework makes heavy use of `pytest` fixtures in order to spin up full integration environment consisting of a kubernetes cluster using [kind](https://kind.sigs.k8s.io/) and [pytest-kind](https://codeberg.org/hjacobs/pytest-kind) and kafka using [strimzi](https://strimzi.io/)

`pytest`

## License

MIT © [Jose Rojas](https://github.com/jarojasm95)

