Metadata-Version: 2.1
Name: distify
Version: 0.3.3
Summary: Easy wrapper for parallelizing Python executions
Home-page: https://github.com/jordiae/distify
Author: Jordi Armengol-Estapé
Author-email: me@jordiae.com
License: MIT
Platform: UNKNOWN
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.0
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp (==3.7.4.post0)
Requires-Dist: aiohttp-cors (==0.7.0)
Requires-Dist: aioredis (==1.3.1)
Requires-Dist: antlr4-python3-runtime (==4.8)
Requires-Dist: async-timeout (==3.0.1)
Requires-Dist: attrs (==21.2.0)
Requires-Dist: blessings (==1.7)
Requires-Dist: boto3 (==1.15.6)
Requires-Dist: botocore (==1.18.18)
Requires-Dist: cachetools (==4.2.2)
Requires-Dist: certifi (==2021.5.30)
Requires-Dist: chardet (==4.0.0)
Requires-Dist: charset-normalizer (==2.0.5)
Requires-Dist: click (==8.0.1)
Requires-Dist: cloudpickle (==2.0.0)
Requires-Dist: colorful (==0.5.4)
Requires-Dist: filelock (==3.0.12)
Requires-Dist: gitdb (==4.0.7)
Requires-Dist: GitPython (==3.1.18)
Requires-Dist: google-api-core (==1.31.2)
Requires-Dist: google-auth (==1.35.0)
Requires-Dist: googleapis-common-protos (==1.53.0)
Requires-Dist: gpustat (==0.6.0)
Requires-Dist: grpcio (==1.40.0)
Requires-Dist: hiredis (==2.0.0)
Requires-Dist: hydra-core (==1.1.1)
Requires-Dist: hydra-ray-launcher (==0.1.2)
Requires-Dist: idna (==3.2)
Requires-Dist: jmespath (==0.10.0)
Requires-Dist: jsonschema (==3.2.0)
Requires-Dist: msgpack (==1.0.2)
Requires-Dist: multidict (==5.1.0)
Requires-Dist: multiprocessing-logging (==0.3.1)
Requires-Dist: numpy (==1.21.2)
Requires-Dist: nvidia-ml-py3 (==7.352.0)
Requires-Dist: omegaconf (==2.1.1)
Requires-Dist: opencensus (==0.7.13)
Requires-Dist: opencensus-context (==0.1.2)
Requires-Dist: packaging (==21.0)
Requires-Dist: pickle5 (==0.0.11)
Requires-Dist: prometheus-client (==0.11.0)
Requires-Dist: protobuf (==3.17.3)
Requires-Dist: psutil (==5.8.0)
Requires-Dist: py-spy (==0.3.9)
Requires-Dist: pyasn1 (==0.4.8)
Requires-Dist: pyasn1-modules (==0.2.8)
Requires-Dist: pyparsing (==2.4.7)
Requires-Dist: pyrsistent (==0.18.0)
Requires-Dist: python-dateutil (==2.8.2)
Requires-Dist: pytz (==2021.1)
Requires-Dist: PyYAML (==5.4.1)
Requires-Dist: ray (==1.6.0)
Requires-Dist: redis (==3.5.3)
Requires-Dist: requests (==2.26.0)
Requires-Dist: rsa (==4.7.2)
Requires-Dist: s3transfer (==0.3.7)
Requires-Dist: six (==1.16.0)
Requires-Dist: smmap (==4.0.0)
Requires-Dist: timeout-decorator (==0.5.0)
Requires-Dist: tqdm (==4.62.2)
Requires-Dist: typing-extensions (==3.10.0.2)
Requires-Dist: urllib3 (==1.25.11)
Requires-Dist: yarl (==1.6.3)

# distify

Wrapper around Ray for easy distributed processing in Python.

**Disclaimer**: I use it for some of my projects, but there is no guarantee that the code will work as expected or that future versions will be backward compatible.

## Install

    pip install distify --upgrade

## Features

- Multiple backends: Ray, Multiprocessing, Multithreading, Sequential.
- Logging.
- Progress bar.
- Can run in local or in multiple nodes.
- Individual timeout for map applications.
- Resume from checkpointing.
- Hydra integration.

## Quickstart/usage

Please refer to this example: https://github.com/jordiae/distify/tree/main/examples/basic

## Documentation

TODO

## LICENSE

MIT

## Author

Jordi Armengol Estapé


