# Qaviton IO  
![logo](https://www.qaviton.com/wp-content/uploads/logo-svg.svg)  
[![version](https://img.shields.io/pypi/v/qaviton_io.svg)](https://pypi.python.org/pypi)
[![license](https://img.shields.io/pypi/l/qaviton_io.svg)](https://pypi.python.org/pypi)
[![open issues](https://img.shields.io/github/issues/yehonadav/qaviton_io)](https://github/issues-raw/yehonadav/qaviton_io)
[![downloads](https://img.shields.io/pypi/dm/qaviton_io.svg)](https://pypi.python.org/pypi)
![code size](https://img.shields.io/github/languages/code-size/yehonadav/qaviton_io)
-------------------------  
  
Qaviton IO  
is a package with a simple API, making use of python's async & multiprocessing  
to enable fast execution of many asyncable operations.  
  
## Installation  
```sh  
pip install qaviton-io -U
```  
  
### Requirements  
- Python 3.6+  
  
## Features  
* async task manager  
* process task manager  
* task logger  
  
## Usage  
  
#### async manager:  
  
```python
from time import time
from typing import List
from requests import get, Response  # lets make use of requests to make async http calls
from qaviton_io import AsyncManager, Log

# we can save the responses here
rs: List[Response] = []

# let's create an async manager & log
manager = AsyncManager()
log = Log()

# first we make a simple function to make an http call
# we want to log the result
# and make sure that in case of an exception
# the manager won't stop
@log.task(exceptions=Exception)
def task():
    r = get("https://qaviton.com")
    r.raise_for_status()
    rs.append(r)

# let's run our task once and see how long it takes
t = time()
manager.run([task for _ in range(1)])
t = time() - t
print(f'took {round(t, 3)}s')

# now let's run our task 20 times and see how long it takes
t = time()
manager.run([task for _ in range(20)])
t = time() - t
print(f'took {round(t, 3)}s')

# let's view the results in the log report
log.report()
```  
  
#### process manager:  
  
```python
from time import time
from requests import get
from qaviton_io.types import Tasks
from qaviton_io import ProcessManager, Log
from traceback import format_exc

# make sure your tasks are defined at the module level,
# so they can be pickled by multiprocessing

# first we need to make a log registry
log = Log()


# now we make some tasks
# this is a nested task
# we don't want to handle any exceptions
# so in case of failure the parent will not proceed
@log.task()
def task(url):
    r = get(url)
    r.raise_for_status()


# this is the prent task
# we want to handle all exceptions
# so in case of failure the next task will execute
@log.task(exceptions=Exception)
def multi_task():
    for url in [
        "https://qaviton.com",
        "https://qaviton.co.il",  # make sure you enter a valid address
        "https://qaviton.com1",  # make sure you enter a valid address
    ]:
        task(url)


# let's create a function to execute tasks
def execute_tasks(tasks: Tasks, timeout):
    manager = ProcessManager()
    t = time()
    try:
        manager.run_until_complete(tasks, timeout=timeout)
        timed_out = None
    except TimeoutError:
        timed_out = format_exc()
    t = time() - t
    manager.log.report()
    print(f'took {round(t, 3)}s\n')
    manager.log.clear()
    return timed_out


# now all that's left is to run the tasks
if __name__ == "__main__":
    timeouts = [
        execute_tasks([multi_task for _ in range(1)], timeout=3),
        execute_tasks([multi_task for _ in range(20)], timeout=6),
        execute_tasks([multi_task for _ in range(80)], timeout=9),
    ]
    for timeout in timeouts:
        if timeout:
            print(timeout)

```  
  
## notes:  
  
* for good performance and easy usage  
you should probably stick with using the AsyncManager  
  
* The ProcessManager uses async operations as well as multi-processing.  
It distributes tasks across cpus, and those tasks are executed using the AsyncManager  
if you want maximum efficiency you should consider using the ProcessManager  
  
* The ProcessManager uses the multiprocessing module  
and should be treated with it's restrictions & limitations accordingly  
  
* The ProcessManager gets stuck easily,  
make sure to use timeouts when using it  
