
asyncoro: Asynchronous, Concurrent, Distributed Programming with Python
***********************************************************************

asyncoro is a Python framework for asynchronous, *concurrent*,
*network*, *distributed programming* and *distributed computing*,
using generator functions, asynchronous completions and message
passing. asyncoro can be used to create coroutines with generator
functions, similar to the way threads are created with functions with
Python's threading module. Programs developed with asyncoro have
**same logic and structure** as programs with threads, except for a
few syntactic changes - mostly using *yield* with asynchronous
completions that give control to asyncoro's scheduler, which
interleaves executions of generators, similar to the way an operating
system executes multiple processes.

Unlike threads, creating processes (coroutines) with asyncoro is very
efficient. Moreover, with asyncoro context switch occurs only when
coroutines use *yield* (typically with an asychronous call), so there
is no need for locking and there is no overhead of unnecessary context
switches.

asyncoro features include:

* No callbacks or event loops! No need to lock critical sections
  either,

* Efficient polling mechanisms epoll, kqueue, /dev/poll, Windows I/O
  Completion Ports (IOCP) for high performance and scalability,

* Asynchronous (non-blocking) *sockets* and *pipes*, for concurrent
  processing of I/O,

* SSL for security,

* Asynchronous timers, including non-blocking sleep,

* Asynchronous locking primitives similar to Python threading
  module,

* Message passing for (local and remote) coroutines to exchange
  messages one-to-one with Message Queue Pattern or through
  broadcasting channels with Publish-Subscribe Pattern,

* Location transparency with naming and locating (local and remote)
  resources,

* Monitoring and restarting of (local or remote) coroutines, for
  fault detection and fault-tolerance,

* Distributing computation components (code and data) to execute
  *Distributed Communicating Processes (discoro)*, for wide range of
  use cases, covering SIMD, MISD, MIMD system architectures at the
  process level, and web interface to monitor cluster/application
  status/performance; in-memory processing, data streaming, real-time
  (live) analytics and Cloud Computing are supported as well,

* Remote execution of coroutines for distributed programming with
  Remote Coroutine Invocation RCI (Remote Coroutine Invocation) and
  message passing,

* Hot-swapping of coroutine functions, for dynamic system
  reconfiguration,

* Thread pools with asynchronous task completions, for executing
  synchronous tasks, e.g., external library calls, such as reading
  standard input.

For reference purposes, asyncoro with Python 2.7 on Raspberry Pi Model
B+ (compliment by James Philips of pyeq2 project) running the program:

   import time
   import asyncoro
   def coro_proc(coro=None):
       yield coro.suspend()

   coros = [asyncoro.Coro(coro_proc) for i in xrange(10000)]
   time.sleep(5)  # wait for coroutines to run (and suspend)
   for coro in coros:
       coro.resume()

   for line in open('/proc/self/status'):
       if line.startswith('VmPeak'):
           print('VM Peak: %d MB' % (int(line.split()[1]) / 1024))
       elif line.startswith('VmHWM'):
           print('VM Max: %d MB' % (int(line.split()[1]) / 1024))
       elif line.startswith('VmStk'):
           print('VM Stack: %d MB' % (int(line.split()[1]) / 1024))

shows that running 10,000 processes takes about 23 MB of memory (with
about 10 MB taken by modules used in asyncoro). Memory and time scale
linearly: For 100,000 processes memory used is 116 MB, for 200,000
processes it is 218 MB, for 300,000 processes it is 315 MB.

asyncoro works with Python 2.7+ and Python 3.1+ and tested on Linux,
Mac OS X and Windows; it may work on other platforms too. asyncoro
works with PyPy as well.


Dependencies
============

asyncoro is implemented with standard modules in Python.

If psutil is available on nodes, node availability status (CPU, memory
and disk) is sent in status messages, and shown in web browser so
cluster/application performance can be monitored.

Under Windows efficient polling notifier I/O Completion Ports is
supported if pywin32 is available; otherwise, inefficient 'select'
notifier is used.


Download/Installation
=====================

* asyncoro package is available in Python Package Index (PyPI) so it
  can be installed with:

     python -m pip install asyncoro

* Docker Container describes how to build Docker image and run
  asyncoro modules in containers.

* asyncoro can also be downloaded from Sourceforge Files. 'py2'
  directory contains files to be used with Python 2.7+ and 'py3'
  contains files to be used with Python 3.1+. To use asyncoro without
  installing, rename 'py2' to 'asyncoro' (to use with Python 2.7+); it
  can then be used from parent directory of 'asyncoro'.


Examples
========

Examples illustrating some of the features of asyncoro are installed
in 'examples' directory under where asyncoro module is installed,
which can be obtained with the program:

   import os, asyncoro
   print(os.path.join(os.path.dirname(asyncoro.__file__), 'examples'))

See "README" file in that directory for brief description of each of
the files.


Citation
--------

If you use asyncoro in academic/research work, please cite asyncoro
as:

   Giridhar Pemmasani, "asyncoro: Asynchronous, Concurrent, Distributed Programming with Python",
     http://asyncoro.sourceforge.net, 2016.


Contents
========

* Introduction

* Asynchronous Concurrenct Programming (asyncoro)

  * Examples

  * AsynCoro scheduler

  * Coroutine

  * Locking Primitives

  * Channel

  * Message Passing

  * AsyncThreadPool

* Asynchronous Network Programming (asyncoro)

  * Examples

  * Asynchronous Socket

* Asynchronous Files and Pipes (asyncfile)

  * Examples

  * Asynchronous File

  * Asynchronous Pipe

* Distributed Programming (disasyncoro)

  * Examples

  * Location

  * AsynCoro

  * Distributed Coroutines

  * Distributed Channels

  * RCI (Remote Coroutine Invocation)

  * SSL (Security / Encryption)

* Distributed Communicating Processes (discoro)

  * Examples

  * Node / Servers

  * Scheduler

  * Computation

  * DiscoroStatus

  * DiscoroNodeAvailInfo

  * HTTP Server

  * Example

  * Client (Browser) Interface

  * RemoteCoroScheduler

  * Docker Container

  * Cloud Computing

* Tutorial / Examples

  * Asynchronous Concurrent Programming

  * Asynchronous Network Programming

  * Distributed Programming

  * Distributed Communicating Processes

* Contribute to / Recommend / Share asyncoro


Indices and tables
==================

* Index

* Module Index

* Search Page
