{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.title-slide {    width: 100%;    display: flex;    flex-direction: row;            /* default value; can be omitted */    flex-wrap: nowrap;              /* default value; can be omitted */    justify-content: space-between;}</style><div class=\"title-slide\">\n",
    "<span style=\"float:left;\">Licence CC BY-NC-ND</span>\n",
    "<span>Thierry Parmentelat - Inria</span>\n",
    "<span><img src=\"media/inria-25.png\" style=\"display:inline\" /></span>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# README"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A simplistic orchestration engine\n",
    "\n",
    "The main and single purpose of this library is to allow for the static description of scenarii involving `asyncio`-compliant jobs, that have dependencies in the sense that a given job cannot start until its requirements have not completed.\n",
    "\n",
    "So in a nutshell you would:\n",
    "\n",
    "* define a set of `Job` objects, \n",
    "* together with their `requires` relationship; that is to say, for each of them, which other jobs need to have completed before this one can be triggered,\n",
    "* and run this logic through an `Scheduler` object, that will orchestrate the whole scenario.\n",
    "\n",
    "Further features allow to\n",
    "\n",
    "* define a job as running `forever`, in which case the scheduler of course won't wait for it, but instead will terminate it when all other jobs are done;\n",
    "* define a job as `critical`; a critical job that raises an exception causes the orchestration to terminate abruptly;\n",
    "* define a global `timeout` for the whole scheduler;\n",
    "* define a *window* in terms of a maximal number of simultaneous jobs that are allowed to run.\n",
    "\n",
    "Finally, knowing that a `Scheduler` instance is also a job, nested schedulers can be very easily created, allowing for re-usable pieces to be returned by regular python functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A job object can be created:\n",
    "\n",
    "* either as a `Job` instance from a regular asyncio coroutine\n",
    "* or by specializing the `AbstractJob` class, and defining its `co_run()` method\n",
    "\n",
    "As a convenience, the `Sequence` class is mostly a helper class that can free you from manually managing the `requires` deps in long strings of jobs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Full documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This document, along with API reference doc, and changelog, is available at <http://asynciojobs.readthedocs.io>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Contact author : *thierry dot parmentelat at inria dot fr*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`asynciojobs` requires `asyncio` and python-3.5 or more recent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "major, minor = sys.version_info[:2]\n",
    "if (major, minor) < (3, 5):\n",
    "    print(\"asynciojobs won't work in this environment\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`asynciojobs` requires python-3.5, and can be installed from the pypi repository:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "pip3 install asynciojobs\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's consider a simple coroutine for the sake of illustration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "# just print a message when entering and exiting, and sleep in the middle\n",
    "async def in_out(timeout):\n",
    "    print(\"-> in_out({})\".format(timeout))\n",
    "    await asyncio.sleep(timeout)\n",
    "    print(\"<- in_out({})\".format(timeout))\n",
    "    # return something easy to recognize: the number of milliseconds\n",
    "    return 1000 * timeout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example A : running in parallel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running a series of coroutines in parallel - a la `gather` - can be done like this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from asynciojobs import Job, Scheduler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a1, a2, a3 = Job(in_out(0.1)), Job(in_out(0.2)), Job(in_out(0.25)),"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we're saying here is that we have three jobs, that have no relationships between them. \n",
    "\n",
    "So when we run them, we would start all 3 coroutines at once, and return once they are all done:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sa = Scheduler(a1, a2, a3)\n",
    "sa.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Retrieving individual results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see right away how to retrieve the results of the various jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a1.result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example B : adding requirements (dependencies)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can add *requirements* dependencies between jobs, like in the following example. Here we want to run:\n",
    " \n",
    "* Job 1 followed by job 2\n",
    "* all this in parallel with job 3\n",
    "\n",
    "We take this chance to show that jobs can be tagged with a label, which can be convenient sometimes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "b1, b2, b3 = (Job(in_out(0.1), label=\"b1\"),\n",
    "              Job(in_out(0.2), label=\"b2\"),\n",
    "              Job(in_out(0.25)))\n",
    "\n",
    "b2.requires(b1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now `b2` needs `b1` to be finished before it can start. And so only the 2 first coroutines get started at the beginning, and only once b1 has finished does b2 start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# with this setup we are certain that b3 ends in the middle of b2\n",
    "sb = Scheduler(b1, b2, b3)\n",
    "sb.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example B' : exact same using a `Sequence`\n",
    "\n",
    "The code above in example B is exactly identical to this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from asynciojobs import Sequence\n",
    "\n",
    "sb2 = Scheduler(\n",
    "    Sequence(Job(in_out(0.1), label=\"bp1\"),\n",
    "             Job(in_out(0.2), label=\"bp2\")),\n",
    "    Job(in_out(0.25)))\n",
    "\n",
    "sb2.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Return value for `Scheduler.run()` "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that because `sb.run()` had returned `True`, we could have inferred that all jobs have completed. As a matter of fact, `run()` returns `True` **if and only if**:\n",
    "\n",
    "* all jobs have completed during the allocated timeout\n",
    "* no critical job has raised an exception"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspecting scheduler and results - `Scheduler.list()`\n",
    "\n",
    "Before we see more examples, let's see more ways to get information about what happened once `run` finishes.\n",
    "For example to check that job `b1` has completed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(b1.is_done())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check that job `b3` has not raised an exception:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(b3.raised_exception())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see an overview of a scheduler, just use the `list()` method that will give you an overview - whether the scheduler has run or not, by the way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sb.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a complete list of the symbols used, with their meaning \n",
    "\n",
    "* `⚐` : idle (read: requirements are not fulfilled)\n",
    "* `⚑` : scheduled (read: waiting for a slot in the jobs window)\n",
    "* `↺` : running\n",
    "* `☓` : complete \n",
    "* `★` : raised an exception\n",
    "* `☉` : went through fine (no exception raised)\n",
    "* `⚠` : defined as critical\n",
    "* `∞` : defined as forever  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And and here's an example of output for `list()` with all possible combinations of jobs:\n",
    "\n",
    "```\n",
    "01 ⚠   ⚐ ∞ <J `forever=True crit.=True status=idle boom=False`>\n",
    "02 ⚠   ⚐   <J `forever=False crit.=True status=idle boom=False`> - requires {01}\n",
    "03     ⚐ ∞ <J `forever=True crit.=False status=idle boom=False`> - requires {02}\n",
    "04     ⚐   <J `forever=False crit.=False status=idle boom=False`> - requires {03}\n",
    "05 ⚠   ⚑ ∞ <J `forever=True crit.=True status=scheduled boom=False`> - requires {04}\n",
    "06 ⚠   ⚑   <J `forever=False crit.=True status=scheduled boom=False`> - requires {05}\n",
    "07     ⚑ ∞ <J `forever=True crit.=False status=scheduled boom=False`> - requires {06}\n",
    "08     ⚑   <J `forever=False crit.=False status=scheduled boom=False`> - requires {07}\n",
    "09 ⚠ ☉ ↺ ∞ <J `forever=True crit.=True status=running boom=False`> - requires {08}\n",
    "10 ⚠ ☉ ↺   <J `forever=False crit.=True status=running boom=False`> - requires {09}\n",
    "11   ☉ ↺ ∞ <J `forever=True crit.=False status=running boom=False`> - requires {10}\n",
    "12   ☉ ↺   <J `forever=False crit.=False status=running boom=False`> - requires {11}\n",
    "13 ⚠ ★ ☓ ∞ <J `forever=True crit.=True status=done boom=True`>!! CRIT. EXC. => bool:True!! - requires {12}\n",
    "14 ⚠ ★ ☓   <J `forever=False crit.=True status=done boom=True`>!! CRIT. EXC. => bool:True!! - requires {13}\n",
    "15   ★ ☓ ∞ <J `forever=True crit.=False status=done boom=True`>!! exception => bool:True!! - requires {14}\n",
    "16   ★ ☓   <J `forever=False crit.=False status=done boom=True`>!! exception => bool:True!! - requires {15}\n",
    "17 ⚠ ☉ ☓ ∞ <J `forever=True crit.=True status=done boom=False`>[[ -> 0]] - requires {16}\n",
    "18 ⚠ ☉ ☓   <J `forever=False crit.=True status=done boom=False`>[[ -> 0]] - requires {17}\n",
    "19   ☉ ☓ ∞ <J `forever=True crit.=False status=done boom=False`>[[ -> 0]] - requires {18}\n",
    "20   ☉ ☓   <J `forever=False crit.=False status=done boom=False`>[[ -> 0]] - requires {19}\n",
    "```\n",
    "\n",
    "Note that if your locale/terminal cannot output these, the code will tentatively resort to pure ASCII output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example C : infinite loops, or coroutines that don't return"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes it is useful to deal with a endless loop; for example if we want to separate completely actions and printing, we can use an `asyncio.Queue` to implement a simple message bus as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "message_bus = asyncio.Queue()\n",
    "\n",
    "async def monitor_loop(bus):\n",
    "    while True:\n",
    "        message = await bus.get()\n",
    "        print(\"BUS: {}\".format(message))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need a modified version of the previous coroutine, that interacts with this message bus instead of printing anything itself&nbsp;:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def in_out_bus(timeout, bus):\n",
    "    await bus.put(\"-> in_out({})\".format(timeout))\n",
    "    await asyncio.sleep(timeout)\n",
    "    await bus.put(\"<- in_out({})\".format(timeout))\n",
    "    # return something easy to recognize\n",
    "    return 10 * timeout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can replay the prevous scenario, adding the monitoring loop as a separate job.\n",
    "\n",
    "However, we need to declare this extra job with `forever=True`, so that the scheduler knows it **does not have to wait** for the monitoring loop, as we know in advance that it will never return."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "c1, c2, c3, c4 = (Job(in_out_bus(0.2, message_bus), label=\"c1\"),\n",
    "                  Job(in_out_bus(0.4, message_bus), label=\"c2\"), \n",
    "                  Job(in_out_bus(0.3, message_bus), label=\"c3\"),\n",
    "                  Job(monitor_loop(message_bus), forever=True, label=\"monitor\"))\n",
    "\n",
    "c3.requires(c1)\n",
    "\n",
    "sc = Scheduler(c1, c2, c3, c4)\n",
    "sc.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `run()` always terminates as soon as all the non-`forever` jobs are complete. The `forever` jobs, on the other hand, get cancelled, so of course no return value is available at the end of the scenario&nbsp;:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sc.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example D : specifying a global timeout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `Scheduler` object has a `timeout` attribute, that can be set to a duration (in seconds). When provided, `run()` will ensure its global duration does not exceed this value, and will return `False` if the timeout triggers.\n",
    "\n",
    "Of course this can be used with any number of jobs and dependencies, but for the sake of simplicity let us see this in action with just one job that loops forever:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def forever():\n",
    "    for i in range(100000):\n",
    "        print(\"{}: forever {}\".format(time.strftime(\"%H:%M:%S\"), i))\n",
    "        await asyncio.sleep(.1)\n",
    "        \n",
    "j = Job(forever(), forever=True)\n",
    "sd = Scheduler(j, timeout=0.25)\n",
    "sd.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see the result of `run()` in this case is `False`, since not all jobs have completed. Apart from that the jobs is now in this state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling exceptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A job instance can be **critical** or not; what this means is as follows\n",
    "\n",
    " * if a critical job raises an exception, the whole scheduler aborts immediately and returns False\n",
    " * if a non-critical job raises an exception, the whole scheduler proceeds regardless\n",
    " \n",
    "In both cases the exception can be retrieved in the corresponding Job object with `raised_exception()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, the **critical** property can be set either at the `Job` or at the `Scheduler` level. Of course the former takes precedence if set. The default for an scheduler object is `critical=False`. Let us see this below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example E : non critical jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def boom(n):\n",
    "    await asyncio.sleep(n)\n",
    "    raise Exception(\"boom after {}s\".format(n))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# by default everything is non critical\n",
    "e1 = Job(in_out(0.2))\n",
    "e2 = Job(boom(0.2), label=\"boom\")\n",
    "e3 = Job(in_out(0.3))\n",
    "e2.requires(e1)\n",
    "e3.requires(e2)\n",
    "\n",
    "se = Scheduler(e1, e2, e3)\n",
    "print(\"orch:\", se.run())\n",
    "se.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example F : critical jobs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Making the *boom* job critical would instead cause the scheduler to bail out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f1 = Job(in_out(0.2))\n",
    "f2 = Job(boom(0.2), label=\"boom\", critical=True)\n",
    "f3 = Job(in_out(0.3))\n",
    "\n",
    "sf = Scheduler(Sequence(f1, f2, f3))\n",
    "print(\"run:\", sf.run())\n",
    "sf.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Limiting the number of simultaneous jobs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `Scheduler` has a `jobs_window` attribute that allows to specify a maximum number of jobs running simultaneously. \n",
    "\n",
    "When `jobs_windows` is not specified or `0`, it means no limit is imposed on the running jobs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's define a simple coroutine\n",
    "async def aprint(message, delay=0.5):\n",
    "     print(message)\n",
    "     await asyncio.sleep(delay)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let us now add 8 jobs that take 0.5 second each\n",
    "s = Scheduler(jobs_window=4)\n",
    "\n",
    "for i in range(1, 9):\n",
    "    s.add(Job(aprint(\"{}-th job\".format(i), 0.5)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# so running them with a window of 4 means approx. 1 second\n",
    "import time\n",
    "beg = time.time()\n",
    "s.run()\n",
    "end = time.time()\n",
    "\n",
    "# expect around 1 second\n",
    "print(\"total duration = {}s\".format(end-beg))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Customizing jobs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Customizing the `Job` class\n",
    "\n",
    "`Job` actually is a specializtion of `AbstractJob`, and the specification is that the `co_run()` method should denote a coroutine itself, as that is what is triggered by `Scheduler` when running said job."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `AbstractJob.co_shutdown()`\n",
    "\n",
    "Before returning, `run()` sends the `co_shutdown()` method on all jobs. The default behaviour - in the `Job` class - is to do nothing, but this can be redefined when relevant. Typically, an implementation of an `SshJob` will allow for a given SSH connection to be shared amongs several `SshJob` instances, and so `co_shutdown()` may be used to  close the underlying SSH connections at the end of the scenario."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The `apssh`  library and the ` SshJob` class\n",
    "\n",
    "You can easily define your own `Job` class by specializing `job.AbstractJob`. As an example, which was the primary target when developping `asynciojobs`, you can find [in the `apssh` library](https://github.com/parmentelat/apssh) a `SshJob` class, with which you can easily orchestrate scenarios involving several hosts that you interact with using ssh."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other useful features on the `Scheduler` class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Inspect / troubleshoot : `Scheduler.debrief()`\n",
    "\n",
    "`Scheduler.debrief()` is designed for schedulers that have run and returned `False`, it does output the same listing as `list()` but with additional statistics on the number of jobs, and, most importantly, on the stacks of jobs that have raised an exception."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup : `Scheduler.sanitize()`\n",
    "\n",
    "In some cases like esp. test scenarios, it can be helpful to add requirements to jobs that are not in the scheduler. The `sanitize` method removes such extra requirements, and unless you are certain it is not your case, it might be a good idea to call it explcitly before an orchestration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Early checks : `Scheduler.rain_check()`\n",
    "\n",
    "`rain_check` will check for cycles in the requirements graph. It returns a boolean. It's a good idea to call it before running an orchestration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Need a coroutine instead ? : `Scheduler.co_run()` \n",
    "\n",
    "`run()` is a regular `def` function (i.e. not an `async def`), but in fact just a wrapper around the native coroutine called `co_run()`.\n",
    "\n",
    "    def run(self, loop=None, *args, **kwds):\n",
    "        if loop is None:\n",
    "            loop = asyncio.get_event_loop()\n",
    "        return loop.run_until_complete(self.co_run(loop=loop, *args, **kwds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization - in a notebook : `Scheduler.graph()`\n",
    "\n",
    "If you have the `graphviz` package installed, you can inspect a scheduler instance in a Jupyter notebook by using the `graph()` method, that returns a `graphviz.Digraph` instance; this way the scheduler graph can be displayed interactively in a notebook - see also <http://graphviz.readthedocs.io/en/stable/manual.html#jupyter-notebooks>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a simple example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# and a simple scheduler with an initialization and 2 concurrent tasks\n",
    "s = Scheduler()\n",
    "j1 = Job(aprint(\"j1\"), label=\"init\", scheduler=s)\n",
    "j2 = Job(aprint(\"j2\"), label=\"critical job\", critical=True, scheduler=s, required=j1)\n",
    "j3 = Job(aprint(\"j3\"), label=\"forever job\", forever=True, scheduler=s, required=j1)\n",
    "s.graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In a regular notebook, that is all you need to do to see the scheduler's graph.\n",
    "In the case of this README though, once rendered on `readthedocs.io` the graph has got lost in translation, so please read on to see that graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization - the long way : `Scheduler.export_as_dotfile()`\n",
    "\n",
    "If visualizing in a notebook is not an option, or if you do not have graphviz installed, you can still produce a dotfile from a scheduler object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.export_as_dotfile('readme.dot')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then later on - and possibly on another host - you can use this dot file as an input to produce a `.png` graphics, using the `dot` program (which is part of `graphviz`), like e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.system(\"dot -Tpng readme.dot -o readme.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which now allows us to render the graph for our last scheduler as a png file:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![manually inserted readme](readme.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if you do have `graphviz` available, you can produce a png file more simply, i.e. without the need for creating the dot file, as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a trick to produce a png file on a box that has graphviz pip-installed\n",
    "g = s.graph()\n",
    "g.format = 'png'\n",
    "g.render('readme')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nesting schedulers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned in the introduction, a `Scheduler` instance can itself be used as a job. This makes it easy to split complex scenarii into pieces, and to combine them in a modular way.\n",
    "\n",
    "Let us consider the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we start with the creation of an internal scheduler\n",
    "# that has a simple diamond structure\n",
    "\n",
    "sub_sched = Scheduler(label=\"critical nested\", critical=True)\n",
    "subj1 = Job(aprint(\"subj1\"), label='subj1', scheduler=sub_sched)\n",
    "subj2 = Job(aprint(\"subj2\"), label='subj2', required=subj1, scheduler=sub_sched)\n",
    "subj3 = Job(aprint(\"subj3\"), label='subj3', required=subj1, scheduler=sub_sched)\n",
    "subj4 = Job(aprint(\"subj4\"), label='subj4', required=(subj2, subj3), scheduler=sub_sched)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now create a main scheduler, in which **one of the jobs is this low-level scheduler**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the main scheduler\n",
    "main_sched = Scheduler(\n",
    "    Sequence(\n",
    "        Job(aprint(\"main-start\"), label=\"main-start\"),\n",
    "        # the way to graft the low-level logic in this main workflow\n",
    "        # is to just use the ShcdulerJOb instance as a job\n",
    "        sub_sched,\n",
    "        Job(aprint(\"main-end\"), label=\"main-end\"),\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This nested structure is rendered by both `list()` and `graph()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_style": "center"
   },
   "outputs": [],
   "source": [
    "# list() shows the contents of sub-schedulers implemented as Scheduler instances\n",
    "main_sched.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using a `Scheduler` to describe nested schedulers, `asynciojobs` will also produce a graphical output that properly exhibits the overall structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_style": "center"
   },
   "outputs": [],
   "source": [
    "# the easiest way to see \n",
    "# a scheduler as a graph is from a notebook:\n",
    "main_sched.graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us do this again another way, so that this shows up properly in *readthedocs*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "main_dot = main_sched.graph()\n",
    "main_dot.format = 'png'\n",
    "main_dot.render(\"readme-nested\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That we now visualize again, using the produced png:\n",
    "\n",
    "![](readme-nested.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which when executed produces this output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "main_sched.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Benefits of nesting schedulers\n",
    "\n",
    "This feature can can come in handy to deal with issues like:\n",
    "\n",
    "* you want to be able to re-use code - as in writing a library - and nesting schedulers is a convenient way to address that; functions can return pieces of workflows implemented as schedulers, that can be easily mixed within a larger scenario;\n",
    "\n",
    "* in another dimension, nested schedulers can be a solution if\n",
    "  * you want the `jobs_window` attribute to apply to only a subset of your jobs;\n",
    "  * or you need the `timeout` attribute to apply to only a subset of your jobs;\n",
    "  * you have `forever` jobs that need to be terminated sooner than the very end of the overall scenario.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Historical note\n",
    "\n",
    "Internally, `asynciojobs` comes with the `PureScheduler` class. \n",
    "\n",
    "A `PureScheduler` instance is a fully functional scheduler, but it cannot be used as a nested scheduler.\n",
    "\n",
    "In terms of implementation, `Scheduler` is a mixin class that inherits from both `PureScheduler` and `AbstractJob`.\n",
    "\n",
    "In previous versions of this library, the `Scheduler` class could not be nested, and a specific class was required for the purpose of creating nestable schedulers, like is shown in this table:\n",
    "\n",
    "<table>\n",
    "  <thead>\n",
    "      <tr><th>version</th><th>just scheduling</th><th>nestable scheduler</th></tr>\n",
    "  </thead>\n",
    "  <tbody>\n",
    "      <tr><td><= 0.8</td><td><code>Scheduler</code></td><td>no class available</td></tr>\n",
    "      <tr><td>== 0.9</td><td><code>Scheduler</code></td><td><code>SchedulerJob</code></td></tr>\n",
    "      <tr><td>>= 0.10</td><td><code>PureScheduler</code></td><td><code>Scheduler</code></td></tr>\n",
    "  </tbody>\n",
    "</table>\n",
    "\n",
    "The bottom line is that, starting with version 0.10, users primarily do not need to worry about that, and creating only nestable `Scheduler` objects is the recommended approach."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Troubleshooting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a general rule, and maybe especially when dealing with nested schedulers, it is important to keep in mind the following constraints."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Don't insert a job in several schedulers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One given job should be inserted in **exactly one** scheduler. Be aware that the code does not check for this, it is the programmer's responsability to enforce this rule. \n",
    "\n",
    "A job that is not inserted in any scheduler will of course never be run. \n",
    "\n",
    "A job inserted in several schedulers will most likely behave very oddly, as each scheduler will be in a position to have it move along.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### You can only create requirements between jobs in the same scheduler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With nested schedulers, it can be tempting to create dependencies between jobs that are not part of the same scheduler, but that belong in sibling or cousin schedulers. \n",
    "\n",
    "This is **currently not supported**, a job can only have requirements to **other jobs in the same scheduler**.\n",
    "\n",
    "Like for the previous topic, as of now there is no provision in the code to enforce that, and failing to comply with that rule will result in unexpected behaviour."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create as many job instances as needed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another common mistake is to try and reuse a job instance in several places in a scheduler. Each instance carries the state of the job progress, so it is important to create as many instances/copies as there are tasks, and to not try and share job objects.\n",
    "\n",
    "In particular, if you take one job instance that has completed, and try to insert it into a new scheduler, it will be considered as done, and will not run again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### You can't run the same scheduler twice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In much the same way, once a scheduler is done - assuming all went well - essentially all of its jobs are marked as done, and trying to run it again will either do nothing, or raise an exception."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "notebookname": "Installing",
  "version": "1.0"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
