{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Using Dask for single-machine parallel computing\n\nThis example shows the simplest usage of the\n[Dask](https://docs.dask.org/en/stable/)\nbackend on your local machine.\n\nThis is useful for prototyping a solution, to later be run on a truly\n[distributed Dask cluster](https://docs.dask.org/en/stable/deploying.html#distributed-computing),\nas the only change needed is the cluster class.\n\nAnother realistic usage scenario: combining dask code with joblib code,\nfor instance using dask for preprocessing data, and scikit-learn for\nmachine learning. In such a setting, it may be interesting to use\ndistributed as a backend scheduler for both dask and joblib, to\norchestrate the computation.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Setup the distributed client\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from dask.distributed import Client, LocalCluster\n\n# replace with whichever cluster class you're using\n# https://docs.dask.org/en/stable/deploying.html#distributed-computing\ncluster = LocalCluster()\n# connect client to your cluster\nclient = Client(cluster)\n\n# Monitor your computation with the Dask dashboard\nprint(client.dashboard_link)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run parallel computation using dask.distributed\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import time\nimport joblib\n\n\ndef long_running_function(i):\n    time.sleep(0.1)\n    return i"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The verbose messages below show that the backend is indeed the\ndask.distributed one\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "with joblib.parallel_config(backend=\"dask\"):\n    joblib.Parallel(verbose=100)(\n        joblib.delayed(long_running_function)(i) for i in range(10)\n    )"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}