{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n===================================\nRandom state within joblib.Parallel\n===================================\n\nRandomness is affected by parallel execution differently by the different\nbackends.\n\nIn particular, when using multiple processes, the random sequence can be\nthe same in all processes. This example illustrates the problem and shows\nhow to work around it.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import numpy as np\nfrom joblib import Parallel, delayed"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "A utility function for the example\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def print_vector(vector, backend):\n    \"\"\"Helper function to print the generated vector with a given backend.\"\"\"\n    print('\\nThe different generated vectors using the {} backend are:\\n {}'\n          .format(backend, np.array(vector)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Sequential behavior\n##############################################################################\n\n ``stochastic_function`` will generate five random integers. When\n calling the function several times, we are expecting to obtain\n different vectors. For instance, we will call the function five times\n in a sequential manner, we can check that the generated vectors are all\n different.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def stochastic_function(max_value):\n    \"\"\"Randomly generate integer up to a maximum value.\"\"\"\n    return np.random.randint(max_value, size=5)\n\n\nn_vectors = 5\nrandom_vector = [stochastic_function(10) for _ in range(n_vectors)]\nprint('\\nThe different generated vectors in a sequential manner are:\\n {}'\n      .format(np.array(random_vector)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Parallel behavior\n##############################################################################\n\n Joblib provides three different backend: loky (default), threading, and\n multiprocessing.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "backend = 'loky'\nrandom_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function)(10) for _ in range(n_vectors))\nprint_vector(random_vector, backend)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "backend = 'threading'\nrandom_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function)(10) for _ in range(n_vectors))\nprint_vector(random_vector, backend)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Loky and the threading backends behave exactly as in the sequential case and\ndo not require more care. However, this is not the case regarding the\nmultiprocessing backend.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "backend = 'multiprocessing'\nrandom_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function)(10) for _ in range(n_vectors))\nprint_vector(random_vector, backend)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Some of the generated vectors are exactly the same, which can be a\nproblem for the application.\n\nTechnically, the reason is that all forked Python processes share the\nsame exact random seed. As a results, we obtain twice the same randomly\ngenerated vectors because we are using ``n_jobs=2``. A solution is to\nset the random state within the function which is passed to\n:class:`joblib.Parallel`.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def stochastic_function_seeded(max_value, random_state):\n    rng = np.random.RandomState(random_state)\n    return rng.randint(max_value, size=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "``stochastic_function_seeded`` accepts as argument a random seed. We can\nreset this seed by passing ``None`` at every function call. In this case, we\nsee that the generated vectors are all different.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "random_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function_seeded)(10, None) for _ in range(n_vectors))\nprint_vector(random_vector, backend)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Fixing the random state to obtain deterministic results\n##############################################################################\n\n The pattern of ``stochastic_function_seeded`` has another advantage: it\n allows to control the random_state by passing a known seed. So for instance,\n we can replicate the same generation of vectors by passing a fixed state as\n follows.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "random_state = np.random.randint(np.iinfo(np.int32).max, size=n_vectors)\n\nrandom_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function_seeded)(10, rng) for rng in random_state)\nprint_vector(random_vector, backend)\n\nrandom_vector = Parallel(n_jobs=2, backend=backend)(delayed(\n    stochastic_function_seeded)(10, rng) for rng in random_state)\nprint_vector(random_vector, backend)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.1"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}