{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from ipyparallel import Client, error  # noqa: F401\n",
    "cluster=Client(profile=\"mpi\")\n",
    "view=cluster[:]\n",
    "view.block=True\n",
    "\n",
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode  # noqa: F401\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DOEDriver\n",
    "\n",
    "DOEDriver facilitates performing a design of experiments (DOE) with your OpenMDAO model. It will run your model multiple times with different values for the design variables depending on the selected input generator. A number of generators are available, each with its own parameters that can be specified when it is instantiated:\n",
    "\n",
    "  - UniformGenerator\n",
    "  - FullFactorialGenerator\n",
    "  - PlackettBurmanGenerator\n",
    "  - BoxBehnkenGenerator\n",
    "  - LatinHypercubeGenerator\n",
    "  - CSVGenerator\n",
    "  - ListGenerator\n",
    "\n",
    "See the [source documentation](../../../_srcdocs/packages/drivers/doe_generators) of these generators for details.\n",
    "\n",
    "```{Note}\n",
    "`FullFactorialGenerator`, `PlackettBurmanGenerator`, `BoxBehnkenGenerator` and `LatinHypercubeGenerator` are provided via the [pyDOE3](https://pypi.org/project/pyDOE3/) package, which is an updated version of [pyDOE](https://pythonhosted.org/pyDOE/). See the original [pyDOE](https://pythonhosted.org/pyDOE/) page for information on those algorithms.\n",
    "```\n",
    "\n",
    "The generator instance may be supplied as an argument to the `DOEDriver` or as an option.\n",
    "\n",
    "## DOEDriver Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "import openmdao.api as om\n",
    "om.show_options_table(\"openmdao.drivers.doe_driver.DOEDriver\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DOEDriver Constructor\n",
    "\n",
    "The call signature for the `DOEDriver` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.drivers.doe_driver.DOEDriver.__init__\n",
    "       :noindex:\n",
    "```  \n",
    "\n",
    "## Simple Example\n",
    "\n",
    "`UniformGenerator` implements the simplest method and will generate a requested number of samples randomly selected from a uniform distribution across the valid range for each design variable. This example demonstrates its use with a model built on the [Paraboloid](../../../basic_user_guide/single_disciplinary_optimization/first_analysis) Component. An [SqliteRecorder](../../../_srcdocs/packages/recorders/sqlite_recorder) is used to capture the cases that were generated. We can see that that the model was evaluated at random values of x and y between -10 and 10, per the lower and upper bounds of those design variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src16\", get_code(\"openmdao.test_suite.components.paraboloid.Paraboloid\"), display=False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `Paraboloid` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src16`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.paraboloid import Paraboloid\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('comp', Paraboloid(), promotes=['*'])\n",
    "\n",
    "model.add_design_var('x', lower=-10, upper=10)\n",
    "model.add_design_var('y', lower=-10, upper=10)\n",
    "model.add_objective('f_xy')\n",
    "\n",
    "prob.driver = om.DOEDriver(om.UniformGenerator(num_samples=5))\n",
    "prob.driver.add_recorder(om.SqliteRecorder(\"cases.sql\"))\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 0.0)\n",
    "prob.set_val('y', 0.0)\n",
    "\n",
    "prob.run_driver()\n",
    "prob.cleanup()\n",
    "\n",
    "cr = om.CaseReader(prob.get_outputs_dir() / \"cases.sql\")\n",
    "cases = cr.list_cases('driver')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "values = []\n",
    "for case in cases:\n",
    "    outputs = cr.get_case(case).outputs\n",
    "    values.append((outputs['x'], outputs['y'], outputs['f_xy']))\n",
    "\n",
    "print(\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert(len(cases) == 5)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running a DOE in Parallel\n",
    "\n",
    "In a parallel processing environment, it is possible for `DOEDriver` to run cases concurrently. This is done by setting the `run_parallel` option to True as shown in the following example and running your script using MPI.\n",
    "\n",
    "Here we are using the `FullFactorialGenerator` with 3 levels to generate inputs for our Paraboloid model. With two inputs, $3^2=9$ cases have been generated. In this case we are running on four processors and have specified `options['run_parallel']=True` to run cases on all available processors. The cases have therefore been split with 3 cases run on the first processor and 2 cases on each of the other processors.\n",
    "\n",
    "Note that, when running in parallel, the `SqliteRecorder` will generate a separate case file for each processor on which cases are recorded. The case files will have a suffix indicating the recording rank and a message will be displayed indicating the file name, as seen in the example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{note}\n",
    "This feature requires MPI, and may not be able to be run on Colab or Binder.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.paraboloid import Paraboloid\n",
    "\n",
    "prob = om.Problem()\n",
    "\n",
    "prob.model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])\n",
    "prob.model.add_design_var('x', lower=0.0, upper=1.0)\n",
    "prob.model.add_design_var('y', lower=0.0, upper=1.0)\n",
    "prob.model.add_objective('f_xy')\n",
    "\n",
    "prob.driver = om.DOEDriver(om.FullFactorialGenerator(levels=3))\n",
    "prob.driver.options['run_parallel'] = True\n",
    "prob.driver.options['procs_per_model'] = 1\n",
    "\n",
    "prob.driver.add_recorder(om.SqliteRecorder(\"cases.sql\"))\n",
    "\n",
    "prob.setup()\n",
    "prob.run_driver()\n",
    "prob.cleanup()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "# check recorded cases from each case file\n",
    "from mpi4py import MPI\n",
    "rank = MPI.COMM_WORLD.rank\n",
    "\n",
    "filename = \"cases.sql_%d\" % rank\n",
    "\n",
    "cr = om.CaseReader(prob.get_outputs_dir() / filename)\n",
    "cases = cr.list_cases('driver', out_stream=None)\n",
    "print(cases)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "values = []\n",
    "for case in cases:\n",
    "    outputs = cr.get_case(case).outputs\n",
    "    values.append((outputs['x'], outputs['y'], outputs['f_xy']))\n",
    "\n",
    "print(\"\\n\"+\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "assert(len(cases) == 3 if rank == 0 else 2)\n",
    "if rank == 0:\n",
    "    expected = [\n",
    "        {'x': np.array([0.]), 'y': np.array([0.]), 'f_xy': np.array([22.00])},\n",
    "        {'x': np.array([.5]), 'y': np.array([.5]), 'f_xy': np.array([23.75])},\n",
    "        {'x': np.array([1.]), 'y': np.array([1.]), 'f_xy': np.array([27.00])}]\n",
    "elif rank == 1:\n",
    "    expected = [\n",
    "        {'x': np.array([.5]), 'y': np.array([0.]), 'f_xy': np.array([19.25])},\n",
    "        {'x': np.array([1.]), 'y': np.array([.5]), 'f_xy': np.array([21.75])}]\n",
    "elif rank == 2:\n",
    "    expected = [\n",
    "        {'x': np.array([1.]), 'y': np.array([0.]), 'f_xy': np.array([17.00])},\n",
    "        {'x': np.array([0.]), 'y': np.array([1.]), 'f_xy': np.array([31.00])}]\n",
    "else:\n",
    "    expected = [\n",
    "        {'x': np.array([0.]), 'y': np.array([.5]), 'f_xy': np.array([26.25])},\n",
    "        {'x': np.array([.5]), 'y': np.array([1.]), 'f_xy': np.array([28.75])}]\n",
    "\n",
    "\n",
    "exp_values = []\n",
    "for case in expected:\n",
    "    exp_values.append((case['x'], case['y'], case['f_xy']))\n",
    "expect_text = \"\\n\"+\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % vals_i for vals_i in exp_values])\n",
    "\n",
    "assert(\"\\n\"+\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]) == expect_text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running a DOE in Parallel with a Parallel Model\n",
    "\n",
    "If the model that is being subjected to the DOE is also parallel, then the total number of processors should reflect the model size as well as the desired concurrency.\n",
    "\n",
    "To illustrate this, we will demonstrate performing a DOE on a model based on the [ParallelGroup](../../core_features/working_with_groups/parallel_group.ipynb) example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "class FanInGrouped(om.Group):\n",
    "    \"\"\"\n",
    "    Topology where two components in a Group feed a single component\n",
    "    outside of that Group.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "\n",
    "        self.set_input_defaults('x1', 1.0)\n",
    "        self.set_input_defaults('x2', 1.0)\n",
    "\n",
    "        self.sub = self.add_subsystem('sub', om.ParallelGroup(),\n",
    "                                      promotes_inputs=['x1', 'x2'])\n",
    "\n",
    "        self.sub.add_subsystem('c1', om.ExecComp(['y=-2.0*x']),\n",
    "                               promotes_inputs=[('x', 'x1')])\n",
    "        self.sub.add_subsystem('c2', om.ExecComp(['y=5.0*x']),\n",
    "                               promotes_inputs=[('x', 'x2')])\n",
    "\n",
    "        self.add_subsystem('c3', om.ExecComp(['y=3.0*x1+7.0*x2']))\n",
    "\n",
    "        self.connect(\"sub.c1.y\", \"c3.x1\")\n",
    "        self.connect(\"sub.c2.y\", \"c3.x2\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the model itself requires two processors, so in order to run cases concurrently we need to allocate at least four processors in total. We can allocate as many processors as we have available, however the number of processors must be a multiple of the number of processors per model, which is 2 here. Regardless of how many processors we allocate, we need to tell the `DOEDriver` that the model needs 2 processors, which is done by specifying `options['procs_per_model']=2`. From this, the driver figures out how many models it can run in parallel, which in this case is also 2.\n",
    "\n",
    "The `SqliteRecorder` will record cases on the first two processors, which serve as the “root” processors for the parallel cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%px\n",
    "\n",
    "import openmdao.api as om\n",
    "\n",
    "prob = om.Problem(FanInGrouped())\n",
    "\n",
    "prob.model.add_design_var('x1', lower=0.0, upper=1.0)\n",
    "prob.model.add_design_var('x2', lower=0.0, upper=1.0)\n",
    "prob.model.add_objective('c3.y')\n",
    "\n",
    "prob.driver = om.DOEDriver(om.FullFactorialGenerator(levels=3))\n",
    "prob.driver.add_recorder(om.SqliteRecorder(\"cases.sql\"))\n",
    "\n",
    "# the FanInGrouped model uses 2 processes, so we can run\n",
    "# two instances of the model at a time, each using 2 of our 4 procs\n",
    "prob.driver.options['run_parallel'] = True\n",
    "prob.driver.options['procs_per_model'] = procs_per_model = 2\n",
    "\n",
    "prob.setup()\n",
    "prob.run_driver()\n",
    "prob.cleanup()\n",
    "\n",
    "# a separate case file will be written by rank 0 of each parallel model\n",
    "# (the top two global ranks)\n",
    "rank = prob.comm.rank\n",
    "\n",
    "num_models = prob.comm.size // procs_per_model\n",
    "\n",
    "if rank < num_models:\n",
    "    filename = \"cases.sql_%d\" % rank\n",
    "\n",
    "    cr = om.CaseReader(prob.get_outputs_dir() / filename)\n",
    "    cases = cr.list_cases('driver', out_stream=None)\n",
    "\n",
    "    values = []\n",
    "    for case in cases:\n",
    "        outputs = cr.get_case(case).outputs\n",
    "        values.append((outputs['x1'], outputs['x2'], outputs['c3.y']))\n",
    "\n",
    "    print(\"\\n\"+\"\\n\".join([\"x1: %5.2f, x2: %5.2f, c3.y: %6.2f\" % (x1, x2, y) for x1, x2, y in values]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "%%px\n",
    "if rank == 0:\n",
    "    expected = [\n",
    "            {'x1': np.array([0.]), 'x2': np.array([0.]), 'c3.y': np.array([0.00])},\n",
    "            {'x1': np.array([1.]), 'x2': np.array([0.]), 'c3.y': np.array([-6.00])},\n",
    "            {'x1': np.array([.5]), 'x2': np.array([.5]), 'c3.y': np.array([14.50])},\n",
    "            {'x1': np.array([0.]), 'x2': np.array([1.]), 'c3.y': np.array([35.00])},\n",
    "            {'x1': np.array([1.]), 'x2': np.array([1.]), 'c3.y': np.array([29.00])}]\n",
    "elif rank == 1:\n",
    "    expected = [\n",
    "            {'x1': np.array([.5]), 'x2': np.array([0.]), 'c3.y': np.array([-3.00])},\n",
    "            {'x1': np.array([0.]), 'x2': np.array([.5]), 'c3.y': np.array([17.50])},\n",
    "            {'x1': np.array([1.]), 'x2': np.array([.5]), 'c3.y': np.array([11.50])},\n",
    "            {'x1': np.array([.5]), 'x2': np.array([1.]), 'c3.y': np.array([32.00])}]\n",
    "\n",
    "if rank < 2:\n",
    "    exp_values = []\n",
    "    for idx, case in enumerate(expected):\n",
    "        exp_values.append((case['x1'], case['x2'], case['c3.y']))\n",
    "\n",
    "    expect_text = \"\\n\"+\"\\n\".join([\n",
    "        \"x1: %5.2f, x2: %5.2f, c3.y: %6.2f\" % vals_i for vals_i in exp_values])\n",
    "    assert(\"\\n\"+\"\\n\".join([\"x1: %5.2f, x2: %5.2f, c3.y: %6.2f\" % (x1, x2, y) for x1, x2, y in values]) == expect_text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Prepared Cases\n",
    "\n",
    "If you have a previously generated set of cases that you want to run using `DOEDriver`, there are a couple of ways to do that. The first is to provide those inputs via an external file in the CSV (comma separated values) format. The file should be organized with one column per design variable, with the first row containing the names of the design variables. The following example demonstrates how to use such a file to run a DOE using the `CSVGenerator`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "# This cell creates a file called saved_cases.csv.\n",
    "\n",
    "import json\n",
    "import numpy as np\n",
    "\n",
    "expected_csv = '\\n'.join([\n",
    "    \" x ,   y\",\n",
    "    \"0.0,  0.0\",\n",
    "    \"0.5,  0.0\",\n",
    "    \"1.0,  0.0\",\n",
    "    \"0.0,  0.5\",\n",
    "    \"0.5,  0.5\",\n",
    "    \"1.0,  0.5\",\n",
    "    \"0.0,  1.0\",\n",
    "    \"0.5,  1.0\",\n",
    "    \"1.0,  1.0\",\n",
    "])\n",
    "\n",
    "with open('saved_cases.csv', 'w') as f:\n",
    "    f.write(expected_csv)\n",
    "\n",
    "expected = [\n",
    "    {'x': np.array([0.]), 'y': np.array([0.]), 'f_xy': np.array([22.00])},\n",
    "    {'x': np.array([.5]), 'y': np.array([0.]), 'f_xy': np.array([19.25])},\n",
    "    {'x': np.array([1.]), 'y': np.array([0.]), 'f_xy': np.array([17.00])},\n",
    "\n",
    "    {'x': np.array([0.]), 'y': np.array([.5]), 'f_xy': np.array([26.25])},\n",
    "    {'x': np.array([.5]), 'y': np.array([.5]), 'f_xy': np.array([23.75])},\n",
    "    {'x': np.array([1.]), 'y': np.array([.5]), 'f_xy': np.array([21.75])},\n",
    "\n",
    "    {'x': np.array([0.]), 'y': np.array([1.]), 'f_xy': np.array([31.00])},\n",
    "    {'x': np.array([.5]), 'y': np.array([1.]), 'f_xy': np.array([28.75])},\n",
    "    {'x': np.array([1.]), 'y': np.array([1.]), 'f_xy': np.array([27.00])},\n",
    "]\n",
    "\n",
    "values = []\n",
    "cases = []\n",
    "\n",
    "for case in expected:\n",
    "    values.append((case['x'], case['y'], case['f_xy']))\n",
    "    # converting ndarray to list enables JSON serialization\n",
    "    cases.append((('x', list(case['x'])), ('y', list(case['y']))))\n",
    "\n",
    "expected_text = \"\\n\".join([\n",
    "    \"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % vals_i for vals_i in values\n",
    "])\n",
    "\n",
    "expected_json = json.dumps(cases).replace(']]],', ']]],\\n')\n",
    "with open('cases.json', 'w') as f:\n",
    "    f.write(expected_json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.paraboloid import Paraboloid\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('comp', Paraboloid(), promotes=['x', 'y', 'f_xy'])\n",
    "\n",
    "model.add_design_var('x', lower=0.0, upper=1.0)\n",
    "model.add_design_var('y', lower=0.0, upper=1.0)\n",
    "model.add_objective('f_xy')\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 0.0)\n",
    "prob.set_val('y', 0.0)\n",
    "\n",
    "# this file contains design variable inputs in CSV format\n",
    "with open('saved_cases.csv', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run problem with DOEDriver using the CSV file\n",
    "prob.driver = om.DOEDriver(om.CSVGenerator('saved_cases.csv'))\n",
    "prob.driver.add_recorder(om.SqliteRecorder(\"cases.sql\"))\n",
    "\n",
    "prob.run_driver()\n",
    "prob.cleanup()\n",
    "\n",
    "cr = om.CaseReader(prob.get_outputs_dir() / \"cases.sql\")\n",
    "cases = cr.list_cases('driver', out_stream=None)\n",
    "\n",
    "values = []\n",
    "for case in cases:\n",
    "    outputs = cr.get_case(case).outputs\n",
    "    values.append((outputs['x'], outputs['y'], outputs['f_xy']))\n",
    "\n",
    "print(\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert(\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]) == expected_text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second method is to provide the data directly as a list of cases, where each case is a collection of name/value pairs for the design variables. You might use this method if you want to generate the cases programmatically via another algorithm or if the data is available in some format other than a CSV file and you can reformat it into this simple list structure. The `DOEGenerator` you would use in this case is the `ListGenerator`, but if you pass a list directly to the `DOEDriver` it will construct the `ListGenerator` for you. In the following example, a set of cases has been pre-generated and saved in JSON (JavaScript Object Notation) format. The data is decoded and provided to the `DOEDriver` as a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load design variable inputs from JSON file and decode into list\n",
    "with open('cases.json', 'r') as f:\n",
    "    json_data = f.read()\n",
    "\n",
    "print(json_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create DOEDriver using provided list of cases\n",
    "case_list = json.loads(json_data)\n",
    "prob.driver = om.DOEDriver(case_list)\n",
    "\n",
    "prob.driver.add_recorder(om.SqliteRecorder(\"cases.sql\"))\n",
    "\n",
    "prob.run_driver()\n",
    "prob.cleanup()\n",
    "\n",
    "# check the recorded cases\n",
    "cr = om.CaseReader(prob.get_outputs_dir() / \"cases.sql\")\n",
    "cases = cr.list_cases('driver', out_stream=None)\n",
    "\n",
    "values = []\n",
    "for case in cases:\n",
    "    outputs = cr.get_case(case).outputs\n",
    "    values.append((outputs['x'], outputs['y'], outputs['f_xy']))\n",
    "\n",
    "print(\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert(type(prob.driver.options['generator']) is om.ListGenerator)\n",
    "assert(\"\\n\".join([\"x: %5.2f, y: %5.2f, f_xy: %6.2f\" % xyf for xyf in values]) == expected_text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{Note}\n",
    "When using pre-generated cases via `CSVGenerator` or `ListGenerator`, there is no enforcement of the declared bounds on a design variable as with the algorithmic generators.\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "py311forge",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
