{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PETScKrylov\n",
    "\n",
    "PETScKrylov is an iterative linear solver that wraps the linear solution methods found in PETSc via petsc4py.\n",
    "The default method is \"fgmres\", or the Flexible Generalized Minimal RESidual method, though you may choose any of\n",
    "the other methods in PETSc. This linear solver is capable of handling any system topology\n",
    "effectively. It also solves all subsystems below it in the hierarchy, so assigning different solvers to\n",
    "subsystems will have no effect on the solution at this level.\n",
    "\n",
    "This solver works under MPI, so it is a good alternative to\n",
    "[ScipyKrylov](../../../_srcdocs/packages/solvers.linear/scipy_iter_solver).\n",
    "This solver is also re-entrant, so there are no problems if it is nested during preconditioning.\n",
    "\n",
    "Here, we calculate the total derivatives across the Sellar system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src39\", get_code(\"openmdao.test_suite.components.sellar.SellarDis1withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis1withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src39`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src40\", get_code(\"openmdao.test_suite.components.sellar.SellarDis2withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis2withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src40`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0], 9.61001056, .00001)\n",
    "print(J['obj', 'z'][0][1], 1.78448534, .00001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "\n",
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001056, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448534, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PETScKrylov Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "om.show_options_table(\"openmdao.solvers.linear.petsc_ksp.PETScKrylov\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PETScKrylov Constructor\n",
    "\n",
    "The call signature for the `PETScKrylov` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.solvers.linear.petsc_ksp.PETScKrylov.__init__\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "## PETScKrylov Option Examples\n",
    "\n",
    "**maxiter**\n",
    "\n",
    "  `maxiter` lets you specify the maximum number of GMRES (or other algorithm) iterations to apply. The default maximum is 100, which\n",
    "  is much higher than the other linear solvers because each multiplication by the system Jacobian is considered\n",
    "  to be an iteration. You may have to decrease this value if you have a coupled system that is converging\n",
    "  very slowly. (Of course, in such a case, it may be better to add a preconditioner.)  Alternatively, you\n",
    "  may have to raise `maxiter` if you have an extremely large number of components in your system (a 1000-component\n",
    "  ring would need 1000 iterations just to make it around once.)\n",
    "\n",
    "  This example shows what happens if you set `maxiter` too low (the derivatives should be nonzero, but it stops too\n",
    "  soon.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "model.linear_solver.options['maxiter'] = 3\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(J['obj', 'z'][0][0], 4.93218027, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.73406455, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**atol**\n",
    "\n",
    "  The absolute convergence tolerance, the absolute size of the (possibly preconditioned) residual norm.\n",
    "\n",
    "  You may need to adjust this setting if you have abnormally large or small values in your global Jacobian."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "model.linear_solver.options['atol'] = 1.0e-20\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001055699, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448533563, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**rtol**\n",
    "\n",
    "  The relative convergence tolerance, the relative decrease in the (possibly preconditioned) residual norm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "model.linear_solver.options['rtol'] = 1.0e-20\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001055699, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448533563, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ksp_type**\n",
    "\n",
    "  You can specify which PETSc algorithm to use in place of 'fgmres' by settng the \"ksp_type\" in the options\n",
    "  dictionary.  Here, we use 'gmres' instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "model.linear_solver.options['ksp_type'] = 'gmres'\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001056, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448534, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(petsckrylov-precon)=\n",
    "## Specifying a Preconditioner\n",
    "\n",
    "You can specify a preconditioner to improve the convergence of the iterative linear solution by setting the `precon` attribute. The motivation for using a preconditioner is the observation that iterative methods have better convergence properties if the linear system has a smaller condition number, so the goal of the preconditioner is to\n",
    "improve the condition number in part or all of the Jacobian.\n",
    "\n",
    "Here, we add a Gauss-Seidel preconditioner to the simple Sellar solution with Newton. Note that the number of GMRES iterations is lower when using the preconditioner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "\n",
    "model.linear_solver.precon = om.LinearBlockGS()\n",
    "model.linear_solver.precon.options['maxiter'] = 2\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(prob.get_val('y1'))\n",
    "print(prob.get_val('y2'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('y1'), 25.58830273, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the default preconditioning \"side\" is right-preconditioning, you can also use left-preconditioning provided that you choose a \"ksp_type\" that supports it. Here we solve the same problem with left-preconditioning using the Richardson method and a `DirectSolver`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "model.linear_solver = om.PETScKrylov()\n",
    "\n",
    "model.linear_solver.precon = om.DirectSolver()\n",
    "model.linear_solver.options['precon_side'] = 'left'\n",
    "model.linear_solver.options['ksp_type'] = 'richardson'\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(prob.get_val('y1'))\n",
    "print(prob.get_val('y2'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('y1'), 25.58830273, .00001)\n",
    "assert_near_equal(prob.get_val('y2'), 12.05848819, .00001)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
