{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode  # noqa: F401\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ScipyKrylov\n",
    "\n",
    "ScipyKrylov is an iterative linear solver that wraps the methods found in `scipy.sparse.linalg`.\n",
    "The default method is \"gmres\", or the Generalized Minimal RESidual method. Support for other\n",
    "`scipy.sparse.linalg` solvers will be added over time. This linear solver is capable of handling any\n",
    "system topology very effectively. It also solves all subsystems below it in the hierarchy, so\n",
    "assigning different solvers to subsystems will have no effect on the solution at this level.\n",
    "\n",
    "This is a serial solver, so it should never be used under MPI; use [PETScKrylov](../../../_srcdocs/packages/solvers.linear/petsc_ksp) instead.\n",
    "\n",
    "Here, we calculate the total derivatives across the Sellar system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src41\", get_code(\"openmdao.test_suite.components.sellar.SellarDis1withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis1withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src41`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src42\", get_code(\"openmdao.test_suite.components.sellar.SellarDis2withDerivatives\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis2withDerivatives` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src42`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "\n",
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001056, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448534, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ScipyKrylov Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "om.show_options_table(\"openmdao.solvers.linear.scipy_iter_solver.ScipyKrylov\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ScipyKrylov Constructor\n",
    "\n",
    "The call signature for the `ScipyKrylov` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.solvers.linear.scipy_iter_solver.ScipyKrylov.__init__\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "## ScipyKrylov Option Examples\n",
    "\n",
    "**maxiter**\n",
    "\n",
    "  `maxiter` lets you specify the maximum number of GMRES iterations to apply. The default maximum is 1000, which\n",
    "  is much higher than the other linear solvers because each multiplication by the system Jacobian is considered\n",
    "  to be an iteration. You may have to decrease this value if you have a coupled system that is converging\n",
    "  very slowly. (Of course, in such a case, it may be better to add a preconditioner.)  Alternatively, you\n",
    "  may have to raise it if you have an extremely large number of components in your system (a 1000-component\n",
    "  ring would need 1000 iterations just to make it around once.)\n",
    "\n",
    "  This example shows what happens if you set `maxiter` too low. Since iteration is terminated after only 3 iterations, the solver does not converge and the derivatives will not be correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.ScipyKrylov()\n",
    "model.linear_solver.options['maxiter'] = 3\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on your situation, you may want to set the `err_on_non_converge` option to `True`, so that this will raise an AnalysisError."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "from openmdao.api import AnalysisError\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.ScipyKrylov()\n",
    "model.linear_solver.options['maxiter'] = 3\n",
    "model.linear_solver.options['err_on_non_converge'] = True\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "try:\n",
    "    J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')\n",
    "except AnalysisError:\n",
    "    # Special handling for a failure to converge when computing derivatives\n",
    "    print(\"===\\nLinear Solver failed to converge, derivatives are not valid.\\n===\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**atol**\n",
    "\n",
    "  Here, we set the absolute tolerance to a much tighter value (default is 1.0e-12) to show what happens. In\n",
    "  practice, the tolerance serves a dual role in GMRES. In addition to being a termination criteria, the tolerance\n",
    "  also defines what GMRES considers to be tiny. Tiny numbers are replaced by zero when the argument vector is\n",
    "  normalized at the start of each new matrix-vector product. The end result here is that we iterate longer to get\n",
    "  a marginally better answer.\n",
    "\n",
    "  You may need to adjust this setting if you have abnormally large or small values in your global Jacobian."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarDis1withDerivatives, SellarDis2withDerivatives\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "model.add_subsystem('d1', SellarDis1withDerivatives(), promotes=['x', 'z', 'y1', 'y2'])\n",
    "model.add_subsystem('d2', SellarDis2withDerivatives(), promotes=['z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('obj_cmp', om.ExecComp('obj = x**2 + z[1] + y1 + exp(-y2)',\n",
    "                                           z=np.array([0.0, 0.0]), x=0.0),\n",
    "                    promotes=['obj', 'x', 'z', 'y1', 'y2'])\n",
    "\n",
    "model.add_subsystem('con_cmp1', om.ExecComp('con1 = 3.16 - y1'), promotes=['con1', 'y1'])\n",
    "model.add_subsystem('con_cmp2', om.ExecComp('con2 = y2 - 24.0'), promotes=['con2', 'y2'])\n",
    "\n",
    "model.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "model.linear_solver = om.ScipyKrylov()\n",
    "model.linear_solver.options['atol'] = 1.0e-20\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "prob.set_val('x', 1.)\n",
    "prob.set_val('z', np.array([5.0, 2.0]))\n",
    "\n",
    "prob.run_model()\n",
    "\n",
    "wrt = ['z']\n",
    "of = ['obj']\n",
    "\n",
    "J = prob.compute_totals(of=of, wrt=wrt, return_format='flat_dict')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(J['obj', 'z'][0][0])\n",
    "print(J['obj', 'z'][0][1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(J['obj', 'z'][0][0], 9.61001055699, .00001)\n",
    "assert_near_equal(J['obj', 'z'][0][1], 1.78448533563, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**rtol**\n",
    "\n",
    "  The 'rtol' setting is not supported by SciPy GMRES for versions earlier than SciPy 1.12.0.\n",
    "\n",
    "## Specifying a Preconditioner\n",
    "\n",
    "You can specify a preconditioner to improve the convergence of the iterative linear solution by setting the `precon` attribute. The\n",
    "motivation for using a preconditioner is the observation that iterative methods have better convergence\n",
    "properties if the linear system has a smaller condition number, so the goal of the preconditioner is to\n",
    "improve the condition number in part or all of the Jacobian.\n",
    "\n",
    "Here, we add a Gauss-Seidel preconditioner to a problem that contains two subgroups, each with an implicit component that implements a quadratic\n",
    "equation. These are solved together by a Newton solver at the top.  The goal of the preconditioner here is to solve the smaller linear systems\n",
    "for the quadratic components independently, and use that solution to precondition the full system solution. This is accomplished by setting up\n",
    "the linear solver hierarchy so that the preconditioner is `LinearBlockGS` and the subsytems `sub1` and `sub2` contain a `DirectSolver`.\n",
    "\n",
    "Note that the number of GMRES iterations is lower when using the preconditioner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src43\", get_code(\"openmdao.test_suite.components.quad_implicit.QuadraticComp\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `QuadraticComp` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src43`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": [
     "output_scroll"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.quad_implicit import QuadraticComp\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model\n",
    "\n",
    "sub1 = model.add_subsystem('sub1', om.Group())\n",
    "sub1.add_subsystem('q1', QuadraticComp())\n",
    "sub1.add_subsystem('z1', om.ExecComp('y = -6.0 + .01 * x'))\n",
    "sub2 = model.add_subsystem('sub2', om.Group())\n",
    "sub2.add_subsystem('q2', QuadraticComp())\n",
    "sub2.add_subsystem('z2', om.ExecComp('y = -6.0 + .01 * x'))\n",
    "\n",
    "model.connect('sub1.q1.x', 'sub1.z1.x')\n",
    "model.connect('sub1.z1.y', 'sub2.q2.c')\n",
    "model.connect('sub2.q2.x', 'sub2.z2.x')\n",
    "model.connect('sub2.z2.y', 'sub1.q1.c')\n",
    "\n",
    "model.nonlinear_solver = om.NewtonSolver(solve_subsystems=False)\n",
    "model.linear_solver = om.ScipyKrylov()\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "model.sub1.linear_solver = om.DirectSolver()\n",
    "model.sub2.linear_solver = om.DirectSolver()\n",
    "\n",
    "model.linear_solver.precon = om.LinearBlockGS()\n",
    "\n",
    "prob.set_solver_print(level=2)\n",
    "prob.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(prob.get_val('sub1.q1.x'))\n",
    "print(prob.get_val('sub2.q2.x'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob.get_val('sub1.q1.x'), 0.99857318, .0001)\n",
    "assert_near_equal(prob.get_val('sub2.q2.x'), 0.99857318, .0001)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
