{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BroydenSolver\n",
    "\n",
    "BroydenSolver is a quasi-Newton solver that implements Broyden's second method to solve for values of the model's states that drive their residuals to zero. It does so by maintaining an approximation to the inverse of the Jacobian of the model or a subset of the model. In some cases this can be more efficient than NewtonSolver because updating the approximated inverse Jacobian is cheaper than solving the linear system. It may take more iterations because the search direction depends on an approximation, but the iterations take fewer operations.\n",
    "\n",
    "The BroydenSolver has two different modes of operation. It can operate on the entire model and solve for every state in the containing system and all subsystems. Alternatively, it can operate on a subset of the model and only solve for a list of states that you provide. The advantage of full-model mode is that you don't have to worry about forgetting a state, particularly in large models where you might not be familiar with every component or variable. The disadvantage is that you are computing the inverse of a larger matrix every time you recalculate the inverse jacobian, though ideally you are not recomputing this very often. Operating on a subset of states is more efficient in both the linear solve and the Broyden update, but you do run the risk of missing a state. The BroydenSolver will print a warning if it finds any states in the model that aren't covered by a solver.\n",
    "\n",
    "\n",
    "## BroydenSolver Options"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input"
    ]
   },
   "outputs": [],
   "source": [
    "import openmdao.api as om\n",
    "om.show_options_table(\"openmdao.solvers.nonlinear.broyden.BroydenSolver\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The BroydenSolver also contains a slot for a linear solver and a slot for a linesearch. See the [linesearch section](linesearch-section) for more about these.\n",
    "\n",
    "## BroydenSolver Constructor\n",
    "\n",
    "The call signature for the `BroydenSolver` constructor is:\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.solvers.nonlinear.broyden.BroydenSolver.__init__\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "## BroydenSolver on a Full Model\n",
    "\n",
    "Here we show an example that uses the [electrical circuit model](../../../advanced_user_guide/models_implicit_components/models_with_solvers_implicit.ipynb) from the\n",
    "advanced guide. We have replaced the `NewtonSolver` with a `BroydenSolver`, and set the maximum number of iterations to 20. We also assign a `DirectSolver` into the \"linear_solver\" slot on the `BroydenSolver`.  This is the linear solver that will be used to assemble the Jacobian and compute its inverse. Since we don't specify any states in the `state_vars` option, the BroydenSolver operates on the entire model. If you don't specify a linear_solver here, then the BroydenSolver will use the one from the system.\n",
    "\n",
    "```{Note}\n",
    "In this mode, only the `DirectSolver` can be used as the linear_solver.\n",
    "```\n",
    "\n",
    "Depending on the values of some of the other options such as \"converge_limit\", \"diverge_limit\", and \"max_converge_failures\", the Jacobian might be recalculated if convergence stalls, though this doesn't happen in the electrical circuit example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src21\", get_code(\"openmdao.test_suite.scripts.circuit_analysis.Circuit\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `Circuit` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src21`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openmdao.api as om\n",
    "from openmdao.test_suite.scripts.circuit_analysis import Circuit\n",
    "\n",
    "p = om.Problem()\n",
    "model = p.model\n",
    "\n",
    "model.add_subsystem('circuit', Circuit(), promotes_inputs=[('Vg', 'V'), ('I_in', 'I')])\n",
    "model.set_input_defaults('V', 0., units='V')\n",
    "model.set_input_defaults('I', 0.1, units='A')\n",
    "\n",
    "p.setup()\n",
    "\n",
    "# Replace existing solver with BroydenSolver\n",
    "model.circuit.nonlinear_solver = om.BroydenSolver()\n",
    "model.circuit.nonlinear_solver.options['maxiter'] = 20\n",
    "model.circuit.nonlinear_solver.linear_solver = om.DirectSolver()\n",
    "\n",
    "# set some initial guesses\n",
    "p.set_val('circuit.n1.V', 10.)\n",
    "p.set_val('circuit.n2.V', 1.)\n",
    "\n",
    "p.set_solver_print(level=2)\n",
    "p.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.get_val('circuit.n1.V'))\n",
    "print(p.get_val('circuit.n2.V'))\n",
    "\n",
    "# sanity check: should sum to .1 Amps\n",
    "print(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.assert_utils import assert_near_equal\n",
    "\n",
    "assert_near_equal(p.get_val('circuit.n1.V'), 9.90804735, 1e-5)\n",
    "assert_near_equal(p.get_val('circuit.n2.V'), 0.71278226, 1e-5)\n",
    "\n",
    "# sanity check: should sum to .1 Amps\n",
    "assert_near_equal(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'), .1, 1e-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## BroydenSolver on a Subset of States\n",
    "\n",
    "The `BroydenSolver` can also be used to solve for specific states. Here we consider the same circuit example, but instead we specify the two voltages n1.V' and 'n2.V' as our \"state_vars\".  In this mode, we aren't limited to just using the `DirectSolver`, and in this example we choose `LinearBlockGS` instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.scripts.circuit_analysis import Circuit\n",
    "\n",
    "p = om.Problem()\n",
    "model = p.model\n",
    "\n",
    "model.add_subsystem('circuit', Circuit(), promotes_inputs=[('Vg', 'V'), ('I_in', 'I')])\n",
    "model.set_input_defaults('V', 0., units='V')\n",
    "model.set_input_defaults('I', 0.1, units='A')\n",
    "\n",
    "p.setup()\n",
    "\n",
    "# Replace existing solver with om.BroydenSolver\n",
    "model.circuit.nonlinear_solver = om.BroydenSolver()\n",
    "model.circuit.nonlinear_solver.options['maxiter'] = 20\n",
    "\n",
    "# Specify states for Broyden to solve\n",
    "model.circuit.nonlinear_solver.options['state_vars'] = ['n1.V', 'n2.V']\n",
    "\n",
    "model.nonlinear_solver.linear_solver = om.LinearBlockGS()\n",
    "\n",
    "# set some initial guesses\n",
    "p.set_val('circuit.n1.V', 10.)\n",
    "p.set_val('circuit.n2.V', 1.)\n",
    "\n",
    "p.set_solver_print(level=2)\n",
    "p.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.get_val('circuit.n1.V'))\n",
    "print(p.get_val('circuit.n2.V'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sanity check: should sum to .1 Amps\n",
    "print(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(p.get_val('circuit.n1.V'), 9.90804735, 1e-5)\n",
    "assert_near_equal(p.get_val('circuit.n2.V'), 0.71278226, 1e-5)\n",
    "\n",
    "# sanity check: should sum to .1 Amps\n",
    "assert_near_equal(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'), .1, 1e-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## BroydenSolver for Models Without Derivatives\n",
    "\n",
    "The `BroydenSolver` can be used for models where you don't have any partial derivatives defined, and don't wish to use finite difference to calculate them. This behavior is activated by setting the \"compute_jacobian\" option to False. Instead of calculating an initial Jacobian, we start with an estimate that is just the identity matrix scaled by a tunable parameter in the options called \"alpha\". As the `BroydenSolver` iterates, this estimate of the Jacobian is improved, and for some problems, a solution can be reached that satisfies the residual equations.\n",
    "\n",
    "In this example, we solve for the coupling variable in a version of the Sellar model that severs the cycle\n",
    "and expresses the difference across the broken cycle as an implicit state, which the `BroydenSolver` will\n",
    "solve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src22\", get_code(\"openmdao.test_suite.components.sellar.SellarStateConnection\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarStateConnection` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src22`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarStateConnection\n",
    "\n",
    "prob = om.Problem()\n",
    "model = prob.model = SellarStateConnection(nonlinear_solver=om.BroydenSolver(),\n",
    "                                           linear_solver=om.LinearRunOnce())\n",
    "\n",
    "prob.setup()\n",
    "\n",
    "model.nonlinear_solver.options['state_vars'] = ['state_eq.y2_command']\n",
    "model.nonlinear_solver.options['compute_jacobian'] = False\n",
    "\n",
    "prob.set_solver_print(level=2)\n",
    "prob.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(prob['y1'])\n",
    "print(prob['state_eq.y2_command'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(prob['y1'], 25.58830273, .00001)\n",
    "assert_near_equal(prob['state_eq.y2_command'], 12.05848819, .00001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## BroydenSolver Option Examples\n",
    "\n",
    "There are a few additional options that give you more control over when and how often the Jacobian is recomputed.\n",
    "The \"diverge_limit\" option allows you to define a limit to the ratio of current residual and the previous iteration's residual above which the solution is considered to be diverging. If this limit is exceeded, then the Jacobian is always recomputed on the next iteration. There is also a \"converge_limit\" that allows you similarly define a limit above which the solution is considered to be non-converging. When this limit is exceeded, the Jacobian is not immediately recomputed until the limit has been exceeded a number of consecutive times as defined by the \"max_converge_failures\" option. The default value for \"max_converge_failures\" is 3, and the default \"converge_limit\" is 1.0. Exploring these options can help you solve more quickly (or in some cases solve at all) some tougher problems.\n",
    "\n",
    "Here, we take the same circuit example from above and specify a much lower \"converge_limit\" and \"max_converge_failures\" to force recomputation of the Jacobian much more frequently. This results in a quicker convergence in terms of the number of iterations, though keep in mind that solving for the derivatives adds computational cost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.scripts.circuit_analysis import Circuit\n",
    "\n",
    "p = om.Problem()\n",
    "model = p.model\n",
    "\n",
    "model.add_subsystem('circuit', Circuit(), promotes_inputs=[('Vg', 'V'), ('I_in', 'I')])\n",
    "model.set_input_defaults('V', 0., units='V')\n",
    "model.set_input_defaults('I', 0.1, units='A')\n",
    "\n",
    "p.setup()\n",
    "\n",
    "# Replace existing solver with BroydenSolver\n",
    "model.circuit.nonlinear_solver = om.BroydenSolver()\n",
    "model.circuit.nonlinear_solver.options['maxiter'] = 20\n",
    "model.circuit.nonlinear_solver.options['converge_limit'] = 0.1\n",
    "model.circuit.nonlinear_solver.options['max_converge_failures'] = 1\n",
    "\n",
    "# Specify states for Broyden to solve\n",
    "model.circuit.nonlinear_solver.options['state_vars'] = ['n1.V', 'n2.V']\n",
    "\n",
    "# set some initial guesses\n",
    "p.set_val('circuit.n1.V', 10.)\n",
    "p.set_val('circuit.n2.V', 1.)\n",
    "\n",
    "p.set_solver_print(level=2)\n",
    "p.run_model()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(p.get_val('circuit.n1.V'))\n",
    "print(p.get_val('circuit.n2.V'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sanity check: should sum to .1 Amps\n",
    "print(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "assert_near_equal(p.get_val('circuit.n1.V'), 9.90804735, 1e-5)\n",
    "assert_near_equal(p.get_val('circuit.n2.V'), 0.71278226, 1e-5)\n",
    "\n",
    "# sanity check: should sum to .1 Amps\n",
    "assert_near_equal(p.get_val('circuit.R1.I') + p.get_val('circuit.D1.I'), .1, 1e-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**stall_limit and stall_tol**\n",
    "\n",
    "  In some cases, nonlinear solvers can stall out where the norm of the residual stops changing at all. This\n",
    "  can happen for a couple of reasons. You can hit numerical noise problems and just be wandering around in\n",
    "  a circle, or you can get stuck on a bound and the line search just keeps running into the same spot no\n",
    "  matter what. Either way, if you have say 100 max iterations and you stall at 15 ... you waste a lot of\n",
    "  compute time. To remedy this, you can turn on stall detection in all nonlinear solvers by setting the\n",
    "  \"stall_limit\" option to a number greater than zero.\n",
    "\n",
    "  In this example, we set stall_limit to 3. While the solver iterates, it will compare the value of the\n",
    "  residual norm to the value computed in the previous iteration.  If the value matches for three iterations\n",
    "  in a row, then iteration will terminate due to detection of a stall. If \"err_on_non_converge\" is set\n",
    "  to True, then an ``AnalysisError`` will be raised just as if we had reached the iteration count limit.\n",
    "\n",
    "  We also set the `stall_tol` to 1e-6, which is the threshold below which a change in the relative residual\n",
    "  norm is considered to be unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prob = om.Problem()\n",
    "\n",
    "prob.model.add_subsystem('comp', om.ExecComp('y=3*x+1'), promotes=['*'])\n",
    "\n",
    "balance = prob.model.add_subsystem('balance', om.BalanceComp(),\n",
    "                                   promotes=['*'])\n",
    "balance.add_balance('x', lower=-.1, upper=10, rhs_val=0, lhs_name='y')\n",
    "\n",
    "nl_solver = prob.model.nonlinear_solver = om.BroydenSolver()\n",
    "nl_solver.options['stall_limit'] = 3\n",
    "nl_solver.options['stall_tol'] = 1e-8\n",
    "nl_solver.options['maxiter'] = 100\n",
    "\n",
    "prob.model.linear_solver = om.DirectSolver()\n",
    "\n",
    "prob.setup()\n",
    "prob.set_solver_print()\n",
    "\n",
    "prob.run_model()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.1"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
