{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "active-ipynb",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    from openmdao.utils.notebook_utils import notebook_mode  # noqa: F401\n",
    "except ImportError:\n",
    "    !python -m pip install openmdao[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Validate a Model Post Run\n",
    "\n",
    "After executing a model, there may be an interest in checking the values of certain inputs or outputs. Although the values may be mathematically correct and converged, the inputs may have been set resulting in an output value that you do not want or that is not physical. If an input / output is found to have an undesired value, this may prompt you to change the value of an input, discretely change the structure and/or connections of the model, or raise a warning if a variable is close to an undesired bound. The `validate` method of a system in the model hierarchy can be overwritten and used to do validation of any inputs and / or outputs. The `validate` method on a system takes in the inputs / outputs in a read-only mode and can be overwritten to do whatever post-run checks are necessary in the system.\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.core.system.System.validate\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "If we, for example, take the systems from the Sellar problem, validate methods can be added to the systems to check values after the model has converged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remove-input",
     "remove-output"
    ]
   },
   "outputs": [],
   "source": [
    "from openmdao.utils.notebook_utils import get_code\n",
    "from myst_nb import glue\n",
    "glue(\"code_src50\", get_code(\"openmdao.test_suite.components.sellar.SellarDis1\"), display=False)\n",
    "glue(\"code_src51\", get_code(\"openmdao.test_suite.components.sellar.SellarDis2\"), display=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ":::{Admonition} `SellarDis1` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src50`\n",
    ":::\n",
    "\n",
    ":::{Admonition} `SellarDis2` class definition \n",
    ":class: dropdown\n",
    "\n",
    "{glue:}`code_src51`\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "import numpy as np\n",
    "import openmdao.api as om\n",
    "from openmdao.test_suite.components.sellar import SellarDis1, SellarDis2\n",
    "\n",
    "class ValidatedSellar1(SellarDis1):\n",
    "    def validate(self, inputs, outputs):\n",
    "        if outputs['y1'] > 20.0:\n",
    "            raise ValueError('Output \"y1\" is greater than 20.')\n",
    "\n",
    "\n",
    "class SellarMDA(om.Group):\n",
    "    def setup(self):\n",
    "        cycle = self.add_subsystem('cycle', om.Group(), promotes=['*'])\n",
    "        cycle.add_subsystem('d1', ValidatedSellar1(), promotes_inputs=['x', 'z', 'y2'],\n",
    "                            promotes_outputs=['y1'])\n",
    "        cycle.add_subsystem('d2', SellarDis2(), promotes_inputs=['z', 'y1'],\n",
    "                            promotes_outputs=['y2'])\n",
    "\n",
    "        self.set_input_defaults('x', 1.0)\n",
    "        self.set_input_defaults('z', np.array([5.0, 2.0]))\n",
    "\n",
    "        # Nonlinear Block Gauss Seidel is a gradient free solver\n",
    "        cycle.nonlinear_solver = om.NonlinearBlockGS()\n",
    "\n",
    "    def validate(self, inputs, outputs):\n",
    "        if outputs['y2'] < 100.0:\n",
    "            warnings.warn('Output \"y2\" is less than 100.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If any of the systems in a model have a validate method that you would like to run, the hierarchy of validates can be run using the `run_validation` method. This method can be run from any system and will go down the model hierarchy relative to the system that called it and run the `validate` method on each system in the hierarchy. Once `run_model()` or `run_driver()` is finished the `run_validation` method may be run.\n",
    "\n",
    "```{eval-rst}\n",
    "    .. automethod:: openmdao.core.system.System.run_validation\n",
    "        :noindex:\n",
    "```\n",
    "\n",
    "Note that when `run_validation` is run, it will collect all of the errors and warnings raised during the `validate` methods that it calls. If there are any errors among them, it will show everything collected under a ValidationError.\n",
    "\n",
    "If the above problem is run, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prob = om.Problem(model=om.Group())\n",
    "prob.model.add_subsystem('cycle', SellarMDA())\n",
    "prob.setup()\n",
    "prob.run_model()\n",
    "try:\n",
    "    prob.model.run_validation()\n",
    "except om.ValidationError as e:\n",
    "    # Avoid printing the entire error traceback for easier viewing\n",
    "    print(f'Validation Error: {e}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If there are no errors among what is collected from the `validate` methods, `run_validation` will print out everything collected without an error.\n",
    "\n",
    "If the above problem is edited to only have warnings with no errors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ValidatedSellar1(SellarDis1):\n",
    "    def validate(self, inputs, outputs):\n",
    "        if outputs['y1'] > 20.0:\n",
    "            warnings.warn('Output \"y1\" is greater than 20.')\n",
    "\n",
    "prob = om.Problem(model=om.Group())\n",
    "prob.model.add_subsystem('cycle', SellarMDA())\n",
    "prob.setup()\n",
    "prob.run_model()\n",
    "prob.model.run_validation()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally, if there are no warnings nor errors collected from the `validate` methods, the `run_validation` will print out a simple message confirming that nothing was raised.\n",
    "\n",
    "If the Sellar problem is run with no validate errors / warnings raised:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openmdao.test_suite.components.sellar import SellarNoDerivatives\n",
    "\n",
    "prob = om.Problem(model=om.Group())\n",
    "prob.model.add_subsystem('cycle', SellarNoDerivatives())\n",
    "prob.setup()\n",
    "prob.run_model()\n",
    "prob.model.run_validation()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "om-dev",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.13"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
