{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Unit Testing\n",
    "\n",
    "## Overview\n",
    "\n",
    "Generally, every sub-package in the OpenMDAO hierarchy has a `tests` directory with test modules for the various features of that package.  If your change impacts one of those features, then you should add your test(s) to the relevant module. Optionally, you can create a new test module under the appropriate sub-package.\n",
    "\n",
    "\n",
    "## testflo\n",
    "\n",
    "While you can use `python -m unittest` to run many of the OpenMDAO unit tests, it is recommended that you use [testflo](https://pypi.org/project/testflo/), which is a testing framework that was designed specifically to facilitate OpenMDAO testing.\n",
    "\n",
    "Per the [README](https://github.com/OpenMDAO/testflo/blob/master/README.md), `testflo` is a python testing framework that uses a pipeline of iterators to process test specifications, run the tests, and process the results. Some OpenMDAO features require execution under MPI while some others don't, so\n",
    "`testflo` has the basic functionality of other test frameworks but with the additional\n",
    "ability to run MPI unit tests using `unittest.TestCase` in the same way as all of our other tests.\n",
    "\n",
    "In addition to MPI unit testing, there are a number of other convenient features that `testflo` provides over and above other testing frameworks.  Some of these are:\n",
    "\n",
    "*    concurrent testing\n",
    "*    can be given a directory, a file, a module path,\n",
    "     *file:testcase.method*, *module:testcase.method*, or a file containing\n",
    "     a list of any of these\n",
    "*    *pre_announce* option to print test name before running in order to\n",
    "     quickly identify hanging MPI tests\n",
    "*    end of testing summary including failed tests and optionally skipped tests,\n",
    "     test coverage, all tests that run for longer than a certain time threshold, and\n",
    "     tests that generate deprecation warnings\n",
    "\n",
    "You can install testflo with several other optional testing dependencies by appending `[test]` to the pip install target. For example:\n",
    "\n",
    "`pip install openmdao[test]` or\n",
    "\n",
    "`pip install -e .[test]`\n",
    "\n",
    "## Test Suite\n",
    "\n",
    "The `openmdao.test_suite` package provides a number of test models, scripts and utilities that can be useful when writing unit tests.  Here you will find multiple variations on the simple Paraboloid and Sellar models, for example, as well as more complex examples like a circuit analysis and a multipoint beam optimization.  Using components and models from the test suite can expedite the task of writing new tests and reduce the occurrence of repetitive code in unit tests.  When creating new tests, consider whether there is an existing component or model in the test suite that can be used to exercise the feature under test, and if not then adding one for others to use.\n",
    "\n",
    "\n",
    "## Testing Utilities\n",
    "\n",
    "The `openmdao.utils` package provides a couple of modules to assist with common needs of unit testing.\n",
    "\n",
    "### testing_utils\n",
    "\n",
    "The `testing_utils` module has decorators and context managers to control if and how a test is run.  Here are a few examples:\n",
    "\n",
    "- The `@use_tempdirs` decorator will run test functions in a temporary directory, which is useful when a test generates outputs files that need to be cleaned up or can cause collisions with other tests.\n",
    "\n",
    "- [pyOptSparse](https://github.com/mdolab/pyoptsparse) is an optional dependency for OpenMDAO, but it provides some of the most powerful optimizers (IPOPT and SNOPT in particular).  If your test makes use of one of these, you should use the `@require_pyoptsparse` decorator so that the test is skipped for users that have not installed `pyOptSparse` and the specified optimizer.\n",
    "\n",
    "- For tests that depend on environment variables, the `set_env_vars` context manager is provided to temporarily modify the environment to suit a specific test.\n",
    "\n",
    "- Normally, the `check_partials` function will not check the partial derivatives of Components that are part of OpenMDAO itself. The `force_check_partials` function provided here will override that behavior if needed in a particular test.\n",
    "\n",
    "\n",
    "### assert_utils\n",
    "\n",
    "The `assert_utils` module provides helper functions and context managers that are useful for asserting the outcome of tests. Some of these are:\n",
    "\n",
    "- `assert_near_equal` is a convenient function for comparing values, including a variety of supported data structures, to the values expected by a test.  It is more flexible and can be used in place of various other assert functions provided in the Python `unittest` and `numpy.testing` packages.\n",
    "\n",
    "- `assert_check_partials`, `assert_check_totals`, `assert_no_approx_partials` and `assert_no_dict_jacobians` are beneficial for checking that derivatives are calculated correctly for the model under test.\n",
    "\n",
    "- The `assert_warning`, `assert_warnings` and `assert_no_warning` context managers can be used to check that warnings are issued (or not) as expected.\n",
    "\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
