{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# iminuit and an external minimizer\n",
    "\n",
    "This is a topical tutorial. If you are new to iminuit, you should go through the basic tutorial first. \n",
    "\n",
    "It is possible to use another minimizer to find the minimum of a function, and then use iminuit to compute the error matrix. We will demonstrate this with a maximum-likelihood fit of a normal distribution, which is carried out with scipy.optimize.minimize. iminuit is then used to compute the parameter uncertainties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from iminuit import Minuit\n",
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "from scipy.optimize import minimize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "      fun: 1391.4995267030417\n",
       " hess_inv: array([[ 4.04113200e-04, -1.35575101e-06],\n",
       "       [-1.35575101e-06,  4.76295795e-04]])\n",
       "      jac: array([1.52587891e-05, 0.00000000e+00])\n",
       "  message: 'Desired error not necessarily achieved due to precision loss.'\n",
       "     nfev: 65\n",
       "      nit: 12\n",
       "     njev: 16\n",
       "   status: 2\n",
       "  success: False\n",
       "        x: array([-0.02758092,  0.97293402])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# normally distributed data\n",
    "x = norm(0, 1).rvs(1000)\n",
    "\n",
    "# negative log-likelihood for a normal distribution\n",
    "def nll(par):\n",
    "    return -np.sum(norm(par[0], par[1]).logpdf(x))\n",
    "\n",
    "# minimize nll with scipy.optimize.minimize\n",
    "result = minimize(nll, np.ones(2))\n",
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr style=\"background-color:#F4F4F4;\">\n",
       "<td/>\n",
       "<th title=\"Variable name\">\n",
       "Name\n",
       "</th>\n",
       "<th title=\"Value of parameter\">\n",
       "Value\n",
       "</th>\n",
       "<th title=\"Hesse error\">\n",
       "Hesse Error\n",
       "</th>\n",
       "<th title=\"Minos lower error\">\n",
       "Minos Error-\n",
       "</th>\n",
       "<th title=\"Minos upper error\">\n",
       "Minos Error+\n",
       "</th>\n",
       "<th title=\"Lower limit of the parameter\">\n",
       "Limit-\n",
       "</th>\n",
       "<th title=\"Upper limit of the parameter\">\n",
       "Limit+\n",
       "</th>\n",
       "<th title=\"Is the parameter fixed in the fit\">\n",
       "Fixed\n",
       "</th>\n",
       "</tr>\n",
       "<tr style=\"background-color:#FFFFFF;\">\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "x0\n",
       "</td>\n",
       "<td>\n",
       "-0.03\n",
       "</td>\n",
       "<td>\n",
       "0.04\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "</tr>\n",
       "<tr style=\"background-color:#F4F4F4;\">\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "x1\n",
       "</td>\n",
       "<td>\n",
       "0.973\n",
       "</td>\n",
       "<td>\n",
       "0.031\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "</tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "------------------------------------------------------------------------------------------\n",
       "|   | Name |   Value   | Hesse Err | Minos Err- | Minos Err+ | Limit-  | Limit+  | Fixed |\n",
       "------------------------------------------------------------------------------------------\n",
       "| 0 | x0   |   -0.03   |   0.04    |            |            |         |         |       |\n",
       "| 1 | x1   |   0.973   |   0.031   |            |            |         |         |       |\n",
       "------------------------------------------------------------------------------------------"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# initialize Minuit with the fit result from scipy.optimize.minimize\n",
    "m = Minuit.from_array_func(nll, result.x, pedantic=False)\n",
    "m.hesse() # this also works without calling MIGRAD before"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the confidence intervals produced by HESSE are sufficient, iminuit can be used effectively together with other minimizers.\n",
    "\n",
    "We can also compute the \"Hesse errors\" at any other point than the minimum. These cannot be interpreted as confidence intervals, they are just some numbers related to the second derivative of the cost function at that point.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr style=\"background-color:#F4F4F4;\">\n",
       "<td/>\n",
       "<th title=\"Variable name\">\n",
       "Name\n",
       "</th>\n",
       "<th title=\"Value of parameter\">\n",
       "Value\n",
       "</th>\n",
       "<th title=\"Hesse error\">\n",
       "Hesse Error\n",
       "</th>\n",
       "<th title=\"Minos lower error\">\n",
       "Minos Error-\n",
       "</th>\n",
       "<th title=\"Minos upper error\">\n",
       "Minos Error+\n",
       "</th>\n",
       "<th title=\"Lower limit of the parameter\">\n",
       "Limit-\n",
       "</th>\n",
       "<th title=\"Upper limit of the parameter\">\n",
       "Limit+\n",
       "</th>\n",
       "<th title=\"Is the parameter fixed in the fit\">\n",
       "Fixed\n",
       "</th>\n",
       "</tr>\n",
       "<tr style=\"background-color:#FFFFFF;\">\n",
       "<td>\n",
       "0\n",
       "</td>\n",
       "<td>\n",
       "x0\n",
       "</td>\n",
       "<td>\n",
       "1.00\n",
       "</td>\n",
       "<td>\n",
       "0.04\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "</tr>\n",
       "<tr style=\"background-color:#F4F4F4;\">\n",
       "<td>\n",
       "1\n",
       "</td>\n",
       "<td>\n",
       "x1\n",
       "</td>\n",
       "<td>\n",
       "500m\n",
       "</td>\n",
       "<td>\n",
       "9m\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "<td>\n",
       "\n",
       "</td>\n",
       "</tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "------------------------------------------------------------------------------------------\n",
       "|   | Name |   Value   | Hesse Err | Minos Err- | Minos Err+ | Limit-  | Limit+  | Fixed |\n",
       "------------------------------------------------------------------------------------------\n",
       "| 0 | x0   |   1.00    |   0.04    |            |            |         |         |       |\n",
       "| 1 | x1   |   500m    |    9m     |            |            |         |         |       |\n",
       "------------------------------------------------------------------------------------------"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.args[:] = (1.0, 0.5)\n",
    "m.hesse()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using MINOS does not work (yet...), however. MINOS requires an existing function minimum produced by MIGRAD, which contains more information than just the parameter values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"<ipython-input-5-068aa930b7fa>\", line 2, in <module>\n",
      "    m.minos()\n",
      "  File \"src/iminuit/_libiminuit.pyx\", line 954, in iminuit._libiminuit.Minuit.minos\n",
      "RuntimeError: MINOS require function to be at the minimum. Run MIGRAD first.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    m.minos()\n",
    "except:\n",
    "    import traceback\n",
    "    traceback.print_exc()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
