{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Geometric representation\n",
    "\n",
    "**In this example, we compare the profile likelihood paths to the cost contour of the model and show that the paths follows the direction of the canyon.\n",
    "We will also give some interpretation of the results.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:28.850304Z",
     "start_time": "2022-01-27T04:20:28.836321Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from profile_likelihood import profile_likelihood\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the model\n",
    "\n",
    "The model is defined as the sum of trigonometric functions, in the form of\n",
    "$$ f\\left(t; \\vec{\\theta}\\right) = \\sin(\\theta_0 t) + \\cos(\\theta_1 t), $$\n",
    "where we sample at 3 time points $t = [1.0, 1.2, 2.0]^T$.\n",
    "The default parameters are chosen to be $\\vec{\\theta} = [1.0, 2.5]^T$.\n",
    "The data is set to be the predictions of the model at the default parameters and the standard deviation is chosen to be 30% of the data.\n",
    "The implementation of this model is given in `model_trigonometric.py`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:28.860559Z",
     "start_time": "2022-01-27T04:20:28.853686Z"
    }
   },
   "outputs": [],
   "source": [
    "import model_trigonometric as model\n",
    "import manifold\n",
    "\n",
    "best_fit = model.best_fit\n",
    "param_names = model.param_names\n",
    "nparams = model.nparams\n",
    "npred = model.npred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cost contour of this model is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:29.140130Z",
     "start_time": "2022-01-27T04:20:28.864217Z"
    }
   },
   "outputs": [],
   "source": [
    "# Get cost contour data\n",
    "Theta0, Theta1, Cost = manifold.cost_contour_data(\n",
    "    model.cost,\n",
    "    xlist=np.linspace(-5, 5, 100),\n",
    "    ylist=np.linspace(-5, 5, 100),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:30.096474Z",
     "start_time": "2022-01-27T04:20:29.144669Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 58828 characters]\n",
      "text/plain": [
       "<Figure size 900x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax, clb = manifold.plot_cost_contour(Theta0, Theta1, Cost, levels=25, scale=\"log\")\n",
    "ax.plot(best_fit[0], best_fit[1], \"wx\", label=\"best fit\")\n",
    "ax.set_xlabel(param_names[0])\n",
    "ax.set_ylabel(param_names[1])\n",
    "clb.set_label(r\"$cost$\")\n",
    "plt.legend(fontsize=8)\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Profile likelihood computation\n",
    "\n",
    "After the model is defined, then we create a `profile_likelihood` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:30.103056Z",
     "start_time": "2022-01-27T04:20:30.099112Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the likelihood object\n",
    "likelihood = profile_likelihood(model.residuals, nparams, npred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The computation is done by calling the `compute` method.\n",
    "By specifying the boundaries of the parameters and the spacing between points, linearly spaced points are created as the fixed parameter values.\n",
    "The results of the computation are stored as a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:31.350448Z",
     "start_time": "2022-01-27T04:20:30.105510Z"
    }
   },
   "outputs": [],
   "source": [
    "bounds = [-10, 10]\n",
    "dt = 0.1\n",
    "\n",
    "# Calculate profile likelihood\n",
    "results = likelihood.compute(best_fit, bounds=bounds, dt=dt)\n",
    "ldata0 = likelihood.results[\"parameter0\"]\n",
    "ldata1 = likelihood.results[\"parameter1\"]\n",
    "\n",
    "# # Print the results\n",
    "# print(likelihood.results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the results\n",
    "\n",
    "We will now compare the profile likelihood paths to the cost contour of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:32.111780Z",
     "start_time": "2022-01-27T04:20:31.352746Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 84828 characters]\n",
      "text/plain": [
       "<Figure size 900x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the profile likelihood paths on top of the cost contour\n",
    "ax, clb = manifold.plot_cost_contour(Theta0, Theta1, Cost, levels=25, scale=\"log\")\n",
    "ax.plot(\n",
    "    ldata0[\"parameters\"][:, 0],\n",
    "    ldata0[\"parameters\"][:, 1],\n",
    "    \"r--\",\n",
    "    label=f\"scan {param_names[0]}\",\n",
    "    zorder=10,\n",
    ")\n",
    "ax.plot(\n",
    "    ldata1[\"parameters\"][:, 0],\n",
    "    ldata1[\"parameters\"][:, 1],\n",
    "    \"k-\",\n",
    "    label=f\"scan {param_names[1]}\",\n",
    ")\n",
    "ax.plot(best_fit[0], best_fit[1], \"wx\", label=\"best fit\", zorder=20)\n",
    "ax.set_xlim((-5, 5))\n",
    "ax.set_ylim((-5, 5))\n",
    "ax.set_xlabel(param_names[0])\n",
    "ax.set_ylabel(param_names[1])\n",
    "clb.set_label(r\"$cost$\")\n",
    "plt.legend(fontsize=8)\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen above, the profile likelihood paths follow the canyon in the parameter space.\n",
    "Using this behavior, we can deduce the location of the canyon when the cost cost contour is not available for some models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot the cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:32.749116Z",
     "start_time": "2022-01-27T04:20:32.115435Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 49968 characters]\n",
      "text/plain": [
       "<Figure size 900x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the cost along theta_0 direction\n",
    "plt.figure(dpi=150, facecolor=\"w\")\n",
    "plt.plot(ldata0[\"parameters\"][:, 0], ldata0[\"cost\"])\n",
    "plt.xlim(-5, 5)\n",
    "plt.yscale(\"log\")\n",
    "plt.ylim(1e-2, 1e3)\n",
    "plt.xlabel(param_names[0])\n",
    "plt.ylabel(\"Cost\")\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-01-27T04:20:33.353469Z",
     "start_time": "2022-01-27T04:20:32.751846Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46244 characters]\n",
      "text/plain": [
       "<Figure size 900x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the cost along theta_1 direction\n",
    "plt.figure(dpi=150, facecolor=\"w\")\n",
    "plt.plot(ldata1[\"parameters\"][:, 1], ldata1[\"cost\"])\n",
    "plt.xlim(-5, 5)\n",
    "plt.yscale(\"log\")\n",
    "plt.ylim(1e-2, 1e3)\n",
    "plt.ylabel(\"Cost\")\n",
    "plt.xlabel(param_names[1])\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plots of the cost along the paths (the negative profile log-likelihood) show the profiles of the canyon."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
