{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "908099da-3ae1-4cb5-86c6-b123eb16a067",
   "metadata": {},
   "source": [
    "# Asymmetric errors\n",
    "\n",
    "Sherpa provides *minimal* support for asymmetric errors - where the low and high error values do not match. The approach taken is currently designed to work best wih the Astronomy UI interface, so that will be used in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5c570ab6-a1c8-4b0d-a7f7-1fe18297804e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING: failed to import sherpa.astro.xspec; XSPEC models will not be available\n"
     ]
    }
   ],
   "source": [
    "from sherpa.astro import ui\n",
    "from sherpa.utils.testing import get_datadir"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c715c4ec-fdd1-4867-b2ea-35c3a6740ba5",
   "metadata": {},
   "source": [
    "For this example we shall use one of the test datasets, which contains `x` and `y` values along with the error range on `y`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9608cbc6-3dac-4c4c-9164-2612754f6ad3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.00310   13.383   12.379   15.002\n",
      "0.00284   13.352   12.437   14.323\n",
      "0.00413   14.717   12.992   15.693\n",
      "0.00469   14.275   13.518   15.093\n",
      "0.00451   13.169   12.321   14.319\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "inpath = Path(get_datadir()) / 'gro.txt'\n",
    "\n",
    "# Display the first 5 lines\n",
    "with open(inpath) as fh:\n",
    "    for inval in fh.readlines()[:5]:\n",
    "        print(inval.strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd93e25-b3a5-4425-af15-601efb2148aa",
   "metadata": {},
   "source": [
    "This can be read in with the `load_ascii_with_errors` command (it can also files where the errors are given as relative values by setting `delta=True` in the call):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "74546cad-9017-4924-8b4f-659179a8d48e",
   "metadata": {},
   "outputs": [],
   "source": [
    "ui.load_ascii_with_errors(str(inpath))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d4437e3-3044-4c83-90cc-11e0ff2ff39e",
   "metadata": {},
   "source": [
    "Let's change the name field to avoid the full path:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f7d2ddce-606d-4c5b-bf59-cc1a1fd85e8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "ui.get_data().name = 'gro.txt'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ba44f03-f7c6-471c-9b54-c40420be93a6",
   "metadata": {},
   "source": [
    "If we display the object we can see there's a `staterror` field as well as `elo` and `ehi`. The `staterror` field is set to be the average of the `elo` and `ehi` fields, which give the low and high errors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7534f5a6-a8c9-4d2f-b757-93eb1823ccd9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name      = gro.txt\n",
      "x         = Float64[61]\n",
      "y         = Float64[61]\n",
      "staterror = Float64[61]\n",
      "syserror  = None\n",
      "elo       = Float64[61]\n",
      "ehi       = Float64[61]\n"
     ]
    }
   ],
   "source": [
    "print(ui.get_data())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c33c3ab9-4b44-4264-af01-4f776ee78071",
   "metadata": {},
   "source": [
    "If we access the fields we can see how they are related:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7fa77dae-e6e9-4103-a9fc-d94435cbf0ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  1.312    1.004 - 1.619\n",
      "  0.943    0.915 - 0.971\n",
      "  1.350    1.725 - 0.976\n",
      "  0.787    0.757 - 0.818\n",
      "  0.999    0.848 - 1.150\n"
     ]
    }
   ],
   "source": [
    "d = ui.get_data()\n",
    "\n",
    "for idx, (err, elo, ehi) in enumerate(zip(d.staterror, d.elo, d.ehi)):\n",
    "    if idx == 5:\n",
    "        break\n",
    "        \n",
    "    print(f\"  {err:.3f}    {elo:.3f} - {ehi:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb1426de-d6a8-4f57-934d-1a66aa273155",
   "metadata": {},
   "source": [
    "When the data is plotted the asymmetric errors are shown:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c8444cf3-9c50-4411-a0a1-d80ca0bc5437",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15524 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.plot_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aac4691f-b4e8-4f5e-b985-683e13aa0399",
   "metadata": {},
   "source": [
    "However, the `staterror` field is used when a statistic is needed - e.g. with `calc_stat` or when fitting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "acc68f36-7d2c-45d9-9f33-ec021593d6c6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "248263792.37785548\n"
     ]
    }
   ],
   "source": [
    "ui.set_stat(\"chi2\")\n",
    "ui.set_source(ui.powlaw1d.pl)\n",
    "\n",
    "print(ui.calc_stat())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "45d1b8ae-9c60-45b8-badc-3d25c4a777e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset               = 1\n",
      "Method                = levmar\n",
      "Statistic             = chi2\n",
      "Initial fit statistic = 2.48264e+08\n",
      "Final fit statistic   = 84.7331 at function evaluation 61\n",
      "Data points           = 61\n",
      "Degrees of freedom    = 59\n",
      "Probability [Q-value] = 0.0157157\n",
      "Reduced statistic     = 1.43615\n",
      "Change in statistic   = 2.48264e+08\n",
      "   pl.gamma       -0.598396    +/- 0.0207753   \n",
      "   pl.ampl        332.533      +/- 38.2647     \n"
     ]
    }
   ],
   "source": [
    "ui.fit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "aa1292fb-7c9a-447d-803e-45a6a4848ebe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20504 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.plot_fit(xlog=True, ylog=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "667c9d5c-aef8-41cc-96d9-e9f32da7c108",
   "metadata": {},
   "source": [
    "As with the `fit` call, `covar` and `conf` use the \"average\" error bar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b2320d4a-b2f3-4ecd-8b50-b419e095c480",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pl.gamma lower bound:\t-0.02077\n",
      "pl.ampl lower bound:\t-35.0532\n",
      "pl.gamma upper bound:\t0.0201346\n",
      "pl.ampl upper bound:\t40.4317\n",
      "Dataset               = 1\n",
      "Confidence Method     = confidence\n",
      "Iterative Fit Method  = None\n",
      "Fitting Method        = levmar\n",
      "Statistic             = chi2\n",
      "confidence 1-sigma (68.2689%) bounds:\n",
      "   Param            Best-Fit  Lower Bound  Upper Bound\n",
      "   -----            --------  -----------  -----------\n",
      "   pl.gamma        -0.598396     -0.02077    0.0201346\n",
      "   pl.ampl           332.533     -35.0532      40.4317\n"
     ]
    }
   ],
   "source": [
    "ui.conf()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1015787-8bac-4c2e-9038-3b0f51195598",
   "metadata": {},
   "source": [
    "We will use the `confidence` results below, so store them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3f12d1bf-d8cd-4571-9b83-d1fa1462d73d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "datasets    = (1,)\n",
      "methodname  = confidence\n",
      "iterfitname = none\n",
      "fitname     = levmar\n",
      "statname    = chi2\n",
      "sigma       = 1\n",
      "percent     = 68.26894921370858\n",
      "parnames    = ('pl.gamma', 'pl.ampl')\n",
      "parvals     = (-0.5983957573984634, 332.53266628336416)\n",
      "parmins     = (-0.020770017510067684, -35.053209709661644)\n",
      "parmaxes    = (0.020134631729182173, 40.43171458331858)\n",
      "nfits       = 18\n"
     ]
    }
   ],
   "source": [
    "conf = ui.get_conf_results()\n",
    "\n",
    "print(conf)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01fc6b66-72b9-4cbf-84fb-455be43c366f",
   "metadata": {},
   "source": [
    "We can draw a vertical rectangle on a plot to show the gamma range (from `parvals + parmins` to `parvals + parmaxes`, since `parmins < 0`). As I plan to use this more-than once I make a helper routine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f9641ac8-66c7-4d04-b6fa-ab5c96ec71c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "\n",
    "def add_conf_gamma():\n",
    "    \"\"\"Draw a rectangle representing the gamma conf results on the plot.\"\"\"\n",
    "\n",
    "    # Label on the conf result as a rectangle\n",
    "    ymin, ymax = plt.ylim()\n",
    "    xy = (conf.parvals[0] + conf.parmins[0], ymin)\n",
    "    dx = conf.parmaxes[0] - conf.parmins[0]\n",
    "    dy = ymax - ymin\n",
    "    patch = plt.Rectangle(xy, dx, dy, color='k', alpha=0.2)\n",
    "    ax = plt.gca()\n",
    "    ax.add_patch(patch)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5316a58d-b3b0-414f-b9ea-f74d7d823c06",
   "metadata": {},
   "source": [
    "This routine can be used to overplot the `conf` results on a `region-projection` plot, showing that the 1D results (e.g. from `conf`) are slightly smaller than the 2D results (as expected):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3973d17f-7040-4556-9731-8cad73a30585",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 61472 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.reg_proj(pl.gamma, pl.ampl, nloop=[101, 101])\n",
    "add_conf_gamma()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a9b27dd-afb9-44c0-83cf-8564d9ac52ba",
   "metadata": {},
   "source": [
    "So, **all** the above used the average error values when calculating statistic values. The question is, how much different would it be if the fit could use the asymmetric errors?\n",
    "\n",
    "The `resample_data` call allows us to estimate what the asymmetry means for any error estimation. It performs a parametric bootstrap assuming a skewed normal distribution centered on the observed data point with the variance given by the low and high measurement errors. The function simulates a number of realizations of the data and fits each realization with the assumed model to obtain the best fit parameters. It returns the best fit parameters for each realization, and displays the average and standard deviation for each parameter.\n",
    "\n",
    "In the following we explicitly ask for 1000 iterations (although that is the default value) and fix the random-number seed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "dfe66669-a317-45f0-9077-574183bbf9ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pl.gamma : avg = -0.4180195954679258 , std = 0.19844933475231044\n",
      "pl.ampl : avg = 163.0552687577543 , std = 94.50832743941801\n"
     ]
    }
   ],
   "source": [
    "result = ui.resample_data(niter=1000, seed=123)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc6f7ad8-b5d2-466e-8315-e687722b9f7e",
   "metadata": {},
   "source": [
    "The routine returns a dictionary with arrays as values; we can check what the keys are with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f5577f8f-ce02-4d24-9d64-4bb1878dd028",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['samples', 'statistic', 'pl.gamma', 'pl.ampl'])\n"
     ]
    }
   ],
   "source": [
    "print(result.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2470998c-2cbd-47de-86fa-9309e38d5d8c",
   "metadata": {},
   "source": [
    "A trace of the statistic value can be created (at present this is always calculated using the least-squares statistic):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "eb325127-62b6-4fef-a8ec-775d7dad6daa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 50160 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.plot_trace(result['statistic'], name='statistic')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f9e8aa2-a2af-4c17-8215-80b4ccb8fa51",
   "metadata": {},
   "source": [
    "The other fields can be displayed, either with Sherpa commands like `plot_trace`, `plot_scatter`, `plot_cdf`, and `plot_pdf`, or directly with a plotting library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e1c08a8c-95ab-4cea-9ab2-a717d633cead",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 22356 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.plot_scatter(result['pl.gamma'], result['pl.ampl'], xlabel=r'$\\Gamma$', ylabel='ampl')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70f782e9-c851-4432-bf3d-3dcc2f6c1a89",
   "metadata": {},
   "source": [
    "We can compare the fit result from earlier to that from the resampled data. In this particular case we can see that the `conf` result (the gray band) is both \n",
    "\n",
    "- too narrow\n",
    "- and biased\n",
    "\n",
    "due to the assumption of symmetric errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0ed4d923-782b-408b-80f7-c1065a86cc77",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23392 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ui.plot_cdf(result['pl.gamma'], xlabel=r'$\\Gamma$', name=r'$\\Gamma$')\n",
    "add_conf_gamma()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c9c1c69-6841-48f2-94a9-24aa8f0734a6",
   "metadata": {},
   "source": [
    "We can use the parameter distribution to estimate errors. Sherpa has useful routines for this but other modules - such as NumPy - can be used. Here we use the `get_error_estimates` routine to return the median value along with the 1-sigma range calculated from the input distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "c87f24ef-164a-4c95-9a75-7a2b4aa282ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "median = -0.472\n",
      " range = -0.547 - -0.269\n"
     ]
    }
   ],
   "source": [
    "from sherpa.utils import get_error_estimates\n",
    "\n",
    "mval, lval, uval = get_error_estimates(result['pl.gamma'])\n",
    "print(f\"median = {mval:.3f}\")\n",
    "print(f\" range = {lval:.3f} - {uval:.3f}\") "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3075d30c-fffc-4063-a7ae-03a3769e41f5",
   "metadata": {},
   "source": [
    "Or use `quantile` to select a non-standard range:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "9030f422-b9c5-4b5b-b051-b3aefcfd4dca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.5890799324170864\n",
      "-0.10309411994420781\n"
     ]
    }
   ],
   "source": [
    "from sherpa.utils import quantile\n",
    "\n",
    "print(quantile(result['pl.gamma'], 0.05))\n",
    "print(quantile(result['pl.gamma'], 0.95))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
