{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyceCREAM\n",
    "\n",
    "Here is a python implementaiton of my accretion disc and emission line lightcurve-fitting code (previously CREAM). This guide briefly covers generating synthetic data and calling a new pycecream object to ingest and fit the accretion disc model (or emission line model) to a set of input light curves. I also demonstrate how to access the output of the pycecream fit. The output includes the fitted light curves, any new light curve data points after merging, fitted response functions and parameter MCMC chain histories for the disc and/or tophat response parameters.\n",
    "\n",
    "Most of these features are used in some form or another from a previous f90 version of this code (CREAM) in the following literature\n",
    "\n",
    "* Grier et al in prep\n",
    "* Grier et al 2018    https://iopscience.iop.org/article/10.3847/1538-4357/aa98dc/pdf\n",
    "* Starkey et al 2017  https://ui.adsabs.harvard.edu/#abs/arXiv:1611.06051\n",
    "* Starkey et al 2016  https://ui.adsabs.harvard.edu/#abs/arXiv:1511.06162\n",
    "\n",
    "Please send questions to ds207@st-andrews.ac.uk. Though I am currently taking a break from academia and may take some time to respond, I will try to do so as soon as possible.\n",
    "\n",
    "\n",
    "## Requirements & Installation\n",
    "\n",
    "Please ensure that you have a fortran compiler installed. I use Gfortran. If you have an alternate (e.g ifort), please indicate the standard command used to call the fortran compiler using the ```fortran_caller``` argument (default is ```fortran_caller = gfortran```).\n",
    "\n",
    "\n",
    "command These are fairly easy to install from macports or wget etc. Also a Python version is required (I am using 3.7 but even 2 should be fine). The it's just...\n",
    "\n",
    "```\n",
    "pip install pycecream\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Section 1: Generate Synthetic Light Curves\n",
    "\n",
    "In this example we generate 4 disk light curves and 2 emission-line light curves modelled as a top-hat with a 20-day lag. The code below generates a list where each index contains an Nx3 numpy array for each light curve. The 3 vertical axis for each light curve are the time, flux and noise respectively (query synthetic_data['echo lightcurves'][0] for an example of the format required when inputting your own light curve data).\n",
    "\n",
    "The example below combines continuum and line light curves and illustrates a case in which you may have two of the same emission line (and so want to fit with the same response function model) but observed from different telescopes that require seperate noise models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import astropy_stark.myfake as mf\n",
    "import matplotlib.pylab as plt\n",
    "\n",
    "'''\n",
    "mf.myfake arguments are\n",
    "\n",
    "wavelengths: enter the wavelengths (-1 indicates an emission line light curve modelled with a top-hat response),\n",
    "\n",
    "snr: set the signal-to-noise relative to light curve rms\n",
    "\n",
    "cadence:set the mean cadence\n",
    "\n",
    "top hat centroid: set the centroid for the top-hat (I think thats what this does but the line lag \n",
    "thing is still newish so Im used to just making continuum light curve)\n",
    "'''\n",
    "\n",
    "\n",
    "synthetic_data = mf.myfake(\n",
    "    [4000.0,5000.0,5000.0,7000.0,-1.0,-1.0],\n",
    "    [50.0,50.0,10.0,50.0,50,10.],\n",
    "    [1.0,1.0,2.0,1.0,1.0,3.0],\n",
    "    thcent = 20.0\n",
    ")\n",
    "\n",
    "'''This recovers the synthetic data'''\n",
    "dat = synthetic_data['echo light curves']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Section 2: Settup and run PyceCREAM\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pycecream path... /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pycecream\n",
      "copying file...\n",
      "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pycecream\n",
      "                   name  type  wavelength            noise model  \\\n",
      "0        continuum 4000  line        -1.0  [var, multiplicative]   \n",
      "0        continuum 5000  line        -1.0  [var, multiplicative]   \n",
      "0    continuum 5000 (b)  line        -1.0  [var, multiplicative]   \n",
      "0        continuum 7000  line        -1.0  [var, multiplicative]   \n",
      "0           test line 1  line        -1.0  [var, multiplicative]   \n",
      "0  test line 1 (shared)  line        -1.0  [var, multiplicative]   \n",
      "\n",
      "  share previous lag temporary file name      mean  standard deviation  \\\n",
      "0              False          line_0.dat  3.800485            0.796293   \n",
      "0              False          line_1.dat  3.590073            0.675132   \n",
      "0              False          line_2.dat  3.593360            0.696788   \n",
      "0              False          line_3.dat  3.277164            0.524617   \n",
      "0              False          line_4.dat -0.000530            1.001517   \n",
      "0               True          line_5.dat  0.003286            1.025419   \n",
      "\n",
      "   tophat centroid  tophat centroid step  tophat width  tophat width step  \n",
      "0              0.0                   5.0           2.0                0.0  \n",
      "0              0.0                   5.1           2.0                0.0  \n",
      "0              0.0                   5.2           2.0                0.0  \n",
      "0              0.0                   5.3           2.0                0.0  \n",
      "0              0.0                   5.4           2.0                0.0  \n",
      "0              0.0                   5.4           2.0                0.0  \n"
     ]
    }
   ],
   "source": [
    "import pycecream\n",
    "\n",
    "#instantiate a pycecream object\n",
    "a = pycecream.pycecream()\n",
    "\n",
    "'''\n",
    "If you use a fortran compiler other than gfortran please indicate here.\n",
    "I just re-enter gfortran here for demonstration purposes even though \n",
    "this is unecassary as gfortran is the default argument.\n",
    "'''\n",
    "a.fortran_caller = 'gfortran'\n",
    "\n",
    "\n",
    "\n",
    "'''Choose an output directory in which to save the results. \n",
    "This will be a new directory that you have not previously created (pycecream will make it automatically).\n",
    "\n",
    "NOTE: Each new cream simulation must have a new name for \"output_directory argument below \n",
    "otherwise an excpetion is raised. This is to prevent accidentally overwriting previous simulations. \n",
    "I might change this in a future version \n",
    "'''\n",
    "a.output_directory = 'fit_synthetic_lightcurves'\n",
    "\n",
    "\n",
    "\n",
    "'''\n",
    "Add each of the light curves in the simulation. \n",
    "In this case we are using the \"dat\" output from the synthetic data above.\n",
    "'''\n",
    "a.add_lc(dat[0], name = 'continuum 4000')\n",
    "a.add_lc(dat[1], name = 'continuum 5000')\n",
    "a.add_lc(dat[2], name = 'continuum 5000 (b)')\n",
    "a.add_lc(dat[3], name = 'continuum 7000')\n",
    "\n",
    "#If adding a line light curve, must indicate using the \"kind\" argument\n",
    "a.add_lc(dat[4],name='test line 1',kind='line')\n",
    "\n",
    "#If we want the same line response function model, set \"share_previous_lag\"=True\n",
    "a.add_lc(dat[5],name='test line 1 (shared)',kind='line',share_previous_lag=True)\n",
    "\n",
    "\n",
    "\n",
    "'''\n",
    "specify the numnber of MCMC iterations. Normally at least several thousand are necessary but shorter numbers \n",
    "can be used just to check everything is working is done here.\n",
    "'''\n",
    "a.N_iterations=40\n",
    "\n",
    "'''\n",
    "specify the step sizes for the fit parameters. \n",
    "Here we are setting the accretion rate step size to vary by ~ 0.1 solar masses per year.\n",
    "'''\n",
    "a.p_accretion_rate_step = 0.1\n",
    "\n",
    "'''\n",
    "Check the input settings are ok prior to running\n",
    "'''\n",
    "print(a.lightcurve_input_params)\n",
    "\n",
    "'''\n",
    "RUN!\n",
    "'''\n",
    "a.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examine the output\n",
    "\n",
    "There are 2 output dataframes.\n",
    "\n",
    "## 1) output_lightcurves = a.get_light_curve_fits():\n",
    "This a dictionary of 3 data frames.\n",
    "\n",
    "    1.1) output_lightcurves['model']: standard time, model, error envelope for each file\n",
    "\n",
    "    1.2) output_lightcurves['merged model'] AS above but with the error bars, vertical and horrizontal scalings applied relative to the reference model. Not sure but I think the reference model defaults to the first occurence of a particular wavelength in the order that it was added in self.add_lc\n",
    "\n",
    "    1.3) output_lightcurves['merged data'] DICTIONARY (since the input data light curves can be different sizes) The same transformations but applied to the input light curve data. useful if using cream only to merge the orriginal light curves from different telescopes to a new scale for further study elsewhere\n",
    "\n",
    "## 2) output_chains = a.get_MCMC_chains(): \n",
    "These are the MCMC chains for each parameter.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cream_lcplot plotting results from... fit_synthetic_lightcurves/simulation_files\n",
      "-15.825983 [3.80048513 3.80048513 3.80048513 3.80048513 3.80048513] 0\n",
      "-15.825983 [3.59007335 3.59007335 3.59007335 3.59007335 3.59007335] 1\n",
      "-15.825983 [3.59336019 3.59336019 3.59336019 3.59336019 3.59336019] 2\n",
      "-15.825983 [3.27716422 3.27716422 3.27716422 3.27716422 3.27716422] 3\n",
      "making posterior plot.... posterior_fit_figures__1.pdf\n",
      "unable to make covariance plot for disc posteriors. Please check at least some of these are set to varyin the fit.\n",
      "fit_synthetic_lightcurves/simulation_files/output_20190406_001/G_plot.pdf\n",
      "Nth  6  Ndisk 1\n",
      "cream_lcplot plotting results from... fit_synthetic_lightcurves/simulation_files/output_20190406_001\n",
      "-15.825983 [3.80048513 3.80048513 3.80048513 3.80048513 3.80048513] 0\n",
      "-15.825983 [3.59007335 3.59007335 3.59007335 3.59007335 3.59007335] 1\n",
      "-15.825983 [3.59336019 3.59336019 3.59336019 3.59336019 3.59336019] 2\n",
      "-15.825983 [3.27716422 3.27716422 3.27716422 3.27716422 3.27716422] 3\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 43648 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 5 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 42548 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cream_lcplot plotting results from... fit_synthetic_lightcurves/simulation_files/output_20190406_001\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40920 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cream_lcplot plotting results from... fit_synthetic_lightcurves/simulation_files/output_20190406_001\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 49800 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 19 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cream_lcplot plotting results from... fit_synthetic_lightcurves/simulation_files/output_20190406_001\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "Get the mcmc chains and output fits. \n",
    "Each of these arguments come with a \"location\" argument where you can point to a \n",
    "previous simulation and recover the outputs. \n",
    "If this is left blank we default to the current simulation\n",
    "'''\n",
    "output_chains = a.get_MCMC_chains(location = None)\n",
    "output_lightcurves = a.get_light_curve_fits(location = None)\n",
    "\n",
    "'''\n",
    "make figures of the fit, posterior, light curves etc. file prefix tells the code where you want to save the output.\n",
    "The figure plotting is somewhat primitive and is a relic of when I still used cream. You may prefer to use your own\n",
    "output figures with the output of the \"get_MCMC_chains\" and \"get_light_curve_fits\" functions above.\n",
    "'''\n",
    "a.plot_results(file_prefix='fit_figures')\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "'''\n",
    "figures can also be made on an indivdual basis with axes objects returned from python plotting functions\n",
    "'''\n",
    "#plot the fitted light curves.\n",
    "a.plot_lightcurves()\n",
    "plt.show()\n",
    "\n",
    "\n",
    "#plot the driving light curve\n",
    "a.plot_driver()\n",
    "plt.show()\n",
    "\n",
    "\n",
    "#plot the parameter trace plots\n",
    "a.plot_trace()\n",
    "plt.show()\n",
    "\n",
    "\n",
    "#plot the covariance parameter plot for the disc parameters\n",
    "a.plot_posterior()\n",
    "plt.show()\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# how to install python 3 environment (skip the netcdf4 line) matplotlib should be ok now\n",
    "# https://salishsea-meopar-docs.readthedocs.io/en/latest/work_env/python3_conda_environment.html\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
