{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "05fad639-5e11-4b6c-8911-0707768ac8ad",
   "metadata": {},
   "source": [
    "# Owl\n",
    "## A Retirement Planning Laboratory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e98ec63-fce8-4098-8179-2d597bfb4c2e",
   "metadata": {},
   "source": [
    "This package is a retirement modeling framework for exploring the sensitivity of retirement financial decisions. Strictly speaking, it is not a planning tool, but more an environment for exploring *what if* scenarios. It provides different realizations of a financial strategy. One can certainly have a savings plan, but due to the volatility of financial investments, it is impossible to have a certain asset earnings plan. This does not mean one cannot make decisions. These decisions need to be guided with an understanding of the sensitivity of the parameters.This is exactly where this tool fits it. Given your savings and spending desires, it can generate different future realizations of your strategy under different market assumptions, helping to better understand your financial situation.\n",
    "\n",
    "The algorithm in Owl is using the open-source HiGHS linear programming solver.\n",
    "The complete formulation and detailed description of the underlying\n",
    "mathematical model can be found\n",
    "[here](https://raw.githubusercontent.com/mdlacasse/Owl/main/docs/owl.pdf).\n",
    "\n",
    "Copyright &copy; 2024 - Martin-D. Lacasse\n",
    "\n",
    "Disclaimers: *I am not a financial planner. You make your own decisions. This program comes with no guarantee. Use at your own risk.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01e6c7da-cd31-41de-9b67-e493ae77dea8",
   "metadata": {},
   "source": [
    "### <span style=\"color: blue\"> README FIRST </span>\n",
    "<span style=\"color:black;background:yellow;font-weight:bold\">\n",
    "Don't make changes directly to this file. Keep it as a working example. Therefore, it is recommended that you build your own case by making a copy of the template files provided where you will be able to enter your own numbers and explore your own assumptions.\n",
    "</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f65fbb58-4975-4a0a-beae-bff23f484b66",
   "metadata": {},
   "source": [
    "# Tutorial 2 - Using the case of Jack and Jill\n",
    "This tutorial shows how to use advanced capabilities of Owl such as Monte Carlo simulations and running simulations over a historical range. We use the same case as the one in Tutorial 1. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6df2872d-93b6-47cf-9a7d-667c44901054",
   "metadata": {},
   "source": [
    "### Just some Python module bookkeeping\n",
    "This command needs to be at the beginning of every Owl notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a46ff04-7c86-4569-84e3-e533a0e9bb47",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If not installing owlplanner as a module\n",
    "import sys\n",
    "sys.path.insert(0, \"../src\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63c9604f-4e6e-4e72-93d7-cdb5788d81d5",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import owlplanner as owl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1033e24-bfab-47d1-9083-e8cf739b086f",
   "metadata": {},
   "source": [
    "### Creating the plan (see Tutorial 1 for details)\n",
    "We first create a plan as we did in the *Tutorial 1*. We removed the comments for reducing the clutter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39121d7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan = owl.Plan(['Jack', 'Jill'], ['1962-01-15', '1965-01-16'], [89, 92], 'jack+jill-spending-MC', verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee55774b-6f33-49fa-ae5c-85d2d466e026",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setAccountBalances(\n",
    "    taxable=[90.5, 60],\n",
    "    taxDeferred=[600.2, 150],\n",
    "    taxFree=[50 + 20.6, 40.8],\n",
    "    startDate='01-01',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e1ac651-ef61-47c1-9d2e-c47a0475e807",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.readHFP('../examples/HFP_jack+jill.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "747f1a39-501e-4560-b856-b838720289fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setInterpolationMethod('s-curve')\n",
    "plan.setAllocationRatios(\n",
    "    'individual',\n",
    "    generic=[[[60, 40, 0, 0], [70, 30, 0, 0]], [[60, 40, 0, 0], [70, 30, 0, 0]]],\n",
    ")\n",
    "plan.showAllocations()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "435a0599-accb-42cd-a1c2-382b97383626",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setPension([0, 10], [65, 65])\n",
    "plan.setSocialSecurity([2333, 2083], [70, 70])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b622cd0-f176-4a2f-8470-a0b457294e87",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setSpendingProfile('smile', 60)\n",
    "plan.showProfile()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec690bc0-cfba-4b4d-a11d-a8fd740346c1",
   "metadata": {},
   "source": [
    "### Running Monte Carlo simulations\n",
    "Before running Monte Carlo simulations, the user has to select one of two rate-generating stochastic methods."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee6a032a-7a62-4c88-9e78-a94754265440",
   "metadata": {},
   "source": [
    "#### Selecting stochastic rates\n",
    "\n",
    "##### The *histochastic* method\n",
    "In Owl, there are two mechanisms that can be used to generate random series of return rates for performing statistical analysis. The first one is to pick a range of historical returns and to derive statistics out of these values. This is achieved by using what I coined the *histochastic* method that is called as follows:\n",
    "\n",
    "    plan.setRates('histochastic', 1970, 2023)\n",
    "\n",
    "As called, this method will extract the statistical features of annual returns which happened from 1970 to 2023 inclusively. Using this approach, the means and covariances are all set under the hood, and the user has nothing more to specify.  \n",
    "The means and covariance matrix calculated from this time series are then used to generate a series with similar characteristics. Note that the mathematical model used assumes that the rates follow a normal distribution which is a known incorrect approximation. This implies that the distribution of events in the tails might be slightly incorrect but for the time being, there is no known distribution that will capture the tails more accurately. The mean values however, are more robust to our choice of distribution, due to a averaging effect known as the central limit theorem. \n",
    "\n",
    "##### The *stochastic* method\n",
    "The means and covariance matrix can also be explicitly specified by the user. Let's look at a specific example:\n",
    "\n",
    "    my_means = [8, 5, 4, 3]\n",
    "    my_stdev = [17, 8, 8, 2]\n",
    "    # my_corr = [[1, 0.46, 0.06, -.12], [0.46, 1, 0.68, -.27], [0.06, 0.68, 1, -.21], [-.12, -.27, -.21, 1]]\n",
    "    offdiag_corr = [.46, .06, -.12, .68, -.27, -.21]\n",
    "    plan.setRates('stochastic', values=my_means, stdev=my_stdev, corr=offdiag_corr)\n",
    "    \n",
    "Here, we set the rates to *stochastic* with mean return values of 8% +/- 17% for the S&P 500, 5% +/- 8% for corporate bonds, 4% +/- 8% for T-notes, and 3% +/- 2% for the inflation/common assets. Specifying the correlation matrix is optional but highly recommended. The one we use here was obtained by looking at historical data using the year range of our choice, for example,\n",
    "\n",
    "    owl.getRatesDistributions(1970, 2019)\n",
    "\n",
    "The matrix `my_corr` can be used to specify the correlation matrix, but to make the call less cumbersome, only the off-diagonal elements of the correlation matrix can be specified and Owl will figure it out.\n",
    "\n",
    "We will now enable the *stochastic* method with the values just mentioned:\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "726966bd-be52-464e-81a5-ac1b9323fcec",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Mean returns\n",
    "my_means = [8, 5, 4, 3]\n",
    "# Volatility\n",
    "my_stdev = [17, 8, 8, 2]\n",
    "# Correlations matrix between rates of return of different assets\n",
    "# my_corr = [[1, 0.46, 0.06, -.12], [0.46, 1, 0.68, -.27], [0.06, 0.68, 1, -.21], [-.12, -.27, -.21, 1]]\n",
    "# These are only the off-diagonal elements\n",
    "offdiag_corr = [.46, .06, -.12, .68, -.27, -.21]\n",
    "plan.setRates('stochastic', values=my_means, stdev=my_stdev, corr=offdiag_corr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cce791e5-1dd6-4b4b-9997-cb8430900034",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display a single instance resulting from this choice.\n",
    "plan.showRatesCorrelations(shareRange=False)\n",
    "plan.showRates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eacb5bc8-aca8-481a-9991-6aa494b493a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setHeirsTaxRate(33)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec99f081-a999-4452-87f2-572c85158643",
   "metadata": {},
   "source": [
    "### Preparing the case to run\n",
    "In order to run Monte Carlo simulations, one has first to set the rates to either *stochastic* or 'histochastic*. Then a call to \n",
    "\n",
    "    runMC(objective, options, N)\n",
    "    \n",
    "specifying the objective and options will generate a desired number of simulations and provide statistics. The last argument $N$ is the number of cases to consider. Unlike event-driven simulator, Owl performs an optimization for each case, which adds considerable computing costs. For this reason, the default behavior of Owl is to turn off Medicare calculations for these runs. It can be turned back on using 'withMedicare' to `True` in the options, but this will significantly increase (about 3X) the computing time while providing little more insight. These simulations are performed with no verbosity as many simulations are performed, but `runMC` accepts the `verbose=True` argument for debugging or curiosity purposes.\n",
    "\n",
    "Running 500 cases takes about 5 min to complete. Be patient. At the end of the run, a histogram of the optimized values is displayed.\n",
    "When running $N$ cases, the error on the estimate for the mean value being estimated decreases as $1/\\sqrt{N - 1}$. Therefore, 500 cases are more than sufficient to obtain reliable and meaningful estimates of mean statistics.\n",
    "\n",
    "Jack and Jill desire to leave a bequest of \\\\$500k (in today's \\\\$). \n",
    "We limit Roth conversions to a maximum of \\\\$150k for Jack and none for Jill.\n",
    "\n",
    "What are these cases telling us?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "856cdfb8-0472-475f-afa9-5dae92f1a52d",
   "metadata": {},
   "outputs": [],
   "source": [
    "options = {'maxRothConversion': 150, 'noRothConversions': 'Jill'}\n",
    "# options['solver'] = 'MOSEK'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ede7fa05-8a6e-41b4-8b7c-a217d28ad41b",
   "metadata": {},
   "outputs": [],
   "source": [
    "options['bequest'] = 500\n",
    "plan.runMC('maxSpending', options, 500);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8553d877-c44c-43f0-a015-9055f7d76e25",
   "metadata": {},
   "source": [
    "All cases are successful at leaving a \\\\$500k bequest, but the net spending required for achieving this constraint can lead to small values in some scenarios. However, the median value for the net spending is about \\\\$88k, very close to the \\\\$90k desired by Jack and Jill. Recall that we are using a more conservative return of 8%, with a volatility of 17% for the S&P 500. It is therefore natural that the number we find here is slightly lower than what the historical return of the market has been able to predict. Some of these scenarios will provide more than the median (\\$M\\$), but some others will require a net spending of about \\\\$50k. All these scenarios make a probability of 100\\% for leaving a \\\\$500k bequest as requested. The mean value is indicated by $\\bar{x}$.\n",
    "\n",
    "A more practical example is to look at the maximum bequest left under the constraint of a fixed net spending."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e63e1b0-480e-4065-bb73-a5d7561cb328",
   "metadata": {},
   "outputs": [],
   "source": [
    "options['netSpending'] = 90\n",
    "plan.runMC('maxBequest', options, 500);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd6356bb-7901-4316-80d9-2ab5e0f531a8",
   "metadata": {},
   "source": [
    "In this case, a relatively large percentage of scenarios are successful in providing the desired \\\\$90k net spending amount over the duration of the plan. The median bequest left at the end of the plan is close to \\\\$1 M, twice what Jack and Jill are intending to leave. This might first suggest that the amount of \\\\$90k might be too conservative on average. The success rate is 85%, which is relatively good, especially considering the fact that Jack and Jill are always able to re-adjust their net spending plan as other forecast estimates are performed in the future."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c706722-4557-4537-94ec-4d13b29e2f61",
   "metadata": {},
   "source": [
    "### Running cases over a historical range or years\n",
    "We can also run cases using the historical data over a range of years and compile the outcomes in a histogram. For this purpose, there is no need to pre-select specific rates using the `setRates()` method as the choice of *historical* rates is implicit. Only the year range is required.\n",
    "\n",
    "Let's look at a specific example. For optimizing the net spending amount, we specify *maxSpending* in the following call to optimize the plan over the 63 years following 1928:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a9be559-2f2e-4326-adef-18af5c14aeb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.runHistoricalRange('maxSpending', options, 1928, 1990);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bccce068-c9a0-4001-a42d-868bde9d481b",
   "metadata": {},
   "source": [
    "When considering more historical data than a single year, we find a slightly larger net spending value than the one dictated from when only looking at the year 1969. This is not surprising as 1969 was not a good year to retire due to the high inflation rates that prevailed during the following decade. \n",
    "\n",
    "Let's now consider the case for *maxBequest* under a desired net spending."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6471720b-4704-4850-a341-1d9a0e5e58e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.runHistoricalRange('maxBequest', options, 1928, 1990);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fa4f26a-0ea2-4804-8013-84b4b24a98e2",
   "metadata": {},
   "source": [
    "The resulting success rate when considering all historical data is 100%. This is also not surprising as the original estimates for the net spending and the bequest were derived from running a case in 1969, the second next worst year of historical sequences of returns. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
