{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ca3f462e",
   "metadata": {},
   "source": [
    "# Owl\n",
    "## A Retirement Planning Laboratory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fbb38b5",
   "metadata": {},
   "source": [
    "This package is a retirement modeling framework for exploring the sensitivity of retirement financial decisions. Strictly speaking, it is not a planning tool, but more an environment for exploring *what if* scenarios. It provides different realizations of a financial strategy. One can certainly have a savings plan, but due to the volatility of financial investments, it is impossible to have a certain asset earnings plan. This does not mean one cannot make decisions. These decisions need to be guided with an understanding of the sensitivity of the parameters.This is exactly where this tool fits it. Given your savings and spending desires, it can generate different future realizations of your strategy under different market assumptions, helping to better understand your financial situation.\n",
    "\n",
    "The algorithm in Owl is using the open-source HiGHS linear programming solver.\n",
    "The complete formulation and detailed description of the underlying\n",
    "mathematical model can be found\n",
    "[here](https://raw.githubusercontent.com/mdlacasse/Owl/main/docs/owl.pdf).\n",
    "\n",
    "Copyright &copy; 2024 - Martin-D. Lacasse\n",
    "\n",
    "Disclaimers: *I am not a financial planner. You make your own decisions. This program comes with no guarantee. Use at your own risk.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79fc57cf-fd5a-4972-8112-078282cde9fc",
   "metadata": {},
   "source": [
    "### <span style=\"color: blue\"> README FIRST </span>\n",
    "<span style=\"color:black;background:yellow;font-weight:bold\">\n",
    "Don't make changes directly to this file. Keep it as a working example. Therefore, it is recommended that you build your own case by making a copy of the template files provided where you will be able to enter your own numbers and explore your own assumptions.\n",
    "</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a289fed6-6b3b-4fd6-9943-f620e1334cf9",
   "metadata": {},
   "source": [
    "# Tutorial 1 - Using the case of Jack and Jill"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "355ed178",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "This file is provided as a tutorial to introduce you to Owl.\n",
    "\n",
    "It is assumed that you have some familiarity with using a jupyter notebook or jupyterLab, and some basic programming skills in Python. If not, a simple tutorial can guide you to the basic skills needed.\n",
    "\n",
    "For simulating your own realizations, use the files beginning with *template*. Make a copy and rename them keeping the same extension and give them your own names. Then you'll be able to personalize a case with your own numbers and start experimenting with Owl."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2939c06-b407-48fb-91d7-560cd96d5206",
   "metadata": {},
   "source": [
    "This notebook named *tutorial_1* describes the case of Jack and Jill, a fictitious couple used for demonstration. Before reading this notebook, it is a good idea to run the calculations so that graphs will be displayed, which can ease learning Owl's capabilities. Running all cells is done differently depending on the platform being used. On Jupyter, this is done by selecting \"Restart Kernel and Run All Cells\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f1fd7ca",
   "metadata": {},
   "source": [
    "### Just some Python module bookkeeping\n",
    "This command needs to be at the beginning of every Owl notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75e0b2e6-ce46-4fb6-8001-2e07ece94881",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If not installing owlplanner as a module\n",
    "import sys\n",
    "sys.path.insert(0, \"../src\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7744b67b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import owlplanner as owl"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2d87b7e",
   "metadata": {},
   "source": [
    "## Initializing the life parameters for the realization\n",
    "In order to be able to generate a realization of the future, one must start with providing the year of birth of each spouse(s) and their expected lifespan(s).\n",
    "\n",
    "For selecting your own numbers, there are plenty of longevity predictors on the web. Pick your favorite:\n",
    "\n",
    "https://longevityillustrator.org\n",
    "\n",
    "https://www.livingto100.com/calculator\n",
    "\n",
    "https://www.sunlife.ca/en/tools-and-resources/tools-and-calculators/life-expectancy-calculator/\n",
    "\n",
    "or just Google life expectancy calculator.\n",
    "\n",
    "There are two values needed for couples. Single individuals just enter one value in each list between square brackets `[ ]`. For couples, always keep the same order in the pair of values when entering the data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b58d346-55a3-40f0-91fe-014a150f5b76",
   "metadata": {},
   "source": [
    "Here Jack was born in 1962 and Jill in 1965. Jack hopes to live to 89 years old, while Jill thinks she might reach age 92. Let's give this case a name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "760e4a8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan = owl.Plan(['Jack', 'Jill'], ['1962-01-15', '1965-01-16'], [89, 92], 'jack+jill-spending-69')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bd6a675-fc57-4322-bdbc-ccc1c8bd4ec4",
   "metadata": {},
   "source": [
    "## Specify account balances and spousal beneficiaries\n",
    "For each spouse, savings accounts have three buckets comprising of the total value of:\n",
    "- Individual **taxable** investment or savings accounts, including bank accounts, and CDs - do not include your safety net account which should typically be sufficient for sustaining 6 months of living expenses;\n",
    "- **Tax-deferred** savings accounts, including all IRAs, 401k, 403b, etc.;\n",
    "- **Tax-free** savings accounts, including Roth IRAs and Roth 401k.\n",
    "\n",
    "For married couples, each spouse will have to enter values for each type of savings account, following the same order as the one used before. For single individuals, only one value is needed between square brackets `[ ]`.\n",
    "\n",
    "Most investment accounts have named beneficiaries. The *beneficiary* values specify the fraction of total of assets left to the other spouse at death. For example, a spouse leaving 3/4 of her fortune to her three children and the other part to her partner would have a beneficiary values of `[1/4, 1/4, 1/4]`. The default values are `[1, 1, 1]`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ea4c5dc-f88d-47c5-a1ec-3474d023bf23",
   "metadata": {},
   "source": [
    "Jack has \\\\$90.5 k in his taxable account, \\\\$600.2 k in his 401k, and $50 k in his Roth 401k and \\\\$20.6 k in Roth IRAs. Jill has \\\\$60 k in her savings bank account, \\\\$150 k in a 403b, and \\\\$40.8 k in Roth IRAs in which she contributed over the years. Units for \\\\$ are in thousands by default. You can use the `units` optional keyword to indicate that the numbers are entered in other units, such as dollars (`1`) or millions (`M`). Notice that being in Python, arithmetic can be used while entering numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa9b0109",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setAccountBalances(\n",
    "    taxable=[90.5, 60],\n",
    "    taxDeferred=[600.2, 150],\n",
    "    taxFree=[50 + 20.6, 40.8],\n",
    "    startDate='01-01',\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5ae2c1d-deab-4967-9cfb-5dec38bde12b",
   "metadata": {},
   "source": [
    "By default, the account balances are assumed to be known as of today, and amounts will be back projected to Jan 1st using the assumed return rates. An arbitrary date can be chosen as a reference point in the current year, using the optional parameter startDate. For example, using startDate='01-01' would select January 1st, the beginning of the year. This is useful for reproducibility studies."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "034559d1-3392-42b6-b666-9e858feb1ae4",
   "metadata": {},
   "source": [
    "Jack is anticipated to pass first and leaves everything to his spouse for all his savings account. These beneficiary fractions would be entered as [1, 1, 1], i.e., one for each type of account. Using fractions other than 1 while optimizing for maximum bequest can lead to unanticipated results. This topic is explored in Tutorial 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43047edc-a21f-44bc-a89c-2383f1d6b2a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# No need for the call as these are the default values.\n",
    "# plan.setBeneficiaryFractions([1, 1, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de5a5c3f-bfad-4d95-a8ae-1d4f3d37661d",
   "metadata": {},
   "source": [
    "## There must be a plan for wages, savings, and Roth conversions\n",
    "The most manageable part of retirement planning is the control one has over work income, contributions to savings accounts, Roth conversions, and other big spending items in the near- and mid-term future.\n",
    "In order to execute a realization, one must provide an earning, saving, and optionally a Roth conversion plan for overriding the optimizer. This is done through providing an Excel workbook with one spreadsheet (tab) per spouse with the following information:\n",
    "\n",
    "|year|anticipated wages|other inc|taxable ctrb | 401k ctrb | Roth 401k ctrb | IRA ctrb | Roth IRA ctrb | Roth conv | big-ticket items|\n",
    "|--|--|--|--|--|--|--|--|--|--|\n",
    "|2024 | | | | | | | | | |\n",
    "|2025 | | | | | | | | | |\n",
    "| ... | | | | | | | | | |\n",
    "|20XX | | | | | | | | | |\n",
    "\n",
    "Here, 20XX is the last row which could be the last year based on the life expectancy values provided. Missing years will be filled with empty values. For the columns, *anticipated wages* is the annual amount (gross minus tax-deferred contributions) that you anticipate to receive from employment or other sources (not including dividends from your taxable investment accounts). Note that column names are case sensitive and all of these entries must be in lower case. Best way to start this process is to use the template file provided rightly named *template.xlsx*.\n",
    "\n",
    "For the purpose of this exercise, there is no clear definition of retirement age. There will be a year, however, from which you will stop having anticipated income, or diminished income due to decreasing your work load. This transition can be gradual or sudden. Therefore there is no need to enter a retirement age for the sole purpose of quantifying your financial future.\n",
    "\n",
    "Contributions to your savings accounts are marked as *ctrb*. We use 401k as a term which includes contributions to 403b as well. Contributions to your 401k/403b must also include your employer's contributions, if any. As this file is in Excel, one can use the native calculator to enter a percentage of the anticipated wages for contributions as this can sometimes be easier. Considering a specific example, assume that Jack earns 100k\\\\$ and contributes 5% to his 401k which his employer matches at up to 4%, then Jack's anticipated wages will be $(1-.05)*100000 = 95000$ and his 401k contributions will be $.09/(1 - .05) * 95000 = 9000 $. The reason for using $95000$ in the last equation allows for making cross-reference between the cells, as the number 100k\\\\$ will not appear directly. Another approach could be to use an additional column with for the total salary and derive numbers from there.\n",
    "\n",
    "Roth conversion are specified in the column marked *Roth conv*. Roth conversion are typically performed in the years when the income is lower (and therefore lower tax rates), typically in the bridge years between having a full-time regular salary and collecting social security. This column is provided to override the Roth conversion optimization in Owl. When the solver is given the option `maxConversion='file'`, then these values will be used and no optimization over Roth conversions will be performed. This column is provided for flexibility and allowing comparisons between an optimized solution and your best guess.\n",
    "\n",
    "Finally, *big-ticket items* are used for accounting for the sale or purchase of a house, or any other major expense or money that you would give or receive (e.g., inheritance, or large gifts to or from you). Therefore, the sign (+/-) of entries in this column is important. Positive numbers will be considered in the cash flow for the year and surplus, if any, will be deposited in the taxable savings accounts. Negative numbers will potentially generate additional withdrawals and distributions from retirement accounts. This is the only column that can contain negative numbers: all other column entries should be positive.\n",
    "\n",
    "The tab name for each spreadsheet represents the name of the spouse for reporting yearly transactions affecting the plan. There has to be one tab for each individual and bearing the same name. Therefore, when running your own case, you will need to rename the tabs in the template file to have the same names as those used to create the plan (i.e., *Jack* and *Jill* in this case).\n",
    "\n",
    "Note that the (free) LibreOffice software can be used if you do not have an Excel license, as LibreOffice can read and save `.xlsx` files. Moreover, the native file format from the LibreOffice software can also be read directly. However, you will have to install the `odfpy` package through `conda install conda-forge::odfpy` first."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b849a5f-7925-438d-a5c7-15130de43e09",
   "metadata": {},
   "source": [
    "Jack and Jill have provided their specific information in the file *jack+jill.xlsx*. This worksheet file needs to have two tabs, one named *Jack* and the other named *Jill*. Open this file in Excel and familiarize yourself with its contents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7748bcf1",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.readHFP('../examples/HFP_jack+jill.xlsx')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ea1d7ff",
   "metadata": {},
   "source": [
    "## What are the current and future asset allocations?\n",
    "Each savings account can invest in 4 major classes of assets:\n",
    "- Equity funds tracking the S&P 500 index;\n",
    "- Bond assets tracking the Corporate bonds (Baa) index;\n",
    "- Fixed-income securities represented by the performance of 10-year Treasury notes;\n",
    "- Inflation-indexed securities tracking the urban Consumer Price Index (common assets).\n",
    "\n",
    "The total of percentages in each class of assets for each savings account must add to 100%.\n",
    "\n",
    "You are asked to provide asset allocation ratios for today, and ones for the time at the end of the plan.\n",
    "Values in between will be interpolated using a linear operator by default. This can be useful\n",
    "if you want to shift asset allocation as you age. One can also select an *S* curve for transitioning\n",
    "from the initial value to the final value. This is done by choosing *s-curve* instead of *linear* as the method for interpolation. *S-curve* take two additional parameters, the *center* of the change which defaults to 15 years from now, and the transition *width* which defaults to 5 years. These parameters will induce\n",
    "a gradual change from the initial allocations now, starting to change significantly in 10 years (15 - 5), and converging to final allocations in 20 years (15 + 5).\n",
    "\n",
    "These parameters are optional and would be called as follows\n",
    "```\n",
    "plan.setInterpolationMethod('s-curve', center=10, width=3)\n",
    "```\n",
    "for a transition over a span of 6 years centered in 10 years from now.\n",
    "\n",
    "The default interpolation method is *linear* which can also be (re-)selected through the following call:\n",
    "```\n",
    "plan.setInterpolationMethod('linear')\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eecc3b25-ae80-4e6a-be38-1f3c66a46899",
   "metadata": {},
   "source": [
    "In the example below, Jack's and Jill's asset allocations start in full stock equities in the tax-free account,\n",
    "gradually transitioning to a more conservative portfolio towards the end of their life. Jack and Jill's tax-deferred account stay as a traditional 60/40 for the duration of the realization. Note that asset allocation ratios are entered as percentages, and that these percentages are for each type of savings account, and for each spouse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac78169f",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setInterpolationMethod('s-curve')\n",
    "plan.setAllocationRatios(\n",
    "    'account',\n",
    "    taxable=[[[60, 40, 0, 0], [70, 30, 0, 0]], [[60, 40, 0, 0], [80, 20, 0, 0]]],\n",
    "    taxDeferred=[[[60, 40, 0, 0], [70, 30, 0, 0]], [[60, 40, 0, 0], [70, 30, 0, 0]]],\n",
    "    taxFree=[[[100, 0, 0, 0], [100, 0, 0, 0]], [[50, 50, 0, 0], [60, 40, 0, 0]]],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79299f3e-1b88-4263-9755-082821538509",
   "metadata": {},
   "source": [
    "### Show asset allocations during the time period\n",
    "The allocation of assets can be plotted for the three types of savings accounts (taxable, tax-deferred, and tax-free) for each spouse and for the 4 types of investments: stocks (S&P 500), corporate bonds (Baa), Treasury notes (10-y), and common assets tracking inflation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8afa2e05-50b1-485f-ba6b-a330bf2dac10",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showAllocations()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dbdb068",
   "metadata": {},
   "source": [
    "## What about anticipated fixed income?\n",
    "Pension and social security are fixed income. Pension income can be inflation adjusted as social security benefits are. Numbers to be provided are the predicted annual amount for each spouse and the age of the commencement of benefits. Values are expressed in today's dollars (as do statements from the Social Security Administration).\n",
    "\n",
    "By default, no pension benefits are assumed. This can also be specified explicitly by entering zeros (0) as entries, as in\n",
    "\n",
    "    plan.setPension([0, 0], [65, 65])\n",
    "    \n",
    "For social security, one must provide the predicted monthly Primary Insurance Amount(s) and the starting age(s) at which benefits are anticipated to be received. There are plenty of social security benefit estimators on the web, including the info you can get directly from your own account at the Social Security Administration (ssa.gov). Another interesting calculator can be found at www.opensocialsecurity.com. This calculator allows you to compare different scenarios regarding your commencement age through a sensitivity plot.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43bd9a2a-e2bf-434f-88ac-b0a70dcdd1fd",
   "metadata": {},
   "source": [
    "Here, Jill has an unindexed pension of \\\\$10 k per year. Both Jack and Jill believe they have good genes and decided to take their social security benefits at age 70. The amounts provided (28k\\\\$ and 25k\\\\$) are estimation of the amounts they would receive at Full Retirement Age (FRA)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c6e5ac9",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setPension([0, 10], [65, 65])\n",
    "plan.setSocialSecurity([2333, 2083], [70, 70])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "632049cd-dbd4-49da-89bd-06e011c063fc",
   "metadata": {},
   "source": [
    "## How much net spending is desirable at retirement?\n",
    "For determining the desirable annual net spending in retirement, certified planners will strongly suggest that you've must have already done a cash flow analysis on your yearly spending. After this exercise, you should have a good idea of how much you'll need in retirement. Another approach is to experiment with multiple spending scenarios and see what your current and future savings can sustain under different market conditions. As one nears retirement, both approaches will need to meet at one point.\n",
    "\n",
    "The desired spending defined here is the minimum annual **net** spending income (i.e., after paying federal income tax and Medicare) that one would like to receive starting at her/his \"retirement age\" (we provided a loose definition of the term *retirement age* above). This desired spending must be adjusted for inflation and can follow an additional adjustment called a *smile* profile. A *smile* profile accounts for the fact that your spending capacity will modulate during retirement as you go from the so-called gogo years to the no-go years. A *flat* profile, on the other hand, will keep the same value, which will only be adjusted for inflation. More realistically, the income could also be modulated by the performance of assets, reducing withdrawals in market down years. Some of these strategies (first proposed by Guyton and Klinger) could be implemented in Owl.\n",
    "\n",
    "These profiles are multipliers to a spending amount to be set, or obtained from an optimization. For example, a flat profile for a single individual would be unity ($1$) for the whole duration of the plan. In contrast, a smile profile would start at a value larger than $1$ and then decrease over the years to a value smaller than unity, then to go above unity toward the end of the plan. The actual spending amount will be determined either by optimization (for `maxSpending`) or be provided by the user (for `maxBequest`). The final spending amount is a *basis* multiplied by the profile. The basis and the net spending amount are the same for a flat profile, but will differ for a *smile* profile.\n",
    "\n",
    "A second optional argument can be provided to specify the fraction of spending left to the surviving spouse. The default is 60%.\n",
    "\n",
    "The target and actual net spending values achieved through the realization can be plotted as will see below.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e253e7c4-9767-411a-bc55-e151c6060b80",
   "metadata": {},
   "source": [
    "Jack and Jill believe that their spending profile will follow a *smile* curve rather than a *flat* line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67a8087a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Next line is the default parameters.\n",
    "# plan.setSpendingProfile('flat', 60)\n",
    "\n",
    "plan.setSpendingProfile('smile', 60)\n",
    "plan.showProfile()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11b572a8-2b93-428d-b121-3dc793cedb79",
   "metadata": {},
   "source": [
    "## Specify rates of return and inflation rate\n",
    "Rates of return for each class of assets can be specified, including the rate of inflation.\n",
    "These are important components for  future assumptions.\n",
    "Setting future rates is done with the `setRates()` method from which one can select different\n",
    "sources for determining the rates.\n",
    "Valid choices are *historical*, *histochastic*, *historical average*, *user*, *conservative*, *stochastic*, or *default*.\n",
    "\n",
    "Also note that the S&P 500 rates provided always include dividends, which are assumed to be reinvested, while the rates for inflation are\n",
    "derived from the urban consumer price index.\n",
    "\n",
    "#### Fixed rates\n",
    "For the *optimistic* case, values reported from MorningStar for the next ten years are used. It is selected by using this call:\n",
    "\n",
    "    plan.setRates('optimistic')\n",
    "\n",
    "*Conservative* rates are lower than the last ten years as most analysts are predicting lower rates from the next decade. These values are fixed rates of 6.0% for the S&P 500, 4.0% for the Corporate bonds markets, 3.5% for the 10-year Treasury notes, with an inflation at around 2.8%. Using these conservative rates can be achieved by using the following call:\n",
    "\n",
    "    plan.setRates('conservative')\n",
    "\n",
    "One can also use user-provided annual rates by providing a list of 4 entries in percent as follows:\n",
    "\n",
    "    myrates = [9.6, 4.0, 3.0, 3.8]\n",
    "    plan.setRates('user', values=myrates)\n",
    "    \n",
    "This example would use fixed rates of 9.6%, 4.0%, 3.0%, and 3.8% as average annual returns on S&P 500, corporate bonds, Treasury notes, and common assets, respectively, with an average annual inflation rate of 3.8% for the full duration of the time simulation. Recall that the common assets class consists of investments tracking inflation only. Therefore the last index serves both to track the common asset class and to adjust values for inflation.\n",
    "\n",
    "Finally, rates can also be set to fixed values obtained from an average over a time interval using the *historical average* option. For example, the call\n",
    "\n",
    "    plan.setRates('historical average', 1990, 2020)\n",
    "\n",
    "would set the rates to constant values being the average observed from 1990 to 2020 inclusively. The word 'means' can also be used.\n",
    "\n",
    "#### Variable rates\n",
    "For the *historical*, *histochastic*, and *historical average* options, data from 1928 to the last year are available for experimenting.\n",
    "Ranges chosen smaller than the life horizon of the longest-lived individual will have rate values repeated in cycle. For example,\n",
    "choosing historical data from 1994 to (up and including) 1996 will repeat these three values over the time span of the realization.\n",
    "This would be called as follows:\n",
    "\n",
    "    plan.setRates('historical', 1994, 1996)\n",
    "\n",
    "This case is only provided as an explanatory example, as it would have little practical value.\n",
    "\n",
    "If the upper bound is not provided as the third argument, then the latest data year (i.e., last year) will be assumed by default.\n",
    "If one chooses a historical range starting from 1970, Owl will use the rates of 1970 for this year and 1971 for next, etc. This would be selected as follows:\n",
    "\n",
    "    plan.setRates('historical', 1970)\n",
    "  \n",
    "Due to its particular sequence of rates, the worst-case historical scenario is a retirement starting in 1966. This can be selected as follows:\n",
    "\n",
    "    plan.setRates('historical', 1966)\n",
    "\n",
    "In this case, the  current year will have the same rates as those that happened in 1966, and next year will have those from 1967, and so on. This choice is given for instructional purposes only. No one should make a plan based on the worst-case historical scenario. Nevertheless, it can be informative to test your own case. In practice, however, a success rate larger than 90% over a reasonable set of historical starting years and market assumptions would be acceptable by a large portion of rational thinkers. But this is all a personal choice. We'll cover how to model multiple starting years below.\n",
    "\n",
    "Alternatively, one can choose a *histochastic* approach in which case the rates are determined from the multivariate distribution for the 4 rates in the selected year range.  The computed statistical distribution of the selected range of data is used to generate new random rate values. For example,\n",
    "\n",
    "    plan.setRates('histochastic', 1945)\n",
    "    \n",
    "will analyze the annual rates from 1945 up to last-year and compute means and covariance to generate new data that are statistically representative of the ones observed during this selected time period. The rates randomly generated for the time span can be plotted and examined as we will see below. Similarly,\n",
    "\n",
    "    plan.setRates('histochastic', 1940, 1970)\n",
    "\n",
    "would generate random rates consistent to those observed during the 1940 - 1970 time period.\n",
    "\n",
    "Rates can also be generated stochastically by choosing the *stochastic* option.\n",
    "In its simplest case, mean values, and standard deviations (volatility) need to be provided (in percent) as follows:\n",
    "\n",
    "    my_means = [8, 5, 4, 3]\n",
    "    my_stdev = [17, 8, 8, 2]\n",
    "    p.setRates('stochastic', values=my_means, stdev=my_stdev)\n",
    "    \n",
    "Here, we select 8% return on S&P 500 with 17% volatility, 5% return on corporate bonds and and 4% on T-notes, both with 8% volatility, and 3% inflation with 2% of volatility.\n",
    "\n",
    "Called as shown will assume that there is no correlation between rates of return (in mathematical terms,\n",
    "the correlation matrix defaults to the identity matrix). However, we know that rates are somehow correlated (i.e., when the inflation goes up, returns on bonds tend to go down). To account for this coupling, we\n",
    "can either provide a correlation matrix as here:\n",
    "    \n",
    "    my_means = [8, 5, 4, 3]\n",
    "    my_stdev = [17, 8, 8, 2]\n",
    "    my_corr = [[1, 0.46, 0.06, -.12], [0.46, 1, 0.68, -.27], [0.06, 0.68, 1, -.21], [-.12, -.27, -.21, 1]]\n",
    "    p.setRates('stochastic', values=my_means, stdev=my_stdev, corr=offdiag_corr)\n",
    "\n",
    "or, as the correlation matrix is symmetric, only the 6 off-diagonal elements can be provided and Owl will reconstruct the covariance matrix using these values and the volatility. This is demonstrated here:\n",
    "\n",
    "    my_means = [8, 5, 4, 3]\n",
    "    my_stdev = [17, 8, 8, 2]\n",
    "    offdiag_corr = [.46, .06, -.12, .68, -.27, -.21]\n",
    "    p.setRates('stochastic', values=my_means, stdev=my_stdev, corr=offdiag_corr)\n",
    "\n",
    "In standard matrix notation, the off-diagonal elements of correlation matrix $C$ are $c_{12}, c_{13}, c_{14}, c_{23}, c_{24}, c_{34}$. As $c_{ii} = 1$ and $C$ is symmetric, it can easily be reconstructed.\n",
    "\n",
    "To get an idea of what to expect as correlations, one can use the `getRatesDistributions(from, to)` over a range of\n",
    "years of historical data as follows:\n",
    "\n",
    "    means, stdev, corr, covar = owl.getRatesDistributions(1970, 2019)\n",
    "\n",
    "This call will analyze the data over the year range and report back on the statistical correlations found. These values can then be used for your simulations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f9074e2-84fa-4498-9e91-a67f4d21d9fb",
   "metadata": {},
   "source": [
    "Jill is interested to know if her situation would survive a retirement started in 1969. The following call would model exactly that situation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b53ff01e",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setRates('historical', 1969)\n",
    "# In contract, the following call selects fixed conservative rates.\n",
    "# plan.setRates('conservative')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2785e1c5",
   "metadata": {},
   "source": [
    "#### Show annual rates used for calculations\n",
    "As described above, there are many choices for selecting the rates or return of investments. This graph will display the annual rates used during the time span of this realization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81cec974",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showRates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3955ac2-6ff5-483c-b0e9-f1e8dccd18b0",
   "metadata": {},
   "source": [
    "##### Show historical rate distribution\n",
    "Since Owl has the historical rates available, one can also display their histograms using a simple function call. This is done with `owl.showRatesDistributions()`.\n",
    "\n",
    "Given the standard deviation of each histogram, the risk/benefit between stocks and bonds is clear. Let's look at the rates distribution over the 30-year period running from 1969 to 1999. Notice how high is the average inflation. The symbol '<>' means everage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73defc19-8df4-48cc-b27e-4320a49fa07f",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showRatesDistributions(1969, 1999)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66f25114",
   "metadata": {},
   "source": [
    "When *stochastic* or *histochastic* rates are used, the distributions and correlations betwen the different rates can be displayed by using the following method, (which can optionally share the same percentage range across all investments). The diagonal represents a histogram of observed values, the upper diagonal the values themselves, and the lower diagonal a representation of the kernel distribution estimates (KDEs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3d2ee89",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showRatesCorrelations(shareRange=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29544a7d-4b18-4fae-a746-93a4b6c170b3",
   "metadata": {},
   "source": [
    "### Other configurable parameters\n",
    "There are a few more parameters that can be configured. The following calls are provided as examples. If not specified, the default values are:\n",
    "- Heirs marginal tax rate: 30%\n",
    "- Dividend return rate on equities: 2%\n",
    "These are provided for reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce0b2cde-9806-4d47-ab12-81597dbdb381",
   "metadata": {},
   "outputs": [],
   "source": [
    "# These calls represent the default values.\n",
    "# plan.setHeirsTaxRate(30)\n",
    "# plan.setDividendRate(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1c97249",
   "metadata": {},
   "source": [
    "## Generating the outcome of a scenario\n",
    "We're now ready to run a single instance of a scenario. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae05b3e9-3d36-4254-bd70-6d3db25f5a54",
   "metadata": {},
   "source": [
    "### Optimizing for maximum net spending\n",
    "Recall that we set the rates above to mimic those from and after 1969. For this purpose, we only need to run a single realization looking at what would happen to Jack and Jill if the future rates were the same as those which took place from 1969 and after. So let's run this plan: The function `solve` runs all the required calculations for the time horizon over Jack and Jill's life expectancy. It will optimize the selected scenario using linear programming. But we first need to select what needs to be optimized by providing one of the parameters `maxSpending` or `maxBequest`.\n",
    "\n",
    "Let's first maximize the net spending subject to leaving a bequest of 500 k\\\\$ and limiting Roth conversions to be less than 150k\\\\$ if needed. If `maxRothConversion` is set to 'file', only conversions in the contributions worksheet file will be performed. Some plans do not allow for Roth conversions. Let's assume Jill's 403b plan does not allow Roth conversions. We will specify `'noRothConversions': 'Jill'` to indicate not to convert for Jill. For cases where none of the individuals can make Roth conversions, just set `maxRothConversion` to `0`.\n",
    "\n",
    "We use `%%time` keyword to report the computer time used to solve this case. It is typically a few seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daf25c7b-f24c-4616-8678-649fc99b53cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "options = {'maxRothConversion': 150, 'bequest': 500, 'noRothConversions': 'Jill'}\n",
    "plan.solve('maxSpending', options=options)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1abc2ec3-2980-41aa-b58d-5550f28ce047",
   "metadata": {},
   "source": [
    "## Saving and retrieving parameters to/from a configuration file\n",
    "All parameters selected can be stored into a configuration file using the `saveConfig()` method. Note that any existing file will be overwritten. This configuration file is readable using a text editor. The name of the file will be the name of the plan with the `.cfg` extension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e02d9e90-50d5-43aa-a424-453aaa2523ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.saveConfig()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6621f075-e948-4cbf-ab68-6ebd617ce8df",
   "metadata": {},
   "source": [
    "This configuration can be read back into the future. This is done using the `readConfig()` function which returns a new plan configured in a way identical to the one saved. To solve this new plan with the same options as the original plan, one uses the function `resolve()`. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c01b3a2-842b-480e-bf65-16f7daa0558f",
   "metadata": {},
   "outputs": [],
   "source": [
    "newplan = owl.readConfig('../examples/case_jack+jill')\n",
    "newplan.resolve()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "466e9f44-ac61-40af-b722-9295d0edea05",
   "metadata": {},
   "source": [
    "At this point, both `plan` and `newplan` are identical plans (twins)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb8f5600",
   "metadata": {},
   "source": [
    "## Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ae26e4d",
   "metadata": {},
   "source": [
    "To get some information about the plan at the last year of the realization, one can use\n",
    "\n",
    "      plan.summary()\n",
    "    \n",
    "which returns the value of the assets in today's \\\\$ at the last year of the scenario, assuming (in this case) a 30\\% tax burden on the taxable portion of the bequeathed estate (read tax-deferred savings accounts). The `summary()` function returns multiple values. These include the total post-tax value of all savings account in today's dollars, the cumulative inflation rate between today and the last day of the realization and much more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eedb9d78",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d961a23-c10c-490b-9b6f-309478c52b68",
   "metadata": {},
   "source": [
    "##### Selecting preferred graphs\n",
    "By default, all graphs are shown in nominal dollars. To change the default behavior of all graphs, the `setDefaultPlots()` method can be called with either *nominal* or *today* as an argument. The default behavior can also be overridden using the `value` argument for each individual call. For example\n",
    "```\n",
    "plan.showGrossIncome(value='today')\n",
    "plan.showNetSpending(value='nominal')\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c87864e-c6af-4a2e-9003-632624ed4bb5",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.setDefaultPlots('today')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bfe0dbd",
   "metadata": {},
   "source": [
    "#### Show net spending compared to target spending over the years\n",
    "This graph shows how the actual net spending generated by the plan realization matches the inflation-adjusted net spending profile specified. Note the 40% drop in target spending as one spouse passes. This drop is configurable through the `setProfile()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6dffb3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showNetSpending()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8123cbc2",
   "metadata": {},
   "source": [
    "#### Show taxable annual ordinary income and anticipated tax brackets\n",
    "Taxable ordinary includes Roth conversions which are not contributing to your net income. This graph shows Jack and Jill's gross taxable income and how it compare with some anticipated federal tax marginal brackets. This visualization is very convenient when one wants to perform Roth conversions and remain below a certain tax bracket.\n",
    "\n",
    "Notice the shift in tax brackets occurring as the Tax Cut and Job Act might expire after 2025."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e305502",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showGrossIncome()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fad7e21",
   "metadata": {},
   "source": [
    "#### Show annual taxes paid over the years of the realization\n",
    "This graph shows how much Jack and Jill paid in federal taxes and IRMAA income-related Medicare insurance monthly adjustments over the years of this realization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a95dad48",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showTaxes()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "460f5547",
   "metadata": {},
   "source": [
    "#### Show sources of income over the years\n",
    "Income will typically come from multiple sources, and it can be quite complex. This graphs shows the breakdown of Jack and Jill's sources of income by spouse and by origin. Notice that additional distributions from tax-deferred accounts (*dist*) are distinguished from required minimum distributions (*rmd*) as they serve different purposes. Other labels should be self-explanatory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f74ec825",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showSources()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d997ff3",
   "metadata": {},
   "source": [
    "#### Show savings accounts balances at the beginning of each year\n",
    "The balance for each savings account for each spouse is calculated at the beginning of each year. Another important aspect of this graph is how much is left at the end of the realization. This depends on the optimization requested (`maxSpending` or `maxBequest`) and the constraint on the `bequest`. In the current case, a bequest of 500 k\\\\$ in today's \\\\$ was requested, which amounts to 2,492 k\\\\$ in nominal value. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b9a9e62",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showAccounts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96d1dd05",
   "metadata": {},
   "source": [
    "#### Show asset distribution\n",
    "The distribution of assets in each savings account can be shown by the following command. It shows the value and the class of assets for each savings account over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3975fc2d-714f-4b6b-9801-77a3b7054746",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.showAssetDistribution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a74f614-f7ce-4543-b0d1-e7cfc4101b3d",
   "metadata": {},
   "source": [
    "## Saving the yearly details of this realization in a spreadsheet\n",
    "This instance of a future realization contains information on the yearly distribution amounts, including the required minimum distribution that had to be performed under the given assumptions. This info can be saved in an excel workbook with one spreadsheet (tab) for each spouse. Worksheet will also contain annual rates, income, income taxes, and account balances. The argument *overwrite* as `True/False` controls if existing file, if any, gets overwritten. Also remember that Windows will not allow the file to be overwritten while the file is being opened in Excel. In that case, the script will ask you to close the file and will retry to save.\n",
    "\n",
    "All values in the workbook are in nominal dollars. The Excel file will be named `workbook_...` where `...` is the name of the plan."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9231c810-31db-45aa-98f1-6632f717d375",
   "metadata": {},
   "source": [
    "Here, this call will create an excel workbook with multiple spreadsheets (tab) representing the data for Jack and Jill. \n",
    "\n",
    "Open the file in Excel to see what it looks like. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e25fdb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan.saveWorkbook(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70386bcb-b523-4034-bfcd-d775c69fe8a7",
   "metadata": {},
   "source": [
    "## Optimizing for maximum bequest\n",
    "Instead of optimizing for Jack and Jill's net spending, one could optimize for the maximum bequest subject to a net spending of 90k\\\\$ per year for the first year. Note that the net spending basis might be different if a *smile* profile is used, as the first year spending amount is higher than in subsequent years.\n",
    "\n",
    "To perform this additional calculation, we could reuse the same plan and just change its name using the `rename()` method. This approach can cause unnecessary confusion if cells are run out of order. We will first discuss how to make copies of a plan."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65c19897-6968-480e-a443-54f2ed0aa703",
   "metadata": {},
   "source": [
    "#### Making a copy of a plan\n",
    "\n",
    "Preferably, we could clone our original plan and provide a new name to it using the `owl.clone()` function as in the following code.\n",
    "\n",
    "```\n",
    " plan2 = owl.clone(plan, 'jack+jill-bequest-1969')\n",
    "```\n",
    "This would create a new plan identical to `plan` but with the new name *jack+jill-bequest-1969*.\n",
    "\n",
    "To make this tutorial more complete, however, we will instead create a new plan configured by reading the configuration file that was saved earlier in this notebook using the `saveConfig()` function. Reading a configuration file is achieved using the `owl.readConfig()`. We would then rename the plan after configuration is read.\n",
    "\n",
    "Here is how this would be done:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "220c58a1-a951-4484-83b9-d51381bfbb2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2 = owl.readConfig('../examples/case_jack+jill')\n",
    "plan2.rename('jack+jill-bequest')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73db9caf-7ca2-4754-b2a3-61034ccbb044",
   "metadata": {},
   "source": [
    "#### Solving for maximum bequest\n",
    "We are now ready to solve for maximum bequest, subject to a netSpending of \\\\$90k in the first year, maximum Roth conversion of \\\\$150k for Jack and no Roth conversions for Jill. The value for *netSpending* is the amount in this year's \\\\$ for the first year of the plan. It is the product of the basis for the net spending amount times the profile factor for the current year. They are the same for a flat profile, but different when using a *smile* profile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20163487",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "options = {'maxRothConversion': 150, 'netSpending': 90, 'noRothConversions': 'Jill'}\n",
    "plan2.solve('maxBequest', options=options)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2347b6eb-da42-41a4-935e-78184f62db1c",
   "metadata": {},
   "source": [
    "### Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77a30382-cd0d-4e5d-b593-6d25faba41e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d103f90-220f-487a-8b78-28f8b076cce8",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showNetSpending()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d94cef1-b510-4dc6-a230-7ac81c02ca1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showGrossIncome()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "236cbeca-db62-427a-9556-d86203fe9279",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showTaxes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83b5687d-b21e-4766-95a7-9ef40bba404e",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showSources()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10124314-e0e7-4f37-a297-9a5011fb4fa1",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showAccounts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a196d2e7-402f-4dfa-936d-9fb5d8e08b5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.showAssetDistribution()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "339bfc58-5d4c-4d19-9f0c-7a3f715964fa",
   "metadata": {},
   "source": [
    "Notice how assets get shifted to the tax-free account towards the end of the plan. Running a plan with a different assumption for the heirs tax rate will give a different outcome."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfc37583-6d22-4c16-81a3-8128cb430469",
   "metadata": {},
   "outputs": [],
   "source": [
    "plan2.saveWorkbook(overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79c42562-575a-4c7b-8b75-ebc61f053562",
   "metadata": {},
   "source": [
    "This notebook is provided as a basic example on what you can do to assess the robustness and sensitivity of your retirement strategy. Go and explore!\n",
    "\n",
    "\n",
    "Enjoy!\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
