{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Stock Price Prediction using Echo State Networks\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Time Series Prediction is one important regression task, which can be solved using several Machine Learning techniques.\n",
    "\n",
    "In this notebook, we would like to briefly introduce one very bais example for Time Series prediction:\n",
    "\n",
    "Stock Price Prediction\n",
    "\n",
    "Please note that this notebook is not supposed to be used for serious stock price analysis, and we will not give any\n",
    "warranty that the approach will work in practice.\n",
    "\n",
    "At first, we need to import all required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "plt.rcParams['image.cmap'] = 'jet'\n",
    "plt.rcParams['pdf.fonttype'] = 42\n",
    "plt.rcParams['ps.fonttype'] = 42\n",
    "%matplotlib inline\n",
    "\n",
    "from IPython.display import set_matplotlib_formats\n",
    "set_matplotlib_formats('png', 'pdf')\n",
    "\n",
    "from pyrcn.echo_state_network import ESNRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading and visualizing stock prices\n",
    "\n",
    "A good ressource for stock prices is Yahoo Finance ([https://finance.yahoo.com/](https://finance.yahoo.com/)), where a\n",
    "lot of financial data can be downloaded as csv files.\n",
    "\n",
    "We have already downloaded several stock prices, which are all stored in the directory \"dataset\".\n",
    "\n",
    "Now, we are working with the gold price in USD (Gold Aug 20(GC=F)) between 2000-02-28 and 2020-05-28."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Adj Close</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000-02-28</td>\n",
       "      <td>292.299988</td>\n",
       "      <td>292.500000</td>\n",
       "      <td>292.299988</td>\n",
       "      <td>292.500000</td>\n",
       "      <td>292.500000</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000-02-29</td>\n",
       "      <td>292.399994</td>\n",
       "      <td>292.399994</td>\n",
       "      <td>292.399994</td>\n",
       "      <td>292.399994</td>\n",
       "      <td>292.399994</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000-03-01</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000-03-02</td>\n",
       "      <td>288.200012</td>\n",
       "      <td>291.500000</td>\n",
       "      <td>288.000000</td>\n",
       "      <td>288.200012</td>\n",
       "      <td>288.200012</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000-03-03</td>\n",
       "      <td>288.799988</td>\n",
       "      <td>288.799988</td>\n",
       "      <td>288.799988</td>\n",
       "      <td>288.799988</td>\n",
       "      <td>288.799988</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date        Open        High         Low       Close   Adj Close  \\\n",
       "0  2000-02-28  292.299988  292.500000  292.299988  292.500000  292.500000   \n",
       "1  2000-02-29  292.399994  292.399994  292.399994  292.399994  292.399994   \n",
       "2  2000-03-01  291.500000  291.500000  291.500000  291.500000  291.500000   \n",
       "3  2000-03-02  288.200012  291.500000  288.000000  288.200012  288.200012   \n",
       "4  2000-03-03  288.799988  288.799988  288.799988  288.799988  288.799988   \n",
       "\n",
       "   Volume  \n",
       "0     4.0  \n",
       "1     7.0  \n",
       "2     0.0  \n",
       "3   100.0  \n",
       "4    10.0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_series = pd.read_csv(filepath_or_buffer=\"./dataset/GC=F.csv\")\n",
    "time_series.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analyzing the dataset, we can analyze the structure. The first column is the date, the second column is the open value,\n",
    "e.g. the first value of a day.\n",
    "The third column is the highest value of a specific day, the fourth column the lowest and \"Close\" is the final value of\n",
    "a day. The \"Adj Close\" is a corrected  final value, I DON 'T YET KNOW ANYTHING ABOUT VOLUME!!!\n",
    "\n",
    "Here, we just use the \"Close\" value, the final value of one day. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Close</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2000-02-28</td>\n",
       "      <td>292.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2000-02-29</td>\n",
       "      <td>292.399994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2000-03-01</td>\n",
       "      <td>291.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2000-03-02</td>\n",
       "      <td>288.200012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2000-03-03</td>\n",
       "      <td>288.799988</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date       Close\n",
       "0  2000-02-28  292.500000\n",
       "1  2000-02-29  292.399994\n",
       "2  2000-03-01  291.500000\n",
       "3  2000-03-02  288.200012\n",
       "4  2000-03-03  288.799988"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_series = time_series.drop(columns=[\"Open\", \"High\", \"Low\", \"Adj Close\", \"Volume\"], axis=1)\n",
    "time_series = time_series.dropna()\n",
    "time_series.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As one can see, the stock price trends upwards for the first few thousand days. Then goes down at around $n=\\num{3300}$ before climbing again after around $n = 4900$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Price')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "application/pdf": "[encoded data, 543688 characters]\n",
      "image/png": "[encoded data, 403084 characters]\n",
      "text/plain": [
       "<Figure size 5700x1980 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, ax = plt.subplots(figsize=(4.75, 1.65), dpi=1200)\n",
    "time_series.plot(x='Date', y='Close', ax=ax, legend=False, grid=True)\n",
    "ax.set_xlabel(\"Timestamp\")\n",
    "ax.set_ylabel(\"Price\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We pre-processed the dataset by removing undefined values, namely, weekends and public holidays. The remaining values were normalized to be in a range of $[0 1]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "prices = np.asarray(time_series['Close'])\n",
    "\n",
    "scaler = MinMaxScaler()\n",
    "prices = scaler.fit_transform(X=prices.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define Train/Test lengths:\n",
    "\n",
    "The first 3000 values were used to train the ESN and to optimize the hyper-parameters and the rest for testing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "train_len = 3000\n",
    "future_len = 1\n",
    "future_total = len(prices) - train_len"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Echo State Network preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "esn = ESNRegressor(k_in=1, input_scaling=0.6, spectral_radius=0.9, bias=0.0, ext_bias=False, leakage=1.0,\n",
    "                   reservoir_size=100, k_res=10, reservoir_activation='tanh', bi_directional=False,\n",
    "                   teacher_scaling=1., teacher_shift=0., solver='ridge', beta=1e-8, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training and Prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE:\t3.7788642391830205e-05\n",
      "Test MSE:\t8.067299296986916e-05\n"
     ]
    }
   ],
   "source": [
    "train_in = prices[0:train_len, :]\n",
    "train_out = prices[0+1:train_len+1, :]\n",
    "test_in = prices[0:train_len+future_total - future_len, :]\n",
    "test_out = prices[future_len:train_len+future_total, :]\n",
    "\n",
    "esn.fit(X=train_in, y=train_out.ravel())\n",
    "train_pred = esn.predict(X=train_in)\n",
    "test_pred = esn.predict(X=test_in)\n",
    "\n",
    "train_err = mean_squared_error(y_true=train_out, y_pred=train_pred)\n",
    "test_err = mean_squared_error(y_true=test_out[train_len+1:, :], y_pred=test_pred[train_len+1:])\n",
    "\n",
    "print(\"Train MSE:\\t{0}\".format(train_err))\n",
    "print(\"Test MSE:\\t{0}\".format(test_err))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We see that the ESN even captures the downward trend in the test set, although it has not seen any longer downward movement during the training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2568b140788>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "application/pdf": "[encoded data, 556468 characters]\n",
      "image/png": "[encoded data, 399352 characters]\n",
      "text/plain": [
       "<Figure size 5700x1980 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "prices_pred = np.empty_like(prices)\n",
    "prices_pred[:] = np.nan\n",
    "prices_pred[train_len + future_len:, 0] = test_pred[train_len:]\n",
    "\n",
    "_, ax = plt.subplots(figsize=(4.75, 1.65), dpi=1200)\n",
    "ax.plot(scaler.inverse_transform(test_out), label='Target')\n",
    "ax.plot(scaler.inverse_transform(prices_pred), label='Predicted')\n",
    "ax.set_xlabel(\"Timestamp\")\n",
    "ax.set_ylabel(\"Price\")\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are experts in signal processing. Please notice that we do not give any kind of financial advice here! In the financial world, we are sure that much more effective models for time series prediction exist, which include actual background information."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
