{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Stock Price Prediction using Echo State Networks\n",
    "\n",
    "## Introduction\n",
    "\n",
    "The Mackey-Glass system is essentially the differential equation, where we set the parameters to $\\alpha = 0.2$, $\\beta = 10$, $\\gamma = 0.1$ and the time delay $\\tau = 17$ in  order to have a mildly chaotic attractor. \n",
    "\n",
    "\\begin{align}\n",
    "\\label{eq:MackeyGlass}\n",
    "\\dot{y}(t) = \\alpha y(t-\\tau) / (1 + y(t - \\tau)^{\\beta}) - \\gamma y(t)\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "plt.rcParams['image.cmap'] = 'jet'\n",
    "plt.rcParams['pdf.fonttype'] = 42\n",
    "plt.rcParams['ps.fonttype'] = 42\n",
    "%matplotlib inline\n",
    "\n",
    "from IPython.display import set_matplotlib_formats\n",
    "set_matplotlib_formats('png', 'pdf')\n",
    "\n",
    "from pyrcn.echo_state_network import ESNRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data = np.loadtxt(\"./dataset/MackeyGlass_t17.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first \\num{500} samples are visualized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "application/pdf": "[encoded data, 514740 characters]\n",
      "image/png": "[encoded data, 494536 characters]\n",
      "text/plain": [
       "<Figure size 5700x1980 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, ax = plt.subplots(figsize=(4.75, 1.65), dpi=1200)\n",
    "ax.plot(data[:500])\n",
    "ax.set_xlabel(\"n\")\n",
    "ax.set_ylabel(\"X[n]\")\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standardization -> From here on, we have a numpy array!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data = data / (data.max() - data.min())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define Train/Test lengths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "initLen = 100 # number of time steps during which internal activations are washed-out during training\n",
    "# we consider trainLen including the warming-up period (i.e. internal activations that are washed-out when training)\n",
    "trainLen = initLen + 1900 # number of time steps during which we train the network\n",
    "testLen = 2000 # number of time steps during which we test/run the network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Echo State Network preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "esn = ESNRegressor(k_in=1, input_scaling=1.0, spectral_radius=1.2, bias=0.0, ext_bias=False, leakage=1.0,\n",
    "                   reservoir_size=500, k_res=10, wash_out=0, reservoir_activation='tanh', bi_directional=False,\n",
    "                   teacher_scaling=1., teacher_shift=0., solver='ridge', beta=1e-4, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training and Prediction. Be careful, this can take a longer time!!!\n",
    "\n",
    "The lowest MSE obtained with this settings were \\num{5.97e-06} for the training set and \\num{43.1e-06} for the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Steiner\\AppData\\Roaming\\Python\\Python37\\site-packages\\pyrcn-0.0.2-py3.7.egg\\pyrcn\\echo_state_network.py:116: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE:\t5.9565490400444104e-06\n",
      "Test MSE:\t4.3113749550908095e-05\n"
     ]
    }
   ],
   "source": [
    "train_in = data[None,0:trainLen]\n",
    "train_out = data[None,0+1:trainLen+1]\n",
    "test_in = data[None,trainLen:trainLen+testLen]\n",
    "test_out = data[None,trainLen+1:trainLen+testLen+1]\n",
    "\n",
    "train_in, train_out = train_in.T, train_out.T\n",
    "test_in, test_out = test_in.T, test_out.T\n",
    "\n",
    "esn.fit(X=train_in, y=train_out)\n",
    "train_pred = esn.predict(X=train_in)\n",
    "test_pred = esn.predict(X=test_in)\n",
    "\n",
    "train_err = mean_squared_error(y_true=train_out, y_pred=train_pred)\n",
    "test_err = mean_squared_error(y_true=test_out, y_pred=test_pred)\n",
    "\n",
    "print(\"Train MSE:\\t{0}\".format(train_err))\n",
    "print(\"Test MSE:\\t{0}\".format(test_err))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prediction of the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'X[n]')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "application/pdf": "[encoded data, 527196 characters]\n",
      "image/png": "[encoded data, 918100 characters]\n",
      "text/plain": [
       "<Figure size 5700x1980 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, ax = plt.subplots(figsize=(4.75, 1.65), dpi=1200)\n",
    "ax.plot(train_out)\n",
    "ax.plot(train_pred)\n",
    "ax.set_xlabel(\"n\")\n",
    "ax.set_ylabel(\"X[n]\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prediction of the test set.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'X[n]')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "application/pdf": "[encoded data, 527248 characters]\n",
      "image/png": "[encoded data, 936084 characters]\n",
      "text/plain": [
       "<Figure size 5700x1980 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, ax = plt.subplots(figsize=(4.75, 1.65), dpi=1200)\n",
    "ax.plot(test_out)\n",
    "ax.plot(test_pred)\n",
    "ax.set_xlabel(\"n\")\n",
    "ax.set_ylabel(\"X[n]\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
