{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "Please go through the \"building_strategies\" notebook first before you go through this notebook\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Some Prebuilt Reporting ##\n",
    "\n",
    "Lets first build the strategy described in that notebook, add it to a portfolio and run the portfolio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.530635Z",
     "start_time": "2019-01-29T04:08:15.495645Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", message=\"numpy.dtype size changed\") # ignore pesky warning, see https://github.com/numpy/numpy/pull/432\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import pyqstrat as pq\n",
    "from pyqstrat.examples.build_example_strategy import build_example_strategy\n",
    "\n",
    "\n",
    "strategy = build_example_strategy(lookback_period = 10, num_std = 2)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.782746Z",
     "start_time": "2019-01-29T04:08:16.532947Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "portfolio = pq.Portfolio()\n",
    "portfolio.add_strategy('bb_strategy', strategy)\n",
    "portfolio.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many objects have functions that return pandas dataframes for ease of use.  Any function that returns a dataframe starts with df_ so its easy to tell which dataframes an object returns.\n",
    "\n",
    "Probably the most common function you will use is df_data on the strategy object.  This returns the market data, indicators, signal values and P&L at each market data bar.  The last column, i is the integer index of that bar, and can be used to query data in other dataframes or objects for that strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.817856Z",
     "start_time": "2019-01-29T04:08:16.784758Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>symbol</th>\n",
       "      <th>v</th>\n",
       "      <th>l</th>\n",
       "      <th>h</th>\n",
       "      <th>o</th>\n",
       "      <th>c</th>\n",
       "      <th>lower_band</th>\n",
       "      <th>mid_band</th>\n",
       "      <th>upper_band</th>\n",
       "      <th>bb_signal</th>\n",
       "      <th>unrealized</th>\n",
       "      <th>realized</th>\n",
       "      <th>fee</th>\n",
       "      <th>net_pnl</th>\n",
       "      <th>position</th>\n",
       "      <th>equity</th>\n",
       "      <th>i</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:40:00</th>\n",
       "      <td>BTC</td>\n",
       "      <td>6.785e+05</td>\n",
       "      <td>8371</td>\n",
       "      <td>8450</td>\n",
       "      <td>8450</td>\n",
       "      <td>8403</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1e+06</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:45:00</th>\n",
       "      <td>BTC</td>\n",
       "      <td>3.081e+05</td>\n",
       "      <td>8369</td>\n",
       "      <td>8412</td>\n",
       "      <td>8403</td>\n",
       "      <td>8395</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>2</td>\n",
       "      <td>-72.93</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-72.93</td>\n",
       "      <td>6</td>\n",
       "      <td>9.999e+05</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:50:00</th>\n",
       "      <td>BTC</td>\n",
       "      <td>1.249e+06</td>\n",
       "      <td>8350</td>\n",
       "      <td>8397</td>\n",
       "      <td>8397</td>\n",
       "      <td>8356</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>2</td>\n",
       "      <td>-308.2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-308.2</td>\n",
       "      <td>6</td>\n",
       "      <td>9.997e+05</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:55:00</th>\n",
       "      <td>BTC</td>\n",
       "      <td>3.085e+06</td>\n",
       "      <td>8265</td>\n",
       "      <td>8357</td>\n",
       "      <td>8356</td>\n",
       "      <td>8311</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>2</td>\n",
       "      <td>-578.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-578.9</td>\n",
       "      <td>6</td>\n",
       "      <td>9.994e+05</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 13:00:00</th>\n",
       "      <td>BTC</td>\n",
       "      <td>6.686e+05</td>\n",
       "      <td>8311</td>\n",
       "      <td>8349</td>\n",
       "      <td>8311</td>\n",
       "      <td>8331</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>nan</td>\n",
       "      <td>2</td>\n",
       "      <td>-460</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-460</td>\n",
       "      <td>6</td>\n",
       "      <td>9.995e+05</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    symbol         v    l    h    o    c  lower_band  \\\n",
       "date                                                                   \n",
       "2018-03-23 12:40:00    BTC 6.785e+05 8371 8450 8450 8403         nan   \n",
       "2018-03-23 12:45:00    BTC 3.081e+05 8369 8412 8403 8395         nan   \n",
       "2018-03-23 12:50:00    BTC 1.249e+06 8350 8397 8397 8356         nan   \n",
       "2018-03-23 12:55:00    BTC 3.085e+06 8265 8357 8356 8311         nan   \n",
       "2018-03-23 13:00:00    BTC 6.686e+05 8311 8349 8311 8331         nan   \n",
       "\n",
       "                     mid_band  upper_band  bb_signal  unrealized  realized  \\\n",
       "date                                                                         \n",
       "2018-03-23 12:40:00       nan         nan          2           0         0   \n",
       "2018-03-23 12:45:00       nan         nan          2      -72.93         0   \n",
       "2018-03-23 12:50:00       nan         nan          2      -308.2         0   \n",
       "2018-03-23 12:55:00       nan         nan          2      -578.9         0   \n",
       "2018-03-23 13:00:00       nan         nan          2        -460         0   \n",
       "\n",
       "                     fee  net_pnl  position    equity  i  \n",
       "date                                                      \n",
       "2018-03-23 12:40:00    0        0         0     1e+06  0  \n",
       "2018-03-23 12:45:00    0   -72.93         6 9.999e+05  1  \n",
       "2018-03-23 12:50:00    0   -308.2         6 9.997e+05  2  \n",
       "2018-03-23 12:55:00    0   -578.9         6 9.994e+05  3  \n",
       "2018-03-23 13:00:00    0     -460         6 9.995e+05  4  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.df_data().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also look at just the PNL or just the marketdata by themselves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.845357Z",
     "start_time": "2019-01-29T04:08:16.820829Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unrealized</th>\n",
       "      <th>realized</th>\n",
       "      <th>fee</th>\n",
       "      <th>net_pnl</th>\n",
       "      <th>position</th>\n",
       "      <th>equity</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:40:00</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:45:00</th>\n",
       "      <td>-72.93</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-72.93</td>\n",
       "      <td>6</td>\n",
       "      <td>9.999e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:50:00</th>\n",
       "      <td>-308.2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-308.2</td>\n",
       "      <td>6</td>\n",
       "      <td>9.997e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:55:00</th>\n",
       "      <td>-578.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-578.9</td>\n",
       "      <td>6</td>\n",
       "      <td>9.994e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 13:00:00</th>\n",
       "      <td>-460</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>-460</td>\n",
       "      <td>6</td>\n",
       "      <td>9.995e+05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     unrealized  realized  fee  net_pnl  position    equity\n",
       "date                                                                       \n",
       "2018-03-23 12:40:00           0         0    0        0         0     1e+06\n",
       "2018-03-23 12:45:00      -72.93         0    0   -72.93         6 9.999e+05\n",
       "2018-03-23 12:50:00      -308.2         0    0   -308.2         6 9.997e+05\n",
       "2018-03-23 12:55:00      -578.9         0    0   -578.9         6 9.994e+05\n",
       "2018-03-23 13:00:00        -460         0    0     -460         6 9.995e+05"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.df_pnl().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.866715Z",
     "start_time": "2019-01-29T04:08:16.850355Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>v</th>\n",
       "      <th>l</th>\n",
       "      <th>h</th>\n",
       "      <th>o</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:40:00</th>\n",
       "      <td>6.785e+05</td>\n",
       "      <td>8371</td>\n",
       "      <td>8450</td>\n",
       "      <td>8450</td>\n",
       "      <td>8403</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:45:00</th>\n",
       "      <td>3.081e+05</td>\n",
       "      <td>8369</td>\n",
       "      <td>8412</td>\n",
       "      <td>8403</td>\n",
       "      <td>8395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:50:00</th>\n",
       "      <td>1.249e+06</td>\n",
       "      <td>8350</td>\n",
       "      <td>8397</td>\n",
       "      <td>8397</td>\n",
       "      <td>8356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 12:55:00</th>\n",
       "      <td>3.085e+06</td>\n",
       "      <td>8265</td>\n",
       "      <td>8357</td>\n",
       "      <td>8356</td>\n",
       "      <td>8311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-23 13:00:00</th>\n",
       "      <td>6.686e+05</td>\n",
       "      <td>8311</td>\n",
       "      <td>8349</td>\n",
       "      <td>8311</td>\n",
       "      <td>8331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            v    l    h    o    c\n",
       "date                                             \n",
       "2018-03-23 12:40:00 6.785e+05 8371 8450 8450 8403\n",
       "2018-03-23 12:45:00 3.081e+05 8369 8412 8403 8395\n",
       "2018-03-23 12:50:00 1.249e+06 8350 8397 8397 8356\n",
       "2018-03-23 12:55:00 3.085e+06 8265 8357 8356 8311\n",
       "2018-03-23 13:00:00 6.686e+05 8311 8349 8311 8331"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.marketdata('BTC').df().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at orders and trades that were created during this run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.889415Z",
     "start_time": "2019-01-29T04:08:16.869613Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>symbol</th>\n",
       "      <th>type</th>\n",
       "      <th>date</th>\n",
       "      <th>qty</th>\n",
       "      <th>params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BTC</td>\n",
       "      <td>StopLimitOrder</td>\n",
       "      <td>2018-03-23 12:40:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8403.4, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BTC</td>\n",
       "      <td>MarketOrder</td>\n",
       "      <td>2018-03-23 16:15:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>{}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BTC</td>\n",
       "      <td>StopLimitOrder</td>\n",
       "      <td>2018-03-23 16:50:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8655.58, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BTC</td>\n",
       "      <td>MarketOrder</td>\n",
       "      <td>2018-03-23 18:00:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>{}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BTC</td>\n",
       "      <td>StopLimitOrder</td>\n",
       "      <td>2018-03-23 18:10:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8716.48, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  symbol            type                date  qty  \\\n",
       "0    BTC  StopLimitOrder 2018-03-23 12:40:00    6   \n",
       "1    BTC     MarketOrder 2018-03-23 16:15:00   -6   \n",
       "2    BTC  StopLimitOrder 2018-03-23 16:50:00    6   \n",
       "3    BTC     MarketOrder 2018-03-23 18:00:00   -6   \n",
       "4    BTC  StopLimitOrder 2018-03-23 18:10:00    6   \n",
       "\n",
       "                                           params  \n",
       "0   {'trigger_price': 8403.4, 'limit_price': nan}  \n",
       "1                                              {}  \n",
       "2  {'trigger_price': 8655.58, 'limit_price': nan}  \n",
       "3                                              {}  \n",
       "4  {'trigger_price': 8716.48, 'limit_price': nan}  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.df_orders().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.914201Z",
     "start_time": "2019-01-29T04:08:16.892427Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>symbol</th>\n",
       "      <th>date</th>\n",
       "      <th>qty</th>\n",
       "      <th>price</th>\n",
       "      <th>fee</th>\n",
       "      <th>commission</th>\n",
       "      <th>order_date</th>\n",
       "      <th>order_qty</th>\n",
       "      <th>order_params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>BTC</td>\n",
       "      <td>2018-03-23 12:45:00</td>\n",
       "      <td>6</td>\n",
       "      <td>8408</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2018-03-23 12:40:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8403.4, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>BTC</td>\n",
       "      <td>2018-03-23 16:20:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>8588</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2018-03-23 16:15:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>{}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>BTC</td>\n",
       "      <td>2018-03-23 16:55:00</td>\n",
       "      <td>6</td>\n",
       "      <td>8664</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2018-03-23 16:50:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8655.58, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>BTC</td>\n",
       "      <td>2018-03-23 18:05:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>8609</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2018-03-23 18:00:00</td>\n",
       "      <td>-6</td>\n",
       "      <td>{}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>BTC</td>\n",
       "      <td>2018-03-23 18:15:00</td>\n",
       "      <td>6</td>\n",
       "      <td>8723</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2018-03-23 18:10:00</td>\n",
       "      <td>6</td>\n",
       "      <td>{'trigger_price': 8716.48, 'limit_price': nan}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  symbol                date  qty  price  fee  commission          order_date  \\\n",
       "0    BTC 2018-03-23 12:45:00    6   8408    0           0 2018-03-23 12:40:00   \n",
       "1    BTC 2018-03-23 16:20:00   -6   8588    0           0 2018-03-23 16:15:00   \n",
       "2    BTC 2018-03-23 16:55:00    6   8664    0           0 2018-03-23 16:50:00   \n",
       "3    BTC 2018-03-23 18:05:00   -6   8609    0           0 2018-03-23 18:00:00   \n",
       "4    BTC 2018-03-23 18:15:00    6   8723    0           0 2018-03-23 18:10:00   \n",
       "\n",
       "   order_qty                                    order_params  \n",
       "0          6   {'trigger_price': 8403.4, 'limit_price': nan}  \n",
       "1         -6                                              {}  \n",
       "2          6  {'trigger_price': 8655.58, 'limit_price': nan}  \n",
       "3         -6                                              {}  \n",
       "4          6  {'trigger_price': 8716.48, 'limit_price': nan}  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.df_trades().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-10T21:09:59.531818Z",
     "start_time": "2018-08-10T21:09:59.499471Z"
    }
   },
   "source": [
    "You can also look at the returns at the portfolio level (i.e. summing up several strategies)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.960090Z",
     "start_time": "2019-01-29T04:08:16.917641Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bb_strategy</th>\n",
       "      <th>equity</th>\n",
       "      <th>ret</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-03-23</th>\n",
       "      <td>1.001e+06</td>\n",
       "      <td>1.001e+06</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-24</th>\n",
       "      <td>1.001e+06</td>\n",
       "      <td>1.001e+06</td>\n",
       "      <td>-1.937e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-25</th>\n",
       "      <td>9.979e+05</td>\n",
       "      <td>9.979e+05</td>\n",
       "      <td>-0.002737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-03-26</th>\n",
       "      <td>9.987e+05</td>\n",
       "      <td>9.987e+05</td>\n",
       "      <td>0.0008119</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            bb_strategy    equity        ret\n",
       "date                                        \n",
       "2018-03-23    1.001e+06 1.001e+06        nan\n",
       "2018-03-24    1.001e+06 1.001e+06 -1.937e-05\n",
       "2018-03-25    9.979e+05 9.979e+05  -0.002737\n",
       "2018-03-26    9.987e+05 9.987e+05  0.0008119"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "portfolio.df_returns()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get data as native Python objects as opposed to pandas dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:16.974232Z",
     "start_time": "2019-01-29T04:08:16.964161Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[BTC 2018-03-23 12:45 qty: 6.0 prc: 8407.545 order: BTC 2018-03-23 12:40 qty: 6.0 trigger_prc: 8403.4 limit_prc: nan enter long filled,\n",
       " BTC 2018-03-23 16:20 qty: -6.0 prc: 8588.060000000001 order: BTC 2018-03-23 16:15 qty: -6.0 exit long filled,\n",
       " BTC 2018-03-23 16:55 qty: 6.0 prc: 8663.805 order: BTC 2018-03-23 16:50 qty: 6.0 trigger_prc: 8655.58 limit_prc: nan enter long filled,\n",
       " BTC 2018-03-23 18:05 qty: -6.0 prc: 8608.575 order: BTC 2018-03-23 18:00 qty: -6.0 exit long filled,\n",
       " BTC 2018-03-23 18:15 qty: 6.0 prc: 8723.24 order: BTC 2018-03-23 18:10 qty: 6.0 trigger_prc: 8716.48 limit_prc: nan enter long filled,\n",
       " BTC 2018-03-23 18:45 qty: -6.0 prc: 8583.775 order: BTC 2018-03-23 18:40 qty: -6.0 exit long filled,\n",
       " BTC 2018-03-23 18:50 qty: -6.0 prc: 8587.705 order: BTC 2018-03-23 18:45 qty: -6.0 trigger_prc: 8591.0 limit_prc: nan enter short filled,\n",
       " BTC 2018-03-23 19:20 qty: 6.0 prc: 8620.275 order: BTC 2018-03-23 19:15 qty: 6.0 exit short filled,\n",
       " BTC 2018-03-23 20:55 qty: 6.0 prc: 8624.75 order: BTC 2018-03-23 20:50 qty: 6.0 trigger_prc: 8615.5 limit_prc: nan enter long filled,\n",
       " BTC 2018-03-23 22:30 qty: -6.0 prc: 8629.685 order: BTC 2018-03-23 22:25 qty: -6.0 exit long filled,\n",
       " BTC 2018-03-23 22:35 qty: -6.0 prc: 8642.130000000001 order: BTC 2018-03-23 22:30 qty: -6.0 trigger_prc: 8648.4 limit_prc: nan enter short filled,\n",
       " BTC 2018-03-23 22:40 qty: 6.0 prc: 8676.875 order: BTC 2018-03-23 22:35 qty: 6.0 exit short filled,\n",
       " BTC 2018-03-23 23:30 qty: 6.0 prc: 8728.505000000001 order: BTC 2018-03-23 23:25 qty: 6.0 trigger_prc: 8726.37 limit_prc: nan enter long filled]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "strategy.trades(start_date = '2018-03-23', end_date = '2018-03-24')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding your Own Metrics ##\n",
    "\n",
    "Each strategy may have metrics that you want to measure that are specific to that strategy.  To add these, you can use the Evaluator object which can make things easier.\n",
    "\n",
    "To evaluate a strategy we use the evaluate returns function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:17.051123Z",
     "start_time": "2019-01-29T04:08:16.977835Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gmean</th>\n",
       "      <th>amean</th>\n",
       "      <th>std</th>\n",
       "      <th>shrp</th>\n",
       "      <th>srt</th>\n",
       "      <th>calmar</th>\n",
       "      <th>mar</th>\n",
       "      <th>mdd_pct</th>\n",
       "      <th>mdd_dates</th>\n",
       "      <th>dd_3y_pct</th>\n",
       "      <th>dd_3y_dates</th>\n",
       "      <th>up_dwn</th>\n",
       "      <th>2018</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>-0.1155</td>\n",
       "      <td>-0.0004861</td>\n",
       "      <td>0.001342</td>\n",
       "      <td>-5.75</td>\n",
       "      <td>-6.526</td>\n",
       "      <td>44.44</td>\n",
       "      <td>44.44</td>\n",
       "      <td>-0.002756</td>\n",
       "      <td>2018-03-23/2018-03-25</td>\n",
       "      <td>-0.002756</td>\n",
       "      <td>2018-03-23/2018-03-25</td>\n",
       "      <td>1/2/0.333</td>\n",
       "      <td>-0.1155</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    gmean       amean       std   shrp     srt calmar    mar    mdd_pct  \\\n",
       "  -0.1155  -0.0004861  0.001342  -5.75  -6.526  44.44  44.44  -0.002756   \n",
       "\n",
       "              mdd_dates  dd_3y_pct            dd_3y_dates     up_dwn     2018  \n",
       "  2018-03-23/2018-03-25  -0.002756  2018-03-23/2018-03-25  1/2/0.333  -0.1155  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "strategy.evaluate_returns(plot = False);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we want to add some more metrics to this.  For example, lets say we want to add a metric that looks at how many long trades we had versus short trades.  We can do this using an Evaluator object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:17.062623Z",
     "start_time": "2019-01-29T04:08:17.054015Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Long Trades: 16 Short Trades: 34\n"
     ]
    }
   ],
   "source": [
    "def compute_num_long_trades(trades):\n",
    "    return len([trade for trade in trades if trade.order.reason_code == pq.ReasonCode.ENTER_LONG])\n",
    "\n",
    "def compute_num_short_trades(trades):\n",
    "    return len([trade for trade in trades if trade.order.reason_code == pq.ReasonCode.ENTER_SHORT])\n",
    "\n",
    "evaluator = pq.Evaluator(initial_metrics = {'trades' : strategy.trades()})\n",
    "\n",
    "evaluator.add_metric('num_long_trades', compute_num_long_trades, dependencies = ['trades'])\n",
    "evaluator.add_metric('num_short_trades', compute_num_short_trades, dependencies = ['trades'])\n",
    "\n",
    "evaluator.compute()\n",
    "\n",
    "print('Long Trades: {} Short Trades: {}'.format(evaluator.metric('num_long_trades'), evaluator.metric('num_short_trades')))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-10T21:35:46.005492Z",
     "start_time": "2018-08-10T21:35:46.002539Z"
    }
   },
   "source": [
    "The Evaluator takes care of dependency management so that if you want to compute a metric that relies on other metrics, it will compute the metrics in the right order.\n",
    "\n",
    "Lets compute Maximum Adverse Execution for each trade.  MAE tells you the maximum loss each trade in its lifetime.  It's useful for figuring out where to put trailing stops. For example, if most of your profitable trades had a maximum loss during their life up to 5% but many losing trades had losses of 50% and 60%, it might make sense to place a trailing stop around 6% or 7% so you don't get stopped out of your profitable trades but get out of the losing ones quickly.  See Jaekle and Tomasini, page 66 for details\n",
    "\n",
    "To do this we first have to first pair up entry and exit trades into a single \"round-trip\" trade."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:17.094945Z",
     "start_time": "2019-01-29T04:08:17.065419Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import collections\n",
    "\n",
    "trades = strategy.trades()\n",
    "\n",
    "entry_trades = [trade for trade in trades if trade.order.reason_code == pq.ReasonCode.ENTER_LONG]\n",
    "exit_trades = [trade for trade in trades if trade.order.reason_code == pq.ReasonCode.EXIT_LONG]\n",
    "\n",
    "def compute_mae(entry_trades, exit_trades, marketdata):\n",
    "    \n",
    "    md = marketdata\n",
    "    mae = np.empty(len(entry_trades)) * np.nan\n",
    "    \n",
    "    round_trip_pnl = np.empty(len(entry_trades)) * np.nan\n",
    "\n",
    "    for i, entry in enumerate(entry_trades):\n",
    "        if i == len(exit_trades): break\n",
    "        exit = exit_trades[i]\n",
    "        _round_trip_pnl = entry.qty * (exit.price - entry.price)\n",
    "        running_price = md.c[(md.dates >= entry.date) & (md.dates <= exit.date)]\n",
    "        running_pnl = entry.qty * (running_price - entry.price)\n",
    "        _mae = -1 * min(_round_trip_pnl, np.min(running_pnl))\n",
    "        _mae = _mae / entry.price # Get mae in % terms\n",
    "        _mae = max(0, _mae) # If we have no drawdown for this trade, set it to 0\n",
    "        mae[i] = _mae\n",
    "        round_trip_pnl[i] = _round_trip_pnl / entry.price # Also store round trip pnl for this trade since we will have to plot it\n",
    "    return mae, round_trip_pnl\n",
    "        \n",
    "def get_trades(trades, entry):\n",
    "    rc = [pq.ReasonCode.ENTER_LONG, pq.ReasonCode.ENTER_SHORT] if entry else [pq.ReasonCode.EXIT_LONG, pq.ReasonCode.EXIT_SHORT]\n",
    "    return [trade for trade in trades if trade.order.reason_code in rc]\n",
    "\n",
    "evaluator = pq.Evaluator(initial_metrics = {'trades' : strategy.trades(), 'marketdata' : strategy.marketdata('BTC')})\n",
    "evaluator.add_metric('entry_trades', lambda trades : get_trades(trades, True), dependencies=['trades'])\n",
    "evaluator.add_metric('exit_trades', lambda trades : get_trades(trades, False), dependencies=['trades'])\n",
    "evaluator.add_metric('mae', compute_mae, dependencies=['entry_trades', 'exit_trades', 'marketdata'])\n",
    "\n",
    "evaluator.compute()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could have easily done the same computation without using the Evaluator.  The main advantage of using the Evaluator is that you can reuse other metrics you are dependent on without having to recompute them each time, i.e it provides a local cache of metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-10T22:14:20.705460Z",
     "start_time": "2018-08-10T22:14:20.700666Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "source": [
    "## Plotting ##\n",
    "\n",
    "Now that we have maximum adverse execution, lets plot it.\n",
    "\n",
    "pyqstrat plots are built on top of matplotlib.  Each pyqstrat Plot object contains a list of Subplot objects which in turn each contain a list of the data you want to plot in that subplot.\n",
    "\n",
    "In this case, we want to plot round trip PNL for each trade versus its max drawdown (MAE).  The goal is to find out where to set a stop loss so we keep most of the winning trades while limiting our losses from the biggest losing trades.  Since we want to look at the effect of setting a stop loss on both winning and losing trades, we flip the sign of the round trip pnl on the losing trades and mark them in a different color to show the negative P&L."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:17.495484Z",
     "start_time": "2019-01-29T04:08:17.097324Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 38616 characters]\n",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mae = evaluator.metric('mae')[0]\n",
    "round_trip_pnl = evaluator.metric('mae')[1]\n",
    "\n",
    "# Separate out positive trades from negative trades\n",
    "round_trip_profit = round_trip_pnl[round_trip_pnl >= 0]\n",
    "mae_profit = mae[round_trip_pnl >= 0]\n",
    "\n",
    "round_trip_loss = round_trip_pnl[round_trip_pnl <= 0]\n",
    "mae_loss = mae[round_trip_pnl <= 0]\n",
    "\n",
    "\n",
    "subplot = pq.Subplot([\n",
    "    pq.XYData('Profitable Trade', mae_profit, round_trip_profit, plot_type = 'scatter', marker = '^', marker_color = 'green'),\n",
    "    pq.XYData('Losing Trade', mae_loss, -1 * round_trip_loss, plot_type = 'scatter', marker = 'v', marker_color = 'red')],\n",
    "    horizontal_lines = [pq.HorizontalLine(y = 0, color = 'black')],\n",
    "    vertical_lines = [pq.VerticalLine(x = 0, color = 'black')],\n",
    "    xlabel = 'Drawdown in %', ylabel = 'Profit / Loss in %')\n",
    "\n",
    "plot = pq.Plot([subplot])\n",
    "plot.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like a good place to put a stop loss so we keep most of the winning trades but don't take big losses might be around 3%.  The next step would be to re-test the system with this stop loss inserted.  Lets plot the same data with a line showing the stop loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-01-29T04:08:17.885336Z",
     "start_time": "2019-01-29T04:08:17.497421Z"
    },
    "run_control": {
     "frozen": false,
     "read_only": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 42172 characters]\n",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "subplot = pq.Subplot([\n",
    "    pq.XYData('Profitable Trade', mae_profit, round_trip_profit, plot_type = 'scatter', marker = '^', marker_color = 'green'),\n",
    "    pq.XYData('Losing Trade', mae_loss, -1 * round_trip_loss, plot_type = 'scatter', marker = 'v', marker_color = 'red')],\n",
    "    horizontal_lines = [pq.HorizontalLine(y = 0, color = 'black')],\n",
    "    vertical_lines = [pq.VerticalLine(x = 0, color = 'black'), pq.VerticalLine(name = 'Stop Loss', x = 0.03, color = 'blue')],\n",
    "    xlabel = 'Drawdown in %', ylabel = 'Profit / Loss in %')\n",
    "\n",
    "plot = pq.Plot([subplot])\n",
    "plot.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several different kinds of subplots, including 3d subplots that you can include in the same plot.  See the code of the plot function in the Strategy class or the Evaluator class for eamples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
