{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to Seeq Data Lab!\n",
    "Seeq Data Lab is a rich Python-based environment for exploring and manipulating time series and industrial process data. It is built on the [Jupyter](https://jupyter.org/) interactive Python platform and [Pandas](https://pandas.pydata.org/) data structures, making extensive use of the features and conventions that their user communities have established.\n",
    "\n",
    "Any datasources that are connected to your Seeq Server are easy to access via Seeq Data Lab's `spy` library. Several simple function calls like `search`, `pull` and `push` consume and/or produce [DataFrames](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html) whenever possible so that you can use their powerful data manipulation capabilities and concise syntax.\n",
    "\n",
    "This tutorial walks you through the basics of searching for data, getting it into a DataFrame and then writing it back to a Seeq Workbench workbook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started\n",
    "This section will walk you through searching for signals in Seeq and pulling data.\n",
    "\n",
    "## Importing the SPy Module\n",
    "The **SPy** module (short for Seeq PYthon) is what you'll use to interact with Seeq Server and access data. Start by importing it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seeq import spy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll also want to import Pandas so that you can work with the DataFrame structures that the **spy** library consumes/produces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logging in to Seeq\n",
    "\n",
    "**If you are using Seeq Data Lab, then this step is NOT necessary.** If you are using the SPy module with Anaconda, AWS SageMaker, Azure Notebooks or any other Jupyter Notebook solution, you will need to execute this step.\n",
    "\n",
    "Create a file in the root folder of your Jupyter project called `credentials.key` and put your username on the first line and your password on the second.\n",
    "\n",
    "If you log in to Seeq Server using your corporate credentials (aka \"Single Sign-On\"), please take a look at the [spy.login](spy.login.ipynb) documentation for information about logging in with an Access Key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Logged in to <strong>http://localhost:34216/api</strong> successfully as <strong>agent_api_key</strong>.<br>Seeq Server Version: <strong>R21.0.44.04</strong><br>Seeq Python Module Version: <strong>0.44.4.137</strong></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "spy.login(url='http://localhost:34216', credentials_file='../credentials.key', force=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Searching for Signals\n",
    "Now let's find some data. We're going to use the built-in Seeq example data to retrieve a few signals and look at them as DataFrames.\n",
    "\n",
    "First we'll use the `spy.search()` function to retrieve metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Query successful</div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">Path</td><td style=\"background-color: #EEFFEE;\">Time</td><td style=\"background-color: #EEFFEE;\">Count</td></tr><tr style=\"background-color: #EEFFEE;\"><td>0</td><td>Example >> Cooling Tower 1 >> Area A</td><td>00:00:00.08</td><td>6</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Path</th>\n",
       "      <th>Asset</th>\n",
       "      <th>Name</th>\n",
       "      <th>Description</th>\n",
       "      <th>Type</th>\n",
       "      <th>Value Unit Of Measure</th>\n",
       "      <th>Datasource Name</th>\n",
       "      <th>Archived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4A264102-AC77-4CEF-9C5F-0191EB3C7931</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Wet Bulb</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>°F</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3B651CDB-E9CA-4699-BB05-F25138C1779A</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Relative Humidity</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>%</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35E96E0C-7761-4270-97C7-71C4E066305C</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Temperature</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>°F</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>EF7AFDB1-D686-456A-8BC9-99551EA8CA24</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Compressor Power</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>kW</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>802DE760-2A0D-4129-8AEB-6D6A73BDD7B9</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Compressor Stage</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>string</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3EAB376F-B4F6-44B5-A788-82B4B2E86146</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Optimizer</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     ID                        Path   Asset  \\\n",
       "0  4A264102-AC77-4CEF-9C5F-0191EB3C7931  Example >> Cooling Tower 1  Area A   \n",
       "1  3B651CDB-E9CA-4699-BB05-F25138C1779A  Example >> Cooling Tower 1  Area A   \n",
       "2  35E96E0C-7761-4270-97C7-71C4E066305C  Example >> Cooling Tower 1  Area A   \n",
       "3  EF7AFDB1-D686-456A-8BC9-99551EA8CA24  Example >> Cooling Tower 1  Area A   \n",
       "4  802DE760-2A0D-4129-8AEB-6D6A73BDD7B9  Example >> Cooling Tower 1  Area A   \n",
       "5  3EAB376F-B4F6-44B5-A788-82B4B2E86146  Example >> Cooling Tower 1  Area A   \n",
       "\n",
       "                Name  Description          Type Value Unit Of Measure  \\\n",
       "0           Wet Bulb          NaN  StoredSignal                    °F   \n",
       "1  Relative Humidity          NaN  StoredSignal                     %   \n",
       "2        Temperature          NaN  StoredSignal                    °F   \n",
       "3   Compressor Power          NaN  StoredSignal                    kW   \n",
       "4   Compressor Stage          NaN  StoredSignal                string   \n",
       "5          Optimizer          NaN  StoredSignal                   NaN   \n",
       "\n",
       "  Datasource Name  Archived  \n",
       "0    Example Data     False  \n",
       "1    Example Data     False  \n",
       "2    Example Data     False  \n",
       "3    Example Data     False  \n",
       "4    Example Data     False  \n",
       "5    Example Data     False  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = spy.search({\n",
    "    \"Path\": \"Example >> Cooling Tower 1 >> Area A\"\n",
    "})\n",
    "\n",
    "# Print the output to the Jupyter page\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Refining the Results\n",
    "\n",
    "Let's pare down the results to just three signals before we pull some data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Path</th>\n",
       "      <th>Asset</th>\n",
       "      <th>Name</th>\n",
       "      <th>Description</th>\n",
       "      <th>Type</th>\n",
       "      <th>Value Unit Of Measure</th>\n",
       "      <th>Datasource Name</th>\n",
       "      <th>Archived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3B651CDB-E9CA-4699-BB05-F25138C1779A</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Relative Humidity</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>%</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35E96E0C-7761-4270-97C7-71C4E066305C</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Temperature</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>°F</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>EF7AFDB1-D686-456A-8BC9-99551EA8CA24</td>\n",
       "      <td>Example &gt;&gt; Cooling Tower 1</td>\n",
       "      <td>Area A</td>\n",
       "      <td>Compressor Power</td>\n",
       "      <td>NaN</td>\n",
       "      <td>StoredSignal</td>\n",
       "      <td>kW</td>\n",
       "      <td>Example Data</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     ID                        Path   Asset  \\\n",
       "1  3B651CDB-E9CA-4699-BB05-F25138C1779A  Example >> Cooling Tower 1  Area A   \n",
       "2  35E96E0C-7761-4270-97C7-71C4E066305C  Example >> Cooling Tower 1  Area A   \n",
       "3  EF7AFDB1-D686-456A-8BC9-99551EA8CA24  Example >> Cooling Tower 1  Area A   \n",
       "\n",
       "                Name  Description          Type Value Unit Of Measure  \\\n",
       "1  Relative Humidity          NaN  StoredSignal                     %   \n",
       "2        Temperature          NaN  StoredSignal                    °F   \n",
       "3   Compressor Power          NaN  StoredSignal                    kW   \n",
       "\n",
       "  Datasource Name  Archived  \n",
       "1    Example Data     False  \n",
       "2    Example Data     False  \n",
       "3    Example Data     False  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_signals = results.loc[results['Name'].isin(['Compressor Power', 'Temperature', 'Relative Humidity'])]\n",
    "my_signals"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's pull in the data using Seeq. The data can come from any connected datasource, including historians like *OSIsoft PI*, time series stores like *InfluxDB*, and SQL databases like *Microsoft SQL Server*.\n",
    "\n",
    "You can specify the `grid` parameter to control the granularity of the data. Seeq's calculation engine will interpolate as needed to produce a uniform DataFrame. Specify `grid=None` if you want the raw data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Pull successful from <strong>2019-01-01 00:00:00-08:00</strong> to <strong>2019-01-07 00:00:00-08:00</strong></div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">ID</td><td style=\"background-color: #EEFFEE;\">Path</td><td style=\"background-color: #EEFFEE;\">Asset</td><td style=\"background-color: #EEFFEE;\">Name</td><td style=\"background-color: #EEFFEE;\">Count</td><td style=\"background-color: #EEFFEE;\">Time</td><td style=\"background-color: #EEFFEE;\">Result</td></tr><tr style=\"background-color: #EEFFEE;\"><td>1</td><td>3B651CDB-E9CA-4699-BB05-F25138C1779A</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Relative Humidity</td><td>289</td><td>00:00:00.04</td><td>Success</td></tr><tr style=\"background-color: #EEFFEE;\"><td>2</td><td>35E96E0C-7761-4270-97C7-71C4E066305C</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Temperature</td><td>289</td><td>00:00:00.04</td><td>Success</td></tr><tr style=\"background-color: #EEFFEE;\"><td>3</td><td>EF7AFDB1-D686-456A-8BC9-99551EA8CA24</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Compressor Power</td><td>289</td><td>00:00:00.05</td><td>Success</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Relative Humidity</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Compressor Power</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:00:00-08:00</th>\n",
       "      <td>42.049933</td>\n",
       "      <td>87.110200</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:30:00-08:00</th>\n",
       "      <td>43.855915</td>\n",
       "      <td>86.364014</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 01:00:00-08:00</th>\n",
       "      <td>45.209910</td>\n",
       "      <td>86.023508</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 01:30:00-08:00</th>\n",
       "      <td>45.082633</td>\n",
       "      <td>85.619200</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 02:00:00-08:00</th>\n",
       "      <td>46.437722</td>\n",
       "      <td>84.459696</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Relative Humidity  Temperature  Compressor Power\n",
       "2019-01-01 00:00:00-08:00          42.049933    87.110200          0.002923\n",
       "2019-01-01 00:30:00-08:00          43.855915    86.364014          0.002923\n",
       "2019-01-01 01:00:00-08:00          45.209910    86.023508          0.002923\n",
       "2019-01-01 01:30:00-08:00          45.082633    85.619200          0.002923\n",
       "2019-01-01 02:00:00-08:00          46.437722    84.459696          0.002923"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_data = spy.pull(my_signals, start='2019-01-01', end='2019-01-07', grid='30min', header='Name')\n",
    "my_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting the Results\n",
    "Let's take a quick peek at the data we just pulled using the built in `DataFrame.plot()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x22428bcd978>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 83264 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Make the plot render at a bigger size than default\n",
    "matplotlib.rcParams['figure.figsize'] = [12, 8]\n",
    "\n",
    "my_data.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE: Sometimes you have to execute the cell above twice for the plot to show. Not sure why.**\n",
    "\n",
    "How about a matrix of scatterplots, with each variable plotted against each other? Pandas can do that easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 84728 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.plotting.scatter_matrix(my_data)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Seeq's Calculation Engine\n",
    "\n",
    "What if we want to use Seeq's calculation engine to reduce the noise in the **Temperature** and **Relative Humidity** signals?\n",
    "\n",
    "We can apply a calculation by specifying a formula as the `calculation` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Pull successful from <strong>2019-01-01 00:00:00-08:00</strong> to <strong>2019-01-07 00:00:00-08:00</strong></div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">ID</td><td style=\"background-color: #EEFFEE;\">Path</td><td style=\"background-color: #EEFFEE;\">Asset</td><td style=\"background-color: #EEFFEE;\">Name</td><td style=\"background-color: #EEFFEE;\">Count</td><td style=\"background-color: #EEFFEE;\">Time</td><td style=\"background-color: #EEFFEE;\">Result</td></tr><tr style=\"background-color: #EEFFEE;\"><td>1</td><td>3B651CDB-E9CA-4699-BB05-F25138C1779A</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Relative Humidity</td><td>1729</td><td>00:00:00.11</td><td>Success</td></tr><tr style=\"background-color: #EEFFEE;\"><td>2</td><td>35E96E0C-7761-4270-97C7-71C4E066305C</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Temperature</td><td>1729</td><td>00:00:00.17</td><td>Success</td></tr><tr style=\"background-color: #EEFFEE;\"><td>3</td><td>EF7AFDB1-D686-456A-8BC9-99551EA8CA24</td><td>Example >> Cooling Tower 1</td><td>Area A</td><td>Compressor Power</td><td>1729</td><td>00:00:00.14</td><td>Success</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Relative Humidity</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Compressor Power</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:00:00-08:00</th>\n",
       "      <td>42.622887</td>\n",
       "      <td>87.023118</td>\n",
       "      <td>0.606612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:05:00-08:00</th>\n",
       "      <td>42.806876</td>\n",
       "      <td>86.900891</td>\n",
       "      <td>0.723195</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:10:00-08:00</th>\n",
       "      <td>42.986413</td>\n",
       "      <td>86.785684</td>\n",
       "      <td>0.808959</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:15:00-08:00</th>\n",
       "      <td>43.161983</td>\n",
       "      <td>86.676838</td>\n",
       "      <td>0.865030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:20:00-08:00</th>\n",
       "      <td>43.333720</td>\n",
       "      <td>86.574218</td>\n",
       "      <td>0.889938</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Relative Humidity  Temperature  Compressor Power\n",
       "2019-01-01 00:00:00-08:00          42.622887    87.023118          0.606612\n",
       "2019-01-01 00:05:00-08:00          42.806876    86.900891          0.723195\n",
       "2019-01-01 00:10:00-08:00          42.986413    86.785684          0.808959\n",
       "2019-01-01 00:15:00-08:00          43.161983    86.676838          0.865030\n",
       "2019-01-01 00:20:00-08:00          43.333720    86.574218          0.889938"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "calculated_data = spy.pull(\n",
    "    my_signals,\n",
    "    start='2019-01-01',\n",
    "    end='2019-01-07',\n",
    "    calculation='$signal.lowPassFilter(200min, 3min, 333)',\n",
    "    grid='5min',\n",
    "    header='Name')\n",
    "calculated_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 89864 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "calculated_data.plot()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pushing a New Condition to Seeq\n",
    "\n",
    "Let's push a Seeq *Condition* into our workbook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Pushed successfully to datasource <strong>Seeq Data Lab [Datasource ID: Seeq Data Lab]</strong> and scoped to workbook ID <strong>1889C1B5-F4B5-42F3-8617-A359DDB7CF5C</strong><br>Click the following link to see what you pushed in Seeq:<br><a href=\"http://localhost:34216/9F4CBC39-E0A9-4DD8-89E8-07715F2DCFAC/workbook/1889C1B5-F4B5-42F3-8617-A359DDB7CF5C/worksheet/84992799-42AB-4C06-8944-BF0F16C86D8C\" target=\"_new\">http://localhost:34216/9F4CBC39-E0A9-4DD8-89E8-07715F2DCFAC/workbook/1889C1B5-F4B5-42F3-8617-A359DDB7CF5C/worksheet/84992799-42AB-4C06-8944-BF0F16C86D8C</a></div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">Asset</td><td style=\"background-color: #EEFFEE;\">Condition</td><td style=\"background-color: #EEFFEE;\">Overall</td><td style=\"background-color: #EEFFEE;\">Relationship</td><td style=\"background-color: #EEFFEE;\">Scalar</td><td style=\"background-color: #EEFFEE;\">Signal</td><td style=\"background-color: #EEFFEE;\">Threshold Metric</td></tr><tr style=\"background-color: #EEFFEE;\"><td>Items pushed</td><td>0</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Formula</th>\n",
       "      <th>Formula Parameters</th>\n",
       "      <th>Name</th>\n",
       "      <th>Type</th>\n",
       "      <th>Datasource Class</th>\n",
       "      <th>Datasource ID</th>\n",
       "      <th>Data ID</th>\n",
       "      <th>ID</th>\n",
       "      <th>Push Result</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>$cp.valueSearch(isGreaterThan(25kW))</td>\n",
       "      <td>[cp=EF7AFDB1-D686-456A-8BC9-99551EA8CA24]</td>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>CalculatedCondition</td>\n",
       "      <td>Seeq Data Lab</td>\n",
       "      <td>Seeq Data Lab</td>\n",
       "      <td>[1889C1B5-F4B5-42F3-8617-A359DDB7CF5C] {Condit...</td>\n",
       "      <td>51DDB467-C50A-4256-B1A9-7679C6E13D7E</td>\n",
       "      <td>Success</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                Formula  \\\n",
       "0  $cp.valueSearch(isGreaterThan(25kW))   \n",
       "\n",
       "                          Formula Parameters                Name  \\\n",
       "0  [cp=EF7AFDB1-D686-456A-8BC9-99551EA8CA24]  Compressor on High   \n",
       "\n",
       "                  Type Datasource Class  Datasource ID  \\\n",
       "0  CalculatedCondition    Seeq Data Lab  Seeq Data Lab   \n",
       "\n",
       "                                             Data ID  \\\n",
       "0  [1889C1B5-F4B5-42F3-8617-A359DDB7CF5C] {Condit...   \n",
       "\n",
       "                                     ID Push Result  \n",
       "0  51DDB467-C50A-4256-B1A9-7679C6E13D7E     Success  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "push_results = spy.push(metadata=pd.DataFrame([{\n",
    "    'Name': 'Compressor on High',\n",
    "    'Type': 'Condition',\n",
    "    'Formula': '$cp.valueSearch(isGreaterThan(25kW))',\n",
    "    'Formula Parameters': {\n",
    "        # Note here that we are just using a row from our search results. The SPy module will figure\n",
    "        # out that it contains an identifier that we can use.\n",
    "        '$cp': results[results['Name'] == 'Compressor Power']\n",
    "    }\n",
    "}]))\n",
    "\n",
    "push_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pulling the Capsules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Pull successful from <strong>2019-01-01 00:00:00-08:00</strong> to <strong>2019-01-07 00:00:00-08:00</strong></div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">ID</td><td style=\"background-color: #EEFFEE;\">Name</td><td style=\"background-color: #EEFFEE;\">Count</td><td style=\"background-color: #EEFFEE;\">Time</td><td style=\"background-color: #EEFFEE;\">Result</td></tr><tr style=\"background-color: #EEFFEE;\"><td>0</td><td>51DDB467-C50A-4256-B1A9-7679C6E13D7E</td><td>Compressor on High</td><td>6</td><td>00:00:00.04</td><td>Success</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Condition</th>\n",
       "      <th>Capsule Start</th>\n",
       "      <th>Capsule End</th>\n",
       "      <th>Capsule Is Uncertain</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-01 16:16:37.331031739-08:00</td>\n",
       "      <td>2019-01-01 21:00:33.238973103-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-02 16:13:18.731947840-08:00</td>\n",
       "      <td>2019-01-02 18:55:03.027869976-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-03 09:28:04.251451273-08:00</td>\n",
       "      <td>2019-01-03 10:21:58.603314005-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-04 09:20:33.782090068-08:00</td>\n",
       "      <td>2019-01-04 10:46:41.571969511-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-04 18:18:11.253715409-08:00</td>\n",
       "      <td>2019-01-04 21:01:23.108876186-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Compressor on High</td>\n",
       "      <td>2019-01-05 09:20:29.256597642-08:00</td>\n",
       "      <td>2019-01-05 10:43:39.192264349-08:00</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Condition                       Capsule Start  \\\n",
       "0  Compressor on High 2019-01-01 16:16:37.331031739-08:00   \n",
       "1  Compressor on High 2019-01-02 16:13:18.731947840-08:00   \n",
       "2  Compressor on High 2019-01-03 09:28:04.251451273-08:00   \n",
       "3  Compressor on High 2019-01-04 09:20:33.782090068-08:00   \n",
       "4  Compressor on High 2019-01-04 18:18:11.253715409-08:00   \n",
       "5  Compressor on High 2019-01-05 09:20:29.256597642-08:00   \n",
       "\n",
       "                          Capsule End  Capsule Is Uncertain  \n",
       "0 2019-01-01 21:00:33.238973103-08:00                 False  \n",
       "1 2019-01-02 18:55:03.027869976-08:00                 False  \n",
       "2 2019-01-03 10:21:58.603314005-08:00                 False  \n",
       "3 2019-01-04 10:46:41.571969511-08:00                 False  \n",
       "4 2019-01-04 21:01:23.108876186-08:00                 False  \n",
       "5 2019-01-05 10:43:39.192264349-08:00                 False  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "capsules = spy.pull(push_results, start='2019-01-01', end='2019-01-07')\n",
    "capsules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting Conditions Alongside Signals\n",
    "Now let's use the new condition in a plot, this time using `spy.plot()`, which has a set of predefined plot configurations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 94460 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "samples = calculated_data\n",
    "\n",
    "spy.plot(samples=samples, capsules=capsules)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating a New Signal using Python\n",
    "One reason you wanted to get the data into Python in the first place is to create new data. Let's clip **Compressor Power** by 10 so it simplifies to just representing power on or off."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Compressor Power</th>\n",
       "      <th>Attenuated Power</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:00:00-08:00</th>\n",
       "      <td>0.002923</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 00:30:00-08:00</th>\n",
       "      <td>0.002923</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 01:00:00-08:00</th>\n",
       "      <td>0.002923</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 01:30:00-08:00</th>\n",
       "      <td>0.002923</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01 02:00:00-08:00</th>\n",
       "      <td>0.002923</td>\n",
       "      <td>0.002923</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Compressor Power  Attenuated Power\n",
       "2019-01-01 00:00:00-08:00          0.002923          0.002923\n",
       "2019-01-01 00:30:00-08:00          0.002923          0.002923\n",
       "2019-01-01 01:00:00-08:00          0.002923          0.002923\n",
       "2019-01-01 01:30:00-08:00          0.002923          0.002923\n",
       "2019-01-01 02:00:00-08:00          0.002923          0.002923"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def attenuate_power(sample):\n",
    "    return min(sample, 10)\n",
    "\n",
    "clipped_data = pd.DataFrame({'Compressor Power': my_data['Compressor Power'],\n",
    "                             'Attenuated Power': my_data['Compressor Power'].apply(attenuate_power)})\n",
    "clipped_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2242c56b9b0>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 46968 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "clipped_data.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pushing Data Into Seeq\n",
    "Jupyter is a great environment for programming, but let's go back to Seeq Workbench and make things truly interactive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"background-color: #EEFFEE;text-align: left;\">Pushed successfully to datasource <strong>Seeq Data Lab [Datasource ID: Seeq Data Lab]</strong> and scoped to workbook ID <strong>1889C1B5-F4B5-42F3-8617-A359DDB7CF5C</strong><br>Click the following link to see what you pushed in Seeq:<br><a href=\"http://localhost:34216/9F4CBC39-E0A9-4DD8-89E8-07715F2DCFAC/workbook/1889C1B5-F4B5-42F3-8617-A359DDB7CF5C/worksheet/84992799-42AB-4C06-8944-BF0F16C86D8C\" target=\"_new\">http://localhost:34216/9F4CBC39-E0A9-4DD8-89E8-07715F2DCFAC/workbook/1889C1B5-F4B5-42F3-8617-A359DDB7CF5C/worksheet/84992799-42AB-4C06-8944-BF0F16C86D8C</a></div><table><tr><td style=\"background-color: #EEFFEE;\"></td><td style=\"background-color: #EEFFEE;\">Count</td><td style=\"background-color: #EEFFEE;\">Result</td><td style=\"background-color: #EEFFEE;\">Time</td></tr><tr style=\"background-color: #EEFFEE;\"><td>Compressor Power</td><td>289</td><td>Success</td><td>00:00:00.06</td></tr><tr style=\"background-color: #EEFFEE;\"><td>Attenuated Power</td><td>289</td><td>Success</td><td>00:00:00.09</td></tr></table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Push Count</th>\n",
       "      <th>Push Time</th>\n",
       "      <th>Push Result</th>\n",
       "      <th>Name</th>\n",
       "      <th>ID</th>\n",
       "      <th>Type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Compressor Power</th>\n",
       "      <td>289.0</td>\n",
       "      <td>0:00:00.061833</td>\n",
       "      <td>Success</td>\n",
       "      <td>Compressor Power</td>\n",
       "      <td>69382339-EE80-4CBE-BEFE-F19F8064BC67</td>\n",
       "      <td>StoredSignal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Attenuated Power</th>\n",
       "      <td>289.0</td>\n",
       "      <td>0:00:00.094747</td>\n",
       "      <td>Success</td>\n",
       "      <td>Attenuated Power</td>\n",
       "      <td>3D9A4346-A9E3-478C-B987-72D9E1946804</td>\n",
       "      <td>StoredSignal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Push Count       Push Time Push Result              Name  \\\n",
       "Compressor Power       289.0  0:00:00.061833     Success  Compressor Power   \n",
       "Attenuated Power       289.0  0:00:00.094747     Success  Attenuated Power   \n",
       "\n",
       "                                                    ID          Type  \n",
       "Compressor Power  69382339-EE80-4CBE-BEFE-F19F8064BC67  StoredSignal  \n",
       "Attenuated Power  3D9A4346-A9E3-478C-B987-72D9E1946804  StoredSignal  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spy.push(clipped_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is now in Seeq's internal time series database and is scoped to a workbook. Click the link above to see it!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
