{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of Creating Aligned Hierarchies for a Mazurka Score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we will walk through the elements of the `repytah` package that pertain to the creation of aligned hierarchies for a music-based data stream (eg. a song).\n",
    "\n",
    "Beginning with features (such as MFCCs and chroma features) for each timestep in your music-based data stream, there are several steps to this process:\n",
    "\n",
    "0. Create the self-dissimilarity matrix (SDM).\n",
    "1. Highlight pairs of timesteps that are close enough to be considered as repetitions of each other. (In other words, threshold the SDM)\n",
    "2. Find pairs of structure repetitions (represented as diagonals within the thresholded SDM).\n",
    "3. Find any pairs of structure repetitions not found in step 2, and group the structure repetitions.\n",
    "4. Remove any repeated structures that have overlapped instances.\n",
    "5. Distill the collection of repeated structures into the _essential structure components_ , i.e. the smallest meaningful repetitions on which all larger repeats are constructed. Each timestep will be contained in no more than one essential structure component\n",
    "\n",
    "_Note_: The walk-through of this example is very similar to the code in `example.py` found in this package. \n",
    "\n",
    "We begin by importing the necessary packages: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script>$(\"div.input\").show()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ee5689babe5144589d2a6166578ba387",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "ToggleButton(value=True, description='Hide code')"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# NumPy and SciPy are required for mathematical operations\n",
    "import scipy.io as sio\n",
    "import numpy as np\n",
    "\n",
    "# Pandas is used to import the csv\n",
    "import pandas as pd\n",
    "\n",
    "# Import all modules from repytah\n",
    "from repytah import *\n",
    "\n",
    "# Matplotlib is used to display outputs\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Make the images clear\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "# Hide code (optional)\n",
    "import ipywidgets as widgets\n",
    "from IPython.display import display, HTML\n",
    "javascript_functions = {False: \"hide()\", True: \"show()\"}\n",
    "button_descriptions  = {False: \"Show code\", True: \"Hide code\"}\n",
    "def toggle_code(state):\n",
    "    output_string = \"<script>$(\\\"div.input\\\").{}</script>\"\n",
    "    output_args   = (javascript_functions[state],)\n",
    "    output        = output_string.format(*output_args)\n",
    "    display(HTML(output))\n",
    "def button_action(value):\n",
    "    state = value.new\n",
    "    toggle_code(state)\n",
    "    value.owner.description = button_descriptions[state]\n",
    "state = True\n",
    "toggle_code(state)\n",
    "button = widgets.ToggleButton(state, description = button_descriptions[state])\n",
    "button.observe(button_action, \"value\")\n",
    "display(button)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 0: Create the SDM \n",
    "\n",
    "In this phase, there are a few crucial details, namely importing the data file that we would like to unearth hierarchical structural information for and determining the appropriate dissimilarity measure to use. If you already have (symmetrical) matrix representations for your data stream, then you may find it more appropriate to load your matrix and then skip ahead to Step 1 or Step 2. \n",
    "\n",
    "This step assumes that your music-based data stream (ie. a recording or score) has already had your preferred features extracted (like chroma or MFCCs) and is arranged into columns such that each column represents a time step (or beat). We refer to this as a _feature vector matrix_ as each feature vector is laid out as column within one cohesive matrix. \n",
    "\n",
    "### Importing data for structure analysis\n",
    "\n",
    "Note: For this demonstration, we are using the `load_ex_data()` built-in function from the `example` module to load our data. To recreate our example, you could do the same. This is an optional task and the normal method of reading in a data file will work.\n",
    "\n",
    "We are using Chopin's Mazurka Op.6, No.1 as input for this demonstration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import csv\n",
    "file_in = load_ex_data('data/input.csv').to_numpy()\n",
    "\n",
    "fv_mat = file_in"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating the SDM\n",
    "\n",
    "In just one line, we define the self-dissimilarity matrix. This function `create_sdm`, uses feature vectors to create an audio shingle for each time step and represents these shingles as vectors by stacking the relevant feature vectors on top of each other. Then, the cosine distance is found between these shingles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "self_dissim_mat:\n",
      " [[0.         0.36468911 0.58429924 ... 0.57425539 0.77909953 0.82434591]\n",
      " [0.36468911 0.         0.41660702 ... 0.68829818 0.64039838 0.80480641]\n",
      " [0.58429924 0.41660702 0.         ... 0.72072778 0.59670143 0.57191158]\n",
      " ...\n",
      " [0.57425539 0.68829818 0.72072778 ... 0.         0.60941866 0.70720396]\n",
      " [0.77909953 0.64039838 0.59670143 ... 0.60941866 0.         0.39416349]\n",
      " [0.82434591 0.80480641 0.57191158 ... 0.70720396 0.39416349 0.        ]]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 618732 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 263,
       "width": 257
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Number of feature vectors per shingle\n",
    "num_fv_per_shingle = 12\n",
    "\n",
    "# Create the self-dissimilarity matrix\n",
    "self_dissim_mat = create_sdm(fv_mat, num_fv_per_shingle)\n",
    "\n",
    "print('self_dissim_mat:\\n', self_dissim_mat)\n",
    "\n",
    "# Produce a visualization\n",
    "SDM = plt.imshow(self_dissim_mat, cmap=\"RdBu\")\n",
    "plt.title('Self-Dissimilarity Matrix')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Threshold the SDM \n",
    "\n",
    "In this step, the self-dissimilarity matrix is thresholded to produce a binary matrix of the same dimensions. This matrix is used to identify repeated structures, which are represented by diagonals of the same length. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "thresh_dist_mat:\n",
      " [[ True False False ... False False False]\n",
      " [False  True False ... False False False]\n",
      " [False False  True ... False False False]\n",
      " ...\n",
      " [False False False ...  True False False]\n",
      " [False False False ... False  True False]\n",
      " [False False False ... False False  True]]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 51984 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 263,
       "width": 257
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "song_length = self_dissim_mat.shape[0]\n",
    "thresh = 0.02\n",
    "\n",
    "# Threshold the SDM to produce a binary matrix \n",
    "thresh_dist_mat = (self_dissim_mat <= thresh) \n",
    "\n",
    "print('thresh_dist_mat:\\n', thresh_dist_mat)\n",
    "\n",
    "# Produce a visualization\n",
    "SDM = plt.imshow(thresh_dist_mat, cmap=\"Greys\")\n",
    "plt.title('Thresholded Matrix')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Find the diagonals present and store all pairs of repeats found in a list\n",
    "\n",
    "The diagonals in the thresholded matrix are found and recorded in an array of repeats. `find_initial_repeats` does this by looking for the largest repeated structures in **thresh_dist_mat**, which are illustrated in the above Thresholded Matrix diagram. Once all repeated structures are found, which are represented as diagonals present in **thresh_dist_mat**, they are stored with their start/end indices and lengths in a list. As each diagonal is found, it is removed to avoid identifying repeated sub-structures.\n",
    "\n",
    "Below is the listing of the pairs of repeats found by `find_initial_repeats`: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "all_lst:\n",
      " [[  2   2  26  26   1]\n",
      " [  2   2  74  74   1]\n",
      " [  2   2 146 146   1]\n",
      " [  2   2 218 218   1]\n",
      " [  2   2 314 314   1]\n",
      " [ 26  26  50  50   1]\n",
      " [ 50  50  74  74   1]\n",
      " [ 50  50 146 146   1]\n",
      " [ 50  50 218 218   1]\n",
      " [ 50  50 314 314   1]\n",
      " [247 250 271 274   4]\n",
      " [124 156 292 324  33]\n",
      " [196 228 292 324  33]\n",
      " [  2  36 290 324  35]\n",
      " [ 50  84 290 324  35]\n",
      " [  3  38 123 158  36]\n",
      " [ 51  86 195 230  36]\n",
      " [  3  39 195 231  37]\n",
      " [  1  38  49  86  38]\n",
      " [ 51 122 123 194  72]\n",
      " [ 87 158 159 230  72]\n",
      " [  1 325   1 325 325]]\n"
     ]
    }
   ],
   "source": [
    "all_lst = find_initial_repeats(thresh_dist_mat, np.arange(1, song_length + 1), 0)\n",
    "    \n",
    "print('all_lst:\\n', all_lst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take moment to examine what we have and time it back to the original thresholded SDM. The first ten pairs listed in **all_lst** are repeats of length 1. We also note that the last \"pair\" of repeats is the whole score being matched to itself. For now, we set this \"repeat\" and the ones that are of length 1 aside. \n",
    "\n",
    "In the image below, for each pair of repeats, we color one of the two diagonals associated to that pairing. The second one is the matching diagonal when flipped over the main diagonal. \n",
    "\n",
    "For example, the red diagonal represents the pair of repeats that start at beats 124 and 292 and are 33 beats long, and the blue diagonal represents the pair of repeats that start at beat 51 and 123 and are 72 beats long."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46292 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 251,
       "width": 257
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_all_lst(thresh_dist_mat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Find any diagonals in the thresholded matrix T that are contained in larger diagonals in T but not found in step 2, then group pairs of repeats\n",
    "\n",
    "Any smaller diagonals that are contained in larger diagonals and are not found in step 2 are found and added to the array of repeats in this step. All possible repeat lengths are looped over in **all_lst**, and larger repeats containing smaller repeats are broken up into up to 3 sections each, the part before the overlap, the overlap, and the part after the overlap. With this, a more complete list of repeated structures is created. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "complete_lst:\n",
      " [[  1   1  49  49   1   1]\n",
      " [  2   2  26  26   1   2]\n",
      " [  2   2  50  50   1   2]\n",
      " [  2   2  74  74   1   2]\n",
      " [  2   2 146 146   1   2]\n",
      " [  2   2 218 218   1   2]\n",
      " [  2   2 290 290   1   2]\n",
      " [  2   2 314 314   1   2]\n",
      " [ 26  26  50  50   1   2]\n",
      " [ 26  26  74  74   1   2]\n",
      " [ 26  26 146 146   1   2]\n",
      " [ 26  26 218 218   1   2]\n",
      " [ 26  26 314 314   1   2]\n",
      " [ 50  50  74  74   1   2]\n",
      " [ 50  50 146 146   1   2]\n",
      " [ 50  50 218 218   1   2]\n",
      " [ 50  50 290 290   1   2]\n",
      " [ 50  50 314 314   1   2]\n",
      " [ 74  74 146 146   1   2]\n",
      " [ 74  74 218 218   1   2]\n",
      " [ 74  74 314 314   1   2]\n",
      " [146 146 218 218   1   2]\n",
      " [146 146 314 314   1   2]\n",
      " [218 218 314 314   1   2]\n",
      " [  3   3 123 123   1   3]\n",
      " [  3   3 195 195   1   3]\n",
      " [ 51  51 123 123   1   3]\n",
      " [ 51  51 195 195   1   3]\n",
      " [ 39  39 231 231   1   4]\n",
      " [  1   2  49  50   2   1]\n",
      " [  2   3 290 291   2   2]\n",
      " [ 50  51 290 291   2   2]\n",
      " [ 37  38  85  86   2   3]\n",
      " [ 37  38 157 158   2   3]\n",
      " [ 85  86 229 230   2   3]\n",
      " [157 158 229 230   2   3]\n",
      " [ 37  39 229 231   3   1]\n",
      " [247 250 271 274   4   1]\n",
      " [ 27  36 315 324  10   1]\n",
      " [ 75  84 315 324  10   1]\n",
      " [147 156 315 324  10   1]\n",
      " [219 228 315 324  10   1]\n",
      " [ 27  38  75  86  12   1]\n",
      " [ 27  38 147 158  12   1]\n",
      " [ 75  86 219 230  12   1]\n",
      " [147 158 219 230  12   1]\n",
      " [ 27  39 219 231  13   1]\n",
      " [124 145 292 313  22   1]\n",
      " [196 217 292 313  22   1]\n",
      " [  3  25 123 145  23   1]\n",
      " [  3  25 195 217  23   1]\n",
      " [ 51  73 123 145  23   1]\n",
      " [ 51  73 195 217  23   1]\n",
      " [  2  25 290 313  24   1]\n",
      " [ 50  73 290 313  24   1]\n",
      " [  1  25  49  73  25   1]\n",
      " [  4  36 124 156  33   1]\n",
      " [  4  36 196 228  33   1]\n",
      " [  4  36 292 324  33   1]\n",
      " [ 52  84 124 156  33   1]\n",
      " [ 52  84 196 228  33   1]\n",
      " [ 52  84 292 324  33   1]\n",
      " [124 156 196 228  33   1]\n",
      " [124 156 292 324  33   1]\n",
      " [196 228 292 324  33   1]\n",
      " [  3  36 291 324  34   1]\n",
      " [ 51  84 291 324  34   1]\n",
      " [  2  36  50  84  35   1]\n",
      " [  2  36 290 324  35   1]\n",
      " [ 50  84 290 324  35   1]\n",
      " [  3  38  51  86  36   1]\n",
      " [  3  38 123 158  36   1]\n",
      " [  3  38 195 230  36   1]\n",
      " [ 51  86 123 158  36   1]\n",
      " [ 51  86 195 230  36   1]\n",
      " [123 158 195 230  36   1]\n",
      " [ 87 122 159 194  36   2]\n",
      " [  3  39 195 231  37   1]\n",
      " [ 87 123 159 195  37   2]\n",
      " [  1  38  49  86  38   1]\n",
      " [ 85 122 157 194  38   2]\n",
      " [ 75 122 147 194  48   1]\n",
      " [ 87 145 159 217  59   1]\n",
      " [ 51 122 123 194  72   1]\n",
      " [ 87 158 159 230  72   2]]\n"
     ]
    }
   ],
   "source": [
    "complete_lst = find_complete_list(all_lst, song_length)\n",
    "    \n",
    "print('complete_lst:\\n', complete_lst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is clear that **complete_list** is much longer that **all_lst**, which makes sense because we are adding smaller pieces of the larger repeats, when an overlap in time has occurred between a smaller repeat and a larger one. \n",
    "\n",
    "For example, the repeat of length 72 starting at beat 51 overlaps with the smaller repeat of length 48 starting at beat 75 (the purple section). \n",
    "\n",
    "The image below visualizes the idea of distilling smaller parts from larger repeats in step 2 by showing some of the smaller repetitions found within the larger repeats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46504 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 251,
       "width": 257
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize_complete_lst(thresh_dist_mat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Remove any repeated structure that has at least two repeats that overlap in time\n",
    "\n",
    "In this step, repeated structures with the same annotation and length are removed if they are overlapping.\n",
    "\n",
    "This is done by looping over all possible repeat lengths, finding all the groups of repeats of the same length. For each of those groups, `remove_overlaps` determines whether there exists any pair of repeats that overlaps in time. If a pair like that exists, all the overlapping repeats are removed. \n",
    "\n",
    "Along with a list of all the repeats with no overlaps, this step also forms a matrix, a list of the associated lengths of the repeats in the matrix, the annotations of the repeats in the matrix, and a list of the overlaps. The matrix is a binary matrix that visualizes the repeats, representing the start of a repeat with a 1. This matrix, in combination with the list of the lengths of the repeats, can be used to visualize the repeats. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "List with no overlaps:\n",
      " [[  1   1  49  49   1   1]\n",
      " [  2   2  26  26   1   2]\n",
      " [  2   2  50  50   1   2]\n",
      " [  2   2  74  74   1   2]\n",
      " [  2   2 146 146   1   2]\n",
      " [  2   2 218 218   1   2]\n",
      " [  2   2 290 290   1   2]\n",
      " [  2   2 314 314   1   2]\n",
      " [  3   3 123 123   1   3]\n",
      " [  3   3 195 195   1   3]\n",
      " [ 26  26  50  50   1   2]\n",
      " [ 26  26  74  74   1   2]\n",
      " [ 26  26 146 146   1   2]\n",
      " [ 26  26 218 218   1   2]\n",
      " [ 26  26 314 314   1   2]\n",
      " [ 39  39 231 231   1   4]\n",
      " [ 50  50  74  74   1   2]\n",
      " [ 50  50 146 146   1   2]\n",
      " [ 50  50 218 218   1   2]\n",
      " [ 50  50 290 290   1   2]\n",
      " [ 50  50 314 314   1   2]\n",
      " [ 51  51 123 123   1   3]\n",
      " [ 51  51 195 195   1   3]\n",
      " [ 74  74 146 146   1   2]\n",
      " [ 74  74 218 218   1   2]\n",
      " [ 74  74 314 314   1   2]\n",
      " [146 146 218 218   1   2]\n",
      " [146 146 314 314   1   2]\n",
      " [218 218 314 314   1   2]\n",
      " [  1   2  49  50   2   1]\n",
      " [  2   3 290 291   2   2]\n",
      " [ 37  38  85  86   2   3]\n",
      " [ 37  38 157 158   2   3]\n",
      " [ 50  51 290 291   2   2]\n",
      " [ 85  86 229 230   2   3]\n",
      " [157 158 229 230   2   3]\n",
      " [ 37  39 229 231   3   1]\n",
      " [247 250 271 274   4   1]\n",
      " [ 27  36 315 324  10   1]\n",
      " [ 75  84 315 324  10   1]\n",
      " [147 156 315 324  10   1]\n",
      " [219 228 315 324  10   1]\n",
      " [ 27  38  75  86  12   1]\n",
      " [ 27  38 147 158  12   1]\n",
      " [ 75  86 219 230  12   1]\n",
      " [147 158 219 230  12   1]\n",
      " [ 27  39 219 231  13   1]\n",
      " [124 145 292 313  22   1]\n",
      " [196 217 292 313  22   1]\n",
      " [  3  25 123 145  23   1]\n",
      " [  3  25 195 217  23   1]\n",
      " [ 51  73 123 145  23   1]\n",
      " [ 51  73 195 217  23   1]\n",
      " [  2  25 290 313  24   1]\n",
      " [ 50  73 290 313  24   1]\n",
      " [  1  25  49  73  25   1]\n",
      " [  4  36 124 156  33   1]\n",
      " [  4  36 196 228  33   1]\n",
      " [  4  36 292 324  33   1]\n",
      " [ 52  84 124 156  33   1]\n",
      " [ 52  84 196 228  33   1]\n",
      " [ 52  84 292 324  33   1]\n",
      " [124 156 196 228  33   1]\n",
      " [124 156 292 324  33   1]\n",
      " [196 228 292 324  33   1]\n",
      " [  3  36 291 324  34   1]\n",
      " [ 51  84 291 324  34   1]\n",
      " [  2  36  50  84  35   1]\n",
      " [  2  36 290 324  35   1]\n",
      " [ 50  84 290 324  35   1]\n",
      " [  3  38  51  86  36   1]\n",
      " [  3  38 123 158  36   1]\n",
      " [  3  38 195 230  36   1]\n",
      " [ 51  86 123 158  36   1]\n",
      " [ 51  86 195 230  36   1]\n",
      " [ 87 122 159 194  36   2]\n",
      " [123 158 195 230  36   1]\n",
      " [  3  39 195 231  37   1]\n",
      " [ 87 123 159 195  37   2]\n",
      " [  1  38  49  86  38   1]\n",
      " [ 85 122 157 194  38   2]\n",
      " [ 75 122 147 194  48   1]\n",
      " [ 87 145 159 217  59   1]\n",
      " [ 51 122 123 194  72   1]\n",
      " [ 87 158 159 230  72   2]]\n",
      "Matrix with no overlaps:\n",
      " [[1 0 0 ... 0 0 0]\n",
      " [0 1 0 ... 0 0 0]\n",
      " [0 0 1 ... 0 0 0]\n",
      " ...\n",
      " [0 0 0 ... 0 0 0]\n",
      " [0 0 0 ... 0 0 0]\n",
      " [0 0 0 ... 0 0 0]]\n",
      "Lengths of the repeats in the matrix: [ 1  1  1  1  2  2  2  3  4 10 12 13 22 23 24 25 33 34 35 36 36 37 37 38\n",
      " 38 48 59 72 72]\n",
      "Annotations of the repeats in the matrix: [1 2 3 4 1 2 3 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 2 1 1 1 2]\n",
      "List of overlaps:\n",
      " []\n"
     ]
    }
   ],
   "source": [
    "output_tuple = remove_overlaps(complete_lst, song_length)\n",
    "\n",
    "print('List with no overlaps:\\n', output_tuple[0])\n",
    "print('Matrix with no overlaps:\\n', output_tuple[1])\n",
    "print('Lengths of the repeats in the matrix:', output_tuple[2])\n",
    "print('Annotations of the repeats in the matrix:', output_tuple[3])\n",
    "print('List of overlaps:\\n', output_tuple[4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Find the essential structure components of the song and build the aligned hierarchies\n",
    "\n",
    "Building the aligned hierarchies takes two final steps: finding the essential structure components and using them to create the full aligned hierarchies. `hierarchical_structure` first finds the essential structure components with `breakup_overlaps_by_intersect`. After this, modified versions of steps 2-4 take place to create the full hierarchical structure. \n",
    "\n",
    "### Essential Structure Components\n",
    "\n",
    "The essential structure components are the structure building blocks; that is they are the smallest meaningful repeated structures. All repeated structures within a piece are constructed with right and left unions of the essential structure components. Each timestep can be in at most **one** essential structure component. The first panel in the below image shows the essential structure components for our example piece. The visualization is organized such that there is one row per type of repeat. Notice that we have nine types of repeats in the below example. \n",
    "\n",
    "### Creating the aligned hierarchies\n",
    "\n",
    "After finding the essential structure components for a song, we build the aligned hierarchies using a process whose result is akin to taking right and left unions of the essential structure components. The process begins by creating a list of the essential structure components in the order they appear. We assign each essential structure component the number of the row that it sits in. For consectutive timesteps where there is not essential structure component, we use 0 for that block. In the below example, this list would be: 1,2,3,4,2,5,6,7,0,1,2,3,4,2,5,6,8,3,4,2,5,6,8,3,4,2,5,6,7,0,9,0,9,0,2,3,4,2,5,0. \n",
    "\n",
    "Using this list, a thresholded dissimilarity matrix is created. Using steps similar to steps two through four above, we can extract information about the combinations of essential structures. From the thresholded dissimilarity matrix, all diagonals are extracted, starting with the longest one, but are not removed. This way, all possible combinations of essential structure components are found. Then, all these combinations are grouped and checked for overlapping repeated combinations. The final product is then restructured to show the widths and annotations for each type of repeated structure (represented by each row).\n",
    "\n",
    "The figures below represent the final result we get, including two of the most important images: the essential structure components (top most panel) and the complete hierarchical structure (bottom panel).\n",
    "\n",
    "The figure below with the name _Threshold Self-dissimilarity matrix of the ordering Essential Structure Components_ shows the square thresholded self-dissimilarity matrix such that the (i,j) entry is 1 if the following three conditions are true:\n",
    "\n",
    "* A repeat of an essential structure component is the i-th item in the ordering.\n",
    "* A repeat of an essential structure component is the j-th item in the ordering.\n",
    "* The repeat occurring in the i-th place of the ordering and the one occurring in the j-th place of the ordering are repeats of the same essential structure component.\n",
    "\n",
    "The figure below with the name _Repeated ordered sublists of the Essential Structure Components_ is the result of extracting all the diagonals and getting pairs of repeated ordered sublists of the essential structure components. The repetitive copies of the same repeat and overlaps are also removed. \n",
    "\n",
    "The second to last figure labeled _Repeated ordered sublists of the Essential Structure Components with leading index highlighted_ contains the same repeats as _Repeated ordered sublists of the Essential Structure Components_ but only notes the starting item in black and shows the rest of the repeat in gray. \n",
    "\n",
    "The aligned hierarchies are shown in the last image. This is the result of stretching each essential structure component to its correct length (ie. the one noted in the first image below). This transformation results in a visualization that is the number of time steps of the original song. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 36212 characters]\n",
      "text/plain": [
       "<Figure size 720x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 535,
       "width": 648
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 57584 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 481,
       "width": 592
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 116700 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 481,
       "width": 642
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 128768 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 698,
       "width": 656
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 122440 characters]\n",
      "text/plain": [
       "<Figure size 720x936 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 753,
       "width": 648
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "(mat_no_overlaps, key_no_overlaps) = output_tuple[1:3]\n",
    "\n",
    "# Distill non-overlapping repeats into essential structure components and\n",
    "# use them to build the hierarchical representation\n",
    "output_tuple = hierarchical_structure(mat_no_overlaps, key_no_overlaps, song_length, vis=True)\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "dc9f90ef3f93cd59b0a3dd3f5f7fc4f99e8f9db7b3a3baf1d7cec7bd52107718"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
