{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Assemble Module\n",
    "\n",
    "The `assemble` module of the `repytah` package finds and forms essential structure components. These components are the smallest building blocks that form the basis for every repeat in the song. The functions in this module ensure that each time step of a song is contained in at most one of the song's essential structure components by making none of the repeats overlap in time. When repeats overlap, these repeats undergo a process where they are divided until there are only non-overlapping pieces left. \n",
    "\n",
    "The following functions are exported from the `assemble` module:\n",
    "\n",
    "- `breakup_overlaps_by_intersect`: Extracts repeats in **input\\_pattern\\_obj** that has the starting indices of the repeats into the essential structure components using **bw\\_vec** that has the lengths of each repeat.\n",
    "    \n",
    "- `check_overlaps`: Compares every pair of groups, determining if there are any repeats in any pairs of the groups that overlap. \n",
    "\n",
    "- `hierarchical_structure`: Distills the repeats encoded in **matrix\\_no\\_overlaps** (and **key\\_no\\_overlaps**) to the essential structure components and then builds the hierarchical representation. Also optionally outputs visualizations of the hierarchical representations.\n",
    "\n",
    "This module uses `find_all_repeats` from the [`search`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/search_vignette.ipynb) module and `reconstruct_full_block` from the [`utilities`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/utilities_vignette.ipynb) module. \n",
    "\n",
    "For more in-depth information on the function calls, an example function pipeline is shown below. Functions from the current module are shown in red.\n",
    "\n",
    "<img src=\"pictures/function_pipeline.jpg\" width=\"380\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NumPy is used for mathematical calculations\n",
    "import numpy as np\n",
    "\n",
    "# Import other modules\n",
    "from inspect import signature \n",
    "\n",
    "# Import assemble\n",
    "from repytah.assemble import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## breakup_overlaps_by_intersect \n",
    "\n",
    "The purpose of this function is to create the essential structure components matrix. Essential structure components contain the smallest building blocks that form every repeat in the song. This matrix is created using **input\\_pattern\\_obj** that has the starting indices of the repeats and a vector **bw\\_vec** that has the lengths of each repeat. \n",
    " \n",
    "The inputs for this function are: \n",
    "- **input_pattern_obj** (np.ndarray): A binary matrix with 1's where repeats begin and 0's otherwise\n",
    "-  **bw_vec** (np.ndarray): Lengths of the repeats encoded in **input\\_pattern\\_obj**\n",
    "- **thresh_bw** (int): The smallest allowable repeat length \n",
    "\n",
    "The outputs for this function are: \n",
    "- **pattern_no_overlaps** (np.ndarray): A binary matrix with 1's where repeats of essential structure components begin \n",
    "- **pattern_no_overlaps_key** (np.ndarray): A vector containing the lengths of the repeats of essential structure components in **pattern\\_no\\_overlaps** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The input array is: \n",
      " [[1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0]\n",
      " [0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]\n",
      " [1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0]\n",
      " [0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]]\n",
      "The lengths of the repeats in the input array is: \n",
      " [[3]\n",
      " [5]\n",
      " [8]\n",
      " [8]]\n",
      "The smallest allowable repeat length is:  0\n"
     ]
    }
   ],
   "source": [
    "input_pattern_obj = np.array([[1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0],\n",
    "                              [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],\n",
    "                              [1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
    "                              [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]])\n",
    "bw_vec = np.array([[3],\n",
    "                   [5],\n",
    "                   [8],\n",
    "                   [8]])\n",
    "thresh_bw = 0\n",
    "print(\"The input array is: \\n\", input_pattern_obj)\n",
    "print(\"The lengths of the repeats in the input array is: \\n\", bw_vec)\n",
    "print(\"The smallest allowable repeat length is: \", thresh_bw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output array is: \n",
      " [[1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0]\n",
      " [0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]]\n",
      "The lengths of the repeats in the output array is: \n",
      " [[3]\n",
      " [5]]\n"
     ]
    }
   ],
   "source": [
    "output = breakup_overlaps_by_intersect(input_pattern_obj, bw_vec, thresh_bw)\n",
    "\n",
    "print(\"The output array is: \\n\", output[0])\n",
    "print(\"The lengths of the repeats in the output array is: \\n\", output[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## check_overlaps\n",
    "\n",
    "This function compares every pair of groups and checks for overlaps between those pairs. To check every pair of groups, the function creates *compare\\_left* and *compare\\_right*. *compare\\_left* repeats each row the number of rows times, and *compare\\_right* repeats the whole input the number of rows times. By comparing each corresponding time step in *compare\\_left* and *compare\\_right*, it determines if there are any overlaps between groups.\n",
    "\n",
    "The input for this function is: \n",
    "\n",
    "- **input_mat** (np.ndarray): An array waiting to be checked for overlaps\n",
    "\n",
    "The output for this function is: \n",
    "- **overlaps\\_yn** (np.ndarray): A logical array where (i,j) = 1 if row i of input matrix and row j of input matrix overlap and (i, j) = 0 elsewhere"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The input array waiting to be checked for overlaps is: \n",
      " [[0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1]\n",
      " [1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0]\n",
      " [0 0 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 0 0]\n",
      " [1 1 1 0 0 0 0 0 1 1 1 0 0 0 0 0 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "input_mat = np.array([[0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],\n",
    "                      [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0],\n",
    "                      [0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0],\n",
    "                      [1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1]])\n",
    "\n",
    "print(\"The input array waiting to be checked for overlaps is: \\n\", input_mat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output logical array is: \n",
      " [[False  True  True  True]\n",
      " [False False  True  True]\n",
      " [False False False False]\n",
      " [False False False False]]\n"
     ]
    }
   ],
   "source": [
    "output = check_overlaps(input_mat)\n",
    "\n",
    "print(\"The output logical array is: \\n\", output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  hierarchical\\_structure\n",
    "\n",
    "This function distills the repeats encoded in **matrix\\_no\\_overlaps** (and **key\\_no\\_overlaps**), which are the outputs from the [`remove_overlaps`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/transform_vignette.ipynb) function from the transform module, to the essential structure components and then builds the hierarchical representation. It optionally shows visualizations of the hierarchical structure via the **vis** argument.\n",
    "\n",
    "The inputs for this function are: \n",
    "\n",
    "- **matrix\\_no\\_overlaps** (np.array\\[int]): A binary matrix with 1's where repeats begin and 0's otherwise\n",
    "- **key\\_no\\_overlaps** (np.array\\[int]): A vector containing the lengths of the repeats encoded in **matrix_no_overlaps**\n",
    "- **sn** (int): The song length, which is the number of audio shingles\n",
    "- **vis** (bool): Shows visualizations if True (default = False)\n",
    "    \n",
    "The outputs for this function are: \n",
    "\n",
    "- **full_visualization** (np.array\\[int]): A binary matrix representation for **full_matrix_no_overlaps** with blocks of 1's equal to the length's prescribed in **full_key**\n",
    "            \n",
    "- **full_key** (np.array\\[int]): A vector containing the lengths of the hierarchical structure encoded in **full_matrix_no_overlaps**\n",
    "            \n",
    "- **full_matrix_no_overlaps** (np.array\\[int]): A binary matrix with 1's where hierarchical structure begins and 0's otherwise\n",
    "            \n",
    "- **full_anno_lst** (np.array\\[int]): A vector containing the annotation markers of the hierarchical structure encoded in each row of **full_matrix_no_overlaps**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The matrix representation of the non-overlapping repeats is: \n",
      " [[0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0]]\n",
      "The lengths of the repeats in matrix_no_overlaps are: \n",
      " [2]\n",
      "The song length is: \n",
      " 20\n"
     ]
    }
   ],
   "source": [
    "matrix_no_overlaps = np.array([[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]])\n",
    "key_no_overlaps = np.array([2])\n",
    "sn = 20\n",
    "\n",
    "print(\"The matrix representation of the non-overlapping repeats is: \\n\", matrix_no_overlaps)\n",
    "print(\"The lengths of the repeats in matrix_no_overlaps are: \\n\", key_no_overlaps)\n",
    "print(\"The song length is: \\n\", sn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 9148 characters]\n",
      "text/plain": [
       "<Figure size 720x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 11212 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 9324 characters]\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 12420 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 10376 characters]\n",
      "text/plain": [
       "<Figure size 720x936 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The binary matrix representation for the full_matrix_no_overlaps is: \n",
      " [[0 1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0]]\n",
      "The vector containing the lengths of the hierarchical structure encoded in full_matrix_no_overlaps is: \n",
      " [[2]]\n",
      "The binary matrix with 1's where hierarchical structure begins and 0's otherwise is: \n",
      " [[0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0]]\n",
      "The vector containing the annotation markers of the hierarchical structure encoded in each row \n",
      " of full_matrix_no_overlaps is: \n",
      " [1]\n"
     ]
    }
   ],
   "source": [
    "output = hierarchical_structure(matrix_no_overlaps, key_no_overlaps, sn, vis=True)\n",
    "\n",
    "full_visualization = output[0]\n",
    "full_key = output[1]\n",
    "full_matrix_no_overlaps = output[2]\n",
    "full_anno_lst = output[3]\n",
    "\n",
    "print(\"The binary matrix representation for the full_matrix_no_overlaps is: \\n\", full_visualization)\n",
    "print(\"The vector containing the lengths of the hierarchical structure encoded in full_matrix_no_overlaps is: \\n\", full_key)\n",
    "print(\"The binary matrix with 1's where hierarchical structure begins and 0's otherwise is: \\n\", full_matrix_no_overlaps)\n",
    "print(\"The vector containing the annotation markers of the hierarchical structure encoded in each row \\n of full_matrix_no_overlaps is: \\n\", full_anno_lst)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "f9558a7a9c0790c2334a9072034da5d30cf10fd46d803cfe3584dbca0180200c"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
