{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# The Search Module\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "The `search` module of the `repytah` package holds functions used to find and record the diagonals in the thresholded matrix, T. These functions prepare the found diagonals for transformation and assembling later.\n",
    "\n",
    "- `find_complete_list`: Finds all smaller diagonals (and the associated pairs of repeats) that are contained in **pair\\_list**, which is composed of larger diagonals found in [`find_initial_repeats`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/utilities_vignette.ipynb).\n",
    "\n",
    "- `find_all_repeats`: Finds all the diagonals present in **thresh\\_mat**. This function is nearly identical to `find_initial_repeats` except for two crucial differences. First, we do not remove diagonals after we find them. Second, there is no smallest bandwidth size as we are looking for all diagonals.\n",
    "\n",
    "- `find_complete_list_anno_only`: Finds annotations for all pairs of repeats found in find_all_repeats. This list contains all the pairs of repeated structures with their starting/ending indices and lengths.\n",
    "\n",
    "The following functions are imported from the [`utilities`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/utilities_vignette.ipynb) module to reformat outputs and assist with the operations of the [`search`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/search_vignette.ipynb) functions.  \n",
    "\n",
    "- `add_annotations`\n",
    "\n",
    "For more in-depth information on the function calls, an example function pipeline is shown below. Functions from the current module are shown in purple.\n",
    "\n",
    "<img src=\"pictures/function_pipeline.jpg\" width=\"380\">"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Importing necessary modules"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "# NumPy is used for mathematical calculations \n",
    "import numpy as np\n",
    "\n",
    "# Import search\n",
    "from repytah.search import *"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "\n",
    "## find_complete_list\n",
    "\n",
    "As seen in the flow chart, `find_initial_repeats` is called by [`example`](https://github.com/smith-tinkerlab/repytah/blob/main/docs/example_vignette.ipynb) right before `find_complete_list`. In `find_complete_list`, smaller pairs of repeats are added to the original list of pairs of repeats made in `find_initial_repeats`. All of the pairs of repeats correspond to each repeated structure in another numpy array called **thresh\\_mat**. This array holds all the repeated structures in a sequential data stream and the repeated structures are represented as diagonals.   \n",
    "\n",
    "The inputs for the function are:\n",
    "\n",
    "- __pair_list__ (np.ndarray): List of pairs of repeats found in earlier steps (bandwidths MUST be in ascending order). If you have run `find_initial_repeats` before this script, then **pair\\_list** will be ordered correctly.\n",
    "   \n",
    "- __song_length__ (int): Song length, which is the number of audio shingles.\n",
    "\n",
    "The output for the function is: \n",
    "\n",
    "- __lst_out__ (np.ndarray): List of pairs of repeats with smaller repeats added."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "pair_list = np.array([[ 1, 10, 46, 55, 10], \n",
    "                      [31, 40, 46, 55, 10],\n",
    "                      [10, 20, 40, 50, 11],\n",
    "                      [ 1, 15, 31, 45, 15]])\n",
    "song_length = 55\n",
    "\n",
    "print(\"The input array is: \\n\", pair_list)\n",
    "print(\"The number of audio shingles is: \\n\", song_length)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The input array is: \n",
      " [[ 1 10 46 55 10]\n",
      " [31 40 46 55 10]\n",
      " [10 20 40 50 11]\n",
      " [ 1 15 31 45 15]]\n",
      "The number of audio shingles is: \n",
      " 55\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "output = find_complete_list(pair_list, song_length)\n",
    "\n",
    "print(\"The output array is: \\n\", output)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output array is: \n",
      " [[11 15 41 45  5  1]\n",
      " [ 1 10 31 40 10  1]\n",
      " [ 1 10 46 55 10  1]\n",
      " [31 40 46 55 10  1]\n",
      " [10 20 40 50 11  1]\n",
      " [ 1 15 31 45 15  1]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "In this example, there are two more rows added to the initial **pair\\_list** input, as `find_complete_list` can detect smaller diagonals contained in larger diagonals already found in `find_initial_repeats`. The repeats now look like this:\n",
    "\n",
    "![alt text](pictures/find_all_repeats1.png)\n",
    "\n",
    "Each row represents a pair of repeats, and each column represents a time step. The time steps with the color black are the starting indices for repeats of length k that we use to check **lst\\_no\\_anno** for more repeats of length k.\n",
    "\n",
    "![alt text](pictures/find_all_repeats2.png)\n",
    "\n",
    "With the same starting index and same length, we can find the same repeats in the color yellow.\n",
    "\n",
    "![alt text](pictures/find_all_repeats3.png)\n",
    "\n",
    "Then we find two more groups of repeats. \n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "##  find_all_repeats \n",
    "\n",
    "`find_all_repeats` finds all the diagonals present in **thresh\\_mat**. This function is nearly identical to `find_initial_repeats`, with two crucial differences. First, we do not remove diagonals after we find them. Second, there is no smallest bandwidth size as we are looking for all diagonals.\n",
    "\n",
    "The inputs for the function are: \n",
    "\n",
    "- __thresh_mat__ (np.ndarray): Thresholded matrix that we extract diagonals from\n",
    "- __band_width_vec__ (np.ndarray): Vector of lengths of diagonals to be found. Should be 1, 2, 3, ..., n where n is the number of timesteps.\n",
    "\n",
    "The output for the function is:\n",
    "\n",
    "- __all_lst__ (np.ndarray): Pairs of repeats that correspond to diagonals in **thresh\\_mat**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "thresh_mat = np.array([[1, 0, 1, 0, 0],\n",
    "                       [0, 1, 0, 1, 0],\n",
    "                       [1, 0, 1, 0, 1],\n",
    "                       [0, 1, 0, 1, 0],\n",
    "                       [0, 0, 1, 0, 1]])\n",
    "\n",
    "bandwidth_vec = np.array([1, 2, 3, 4, 5])\n",
    "\n",
    "print(\"The threshold matrix is: \\n\", thresh_mat)\n",
    "print(\"The lengths of the diagonals to be found are: \\n\", bandwidth_vec)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The threshold matrix is: \n",
      " [[1 0 1 0 0]\n",
      " [0 1 0 1 0]\n",
      " [1 0 1 0 1]\n",
      " [0 1 0 1 0]\n",
      " [0 0 1 0 1]]\n",
      "The lengths of the diagonals to be found are: \n",
      " [1 2 3 4 5]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "output = find_all_repeats(thresh_mat, bandwidth_vec)\n",
    "\n",
    "print(\"The output array is: \\n\", output )"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output array is: \n",
      " [[1 1 3 3 1]\n",
      " [2 2 4 4 1]\n",
      " [3 3 5 5 1]\n",
      " [1 2 3 4 2]\n",
      " [2 3 4 5 2]\n",
      " [1 2 3 4 2]\n",
      " [2 3 4 5 2]]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## find_complete_list_anno_only\n",
    "\n",
    "`find_complete_list_anno_only` finds annotations for all pairs of repeats found in `find_initial_repeats`. This list contains all the pairs of repeated structures with their starting/ending indices and lengths.\n",
    "\n",
    "The inputs for the function are: \n",
    "\n",
    "- __pair_list__ (np.ndarray): List of pairs of repeats.\n",
    "- __song_length__ (int): Number of audio shingles in song.\n",
    "\n",
    "The output for the function is: \n",
    "\n",
    "- __out_lst__ (np.ndarray): List of pairs of repeats with smaller repeats added and with annotation markers."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "pair_list = np.array([[3,  3,  5,  5, 1],\n",
    "                      [2,  2,  8,  8, 1],\n",
    "                      [3,  3,  9,  9, 1],\n",
    "                      [2,  2, 15, 15, 1],\n",
    "                      [8,  8, 15, 15, 1],\n",
    "                      [4,  4, 17, 17, 1],\n",
    "                      [2,  3,  8,  9, 2],\n",
    "                      [3,  4,  9, 10, 2],\n",
    "                      [2,  3, 15, 16, 2],\n",
    "                      [8,  9, 15, 16, 2],\n",
    "                      [3,  4, 16, 17, 2],\n",
    "                      [2,  4,  8, 10, 3],\n",
    "                      [3,  5,  9, 11, 3],\n",
    "                      [7,  9, 14, 16, 3],\n",
    "                      [2,  4, 15, 17, 3],\n",
    "                      [3,  5, 16, 18, 3],\n",
    "                      [9, 11, 16, 18, 3],\n",
    "                      [7, 10, 14, 17, 4],\n",
    "                      [7, 11, 14, 18, 5],\n",
    "                      [8, 12, 15, 19, 5],\n",
    "                      [7, 12, 14, 19, 6]])\n",
    "song_length = 19\n",
    "\n",
    "print(\"The pairs of repeats are: \\n\", pair_list)\n",
    "print(\"The number of audio shingles in the song is:\", song_length)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The pairs of repeats are: \n",
      " [[ 3  3  5  5  1]\n",
      " [ 2  2  8  8  1]\n",
      " [ 3  3  9  9  1]\n",
      " [ 2  2 15 15  1]\n",
      " [ 8  8 15 15  1]\n",
      " [ 4  4 17 17  1]\n",
      " [ 2  3  8  9  2]\n",
      " [ 3  4  9 10  2]\n",
      " [ 2  3 15 16  2]\n",
      " [ 8  9 15 16  2]\n",
      " [ 3  4 16 17  2]\n",
      " [ 2  4  8 10  3]\n",
      " [ 3  5  9 11  3]\n",
      " [ 7  9 14 16  3]\n",
      " [ 2  4 15 17  3]\n",
      " [ 3  5 16 18  3]\n",
      " [ 9 11 16 18  3]\n",
      " [ 7 10 14 17  4]\n",
      " [ 7 11 14 18  5]\n",
      " [ 8 12 15 19  5]\n",
      " [ 7 12 14 19  6]]\n",
      "The number of audio shingles in the song are: 19\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "output = find_complete_list_anno_only(pair_list, song_length)\n",
    "\n",
    "print(\"The output array is: \\n\", output)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The output array is: \n",
      " [[ 2  2  8  8  1  1]\n",
      " [ 2  2 15 15  1  1]\n",
      " [ 8  8 15 15  1  1]\n",
      " [ 3  3  5  5  1  2]\n",
      " [ 3  3  9  9  1  2]\n",
      " [ 4  4 17 17  1  3]\n",
      " [ 2  3  8  9  2  1]\n",
      " [ 2  3 15 16  2  1]\n",
      " [ 8  9 15 16  2  1]\n",
      " [ 3  4  9 10  2  2]\n",
      " [ 3  4 16 17  2  2]\n",
      " [ 2  4  8 10  3  1]\n",
      " [ 2  4 15 17  3  1]\n",
      " [ 3  5  9 11  3  2]\n",
      " [ 3  5 16 18  3  2]\n",
      " [ 9 11 16 18  3  2]\n",
      " [ 7  9 14 16  3  3]\n",
      " [ 7 10 14 17  4  1]\n",
      " [ 7 11 14 18  5  1]\n",
      " [ 8 12 15 19  5  2]\n",
      " [ 7 12 14 19  6  1]]\n"
     ]
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "f9558a7a9c0790c2334a9072034da5d30cf10fd46d803cfe3584dbca0180200c"
  },
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}