{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 11.4 s, sys: 1.39 s, total: 12.8 s\n",
      "Wall time: 16.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import malaya"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is word mover distance?\n",
    "\n",
    "<img src=\"https://vene.ro/images/wmd-obama.png\" width=\"40%\" align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "between two documents in a meaningful way, even when they have no words in common. It uses vector embeddings of words. It been shown to outperform many of the state-of-the-art methods in k-nearest neighbors classification.\n",
    "\n",
    "You can read more about word mover distance from [Word Distance between Word Embeddings](https://towardsdatascience.com/word-distance-between-word-embeddings-cc3e9cf1d632).\n",
    "\n",
    "**Closest to 0 is better**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "left_sentence = 'saya suka makan ayam'\n",
    "right_sentence = 'saya suka makan ikan'\n",
    "left_token = left_sentence.split()\n",
    "right_token = right_sentence.split()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "w2v_wiki = malaya.word2vec.load_wiki()\n",
    "w2v_wiki = malaya.word2vec.word2vec(w2v_wiki['nce_weights'],w2v_wiki['dictionary'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "fasttext_wiki, ngrams = malaya.fast_text.load_wiki()\n",
    "fasttext_wiki = malaya.fast_text.fast_text(fasttext_wiki['embed_weights'],\n",
    "                                           fasttext_wiki['dictionary'], ngrams)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using word2vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8225146532058716"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.word_mover.distance(left_token, right_token, w2v_wiki)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.word_mover.distance(left_token, left_token, w2v_wiki)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using fast-text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.82466983795166"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.word_mover.distance(left_token, right_token, fasttext_wiki)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.word_mover.distance(left_token, left_token, fasttext_wiki)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why word mover distance?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maybe you heard about skipthought or siamese network to train sentences similarity, but both required a good corpus plus really slow to train. Malaya provided both models to train your own text similarity, can check here, [Malaya text-similarity](https://malaya.readthedocs.io/en/latest/Similarity.html)\n",
    "\n",
    "`word2vec` or `fast-text` are really good to know semantic definitions between 2 words, like below,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['zaid', 0.7285637855529785],\n",
       " ['khairy', 0.6839416027069092],\n",
       " ['zabidi', 0.6709405183792114],\n",
       " ['nizar', 0.6695379018783569],\n",
       " ['harussani', 0.6595045328140259],\n",
       " ['shahidan', 0.6565827131271362],\n",
       " ['azalina', 0.6541041135787964],\n",
       " ['shahrizat', 0.6538639068603516]]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w2v_wiki.n_closest(word = 'anwar', num_closest=8, metric='cosine')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we got some suggestion from the interface included distance between 0-1, closest to 1 is better.\n",
    "\n",
    "Now let say I want to compare similarity between 2 sentences, and using vectors representation from our word2vec and fast-text.\n",
    "\n",
    "I got, `rakyat sebenarnya sukakan mahathir`, and `rakyat sebenarnya sukakan najib`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9017602205276489"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mahathir = 'rakyat sebenarnya sukakan mahathir'\n",
    "najib = 'rakyat sebenarnya sukakan najib'\n",
    "malaya.word_mover.distance(mahathir.split(), najib.split(), w2v_wiki)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "0.9, quite good. What happen if we make our sentence quite polarity ambigious for najib? (Again, this is just example)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.7690724730491638"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mahathir = 'rakyat sebenarnya sukakan mahathir'\n",
    "najib = 'rakyat sebenarnya gilakan najib'\n",
    "malaya.word_mover.distance(mahathir.split(), najib.split(), w2v_wiki)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We just changed `sukakan` with `gilakan`, but our word2vec representation based on `rakyat sebenarnya <word> <person>` not able to correlate same polarity, real definition of `gilakan` is positive polarity, but word2vec learnt `gilakan` is negative or negate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Soft mode\n",
    "\n",
    "What happened if a word is not inside vectorizer dictionary? `malaya.word_mover.distance` will throw an exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "Exception",
     "evalue": "input not found in dictionary, here top-5 nearest words [qw, qe, we, qwest, qwabe]",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mException\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m~/Documents/Malaya/malaya/word_mover.py\u001b[0m in \u001b[0;36m_word_mover\u001b[0;34m(left_token, right_token, vectorizer, soft)\u001b[0m\n\u001b[1;32m     45\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 46\u001b[0;31m             \u001b[0mwordvecs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtoken\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_vector_by_name\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtoken\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     47\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Documents/Malaya/malaya/word2vec.py\u001b[0m in \u001b[0;36mget_vector_by_name\u001b[0;34m(self, word)\u001b[0m\n\u001b[1;32m    289\u001b[0m                 \u001b[0;34m'input not found in dictionary, here top-5 nearest words [%s]'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 290\u001b[0;31m                 \u001b[0;34m%\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mstrings\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    291\u001b[0m             )\n",
      "\u001b[0;31mException\u001b[0m: input not found in dictionary, here top-5 nearest words [qw, qe, we, qwest, qwabe]",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[0;31mException\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-13-4acdc71ff70d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mleft\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'tyi'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mright\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'qwe'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mmalaya\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mword_mover\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdistance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mleft\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mright\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw2v_wiki\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Documents/Malaya/malaya/word_mover.py\u001b[0m in \u001b[0;36mdistance\u001b[0;34m(left_token, right_token, vectorizer, soft)\u001b[0m\n\u001b[1;32m    111\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvectorizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'get_vector_by_name'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    112\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'vectorizer must has `get_vector_by_name` method'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 113\u001b[0;31m     \u001b[0mprob\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_word_mover\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mleft_token\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mright_token\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msoft\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msoft\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    114\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mpulp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobjective\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    115\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Documents/Malaya/malaya/word_mover.py\u001b[0m in \u001b[0;36m_word_mover\u001b[0;34m(left_token, right_token, vectorizer, soft)\u001b[0m\n\u001b[1;32m     47\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     48\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0msoft\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 49\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0mException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     50\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     51\u001b[0m                 \u001b[0marr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfuzz\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mratio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtoken\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvectorizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwords\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mException\u001b[0m: input not found in dictionary, here top-5 nearest words [qw, qe, we, qwest, qwabe]"
     ]
    }
   ],
   "source": [
    "left = 'tyi'\n",
    "right = 'qwe'\n",
    "malaya.word_mover.distance(left.split(), right.split(), w2v_wiki)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So if use `soft = True`, if the word is not inside vectorizer, it will find the nearest word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.273216962814331"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "left = 'tyi'\n",
    "right = 'qwe'\n",
    "malaya.word_mover.distance(left.split(), right.split(), w2v_wiki, soft = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load expander\n",
    "\n",
    "We want to expand shortforms based on `malaya.normalize.spell` by using word mover distance. If our vector knows that `mkn` semantically similar to `makan` based on `saya suka mkn ayam` sentence, word mover distance will become closer.\n",
    "\n",
    "It is really depends on our vector, and word2vec may not able to understand shortform, so we will use fast-text to fix `OUT-OF-VOCAB` problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "downloading Malay texts\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "1.00MB [00:00, 1.70MB/s]                   \n"
     ]
    }
   ],
   "source": [
    "malays = malaya.load_malay_dictionary()\n",
    "wiki, ngrams = malaya.fast_text.load_wiki()\n",
    "fast_text_embed = malaya.fast_text.fast_text(wiki['embed_weights'],wiki['dictionary'],ngrams)\n",
    "expander = malaya.word_mover.expander(malays, fast_text_embed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "string = 'y u xsuka makan HUSEIN kt situ tmpt'\n",
    "another = 'i mmg xska mknn HUSEIN kampng tempt'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tut',\n",
       "   0.8088938253521919),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tuit',\n",
       "   0.863929785296917),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tat',\n",
       "   0.8680638003787995),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ top',\n",
       "   0.8688952446055412),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tip',\n",
       "   0.8978437346220016),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ taat',\n",
       "   0.936883625289917),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ topi',\n",
       "   0.9442774548711776),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tumit',\n",
       "   0.9495834815340042),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tempe',\n",
       "   0.9758907731723786),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ ampe',\n",
       "   0.9821926467533112),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tempo',\n",
       "   0.9836614096956253),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tepet',\n",
       "   0.994007917971611),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ amit',\n",
       "   0.9999424153804779),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tuat',\n",
       "   1.0002889167022706),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ mat',\n",
       "   1.0071370331926346),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ temut',\n",
       "   1.011553812426567),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ ampit',\n",
       "   1.022653616695404),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ ampo',\n",
       "   1.0231078831071854),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tipu',\n",
       "   1.0246861065587998),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tepi',\n",
       "   1.0285266551542283),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ umut',\n",
       "   1.0287358275117875),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ emat',\n",
       "   1.0357482937116622),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ empat',\n",
       "   1.0431590774860382),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tapi',\n",
       "   1.0562509994459153),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tepu',\n",
       "   1.0601519473543166),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tumpat',\n",
       "   1.074669928882599),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ impi',\n",
       "   1.078846170501709),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ umat',\n",
       "   1.0791117155513763),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tampi',\n",
       "   1.0883281208925248),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tumpu',\n",
       "   1.091578345676422),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ umpat',\n",
       "   1.092372225769043),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tepat',\n",
       "   1.0979607516746521),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tampa',\n",
       "   1.1118229238204955),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ amput',\n",
       "   1.1226389572820663),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tapa',\n",
       "   1.129335333744049),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ timpa',\n",
       "   1.1353471846590042),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ empu',\n",
       "   1.1459274488725661),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tempa',\n",
       "   1.164648480837822),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tampu',\n",
       "   1.1812463180065156),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tempat',\n",
       "   1.1856716803007126),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ tamat',\n",
       "   1.2068403679332733),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ amat',\n",
       "   1.2214121790246963),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ ampu',\n",
       "   1.2350379461402894),\n",
       "  ('tmpt',\n",
       "   'kenapa awak tak suka makan Husein kat situ taut',\n",
       "   1.2796957146606445)]]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "expander.expand(string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[('ska', 'saya memang tak soka mknn Husein kampng tempt', 0.7199365496635437),\n",
       "  ('ska', 'saya memang tak suka mknn Husein kampng tempt', 0.8050327301025391),\n",
       "  ('ska', 'saya memang tak sika mknn Husein kampng tempt', 0.8729341626167297),\n",
       "  ('ska', 'saya memang tak saka mknn Husein kampng tempt', 0.875930666923523),\n",
       "  ('ska', 'saya memang tak spa mknn Husein kampng tempt', 0.8995948433876038),\n",
       "  ('ska', 'saya memang tak sua mknn Husein kampng tempt', 0.9496822357177734),\n",
       "  ('ska', 'saya memang tak seka mknn Husein kampng tempt', 0.9891390204429626),\n",
       "  ('ska', 'saya memang tak ski mknn Husein kampng tempt', 1.1318669319152832),\n",
       "  ('ska', 'saya memang tak sia mknn Husein kampng tempt', 1.1666431427001953)],\n",
       " [('mknn', 'saya memang tak ska min Husein kampng tempt', 0.8653836846351624),\n",
       "  ('mknn', 'saya memang tak ska maun Husein kampng tempt', 1.045318603515625),\n",
       "  ('mknn', 'saya memang tak ska kun Husein kampng tempt', 1.0710314512252808),\n",
       "  ('mknn', 'saya memang tak ska ken Husein kampng tempt', 1.0728274583816528),\n",
       "  ('mknn', 'saya memang tak ska kon Husein kampng tempt', 1.0992072820663452),\n",
       "  ('mknn', 'saya memang tak ska ikon Husein kampng tempt', 1.1365187168121338),\n",
       "  ('mknn', 'saya memang tak ska makin Husein kampng tempt', 1.180336833000183),\n",
       "  ('mknn', 'saya memang tak ska main Husein kampng tempt', 1.182568907737732),\n",
       "  ('mknn', 'saya memang tak ska makan Husein kampng tempt', 1.183489203453064),\n",
       "  ('mknn', 'saya memang tak ska makna Husein kampng tempt', 1.184565544128418),\n",
       "  ('mknn', 'saya memang tak ska kan Husein kampng tempt', 1.2368937730789185),\n",
       "  ('mknn', 'saya memang tak ska akan Husein kampng tempt', 1.2527291774749756),\n",
       "  ('mknn', 'saya memang tak ska mani Husein kampng tempt', 1.266147494316101),\n",
       "  ('mknn', 'saya memang tak ska ikan Husein kampng tempt', 1.2773109674453735),\n",
       "  ('mknn', 'saya memang tak ska mini Husein kampng tempt', 1.3020210266113281),\n",
       "  ('mknn', 'saya memang tak ska mana Husein kampng tempt', 1.3099677562713623),\n",
       "  ('mknn', 'saya memang tak ska menu Husein kampng tempt', 1.3974181413650513),\n",
       "  ('mknn', 'saya memang tak ska mena Husein kampng tempt', 1.404064655303955),\n",
       "  ('mknn',\n",
       "   'saya memang tak ska makanan Husein kampng tempt',\n",
       "   1.4473483562469482)],\n",
       " [('kampng',\n",
       "   'saya memang tak ska mknn Husein kampung tempt',\n",
       "   0.9272603988647461)],\n",
       " [('tempt',\n",
       "   'saya memang tak ska mknn Husein kampng tempo',\n",
       "   0.7405402660369873),\n",
       "  ('tempt',\n",
       "   'saya memang tak ska mknn Husein kampng tempe',\n",
       "   0.7510019540786743),\n",
       "  ('tempt', 'saya memang tak ska mknn Husein kampng tempa', 0.885798454284668),\n",
       "  ('tempt',\n",
       "   'saya memang tak ska mknn Husein kampng temut',\n",
       "   0.9036741256713867),\n",
       "  ('tempt',\n",
       "   'saya memang tak ska mknn Husein kampng tempat',\n",
       "   0.9161624312400818)]]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "expander.expand(another)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
