{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.83 s, sys: 1.16 s, total: 5.99 s\n",
      "Wall time: 6.98 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import malaya"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "string1 = 'Sis, students from overseas were brought back because they are not in their countries which is if something happens to them, its not the other countries’ responsibility. Student dalam malaysia ni dah dlm tggjawab kerajaan. Mana part yg tak faham?'\n",
    "string2 = 'Harap kerajaan tak bukak serentak. Slowly release week by week. Focus on economy related industries dulu'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load multinomial model\n",
    "\n",
    "All model interface will follow sklearn interface started v3.4,\n",
    "\n",
    "```python\n",
    "model.predict(List[str])\n",
    "\n",
    "model.predict_proba(List[str])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = malaya.sentiment.multinomial()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['positive', 'negative']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict([string1, string2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.008184343650433397,\n",
       "  'positive': 0.18156563495665812,\n",
       "  'neutral': 0.8102500213929085},\n",
       " {'negative': 0.010056240383248257,\n",
       "  'positive': 0.009943759616751778,\n",
       "  'neutral': 0.98}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba([string1, string2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Disable `neutral` probability,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.40921718252166983, 'positive': 0.5907828174783291},\n",
       " {'negative': 0.5028120191624128, 'positive': 0.49718798083758886}]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba([string1, string2], add_neutral = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List available Transformer models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'bert': ['425.6 MB', 'accuracy: 0.993'],\n",
       " 'tiny-bert': ['57.4 MB', 'accuracy: 0.987'],\n",
       " 'albert': ['48.6 MB', 'accuracy: 0.992'],\n",
       " 'tiny-albert': ['22.4 MB', 'accuracy: 0.985'],\n",
       " 'xlnet': ['446.5 MB', 'accuracy: 0.993'],\n",
       " 'alxlnet': ['46.8 MB', 'accuracy: 0.991']}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.sentiment.available_transformer_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure you can check accuracy chart from here first before select a model, https://malaya.readthedocs.io/en/latest/Accuracy.html#sentiment-analysis\n",
    "\n",
    "**You might want to use Tiny-Albert, a very small size, 22.4MB, but the accuracy is still on the top notch.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load XLNET model\n",
    "\n",
    "All model interface will follow sklearn interface started v3.4,\n",
    "\n",
    "```python\n",
    "model.predict(List[str])\n",
    "\n",
    "model.predict_proba(List[str])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:54: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
      "\n",
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:55: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.\n",
      "\n",
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:49: The name tf.InteractiveSession is deprecated. Please use tf.compat.v1.InteractiveSession instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = malaya.sentiment.transformer(model = 'xlnet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Predict batch of strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.00018888633, 'positive': 0.9811114, 'neutral': 0.018699706},\n",
       " {'negative': 0.8079505, 'positive': 0.0019204962, 'neutral': 0.19012898}]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba([string1, string2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.029847767, 'positive': 0.97015226},\n",
       " {'negative': 0.1034979, 'positive': 0.89650214}]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba([string1, string2], add_neutral = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Open emotion visualization dashboard\n",
    "\n",
    "Default when you call `predict_words` it will open a browser with visualization dashboard, you can disable by `visualization=False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.predict_words(string1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 307020 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {
      "image/png": {
       "width": 800
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import Image, display\n",
    "\n",
    "display(Image('sentiment-dashboard.png', width=800))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking models\n",
    "\n",
    "More information, you can read at [https://malaya.readthedocs.io/en/latest/Stack.html](https://malaya.readthedocs.io/en/latest/Stack.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "multinomial = malaya.sentiment.multinomial()\n",
    "alxlnet = malaya.sentiment.transformer(model = 'alxlnet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.0005453552136673502,\n",
       "  'positive': 0.5603020846001405,\n",
       "  'neutral': 0.05399025419995675},\n",
       " {'negative': 0.0002248290781177622,\n",
       "  'positive': 0.21361579430243546,\n",
       "  'neutral': 0.022142383292097452}]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.stack.predict_stack([multinomial, alxlnet, model], [string1, string2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you do not want neutral in `predict_stack`, simply override the parameter,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'negative': 0.05828375571937787, 'positive': 0.8221586003437801},\n",
       " {'negative': 0.014352668987571138, 'positive': 0.7835866999009022}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.stack.predict_stack([multinomial, alxlnet, model], [string1, string2], add_neutral = False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
