{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4.26 s, sys: 828 ms, total: 5.09 s\n",
      "Wall time: 4.26 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import malaya"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "anger_text = 'babi la company ni, aku dah la penat datang dari jauh'\n",
    "fear_text = 'takut doh tengok cerita hantu tadi'\n",
    "happy_text = 'bestnya dapat tidur harini, tak payah pergi kerja'\n",
    "love_text = 'aku sayang sgt dia dah doh'\n",
    "sadness_text = 'kecewa tengok kerajaan baru ni, janji ape pun tak dapat'\n",
    "surprise_text = 'sakit jantung aku, terkejut dengan cerita hantu tadi'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['anger', 'fear', 'happy', 'love', 'sadness', 'surprise']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.emotion.label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All models follow same method as sklearn interface, `predict` to get batch of labels, `predict_proba` to get batch of probabilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load multinomial model\n",
    "\n",
    "All model interface will follow sklearn interface started v3.4,\n",
    "\n",
    "```python\n",
    "model.predict(List[str])\n",
    "\n",
    "model.predict_proba(List[str])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = malaya.emotion.multinomial()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['anger']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict([anger_text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['anger', 'fear', 'happy', 'love', 'sadness', 'surprise']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(\n",
    "    [anger_text, fear_text, happy_text, love_text, sadness_text, surprise_text]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'anger': 0.32948272681734814,\n",
       "  'fear': 0.13959708810717708,\n",
       "  'happy': 0.14671455153216045,\n",
       "  'love': 0.12489192355631354,\n",
       "  'sadness': 0.1285972541671178,\n",
       "  'surprise': 0.13071645581988448},\n",
       " {'anger': 0.11379406005377896,\n",
       "  'fear': 0.4006934391283133,\n",
       "  'happy': 0.11389665647702245,\n",
       "  'love': 0.12481915233837086,\n",
       "  'sadness': 0.0991261507380643,\n",
       "  'surprise': 0.14767054126445014},\n",
       " {'anger': 0.15051890586527464,\n",
       "  'fear': 0.13931406415515296,\n",
       "  'happy': 0.32037710031973415,\n",
       "  'love': 0.13747954667255546,\n",
       "  'sadness': 0.11565866743099411,\n",
       "  'surprise': 0.13665171555628927},\n",
       " {'anger': 0.1590563839629243,\n",
       "  'fear': 0.14687344690114268,\n",
       "  'happy': 0.1419948160674701,\n",
       "  'love': 0.279550441361504,\n",
       "  'sadness': 0.1285927908584157,\n",
       "  'surprise': 0.14393212084854254},\n",
       " {'anger': 0.14268176425895224,\n",
       "  'fear': 0.12178299725318226,\n",
       "  'happy': 0.16187751258299898,\n",
       "  'love': 0.1030494733572262,\n",
       "  'sadness': 0.34277869755707796,\n",
       "  'surprise': 0.1278295549905621},\n",
       " {'anger': 0.06724850384395685,\n",
       "  'fear': 0.1283628050361525,\n",
       "  'happy': 0.05801958643852813,\n",
       "  'love': 0.06666524240157067,\n",
       "  'sadness': 0.06537667186293224,\n",
       "  'surprise': 0.6143271904168589}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba(\n",
    "    [anger_text, fear_text, happy_text, love_text, sadness_text, surprise_text]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List available Transformer models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'bert': ['425.6 MB', 'accuracy: 0.992'],\n",
       " 'tiny-bert': ['57.4 MB', 'accuracy: 0.988'],\n",
       " 'albert': ['48.6 MB', 'accuracy: 0.997'],\n",
       " 'tiny-albert': ['22.4 MB', 'accuracy: 0.981'],\n",
       " 'xlnet': ['446.5 MB', 'accuracy: 0.990'],\n",
       " 'alxlnet': ['46.8 MB', 'accuracy: 0.989']}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.emotion.available_transformer_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure you can check accuracy chart from here first before select a model, https://malaya.readthedocs.io/en/latest/Accuracy.html#emotion-analysis\n",
    "\n",
    "**You might want to use Tiny-Albert, a very small size, 22.4MB, but the accuracy is still on the top notch.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Albert model\n",
    "\n",
    "All model interface will follow sklearn interface started v3.4,\n",
    "\n",
    "```python\n",
    "model.predict(List[str])\n",
    "\n",
    "model.predict_proba(List[str])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:54: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
      "\n",
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:55: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/albert/tokenization.py:240: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
      "\n",
      "INFO:tensorflow:loading sentence piece model\n",
      "WARNING:tensorflow:From /Users/huseinzolkepli/Documents/Malaya/malaya/function/__init__.py:49: The name tf.InteractiveSession is deprecated. Please use tf.compat.v1.InteractiveSession instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = malaya.emotion.transformer(model = 'albert')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Predict batch of strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'anger': 0.9998901,\n",
       "  'fear': 3.2524113e-05,\n",
       "  'happy': 2.620931e-05,\n",
       "  'love': 2.2871463e-05,\n",
       "  'sadness': 9.782951e-06,\n",
       "  'surprise': 1.8502667e-05},\n",
       " {'anger': 1.6941378e-05,\n",
       "  'fear': 0.9999205,\n",
       "  'happy': 9.070281e-06,\n",
       "  'love': 2.044179e-05,\n",
       "  'sadness': 6.7731107e-06,\n",
       "  'surprise': 2.6314676e-05},\n",
       " {'anger': 0.15370166,\n",
       "  'fear': 0.0013852724,\n",
       "  'happy': 0.8268689,\n",
       "  'love': 0.011433229,\n",
       "  'sadness': 0.0011807577,\n",
       "  'surprise': 0.005430276},\n",
       " {'anger': 1.2597201e-05,\n",
       "  'fear': 1.7600481e-05,\n",
       "  'happy': 9.667115e-06,\n",
       "  'love': 0.9999331,\n",
       "  'sadness': 1.3735416e-05,\n",
       "  'surprise': 1.3399296e-05},\n",
       " {'anger': 1.9176923e-05,\n",
       "  'fear': 1.1163729e-05,\n",
       "  'happy': 6.353941e-06,\n",
       "  'love': 7.004002e-06,\n",
       "  'sadness': 0.99994576,\n",
       "  'surprise': 1.0511084e-05},\n",
       " {'anger': 5.8739704e-05,\n",
       "  'fear': 1.9771342e-05,\n",
       "  'happy': 1.8316741e-05,\n",
       "  'love': 2.2319455e-05,\n",
       "  'sadness': 3.646786e-05,\n",
       "  'surprise': 0.9998443}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba(\n",
    "    [anger_text, fear_text, happy_text, love_text, sadness_text, surprise_text]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Open emotion visualization dashboard\n",
    "\n",
    "Default when you call `predict_words` it will open a browser with visualization dashboard, you can disable by `visualization=False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.predict_words(sadness_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 242996 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {
      "image/png": {
       "width": 800
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import Image, display\n",
    "\n",
    "display(Image('emotion-dashboard.png', width=800))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking models\n",
    "\n",
    "More information, you can read at [https://malaya.readthedocs.io/en/latest/Stack.html](https://malaya.readthedocs.io/en/latest/Stack.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "multinomial = malaya.emotion.multinomial()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'anger': 0.5739743139312979,\n",
       "  'fear': 0.002130791264743306,\n",
       "  'happy': 0.0019609404077070573,\n",
       "  'love': 0.0016901068202818533,\n",
       "  'sadness': 0.001121633002361737,\n",
       "  'surprise': 0.0015551851123993595}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.stack.predict_stack([multinomial, model], [anger_text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'anger': 0.5739743139312979,\n",
       "  'fear': 0.002130791264743306,\n",
       "  'happy': 0.0019609404077070573,\n",
       "  'love': 0.0016901068202818533,\n",
       "  'sadness': 0.001121633002361737,\n",
       "  'surprise': 0.0015551858768478731},\n",
       " {'anger': 0.0016541454680912208,\n",
       "  'fear': 0.0011659984542562358,\n",
       "  'happy': 0.001014179551389293,\n",
       "  'love': 0.0008495638318424924,\n",
       "  'sadness': 0.5854571761989077,\n",
       "  'surprise': 0.001159149836587787}]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "malaya.stack.predict_stack([multinomial, model], [anger_text, sadness_text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
