{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default_exp models.ar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AR\n",
    "> Association Rules\n",
    "\n",
    "Simple Association Rules (AR) are a simplified version of the association rule mining technique [Agrawal et al. 1993] with a maximum rule size of two. The method is designed to capture the frequency of two co-occurring events, e.g., “Customers who bought . . . also bought”. Algorithmically, the rules and their corresponding importance are “learned” by counting how often the items i and j occurred together in a session of any user. Let a session s be a chronologically ordered tuple of item click events s = ($s_1$,$s_2$,$s_3$, . . . ,$s_m$) and $S_p$ the set of all past sessions. Given a user’s current session s with $s_{|s|}$ being the last item interaction in s, we can define the score for a recommendable item i as follows, where the indicator function $1_{EQ}(a,b)$ is 1 in case a and b refer to the same item and 0 otherwise.\n",
    "\n",
    "$$score_{AR}(i,s) = \\dfrac{1}{\\sum_{p \\in S_p}\\sum_{x=1}^{|p|}1_{EQ}(s_{|s|},p_x)\\cdot(|p|-1)}\\sum_{p \\in s_p}\\sum_{x=1}^{|p|}\\sum_{y=1}^{|p|}1_{EQ}(s_{|s|},p_x)\\cdot1_{EQ}(i,p_y)$$\n",
    "\n",
    "In the above equation, the sums at the right-hand side represent the counting scheme. The term at the left-hand side normalizes the score by the number of total rule occurrences originating from the current item $s_{|s|}$. A list of recommendations returned by the ar method then contains the items with the highest scores in descending order. No minimum support or confidence thresholds are applied."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "from nbdev.showdoc import *\n",
    "from fastcore.nb_imports import *\n",
    "from fastcore.test import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import collections as col"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class AssosiationRules: \n",
    "    '''\n",
    "    AssosiationRules(pruning=10, session_key='SessionId', item_keys=['ItemId'])\n",
    "    Parameters\n",
    "    --------\n",
    "    pruning : int\n",
    "        Prune the results per item to a list of the top N co-occurrences. (Default value: 10)\n",
    "    session_key : string\n",
    "        The data frame key for the session identifier. (Default value: SessionId)\n",
    "    item_keys : string\n",
    "        The data frame list of keys for the item identifier as first item in list \n",
    "        and features keys next. (Default value: [ItemId])\n",
    "    '''\n",
    "    def __init__( self, pruning=10, session_key='SessionID', item_keys=['ItemID'] ):\n",
    "        self.pruning = pruning\n",
    "        self.session_key = session_key\n",
    "        self.item_keys = item_keys\n",
    "        self.items_features = {}\n",
    "        self.predict_for_item_ids = []\n",
    "        \n",
    "    def fit( self, data):\n",
    "        '''\n",
    "        Trains the predictor.\n",
    "        \n",
    "        Parameters\n",
    "        --------\n",
    "        data: pandas.DataFrame\n",
    "            Training data. It contains the transactions of the sessions. \n",
    "            It has one column for session IDs, one for item IDs and many for the\n",
    "            item features if exist.\n",
    "            It must have a header. Column names are arbitrary, but must \n",
    "            correspond to the ones you set during the initialization of the \n",
    "            network (session_key, item_keys).\n",
    "        '''\n",
    "        cur_session = -1\n",
    "        last_items = []\n",
    "        all_rules = []\n",
    "        indices_item = []\n",
    "        for i in self.item_keys:\n",
    "            all_rules.append(dict())\n",
    "            indices_item.append( data.columns.get_loc(i) )\n",
    "\n",
    "        data.sort_values(self.session_key, inplace=True)\n",
    "        index_session = data.columns.get_loc(self.session_key)\n",
    "        \n",
    "        #Create Dictionary of items and their features\n",
    "        for row in data.itertuples( index=False ):\n",
    "            item_id = row[indices_item[0]]\n",
    "            if not item_id in self.items_features.keys() :\n",
    "                self.items_features[item_id] = []\n",
    "                for i in indices_item:\n",
    "                    self.items_features[item_id].append(row[i])\n",
    "              \n",
    "        for i in range(len(self.item_keys)):\n",
    "            rules = all_rules[i]\n",
    "            index_item = indices_item[i]\n",
    "            for row in data.itertuples( index=False ):\n",
    "                session_id, item_id = row[index_session], row[index_item]\n",
    "                if session_id != cur_session:\n",
    "                    cur_session = session_id\n",
    "                    last_items = []\n",
    "                else: \n",
    "                    for item_id2 in last_items:                \n",
    "                        if not item_id in rules :\n",
    "                            rules[item_id] = dict()                \n",
    "                        if not item_id2 in rules :\n",
    "                            rules[item_id2] = dict()                \n",
    "                        if not item_id in rules[item_id2]:\n",
    "                            rules[item_id2][item_id] = 0           \n",
    "                        if not item_id2 in rules[item_id]:\n",
    "                            rules[item_id][item_id2] = 0\n",
    "                        \n",
    "                        rules[item_id][item_id2] += 1\n",
    "                        rules[item_id2][item_id] += 1\n",
    "                        \n",
    "                last_items.append(item_id)\n",
    "                \n",
    "            if self.pruning > 0:\n",
    "                rules = self.prune(rules) \n",
    "                \n",
    "            all_rules[i] = rules\n",
    "        self.all_rules = all_rules\n",
    "        self.predict_for_item_ids = list(self.all_rules[0].keys())\n",
    "    def predict_next(self, session_items, k = 20):\n",
    "        '''\n",
    "        Gives predicton scores for a selected set of items on how likely they be the next item in the session.\n",
    "                \n",
    "        Parameters\n",
    "        --------\n",
    "        session_items : List\n",
    "            Items IDs in current session.\n",
    "        k : Integer\n",
    "            How many items to recommend\n",
    "        Returns\n",
    "        --------\n",
    "        out : pandas.Series\n",
    "            Prediction scores for selected items on how likely to be the next item of this session. \n",
    "            Indexed by the item IDs.\n",
    "        \n",
    "        '''\n",
    "        all_len = len(self.predict_for_item_ids)\n",
    "        input_item_id = session_items[-1]\n",
    "        preds = np.zeros( all_len ) \n",
    "             \n",
    "        if input_item_id in self.all_rules[0].keys():\n",
    "            for k_ind in range(all_len):\n",
    "                key = self.predict_for_item_ids[k_ind]\n",
    "                if key in session_items:\n",
    "                    continue\n",
    "                try:\n",
    "                    preds[ k_ind ] += self.all_rules[0][input_item_id][key]\n",
    "                except:\n",
    "                    pass\n",
    "                for i in range(1, len(self.all_rules)):\n",
    "                    input_item_feature = self.items_features[input_item_id][i]\n",
    "                    key_feature = self.items_features[key][i]\n",
    "                    try:\n",
    "                        preds[ k_ind ] += self.all_rules[i][input_item_feature][key_feature]\n",
    "                    except:\n",
    "                        pass\n",
    "        \n",
    "        series = pd.Series(data=preds, index=self.predict_for_item_ids)\n",
    "        series = series / series.max()\n",
    "        \n",
    "        return series.nlargest(k).index.values\n",
    "    \n",
    "    def prune(self, rules): \n",
    "        '''\n",
    "        Gives predicton scores for a selected set of items on how likely they be the next item in the session.\n",
    "        Parameters\n",
    "            --------\n",
    "            rules : dict of dicts\n",
    "                The rules mined from the training data\n",
    "        '''\n",
    "        for k1 in rules:\n",
    "            tmp = rules[k1]\n",
    "            if self.pruning < 1:\n",
    "                keep = len(tmp) - int( len(tmp) * self.pruning )\n",
    "            elif self.pruning >= 1:\n",
    "                keep = self.pruning\n",
    "            counter = col.Counter( tmp )\n",
    "            rules[k1] = dict()\n",
    "            for k2, v in counter.most_common( keep ):\n",
    "                rules[k1][k2] = v\n",
    "        return rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import argparse\n",
    "import pandas as pd\n",
    "from recohut.utils.common_utils import download_url"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading https://github.com/RecoHut-Datasets/yoochoose/raw/v4/yoochoose_train.txt\n",
      "Downloading https://github.com/RecoHut-Datasets/yoochoose/raw/v4/yoochoose_valid.txt\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'/content/data/yoochoose_valid.txt'"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_root = '/content/data'\n",
    "download_url('https://github.com/RecoHut-Datasets/yoochoose/raw/v4/yoochoose_train.txt', data_root)\n",
    "download_url('https://github.com/RecoHut-Datasets/yoochoose/raw/v4/yoochoose_valid.txt', data_root)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished Reading Data.\n",
      "Start Model Fitting...\n",
      "End Model Fitting with total time = 47.870760679244995\n",
      "Start Predictions...\n",
      "Finished Prediction for  5000 items.\n",
      "Recall: 0.26574500768049153\n",
      "\n",
      "MRR: 0.1308005998098165\n",
      "End Model Predictions with total time = 110.56373572349548\n"
     ]
    }
   ],
   "source": [
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--prune', type=int, default=10, help=\"Association Rules Pruning Parameter\")\n",
    "parser.add_argument('--K', type=int, default=20, help=\"K items to be used in Recall@K and MRR@K\")\n",
    "parser.add_argument('--itemid', default='sid', type=str)\n",
    "parser.add_argument('--sessionid', default='uid', type=str)\n",
    "parser.add_argument('--item_feats', default='', type=str, \n",
    "                    help=\"Names of Columns containing items features separated by #\")\n",
    "parser.add_argument('--valid_data', default='yoochoose_valid.txt', type=str)\n",
    "parser.add_argument('--train_data', default='yoochoose_train.txt', type=str)\n",
    "parser.add_argument('--data_folder', default=data_root, type=str)\n",
    "\n",
    "# Get the arguments\n",
    "args = parser.parse_args([])\n",
    "train_data = os.path.join(args.data_folder, args.train_data)\n",
    "x_train = pd.read_csv(train_data)\n",
    "valid_data = os.path.join(args.data_folder, args.valid_data)\n",
    "x_valid = pd.read_csv(valid_data)\n",
    "x_valid.sort_values(args.sessionid, inplace=True)\n",
    "\n",
    "items_feats = [args.itemid]\n",
    "ffeats = args.item_feats.strip().split(\"#\")\n",
    "if ffeats[0] != '':\n",
    "    items_feats.extend(ffeats)\n",
    "\n",
    "print('Finished Reading Data.')\n",
    "# Fitting AR Model\n",
    "print('Start Model Fitting...')\n",
    "t1 = time.time()\n",
    "model = AssosiationRules(session_key = args.sessionid, item_keys = items_feats, pruning=args.prune)\n",
    "model.fit(x_train)\n",
    "t2 = time.time()\n",
    "print('End Model Fitting with total time =', t2 - t1)\n",
    "\n",
    "print('Start Predictions...')\n",
    "# Test Set Evaluation\n",
    "test_size = 0.0\n",
    "hit = 0.0\n",
    "MRR = 0.0\n",
    "cur_length = 0\n",
    "cur_session = -1\n",
    "last_items = []\n",
    "t1 = time.time()\n",
    "index_item = x_valid.columns.get_loc(args.itemid)\n",
    "index_session = x_valid.columns.get_loc(args.sessionid)\n",
    "train_items = model.items_features.keys()\n",
    "counter = 0\n",
    "for row in x_valid.itertuples(index=False):\n",
    "    counter += 1\n",
    "    if counter % 5000 == 0:\n",
    "        print('Finished Prediction for ', counter, 'items.')\n",
    "    session_id, item_id = row[index_session], row[index_item]\n",
    "    if session_id != cur_session:\n",
    "        cur_session = session_id\n",
    "        last_items = []\n",
    "        cur_length = 0\n",
    "    \n",
    "    if not item_id in last_items and item_id in train_items:\n",
    "        if len(last_items) > cur_length: #make prediction\n",
    "            cur_length += 1\n",
    "            test_size += 1\n",
    "            # Predict the most similar items to items\n",
    "            predictions = model.predict_next(last_items, k = args.K)\n",
    "            #print('preds:', predictions)\n",
    "            # Evaluation\n",
    "            rank = 0\n",
    "            for predicted_item in predictions:\n",
    "                rank += 1\n",
    "                if predicted_item == item_id:\n",
    "                    hit += 1.0\n",
    "                    MRR += 1/rank\n",
    "                    break\n",
    "        \n",
    "        last_items.append(item_id)\n",
    "t2 = time.time()\n",
    "print('Recall: {}'.format(hit / test_size))\n",
    "print ('\\nMRR: {}'.format(MRR / test_size))\n",
    "print('End Model Predictions with total time =', t2 - t1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **References:-**\n",
    "- [https://arxiv.org/pdf/1803.09587.pdf](https://arxiv.org/pdf/1803.09587.pdf)\n",
    "- [http://www.rakesh.agrawal-family.com/papers/sigmod93assoc.pdf](http://www.rakesh.agrawal-family.com/papers/sigmod93assoc.pdf)\n",
    "- [https://github.com/mmaher22/iCV-SBR/tree/master/Source Codes/AR%26SR_Python](https://github.com/mmaher22/iCV-SBR/tree/master/Source%20Codes/AR%26SR_Python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Author: Sparsh A.\n",
      "\n",
      "Last updated: 2022-01-01 06:12:09\n",
      "\n",
      "Compiler    : GCC 7.5.0\n",
      "OS          : Linux\n",
      "Release     : 5.4.144+\n",
      "Machine     : x86_64\n",
      "Processor   : x86_64\n",
      "CPU cores   : 2\n",
      "Architecture: 64bit\n",
      "\n",
      "pandas  : 1.1.5\n",
      "numpy   : 1.19.5\n",
      "argparse: 1.1\n",
      "IPython : 5.5.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#hide\n",
    "%reload_ext watermark\n",
    "%watermark -a \"Sparsh A.\" -m -iv -u -t -d"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
