{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default_exp models.xdeepfm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# xDeepFM\n",
    "> A pytorch implementation of Extreme Deep Factorization Machines (xDeepFM).\n",
    "\n",
    "xDeepFM combines the CIN and a classical DNN into one unified model. xDeepFM is able to learn certain bounded-degree feature interactions explicitly; on the other hand, it can learn arbitrary low- and high-order feature interactions implicitly.\n",
    "\n",
    "**The architecture of xDeepFM.**\n",
    "<img src='https://github.com/RecoHut-Stanzas/S021355/raw/main/images/img13.png'>\n",
    "\n",
    "Compressed Interaction Network (CIN) aims to generate feature interactions in an explicit fashion and at the vector-wise level. CIN share some functionalities with convolutional neural networks (CNNs) and recurrent neural networks (RNNs).\n",
    "\n",
    "**Components and architecture of the Compressed Interaction Network (CIN).**\n",
    "<img src='https://github.com/RecoHut-Stanzas/S021355/raw/main/images/img14.png'>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "from nbdev.showdoc import *\n",
    "from fastcore.nb_imports import *\n",
    "from fastcore.test import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "import torch\n",
    "\n",
    "from recohut.models.layers.common import FeaturesEmbedding, FeaturesLinear, MultiLayerPerceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export\n",
    "class CompressedInteractionNetwork(torch.nn.Module):\n",
    "\n",
    "    def __init__(self, input_dim, cross_layer_sizes, split_half=True):\n",
    "        super().__init__()\n",
    "        self.num_layers = len(cross_layer_sizes)\n",
    "        self.split_half = split_half\n",
    "        self.conv_layers = torch.nn.ModuleList()\n",
    "        prev_dim, fc_input_dim = input_dim, 0\n",
    "        for i in range(self.num_layers):\n",
    "            cross_layer_size = cross_layer_sizes[i]\n",
    "            self.conv_layers.append(torch.nn.Conv1d(input_dim * prev_dim, cross_layer_size, 1,\n",
    "                                                    stride=1, dilation=1, bias=True))\n",
    "            if self.split_half and i != self.num_layers - 1:\n",
    "                cross_layer_size //= 2\n",
    "            prev_dim = cross_layer_size\n",
    "            fc_input_dim += prev_dim\n",
    "        self.fc = torch.nn.Linear(fc_input_dim, 1)\n",
    "\n",
    "    def forward(self, x):\n",
    "        \"\"\"\n",
    "        :param x: Float tensor of size ``(batch_size, num_fields, embed_dim)``\n",
    "        \"\"\"\n",
    "        xs = list()\n",
    "        x0, h = x.unsqueeze(2), x\n",
    "        for i in range(self.num_layers):\n",
    "            x = x0 * h.unsqueeze(1)\n",
    "            batch_size, f0_dim, fin_dim, embed_dim = x.shape\n",
    "            x = x.view(batch_size, f0_dim * fin_dim, embed_dim)\n",
    "            x = F.relu(self.conv_layers[i](x))\n",
    "            if self.split_half and i != self.num_layers - 1:\n",
    "                x, h = torch.split(x, x.shape[1] // 2, dim=1)\n",
    "            else:\n",
    "                h = x\n",
    "            xs.append(x)\n",
    "        return self.fc(torch.sum(torch.cat(xs, dim=1), 2))\n",
    "\n",
    "\n",
    "class xDeepFM(torch.nn.Module):\n",
    "    \"\"\"\n",
    "    A pytorch implementation of xDeepFM.\n",
    "    Reference:\n",
    "        J Lian, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems, 2018.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, field_dims, embed_dim, mlp_dims, dropout, cross_layer_sizes, split_half=True):\n",
    "        super().__init__()\n",
    "        self.embedding = FeaturesEmbedding(field_dims, embed_dim)\n",
    "        self.embed_output_dim = len(field_dims) * embed_dim\n",
    "        self.cin = CompressedInteractionNetwork(len(field_dims), cross_layer_sizes, split_half)\n",
    "        self.mlp = MultiLayerPerceptron(self.embed_output_dim, mlp_dims, dropout)\n",
    "        self.linear = FeaturesLinear(field_dims)\n",
    "\n",
    "    def forward(self, x):\n",
    "        \"\"\"\n",
    "        :param x: Long tensor of size ``(batch_size, num_fields)``\n",
    "        \"\"\"\n",
    "        embed_x = self.embedding(x)\n",
    "        x = self.linear(x) + self.cin(embed_x) + self.mlp(embed_x.view(-1, self.embed_output_dim))\n",
    "        return torch.sigmoid(x.squeeze(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **References:-**\n",
    "- J Lian, et al. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems, 2018.\n",
    "- https://github.com/rixwew/pytorch-fm/blob/master/torchfm/model/xdfm.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#hide\n",
    "%reload_ext watermark\n",
    "%watermark -a \"Sparsh A.\" -m -iv -u -t -d -p recohut"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
