--- title: DeepFM keywords: fastai sidebar: home_sidebar summary: "A pytorch implementation of DeepFM." description: "A pytorch implementation of DeepFM." nb_path: "nbs/models/models.deepfm.ipynb" ---
{% raw %}
{% endraw %}

## v1

> References:-- H Guo, et al. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction, 2017.

> - https://github.com/rixwew/pytorch-fm/blob/master/torchfm/model/dfm.py

{% raw %}
{% endraw %} {% raw %}

## `class` `FactorizationMachine`[source]

> `FactorizationMachine`(`reduce_sum`=`True`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}

## `class` `DeepFM`[source]

> `DeepFM`(`field_dims`, `embed_dim`, `mlp_dims`, `dropout`) :: `Module`

A pytorch implementation of DeepFM. Reference: H Guo, et al. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction, 2017.

{% endraw %} {% raw %}
{% endraw %}

## v2

> References:-- https://github.com/huangjunheng/recommendation_model/tree/master/deepFM

{% raw %}
{% endraw %} {% raw %}

## `class` `DeepFMv2`[source]

> `DeepFMv2`(`feat_sizes`, `sparse_feature_columns`, `dense_feature_columns`, `dnn_hidden_units`=`[400, 400, 400]`, `dnn_dropout`=`0.0`, `ebedding_size`=`4`, `l2_reg_linear`=`1e-05`, `l2_reg_embedding`=`1e-05`, `l2_reg_dnn`=`0`, `init_std`=`0.0001`, `seed`=`1024`, `device`=`'cpu'`) :: `Module`

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

    import torch.nn as nn
    import torch.nn.functional as F

    class Model(nn.Module):
        def __init__(self):
            super(Model, self).__init__()
            self.conv1 = nn.Conv2d(1, 20, 5)
            self.conv2 = nn.Conv2d(20, 20, 5)

        def forward(self, x):
            x = F.relu(self.conv1(x))
            return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:`to`, etc.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool

{% endraw %} {% raw %}
{% endraw %} {% raw %}

    import numpy as np
    import pandas as pd
    import torch
    import torch.nn as nn
    from torch.utils.data import DataLoader
    from sklearn.metrics import log_loss, roc_auc_score

    from model import DeepFMv2 as DeepFM
    from recohut.datasets.criteo import CriteoSampleDataset

    def get_auc(loader, model):
        pred, target = [], []
        model.eval()
        with torch.no_grad():
            for x, y in loader:
                x, y = x.to(device).float(), y.to(device).float()
                y_hat = model(x)
                pred += list(y_hat.cpu().numpy())
                target += list(y.cpu().numpy())
        auc = roc_auc_score(target, pred)
        return auc

    root = '/content/data'
    batch_size = 1024
    epochs = 10
    seed = 1024
    lr = 0.00005
    wd = 0.00001
    device = 'cpu'

    ds = CriteoSampleDataset(root=root)
    train_tensor_data, test_tensor_data = ds.load()
    train_loader = DataLoader(train_tensor_data, shuffle=True, batch_size=batch_size)
    test_loader = DataLoader(test_tensor_data, batch_size=batch_size)

    sparse_features = ['C' + str(i) for i in range(1, 27)]
    dense_features = ['I' + str(i) for i in range(1, 14)]

    # model = NFM(ds.feat_sizes, embedding_size, ds.linear_feature_columns, ds.dnn_feature_columns).to(device)
    model = DeepFMv2(ds.feat_sizes, sparse_feature_columns=sparse_features, dense_feature_columns=dense_features,
                    dnn_hidden_units=[1000, 500, 250], dnn_dropout=0.9, ebedding_size=16,
                    l2_reg_linear=1e-3, device=device)
    loss_func = nn.BCELoss(reduction='mean')
    optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=wd)

    for epoch in range(epochs):
        total_loss_epoch = 0.0
        total_tmp = 0
        model.train()
        for index, (x, y) in enumerate(train_loader):
            x, y = x.to(device).float(), y.to(device).float()
            y_hat = model(x)

            optimizer.zero_grad()
            loss = loss_func(y_hat, y)
            loss.backward()
            optimizer.step()
            total_loss_epoch += loss.item()
            total_tmp += 1
        auc = get_auc(test_loader, model)
        print('epoch/epoches: {}/{}, train loss: {:.3f}, test auc: {:.3f}'.format(epoch, epochs, total_loss_epoch / total_tmp, auc))

    epoch/epoches: 0/10, train loss: 0.570, test auc: 0.684
    epoch/epoches: 1/10, train loss: 0.534, test auc: 0.714
    epoch/epoches: 2/10, train loss: 0.511, test auc: 0.725
    epoch/epoches: 3/10, train loss: 0.486, test auc: 0.732
    epoch/epoches: 4/10, train loss: 0.459, test auc: 0.738
    epoch/epoches: 5/10, train loss: 0.431, test auc: 0.743
    epoch/epoches: 6/10, train loss: 0.401, test auc: 0.743
    epoch/epoches: 7/10, train loss: 0.368, test auc: 0.740
    epoch/epoches: 8/10, train loss: 0.337, test auc: 0.735
    epoch/epoches: 9/10, train loss: 0.312, test auc: 0.729

{% endraw %} {% raw %}

    #     def __init__(self, n_users, n_items, embedding_dim, batch_norm=True, dropout=0.1, num_layers=3, act_function='relu'):
    #         """
    #         Args:
    #             n_users : int, the number of users
    #             n_items : int, the number of items
    #             embedding_dim : int, the number of latent factoact_function : str, activation function for hidden layer
    #             num_layers : int, number of hidden layers
    #             batch_norm : bool, whether to normalize a batch of data
    #             dropout : float, dropout rate
    #         """
    #         super().__init__()

    #         self.num_layers = num_layers

    #         self.user_embedding = nn.Embedding(
    #             num_embeddings=n_users, embedding_dim=embedding_dim
    #         )
    #         self.item_embedding = nn.Embedding(
    #             num_embeddings=n_items, embedding_dim=embedding_dim
    #         )
    #         self.user_bias = nn.Embedding(n_users, 1)
    #         self.item_bias = nn.Embedding(n_items, 1)
    #         self.bias_ = nn.Parameter(torch.tensor([0.0]))

    #         fm_modules = []
    #         if batch_norm:
    #             fm_modules.append(nn.BatchNorm1d(embedding_dim))
    #         fm_modules.append(nn.Dropout(dropout))
    #         self.fm_layers = nn.Sequential(*fm_modules)

    #         deep_modules = []
    #         in_dim = embedding_dim * 2   # user & item
    #         for _ in range(num_layers):  # _ is dim if layers is list
    #             out_dim = in_dim
    #             deep_modules.append(nn.Linear(in_dim, out_dim))
    #             in_dim = out_dim
    #             if batch_norm:
    #                 deep_modules.append(nn.BatchNorm1d(out_dim))
    #             if act_function == 'relu':
    #                 deep_modules.append(nn.ReLU())
    #             elif act_function == 'sigmoid':
    #                 deep_modules.append(nn.Sigmoid())
    #             elif act_function == 'tanh':
    #                 deep_modules.append(nn.Tanh())
    #             deep_modules.append(nn.Dropout(dropout))

    #         self.deep_layers = nn.Sequential(*deep_modules)
    #         self.deep_out = nn.Linear(in_dim, 1, bias=False)

    #         self._init_weights()

    #     def _init_weights(self):
    #         nn.init.normal_(self.item_embedding.weight, std=0.01)
    #         nn.init.normal_(self.user_embedding.weight, std=0.01)
    #         nn.init.constant_(self.user_bias.weight, 0.0)
    #         nn.init.constant_(self.item_bias.weight, 0.0)

    #         # for deep layers
    #         for m in self.deep_layers:
    #             if isinstance(m, nn.Linear):
    #                 nn.init.xavier_normal_(m.weight)
    #         nn.init.xavier_normal_(self.deep_out.weight)

    #     def forward(self, users, items):
    #         embed_user = self.user_embedding(users)
    #         embed_item = self.item_embedding(items)

    #         fm = embed_user * embed_item
    #         fm = self.fm_layers(fm)
    #         y_fm = fm.sum(dim=-1)

    #         y_fm = y_fm + self.user_bias(users) + self.item_bias(items) + self.bias_

    #         if self.num_layers:
    #             fm = self.deep_layers(fm)

    #         y_deep = torch.cat((embed_user, embed_item), dim=-1)
    #         y_deep = self.deep_layers(y_deep)

    #         # since BCELoss will automatically transfer pred with sigmoid
    #         # there is no need to use extra nn.Sigmoid(pred)
    #         pred = y_fm + y_deep

    #         return pred.view(-1)

{% endraw %}