{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a62a0c59",
   "metadata": {},
   "source": [
    "# Non-negative Matrix Factorization (NMF)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1881bc9e",
   "metadata": {},
   "source": [
    "[![Slides](https://img.shields.io/badge/🦌-ReHLine-blueviolet)](https://rehline-python.readthedocs.io/en/latest/)\n",
    "\n",
    "Non-negative Matrix Factorizaation (NMF) solves the following optimiation problem:\n",
    "\n",
    "$$\n",
    "  \\min_{\\substack{\n",
    "      \\mathbf{P} \\in \\mathbb{R}^{n \\times k}\\ \n",
    "      \\pmb{\\alpha} \\in \\mathbb{R}^n \\\\\n",
    "      \\mathbf{Q} \\in \\mathbb{R}^{m \\times k}\\ \n",
    "      \\pmb{\\beta} \\in \\mathbb{R}^m\n",
    "  }} \n",
    "  \\left[\n",
    "      \\sum_{(u,i)\\in \\Omega} C \\cdot \\phi(r_{ui}, \\ \\mathbf{p}_u^T \\mathbf{q}_i + \\alpha_u + \\beta_i) \n",
    "  \\right]  \n",
    "  + \n",
    "  \\left[ \n",
    "      \\frac{\\rho}{n}\\sum_{u=1}^n(\\|\\mathbf{p}_u\\|_2^2 + \\alpha_u^2) \n",
    "      + \\frac{1-\\rho}{m}\\sum_{i=1}^m(\\|\\mathbf{q}_i\\|_2^2 + \\beta_i^2) \n",
    "  \\right]\n",
    "$$\n",
    "\n",
    "$$\n",
    "  \\ \\text{ s.t. } \\ \n",
    "  \\mathbf{P} \\geq \\mathbf{0},\\ \\mathbf{Q} \\geq \\mathbf{0},\\  \\pmb{\\alpha} \\geq \\mathbf{0},\\ \\text{and}\\  \\pmb{\\beta} \\geq \\mathbf{0}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "- $\\phi(\\cdot , \\cdot)$ \n",
    "  is a convex piecewise linear-quadratic loss function\n",
    "\n",
    "- $\\Omega$\n",
    "  is a user-item collection that records all training data\n",
    "\n",
    "- $n$ is number of rows in target matrix, $m$ is number of columns in target matrix\n",
    "\n",
    "- $k$ is length of latent factors (rank of MF) \n",
    "\n",
    "- $C$ is regularization parameter, $\\rho$ balances regularization strength between user and item\n",
    "\n",
    "- $\\mathbf{p}_u$ and $\\alpha_u$\n",
    "  are latent vector and individual bias of u-th row. Specifically, $\\mathbf{p}_u$ is the u-th row of $\\mathbf{P}$, and $\\alpha_u$ is the u-th element of $\\pmb{\\alpha}$\n",
    "  \n",
    "- $\\mathbf{q}_i$ and $\\beta_i$\n",
    "  are latent vector and individual bias of i-th column. Specifically, $\\mathbf{q}_i$ is the i-th row of $\\mathbf{Q}$, and $\\beta_i$ is the i-th element of $\\pmb{\\beta}$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cb9c3754",
   "metadata": {},
   "outputs": [],
   "source": [
    "## packages\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from numpy.random import RandomState\n",
    "from sklearn.datasets import fetch_olivetti_faces\n",
    "from rehline import plqMF_Ridge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ae1700c3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 56088 characters]",
      "text/plain": [
       "<Figure size 600x460 with 7 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## set random state\n",
    "rng = RandomState(0)\n",
    "\n",
    "\n",
    "## import face data and convert to triplet dataset\n",
    "faces, _ = fetch_olivetti_faces(return_X_y=True, shuffle=True, random_state=rng)\n",
    "n_samples, n_features = faces.shape\n",
    "\n",
    "rows, cols = np.nonzero(faces)\n",
    "X = np.column_stack((rows, cols))\n",
    "y = faces[rows, cols]\n",
    "\n",
    "\n",
    "## define a face-visualization function \n",
    "n_row, n_col = 2, 3\n",
    "n_components = n_row * n_col\n",
    "image_shape = (64, 64)\n",
    "\n",
    "def plot_gallery(title, images, n_col=n_col, n_row=n_row, cmap=plt.cm.gray):\n",
    "    fig, axs = plt.subplots(\n",
    "        nrows=n_row,\n",
    "        ncols=n_col,\n",
    "        figsize=(2.0 * n_col, 2.3 * n_row),\n",
    "        facecolor=\"white\",\n",
    "        constrained_layout=True,\n",
    "    )\n",
    "    fig.set_constrained_layout_pads(w_pad=0.01, h_pad=0.02, hspace=0, wspace=0)\n",
    "    fig.set_edgecolor(\"black\")\n",
    "    fig.suptitle(title, size=16)\n",
    "    for ax, vec in zip(axs.flat, images):\n",
    "        vmax = max(vec.max(), -vec.min())\n",
    "        im = ax.imshow(\n",
    "            vec.reshape(image_shape),\n",
    "            cmap=cmap,\n",
    "            interpolation=\"nearest\",\n",
    "            vmin=-vmax,\n",
    "            vmax=vmax,\n",
    "        )\n",
    "        ax.axis(\"off\")\n",
    "\n",
    "    fig.colorbar(im, ax=axs, orientation=\"horizontal\", shrink=0.99, aspect=40, pad=0.01)\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "## show some faces\n",
    "plot_gallery(\"Faces from dataset\", faces[:n_components])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1b25add7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration    Average Loss(mse)    Objective Function  \n",
      "1            0.020348             18.630042           \n",
      "2            0.012533             12.627574           \n",
      "3            0.010705             10.203743           \n",
      "4            0.009903             8.819946            \n",
      "5            0.009485             7.921505            \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  display: none;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  overflow: visible;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".estimator-table summary {\n",
       "    padding: .5rem;\n",
       "    font-family: monospace;\n",
       "    cursor: pointer;\n",
       "}\n",
       "\n",
       ".estimator-table details[open] {\n",
       "    padding-left: 0.1rem;\n",
       "    padding-right: 0.1rem;\n",
       "    padding-bottom: 0.3rem;\n",
       "}\n",
       "\n",
       ".estimator-table .parameters-table {\n",
       "    margin-left: auto !important;\n",
       "    margin-right: auto !important;\n",
       "}\n",
       "\n",
       ".estimator-table .parameters-table tr:nth-child(odd) {\n",
       "    background-color: #fff;\n",
       "}\n",
       "\n",
       ".estimator-table .parameters-table tr:nth-child(even) {\n",
       "    background-color: #f6f6f6;\n",
       "}\n",
       "\n",
       ".estimator-table .parameters-table tr:hover {\n",
       "    background-color: #e0e0e0;\n",
       "}\n",
       "\n",
       ".estimator-table table td {\n",
       "    border: 1px solid rgba(106, 105, 104, 0.232);\n",
       "}\n",
       "\n",
       ".user-set td {\n",
       "    color:rgb(255, 94, 0);\n",
       "    text-align: left;\n",
       "}\n",
       "\n",
       ".user-set td.value pre {\n",
       "    color:rgb(255, 94, 0) !important;\n",
       "    background-color: transparent !important;\n",
       "}\n",
       "\n",
       ".default td {\n",
       "    color: black;\n",
       "    text-align: left;\n",
       "}\n",
       "\n",
       ".user-set td i,\n",
       ".default td i {\n",
       "    color: black;\n",
       "}\n",
       "\n",
       ".copy-paste-icon {\n",
       "    background-image: url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCA0NDggNTEyIj48IS0tIUZvbnQgQXdlc29tZSBGcmVlIDYuNy4yIGJ5IEBmb250YXdlc29tZSAtIGh0dHBzOi8vZm9udGF3ZXNvbWUuY29tIExpY2Vuc2UgLSBodHRwczovL2ZvbnRhd2Vzb21lLmNvbS9saWNlbnNlL2ZyZWUgQ29weXJpZ2h0IDIwMjUgRm9udGljb25zLCBJbmMuLS0+PHBhdGggZD0iTTIwOCAwTDMzMi4xIDBjMTIuNyAwIDI0LjkgNS4xIDMzLjkgMTQuMWw2Ny45IDY3LjljOSA5IDE0LjEgMjEuMiAxNC4xIDMzLjlMNDQ4IDMzNmMwIDI2LjUtMjEuNSA0OC00OCA0OGwtMTkyIDBjLTI2LjUgMC00OC0yMS41LTQ4LTQ4bDAtMjg4YzAtMjYuNSAyMS41LTQ4IDQ4LTQ4ek00OCAxMjhsODAgMCAwIDY0LTY0IDAgMCAyNTYgMTkyIDAgMC0zMiA2NCAwIDAgNDhjMCAyNi41LTIxLjUgNDgtNDggNDhMNDggNTEyYy0yNi41IDAtNDgtMjEuNS00OC00OEwwIDE3NmMwLTI2LjUgMjEuNS00OCA0OC00OHoiLz48L3N2Zz4=);\n",
       "    background-repeat: no-repeat;\n",
       "    background-size: 14px 14px;\n",
       "    background-position: 0;\n",
       "    display: inline-block;\n",
       "    width: 14px;\n",
       "    height: 14px;\n",
       "    cursor: pointer;\n",
       "}\n",
       "</style><body><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>plqMF_Ridge(C=0.0003, biased=False, constraint_item=[{&#x27;name&#x27;: &#x27;&gt;=0&#x27;}],\n",
       "            constraint_user=[{&#x27;name&#x27;: &#x27;&gt;=0&#x27;}], loss={&#x27;name&#x27;: &#x27;mse&#x27;},\n",
       "            max_iter_CD=5, n_items=4096, n_users=400, random_state=0, rank=6,\n",
       "            tol=0.01, verbose=1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label  sk-toggleable__label-arrow\"><div><div>plqMF_Ridge</div></div><div><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></div></label><div class=\"sk-toggleable__content \" data-param-prefix=\"\">\n",
       "        <div class=\"estimator-table\">\n",
       "            <details>\n",
       "                <summary>Parameters</summary>\n",
       "                <table class=\"parameters-table\">\n",
       "                  <tbody>\n",
       "                    \n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('n_users',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">n_users&nbsp;</td>\n",
       "            <td class=\"value\">400</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('n_items',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">n_items&nbsp;</td>\n",
       "            <td class=\"value\">4096</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('loss',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">loss&nbsp;</td>\n",
       "            <td class=\"value\">{&#x27;name&#x27;: &#x27;mse&#x27;}</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('biased',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">biased&nbsp;</td>\n",
       "            <td class=\"value\">False</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('constraint_user',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">constraint_user&nbsp;</td>\n",
       "            <td class=\"value\">[{&#x27;name&#x27;: &#x27;&gt;=0&#x27;}]</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('constraint_item',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">constraint_item&nbsp;</td>\n",
       "            <td class=\"value\">[{&#x27;name&#x27;: &#x27;&gt;=0&#x27;}]</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('rank',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">rank&nbsp;</td>\n",
       "            <td class=\"value\">6</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('C',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">C&nbsp;</td>\n",
       "            <td class=\"value\">0.0003</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('rho',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">rho&nbsp;</td>\n",
       "            <td class=\"value\">0.5</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('init_mean',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">init_mean&nbsp;</td>\n",
       "            <td class=\"value\">0.0</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('init_sd',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">init_sd&nbsp;</td>\n",
       "            <td class=\"value\">0.1</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('random_state',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">random_state&nbsp;</td>\n",
       "            <td class=\"value\">0</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('max_iter',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">max_iter&nbsp;</td>\n",
       "            <td class=\"value\">10000</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('tol',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">tol&nbsp;</td>\n",
       "            <td class=\"value\">0.01</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('shrink',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">shrink&nbsp;</td>\n",
       "            <td class=\"value\">1</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('trace_freq',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">trace_freq&nbsp;</td>\n",
       "            <td class=\"value\">100</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('max_iter_CD',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">max_iter_CD&nbsp;</td>\n",
       "            <td class=\"value\">5</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"default\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('tol_CD',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">tol_CD&nbsp;</td>\n",
       "            <td class=\"value\">0.0001</td>\n",
       "        </tr>\n",
       "    \n",
       "\n",
       "        <tr class=\"user-set\">\n",
       "            <td><i class=\"copy-paste-icon\"\n",
       "                 onclick=\"copyToClipboard('verbose',\n",
       "                          this.parentElement.nextElementSibling)\"\n",
       "            ></i></td>\n",
       "            <td class=\"param\">verbose&nbsp;</td>\n",
       "            <td class=\"value\">1</td>\n",
       "        </tr>\n",
       "    \n",
       "                  </tbody>\n",
       "                </table>\n",
       "            </details>\n",
       "        </div>\n",
       "    </div></div></div></div></div><script>function copyToClipboard(text, element) {\n",
       "    // Get the parameter prefix from the closest toggleable content\n",
       "    const toggleableContent = element.closest('.sk-toggleable__content');\n",
       "    const paramPrefix = toggleableContent ? toggleableContent.dataset.paramPrefix : '';\n",
       "    const fullParamName = paramPrefix ? `${paramPrefix}${text}` : text;\n",
       "\n",
       "    const originalStyle = element.style;\n",
       "    const computedStyle = window.getComputedStyle(element);\n",
       "    const originalWidth = computedStyle.width;\n",
       "    const originalHTML = element.innerHTML.replace('Copied!', '');\n",
       "\n",
       "    navigator.clipboard.writeText(fullParamName)\n",
       "        .then(() => {\n",
       "            element.style.width = originalWidth;\n",
       "            element.style.color = 'green';\n",
       "            element.innerHTML = \"Copied!\";\n",
       "\n",
       "            setTimeout(() => {\n",
       "                element.innerHTML = originalHTML;\n",
       "                element.style = originalStyle;\n",
       "            }, 2000);\n",
       "        })\n",
       "        .catch(err => {\n",
       "            console.error('Failed to copy:', err);\n",
       "            element.style.color = 'red';\n",
       "            element.innerHTML = \"Failed!\";\n",
       "            setTimeout(() => {\n",
       "                element.innerHTML = originalHTML;\n",
       "                element.style = originalStyle;\n",
       "            }, 2000);\n",
       "        });\n",
       "    return false;\n",
       "}\n",
       "\n",
       "document.querySelectorAll('.fa-regular.fa-copy').forEach(function(element) {\n",
       "    const toggleableContent = element.closest('.sk-toggleable__content');\n",
       "    const paramPrefix = toggleableContent ? toggleableContent.dataset.paramPrefix : '';\n",
       "    const paramName = element.parentElement.nextElementSibling.textContent.trim();\n",
       "    const fullParamName = paramPrefix ? `${paramPrefix}${paramName}` : paramName;\n",
       "\n",
       "    element.setAttribute('title', fullParamName);\n",
       "});\n",
       "</script></body>"
      ],
      "text/plain": [
       "plqMF_Ridge(C=0.0003, biased=False, constraint_item=[{'name': '>=0'}],\n",
       "            constraint_user=[{'name': '>=0'}], loss={'name': 'mse'},\n",
       "            max_iter_CD=5, n_items=4096, n_users=400, random_state=0, rank=6,\n",
       "            tol=0.01, verbose=1)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## solve nmf via rehline\n",
    "nmf_rehline = plqMF_Ridge(C=0.0003,\n",
    "                          biased=False,\n",
    "                          n_users=n_samples, \n",
    "                          n_items=n_features, \n",
    "                          rank=n_components, \n",
    "                          loss={'name':'mse'}, \n",
    "                          constraint_user=[{'name': '>=0'}],\n",
    "                          constraint_item=[{'name': '>=0'}],\n",
    "                          verbose=1,\n",
    "                          max_iter_CD=5,\n",
    "                          random_state=0,\n",
    "                          tol=0.01)\n",
    "\n",
    "nmf_rehline.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7249b602",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 50420 characters]",
      "text/plain": [
       "<Figure size 600x460 with 7 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## check eigen faces\n",
    "plot_gallery(\"Non-negative components - ReHLine\", np.transpose(nmf_rehline.Q))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ReHLine",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
