{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **FairSVM**\n",
    "\n",
    "[![Slides](https://img.shields.io/badge/🦌-ReHLine-blueviolet)](https://rehline-python.readthedocs.io/en/latest/)\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1f_7t1t6FNxAooQOmpyhHCOVq0IKgMxe-?usp=sharing)\n",
    "\n",
    "The FairSVM solves the following optimization problem:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "  & \\min_{\\mathbf{\\beta} \\in \\mathbb{R}^d} \\frac{C}{n} \\sum_{i=1}^n ( 1 - y_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i )_+ + \\frac{1}{2} \\| \\mathbf{\\beta} \\|_2^2, \\nonumber \\\\\n",
    "  \\text{subject to } & \\quad \\frac{1}{n} \\sum_{i=1}^n \\mathbf{z}_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i \\leq \\mathbf{\\rho}, \\quad \\frac{1}{n} \\sum_{i=1}^n \\mathbf{z}_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i \\geq -\\mathbf{\\rho},\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "where:\n",
    "\n",
    "* $\\mathbf{x}_i \\in \\mathbb{R}^d$ is a feature vector\n",
    "* $y_i \\in \\{-1, 1\\}$ is a binary label\n",
    "* $\\mathbf{z}_i$ is a collection of **centered sensitive features**, such as gender and/or race, satisfying:\n",
    "\n",
    "$$\\sum_{i=1}^n z_{ij} = 0,$$\n",
    "\n",
    "* $\\mathbf{z}_i \\in \\mathbb{R}^{d_0}$ is a $d_0$-length sensitive feature vector\n",
    "* $\\mathbf{\\rho} \\in \\mathbb{R}_+^{d_0}$ is a vector of constants that trade-off predictive accuracy and fairness\n",
    "\n",
    "The constraints limit the correlation between the sensitive features and the decision function, ensuring fairness in the predictions.\n",
    "\n",
    "> **Note.** Since the hinge loss is a plq function, and fairness constraints are linear, thus we can solve it by `rehline.plqERM_Ridge`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "## simulate data\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import numpy as np\n",
    "\n",
    "scaler = StandardScaler()\n",
    "\n",
    "n, d = 10000, 5\n",
    "X, y = make_classification(n_samples=n, n_features=d, n_redundant=0)\n",
    "## convert y to +1/-1\n",
    "y = 2*y - 1\n",
    "X = scaler.fit_transform(X)\n",
    "\n",
    "## we take the first column of X as sensetive features, and tol is 0.1\n",
    "sen_idx = [0]\n",
    "tol_sen = 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM as baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## we first run a SVM\n",
    "from rehline import plqERM_Ridge\n",
    "\n",
    "clf = plqERM_Ridge(loss={'name': 'svm'}, C=1.0, max_iter=50000)\n",
    "clf.fit(X=X, y=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FairSVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "## solve FairSVM via `plqERM_Ridge` by adding `constraint`\n",
    "from rehline import plqERM_Ridge\n",
    "\n",
    "fclf = plqERM_Ridge(loss={'name': 'svm'},\n",
    "                   constraint=[{'name': 'fair',\n",
    "                                'sen_idx': sen_idx,\n",
    "                                'tol_sen': tol_sen}],\n",
    "                   C=1.0,\n",
    "                   max_iter=50000)\n",
    "fclf.fit(X=X, y=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Model  Train Performance  Correlation with Sensitive Features\n",
      "    SVM             0.8853                             2.535203\n",
      "FairSVM             0.5856                             0.100212\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "## score\n",
    "score = clf.decision_function(X)\n",
    "fscore = fclf.decision_function(X)\n",
    "\n",
    "svm_perf = len(y[score*y > 0])/n\n",
    "fsvm_perf = len(y[fscore*y > 0])/n\n",
    "\n",
    "svm_corr = score.dot(X[:,sen_idx]) / n\n",
    "fsvm_corr = fscore.dot(X[:,sen_idx]) / n\n",
    "\n",
    "# Create a pandas DataFrame to store the results\n",
    "results = pd.DataFrame({\n",
    "    'Model': ['SVM', 'FairSVM'],\n",
    "    'Train Performance': [svm_perf, fsvm_perf],\n",
    "    'Correlation with Sensitive Features': [svm_corr, fsvm_corr]\n",
    "})\n",
    "\n",
    "# Print the results as a table\n",
    "print(results.to_string(index=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 19288 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 26412 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import warnings\n",
    "import matplotlib.pyplot as plt\n",
    "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n",
    "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n",
    "\n",
    "df = pd.DataFrame({'score': score, 'fscore': fscore, 'y': y})\n",
    "\n",
    "sns.histplot(df, x=\"score\", hue=\"y\").set_title(\"SVM\")\n",
    "plt.show()\n",
    "sns.histplot(df, x=\"fscore\", hue=\"y\").set_title(\"FairSVM\")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
