{
  "nbformat_minor": 0, 
  "nbformat": 4, 
  "cells": [
    {
      "execution_count": null, 
      "cell_type": "code", 
      "source": [
        "%matplotlib inline"
      ], 
      "outputs": [], 
      "metadata": {
        "collapsed": false
      }
    }, 
    {
      "source": [
        "\n=============================================\nExample: detecting defaults on retail credits\n=============================================\n\n\nSkopeRules finds logical rules with high precision and fuse them. Finding\ngood rules is done by fitting classification and regression trees\nto sub-samples.\nA fitted tree defines a set of rules (each tree node defines a rule); rules\nare then tested out of the bag, and the ones with higher precision are kept.\n\nThis example aims at finding logical rules to predict credit defaults. The\nanalysis shows that setting.\n\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }, 
    {
      "source": [
        "Data import and preparation\n...........................\n\nThere are 3 categorical variables (SEX, EDUCATION and MARRIAGE) and 20\nnumerical variables.\nThe target (credit defaults) is transformed in a binary variable with\nintegers 0 (no default) and 1 (default).\nFrom the 30000 credits, 50% are used for training and 50% are used\nfor testing. The target is unbalanced with a 22%/78% ratio.\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }, 
    {
      "execution_count": null, 
      "cell_type": "code", 
      "source": [
        "import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import roc_curve, precision_recall_curve\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.model_selection import GridSearchCV\nfrom sklearn.utils import shuffle\nfrom skrules import SkopeRules\nfrom skrules.datasets import load_credit_data\n\nprint(__doc__)\n\nrng = np.random.RandomState(1)\n\n# Importing data\ndataset = load_credit_data()\nX = dataset.data\ny = dataset.target\n# Shuffling data, preparing target and variables\ndata, y = shuffle(np.array(X), y, random_state=rng)\ndata = pd.DataFrame(data, columns=X.columns)\n\nfor col in ['ID']:\n    del data[col]\n\n# Quick feature engineering\ndata = data.rename(columns={\"PAY_0\": \"PAY_1\"})\nold_PAY = ['PAY_3', 'PAY_4', 'PAY_5', 'PAY_6']\ndata['PAY_old_mean'] = data[old_PAY].apply(lambda x: np.mean(x), axis=1)\n\nold_BILL_AMT = ['BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6']\ndata['BILL_AMT_old_mean'] = data[old_BILL_AMT].apply(\n    lambda x: np.mean(x), axis=1)\ndata['BILL_AMT_old_std'] = data[old_BILL_AMT].apply(\n    lambda x: np.std(x),\n    axis=1)\n\nold_PAY_AMT = ['PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6']\ndata['PAY_AMT_old_mean'] = data[old_PAY_AMT].apply(\n    lambda x: np.mean(x), axis=1)\ndata['PAY_AMT_old_std'] = data[old_PAY_AMT].apply(\n    lambda x: np.std(x), axis=1)\n\ndata.drop(old_PAY_AMT + old_BILL_AMT + old_PAY, axis=1, inplace=True)\n\n# Creating the train/test split\nfeature_names = list(data.columns)\nprint(\"List of variables used to train models : \" + str(feature_names))\ndata = data.values\nn_samples = data.shape[0]\nn_samples_train = int(n_samples / 2)\ny_train = y[:n_samples_train]\ny_test = y[n_samples_train:]\nX_train = data[:n_samples_train]\nX_test = data[n_samples_train:]"
      ], 
      "outputs": [], 
      "metadata": {
        "collapsed": false
      }
    }, 
    {
      "source": [
        "Benchmark with a Random Forest classifier\n.........................................\n\nThis part shows the training and performance evaluation of a random forest\nmodel. The objective remains to extract rules which targets credit defaults.\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }, 
    {
      "execution_count": null, 
      "cell_type": "code", 
      "source": [
        "rf = GridSearchCV(\n    RandomForestClassifier(\n        random_state=rng,\n        n_estimators=30,\n        class_weight='balanced'),\n    param_grid={'max_depth': range(3, 8, 1),\n                'max_features': np.linspace(0.1, 1., 5)},\n    scoring={'AUC': 'roc_auc'}, cv=5,\n    refit='AUC', n_jobs=-1)\n\nrf.fit(X_train, y_train)\nscoring_rf = rf.predict_proba(X_test)[:, 1]\n\nprint(\"Random Forest selected parameters : %s\" % rf.best_params_)\n\n# Plot ROC and PR curves\n\nfig, axes = plt.subplots(1, 2, figsize=(12, 5),\n                         sharex=True, sharey=True)\n\nax = axes[0]\nfpr_RF, tpr_RF, _ = roc_curve(y_test, scoring_rf)\nax.step(fpr_RF, tpr_RF, linestyle='-.', c='g', lw=1, where='post')\nax.set_title(\"ROC\", fontsize=20)\nax.legend(loc='upper center', fontsize=8)\nax.set_xlabel('False Positive Rate', fontsize=18)\nax.set_ylabel('True Positive Rate (Recall)', fontsize=18)\n\nax = axes[1]\nprecision_RF, recall_RF, _ = precision_recall_curve(y_test, scoring_rf)\nax.step(recall_RF, precision_RF, linestyle='-.', c='g', lw=1, where='post')\nax.set_title(\"Precision-Recall\", fontsize=20)\nax.set_xlabel('Recall (True Positive Rate)', fontsize=18)\nax.set_ylabel('Precision', fontsize=18)\nplt.show()"
      ], 
      "outputs": [], 
      "metadata": {
        "collapsed": false
      }
    }, 
    {
      "source": [
        "The ROC and Precision-Recall curves illustrate the performance of Random\nForests in this classification task.\nSuppose now that we add an interpretability contraint to this setting:\nTypically, we want to express our model in terms of logical rules detecting\ndefaults. A random forest could be expressed in term of weighted sum of\nrules, but 1) such a large weighted sum, is hardly interpretable and 2)\nsimplifying it by removing rules/weights is not easy, as optimality is\ntargeted by the ensemble of weighted rules, not by each rule.\nIn the following section, we show how SkopeRules can be used to produce\na number of rules, each seeking for high precision on a potentially small\narea of detection (low recall).\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }, 
    {
      "source": [
        "Getting rules with skrules\n..........................\n\nThis part shows how SkopeRules can be fitted to detect credit defaults.\nPerformances are compared with the random forest model previously trained.\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }, 
    {
      "execution_count": null, 
      "cell_type": "code", 
      "source": [
        "# fit the model\n\nclf = SkopeRules(\n    similarity_thres=.9, max_depth=3, max_features=0.5,\n    max_samples_features=0.5, random_state=rng, n_estimators=30,\n    feature_names=feature_names, recall_min=0.02, precision_min=0.6)\nclf.fit(X_train, y_train)\n\n# in the separate_rules_score method, a score of k means that rule number k\n# vote positively, but not rules 1, ..., k-1. It will allow us to plot\n# performance of each rule separately on ROC and PR plots.\nscoring = clf.separate_rules_score(X_test)\n\nprint(str(len(clf.rules_)) + ' rules have been built.')\nprint('The most precise rules are the following:')\nprint(clf.rules_[:5])\n\ncurves = [roc_curve, precision_recall_curve]\nxlabels = ['False Positive Rate', 'Recall (True Positive Rate)']\nylabels = ['True Positive Rate (Recall)', 'Precision']\n\n\nfig, axes = plt.subplots(1, 2, figsize=(12, 5),\n                         sharex=True, sharey=True)\n\nax = axes[0]\nfpr, tpr, _ = roc_curve(y_test, scoring)\nfpr_rf, tpr_rf, _ = roc_curve(y_test, scoring_rf)\nax.scatter(fpr[:-1], tpr[:-1], c='b', s=10)\nax.step(fpr_RF, tpr_RF, linestyle='-.', c='g', lw=1, where='post')\nax.set_title(\"ROC\", fontsize=20)\nax.legend(loc='upper center', fontsize=8)\nax.set_xlabel('False Positive Rate', fontsize=18)\nax.set_ylabel('True Positive Rate (Recall)', fontsize=18)\n\nax = axes[1]\nprecision, recall, _ = precision_recall_curve(y_test, scoring)\nprecision_rf, recall_rf, _ = precision_recall_curve(y_test, scoring_rf)\nax.scatter(recall[1:-1], precision[1:-1], c='b', s=10)\nax.step(recall_RF, precision_RF, linestyle='-.', c='g', lw=1, where='post')\nax.set_title(\"Precision-Recall\", fontsize=20)\nax.set_xlabel('Recall (True Positive Rate)', fontsize=18)\nax.set_ylabel('Precision', fontsize=18)\nplt.show()"
      ], 
      "outputs": [], 
      "metadata": {
        "collapsed": false
      }
    }, 
    {
      "source": [
        "The ROC and Precision-Recall curves show the performance of the rules\ngenerated by SkopeRules the (the blue points) and the performance of the\nRandom Forest classifier fitted above.\nEach blue point represents the performance of a set of rules: The kth point\nrepresents the score associated to the concatenation (union) of the k first\nrules, etc. Thus, each blue point is associated with an interpretable\nclassifier.\nIn terms of performance, each of these interpretable classifiers compare well\nwith Random Forest, while offering complete interpretation.\nThe range of recall and precision can be controlled by the precision_min and\nrecall_min parameters. Here, setting precision_min to 0.6 force the rules to\nhave a limited recall.\n\n"
      ], 
      "cell_type": "markdown", 
      "metadata": {}
    }
  ], 
  "metadata": {
    "kernelspec": {
      "display_name": "Python 2", 
      "name": "python2", 
      "language": "python"
    }, 
    "language_info": {
      "mimetype": "text/x-python", 
      "nbconvert_exporter": "python", 
      "name": "python", 
      "file_extension": ".py", 
      "version": "2.7.12", 
      "pygments_lexer": "ipython2", 
      "codemirror_mode": {
        "version": 2, 
        "name": "ipython"
      }
    }
  }
}