{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LizyML Widget Tutorial — Binary Classification\n",
    "\n",
    "This tutorial demonstrates using LizyML Widget for a **binary classification** task\n",
    "with the [Breast Cancer Wisconsin](https://scikit-learn.org/stable/datasets/toy_dataset.html#breast-cancer-wisconsin-diagnostic-dataset) dataset.\n",
    "\n",
    "You will learn:\n",
    "1. Loading real-world data into the widget\n",
    "2. Reviewing auto-detected settings\n",
    "3. Configuring evaluation metrics (including `precision_at_k` with custom k)\n",
    "4. Enabling calibration for probability estimates\n",
    "5. Reviewing binary-specific metrics and plots (ROC curve, calibration, feature importance)\n",
    "6. Hyperparameter tuning\n",
    "7. Running inference with probability predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Installation\n",
    "\n",
    "```bash\n",
    "pip install lizyml-widget lizyml[plots,tuning,calibration,explain]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load the Breast Cancer Dataset\n",
    "\n",
    "This dataset contains 569 samples with 30 features computed from\n",
    "digitized images of fine needle aspirates of breast masses.\n",
    "The target is binary: malignant (0) or benign (1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "\n",
    "data = load_breast_cancer(as_frame=True)\n",
    "df = data.frame\n",
    "print(f\"Shape: {df.shape}\")\n",
    "print(f\"Target distribution:\\n{df['target'].value_counts()}\")\n",
    "print(f\"  0 = malignant, 1 = benign\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Launch the Widget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lizyml_widget import LizyWidget\n",
    "\n",
    "w = LizyWidget()\n",
    "w.load(df, target=\"target\")\n",
    "w"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Data Tab — Auto-Detection\n",
    "\n",
    "The widget auto-detects:\n",
    "- **Task**: `binary` (2 unique target values)\n",
    "- **CV**: `stratified_kfold` (preserves class balance)\n",
    "- **Columns**: All 30 features included"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Task:     {w.task}\")\n",
    "print(f\"CV:       {w.cv_method} ({w.cv_n_splits} folds)\")\n",
    "print(f\"Shape:    {w.df_shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Config — Model & Evaluation Settings\n",
    "\n",
    "For binary classification, we configure:\n",
    "- Multiple evaluation metrics including `precision_at_k` with custom k\n",
    "- Calibration enabled (Platt scaling) for well-calibrated probabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.set_config({\n",
    "    \"model\": {\n",
    "        \"name\": \"lgbm\",\n",
    "        \"params\": {\n",
    "            \"n_estimators\": 300,\n",
    "            \"learning_rate\": 0.05,\n",
    "            \"max_depth\": 4,\n",
    "        },\n",
    "    },\n",
    "    \"training\": {\n",
    "        \"seed\": 42,\n",
    "        \"early_stopping\": {\"enabled\": True, \"rounds\": 50},\n",
    "    },\n",
    "    \"evaluation\": {\n",
    "        \"metrics\": [\"auc\", \"logloss\", \"f1\", \"precision_at_k\"],\n",
    "        \"params\": {\"precision_at_k_k\": 20},\n",
    "    },\n",
    "    \"calibration\": {\n",
    "        \"method\": \"platt\",\n",
    "        \"n_splits\": 5,\n",
    "        \"params\": {},\n",
    "    },\n",
    "})\n",
    "\n",
    "print(\"Config set with AUC, LogLoss, F1, precision_at_k (k=20), and Platt calibration\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Run Fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Results — Binary Classification Metrics & Plots\n",
    "\n",
    "After fitting, the Results tab shows:\n",
    "\n",
    "**Metrics** (IS = in-sample, OOS = out-of-sample):\n",
    "- **AUC** — Area Under ROC Curve\n",
    "- **LogLoss** — Logarithmic Loss\n",
    "- **F1** — F1 Score\n",
    "- **precision_at_k (k=20)** — Precision among top-k predictions\n",
    "\n",
    "**Binary-specific plots**:\n",
    "- **ROC Curve** — Receiver Operating Characteristic\n",
    "- **Calibration** — Predicted vs actual probability (when calibration enabled)\n",
    "- **Probability Histogram** — Distribution of predicted probabilities\n",
    "- **Feature Importance** — Split, Gain, and SHAP variants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary = w.get_fit_summary()\n",
    "if summary:\n",
    "    print(f\"Fold count: {summary.fold_count}\")\n",
    "    for name, vals in summary.metrics.items():\n",
    "        if isinstance(vals, dict):\n",
    "            oos = vals.get(\"oos\", \"N/A\")\n",
    "            print(f\"  {name}: OOS={oos}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Tune — Hyperparameter Optimization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.tune()\n",
    "\n",
    "tune_summary = w.get_tune_summary()\n",
    "if tune_summary:\n",
    "    print(f\"Best score:  {tune_summary.best_score:.4f}\")\n",
    "    print(f\"Metric:      {tune_summary.metric_name} ({tune_summary.direction})\")\n",
    "    print(f\"Best params: {tune_summary.best_params}\")\n",
    "    print(f\"Trials:      {len(tune_summary.trials)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Inference — Predict with Probabilities\n",
    "\n",
    "For binary classification, predictions include probability estimates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test = df.drop(columns=[\"target\"]).tail(10)\n",
    "\n",
    "result = w.predict(df_test)\n",
    "if result:\n",
    "    print(f\"Predictions shape: {result.predictions.shape}\")\n",
    "    print(f\"Columns: {list(result.predictions.columns)}\")\n",
    "    result.predictions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Save Config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.save_config(\"binary_config.yaml\")\n",
    "print(\"Config saved to binary_config.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10. Export Code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export standalone training code (no LizyML dependency needed)\n",
    "export_path = w.export_code(\"./exported_code\")\n",
    "print(f\"Code exported to: {export_path}\")\n",
    "print(\"Generated files:\")\n",
    "for f in sorted(export_path.rglob(\"*\")):\n",
    "    if f.is_file():\n",
    "        print(f\"  {f.relative_to(export_path)}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
