{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.0"
  },
  "colab": {
   "provenance": []
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cell-0",
   "metadata": {},
   "source": [
    "# insurance-gam\n",
    "\n",
    "**Interpretable GAM toolkit for insurance pricing.**\n",
    "\n",
    "GLMs are the industry standard in UK insurance pricing: interpretable,\n",
    "well-understood, regulator-friendly. But they leave predictive power on the\n",
    "table when non-linear effects and interactions matter.\n",
    "\n",
    "This package gives you three alternatives that sit between a GLM and a\n",
    "black-box gradient booster: all interpretable, all exposure-aware, all testable.\n",
    "This notebook covers the most accessible entry point: `InsuranceEBM`\n",
    "(Explainable Boosting Machine) from the `ebm` subpackage.\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/burning-cost/insurance-gam/blob/main/notebooks/quickstart.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-1",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -q \"insurance-gam[ebm]\" polars numpy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-2",
   "metadata": {},
   "source": [
    "## Synthetic motor pricing data\n",
    "\n",
    "We generate 3,000 policies with a Poisson claim count target.\n",
    "The true model has:\n",
    "- A non-linear young-driver effect (age < 25 adds 50% frequency)\n",
    "- A linear NCD discount (−12% per year)\n",
    "- An old-vehicle load (vehicle age > 10 adds 30%)\n",
    "\n",
    "No interaction between rating factors. The EBM should recover this cleanly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import polars as pl\n",
    "\n",
    "rng = np.random.default_rng(42)\n",
    "n = 3_000\n",
    "\n",
    "df = pl.DataFrame({\n",
    "    \"driver_age\":    rng.integers(17, 75, n).astype(float),\n",
    "    \"vehicle_age\":   rng.integers(0, 15, n).astype(float),\n",
    "    \"ncd_years\":     rng.integers(0, 9, n).astype(float),\n",
    "    \"annual_miles\":  rng.integers(3000, 20000, n).astype(float),\n",
    "    \"area\":          rng.integers(0, 5, n).astype(float),\n",
    "})\n",
    "exposure = rng.uniform(0.3, 1.0, n)\n",
    "\n",
    "log_rate = (\n",
    "    -2.5\n",
    "    + 0.5 * (df[\"driver_age\"].to_numpy() < 25).astype(float)\n",
    "    - 0.12 * df[\"ncd_years\"].to_numpy()\n",
    "    + 0.3 * (df[\"vehicle_age\"].to_numpy() > 10).astype(float)\n",
    ")\n",
    "y = rng.poisson(np.exp(log_rate) * exposure)\n",
    "\n",
    "print(f\"n={n:,} policies | mean claim freq: {(y/exposure).mean():.4f}\")\n",
    "print(f\"Claims: {y.sum()} total\")\n",
    "df.head(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-4",
   "metadata": {},
   "source": [
    "## Fit the EBM\n",
    "\n",
    "`InsuranceEBM` wraps interpretML's ExplainableBoostingRegressor with\n",
    "insurance-specific defaults: Poisson loss, exposure offset,\n",
    "and a `RelativitiesTable` that produces the kind of output a pricing\n",
    "actuary can review factor by factor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from insurance_gam.ebm import InsuranceEBM, RelativitiesTable\n",
    "\n",
    "train_n = 2400\n",
    "model = InsuranceEBM(loss=\"poisson\", interactions=\"3x\")\n",
    "model.fit(df[:train_n], y[:train_n], exposure=exposure[:train_n])\n",
    "\n",
    "print(\"EBM fitted. Feature importances:\")\n",
    "importances = model.feature_importances()\n",
    "for feat, imp in sorted(importances.items(), key=lambda x: -x[1]):\n",
    "    bar = '#' * int(imp * 40)\n",
    "    print(f\"  {feat:<20} {imp:.4f}  {bar}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-6",
   "metadata": {},
   "source": [
    "## Relativities table\n",
    "\n",
    "`RelativitiesTable` converts EBM shape functions into the multiplicative\n",
    "relativity tables pricing teams use. Each value is relative to the\n",
    "grand mean — 1.0 means no loading, 1.5 means +50%.\n",
    "\n",
    "This is the output format a pricing actuary can compare against their\n",
    "existing GLM relativities and challenge rationally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-7",
   "metadata": {},
   "outputs": [],
   "source": [
    "rt = RelativitiesTable(model)\n",
    "\n",
    "# NCD relativities — should be monotone decreasing\n",
    "print(\"NCD years relativities:\")\n",
    "print(rt.table(\"ncd_years\").to_pandas().to_string(index=False))\n",
    "\n",
    "print()\n",
    "print(\"Driver age relativities (first 8 bands):\")\n",
    "age_rel = rt.table(\"driver_age\")\n",
    "print(age_rel.head(8).to_pandas().to_string(index=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-8",
   "metadata": {},
   "source": [
    "## Out-of-sample Gini\n",
    "\n",
    "The acid test: does the EBM discriminate better than a flat GLM on held-out data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cell-9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from insurance_gam.ebm import gini\n",
    "\n",
    "# Predictions on holdout set\n",
    "y_test = y[train_n:]\n",
    "exposure_test = exposure[train_n:]\n",
    "y_pred_ebm = model.predict(df[train_n:])\n",
    "\n",
    "# Flat (null) model: same prediction for everyone\n",
    "y_pred_flat = np.full_like(y_pred_ebm, y_pred_ebm.mean())\n",
    "\n",
    "gini_ebm  = gini(y_test, y_pred_ebm,  exposure=exposure_test)\n",
    "gini_flat = gini(y_test, y_pred_flat, exposure=exposure_test)\n",
    "\n",
    "print(f\"Gini (EBM):       {gini_ebm:.3f}\")\n",
    "print(f\"Gini (flat null): {gini_flat:.3f}\")\n",
    "print(f\"Lift over null:   {gini_ebm - gini_flat:+.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cell-10",
   "metadata": {},
   "source": [
    "## What you should see\n",
    "\n",
    "- `ncd_years` and `driver_age` should be the top two features by importance.\n",
    "- NCD relativities should be monotone decreasing (0 NCD = highest multiplier,\n",
    "  8 NCD = lowest). Annual miles and area should be near 1.0.\n",
    "- EBM Gini should be materially above the flat model Gini.\n",
    "\n",
    "## Next steps\n",
    "\n",
    "- **`MonotonicityEditor`** — enforce monotone constraints on any feature\n",
    "  (NCD must decrease, age must be U-shaped)\n",
    "- **`GLMComparison`** — run a side-by-side lift comparison between the EBM and\n",
    "  an existing GLM on the same holdout\n",
    "- **`insurance_gam.anam`** — Actuarial Neural Additive Model (deep learning,\n",
    "  same interpretability)\n",
    "- **`insurance_gam.pin`** — Pairwise Interaction Networks for modelling\n",
    "  two-way interactions explicitly\n",
    "\n",
    "**GitHub:** https://github.com/burning-cost/insurance-gam  \n",
    "**PyPI:** https://pypi.org/project/insurance-gam/"
   ]
  }
 ]
}
