{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "plaintext"
    }
   },
   "source": [
    "# Combined projections\n",
    "While the **Projections** notebook touched upon how to set up simple projections it is more interesting to see how more complex models can be set up and utilized. \n",
    "`suppy.projections`contains general implementations for projection methods that allow to find a poin in the intersection of the individual feasible sets. \n",
    "\n",
    "These are:\n",
    "- SequentialProjection\n",
    "- SimultaneousProjection\n",
    "- BlockIterativeProjection\n",
    "- StringAveragedProjection\n",
    "\n",
    "Each of these projection methods has a `project` function that allows to perform a single feasibility seeking step, as well as a general `solve` function.\n",
    "While the mathematical formulation of each of the latter three allows for parallel computation the current framework implements them sequentially."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's set up a simple model that tries to find a point in the intersection of two halfspaces $H_1$ and $H_2$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from suppy.projections import HalfspaceProjection\n",
    "from suppy.projections import SequentialProjection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "a_1 = np.array([1,1])\n",
    "a_2 = np.array([-1,0])\n",
    "b_1 = 0\n",
    "b_2 = 0\n",
    "Halfspace_1 = HalfspaceProjection(a_1, b_1)\n",
    "Halfspace_2 = HalfspaceProjection(a_2, b_2)\n",
    "seq_model = SequentialProjection([Halfspace_1, Halfspace_2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25436 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# We can also visualize the model using matplotlib:\n",
    "import matplotlib.pyplot as plt\n",
    "fig,ax = plt.subplots()\n",
    "seq_model.visualize(ax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Projection\n",
    "In this case we try to find a point in the above dark blue shaded area.\n",
    "For this the `SequentialProjection` iteratively projects onto $H_1$ and $H_2$:\n",
    "\\begin{equation}\n",
    "x_1 = P_{H_2}(P_{H_1}(x_0))\n",
    "\\end{equation}\n",
    "with $P_{H_i}$ being the projecton onto $H_i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acfe9b8ad0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 38748 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_0 = np.array([2,7.5])\n",
    "x_1 = seq_model.project(x_0.copy())\n",
    "fig,ax = plt.subplots()\n",
    "seq_model.visualize(ax)\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "ax.scatter(x_1[0], x_1[1], color='blue', label = '$x$ after performing $P_{H_2}(P_{H_1}(x_0))$')\n",
    "plt.legend(loc = 'lower left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feasibility seeking\n",
    "While this does not result in a point in the intersection we can perform this operation several times in a row. For this the `solve` function is used which runs until a stopping criterium is met (In this case that the weighted sum of residuals falls below a lower bound or a maximal number of iterations is performed).\n",
    "This results in the following feasible point:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acff9b4710>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 37000 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_0 = np.array([2,7.5])\n",
    "x_1 = seq_model.solve(x_0.copy(),storage = True)\n",
    "fig,ax = plt.subplots()\n",
    "seq_model.visualize(ax)\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "ax.scatter(x_1[0], x_1[1], color='blue', label = 'Feasible solution')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Things to note here:\n",
    "1. The `project` and `solve` method are passed copies of $x_0$. This is due to the `project` function, which is used in all algorithms, **changing the input in place** in a lot of cases.\n",
    "To still have access to the initial vector afterwards it is therefore recommended to just pass a copy.\n",
    "2. The solve function has a `storage` option, which is by default deactivated,that allows to store intermediate solutions $x^k$ of the feasibility seeking process.\n",
    "After termination of the intermediate solutions can be accessed via `model.all_x`\n",
    "3. The algorithm terminates once the proximity drops below a given threshold. By default the proximity value is calculated as a weighted sum of the proximities to the individual  constraints: (weights are chosen uniformly be default)\n",
    "\\begin{equation}\n",
    "\\sum_i w_i Prox_i(x)^2 \\leq Prox_{max}\n",
    "\\end{equation}\n",
    "4. It is possible to select other proximity measures like a max distance norm. For this a LIST of `proximity_measures` can be passed to the solve function. As of now the following options are available:\n",
    "\n",
    "    a) `p_norm` distance ($\\sum_id(x,C_i)^p$) by passing a tuple `(\"p_norm\",p)` with p being the wanted power (default p=2) \n",
    "\n",
    "    b) `max_norm` distance ($\\max_i d(x,C_i)$) by simply passing the string `max_norm`\n",
    "\n",
    "If the passed list contains more than one option all quantities are calculated, however (as of now) only the first one can be chosen for the stopping criterion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acffb58210>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 42520 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig,ax = plt.subplots()\n",
    "seq_model.visualize(ax)\n",
    "ax.plot(seq_model.all_x[:,0],seq_model.all_x[:,1], '-o', color='green', label = 'Intermediate iterates', zorder = 1)\n",
    "ax.scatter(x_1[0], x_1[1], color='blue', label = 'Feasible solution')\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simultaneous Projection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the `SimultaneousProjection`, a new value for $x$ is calculated by projecting the current iterate $x^k$ onto each constraint and then performing a convex combination of the individual ones:\n",
    "\\begin{align}\n",
    "x^{k+1} &= \\sum_i{w_i P_i(x^k)}\\\\\n",
    "\\sum_i{w_i} &= 1\n",
    "\\end{align}\n",
    "By default weights are chosen to be uniform but can be adapted by setting `weights`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from suppy.projections import SimultaneousProjection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Uniform weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acffbea410>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 38568 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "a_1 = np.array([1,1])\n",
    "a_2 = np.array([-1,0])\n",
    "b_1 = 0\n",
    "b_2 = 0\n",
    "Halfspace_1 = HalfspaceProjection(a_1, b_1)\n",
    "Halfspace_2 = HalfspaceProjection(a_2, b_2)\n",
    "sim_model_1 = SimultaneousProjection([Halfspace_1, Halfspace_2])\n",
    "x_1 = sim_model_1.project(x_0.copy())\n",
    "fig,ax = plt.subplots()\n",
    "sim_model_1.visualize(ax)\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "ax.scatter(x_1[0], x_1[1], color='blue', label = '$x$ after performing $P_{H_21}(x_0))$')\n",
    "plt.legend(loc = 'lower left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## With different weights "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acffd54710>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 38668 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "a_1 = np.array([1,1])\n",
    "a_2 = np.array([-1,0])\n",
    "b_1 = 0\n",
    "b_2 = 0\n",
    "Halfspace_1 = HalfspaceProjection(a_1, b_1)\n",
    "Halfspace_2 = HalfspaceProjection(a_2, b_2)\n",
    "sim_model_2 = SimultaneousProjection([Halfspace_1, Halfspace_2],weights = np.array([1/3,2/3]))\n",
    "x_1 = sim_model_2.project(x_0.copy())\n",
    "fig,ax = plt.subplots()\n",
    "sim_model_2.visualize(ax)\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "ax.scatter(x_1[0], x_1[1], color='blue', label = '$x$ after performing $P_{H_21}(x_0))$')\n",
    "plt.legend(loc = 'lower left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since one of the constraints is already met, only the other one affects the projection. With the weight being only $1/3$ the new iterate is different to the uniform weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1acffe7a410>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 46588 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_sim_1 = sim_model_1.solve(x_0.copy(),storage = True)\n",
    "x_sim_2 = sim_model_2.solve(x_0.copy(),storage = True)\n",
    "fig,ax = plt.subplots()\n",
    "sim_model_2.visualize(ax)\n",
    "ax.plot(sim_model_1.all_x[:,0],sim_model_1.all_x[:,1], '-o', color='orange', label = 'Uniform weights', zorder = 1)\n",
    "ax.plot(sim_model_2.all_x[:,0],sim_model_2.all_x[:,1], '-o', color='green', label = 'Different weights', zorder = 1)\n",
    "ax.scatter(x_sim_1[0], x_sim_1[1], color='blue', label = 'Feasible solution')\n",
    "ax.scatter(x_0[0], x_0[1], color='red',label = 'Initial point')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In case that the constraints can be modelled as linear equations, especially if the size of the system increases by a lot it is more efficient to use a dedicated implementation for these linear algorithms. On how this is handled in **SupPy** check out the `linear_feasible_models` notebook."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "suppy",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
