{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ae2b8c7c-6795-4ce6-9367-22dbffe61e45",
   "metadata": {},
   "source": [
    "# Evaluating Experiment Designs Using Fisher Information\n",
    "In this notebook we step through an example of using the Fisher information matrix of an experiment design to evaluate its theoretical performance. This sort of analysis was done in these two papers : [1](https://arxiv.org/abs/2307.15767) [2](https://arxiv.org/abs/2308.08781)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "37b4eeb1-a0e1-4257-972c-3db5eefb074c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pygsti\n",
    "from pygsti.modelpacks import smq1Q_XYI\n",
    "from pygsti.tools.edesigntools import calculate_fisher_information_matrices_by_L\n",
    "from pygsti.protocols import StandardGSTDesign\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69337517-acb6-4416-8c0b-379b5583928a",
   "metadata": {},
   "source": [
    "Start by loading in a target model from a model pack and creating an experiment design."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "759160df-300e-4eb9-9b82-74a3a2c967cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "target_model = smq1Q_XYI.target_model('full TP')\n",
    "edesign = smq1Q_XYI.create_gst_experiment_design(128)\n",
    "maxLs = [1,2,4,8,16,32,64,128]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8322cc9-c3b9-4da2-9da6-48429bc8d129",
   "metadata": {},
   "source": [
    "The fisher information is a tool for evaluating the theoretical performance of an experiment design. It corresponds to the variance of the score, which under suitable regularity conditions reducts to the hessian of the log-likelihood function with respect to the model parameters. Definitition aside, the power of the Fisher information comes from the Cramer-Rao bound, which says that the covariance matrix of estimates generated with respect to an experiment design is lower bounded by the inverse of that design's Fisher information matrix.\n",
    "\n",
    "Below we will calculate the Fisher information matrices for each nested iteration of the GST experiment design we have constructed (we'll assume 100 shots per-circuit)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "59395181-0eca-4238-829a-2d8a158e8816",
   "metadata": {},
   "outputs": [],
   "source": [
    "fisher_infos_by_L = calculate_fisher_information_matrices_by_L(target_model, edesign.circuit_lists, maxLs, num_shots=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcf2a397-0f48-4064-82eb-4a68b95e9a30",
   "metadata": {},
   "source": [
    "Next we will calculate the eigenvalues of each of these Fisher information matrices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ef975c3e-945b-4a35-8335-0fb698e07c38",
   "metadata": {},
   "outputs": [],
   "source": [
    "fim_spectra_by_L = [np.linalg.eigvalsh(fim) for fim in fisher_infos_by_L.values()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee25f282-0c46-4ceb-bd37-77caab7f8452",
   "metadata": {},
   "source": [
    "The reciprocals of these eigenvalues correspond to the lengths of the axes of the uncertainty ellipsoid defined by the covariance for our estimate. So, for a well-constructed GST experiment design we expect to see a linear increase in the Fisher information eigenvalues associated with amplifiable parameters (non-SPAM, non-gauge). \n",
    "\n",
    "We can evaluate this by simply plotting these directly. (Helper function defined below)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f3620cb9-be82-408c-9c30-65734c9738a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_spectra_by_L(fisher_spectra_by_L, Ls, num_gauge, figure_size=None):\n",
    "    #In this version only plot the positive part of the spectra\n",
    "    \n",
    "    #convert this spectra to a numpy array\n",
    "    fisher_array= np.asarray(fisher_spectra_by_L)\n",
    "    #this should have rows corresponding to the length and columns corresponding to the eigenvalues at that length in increasing order.\n",
    "    #to get eigenvalue bands we should be able to just take the columns and plot those.\n",
    "        \n",
    "    #drop the stated number of gauge parameters\n",
    "    if figure_size is not None:\n",
    "        plt.figure(figsize=figure_size)\n",
    "    \n",
    "    for i in range(num_gauge, fisher_array.shape[1]):\n",
    "        plt.plot(Ls, fisher_array[:,i])\n",
    "\n",
    "    #also plot a line for the x axis.\n",
    "    plt.plot(Ls, [0]*len(Ls), 'k')\n",
    "    plt.xscale('log')\n",
    "    plt.yscale('log')\n",
    "    plt.xticks(Ls, Ls)\n",
    "    \n",
    "    #Addtional Formatting\n",
    "    plt.xlabel(r'$L$, (maximum base circuit depth)', fontsize=16)\n",
    "    plt.xlim([0.9, Ls[-1]])\n",
    "    plt.ylabel('Eigenvalue', fontsize=16)\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "b64bf89e-a2f2-426c-9d73-91e1afd5b9de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 171300 characters]\n",
      "text/plain": [
       "<Figure size 500x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_spectra_by_L(fim_spectra_by_L, maxLs, num_gauge=target_model.num_gauge_params, figure_size=(5,8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61d3cdfd-bee1-4911-8a74-7b647718afde",
   "metadata": {},
   "source": [
    "A linear scaling of the Fisher information eigenvalues corresponding to amplifiable parameters is exactly what we get here. You may notice that there are 6 traces that are very clearly plateauing. These correspond to the 6 non-gauge SPAM parameters in this particular gate set model, so this too is as expected since SPAM parameters are not amplifiable (essentially because SPAM operations only ever happen once in a circuit).\n",
    "\n",
    "What about with an experiment design that we know is incomplete? One simple example would be an experiment design constructed with respect to a germ set we call the 'bare' germs, which are simply germs correspoding to the gates of the gate set themselves, and is known not to be amplificationally complete for this gate set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "da306201-0fac-46cf-9c77-cdad1c9ba25d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Circuit([]@(0)), Circuit(Gxpi2:0@(0)), Circuit(Gypi2:0@(0))]\n"
     ]
    }
   ],
   "source": [
    "bare_germs= smq1Q_XYI.germs()[0:3]\n",
    "print(bare_germs)\n",
    "prep_fids = smq1Q_XYI.prep_fiducials()\n",
    "meas_fids = smq1Q_XYI.meas_fiducials()\n",
    "incomplete_edesign = StandardGSTDesign(target_model, prep_fids, meas_fids, bare_germs, maxLs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "c7bee205-3bb4-4bdb-9739-2b470a584ea1",
   "metadata": {},
   "outputs": [],
   "source": [
    "fisher_infos_by_L_incomplete = calculate_fisher_information_matrices_by_L(target_model, incomplete_edesign.circuit_lists, maxLs, num_shots=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "42da40f2-b69f-44f0-ae4a-88d3b506de0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "fim_spectra_by_L_incomplete = [np.linalg.eigvalsh(fim) for fim in fisher_infos_by_L_incomplete.values()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "e22a84c6-7cc9-49ee-90f1-4e12317ec5c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 149952 characters]\n",
      "text/plain": [
       "<Figure size 500x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_spectra_by_L(fim_spectra_by_L_incomplete, maxLs, num_gauge=target_model.num_gauge_params, figure_size=(5,8))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79594032-4963-4b07-b05c-c8dab6dab425",
   "metadata": {},
   "source": [
    "Here we can see that we now have at least 10 fisher information eigenvalues plateuing, more than can be accounted for due to SPAM alone. This indicates that we are no longer achieving heisenberg-limited precision scaling for our estimates of at least 4 amplifiable parameters of the gate set."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "New_FPR",
   "language": "python",
   "name": "new_fpr"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
