{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quantum Circuit Learning\n",
    "**Quantum Circuit Learning** (QCL) is an algorithm for applying quantum computers to machine learning [1].\n",
    "Just like the VQE (Variational Quantum Eigensolver) we have learned in the previous section, it is a **quantum-classical hybrid algorithm**, designed to operate on **NISQ** (Noisy Intermediate-Scale Quantum Computer), a medium-scale quantum computer without error correction function.\n",
    "\n",
    "Experiments using actual NISQ devices have already been performed, and in March 2019, a paper on actual implementation of QCL by an IBM experiment team [2] was published in Nature and became a hot topic.\n",
    "\n",
    "In the following, the outline of the algorithm and the specific learning procedure are introduced, then an implementation example using the quantum simulator Qulacs is presented.\n",
    "\n",
    "This notebook is tranlated from https://dojo.qulacs.org/ja/latest/notebooks/5.1_variational_quantum_eigensolver.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview of QCL\n",
    "In recent years, deep learning has been spotlighted in the field of machine learning.\n",
    "In deep learning, by approximating a complex function using a deep neural network, the relationship between input and output can be learned and predictions can be performed on new data. QCL is a machine learning method that replaces this neural network with a **quantum circuit**, and hence a quantum computer. \n",
    "\n",
    "By using the quantum circuit, the learning process can be **performed exponentially by using a large number of basis functions** by utilizing the principle of superposition of quantum mechanics, so that the expression capacity of the model is improved. \n",
    "\n",
    "Furthermore, it is considered that overfitting can be automatically prevented depending on the condition (unitary property) to be satisfied by the quantum circuit. As a result, higher performance can be expected beyond machine learning in classical computers. (See Reference [1] for details)\n",
    "\n",
    "In a neural network, the function is approximated by adjusting the weight parameter *W* of each layer, and the concept is exactly the same in QCL.\n",
    "\n",
    "That is, although the quantum circuit used in the QCL includes multiple “rotating gates”, the function is approximated by adjusting the rotating angle $\\theta$ of the rotating gate. The specific procedure is shown below.\n",
    "\n",
    "## Learning procedure\n",
    "1. Prepare training data {($x_i,y_i$)}. ($x_i$ is input data (teacher data), $y_i$ is the correct output data expected to be predicted from $x_i$)\n",
    "2. Prepare a circuit called $U_\\text{in}(x)$ that is determined by some rule from the input $x$, and create an input state $\\{\\left|\\psi_{\\text{in}}(x_i)\\right>\\}_i=\\{U_{\\text{in}}(x_i)\\left|0\\right>\\}_i$ with the information of $x_i$ embedded.\n",
    "3. Multiply gate $U(\\theta)$ which depends on parameter $\\theta$ with the input state to obtain the output state $\\{\\left|\\psi_{\\text{out}}(x_i,\\theta)\\right>=U(\\theta)\\left|\\psi_{\\text{in}}(x_i)\\right>\\}_i$.\n",
    "4. The measurement is done by measuring some observable under the output state. (eg. the $Z$ expection of the first qubit: $\\left<Z_1\\right>=\\left<\\psi_{\\text{out}}\\left|Z_1\\right|\\psi_{\\text{out}}\\right>$)\n",
    "5. Set F as an appropriate function (sigmoid, softmax or constant function, etc.), and the output $y(x_i,\\theta)$ of the model is F(measurement_i).\n",
    "6. Calculate the cost function $L(\\theta)$ representing the divergence between the correct data $\\{y_i\\}$ and the output $\\{y(x_i,\\theta)\\}_i$ of the model.\n",
    "7. Obtain the $\\theta=\\theta^*$ which minimizes the cost function.\n",
    "8. Then $y(x,\\theta^*)$ is the desired prediction model.\n"
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "[encoded data, 78720 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![QCL.png](img/QCL.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(In the QCL, input data $x$ is first converted to a quantum state using $U_{\\text{in}}(x)$, and an output $y$ is obtained there from using a variational quantum circuit $U(\\theta)$ and measurement (In the figure, the output is $\\left<B(x,\\theta)\\right>$.) Source: Revised Figure 1 in reference [1].)\n",
    "\n",
    "## Implementation using quantum simulator Qulacs \n",
    "In the following, a fitting of sin function $y=sin(\\pi x)$ is performed as a demonstration of approximating function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from functools import reduce"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "########  Parameter  #############\n",
    "nqubit = 3 ## number of qubit\n",
    "c_depth = 3 ## depth of circuit\n",
    "time_step = 0.77  ## elapsed time of time evolution with random Hamiltonian\n",
    "\n",
    "## randomly take num_x_train points from [x_min, x_max] as teacher data.\n",
    "x_min = - 1.; x_max = 1.;\n",
    "num_x_train = 50\n",
    "\n",
    "## one variable function to learn\n",
    "func_to_learn = lambda x: np.sin(x*np.pi)\n",
    "\n",
    "## seed of random number\n",
    "random_seed = 0\n",
    "## initialization of random number generator\n",
    "np.random.seed(random_seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 8340 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#### Prepare teacher data\n",
    "x_train = x_min + (x_max - x_min) * np.random.rand(num_x_train)\n",
    "y_train = func_to_learn(x_train)\n",
    "\n",
    "# Add noise to pure sine function assuming real data used\n",
    "mag_noise = 0.05\n",
    "y_train = y_train + mag_noise * np.random.randn(num_x_train)\n",
    "\n",
    "plt.plot(x_train, y_train, \"o\"); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Construct the input state\n",
    "Firstly, create a gate $U_{\\text{in}}(x_i)$ for embedding the input value $x_i$ in the initial state $\\left|00...0\\right>$.\n",
    "According to reference [1], define $U_{\\text{in}}(x_i)=\\prod_jR^Z_j(\\text{cos}^{-1}x^2)R^Y_j(\\text{sin}^{-1}x)$ using rotation gates $R^Y_j(\\theta)=e^{i\\theta Y_j/2}, R^Z_j(\\theta)=e^{i\\theta Z_j/2}$.\n",
    "The input $x_i$ is converted into quantum state $\\left|\\psi_\\text{in}(x_i)\\right>=U_{\\text{in}}(x_i)\\left|00...0\\right>$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: qulacs in /anaconda3/envs/VibSpec/lib/python3.7/site-packages (0.1.9)\r\n"
     ]
    }
   ],
   "source": [
    "## When using Google Colaboratory・please run in a local environment where Qulacs is not installed.\n",
    "!pip install qulacs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.+0.j 0.+0.j 0.+0.j 0.+0.j 0.+0.j 0.+0.j 0.+0.j 0.+0.j]\n"
     ]
    }
   ],
   "source": [
    "# Create intial state\n",
    "from qulacs import QuantumState, QuantumCircuit\n",
    "\n",
    "state = QuantumState(nqubit) # Initial state |000>\n",
    "state.set_zero_state()\n",
    "print(state.get_vector())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function that creats a gate encoding x\n",
    "def U_in(x):\n",
    "    U = QuantumCircuit(nqubit)\n",
    "\n",
    "    angle_y = np.arcsin(x)\n",
    "    angle_z = np.arccos(x**2)\n",
    "\n",
    "    for i in range(nqubit):\n",
    "        U.add_RY_gate(i, angle_y)\n",
    "        U.add_RZ_gate(i, angle_z)\n",
    "\n",
    "    return U"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-6.93804351e-01+7.14937415e-01j -3.54871219e-02-3.51340074e-02j\n",
      " -3.54871219e-02-3.51340074e-02j  1.77881430e-03-1.76111422e-03j\n",
      " -3.54871219e-02-3.51340074e-02j  1.77881430e-03-1.76111422e-03j\n",
      "  1.77881430e-03-1.76111422e-03j  8.73809020e-05+9.00424970e-05j]\n"
     ]
    }
   ],
   "source": [
    "# Test initial state\n",
    "x = 0.1 # appropriate value\n",
    "U_in(x).update_quantum_state(state) # calculation of U_in|000>\n",
    "print(state.get_vector())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Construct variational quantum circuit $U(\\theta)$\n",
    "Next, a variational quantum circuit $U(\\theta)$ to be optimized is created. The procedure is the following three steps:\n",
    "1. Create a transverse magnetic field Ising Hamiltonian.\n",
    "2. Create rotation gates.\n",
    "3. Combine step1 and step2's gates alternatively to make a large variational quantum circuit $U(\\theta)$.\n",
    "\n",
    "#### 1.Create a transverse magnetic field Ising Hamiltonian\n",
    "The expressiveness of the model is enhanced by increasing the complexity (entanglement) of the quantum circuit after performing time evolution based on the transverse magnetic field Ising model learned in section 4-2. (This part can be skipped unless you want to know the details.)\n",
    "\n",
    "The Hamiltonian of transverse magnetic file Ising model is shown below, time evolution operator is defined as $U_{\\text{rand}}=e^{-iHt}$:\n",
    "\n",
    "\\begin{equation}\n",
    "H=\\sum_{j=1}^N a_jX_j+\\sum_{j=1}^N\\sum_{k=1}^{j-1}J_{jk}Z_jZ_k\n",
    "\\end{equation}\n",
    "\n",
    "Coefficient $a$ and $J$ is a uniform distribution of $[-1,1]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Basic gate\n",
    "from qulacs.gate import X, Z\n",
    "I_mat = np.eye(2, dtype=complex)\n",
    "X_mat = X(0).get_matrix()\n",
    "Z_mat = Z(0).get_matrix()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Function that creates fullsize gate.\n",
    "def make_fullgate(list_SiteAndOperator, nqubit):\n",
    "    '''\n",
    "    Take list_SiteAndOperator = [ [i_0, O_0], [i_1, O_1], ...],\n",
    "    Insert Identity into unrelated qubit\n",
    "    make (2**nqubit, 2**nqubit) matrix:\n",
    "    I(0) * ... * O_0(i_0) * ... * O_1(i_1) ...\n",
    "    '''\n",
    "    list_Site = [SiteAndOperator[0] for SiteAndOperator in list_SiteAndOperator]\n",
    "    list_SingleGates = [] ## Arrange 1-qubit gates and reduce with np.kron\n",
    "    cnt = 0\n",
    "    for i in range(nqubit):\n",
    "        if (i in list_Site):\n",
    "            list_SingleGates.append( list_SiteAndOperator[cnt][1] )\n",
    "            cnt += 1\n",
    "        else: ## an empty site is identity\n",
    "            list_SingleGates.append(I_mat)\n",
    "\n",
    "    return reduce(np.kron, list_SingleGates)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### Create time evolution operator by making random magnetic field and random coupling Ising Hamiltonian\n",
    "ham = np.zeros((2**nqubit,2**nqubit), dtype = complex)\n",
    "for i in range(nqubit): ## i runs 0 to nqubit-1\n",
    "    Jx = -1. + 2.*np.random.rand() ## random number in -1~1\n",
    "    ham += Jx * make_fullgate( [ [i, X_mat] ], nqubit)\n",
    "    for j in range(i+1, nqubit):\n",
    "        J_ij = -1. + 2.*np.random.rand()\n",
    "        ham += J_ij * make_fullgate ([ [i, Z_mat], [j, Z_mat]], nqubit)\n",
    "\n",
    "## Create a time evolution operator by diagonalization. H*P = P*D <-> H = P*D*P^dagger\n",
    "diag, eigen_vecs = np.linalg.eigh(ham)\n",
    "time_evol_op = np.dot(np.dot(eigen_vecs, np.diag(np.exp(-1j*time_step*diag))), eigen_vecs.T.conj()) # e^-iHT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8, 8)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_evol_op.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert to qulacs gate\n",
    "from qulacs.gate import DenseMatrix\n",
    "time_evol_gate = DenseMatrix([i for i in range(nqubit)], time_evol_op)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.Create rotation gates, 3.Create $U(\\theta)$\n",
    "Combine the time evolution operator $U_{\\text{rand}}$ accroding to random transverse magnetic field Ising model and the operator of rotation gates operating on $j(=1,2,\\dots n)$th qubit\n",
    "\n",
    "\\begin{equation}\n",
    "U_\\text{rot}(\\theta_j^{(i)})=R_j^X(\\theta_{j1}^{(i)})R_j^Z(\\theta_{j2}^{(i)})R_j^X(\\theta_{j3}^{(i)})\n",
    "\\end{equation}\n",
    "\n",
    "to create variational quantum circuit $U(\\theta)$:\n",
    "\n",
    "\\begin{equation}\n",
    "U\\Big(\\{\\theta_j^{(i)}\\}_{i,j}\\Big)= \\prod_{i=1}^d\\bigg(\\bigg(\\prod_{j=1}^nU_\\text{rot}(\\theta_j^{(i)})\\bigg)\\cdot U_\\text{rand} \\bigg)\n",
    "\\end{equation}\n",
    "\n",
    "Here $i$ is a suffix representing the layer of the quantum circuit, and $U_\\text{rand}$ and the above rotation are repeated in $d$ layers in total. There are $3\\times n \\times d$ parameters. The intial value of each $\\theta$ is a uniform distribution of $[0,2\\pi]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qulacs import ParametricQuantumCircuit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assemble output gate U_out & set initial parameter values\n",
    "U_out = ParametricQuantumCircuit(nqubit)\n",
    "for d in range(c_depth):\n",
    "    U_out.add_gate(time_evol_gate)\n",
    "    for i in range(nqubit):\n",
    "        angle = 2.0 * np.pi * np.random.rand()\n",
    "        U_out.add_parametric_RX_gate(i,angle)\n",
    "        angle = 2.0 * np.pi * np.random.rand()\n",
    "        U_out.add_parametric_RZ_gate(i,angle)\n",
    "        angle = 2.0 * np.pi * np.random.rand()\n",
    "        U_out.add_parametric_RX_gate(i,angle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the list of initial values of the parameter theta\n",
    "parameter_count = U_out.get_parameter_count()\n",
    "theta_init = [U_out.get_parameter(ind) for ind in range(parameter_count)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[6.007250646127814,\n",
       " 4.046309757767312,\n",
       " 2.663159813474645,\n",
       " 3.810080933381979,\n",
       " 0.12059442161498848,\n",
       " 1.8948504571449056,\n",
       " 4.14799267096281,\n",
       " 1.8226113595664735,\n",
       " 3.88310546309581,\n",
       " 2.6940332019609157,\n",
       " 0.851208649826403,\n",
       " 1.8741631278382846,\n",
       " 3.5811951525261123,\n",
       " 3.7125630518871535,\n",
       " 3.6085919651139333,\n",
       " 4.104181793964002,\n",
       " 4.097285684838374,\n",
       " 2.71068197476515,\n",
       " 5.633168398253273,\n",
       " 2.309459341364396,\n",
       " 2.738620094343915,\n",
       " 5.6041197193647925,\n",
       " 5.065466226710866,\n",
       " 4.4226624059922806,\n",
       " 0.6297441057449945,\n",
       " 5.777279648887616,\n",
       " 4.487710439107831]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta_init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, a function for updating parameter $\\theta$ in $U(\\theta)$ is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function that updates parameter theta\n",
    "def set_U_out(theta):\n",
    "    global U_out\n",
    "\n",
    "    parameter_count = U_out.get_parameter_count()\n",
    "\n",
    "    for i in range(parameter_count):\n",
    "        U_out.set_parameter(i, theta[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Measurement\n",
    "In this demonstration, the output of the model is the expectation value of 0th qubit's Pauli Z on output state $\\left|\\psi_\\text{out}\\right>$, that is:\n",
    "\n",
    "\\begin{equation}\n",
    "y(\\theta,x_i)=\\left<Z_0\\right>=\\left<\\psi_\\text{out}|Z_0|\\psi_\\text{out}\\right>\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create observable Z_0\n",
    "from qulacs import Observable\n",
    "obs = Observable(nqubit)\n",
    "obs.add_operator(2.,'Z 0') \n",
    "# Set observable as 2*Z。\n",
    "# The reason for multiplying by 2 here is to expand the value range of the final <Z>.\n",
    "# In order to cope with any unknown function, this constant also needs to be optimized as one parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.9899748742132415"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obs.get_expectation_value(state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combine a series of procedures into one function\n",
    "The procedures up to this point can be combined to define a function that returns the predicted value $y(x_i,\\theta)$ of the model from the input $x_i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function that gives prediction value y(x_i, theta) of the model from input x_i\n",
    "def qcl_pred(x, U_out):\n",
    "    state = QuantumState(nqubit)\n",
    "    state.set_zero_state()\n",
    "\n",
    "    # Calculate input state\n",
    "    U_in(x).update_quantum_state(state)\n",
    "\n",
    "    # Calculate output state\n",
    "    U_out.update_quantum_state(state)\n",
    "\n",
    "    # Output of the model\n",
    "    res = obs.get_expectation_value(state)\n",
    "\n",
    "    return res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculation of cost function\n",
    "The cost function $L(\\theta)$ is a mean square error (MSE) between the teacher data and the prediction data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate cost function L\n",
    "def cost_func(theta):\n",
    "    '''\n",
    "    theta: ndarray of length c_depth * nqubit * 3\n",
    "    '''\n",
    "    # update the parameter theta of U_out\n",
    "    # global U_out\n",
    "    set_U_out(theta)\n",
    "\n",
    "    # calculate basing on data of num_x_train in total\n",
    "    y_pred = [qcl_pred(x, U_out) for x in x_train]\n",
    "\n",
    "    # quadratic loss\n",
    "    L = ((y_pred - y_train)**2).mean()\n",
    "\n",
    "    return L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.38892593161935"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Value of cost function with initial parameter theta\n",
    "cost_func(theta_init)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11e0de090>]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 12552 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Figure basing on inital parameter theta\n",
    "xlist = np.arange(x_min, x_max, 0.02)\n",
    "y_init = [qcl_pred(x, U_out) for x in xlist]\n",
    "plt.plot(xlist, y_init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning (optimization by scipy.optimize.minimize)\n",
    "Preparation is finally finished, and let's start learning from now on. Here, for simplicity, optimization is performed using the Nelder-Mead method, which does not need a gradient calculation formula. When using an optimization method that needs gradients (eg: the BFGS method), refer to the useful gradients calculation formulas introduced in Reference [1]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.optimize import minimize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 11.7 s, sys: 11.7 s, total: 23.4 s\n",
      "Wall time: 14.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Learning (takes 14 seconds with the writer's PC)\n",
    "result = minimize(cost_func, theta_init, method='Nelder-Mead')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.003987076559624772"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Value of cost_function after optimization\n",
    "result.fun"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7.17242144 5.4043736  1.27744316 3.09192904 0.13144047 2.13757354\n",
      " 4.58470259 2.01924008 2.96107066 2.91843537 1.0609229  1.70351774\n",
      " 6.41114609 6.25686828 2.41619471 3.69387805 4.07551328 1.47666316\n",
      " 3.4108701  2.28524042 1.75253621 7.44181397 3.20314179 5.11364648\n",
      " 1.2831137  2.07306927 3.75112591]\n"
     ]
    }
   ],
   "source": [
    "# Solution of theta by optimization\n",
    "theta_opt = result.x\n",
    "print(theta_opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Insert optimized theta into U_out\n",
    "set_U_out(theta_opt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 36736 characters]",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot\n",
    "plt.figure(figsize=(10, 6))\n",
    "\n",
    "xlist = np.arange(x_min, x_max, 0.02)\n",
    "\n",
    "# teacher data\n",
    "plt.plot(x_train, y_train, \"o\", label='Teacher')\n",
    "\n",
    "# Figure basing on inital parameter theta\n",
    "plt.plot(xlist, y_init, '--', label='Initial Model Prediction', c='gray')\n",
    "\n",
    "# Prediction of the model\n",
    "y_pred = np.array([qcl_pred(x, U_out) for x in xlist])\n",
    "plt.plot(xlist, y_pred, label='Final Model Prediction')\n",
    "\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is clear that the approximation of the sin function was successful.\n",
    "\n",
    "Here we dealt with a very simple task of a one-dimensional function approximation for both input and output, but it can be extended to approximation and classification problems with multidimensional inputs and outputs.\n",
    "\n",
    "Motivated readers are encouraged to try to classify the [Iris dataset](https://scikit-learn.org/stable/auto_examples/datasets/plot_iris_dataset.html), one of the typical machine learning datasets, in column `5.2c. Application of QCL to Machine Learning`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference\n",
    "[1] K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, “Quantum circuit learning”, [Phys. Rev. A 98, 032309 (2018)](https://journals.aps.org/pra/abstract/10.1103/PhysRevA.98.032309), arXiv：https://arxiv.org/abs/1803.00745\n",
    "\n",
    "[2] V. Havlicek et al. , “Supervised learning with quantum-enhanced feature spaces”, [Nature 567, 209–212 (2019)](https://www.nature.com/articles/s41586-019-0980-2), arXiv：https://arxiv.org/abs/1804.11326"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
