{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gradients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a couple of ways we can interact with the gradients during and after a backward pass."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following example, we save the hidden states of the last layer and do a backward pass on the sum of the logits.\n",
    "\n",
    "Note two things:\n",
    "\n",
    "1. `requires_grad=True` by default.\n",
    "2. We can all `.backward()` on a value within the tracing context just like you normally would."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/share/u/caden/.conda/envs/interp/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from nnsight import LanguageModel\n",
    "\n",
    "model = LanguageModel(\"openai-community/gpt2\", device_map=\"cuda\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a GPT2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[ 0.5216, -1.1755, -0.4617,  ..., -1.1919,  0.0204, -2.0075],\n",
      "         [ 0.9841,  2.2175,  3.5851,  ...,  0.5212, -2.2286,  5.7334]]],\n",
      "       device='cuda:0', grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "with model.trace(\"Hello World\") as tracer:\n",
    "\n",
    "    hidden_states = model.transformer.h[-1].output[0].save()\n",
    "\n",
    "    logits = model.output.logits\n",
    "\n",
    "    logits.sum().backward()\n",
    "\n",
    "print(hidden_states)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we wanted to see the gradients for the hidden_states, we can call `.retain_grad()` on it and access the `.grad` attribute after execution. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[ 0.5216, -1.1755, -0.4617,  ..., -1.1919,  0.0204, -2.0075],\n",
      "         [ 0.9841,  2.2175,  3.5851,  ...,  0.5212, -2.2286,  5.7334]]],\n",
      "       device='cuda:0', grad_fn=<AddBackward0>)\n",
      "tensor([[[  28.7976, -282.5981,  868.7354,  ...,  120.1743,   52.2264,\n",
      "           168.6449],\n",
      "         [  79.4181, -253.6225, 1322.1293,  ...,  208.3980,  -19.5545,\n",
      "           509.9857]]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "with model.trace(\"Hello World\") as tracer:\n",
    "\n",
    "    hidden_states = model.transformer.h[-1].output[0].save()\n",
    "    hidden_states_grad = model.transformer.h[-1].output[0].grad.save()\n",
    "\n",
    "    model.output.logits.sum().backward()\n",
    "\n",
    "print(hidden_states)\n",
    "print(hidden_states_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even better, `nnsight` also provides proxy access into the backward process via the `.grad` attribute on proxies. This works just like  `.input` and `.output` where operations , including getting and setting, are traced and performed on the model at runtime. (assuming it's a proxy of a Tensor as this calls .register_hook(...) on it!)\n",
    "\n",
    "The following examples demonstrate ablating (setting to zero) the gradients for a hidden state in gpt2. The first example is an in-place operation and the second swaps the gradient out for a new tensor of zeroes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a GPT2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before tensor([[[  28.7976, -282.5981,  868.7354,  ...,  120.1743,   52.2264,\n",
      "           168.6449],\n",
      "         [  79.4181, -253.6225, 1322.1293,  ...,  208.3980,  -19.5545,\n",
      "           509.9857]]], device='cuda:0')\n",
      "After tensor([[[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]]], device='cuda:0')\n",
      "Before tensor([[[  28.7976, -282.5981,  868.7354,  ...,  120.1743,   52.2264,\n",
      "           168.6449],\n",
      "         [  79.4181, -253.6225, 1322.1293,  ...,  208.3980,  -19.5545,\n",
      "           509.9857]]], device='cuda:0')\n",
      "After tensor([[[0., 0., 0.,  ..., 0., 0., 0.],\n",
      "         [0., 0., 0.,  ..., 0., 0., 0.]]], device='cuda:0')\n"
     ]
    }
   ],
   "source": [
    "from nnsight import LanguageModel\n",
    "import torch\n",
    "\n",
    "model = LanguageModel('gpt2', device_map='cuda')\n",
    "\n",
    "with model.trace(\"Hello World\") as tracer:\n",
    "    hidden_states = model.transformer.h[-1].output[0].save()\n",
    "\n",
    "    hidden_states_grad_before = hidden_states.grad.clone().save()\n",
    "    hidden_states.grad[:] = 0\n",
    "    hidden_states_grad_after = hidden_states.grad.save()\n",
    "\n",
    "    logits = model.output.logits\n",
    "\n",
    "    logits.sum().backward()\n",
    "\n",
    "print(\"Before\", hidden_states_grad_before)\n",
    "print(\"After\", hidden_states_grad_after)\n",
    "\n",
    "with model.trace(\"Hello World\") as tracer:\n",
    "    hidden_states = model.transformer.h[-1].output[0].save()\n",
    "\n",
    "    hidden_states_grad_before = hidden_states.grad.clone().save()\n",
    "    hidden_states.grad = torch.zeros(hidden_states.grad.shape)\n",
    "    hidden_states_grad_after = hidden_states.grad.save()\n",
    "\n",
    "    logits = model.output.logits\n",
    "\n",
    "    logits.sum().backward()\n",
    "\n",
    "print(\"Before\", hidden_states_grad_before)\n",
    "print(\"After\", hidden_states_grad_after)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ndif",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
