{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with Hyrax Custom Dataset Classes\n",
    "\n",
    "In this notebook we are going to build up a custom dataset class for hyrax, and show how you can use the \n",
    "`prepare` verb in hyrax to test various aspects of your new dataclass.\n",
    "\n",
    "First we will create some synthetic data. We'll create 1000 10x10 numpy arrays with associated random file names.\n",
    "Nothing in this cell is specific to developing a Hyrax dataset, it's just setting things up in a semi-realistic way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "rng = np.random.default_rng()\n",
    "num_tensors = 1000\n",
    "\n",
    "# Generate filenames\n",
    "alphabet = list(\"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ\")\n",
    "filename_length = 15\n",
    "filenames = [\"\".join(list(rng.choice(alphabet, 15))) for _ in range(num_tensors)]\n",
    "\n",
    "# Generate numpy arrays\n",
    "shape = (3, 10, 10)\n",
    "random_data = {file: rng.random(size=shape, dtype=np.float32) for file in filenames}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a custom Dataset class\n",
    "\n",
    "We will treat these tensors as if they are on the filesystem, and write a dataclass that gives hyrax access to \n",
    "these \"files\" treating `_read_tensor` as a library function which returns a torch.Tensor from our \"files\", \n",
    "and `_list_filenames` as a library function which lists the filenames in a particular path.\n",
    "\n",
    "The first thing we need to do is make a new class derived from HyraxDataset and torch.Dataset as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset\n",
    "from hyrax.data_sets import HyraxDataset\n",
    "from pathlib import Path\n",
    "from typing import Union\n",
    "\n",
    "\n",
    "class MyDataset(HyraxDataset, Dataset):\n",
    "    def __init__(self, config: dict, data_location: Union[Path, str] = None):\n",
    "        self.filenames = MyDataset._list_filenames(data_location)\n",
    "\n",
    "        super().__init__(config)\n",
    "\n",
    "    def get_image(self, index):\n",
    "        \"\"\"Pretend to read specific data from the disk.\"\"\"\n",
    "        filename = self.filenames[index]\n",
    "        global random_data\n",
    "        return random_data[filename]\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return {\n",
    "            \"data\": {\"image\": MyDataset.get_image(self.filenames[idx])},\n",
    "        }\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.filenames)\n",
    "\n",
    "    @staticmethod\n",
    "    def _list_filenames(data_location):\n",
    "        \"\"\"This is a pretend implementation so we ignore data_location\"\"\"\n",
    "        global filenames\n",
    "        return filenames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Key aspects of this class that you will need to replicate are:\n",
    "\n",
    "* `__init__` must call `super().__init__(config)` This is important for hyrax to function appropriately, and \n",
    "gives you access to hyrax's config in other functions should you want it later. You will probably want to \n",
    "access `config[\"general\"][\"data_dir\"]` to figure out what directory to start in.\n",
    "\n",
    "* `__getitem__` You must implement this function, it takes an index and return the appropriate torch.Tensor\n",
    "for your data.\n",
    "\n",
    "* `__len__` must return the length of your tensorial data.\n",
    "\n",
    "Note that all of these are instance methods that use `self` as the first argument. This `self` is the current\n",
    "`MyDataset` object, and allows you to set and get values as is done with `self.filenames` in the code above.\n",
    "\n",
    "The functions `_list_filenames()` and `_read_tensor()` are both reading our fake data, and are there so we \n",
    "have an effective demonstration. The functional organization of your analogous file reading code is entirely \n",
    "up to you!\n",
    "\n",
    "\n",
    "We're now going to start up Hyrax and use the `prepare` verb to create an instance of this class and see\n",
    "that it works correctly. Note that we have set `config[\"general][\"data_dir\"]` to specify the location of our\n",
    "data for the `__init__`  function we wrote earlier, as well as the `config[\"data_set\"][\"name\"]` to the \n",
    "name of our class, so that Hyrax knows to use our dataset class rather than one of the built-in ones.\n",
    "\n",
    "Our `h.prepare()` line in the script will have the effect of calling our `__init__` function with the \n",
    "current hyrax config."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 14:25:05,001 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 14:25:06,348 hyrax.prepare:INFO] Finished Prepare\n"
     ]
    }
   ],
   "source": [
    "import hyrax\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "    }\n",
    "}\n",
    "\n",
    "dataset = h.prepare()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing\n",
    "\n",
    "The object we recieved from `h.prepare()` is an instance of our dataset, which we can test for functionality\n",
    "\n",
    "We're going to index into the dataset object with `[]` this has the effect of calling our `__getitem__` function\n",
    "and returning the result.\n",
    "\n",
    "We're also going to call `len()` on the dataset which will have the effect of calling our `__len__` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking __getitem__ ...\n",
      "\n",
      "Shape of our first element, should be \"torch.Size([3,10,10])\": \n",
      "(3, 10, 10)\n",
      "\n",
      "Type of our first element, should be \"<class 'numpy.ndarray'>\": \n",
      "<class 'numpy.ndarray'>\n",
      "\n",
      "Checking __len__, should print 1000: \n",
      "1000\n"
     ]
    }
   ],
   "source": [
    "print(\"Checking __getitem__ ...\", end=\"\\n\\n\")\n",
    "item = dataset[0]\n",
    "\n",
    "print('Shape of our first element, should be \"torch.Size([3,10,10])\": ')\n",
    "print(item[\"data\"][\"image\"].shape, end=\"\\n\\n\")\n",
    "\n",
    "print(\"Type of our first element, should be \\\"<class 'numpy.ndarray'>\\\": \")\n",
    "print(type(item[\"data\"][\"image\"]), end=\"\\n\\n\")\n",
    "\n",
    "print(\"Checking __len__, should print 1000: \")\n",
    "print(len(dataset))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset class is suitable for training or inference with Hyrax; however, you may want to read on to learn\n",
    "about more advanced features such as custom IDs for your data elements, metadata, and configuration access.\n",
    "\n",
    "Below is a short example that uses the HyraxAutoencoder built-in model, demonstrating that training is possible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 14:25:06,382 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 14:25:06,464 hyrax.models.hyrax_autoencoder:INFO] Found shape: (3, 10, 10) in data sample, using this to initialize model.\n",
      "[2025-09-17 14:25:06,467 hyrax.models.model_registry:INFO] Using criterion: torch.nn.CrossEntropyLoss with default arguments.\n",
      "2025-09-17 14:25:06,575 ignite.distributed.auto.auto_dataloader INFO: Use data loader kwargs for dataset 'Name: data\n",
      "  Dataset': \n",
      "\t{'sampler': <hyrax.pytorch_ignite.SubsetSequentialSampler object at 0x7c2da90a0ec0>, 'batch_size': 512, 'shuffle': False, 'pin_memory': True}\n",
      "2025-09-17 14:25:06,576 ignite.distributed.auto.auto_dataloader INFO: Use data loader kwargs for dataset 'Name: data\n",
      "  Dataset': \n",
      "\t{'sampler': <hyrax.pytorch_ignite.SubsetSequentialSampler object at 0x7c2da8728910>, 'batch_size': 512, 'shuffle': False, 'pin_memory': True}\n",
      "/home/drew/miniconda3/envs/hyrax/lib/python3.13/site-packages/ignite/handlers/tqdm_logger.py:127: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
      "  from tqdm.autonotebook import tqdm\n",
      "2025/09/17 14:25:06 INFO mlflow.system_metrics.system_metrics_monitor: Started monitoring system metrics.\n",
      "[2025-09-17 14:25:06,938 hyrax.pytorch_ignite:INFO] Training model on device: cuda\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5c2cc4d5bd944e52bf3f89350af8a2ee",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0dcd5b1a773a44e79b8a8fe1ee97f3c2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e276d9f03cc340f19e754731f6343d07",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d8aa39af04ed4e56bb780a7bce7c1167",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ff5134c3d50945459d45d8e7b5e9de1c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e82dd95163824b45aab236bd5ba78e13",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5ba2db9ce4f5443da9348bfc2415d8d1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "041d509e57a04b79bfc853864a4e61cd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d773bb6b39c94f2f9047cd51dfeb7a71",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9e9689df678847ccb1e701b8bb1a1428",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 14:25:07,884 hyrax.pytorch_ignite:INFO] Total training time: 0.95[s]\n",
      "[2025-09-17 14:25:07,885 hyrax.pytorch_ignite:INFO] Latest checkpoint saved as: /home/drew/code/hyrax/docs/pre_executed/results/20250917-142506-train-pSy4/checkpoint_epoch_10.pt\n",
      "[2025-09-17 14:25:07,885 hyrax.pytorch_ignite:INFO] Best metric checkpoint saved as: /home/drew/code/hyrax/docs/pre_executed/results/20250917-142506-train-pSy4/checkpoint_10_loss=-31.1603.pt\n",
      "2025/09/17 14:25:07 INFO mlflow.system_metrics.system_metrics_monitor: Stopping system metrics monitoring...\n",
      "2025/09/17 14:25:07 INFO mlflow.system_metrics.system_metrics_monitor: Successfully terminated system metrics monitoring!\n",
      "[2025-09-17 14:25:07,894 hyrax.verbs.train:INFO] Finished Training\n",
      "[2025-09-17 14:25:07,958 hyrax.model_exporters:INFO] Exported model to ONNX format: /home/drew/code/hyrax/docs/pre_executed/results/20250917-142506-train-pSy4/example_model_opset_20.onnx\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "HyraxAutoencoder(\n",
       "  (encoder): Sequential(\n",
       "    (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
       "    (1): GELU(approximate='none')\n",
       "    (2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (3): GELU(approximate='none')\n",
       "    (4): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
       "    (5): GELU(approximate='none')\n",
       "    (6): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (7): GELU(approximate='none')\n",
       "    (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))\n",
       "    (9): GELU(approximate='none')\n",
       "    (10): Flatten(start_dim=1, end_dim=-1)\n",
       "    (11): Linear(in_features=256, out_features=64, bias=True)\n",
       "  )\n",
       "  (dec_linear): Sequential(\n",
       "    (0): Linear(in_features=64, out_features=256, bias=True)\n",
       "    (1): GELU(approximate='none')\n",
       "  )\n",
       "  (decoder): Sequential(\n",
       "    (0): ConvTranspose2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))\n",
       "    (1): GELU(approximate='none')\n",
       "    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (3): GELU(approximate='none')\n",
       "    (4): ConvTranspose2d(64, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))\n",
       "    (5): GELU(approximate='none')\n",
       "    (6): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n",
       "    (7): GELU(approximate='none')\n",
       "    (8): ConvTranspose2d(32, 3, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), output_padding=(1, 1))\n",
       "    (9): Tanh()\n",
       "  )\n",
       "  (criterion): CrossEntropyLoss()\n",
       ")"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import hyrax\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model\"][\"name\"] = \"HyraxAutoencoder\"\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "    }\n",
    "}\n",
    "\n",
    "h.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extending to support visualization\n",
    "\n",
    "This section is primarily concerned with binding different sorts of metadata to your dataset. This metadata\n",
    "is used by the Hyrax visualization components to identify the source data of your latent space representation\n",
    "and link it back to a particular object/event in your astronomical dataset. \n",
    "\n",
    "When we built `MyDataclass` above, we invisibly picked up two major aspects from `HyraxDataset`:\n",
    "\n",
    "1. Unique IDs: Every tensor in our dataset got an ID of a sequential zero-based index, which was exactly the \n",
    "argument to `__getitem__`/`[]`. This list of ids is available as an iterator by calling `ids()` on the dataset\n",
    "object. These IDs are used in inference results and visualizations of the data, but they can be overriden.\n",
    "\n",
    "2. Metadata Interface: Every `HyraxDataset` can provide an astropy `Table` of values in the same order as \n",
    "their `__getitem__`/`[]` This allows each tensor in the dataset to have associated scalar data such as ra/dec, \n",
    "ephemeris parameters, redshift, magnitude, etc. For our class there currently is no metadata.\n",
    "\n",
    "Below is how we would access the metadata and IDs demonstrating the default behavior if your custom class\n",
    "does no overrides:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 14:25:07,987 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 14:25:08,001 hyrax.prepare:INFO] Finished Prepare\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "IDs:\n",
      "list(dataset.ids())[0:10] = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']\n",
      "\n",
      "Metadata field list:\n",
      "dataset.metadata_fields() = ['object_id'] (there is no metadata)\n"
     ]
    }
   ],
   "source": [
    "import hyrax\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "    }\n",
    "}\n",
    "\n",
    "dataset = h.prepare()\n",
    "\n",
    "print(\"\\nIDs:\")\n",
    "print(f\"list(dataset.ids())[0:10] = {list(dataset.ids())[0:10]}\")\n",
    "\n",
    "\n",
    "print(\"\\nMetadata field list:\")\n",
    "print(f\"dataset.metadata_fields() = {dataset.metadata_fields()} (there is no metadata)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding IDs\n",
    "\n",
    "We're going to use the filename in our fake data as IDs by adding a single `ids()` method to our `MyDataset` \n",
    "object. The most expedient way to do this will be to redefine the entire class below. Note that functions \n",
    "marked with a comment are just the same as earlier.\n",
    "\n",
    "Note that the `ids()` function is required to return a generator, so we will use a `for` loop and `yield`\n",
    "each sequential value. This interface allows Hyrax to partially enumerate the IDs in a dataset when that\n",
    "is desirable. It is easy enough to get all the ids in order with `list(dataset.ids())`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset\n",
    "from hyrax.data_sets import HyraxDataset\n",
    "\n",
    "\n",
    "class MyDataset(HyraxDataset, Dataset):\n",
    "    def ids(self):\n",
    "        for filename in self.filenames:\n",
    "            yield filename\n",
    "\n",
    "    # Unchanged from before below this comment ...\n",
    "    def __init__(self, config: dict, data_location: Union[Path, str] = None):\n",
    "        self.filenames = MyDataset._list_filenames(data_location)\n",
    "\n",
    "        super().__init__(config)\n",
    "\n",
    "    def get_image(self, index):\n",
    "        \"\"\"Pretend to read specific data from the disk.\"\"\"\n",
    "        filename = self.filenames[index]\n",
    "        global random_data\n",
    "        return random_data[filename]\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return {\n",
    "            \"data\": {\"image\": MyDataset.get_image(self.filenames[idx])},\n",
    "        }\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.filenames)\n",
    "\n",
    "    @staticmethod\n",
    "    def _list_filenames(data_location):\n",
    "        \"\"\"This is a pretend implementation so we ignore data_location\"\"\"\n",
    "        global filenames\n",
    "        return filenames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running `prepare` again on our newly defined dataset class, we can see that the ids are now the fake \n",
    "\"filenames\" we generated at the top of the notebook, rather than sequential integers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 14:25:08,051 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 14:25:08,063 hyrax.prepare:INFO] Finished Prepare\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "IDs:\n",
      "list(dataset.ids())[0:5] = ['UYvkmfDTyMHydyA', 'SOErTFtdwGspvJS', 'QVsvWTasGFiEtKc', 'QWqksbRihihOqde', 'BFXVfmWywdOSWmi']\n"
     ]
    }
   ],
   "source": [
    "import hyrax\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "    }\n",
    "}\n",
    "\n",
    "dataset = h.prepare()\n",
    "\n",
    "print(\"\\nIDs:\")\n",
    "print(f\"list(dataset.ids())[0:5] = {list(dataset.ids())[0:5]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding Metadata\n",
    "\n",
    "Now we are going to generate some fake metadata for our fake data. This will take the form of \n",
    "random ra/dec pairs for each fake object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import astropy.units as u\n",
    "\n",
    "ras = rng.uniform(low=0.0, high=360.0, size=num_tensors) * u.deg\n",
    "decs = rng.uniform(low=-90.0, high=90.0, size=num_tensors) * u.deg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to override metadata we will provide `HyraxDataset` with an astropy table containing all of the metadata in the constructor for our class as shown below. We do this in `__init__` by passing an astropy table of our metadata to `super().__init__` as a second, optional argument.\n",
    "\n",
    "Note the new function `_read_metadata()` which constructs this table. On a real dataset this function would\n",
    "most likely call astropy's `Table.read` [high level interface](https://docs.astropy.org/en/latest/io/unified.html) to construct a table directly from your catalog.\n",
    "\n",
    "As before we re-implement the entire class below with small modifications marked with comments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset\n",
    "from hyrax.data_sets import HyraxDataset\n",
    "\n",
    "\n",
    "class MyDataset(HyraxDataset, Dataset):\n",
    "    def __init__(self, config: dict, data_location: Union[Path, str] = None):\n",
    "        self.filenames = MyDataset._list_filenames(config[\"general\"][\"data_dir\"])\n",
    "        metadata_table = MyDataset._read_metadata(config[\"general\"][\"data_dir\"])\n",
    "        super().__init__(config, metadata_table=metadata_table)\n",
    "\n",
    "    def _read_metadata(path_to_data):\n",
    "        \"\"\"This is a pretend implementation so we don't use the path passed, which you might use\n",
    "        to find your .csv/.fits/.tsv catalog file and call astropy's Table.read().\n",
    "\n",
    "        We simply construct a table from our mock data\"\"\"\n",
    "        from astropy.table import Table\n",
    "\n",
    "        global ras, decs, filenames\n",
    "        return Table({\"object_id\": filenames, \"ra\": ras, \"dec\": decs})\n",
    "\n",
    "    # Unchanged from before below this comment ...\n",
    "    def ids(self):\n",
    "        for filename in self.filenames:\n",
    "            yield filename\n",
    "\n",
    "    def get_image(self, index):\n",
    "        \"\"\"Pretend to read specific data from the disk.\"\"\"\n",
    "        filename = self.filenames[index]\n",
    "        global random_data\n",
    "        return random_data[filename]\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        return {\n",
    "            \"data\": {\"image\": MyDataset.get_image(self.filenames[idx])},\n",
    "            \"object_id\": self.filenames[idx],\n",
    "        }\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.filenames)\n",
    "\n",
    "    @staticmethod\n",
    "    def _list_filenames(data_location):\n",
    "        \"\"\"This is a pretend implementation so we ignore data_location\"\"\"\n",
    "        global filenames\n",
    "        return filenames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that our dataset class supports metadata, we can access the metadata interface directly on the dataset object using the `metadata_fields` and `metadata` functions on the dataset object.\n",
    "\n",
    "- `metadata_fields` lists the available fields, in our case only \"ra\" and \"dec\" are available, but \n",
    "this is only because that is what was defined in the cell above\n",
    "- `metadata` takes a list (or array) of indexes, and a list (or array) of valid fields. It returns a numpy rec-array of the selected metadata fields for the selected data indexes. It is essentially\n",
    "equivalent to `metadata_table[indexes][fields].as_array()` where `metadata_table` is the original astropy table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:47:01,811 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 16:47:01,832 hyrax.prepare:INFO] Finished Prepare\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Metadata field list:\n",
      "dataset.metadata_fields() = ['object_id_data', 'ra_data', 'dec_data', 'object_id']\n",
      "Table(dataset.metadata([1, 3, 4], [\"ra_data\", \"dec_data\"])) =>\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><i>Table length=3</i>\n",
       "<table id=\"table136522054300464\" class=\"table-striped table-bordered table-condensed\">\n",
       "<thead><tr><th>ra_data</th><th>dec_data</th></tr></thead>\n",
       "<thead><tr><th>float64</th><th>float64</th></tr></thead>\n",
       "<tr><td>292.4007388647287</td><td>-87.13940122247789</td></tr>\n",
       "<tr><td>84.67360544447406</td><td>-59.027978794560354</td></tr>\n",
       "<tr><td>218.76113549876985</td><td>-1.0705419754372798</td></tr>\n",
       "</table></div>"
      ],
      "text/plain": [
       "<Table length=3>\n",
       "     ra_data             dec_data     \n",
       "     float64             float64      \n",
       "------------------ -------------------\n",
       " 292.4007388647287  -87.13940122247789\n",
       " 84.67360544447406 -59.027978794560354\n",
       "218.76113549876985 -1.0705419754372798"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import hyrax\n",
    "from astropy.table import Table\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "    }\n",
    "}\n",
    "\n",
    "dataset = h.prepare()\n",
    "\n",
    "print(\"\\nMetadata field list:\")\n",
    "print(f\"dataset.metadata_fields() = {dataset.metadata_fields()}\")\n",
    "print(f'Table(dataset.metadata([1, 3, 4], [\"ra_data\", \"dec_data\"])) =>')\n",
    "Table(dataset.metadata([1, 3, 4], [\"ra_data\", \"dec_data\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a Dataset capable of 'ra' and 'dec' metadata, we can do a full analysis with hyrax, `train`ing the model, `infer`ing the latent space, `umap`ping the latent space to a 2d representation, and `visualize`-ing the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:44:01,044 hyrax:INFO] Runtime Config read from: /home/drew/code/hyrax/src/hyrax/hyrax_default_config.toml\n",
      "[2025-09-17 16:44:01,062 hyrax.models.hyrax_autoencoder:INFO] Found shape: (3, 10, 10) in data sample, using this to initialize model.\n",
      "[2025-09-17 16:44:01,065 hyrax.models.model_registry:INFO] Using criterion: torch.nn.CrossEntropyLoss with default arguments.\n",
      "2025-09-17 16:44:01,066 ignite.distributed.auto.auto_dataloader INFO: Use data loader kwargs for dataset 'Name: data\n",
      "  Dataset': \n",
      "\t{'sampler': <hyrax.pytorch_ignite.SubsetSequentialSampler object at 0x7c2a6f2d5630>, 'batch_size': 512, 'shuffle': False, 'pin_memory': True}\n",
      "2025-09-17 16:44:01,066 ignite.distributed.auto.auto_dataloader INFO: Use data loader kwargs for dataset 'Name: data\n",
      "  Dataset': \n",
      "\t{'sampler': <hyrax.pytorch_ignite.SubsetSequentialSampler object at 0x7c2a6f4c7da0>, 'batch_size': 512, 'shuffle': False, 'pin_memory': True}\n",
      "2025/09/17 16:44:01 INFO mlflow.system_metrics.system_metrics_monitor: Started monitoring system metrics.\n",
      "[2025-09-17 16:44:01,085 hyrax.pytorch_ignite:INFO] Training model on device: cuda\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5bab3ad0f0644efd980ecaf0647061b0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1589ab71ec774a4aaea03d0c3a255ae1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0b3447bb9bc8491c91bba11f9ae7a850",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2f53d0f482ef4a35ae10435b92907a05",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9e5eb402958b42efba4f5c026c15bc87",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8126c61aef1849d9b51985bbb8d0214f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8c7d10c22bdd4e20be274c73ef673b35",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "016b40479268414ba0623f57feebf7bb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "68831b2270b34fbea4e2a7b0c626ade6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cc15fc70087b49b4ac6b4c301656a8ba",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:44:01,760 hyrax.pytorch_ignite:INFO] Total training time: 0.67[s]\n",
      "[2025-09-17 16:44:01,760 hyrax.pytorch_ignite:INFO] Latest checkpoint saved as: /home/drew/code/hyrax/docs/pre_executed/results/20250917-164401-train-tfhe/checkpoint_epoch_10.pt\n",
      "[2025-09-17 16:44:01,761 hyrax.pytorch_ignite:INFO] Best metric checkpoint saved as: /home/drew/code/hyrax/docs/pre_executed/results/20250917-164401-train-tfhe/checkpoint_10_loss=-28.0658.pt\n",
      "2025/09/17 16:44:01 INFO mlflow.system_metrics.system_metrics_monitor: Stopping system metrics monitoring...\n",
      "2025/09/17 16:44:01 INFO mlflow.system_metrics.system_metrics_monitor: Successfully terminated system metrics monitoring!\n",
      "[2025-09-17 16:44:01,769 hyrax.verbs.train:INFO] Finished Training\n",
      "[2025-09-17 16:44:01,817 hyrax.model_exporters:INFO] Exported model to ONNX format: /home/drew/code/hyrax/docs/pre_executed/results/20250917-164401-train-tfhe/example_model_opset_20.onnx\n",
      "[2025-09-17 16:44:01,832 hyrax.models.hyrax_autoencoder:INFO] Found shape: (3, 10, 10) in data sample, using this to initialize model.\n",
      "[2025-09-17 16:44:01,834 hyrax.models.model_registry:INFO] Using criterion: torch.nn.CrossEntropyLoss with default arguments.\n",
      "[2025-09-17 16:44:01,835 hyrax.verbs.infer:INFO] data set has length 1000\n",
      "2025-09-17 16:44:01,835 ignite.distributed.auto.auto_dataloader INFO: Use data loader kwargs for dataset 'Name: data\n",
      "  Dataset': \n",
      "\t{'sampler': None, 'batch_size': 512, 'shuffle': False, 'pin_memory': True}\n",
      "[2025-09-17 16:44:01,840 hyrax.verbs.infer:INFO] Saving inference results at: /home/drew/code/hyrax/docs/pre_executed/results/20250917-164401-infer-ZP0M\n",
      "[2025-09-17 16:44:02,237 hyrax.pytorch_ignite:INFO] Evaluating model on device: cuda\n",
      "[2025-09-17 16:44:02,239 hyrax.pytorch_ignite:INFO] Total epochs: 1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "07f5f9dd6dd74a9cb8f161f0024ff1b2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       " 50%|#####     | 1/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:44:02,293 hyrax.pytorch_ignite:INFO] Total evaluation time: 0.06[s]\n",
      "[2025-09-17 16:44:02,369 hyrax.verbs.infer:INFO] Inference Complete.\n",
      "[2025-09-17 16:44:02,406 hyrax.data_sets.inference_dataset:INFO] Using most recent results dir /home/drew/code/hyrax/docs/pre_executed/results/20250917-164401-infer-ZP0M for lookup. Use the [results] inference_dir config to set a directory or pass it to this verb.\n",
      "[2025-09-17 16:44:02,441 hyrax.verbs.umap:INFO] Saving UMAP results to /home/drew/code/hyrax/docs/pre_executed/results/20250917-164402-umap-Volp\n",
      "[2025-09-17 16:44:02,826 hyrax.verbs.umap:INFO] Fitting the UMAP\n",
      "[2025-09-17 16:44:03,793 hyrax.verbs.umap:INFO] Saving fitted UMAP Reducer\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c1ed3a3b547b43cba430768e50eac9ea",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Creating lower dimensional representation using UMAP::   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2025-09-17 16:44:05,269 hyrax.verbs.umap:INFO] Finished transforming all data through UMAP\n",
      "[2025-09-17 16:44:05,308 hyrax.verbs.visualize:INFO] UMAP directory not specified at runtime. Reading from config values.\n",
      "[2025-09-17 16:44:05,310 hyrax.data_sets.inference_dataset:INFO] Using most recent results dir /home/drew/code/hyrax/docs/pre_executed/results/20250917-164402-umap-Volp for lookup. Use the [results] inference_dir config to set a directory or pass it to this verb.\n",
      "[2025-09-17 16:44:05,350 hyrax.verbs.visualize:INFO] Rendering UMAP from the following directory: /home/drew/code/hyrax/docs/pre_executed/results/20250917-164402-umap-Volp\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<script type=\"esms-options\">{\"shimMode\": true}</script><style>*[data-root-id],\n",
       "*[data-root-id] > * {\n",
       "  box-sizing: border-box;\n",
       "  font-family: var(--jp-ui-font-family);\n",
       "  font-size: var(--jp-ui-font-size1);\n",
       "  color: var(--vscode-editor-foreground, var(--jp-ui-font-color1));\n",
       "}\n",
       "\n",
       "/* Override VSCode background color */\n",
       ".cell-output-ipywidget-background:has(\n",
       "  > .cell-output-ipywidget-background > .lm-Widget > *[data-root-id]\n",
       "),\n",
       ".cell-output-ipywidget-background:has(> .lm-Widget > *[data-root-id]) {\n",
       "  background-color: transparent !important;\n",
       "}\n",
       "</style>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": "(function(root) {\n  function now() {\n    return new Date();\n  }\n\n  const force = true;\n  const py_version = '3.7.3'.replace('rc', '-rc.').replace('.dev', '-dev.');\n  const reloading = false;\n  const Bokeh = root.Bokeh;\n\n  // Set a timeout for this load but only if we are not already initializing\n  if (typeof (root._bokeh_timeout) === \"undefined\" || (force || !root._bokeh_is_initializing)) {\n    root._bokeh_timeout = Date.now() + 5000;\n    root._bokeh_failed_load = false;\n  }\n\n  function run_callbacks() {\n    try {\n      root._bokeh_onload_callbacks.forEach(function(callback) {\n        if (callback != null)\n          callback();\n      });\n    } finally {\n      delete root._bokeh_onload_callbacks;\n    }\n    console.debug(\"Bokeh: all callbacks have finished\");\n  }\n\n  function load_libs(css_urls, js_urls, js_modules, js_exports, callback) {\n    if (css_urls == null) css_urls = [];\n    if (js_urls == null) js_urls = [];\n    if (js_modules == null) js_modules = [];\n    if (js_exports == null) js_exports = {};\n\n    root._bokeh_onload_callbacks.push(callback);\n\n    if (root._bokeh_is_loading > 0) {\n      // Don't load bokeh if it is still initializing\n      console.debug(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n      return null;\n    } else if (js_urls.length === 0 && js_modules.length === 0 && Object.keys(js_exports).length === 0) {\n      // There is nothing to load\n      run_callbacks();\n      return null;\n    }\n\n    function on_load() {\n      root._bokeh_is_loading--;\n      if (root._bokeh_is_loading === 0) {\n        console.debug(\"Bokeh: all BokehJS libraries/stylesheets loaded\");\n        run_callbacks()\n      }\n    }\n    window._bokeh_on_load = on_load\n\n    function on_error(e) {\n      const src_el = e.srcElement\n      console.error(\"failed to load \" + (src_el.href || src_el.src));\n    }\n\n    const skip = [];\n    if (window.requirejs) {\n      window.requirejs.config({'packages': {}, 'paths': {}, 'shim': {}});\n      root._bokeh_is_loading = css_urls.length + 0;\n    } else {\n      root._bokeh_is_loading = css_urls.length + js_urls.length + js_modules.length + Object.keys(js_exports).length;\n    }\n\n    const existing_stylesheets = []\n    const links = document.getElementsByTagName('link')\n    for (let i = 0; i < links.length; i++) {\n      const link = links[i]\n      if (link.href != null) {\n        existing_stylesheets.push(link.href)\n      }\n    }\n    for (let i = 0; i < css_urls.length; i++) {\n      const url = css_urls[i];\n      const escaped = encodeURI(url)\n      if (existing_stylesheets.indexOf(escaped) !== -1) {\n        on_load()\n        continue;\n      }\n      const element = document.createElement(\"link\");\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.rel = \"stylesheet\";\n      element.type = \"text/css\";\n      element.href = url;\n      console.debug(\"Bokeh: injecting link tag for BokehJS stylesheet: \", url);\n      document.body.appendChild(element);\n    }    var existing_scripts = []\n    const scripts = document.getElementsByTagName('script')\n    for (let i = 0; i < scripts.length; i++) {\n      var script = scripts[i]\n      if (script.src != null) {\n        existing_scripts.push(script.src)\n      }\n    }\n    for (let i = 0; i < js_urls.length; i++) {\n      const url = js_urls[i];\n      const escaped = encodeURI(url)\n      if (skip.indexOf(escaped) !== -1 || existing_scripts.indexOf(escaped) !== -1) {\n        if (!window.requirejs) {\n          on_load();\n        }\n        continue;\n      }\n      const element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n    for (let i = 0; i < js_modules.length; i++) {\n      const url = js_modules[i];\n      const escaped = encodeURI(url)\n      if (skip.indexOf(escaped) !== -1 || existing_scripts.indexOf(escaped) !== -1) {\n        if (!window.requirejs) {\n          on_load();\n        }\n        continue;\n      }\n      var element = document.createElement('script');\n      element.onload = on_load;\n      element.onerror = on_error;\n      element.async = false;\n      element.src = url;\n      element.type = \"module\";\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      document.head.appendChild(element);\n    }\n    for (const name in js_exports) {\n      const url = js_exports[name];\n      const escaped = encodeURI(url)\n      if (skip.indexOf(escaped) >= 0 || root[name] != null) {\n        if (!window.requirejs) {\n          on_load();\n        }\n        continue;\n      }\n      var element = document.createElement('script');\n      element.onerror = on_error;\n      element.async = false;\n      element.type = \"module\";\n      console.debug(\"Bokeh: injecting script tag for BokehJS library: \", url);\n      element.textContent = `\n      import ${name} from \"${url}\"\n      window.${name} = ${name}\n      window._bokeh_on_load()\n      `\n      document.head.appendChild(element);\n    }\n    if (!js_urls.length && !js_modules.length) {\n      on_load()\n    }\n  };\n\n  function inject_raw_css(css) {\n    const element = document.createElement(\"style\");\n    element.appendChild(document.createTextNode(css));\n    document.body.appendChild(element);\n  }\n\n  const js_urls = [\"https://cdn.holoviz.org/panel/1.7.5/dist/bundled/reactiveesm/es-module-shims@^1.10.0/dist/es-module-shims.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-3.7.3.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-gl-3.7.3.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.7.3.min.js\", \"https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.7.3.min.js\", \"https://cdn.holoviz.org/panel/1.7.5/dist/panel.min.js\"];\n  const js_modules = [];\n  const js_exports = {};\n  const css_urls = [];\n  const inline_js = [    function(Bokeh) {\n      Bokeh.set_log_level(\"info\");\n    },\nfunction(Bokeh) {} // ensure no trailing comma for IE\n  ];\n\n  function run_inline_js() {\n    if ((root.Bokeh !== undefined) || (force === true)) {\n      for (let i = 0; i < inline_js.length; i++) {\n        try {\n          inline_js[i].call(root, root.Bokeh);\n        } catch(e) {\n          if (!reloading) {\n            throw e;\n          }\n        }\n      }\n      // Cache old bokeh versions\n      if (Bokeh != undefined && !reloading) {\n        var NewBokeh = root.Bokeh;\n        if (Bokeh.versions === undefined) {\n          Bokeh.versions = new Map();\n        }\n        if (NewBokeh.version !== Bokeh.version) {\n          Bokeh.versions.set(NewBokeh.version, NewBokeh)\n        }\n        root.Bokeh = Bokeh;\n      }\n    } else if (Date.now() < root._bokeh_timeout) {\n      setTimeout(run_inline_js, 100);\n    } else if (!root._bokeh_failed_load) {\n      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n      root._bokeh_failed_load = true;\n    }\n    root._bokeh_is_initializing = false\n  }\n\n  function load_or_wait() {\n    // Implement a backoff loop that tries to ensure we do not load multiple\n    // versions of Bokeh and its dependencies at the same time.\n    // In recent versions we use the root._bokeh_is_initializing flag\n    // to determine whether there is an ongoing attempt to initialize\n    // bokeh, however for backward compatibility we also try to ensure\n    // that we do not start loading a newer (Panel>=1.0 and Bokeh>3) version\n    // before older versions are fully initialized.\n    if (root._bokeh_is_initializing && Date.now() > root._bokeh_timeout) {\n      // If the timeout and bokeh was not successfully loaded we reset\n      // everything and try loading again\n      root._bokeh_timeout = Date.now() + 5000;\n      root._bokeh_is_initializing = false;\n      root._bokeh_onload_callbacks = undefined;\n      root._bokeh_is_loading = 0\n      console.log(\"Bokeh: BokehJS was loaded multiple times but one version failed to initialize.\");\n      load_or_wait();\n    } else if (root._bokeh_is_initializing || (typeof root._bokeh_is_initializing === \"undefined\" && root._bokeh_onload_callbacks !== undefined)) {\n      setTimeout(load_or_wait, 100);\n    } else {\n      root._bokeh_is_initializing = true\n      root._bokeh_onload_callbacks = []\n      const bokeh_loaded = root.Bokeh != null && (root.Bokeh.version === py_version || (root.Bokeh.versions !== undefined && root.Bokeh.versions.has(py_version)));\n      if (!reloading && !bokeh_loaded) {\n        if (root.Bokeh) {\n          root.Bokeh = undefined;\n        }\n        console.debug(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n      }\n      load_libs(css_urls, js_urls, js_modules, js_exports, function() {\n        console.debug(\"Bokeh: BokehJS plotting callback run at\", now());\n        run_inline_js();\n      });\n    }\n  }\n  // Give older versions of the autoload script a head-start to ensure\n  // they initialize before we start loading newer version.\n  setTimeout(load_or_wait, 100)\n}(window));",
      "application/vnd.holoviews_load.v0+json": ""
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": "\nif ((window.PyViz === undefined) || (window.PyViz instanceof HTMLElement)) {\n  window.PyViz = {comms: {}, comm_status:{}, kernels:{}, receivers: {}, plot_index: []}\n}\n\n\n    function JupyterCommManager() {\n    }\n\n    JupyterCommManager.prototype.register_target = function(plot_id, comm_id, msg_handler) {\n      if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) {\n        var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager;\n        comm_manager.register_target(comm_id, function(comm) {\n          comm.on_msg(msg_handler);\n        });\n      } else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) {\n        window.PyViz.kernels[plot_id].registerCommTarget(comm_id, function(comm) {\n          comm.onMsg = msg_handler;\n        });\n      } else if (typeof google != 'undefined' && google.colab.kernel != null) {\n        google.colab.kernel.comms.registerTarget(comm_id, (comm) => {\n          var messages = comm.messages[Symbol.asyncIterator]();\n          function processIteratorResult(result) {\n            var message = result.value;\n            var content = {data: message.data, comm_id};\n            var buffers = []\n            for (var buffer of message.buffers || []) {\n              buffers.push(new DataView(buffer))\n            }\n            var metadata = message.metadata || {};\n            var msg = {content, buffers, metadata}\n            msg_handler(msg);\n            return messages.next().then(processIteratorResult);\n          }\n          return messages.next().then(processIteratorResult);\n        })\n      }\n    }\n\n    JupyterCommManager.prototype.get_client_comm = function(plot_id, comm_id, msg_handler) {\n      if (comm_id in window.PyViz.comms) {\n        return window.PyViz.comms[comm_id];\n      } else if (window.comm_manager || ((window.Jupyter !== undefined) && (Jupyter.notebook.kernel != null))) {\n        var comm_manager = window.comm_manager || Jupyter.notebook.kernel.comm_manager;\n        var comm = comm_manager.new_comm(comm_id, {}, {}, {}, comm_id);\n        if (msg_handler) {\n          comm.on_msg(msg_handler);\n        }\n      } else if ((plot_id in window.PyViz.kernels) && (window.PyViz.kernels[plot_id])) {\n        var comm = window.PyViz.kernels[plot_id].connectToComm(comm_id);\n        let retries = 0;\n        const open = () => {\n          if (comm.active) {\n            comm.open();\n          } else if (retries > 3) {\n            console.warn('Comm target never activated')\n          } else {\n            retries += 1\n            setTimeout(open, 500)\n          }\n        }\n        if (comm.active) {\n          comm.open();\n        } else {\n          setTimeout(open, 500)\n        }\n        if (msg_handler) {\n          comm.onMsg = msg_handler;\n        }\n      } else if (typeof google != 'undefined' && google.colab.kernel != null) {\n        var comm_promise = google.colab.kernel.comms.open(comm_id)\n        comm_promise.then((comm) => {\n          window.PyViz.comms[comm_id] = comm;\n          if (msg_handler) {\n            var messages = comm.messages[Symbol.asyncIterator]();\n            function processIteratorResult(result) {\n              var message = result.value;\n              var content = {data: message.data};\n              var metadata = message.metadata || {comm_id};\n              var msg = {content, metadata}\n              msg_handler(msg);\n              return messages.next().then(processIteratorResult);\n            }\n            return messages.next().then(processIteratorResult);\n          }\n        })\n        var sendClosure = (data, metadata, buffers, disposeOnDone) => {\n          return comm_promise.then((comm) => {\n            comm.send(data, metadata, buffers, disposeOnDone);\n          });\n        };\n        var comm = {\n          send: sendClosure\n        };\n      }\n      window.PyViz.comms[comm_id] = comm;\n      return comm;\n    }\n    window.PyViz.comm_manager = new JupyterCommManager();\n    \n\n\nvar JS_MIME_TYPE = 'application/javascript';\nvar HTML_MIME_TYPE = 'text/html';\nvar EXEC_MIME_TYPE = 'application/vnd.holoviews_exec.v0+json';\nvar CLASS_NAME = 'output';\n\n/**\n * Render data to the DOM node\n */\nfunction render(props, node) {\n  var div = document.createElement(\"div\");\n  var script = document.createElement(\"script\");\n  node.appendChild(div);\n  node.appendChild(script);\n}\n\n/**\n * Handle when a new output is added\n */\nfunction handle_add_output(event, handle) {\n  var output_area = handle.output_area;\n  var output = handle.output;\n  if ((output.data == undefined) || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n    return\n  }\n  var id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n  var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n  if (id !== undefined) {\n    var nchildren = toinsert.length;\n    var html_node = toinsert[nchildren-1].children[0];\n    html_node.innerHTML = output.data[HTML_MIME_TYPE];\n    var scripts = [];\n    var nodelist = html_node.querySelectorAll(\"script\");\n    for (var i in nodelist) {\n      if (nodelist.hasOwnProperty(i)) {\n        scripts.push(nodelist[i])\n      }\n    }\n\n    scripts.forEach( function (oldScript) {\n      var newScript = document.createElement(\"script\");\n      var attrs = [];\n      var nodemap = oldScript.attributes;\n      for (var j in nodemap) {\n        if (nodemap.hasOwnProperty(j)) {\n          attrs.push(nodemap[j])\n        }\n      }\n      attrs.forEach(function(attr) { newScript.setAttribute(attr.name, attr.value) });\n      newScript.appendChild(document.createTextNode(oldScript.innerHTML));\n      oldScript.parentNode.replaceChild(newScript, oldScript);\n    });\n    if (JS_MIME_TYPE in output.data) {\n      toinsert[nchildren-1].children[1].textContent = output.data[JS_MIME_TYPE];\n    }\n    output_area._hv_plot_id = id;\n    if ((window.Bokeh !== undefined) && (id in Bokeh.index)) {\n      window.PyViz.plot_index[id] = Bokeh.index[id];\n    } else {\n      window.PyViz.plot_index[id] = null;\n    }\n  } else if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n    var bk_div = document.createElement(\"div\");\n    bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n    var script_attrs = bk_div.children[0].attributes;\n    for (var i = 0; i < script_attrs.length; i++) {\n      toinsert[toinsert.length - 1].childNodes[1].setAttribute(script_attrs[i].name, script_attrs[i].value);\n    }\n    // store reference to server id on output_area\n    output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n  }\n}\n\n/**\n * Handle when an output is cleared or removed\n */\nfunction handle_clear_output(event, handle) {\n  var id = handle.cell.output_area._hv_plot_id;\n  var server_id = handle.cell.output_area._bokeh_server_id;\n  if (((id === undefined) || !(id in PyViz.plot_index)) && (server_id !== undefined)) { return; }\n  var comm = window.PyViz.comm_manager.get_client_comm(\"hv-extension-comm\", \"hv-extension-comm\", function () {});\n  if (server_id !== null) {\n    comm.send({event_type: 'server_delete', 'id': server_id});\n    return;\n  } else if (comm !== null) {\n    comm.send({event_type: 'delete', 'id': id});\n  }\n  delete PyViz.plot_index[id];\n  if ((window.Bokeh !== undefined) & (id in window.Bokeh.index)) {\n    var doc = window.Bokeh.index[id].model.document\n    doc.clear();\n    const i = window.Bokeh.documents.indexOf(doc);\n    if (i > -1) {\n      window.Bokeh.documents.splice(i, 1);\n    }\n  }\n}\n\n/**\n * Handle kernel restart event\n */\nfunction handle_kernel_cleanup(event, handle) {\n  delete PyViz.comms[\"hv-extension-comm\"];\n  window.PyViz.plot_index = {}\n}\n\n/**\n * Handle update_display_data messages\n */\nfunction handle_update_output(event, handle) {\n  handle_clear_output(event, {cell: {output_area: handle.output_area}})\n  handle_add_output(event, handle)\n}\n\nfunction register_renderer(events, OutputArea) {\n  function append_mime(data, metadata, element) {\n    // create a DOM node to render to\n    var toinsert = this.create_output_subarea(\n    metadata,\n    CLASS_NAME,\n    EXEC_MIME_TYPE\n    );\n    this.keyboard_manager.register_events(toinsert);\n    // Render to node\n    var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n    render(props, toinsert[0]);\n    element.append(toinsert);\n    return toinsert\n  }\n\n  events.on('output_added.OutputArea', handle_add_output);\n  events.on('output_updated.OutputArea', handle_update_output);\n  events.on('clear_output.CodeCell', handle_clear_output);\n  events.on('delete.Cell', handle_clear_output);\n  events.on('kernel_ready.Kernel', handle_kernel_cleanup);\n\n  OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n    safe: true,\n    index: 0\n  });\n}\n\nif (window.Jupyter !== undefined) {\n  try {\n    var events = require('base/js/events');\n    var OutputArea = require('notebook/js/outputarea').OutputArea;\n    if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n      register_renderer(events, OutputArea);\n    }\n  } catch(err) {\n  }\n}\n",
      "application/vnd.holoviews_load.v0+json": ""
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.holoviews_exec.v0+json": "",
      "text/html": [
       "<div id='46c122b0-13a3-4165-b348-41b97a76a09d'>\n",
       "  <div id=\"e959b8d6-9f53-4168-b226-bcce51b55fa9\" data-root-id=\"46c122b0-13a3-4165-b348-41b97a76a09d\" style=\"display: contents;\"></div>\n",
       "</div>\n",
       "<script type=\"application/javascript\">(function(root) {\n",
       "  var docs_json = {\"c2855661-d322-4f91-9b81-3c200a2ef30c\":{\"version\":\"3.7.3\",\"title\":\"Bokeh Application\",\"roots\":[{\"type\":\"object\",\"name\":\"panel.models.browser.BrowserInfo\",\"id\":\"46c122b0-13a3-4165-b348-41b97a76a09d\"},{\"type\":\"object\",\"name\":\"panel.models.comm_manager.CommManager\",\"id\":\"01f6329f-7694-49de-8d1b-da775226e3c9\",\"attributes\":{\"plot_id\":\"46c122b0-13a3-4165-b348-41b97a76a09d\",\"comm_id\":\"d0d56cc235a941edb126885cf39cfff1\",\"client_comm_id\":\"37c6f0bbda274cceb6d27efe9c27789f\"}}],\"defs\":[{\"type\":\"model\",\"name\":\"ReactiveHTML1\"},{\"type\":\"model\",\"name\":\"FlexBox1\",\"properties\":[{\"name\":\"align_content\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"align_items\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"flex_direction\",\"kind\":\"Any\",\"default\":\"row\"},{\"name\":\"flex_wrap\",\"kind\":\"Any\",\"default\":\"wrap\"},{\"name\":\"gap\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"justify_content\",\"kind\":\"Any\",\"default\":\"flex-start\"}]},{\"type\":\"model\",\"name\":\"FloatPanel1\",\"properties\":[{\"name\":\"config\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"contained\",\"kind\":\"Any\",\"default\":true},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"right-top\"},{\"name\":\"offsetx\",\"kind\":\"Any\",\"default\":null},{\"name\":\"offsety\",\"kind\":\"Any\",\"default\":null},{\"name\":\"theme\",\"kind\":\"Any\",\"default\":\"primary\"},{\"name\":\"status\",\"kind\":\"Any\",\"default\":\"normalized\"}]},{\"type\":\"model\",\"name\":\"GridStack1\",\"properties\":[{\"name\":\"ncols\",\"kind\":\"Any\",\"default\":null},{\"name\":\"nrows\",\"kind\":\"Any\",\"default\":null},{\"name\":\"allow_resize\",\"kind\":\"Any\",\"default\":true},{\"name\":\"allow_drag\",\"kind\":\"Any\",\"default\":true},{\"name\":\"state\",\"kind\":\"Any\",\"default\":[]}]},{\"type\":\"model\",\"name\":\"drag1\",\"properties\":[{\"name\":\"slider_width\",\"kind\":\"Any\",\"default\":5},{\"name\":\"slider_color\",\"kind\":\"Any\",\"default\":\"black\"},{\"name\":\"value\",\"kind\":\"Any\",\"default\":50}]},{\"type\":\"model\",\"name\":\"click1\",\"properties\":[{\"name\":\"terminal_output\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"debug_name\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"clears\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"ReactiveESM1\",\"properties\":[{\"name\":\"esm_constants\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}}]},{\"type\":\"model\",\"name\":\"JSComponent1\",\"properties\":[{\"name\":\"esm_constants\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}}]},{\"type\":\"model\",\"name\":\"ReactComponent1\",\"properties\":[{\"name\":\"use_shadow_dom\",\"kind\":\"Any\",\"default\":true},{\"name\":\"esm_constants\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}}]},{\"type\":\"model\",\"name\":\"AnyWidgetComponent1\",\"properties\":[{\"name\":\"use_shadow_dom\",\"kind\":\"Any\",\"default\":true},{\"name\":\"esm_constants\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}}]},{\"type\":\"model\",\"name\":\"FastWrapper1\",\"properties\":[{\"name\":\"object\",\"kind\":\"Any\",\"default\":null},{\"name\":\"style\",\"kind\":\"Any\",\"default\":null}]},{\"type\":\"model\",\"name\":\"NotificationArea1\",\"properties\":[{\"name\":\"js_events\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"max_notifications\",\"kind\":\"Any\",\"default\":5},{\"name\":\"notifications\",\"kind\":\"Any\",\"default\":[]},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0},{\"name\":\"types\",\"kind\":\"Any\",\"default\":[{\"type\":\"map\",\"entries\":[[\"type\",\"warning\"],[\"background\",\"#ffc107\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-exclamation-triangle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]},{\"type\":\"map\",\"entries\":[[\"type\",\"info\"],[\"background\",\"#007bff\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-info-circle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]}]}]},{\"type\":\"model\",\"name\":\"Notification\",\"properties\":[{\"name\":\"background\",\"kind\":\"Any\",\"default\":null},{\"name\":\"duration\",\"kind\":\"Any\",\"default\":3000},{\"name\":\"icon\",\"kind\":\"Any\",\"default\":null},{\"name\":\"message\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"notification_type\",\"kind\":\"Any\",\"default\":null},{\"name\":\"_rendered\",\"kind\":\"Any\",\"default\":false},{\"name\":\"_destroyed\",\"kind\":\"Any\",\"default\":false}]},{\"type\":\"model\",\"name\":\"TemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"BootstrapTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"TemplateEditor1\",\"properties\":[{\"name\":\"layout\",\"kind\":\"Any\",\"default\":[]}]},{\"type\":\"model\",\"name\":\"MaterialTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"request_value1\",\"properties\":[{\"name\":\"fill\",\"kind\":\"Any\",\"default\":\"none\"},{\"name\":\"_synced\",\"kind\":\"Any\",\"default\":null},{\"name\":\"_request_sync\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"holoviews.plotting.bokeh.raster.HoverModel\",\"properties\":[{\"name\":\"xy\",\"kind\":\"Any\",\"default\":null},{\"name\":\"data\",\"kind\":\"Any\",\"default\":null}]}]}};\n",
       "  var render_items = [{\"docid\":\"c2855661-d322-4f91-9b81-3c200a2ef30c\",\"roots\":{\"46c122b0-13a3-4165-b348-41b97a76a09d\":\"e959b8d6-9f53-4168-b226-bcce51b55fa9\"},\"root_ids\":[\"46c122b0-13a3-4165-b348-41b97a76a09d\"]}];\n",
       "  var docs = Object.values(docs_json)\n",
       "  if (!docs) {\n",
       "    return\n",
       "  }\n",
       "  const py_version = docs[0].version.replace('rc', '-rc.').replace('.dev', '-dev.')\n",
       "  async function embed_document(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    await Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
       "    for (const render_item of render_items) {\n",
       "      for (const root_id of render_item.root_ids) {\n",
       "\tconst id_el = document.getElementById(root_id)\n",
       "\tif (id_el.children.length && id_el.children[0].hasAttribute('data-root-id')) {\n",
       "\t  const root_el = id_el.children[0]\n",
       "\t  root_el.id = root_el.id + '-rendered'\n",
       "\t  for (const child of root_el.children) {\n",
       "            // Ensure JupyterLab does not capture keyboard shortcuts\n",
       "            // see: https://jupyterlab.readthedocs.io/en/4.1.x/extension/notebook.html#keyboard-interaction-model\n",
       "\t    child.setAttribute('data-lm-suppress-shortcuts', 'true')\n",
       "\t  }\n",
       "\t}\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  function get_bokeh(root) {\n",
       "    if (root.Bokeh === undefined) {\n",
       "      return null\n",
       "    } else if (root.Bokeh.version !== py_version) {\n",
       "      if (root.Bokeh.versions === undefined || !root.Bokeh.versions.has(py_version)) {\n",
       "\treturn null\n",
       "      }\n",
       "      return root.Bokeh.versions.get(py_version);\n",
       "    } else if (root.Bokeh.version === py_version) {\n",
       "      return root.Bokeh\n",
       "    }\n",
       "    return null\n",
       "  }\n",
       "  function is_loaded(root) {\n",
       "    var Bokeh = get_bokeh(root)\n",
       "    return (Bokeh != null && Bokeh.Panel !== undefined)\n",
       "  }\n",
       "  if (is_loaded(root)) {\n",
       "    embed_document(root);\n",
       "  } else {\n",
       "    var attempts = 0;\n",
       "    var timer = setInterval(function(root) {\n",
       "      if (is_loaded(root)) {\n",
       "        clearInterval(timer);\n",
       "        embed_document(root);\n",
       "      } else if (document.readyState == \"complete\") {\n",
       "        attempts++;\n",
       "        if (attempts > 200) {\n",
       "          clearInterval(timer);\n",
       "\t  var Bokeh = get_bokeh(root)\n",
       "\t  if (Bokeh == null || Bokeh.Panel == null) {\n",
       "            console.warn(\"Panel: ERROR: Unable to run Panel code because Bokeh or Panel library is missing\");\n",
       "\t  } else {\n",
       "\t    console.warn(\"Panel: WARNING: Attempting to render but not all required libraries could be resolved.\")\n",
       "\t    embed_document(root)\n",
       "\t  }\n",
       "        }\n",
       "      }\n",
       "    }, 25, root)\n",
       "  }\n",
       "})(window);</script>"
      ]
     },
     "metadata": {
      "application/vnd.holoviews_exec.v0+json": {
       "id": "46c122b0-13a3-4165-b348-41b97a76a09d"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "\n",
       "\n",
       "<div class=\"logo-block\">\n",
       "<a href=\"https://holoviews.org\" target=\"_blank\" title=\"HoloViews 1.21.0\">\n",
       "  <img src='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAAB+wAAAfsBxc2miwAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAA6zSURB\n",
       "VHic7ZtpeFRVmsf/5966taWqUlUJ2UioBBJiIBAwCZtog9IOgjqACsogKtqirT2ttt069nQ/zDzt\n",
       "tI4+CrJIREFaFgWhBXpUNhHZQoKBkIUASchWla1S+3ar7r1nPkDaCAnZKoQP/D7mnPOe9/xy76n3\n",
       "nFSAW9ziFoPFNED2LLK5wcyBDObkb8ZkxuaoSYlI6ZcOKq1eWFdedqNzGHQBk9RMEwFAASkk0Xw3\n",
       "ETacDNi2vtvc7L0ROdw0AjoSotQVkKSvHQz/wRO1lScGModBFbDMaNRN1A4tUBCS3lk7BWhQkgpD\n",
       "lG4852/+7DWr1R3uHAZVQDsbh6ZPN7CyxUrCzJMRouusj0ipRwD2uKm0Zn5d2dFwzX1TCGhnmdGo\n",
       "G62Nna+isiUqhkzuKrkQaJlPEv5mFl2fvGg2t/VnzkEV8F5ioioOEWkLG86fvbpthynjdhXYZziQ\n",
       "x1hC9J2NFyi8vCTt91Fh04KGip0AaG9zuCk2wQCVyoNU3Hjezee9bq92duzzTmxsRJoy+jEZZZYo\n",
       "GTKJ6SJngdJqAfRzpze0+jHreUtPc7gpBLQnIYK6BYp/uGhw9YK688eu7v95ysgshcg9qSLMo3JC\n",
       "4jqLKQFBgdKDPoQ+Pltb8dUyQLpeDjeVgI6EgLIQFT5tEl3rn2losHVsexbZ3EyT9wE1uGdkIPcy\n",
       "BGxn8QUq1QrA5nqW5i2tLqvrrM9NK6AdkVIvL9E9bZL/oyfMVd/jqvc8LylzRBKDJSzIExwhQzuL\n",
       "QYGQj4rHfFTc8mUdu3E7yoLtbTe9gI4EqVgVkug2i5+uXGo919ixbRog+3fTbQ8qJe4ZOYNfMoTI\n",
       "OoshUNosgO60AisX15aeI2PSIp5KiFLI9ubb1vV3Qb2ltwLakUCDAkWX7/nHKRmmGIl9VgYsUhJm\n",
       "2NXjKYADtM1ygne9QQDIXlk49FBstMKx66D1v4+XuQr7vqTe0VcBHQlRWiOCbmmSYe2SqtL6q5rJ\n",
       "zsTb7lKx3FKOYC4DoqyS/B5bvLPxvD9Qtf6saxYLQGJErmDOdOMr/zo96km1nElr8bmPOBwI9COv\n",
       "HnFPRIwmkSOv9kcAS4heRsidOkpeWBgZM+UBrTFAXNYL5Vf2ii9c1trNzpYdaoVil3WIc+wdk+gQ\n",
       "noie3ecCcxt9ITcLAPWt/laGEO/9U6PmzZkenTtsSMQ8uYywJVW+grCstAvCIaAdArAsIWkRDDs/\n",
       "KzLm2YcjY1Lv0UdW73HabE9n6V66cxSzfEmuJssTpKGVp+0vHq73FwL46eOjpMpbRAnNmJFrGJNu\n",
       "Ukf9Yrz+3rghiumCKNXXWPhLYcjxGsIpoCMsIRoFITkW8AuyM8jC1+/QLx4bozCEJIq38+1rtpR6\n",
       "V/yzb8eBlRb3fo5l783N0CWolAzJHaVNzkrTzlEp2bQ2q3TC5gn6wpnoQAmwSiGh2GitnTmVMc5O\n",
       "UyfKWUKCIsU7+fZDKwqdT6DDpvkzAX4/+AMFjk0tDp5GRXLpQ2MUmhgDp5gxQT8+Y7hyPsMi8uxF\n",
       "71H0oebujHALECjFKaW9Lm68n18wXp2kVzIcABytD5iXFzg+WVXkegpAsOOYziqo0OkK76GyquC3\n",
       "ltZAzMhhqlSNmmWTE5T6e3IN05ITFLM4GdN0vtZ3ob8Jh1NAKXFbm5PtLU/eqTSlGjkNAJjdgn/N\n",
       "aedXa0tdi7+t9G0FIF49rtMSEgAs1kDLkTPO7ebm4IUWeyh1bKomXqlgMG6kJmHcSM0clYLJ8XtR\n",
       "1GTnbV3F6I5wCGikAb402npp1h1s7LQUZZSMIfALFOuL3UUrfnS8+rez7v9qcold5tilgHbO1fjK\n",
       "9ubb17u9oshxzMiUBKXWqJNxd+fqb0tLVs4lILFnK71H0Ind7uiPgACVcFJlrb0tV6DzxqqTIhUM\n",
       "CwDf1/rrVhTa33/3pGPxJYdQ2l2cbgVcQSosdx8uqnDtbGjh9SlDVSMNWhlnilfqZk42Th2ZpLpf\n",
       "xrHec5e815zrr0dfBZSwzkZfqsv+1FS1KUknUwPARVvItfKUY+cn57yP7qv07UE3p8B2uhUwLk09\n",
       "e0SCOrK+hbdYHYLjRIl71wWzv9jpEoeOHhGRrJAzyEyNiJuUqX0g2sBN5kGK6y2Blp5M3lsB9Qh4\n",
       "y2Ja6x6+i0ucmKgwMATwhSjdUu49tKrQ/pvN5d53ml2CGwCmJipmKjgmyuaXzNeL2a0AkQ01Th5j\n",
       "2DktO3Jyk8f9vcOBQHV94OK+fPumJmvQHxJoWkaKWq9Vs+yUsbq0zGT1I4RgeH2b5wef7+c7bl8F\n",
       "eKgoHVVZa8ZPEORzR6sT1BzDUAD/d9F78e2Tzv99v8D+fLVTqAKAsbGamKey1Mt9Ann4eH3gTXTz\n",
       "idWtAJ8PQWOk7NzSeQn/OTHDuEikVF1R4z8BQCy+6D1aWRfY0tTGG2OM8rRoPaeIj5ZHzJxszElN\n",
       "VM8K8JS5WOfv8mzRnQAKoEhmt8gyPM4lU9SmBK1MCQBnW4KONT86v1hZ1PbwSXPw4JWussVjtH9Y\n",
       "NCoiL9UoH/6PSu8jFrfY2t36erQHXLIEakMi1SydmzB31h3GGXFDFNPaK8Rme9B79Ixrd0WN+1ij\n",
       "NRQ/doRmuFLBkHSTOm5GruG+pFjFdAmorG4IXH1Qua6ASniclfFtDYt+oUjKipPrCQB7QBQ2lrgP\n",
       "fFzm+9XWUtcqJ3/5vDLDpJ79XHZk3u8nGZ42qlj1+ydtbxysCezrydp6ugmipNJ7WBPB5tydY0jP\n",
       "HaVNzs3QzeE4ZpTbI+ZbnSFPbVOw9vsfnVvqWnirPyCNGD08IlqtYkh2hjZ5dErEQzoNm+6ykyOt\n",
       "Lt5/PQEuSRRKo22VkydK+vvS1XEKlhCJAnsqvcVvH7f/ZU2R67eXbMEGAMiIV5oWZWiWvz5Fv2xG\n",
       "sjqNJQRvn3Rs2lji/lNP19VjAQDgD7FHhujZB9OGqYxRkZxixgRDVlqS6uEOFaJUVu0rPFzctrnF\n",
       "JqijImVp8dEKVWyUXDk92zAuMZ6bFwpBU1HrOw6AdhQgUooChb0+ItMbWJitSo5Ws3IAOGEOtL53\n",
       "0vHZih9sC4vtofZ7Qu6523V/fmGcds1TY3V36pUsBwAbSlxnVh2xLfAD/IAIMDf7XYIkNmXfpp2l\n",
       "18rkAJAy9HKFaIr/qULkeQQKy9zf1JgDB2uaeFNGijo5QsUyacNUUTOnGO42xSnv4oOwpDi1zYkc\n",
       "efUc3I5Gk6PhyTuVKaOGyLUAYPGIoY9Pu/atL/L92+4q9wbflRJ2Trpm/jPjdBtfnqB/dIThcl8A\n",
       "KG7hbRuKnb8qsQsVvVlTrwQAQMUlf3kwJI24Z4JhPMtcfng5GcH49GsrxJpGvvHIaeem2ma+KSjQ\n",
       "lIwUdYyCY8j4dE1KzijNnIP2llF2wcXNnsoapw9XxsgYAl6k+KzUXbi2yP3KR2ecf6z3BFsBICdW\n",
       "nvnIaG3eHybqX7vbpEqUMT+9OL4Qpe8VON7dXuFd39v19FoAABRVePbGGuXTszO0P7tu6lghUonE\n",
       "llRdrhArLvmKdh9u29jcFiRRkfLUxBiFNiqSU9icoZQHo5mYBI1MBgBH6wMNb+U7Pnw337H4gi1Y\n",
       "ciWs+uks3Z9fztUvfzxTm9Ne8XXkvQLHNytOOZeiD4e0PgkAIAYCYknKUNUDSXEKzdWNpnil7r4p\n",
       "xqkjTarZMtk/K8TQ6Qve78qqvXurGwIJqcOUKfUWHsm8KGvxSP68YudXq4pcj39X49uOK2X142O0\n",
       "Tz5/u/7TVybqH0rSya6ZBwD21/gubbrgWdDgEOx9WUhfBaC2ibcEBYm7a7x+ukrBMNcEZggyR0TE\n",
       "T8zUPjikQ4VosQZbTpS4vqizBKvqmvjsqnpfzaZyx9JPiz1/bfGKdgD45XB1zoIMzYbfTdS/NClB\n",
       "Gct0USiY3YL/g0LHy/uq/Ef6uo5+n0R/vyhp17Klpge763f8rMu6YU/zrn2nml+2WtH+Z+5IAAFc\n",
       "2bUTdTDOSNa9+cQY7YLsOIXhevEkCvzph7a8laecz/Un/z4/Ae04XeL3UQb57IwU9ZDr9UuKVajv\n",
       "nxp1+1UVIo/LjztZkKH59fO3G/JemqCfmaCRqbqbd90ZZ8FfjtkfAyD0J/9+C2h1hDwsSxvGjNDc\n",
       "b4zk5NfrSwiQblLHzZhg+Jf4aPlUwpDqkQqa9nimbt1/TDH8OitGMaQnj+RJS6B1fbF7SY1TqO5v\n",
       "/v0WAADl1f7zokgS7s7VT2DZ7pegUjBM7mjtiDZbcN4j0YrHH0rXpCtY0qPX0cVL0rv5jv/ZXend\n",
       "0u/EESYBAFBU4T4Qa5TflZOhTe7pmKpaP8kCVUVw1+yhXfJWvn1P3hnXi33JsTN6PnP3hHZ8Z3/h\n",
       "aLHzmkNPuPj7Bc/F/Q38CwjTpSwQXgE4Vmwry9tpfq/ZFgqFMy4AVDtCvi8rvMvOmv0N4YwbVgEA\n",
       "sPM72/KVnzfspmH7HQGCRLG2yL1+z8XwvPcdCbsAANh+xPzstgMtxeGKt+6MK3/tacfvwhWvIwMi\n",
       "oKEBtm0H7W+UVfkc/Y1V0BhoPlDr/w1w/eu1vjIgAgDg22OtX6/eYfnEz/focrZTHAFR+PSs56/7\n",
       "q32nwpjazxgwAQCwcU/T62t3WL7r6/jVRa6/byp1rei+Z98ZUAEAhEPHPc8fKnTU9nbgtnOe8h0l\n",
       "9hcGIqmODLQAHCy2Xti6v/XNRivf43f4fFvIteu854+VHnR7q9tfBlwAAGz+pnndB9vM26UebAe8\n",
       "SLHujPOTPVW+rwY+sxskAAC2HrA8t2Vvc7ffP1r9o+vwR2dcr92InIAbKKC1FZ5tB1tf+/G8p8sv\n",
       "N/9Q5zd/XR34LYCwV5JdccMEAMDBk45DH243r/X4xGvqxFa/GNpS7n6rwOwNWwHVE26oAADYurf1\n",
       "zx/utOzt+DMKYM0p17YtZZ5VNzqfsB2HewG1WXE8PoZ7gOclbTIvynZf9JV+fqZtfgs/8F/Nu5rB\n",
       "EIBmJ+8QRMmpU7EzGRsf2FzuePqYRbzh/zE26EwdrT10f6r6o8HOYzCJB9Dpff8tbnGLG8L/A/WE\n",
       "roTBs2RqAAAAAElFTkSuQmCC' style='height:25px; border-radius:12px; display: inline-block; float: left; vertical-align: middle'></img>\n",
       "</a>\n",
       "\n",
       "\n",
       "<a href=\"https://bokeh.org\" target=\"_blank\" title=\"Bokeh 3.7.3\">\n",
       "  <img src='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACMAAAAjCAYAAAAe2bNZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAK6wAACusBgosNWgAAABx0RVh0U29mdHdhcmUAQWRvYmUgRmlyZXdvcmtzIENTNui8sowAAAf9SURBVFiFvZh7cFTVHcc/59y7793sJiFAwkvAYDRqFWwdraLVlj61diRYsDjqCFbFKrYo0CltlSq1tLaC2GprGIriGwqjFu10OlrGv8RiK/IICYECSWBDkt3s695zTv9IAtlHeOn0O7Mzu797z+/3Ob/z+p0VfBq9doNFljuABwAXw2PcvGHt6bgwxhz7Ls4YZNVXxxANLENwE2D1W9PAGmAhszZ0/X9gll5yCbHoOirLzmaQs0F6F8QMZq1v/8xgNm7DYwwjgXJLYL4witQ16+sv/U9HdDmV4WrKw6B06cZC/RMrM4MZ7xz61DAbtzEXmAvUAX4pMOVecg9/MFFu3j3Gz7gQBLygS2RGumBkL0cubiFRsR3LzVBV1UMk3IrW73PT9C2lYOwhQB4ClhX1AuKpjLcV27oEjyUpNUJCg1CvcejykWTCXyQgzic2HIIBjg3pS6+uRLKAhumZvD4U+tq0jTrgkVKQQtLekfTtxIPAkhTNF6G7kZm7aPp6M9myKVQEoaYaIhEQYvD781DML/RfBGNZXAl4irJiwBa07e/y7cQnBaJghIX6ENl2GR/fGCBoz6cm5qeyEqQA5ZYA5x5eeiV0Qph4gjFAUSwAr6QllQgcxS/Jm25Cr2Tmpsk03XI9NfI31FTZBEOgVOk51adqDBNPCNPSRlkiDXbBEwOU2WxH+I7itQZ62g56OjM33suq1YsZHVtGZSUI2QdyYgkgOthQNIF7BIGDnRAJgJSgj69cUx1gB8PkOGwL4E1gPrM27gIg7NlGKLQApc7BmEnAxP5g/rw4YqBrCDB5xHkw5rdR/1qTrN/hKNo6YUwVDNpFsnjYS8RbidBPcPXFP6R6yfExuOXmN4A3jv1+8ZUwgY9D2OWjUZE6lO88jDwHI8ZixGiMKSeYTBamCoDk6kDAb6y1OcH1a6KpD/fZesoFw5FlIXAVCIiH4PxrV+p2npVDToTBmtjY8t1swh2V61E9KqWiyuPEjM8dbfxuvfa49Zayf9R136Wr8mBSf/T7bNteA8zwaGEUbFpckWwq95n59dUIywKl2fbOIS5e8bWSu0tJ1a5redAYfqkdjesodFajcgaVNWhXo1C9SrkN3Usmv3UMJrc6/DDwkwEntkEJLe67tSLhvyzK8rHDQWleve5CGk4VZEB1r+5bg2E2si+Y0QatDK6jUVkX5eg2YYlp++ZM+rfMNYamAj8Y7MAVWFqaR1f/t2xzU4IHjybBtthzuiAASqv7jTF7jOqDMAakFHgDNsFyP+FhwZHBmH9F7cutIYkQCylYYv1AZSqsn1/+bX51OMMjPSl2nAnM7hnjOx2v53YgNWAzHM9Q/9l0lQWPSCBSyokAtOBC1Rj+w/1Xs+STDp4/E5g7Rs2zm2+oeVd7PUuHKDf6A4r5EsPT5K3gfCnBXNUYnvGzb+KcCczYYWOnLpy4eOXuG2oec0PBN8XQQAnpvS35AvAykr56rWhPBiV4MvtceGLxk5Mr6A1O8IfK7rl7xJ0r9kyumuP4fa0lMqTBLJIAJqEf1J3qE92lMBndlyfRD2YBghHC4hlny7ASqCeWo5zaoDdIWfnIefNGTb9fC73QDfhyBUCNOxrGPSUBfPem9us253YTV+3mcBbdkUYfzmHiLqZbYdIGHHON2ZlemXouaJUOO6TqtdHEQuXYY8Yt+EbDgmlS6RdzkaDTv2P9A3gICiq93sWhb5mc5wVhuU3Y7m5hOc3So7qFT3SLgOXHb/cyOfMn7xROegoC/PTcn3v8gbKPgDopJFk3R/uBPWQiwQ+2/GJevRMObLUzqe/saJjQUQTTftEVMW9tWxPgAocwcj9abNcZe7s+6t2R2xXZG7zyYLp8Q1PiRBBHym5bYuXi8Qt+/LvGu9f/5YDAxABsaRNPH6Xr4D4Sk87a897SOy9v/fKwjoF2eQel95yDESGEF6gEMwKhLwKus3wOVjTtes7qzgLdXTMnNCNoEpbcrtNuq6N7Xh/+eqcbj94xQkp7mdKpW5XbtbR8Z26kgMCAf2UU5YEovRUVRHbu2b3vK1UdDFkDCyMRQxbpdv8nhKAGIa7QaQedzT07fFPny53R738JoVYBdVrnsNx9XZ9v33UeGO+AA2MMUkgqQ5UcdDLZSFeVgONnXeHqSAC5Ew1BXwko0D1Zct3dT1duOjS3MzZnEUJtBuoQAq3SGOLR4ekjn9NC5nVOaYXf9lETrUkmOJy3pOz8OKIb2A1cWhJCCEzOxU2mUPror+2/L3yyM3pkM7jTjr1nBOgkGeyQ7erxpdJsMAS9wb2F9rzMxNY1K2PMU0WtZV82VU8Wp6vbKJVo9Lx/+4cydORdxCCQ/kDGTZCWsRpLu7VD7bfKqL8V2orKTp/PtzaXy42jr6TwAuisi+7JolUG4wY+8vyrISCMtRrLKWpvjAOqx/QGhp0rjRo5xD3x98CWQuOQN8qumRMmI7jKZPUEpzNVZsj4Zbaq1to5tZZsKIydLWojhIXrJnES79EaOzv3du2NytKuxzJKAA6wF8xqEE8s2jo/1wd/khslQGxd81Zg62Bbp31XBH+iETt7Y3ELA0iU6iGDlQ5mexe0VEx4a3x8V1AaYwFJgTiwaOsDmeK2J8nMUOqsnB1A+dcA04ucCYt0urkjmflk9iT2v30q/gZn5rQPvor4n9Ou634PeBzoznes/iot/7WnClKoM/+zCIjH5kwT8ChQjTHPIPTjFV3PpU/Hx+DM/A9U3IXI4SPCYAAAAABJRU5ErkJggg=='\n",
       "       style='height:15px; border-radius:12px; display: inline-block; float: left'>\n",
       "  </img>\n",
       "</a>\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "\n",
       "<span style=\"float: left; margin-left: 5px; line-height: 15px; cursor: pointer; opacity: 0.7;\"\n",
       "      onmouseover=\"this.style.opacity='1'\"\n",
       "      onmouseout=\"this.style.opacity='0.7'\"\n",
       "      title=\"Extension loaded. This cell output contains code that enables plot interactivity, it should not be removed.\">ⓘ</span>\n",
       "</div>\n",
       "\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bc7fbc72147445bcb6d3213b3718d6c5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "BokehModel(combine_events=True, render_bundle={'docs_json': {'e14a3181-ca72-4205-9c5a-19c38a5948db': {'version…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import hyrax\n",
    "\n",
    "h = hyrax.Hyrax()\n",
    "h.config[\"model_inputs\"] = {\n",
    "    \"data\": {\n",
    "        \"dataset_class\": \"MyDataset\",\n",
    "        \"data_location\": \"/fake/path/to/some/data\",\n",
    "        \"primary_id_field\": \"object_id\",\n",
    "    }\n",
    "}\n",
    "h.config[\"model\"][\"name\"] = \"HyraxAutoencoder\"\n",
    "\n",
    "h.train()\n",
    "h.infer()\n",
    "h.umap()\n",
    "v = h.visualize()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "hyrax",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
