{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Typed Objects"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "In addition to the interfaces offered by `ObjectAPIClient` and `DownloadedObject`, which provides access to Geoscience Objects in a way that is agnostic to the specific object type, the `evo-objects` package also provides a set of \"typed objects\" that represent specific object types."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "As usual, we need to first authenticate, which we can do using the `ServiceManagerWidget` from the `evo-notebooks` package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.notebooks import ServiceManagerWidget\n",
    "\n",
    "manager = await ServiceManagerWidget.with_auth_code(\n",
    "    client_id=\"your-client-id\",\n",
    "    cache_location=\"./notebook-data\",\n",
    ").login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the widgets extension for rich HTML display of objects\n",
    "%load_ext evo.widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## Creating a regular 3D grid object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "To create a new 'regular-3d-grid' object, we use the `Regular3DGridData` class to define all of the properties of the grid, then pass that to the `Regular3DGrid.create` class method.\n",
    "\n",
    "The `cell_data` and `vertex_data` properties are dataframes that define the attributes of the cells and vertices respectively. Each column in the dataframe represents a different attribute.\n",
    "\n",
    "Note the order of the data is important, and must match the order of the cells and vertices in the grid. The cells are ordered such that the x-axis varies the fastest, then y, then z. So for a grid of size (10, 10, 5), the first 10 rows of the dataframe correspond to the cells at (0, 0, 0) to (9, 0, 0), the next 10 rows correspond to (0, 1, 0) to (9, 1, 0), and so on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from evo.objects.typed import Point3, Regular3DGrid, Regular3DGridData, Rotation, Size3d, Size3i\n",
    "\n",
    "grid = Regular3DGridData(\n",
    "    name=\"Test Grid\",\n",
    "    origin=Point3(0, 0, 0),\n",
    "    size=Size3i(10, 10, 5),\n",
    "    cell_size=Size3d(2.5, 5, 5),\n",
    "    cell_data=pd.DataFrame(\n",
    "        {\n",
    "            \"value\": np.random.rand(10 * 10 * 5),\n",
    "            \"cat\": pd.Categorical.from_codes(np.random.choice(range(4), size=10 * 10 * 5), [\"a\", \"b\", \"c\", \"d\"]),\n",
    "        }\n",
    "    ),\n",
    "    vertex_data=pd.DataFrame(\n",
    "        {\n",
    "            \"elevation\": np.random.rand(11 * 11 * 6),\n",
    "        }\n",
    "    ),\n",
    "    rotation=Rotation(90, 0, 0),\n",
    ")\n",
    "created_grid = await Regular3DGrid.create(manager, grid)\n",
    "print(f\"Created regular-3d-grid object: {created_grid.metadata.url}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "## Downloading a regular 3D grid object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "To download an existing 'regular-3d-grid' object, and inspect it, you can use the `Regular3DGrid.from_reference` class method. This method takes the context and the URL of the object to download.\n",
    "\n",
    "For the example, we will download the object we just created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import object_from_reference\n",
    "\n",
    "grid = await object_from_reference(manager, created_grid.metadata.url)\n",
    "\n",
    "# Simply displaying the object shows a pretty-printed summary with Portal and Viewer links\n",
    "grid"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "### Accessing object properties\n",
    "\n",
    "You can also access individual properties programmatically:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Downloaded regular-3d-grid object with name: {grid.name}\")\n",
    "print(f\"Origin: {grid.origin}\")\n",
    "print(f\"Size: {grid.size}\")\n",
    "print(f\"Cell Size: {grid.cell_size}\")\n",
    "print(f\"Rotation: {grid.rotation}\")\n",
    "print(f\"Bounding Box: {grid.bounding_box}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "### Viewing attributes\n",
    "\n",
    "You can view the available attributes using the `attributes` property, which provides a pretty-printed table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "# View cell attributes (shortcut for grid.cells.attributes)\n",
    "grid.attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "# View vertex attributes\n",
    "grid.vertex_attributes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "### Getting data as DataFrames\n",
    "\n",
    "The `to_dataframe()` method provides a convenient way to get the cell data. You can also use `grid.cells.to_dataframe()` and `grid.vertices.to_dataframe()` for more explicit access."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get all cell data using to_dataframe()\n",
    "await grid.to_dataframe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or use the explicit cells/vertices access\n",
    "await grid.vertices.to_dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19",
   "metadata": {},
   "source": [
    "## Updating a regular 3D grid object"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "To update an existing 'regular-3d-grid' object, we can modify the properties of the `Regular3DGrid` instance, and then call the `update` method to submit the changes to the Geoscience Object service. This will create a new version of the object with the updated properties.\n",
    "\n",
    "Updating the cell or vertex data can be done by calling the `from_dataframe` method on the `cells` or `vertices` attributes respectively. This will upload the new data to the Geoscience Object service, but the object will remain the same version until the `update` method is called."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Version before update: {grid.metadata.version_id}\")\n",
    "\n",
    "grid.name = \"Updated Test Grid\"\n",
    "\n",
    "await grid.cells.from_dataframe(\n",
    "    pd.DataFrame(\n",
    "        {\n",
    "            \"value\": np.ones(10 * 10 * 5),\n",
    "        }\n",
    "    )\n",
    ")\n",
    "\n",
    "# Update the object to create a new version with the modified properties\n",
    "await grid.update()\n",
    "print(f\"Updated version: {grid.metadata.version_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22",
   "metadata": {},
   "source": [
    "## Creating a PointSet object\n",
    "\n",
    "To create a new 'pointset' object, we use the `PointSetData` class to define the point locations and attributes, then pass that to the `PointSet.create` class method.\n",
    "\n",
    "The `locations` DataFrame must have 'x', 'y', 'z' columns for coordinates. Any additional columns will be treated as point attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import PointSet, PointSetData\n",
    "\n",
    "# Create a pointset with 100 random points\n",
    "num_points = 100\n",
    "pointset_data = PointSetData(\n",
    "    name=\"Test PointSet\",\n",
    "    locations=pd.DataFrame(\n",
    "        {\n",
    "            \"x\": np.random.rand(num_points) * 100,\n",
    "            \"y\": np.random.rand(num_points) * 100,\n",
    "            \"z\": np.random.rand(num_points) * 50,\n",
    "            \"grade\": np.random.rand(num_points) * 10,  # Example attribute\n",
    "            \"category\": pd.Categorical.from_codes(\n",
    "                np.random.choice(range(3), size=num_points), [\"low\", \"medium\", \"high\"]\n",
    "            ),\n",
    "        }\n",
    "    ),\n",
    ")\n",
    "\n",
    "created_pointset = await PointSet.create(manager, pointset_data)\n",
    "print(f\"Created pointset object: {created_pointset.metadata.url}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24",
   "metadata": {},
   "source": [
    "## Downloading a PointSet object\n",
    "\n",
    "To download an existing 'pointset' object, use the `PointSet.from_reference` class method. Simply displaying the object shows a pretty-printed summary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import object_from_reference\n",
    "\n",
    "pointset = await object_from_reference(manager, created_pointset.metadata.url)\n",
    "\n",
    "# Pretty-printed display with Portal and Viewer links\n",
    "pointset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26",
   "metadata": {},
   "source": [
    "### Accessing pointset properties and data\n",
    "\n",
    "The `attributes` property provides quick access to point attributes, and `to_dataframe()` returns all data including coordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Number of points: {pointset.num_points}\")\n",
    "print(f\"Bounding Box: {pointset.bounding_box}\")\n",
    "\n",
    "# View available attributes\n",
    "pointset.attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the full dataframe with coordinates and attributes using to_dataframe()\n",
    "await pointset.to_dataframe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Or get just the coordinates\n",
    "await pointset.coordinates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30",
   "metadata": {},
   "source": [
    "## Creating a regular masked 3D grid object\n",
    "\n",
    "A masked grid is similar to a regular grid, but it includes a boolean mask indicating which cells are \"active\". Only active cells have attribute values, which can significantly reduce storage when working with sub-regions of a larger grid (e.g., a mining pit or geological domain).\n",
    "\n",
    "The `mask` is a 1D boolean array with the same length as the total number of cells in the grid. The `cell_data` should only contain rows for the active cells (where `mask=True`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import RegularMasked3DGrid, RegularMasked3DGridData\n",
    "\n",
    "# Create a mask where only some cells are active (e.g., alternating pattern)\n",
    "mask = np.array([True, False, True, False, True] * 100, dtype=bool)  # 500 cells, 300 active\n",
    "number_active = np.sum(mask)\n",
    "\n",
    "masked_grid = RegularMasked3DGridData(\n",
    "    name=\"Test Masked Grid\",\n",
    "    origin=Point3(0, 0, 0),\n",
    "    size=Size3i(10, 10, 5),\n",
    "    cell_size=Size3d(2.5, 5, 5),\n",
    "    mask=mask,\n",
    "    cell_data=pd.DataFrame(\n",
    "        {\n",
    "            # Only provide data for active cells\n",
    "            \"value\": np.random.rand(number_active),\n",
    "            \"category\": pd.Categorical.from_codes(\n",
    "                np.random.choice(range(3), size=number_active), [\"rock\", \"ore\", \"waste\"]\n",
    "            ),\n",
    "        }\n",
    "    ),\n",
    "    rotation=Rotation(45, 0, 0),\n",
    ")\n",
    "created_masked_grid = await RegularMasked3DGrid.create(manager, masked_grid)\n",
    "print(f\"Created regular-masked-3d-grid object: {created_masked_grid.metadata.url}\")\n",
    "print(f\"Total cells: {created_masked_grid.size.total_size}, Active cells: {created_masked_grid.cells.number_active}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32",
   "metadata": {},
   "source": [
    "### Downloading and inspecting a masked grid\n",
    "\n",
    "Simply displaying the masked grid shows a pretty-printed summary. The mask can be retrieved using `cells.get_mask()`, which returns a boolean numpy array. The cell data only contains values for active cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import object_from_reference\n",
    "\n",
    "masked_grid = await object_from_reference(manager, created_masked_grid.metadata.url)\n",
    "\n",
    "# Pretty-printed display\n",
    "masked_grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34",
   "metadata": {},
   "outputs": [],
   "source": [
    "# View available attributes\n",
    "masked_grid.attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Size: {masked_grid.size}, Active cells: {masked_grid.cells.number_active}\")\n",
    "\n",
    "# Get the mask\n",
    "grid_mask = await masked_grid.cells.get_mask()\n",
    "print(f\"Mask shape: {grid_mask.shape}, Active count: {np.sum(grid_mask)}\")\n",
    "\n",
    "# Get the cell data using to_dataframe() (only for active cells)\n",
    "cell_df = await masked_grid.to_dataframe()\n",
    "print(f\"Cell data shape: {cell_df.shape}\")\n",
    "cell_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36",
   "metadata": {},
   "source": [
    "## Creating a tensor 3D grid object\n",
    "\n",
    "A tensor grid is a 3D grid where cells can have different sizes along each axis. Instead of a uniform `cell_size`, you provide arrays of cell sizes for each axis (`cell_sizes_x`, `cell_sizes_y`, `cell_sizes_z`).\n",
    "\n",
    "This is useful for grids with variable resolution, such as grids that are finer in areas of interest and coarser elsewhere."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import Tensor3DGrid, Tensor3DGridData\n",
    "\n",
    "# Define variable cell sizes along each axis\n",
    "# X-axis: 10 cells with varying sizes (finer in the middle)\n",
    "cell_sizes_x = np.array([5.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0, 2.0, 3.0, 5.0])\n",
    "# Y-axis: 8 cells with uniform sizes\n",
    "cell_sizes_y = np.array([2.0] * 8)\n",
    "# Z-axis: 5 cells with increasing sizes (finer at top)\n",
    "cell_sizes_z = np.array([1.0, 1.5, 2.0, 2.5, 3.0])\n",
    "\n",
    "size = Size3i(len(cell_sizes_x), len(cell_sizes_y), len(cell_sizes_z))\n",
    "num_cells = size.total_size\n",
    "num_vertices = (size.nx + 1) * (size.ny + 1) * (size.nz + 1)\n",
    "\n",
    "tensor_grid = Tensor3DGridData(\n",
    "    name=\"Variable Resolution Grid\",\n",
    "    origin=Point3(100, 200, 0),\n",
    "    size=size,\n",
    "    cell_sizes_x=cell_sizes_x,\n",
    "    cell_sizes_y=cell_sizes_y,\n",
    "    cell_sizes_z=cell_sizes_z,\n",
    "    cell_data=pd.DataFrame(\n",
    "        {\n",
    "            \"grade\": np.random.rand(num_cells) * 10,\n",
    "            \"density\": np.random.rand(num_cells) * 2 + 2,\n",
    "        }\n",
    "    ),\n",
    "    vertex_data=pd.DataFrame(\n",
    "        {\n",
    "            \"elevation\": np.random.rand(num_vertices) * 100,\n",
    "        }\n",
    "    ),\n",
    ")\n",
    "created_tensor_grid = await Tensor3DGrid.create(manager, tensor_grid)\n",
    "print(f\"Created tensor-3d-grid object: {created_tensor_grid.metadata.url}\")\n",
    "print(f\"Grid extent: X={np.sum(cell_sizes_x):.1f}, Y={np.sum(cell_sizes_y):.1f}, Z={np.sum(cell_sizes_z):.1f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38",
   "metadata": {},
   "source": [
    "### Downloading and inspecting a tensor grid\n",
    "\n",
    "Simply displaying the tensor grid shows a pretty-printed summary. The cell sizes for each axis can be accessed via the `cell_sizes_x`, `cell_sizes_y`, and `cell_sizes_z` properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.objects.typed import object_from_reference\n",
    "\n",
    "tensor_grid = await object_from_reference(manager, created_tensor_grid.metadata.url)\n",
    "\n",
    "# Pretty-printed display\n",
    "tensor_grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40",
   "metadata": {},
   "outputs": [],
   "source": [
    "# View cell and vertex attributes\n",
    "print(\"Cell attributes:\")\n",
    "display(tensor_grid.attributes)\n",
    "print(\"\\nVertex attributes:\")\n",
    "display(tensor_grid.vertex_attributes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Origin: {tensor_grid.origin}\")\n",
    "print(f\"Size: {tensor_grid.size}\")\n",
    "print(f\"Bounding Box: {tensor_grid.bounding_box}\")\n",
    "print(f\"\\nCell sizes X: {tensor_grid.cell_sizes_x}\")\n",
    "print(f\"Cell sizes Y: {tensor_grid.cell_sizes_y}\")\n",
    "print(f\"Cell sizes Z: {tensor_grid.cell_sizes_z}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the cell data using to_dataframe()\n",
    "cell_df = await tensor_grid.to_dataframe()\n",
    "print(f\"Cell data: {cell_df.shape[0]} cells with columns {list(cell_df.columns)}\")\n",
    "cell_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get vertex data\n",
    "vertex_df = await tensor_grid.vertices.to_dataframe()\n",
    "print(f\"Vertex data: {vertex_df.shape[0]} vertices with columns {list(vertex_df.columns)}\")\n",
    "vertex_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44",
   "metadata": {},
   "outputs": [],
   "source": [
    "from evo.widgets import get_viewer_url_for_objects\n",
    "\n",
    "# View all the objects in the same scene in the viewer\n",
    "get_viewer_url_for_objects(manager, [tensor_grid, masked_grid, pointset, grid])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "evo-python-sdk",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
