{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e520122e",
   "metadata": {
    "papermill": {
     "duration": 0.001927,
     "end_time": "2025-12-07T04:20:07.203064",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.201137",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Working with Complex Hierarchical Data\n",
    "\n",
    "This tutorial demonstrates how linkml-data-qc handles deeply nested data structures with multiple levels of lists and objects. We'll use a disease knowledge base as an example.\n",
    "\n",
    "## The Challenge\n",
    "\n",
    "Real-world knowledge bases often have complex structures:\n",
    "\n",
    "- Diseases with multiple subtypes\n",
    "- Each subtype has evidence citations\n",
    "- Pathophysiology entries with cell type annotations\n",
    "- Phenotypes with nested ontology term bindings\n",
    "\n",
    "Tracking compliance across all these nested structures requires understanding path notation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a5166f",
   "metadata": {
    "papermill": {
     "duration": 0.001186,
     "end_time": "2025-12-07T04:20:07.205760",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.204574",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Setup: A Complex Schema\n",
    "\n",
    "Let's create a schema that models a disease knowledge base with nested structures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dbeedaf3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:07.208576Z",
     "iopub.status.busy": "2025-12-07T04:20:07.208440Z",
     "iopub.status.idle": "2025-12-07T04:20:07.244665Z",
     "shell.execute_reply": "2025-12-07T04:20:07.243972Z"
    },
    "papermill": {
     "duration": 0.038971,
     "end_time": "2025-12-07T04:20:07.245806",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.206835",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Schema created with nested classes!\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat > /tmp/disease_kb_schema.yaml << 'EOF'\n",
    "id: https://example.org/disease-kb\n",
    "name: disease_kb_schema\n",
    "prefixes:\n",
    "  linkml: https://w3id.org/linkml/\n",
    "imports:\n",
    "  - linkml:types\n",
    "default_range: string\n",
    "\n",
    "classes:\n",
    "  Disease:\n",
    "    attributes:\n",
    "      name:\n",
    "        required: true\n",
    "      description:\n",
    "        recommended: true\n",
    "      disease_term:\n",
    "        range: OntologyAnnotation\n",
    "        inlined: true\n",
    "        recommended: true\n",
    "      has_subtypes:\n",
    "        range: DiseaseSubtype\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "      pathophysiology:\n",
    "        range: PathophysiologyStep\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "      phenotypes:\n",
    "        range: Phenotype\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "\n",
    "  OntologyAnnotation:\n",
    "    attributes:\n",
    "      preferred_term:\n",
    "        recommended: true\n",
    "      term:\n",
    "        range: OntologyTerm\n",
    "        inlined: true\n",
    "        recommended: true\n",
    "\n",
    "  OntologyTerm:\n",
    "    attributes:\n",
    "      id:\n",
    "        recommended: true\n",
    "      label:\n",
    "        recommended: true\n",
    "\n",
    "  DiseaseSubtype:\n",
    "    attributes:\n",
    "      name:\n",
    "        required: true\n",
    "      description:\n",
    "        recommended: true\n",
    "      evidence:\n",
    "        range: Evidence\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "\n",
    "  Evidence:\n",
    "    attributes:\n",
    "      reference:\n",
    "        recommended: true\n",
    "      supports:\n",
    "        recommended: true\n",
    "      snippet:\n",
    "        recommended: true\n",
    "      explanation:\n",
    "        recommended: true\n",
    "\n",
    "  PathophysiologyStep:\n",
    "    attributes:\n",
    "      name:\n",
    "        required: true\n",
    "      description:\n",
    "        recommended: true\n",
    "      cell_types:\n",
    "        range: OntologyAnnotation\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "      evidence:\n",
    "        range: Evidence\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "\n",
    "  Phenotype:\n",
    "    attributes:\n",
    "      name:\n",
    "        required: true\n",
    "      category:\n",
    "        recommended: true\n",
    "      frequency:\n",
    "        recommended: true\n",
    "      phenotype_term:\n",
    "        range: OntologyAnnotation\n",
    "        inlined: true\n",
    "        recommended: true\n",
    "      evidence:\n",
    "        range: Evidence\n",
    "        multivalued: true\n",
    "        inlined_as_list: true\n",
    "EOF\n",
    "echo \"Schema created with nested classes!\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a85a5c5",
   "metadata": {
    "papermill": {
     "duration": 0.001152,
     "end_time": "2025-12-07T04:20:07.248505",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.247353",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Test Data: A Disease Entry\n",
    "\n",
    "Now let's create a disease entry with varying levels of completeness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e25ea77a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:07.251736Z",
     "iopub.status.busy": "2025-12-07T04:20:07.251528Z",
     "iopub.status.idle": "2025-12-07T04:20:07.263959Z",
     "shell.execute_reply": "2025-12-07T04:20:07.263587Z"
    },
    "papermill": {
     "duration": 0.014951,
     "end_time": "2025-12-07T04:20:07.264756",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.249805",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complex disease entry created!\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat > /tmp/disease_aps.yaml << 'EOF'\n",
    "name: Antiphospholipid Syndrome\n",
    "description: A systemic autoimmune disorder characterized by antiphospholipid antibodies.\n",
    "disease_term:\n",
    "  preferred_term: antiphospholipid syndrome\n",
    "  term:\n",
    "    id: MONDO:0000810\n",
    "    label: antiphospholipid syndrome\n",
    "\n",
    "has_subtypes:\n",
    "  - name: Primary APS\n",
    "    description: occurs in the absence of any other disease\n",
    "    evidence:\n",
    "      - reference: PMID:16338214\n",
    "        supports: SUPPORT\n",
    "        snippet: the condition can exist on its own\n",
    "        explanation: Supports primary APS definition\n",
    "      - reference: PMID:27550302\n",
    "        supports: SUPPORT\n",
    "        snippet: APS can be isolated (primary APS)\n",
    "        # Missing explanation!\n",
    "  - name: Secondary APS\n",
    "    description: occurs with other autoimmune diseases\n",
    "    evidence:\n",
    "      - reference: PMID:11014973\n",
    "        supports: SUPPORT\n",
    "        snippet: APS may be associated with another autoimmune disease\n",
    "        explanation: Confirms secondary APS\n",
    "  - name: Asymptomatic APS\n",
    "    # Missing description!\n",
    "    evidence:\n",
    "      - reference: PMID:17145604\n",
    "        supports: SUPPORT\n",
    "        # Missing snippet and explanation!\n",
    "\n",
    "pathophysiology:\n",
    "  - name: Antibody Production\n",
    "    description: The immune system produces antiphospholipid antibodies.\n",
    "    cell_types:\n",
    "      - preferred_term: B cell\n",
    "        term:\n",
    "          id: CL:0000236\n",
    "          label: B cell\n",
    "      - preferred_term: T cell\n",
    "        # Missing nested term!\n",
    "    evidence:\n",
    "      - reference: PMID:29867951\n",
    "        supports: SUPPORT\n",
    "        snippet: production of antibodies that bind phospholipid-binding proteins\n",
    "        explanation: Describes antibody production mechanism\n",
    "  - name: Blood Clot Formation\n",
    "    description: Antibodies increase thrombosis risk.\n",
    "    # No cell_types - that's fine, not always applicable\n",
    "    evidence:\n",
    "      - reference: PMID:22100379\n",
    "        supports: SUPPORT\n",
    "        snippet: antibodies associated with thrombosis risk\n",
    "        explanation: Supports clotting mechanism\n",
    "\n",
    "phenotypes:\n",
    "  - name: Deep Vein Thrombosis\n",
    "    category: Thrombosis\n",
    "    frequency: FREQUENT\n",
    "    phenotype_term:\n",
    "      preferred_term: DVT\n",
    "      term:\n",
    "        id: HP:0002625\n",
    "        label: deep vein thrombosis\n",
    "    evidence:\n",
    "      - reference: PMID:36575066\n",
    "        supports: SUPPORT\n",
    "        snippet: venous thromboembolism is frequent\n",
    "        explanation: Confirms DVT frequency\n",
    "  - name: Preeclampsia\n",
    "    category: Pregnancy-Related\n",
    "    # Missing frequency!\n",
    "    phenotype_term:\n",
    "      preferred_term: Preeclampsia\n",
    "      term:\n",
    "        id: HP:0100602\n",
    "        # Missing label!\n",
    "    evidence:\n",
    "      - reference: PMID:26815583\n",
    "        supports: PARTIAL\n",
    "        # Missing snippet and explanation!\n",
    "  - name: Livedo Reticularis\n",
    "    category: Dermatologic\n",
    "    frequency: OCCASIONAL\n",
    "    # Missing phenotype_term entirely!\n",
    "    evidence:\n",
    "      - reference: PMID:26223086\n",
    "        supports: SUPPORT\n",
    "        snippet: common cutaneous manifestation\n",
    "        explanation: Supports dermatologic involvement\n",
    "EOF\n",
    "echo \"Complex disease entry created!\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f759ed68",
   "metadata": {
    "papermill": {
     "duration": 0.001111,
     "end_time": "2025-12-07T04:20:07.267263",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.266152",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Analyzing Nested Compliance\n",
    "\n",
    "Let's run the analysis and see how paths are reported:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "84613d23",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:07.270539Z",
     "iopub.status.busy": "2025-12-07T04:20:07.270343Z",
     "iopub.status.idle": "2025-12-07T04:20:07.634988Z",
     "shell.execute_reply": "2025-12-07T04:20:07.634528Z"
    },
    "papermill": {
     "duration": 0.367422,
     "end_time": "2025-12-07T04:20:07.635936",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.268514",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compliance Report: /tmp/disease_aps.yaml\n",
      "Target Class: Disease\n",
      "Global Compliance: 85.7% (24/28)\n",
      "Weig"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hted Compliance: 82.6%\n",
      "\n",
      "Summary by Slot:\n",
      "  category: 100.0%\n",
      "  disease_term: 100.0%\n",
      "  frequency: 66.7"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%\n",
      "  id: 100.0%\n",
      "  label: 75.0%\n",
      "  phenotype_term: 66.7%\n",
      "  preferred_term: 100.0%\n",
      "  term: 80.0%\n",
      "\n",
      "Aggreg"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ated Scores by List Path:\n",
      "  pathophysiology[].cell_types[].preferred_term: 100.0% (2/2)\n",
      "  pathophysi"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ology[].cell_types[].term: 50.0% (1/2)\n",
      "  pathophysiology[].cell_types[].term.id: 100.0% (1/1)\n",
      "  path"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ophysiology[].cell_types[].term.label: 100.0% (1/1)\n",
      "  phenotypes[].category: 100.0% (3/3)\n",
      "  phenotyp"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "es[].frequency: 66.7% (2/3)\n",
      "  phenotypes[].phenotype_term: 66.7% (2/3)\n",
      "  phenotypes[].phenotype_term"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".preferred_term: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.term: 100.0% (2/2)\n",
      "  phenotypes[].phenot"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ype_term.term.id: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.term.label: 50.0% (1/2)\n",
      "\n",
      "Detailed Path "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scores:\n",
      "  (root) (Disease): 100.0%\n",
      "    - disease_term: OK\n",
      "  disease_term (OntologyAnnotation): 100.0"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  disease_term.term (OntologyTerm): 100.0%\n",
      "    - id: OK\n",
      "  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  - label: OK\n",
      "  pathophysiology[0].cell_types[0] (OntologyAnnotation): 100.0%\n",
      "    - preferred_term: "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK\n",
      "    - term: OK\n",
      "  pathophysiology[0].cell_types[0].term (OntologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "label: OK\n",
      "  pathophysiology[0].cell_types[1] (OntologyAnnotation): 50.0%\n",
      "    - preferred_term: OK\n",
      "  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  - term: MISSING\n",
      "  phenotypes[0] (Phenotype): 100.0%\n",
      "    - category: OK\n",
      "    - frequency: OK\n",
      "    - p"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "henotype_term: OK\n",
      "  phenotypes[0].phenotype_term (OntologyAnnotation): 100.0%\n",
      "    - preferred_term: "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK\n",
      "    - term: OK\n",
      "  phenotypes[0].phenotype_term.term (OntologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - labe"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l: OK\n",
      "  phenotypes[1] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "    - frequency: MISSING\n",
      "    - phenotype"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_term: OK\n",
      "  phenotypes[1].phenotype_term (OntologyAnnotation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    -"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " term: OK\n",
      "  phenotypes[1].phenotype_term.term (OntologyTerm): 50.0%\n",
      "    - id: OK\n",
      "    - label: MISSIN"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "G\n",
      "  phenotypes[2] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "    - frequency: OK\n",
      "    - phenotype_term: MI"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SSING\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "linkml-data-qc /tmp/disease_aps.yaml \\\n",
    "    -s /tmp/disease_kb_schema.yaml \\\n",
    "    -t Disease"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "453ee825",
   "metadata": {
    "papermill": {
     "duration": 0.001529,
     "end_time": "2025-12-07T04:20:07.639338",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.637809",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Understanding the Paths\n",
    "\n",
    "The output shows aggregated paths like:\n",
    "\n",
    "- `has_subtypes[].description` - All subtype descriptions\n",
    "- `has_subtypes[].evidence[].snippet` - All snippets across all evidence in all subtypes\n",
    "- `pathophysiology[].cell_types[].term` - All nested term objects in cell_types\n",
    "- `phenotypes[].phenotype_term.term.label` - Deeply nested labels\n",
    "\n",
    "Let's look at the JSON output for more detail:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fd465d25",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:07.642876Z",
     "iopub.status.busy": "2025-12-07T04:20:07.642747Z",
     "iopub.status.idle": "2025-12-07T04:20:08.062719Z",
     "shell.execute_reply": "2025-12-07T04:20:08.062292Z"
    },
    "papermill": {
     "duration": 0.426109,
     "end_time": "2025-12-07T04:20:08.066928",
     "exception": false,
     "start_time": "2025-12-07T04:20:07.640819",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"file_path\": \"/tmp/disease_aps.yaml\",\n",
      "    \"target_class\": \"Disease\",\n",
      "    \"schema_path\": \"/tmp/"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "disease_kb_schema.yaml\",\n",
      "    \"global_compliance\": 85.71428571428571,\n",
      "    \"weighted_compliance\": 82.6"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "086956521739,\n",
      "    \"total_checks\": 28,\n",
      "    \"total_populated\": 24,\n",
      "    \"path_scores\": [\n",
      "        {\n",
      "    "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        \"path\": \"(root)\",\n",
      "            \"parent_class\": \"Disease\",\n",
      "            \"item_count\": 1,\n",
      "      "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      \"slot_scores\": [\n",
      "                {\n",
      "                    \"path\": \"(root)\",\n",
      "                    \""
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "slot_name\": \"disease_term\",\n",
      "                    \"populated\": 1,\n",
      "                    \"total\": 1,\n",
      "    "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                \"percentage\": 100.0\n",
      "                }\n",
      "            ],\n",
      "            \"overall_percentage"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\": 100.0\n",
      "        },\n",
      "        {\n",
      "            \"path\": \"disease_term\",\n",
      "            \"parent_class\": \"Ontol"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ogyAnnotation\",\n",
      "            \"item_count\": 1,\n",
      "            \"slot_scores\": [\n",
      "                {\n",
      "        "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            \"path\": \"disease_term\",\n",
      "                    \"slot_name\": \"preferred_term\",\n",
      "             "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       \"populated\": 1,\n",
      "                    \"total\": 1,\n",
      "                    \"percentage\": 100.0\n",
      "     "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           },\n",
      "                {\n",
      "                    \"path\": \"disease_term\",\n",
      "                    \"slo"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t_name\": \"term\",\n",
      "                    \"populated\": 1,\n",
      "                    \"total\": 1,\n",
      "               "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     \"percentage\": 100.0\n",
      "                }\n",
      "            ],\n",
      "            \"overall_percentage\": 100.0\n",
      "  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      },\n",
      "        {\n",
      "            \"path\": \"disease_term.term\",\n",
      "            \"parent_class\": \"OntologyTer"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "m\",\n",
      "            \"item_count\": 1,\n",
      "            \"slot_scores\": [\n",
      "                {\n",
      "                    "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"path\": \"disease_term.term\",\n",
      "                    \"slot_name\": \"id\",\n",
      "                    \"populated\":"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 1,\n",
      "                    \"total\": 1,\n",
      "                    \"percentage\": 100.0\n",
      "                },\n",
      "     "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           {\n",
      "                    \"path\": \"disease_term.term\",\n",
      "                    \"slot_name\": \"labe"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l\",\n",
      "                    \"populated\": 1,\n",
      "                    \"total\": 1,\n",
      "                    \"percent"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "age\": 100.0\n",
      "                }\n",
      "            ],\n",
      "            \"overall_percentage\": 100.0\n",
      "        },\n",
      "    "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    {\n",
      "            \"path\": \"pathophysiology[0].cell_types[0]\",\n",
      "            \"parent_class\": \"OntologyA"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nnotation\",\n",
      "            \"item_count\": 1,\n",
      "            \"slot_scores\": [\n",
      "                {\n",
      "            "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        \"path\": \"pathophysiology[0].cell_types[0]\",\n",
      "                    \"slot_name\": \"preferred_term"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\",\n",
      "                    \"populated\": 1,\n",
      "                    \"total\": 1,\n",
      "                    \"percenta"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ge\": 100.0\n",
      "                },\n",
      "                {\n",
      "                    \"path\": \"pathophysiology[0].cell"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_types[0]\",\n",
      "                    \"slot_name\": \"term\",\n",
      "                    \"populated\": 1,\n",
      "           "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         \"total\": 1,\n",
      "                    \"percentage\": 100.0\n",
      "                }\n",
      "            ],\n",
      "      "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      \"overall_percentage\": 100.0\n",
      "        },\n",
      "        {\n",
      "            \"path\": \"pathophysiology[0].cell_"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "types[0].term\",\n",
      "            \"parent_class\": \"OntologyTerm\",\n",
      "            \"item_count\": 1,\n",
      "           "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " \"slot_scores\": [\n",
      "                {\n",
      "                    \"path\": \"pathophysiology[0].cell_types[0].te"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rm\",\n",
      "                    \"slot_name\": \"id\",\n",
      "                    \"populated\": 1,\n",
      "                    "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"total\": 1,\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "linkml-data-qc /tmp/disease_aps.yaml \\\n",
    "    -s /tmp/disease_kb_schema.yaml \\\n",
    "    -t Disease \\\n",
    "    -f json | python -m json.tool | head -100"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1c0e96b",
   "metadata": {
    "papermill": {
     "duration": 0.002731,
     "end_time": "2025-12-07T04:20:08.072444",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.069713",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Configuring Weights for Nested Paths\n",
    "\n",
    "We can prioritize certain nested paths over others:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b705686c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:08.078417Z",
     "iopub.status.busy": "2025-12-07T04:20:08.078235Z",
     "iopub.status.idle": "2025-12-07T04:20:08.090726Z",
     "shell.execute_reply": "2025-12-07T04:20:08.090424Z"
    },
    "papermill": {
     "duration": 0.016372,
     "end_time": "2025-12-07T04:20:08.091586",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.075214",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Configuration created!\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cat > /tmp/disease_qc_config.yaml << 'EOF'\n",
    "default_weight: 1.0\n",
    "\n",
    "# Slot-level configuration\n",
    "slots:\n",
    "  # Ontology term bindings are critical\n",
    "  id:\n",
    "    weight: 3.0\n",
    "    min_compliance: 90.0\n",
    "  label:\n",
    "    weight: 2.0\n",
    "    min_compliance: 80.0\n",
    "  # Evidence snippets are important\n",
    "  snippet:\n",
    "    weight: 2.0\n",
    "    min_compliance: 75.0\n",
    "  explanation:\n",
    "    weight: 1.5\n",
    "  # Descriptions nice-to-have\n",
    "  description:\n",
    "    weight: 0.5\n",
    "\n",
    "# Path-level overrides for specific nested locations\n",
    "paths:\n",
    "  # Phenotype terms are especially critical\n",
    "  \"phenotypes[].phenotype_term.term.id\":\n",
    "    weight: 5.0\n",
    "    min_compliance: 95.0\n",
    "  # Root disease term is essential\n",
    "  \"disease_term.term.id\":\n",
    "    weight: 5.0\n",
    "    min_compliance: 100.0\n",
    "EOF\n",
    "echo \"Configuration created!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6c5596c7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:08.097915Z",
     "iopub.status.busy": "2025-12-07T04:20:08.097682Z",
     "iopub.status.idle": "2025-12-07T04:20:08.433304Z",
     "shell.execute_reply": "2025-12-07T04:20:08.432897Z"
    },
    "papermill": {
     "duration": 0.339937,
     "end_time": "2025-12-07T04:20:08.434350",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.094413",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compliance Report: /tmp/disease_aps.yaml\n",
      "Target Class: Disease\n",
      "Global Compliance: 85.7% (24/28)\n",
      "Weig"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hted Compliance: 86.1%\n",
      "Config: /tmp/disease_qc_config.yaml\n",
      "\n",
      "Summary by Slot:\n",
      "  category: 100.0%\n",
      "  di"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sease_term: 100.0%\n",
      "  frequency: 66.7%\n",
      "  id: 100.0%\n",
      "  label: 75.0%\n",
      "  phenotype_term: 66.7%\n",
      "  preferre"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d_term: 100.0%\n",
      "  term: 80.0%\n",
      "\n",
      "Threshold Violations (1):\n",
      "  phenotypes[].phenotype_term.term.label: 50"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".0% < 80.0% (shortfall: 30.0%)\n",
      "\n",
      "Aggregated Scores by List Path:\n",
      "  pathophysiology[].cell_types[].pre"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ferred_term: 100.0% (2/2)\n",
      "  pathophysiology[].cell_types[].term: 50.0% (1/2)\n",
      "  pathophysiology[].cel"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l_types[].term.id: 100.0% (1/1)\n",
      "  pathophysiology[].cell_types[].term.label: 100.0% (1/1)\n",
      "  phenotyp"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "es[].category: 100.0% (3/3)\n",
      "  phenotypes[].frequency: 66.7% (2/3)\n",
      "  phenotypes[].phenotype_term: 66."
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7% (2/3)\n",
      "  phenotypes[].phenotype_term.preferred_term: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.te"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rm: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.term.id: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.t"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "erm.label: 50.0% (1/2)\n",
      "\n",
      "Detailed Path Scores:\n",
      "  (root) (Disease): 100.0%\n",
      "    - disease_term: OK\n",
      "  di"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sease_term (OntologyAnnotation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  disease_term.term "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(OntologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  pathophysiology[0].cell_types[0] (OntologyAnno"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  pathophysiology[0].cell_types[0].term (Ont"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  pathophysiology[0].cell_types[1] (OntologyAnnotati"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "on): 50.0%\n",
      "    - preferred_term: OK\n",
      "    - term: MISSING\n",
      "  phenotypes[0] (Phenotype): 100.0%\n",
      "    - ca"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tegory: OK\n",
      "    - frequency: OK\n",
      "    - phenotype_term: OK\n",
      "  phenotypes[0].phenotype_term (OntologyAnno"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  phenotypes[0].phenotype_term.term (Ontolog"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  phenotypes[1] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  - frequency: MISSING\n",
      "    - phenotype_term: OK\n",
      "  phenotypes[1].phenotype_term (OntologyAnnotation):"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  phenotypes[1].phenotype_term.term (OntologyTerm): "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "50.0%\n",
      "    - id: OK\n",
      "    - label: MISSING\n",
      "  phenotypes[2] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "    - "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "frequency: OK\n",
      "    - phenotype_term: MISSING\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "linkml-data-qc /tmp/disease_aps.yaml \\\n",
    "    -s /tmp/disease_kb_schema.yaml \\\n",
    "    -t Disease \\\n",
    "    -c /tmp/disease_qc_config.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d4a1cff",
   "metadata": {
    "papermill": {
     "duration": 0.002986,
     "end_time": "2025-12-07T04:20:08.440518",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.437532",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Checking for Threshold Violations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b581def5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:08.447371Z",
     "iopub.status.busy": "2025-12-07T04:20:08.447208Z",
     "iopub.status.idle": "2025-12-07T04:20:08.879679Z",
     "shell.execute_reply": "2025-12-07T04:20:08.878201Z"
    },
    "papermill": {
     "duration": 0.437425,
     "end_time": "2025-12-07T04:20:08.881005",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.443580",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compliance Report: /tmp/disease_aps.yaml\n",
      "Target Class: Disease\n",
      "Global Compliance: 85.7% (24/28)\n",
      "Weig"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hted Compliance: 86.1%\n",
      "Config: /tmp/disease_qc_config.yaml\n",
      "\n",
      "Summary by Slot:\n",
      "  category: 100.0%\n",
      "  di"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sease_term: 100.0%\n",
      "  frequency: 66.7%\n",
      "  id: 100.0%\n",
      "  label: 75.0%\n",
      "  phenotype_term: 66.7%\n",
      "  preferre"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d_term: 100.0%\n",
      "  term: 80.0%\n",
      "\n",
      "Threshold Violations (1):\n",
      "  phenotypes[].phenotype_term.term.label: 50"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".0% < 80.0% (shortfall: 30.0%)\n",
      "\n",
      "Aggregated Scores by List Path:\n",
      "  pathophysiology[].cell_types[].pre"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ferred_term: 100.0% (2/2)\n",
      "  pathophysiology[].cell_types[].term: 50.0% (1/2)\n",
      "  pathophysiology[].cel"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l_types[].term.id: 100.0% (1/1)\n",
      "  pathophysiology[].cell_types[].term.label: 100.0% (1/1)\n",
      "  phenotyp"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "es[].category: 100.0% (3/3)\n",
      "  phenotypes[].frequency: 66.7% (2/3)\n",
      "  phenotypes[].phenotype_term: 66."
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7% (2/3)\n",
      "  phenotypes[].phenotype_term.preferred_term: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.te"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rm: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.term.id: 100.0% (2/2)\n",
      "  phenotypes[].phenotype_term.t"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "erm.label: 50.0% (1/2)\n",
      "\n",
      "Detailed Path Scores:\n",
      "  (root) (Disease): 100.0%\n",
      "    - disease_term: OK\n",
      "  di"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sease_term (OntologyAnnotation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  disease_term.term "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(OntologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  pathophysiology[0].cell_types[0] (OntologyAnno"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  pathophysiology[0].cell_types[0].term (Ont"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ologyTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  pathophysiology[0].cell_types[1] (OntologyAnnotati"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "on): 50.0%\n",
      "    - preferred_term: OK\n",
      "    - term: MISSING\n",
      "  phenotypes[0] (Phenotype): 100.0%\n",
      "    - ca"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tegory: OK\n",
      "    - frequency: OK\n",
      "    - phenotype_term: OK\n",
      "  phenotypes[0].phenotype_term (OntologyAnno"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tation): 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  phenotypes[0].phenotype_term.term (Ontolog"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yTerm): 100.0%\n",
      "    - id: OK\n",
      "    - label: OK\n",
      "  phenotypes[1] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  - frequency: MISSING\n",
      "    - phenotype_term: OK\n",
      "  phenotypes[1].phenotype_term (OntologyAnnotation):"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 100.0%\n",
      "    - preferred_term: OK\n",
      "    - term: OK\n",
      "  phenotypes[1].phenotype_term.term (OntologyTerm): "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "50.0%\n",
      "    - id: OK\n",
      "    - label: MISSING\n",
      "  phenotypes[2] (Phenotype): 66.7%\n",
      "    - category: OK\n",
      "    - "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "frequency: OK\n",
      "    - phenotype_term: MISSING\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "1 threshold violation(s) found:\n",
      "  phenotypes[].phenotype_term.term.label: 50.0% < 80.0%\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Violations detected! Exit code: 1\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "linkml-data-qc /tmp/disease_aps.yaml \\\n",
    "    -s /tmp/disease_kb_schema.yaml \\\n",
    "    -t Disease \\\n",
    "    -c /tmp/disease_qc_config.yaml \\\n",
    "    --fail-on-violations || echo \"Violations detected! Exit code: $?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8caa1859",
   "metadata": {
    "papermill": {
     "duration": 0.00472,
     "end_time": "2025-12-07T04:20:08.890424",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.885704",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Finding Low-Compliance Areas\n",
    "\n",
    "Use CSV output to find the paths with lowest compliance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9ea419b3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:20:08.899551Z",
     "iopub.status.busy": "2025-12-07T04:20:08.899393Z",
     "iopub.status.idle": "2025-12-07T04:20:09.253333Z",
     "shell.execute_reply": "2025-12-07T04:20:09.252855Z"
    },
    "papermill": {
     "duration": 0.359738,
     "end_time": "2025-12-07T04:20:09.254137",
     "exception": false,
     "start_time": "2025-12-07T04:20:08.894399",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Lowest compliance areas ===\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "/tmp/disease_aps.yaml,pathophysiology[0].cell_types[1],OntologyAnnotation,term,0,1,0.0\r\n",
      "/tmp/diseas"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "e_aps.yaml,phenotypes[1],Phenotype,frequency,0,1,0.0\r\n",
      "/tmp/disease_aps.yaml,phenotypes[1].phenotype_"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "term.term,OntologyTerm,label,0,1,0.0\r\n",
      "/tmp/disease_aps.yaml,phenotypes[2],Phenotype,phenotype_term,0"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ",1,0.0\r\n",
      "/tmp/disease_aps.yaml,(root),Disease,disease_term,1,1,100.0\r\n",
      "/tmp/disease_aps.yaml,disease_t"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "erm,OntologyAnnotation,preferred_term,1,1,100.0\r\n",
      "/tmp/disease_aps.yaml,disease_term,OntologyAnnotati"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "on,term,1,1,100.0\r\n",
      "/tmp/disease_aps.yaml,disease_term.term,OntologyTerm,id,1,1,100.0\r\n",
      "/tmp/disease_a"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ps.yaml,disease_term.term,OntologyTerm,label,1,1,100.0\r\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "echo \"=== Lowest compliance areas ===\"\n",
    "linkml-data-qc /tmp/disease_aps.yaml \\\n",
    "    -s /tmp/disease_kb_schema.yaml \\\n",
    "    -t Disease \\\n",
    "    -f csv | tail -n +2 | sort -t, -k7 -n | head -10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "313e5acd",
   "metadata": {
    "papermill": {
     "duration": 0.003715,
     "end_time": "2025-12-07T04:20:09.270799",
     "exception": false,
     "start_time": "2025-12-07T04:20:09.267084",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Path Patterns Summary\n",
    "\n",
    "Here's what we learned about paths in complex data:\n",
    "\n",
    "| Pattern | Example | Meaning |\n",
    "|---------|---------|--------|\n",
    "| `(root)` | - | Top-level object |\n",
    "| `slot` | `disease_term` | Direct child object |\n",
    "| `slot.nested` | `disease_term.term` | Nested object |\n",
    "| `list[]` | `has_subtypes[]` | All items in list |\n",
    "| `list[].slot` | `has_subtypes[].description` | Slot in all list items |\n",
    "| `list[].nested[]` | `has_subtypes[].evidence[]` | Nested list in list |\n",
    "| `list[].obj.slot` | `phenotypes[].phenotype_term.term` | Deeply nested |\n",
    "\n",
    "## Key Takeaways\n",
    "\n",
    "1. **Aggregated paths** (with `[]`) show compliance across all matching items\n",
    "2. **Deeply nested paths** help identify specific curation gaps\n",
    "3. **Path-specific config** lets you prioritize critical nested locations\n",
    "4. **CSV output** is useful for finding lowest-compliance areas to prioritize curation"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 3.21081,
   "end_time": "2025-12-07T04:20:09.490775",
   "environment_variables": {},
   "exception": null,
   "input_path": "docs/notebooks/03_complex_hierarchies.ipynb",
   "output_path": "docs/notebooks/03_complex_hierarchies.ipynb",
   "parameters": {},
   "start_time": "2025-12-07T04:20:06.279965",
   "version": "2.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}