{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b7aead8b",
   "metadata": {
    "papermill": {
     "duration": 0.001989,
     "end_time": "2025-12-07T04:10:19.197393",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.195404",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Python API Tutorial\n",
    "\n",
    "While linkml-data-qc is primarily a CLI tool, it also provides a Python API for programmatic access.\n",
    "\n",
    "## Core Classes\n",
    "\n",
    "The main classes are:\n",
    "\n",
    "- `ComplianceAnalyzer` - Analyzes data files for compliance\n",
    "- `SchemaIntrospector` - Extracts recommended slots from schemas\n",
    "- `QCConfig` - Configuration for weights and thresholds\n",
    "- Formatters (`JSONFormatter`, `CSVFormatter`, `TextFormatter`) - Output formatting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ec27df9",
   "metadata": {
    "papermill": {
     "duration": 0.00117,
     "end_time": "2025-12-07T04:10:19.200003",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.198833",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Setup\n",
    "\n",
    "First, let's create the same test data as in the CLI tutorial:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "daf0a3e1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.203112Z",
     "iopub.status.busy": "2025-12-07T04:10:19.202963Z",
     "iopub.status.idle": "2025-12-07T04:10:19.224662Z",
     "shell.execute_reply": "2025-12-07T04:10:19.224304Z"
    },
    "papermill": {
     "duration": 0.024369,
     "end_time": "2025-12-07T04:10:19.225519",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.201150",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test files created!\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "# Create test schema\n",
    "cat > /tmp/disease_schema.yaml << 'EOF'\n",
    "id: https://example.org/disease\n",
    "name: disease_schema\n",
    "prefixes:\n",
    "  linkml: https://w3id.org/linkml/\n",
    "imports:\n",
    "  - linkml:types\n",
    "default_range: string\n",
    "\n",
    "classes:\n",
    "  Disease:\n",
    "    attributes:\n",
    "      id:\n",
    "        identifier: true\n",
    "      name:\n",
    "        required: true\n",
    "      description:\n",
    "        recommended: true\n",
    "      synonyms:\n",
    "        multivalued: true\n",
    "        recommended: true\n",
    "      ontology_id:\n",
    "        recommended: true\n",
    "EOF\n",
    "\n",
    "# Create test data\n",
    "cat > /tmp/disease_good.yaml << 'EOF'\n",
    "id: DISEASE:001\n",
    "name: Asthma\n",
    "description: A chronic respiratory condition\n",
    "synonyms:\n",
    "  - bronchial asthma\n",
    "ontology_id: MONDO:0004979\n",
    "EOF\n",
    "\n",
    "cat > /tmp/disease_poor.yaml << 'EOF'\n",
    "id: DISEASE:002\n",
    "name: Unknown Disease\n",
    "EOF\n",
    "\n",
    "echo \"Test files created!\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6097faab",
   "metadata": {
    "papermill": {
     "duration": 0.001576,
     "end_time": "2025-12-07T04:10:19.228571",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.226995",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Basic Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4e201874",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.232708Z",
     "iopub.status.busy": "2025-12-07T04:10:19.232570Z",
     "iopub.status.idle": "2025-12-07T04:10:19.504231Z",
     "shell.execute_reply": "2025-12-07T04:10:19.503837Z"
    },
    "papermill": {
     "duration": 0.275258,
     "end_time": "2025-12-07T04:10:19.505385",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.230127",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Global Compliance: 100.0%\n",
      "Total Checks: 3\n",
      "Total Populated: 3\n"
     ]
    }
   ],
   "source": [
    "from linkml_data_qc import ComplianceAnalyzer\n",
    "\n",
    "# Create an analyzer with your schema\n",
    "analyzer = ComplianceAnalyzer(\"/tmp/disease_schema.yaml\")\n",
    "\n",
    "# Analyze a file\n",
    "report = analyzer.analyze_file(\"/tmp/disease_good.yaml\", \"Disease\")\n",
    "\n",
    "print(f\"Global Compliance: {report.global_compliance}%\")\n",
    "print(f\"Total Checks: {report.total_checks}\")\n",
    "print(f\"Total Populated: {report.total_populated}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6516c6e4",
   "metadata": {
    "papermill": {
     "duration": 0.002131,
     "end_time": "2025-12-07T04:10:19.509016",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.506885",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Accessing Detailed Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c02d3819",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.512459Z",
     "iopub.status.busy": "2025-12-07T04:10:19.512323Z",
     "iopub.status.idle": "2025-12-07T04:10:19.514721Z",
     "shell.execute_reply": "2025-12-07T04:10:19.514427Z"
    },
    "papermill": {
     "duration": 0.004836,
     "end_time": "2025-12-07T04:10:19.515388",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.510552",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Summary by Slot:\n",
      "  description: 100.0%\n",
      "  synonyms: 100.0%\n",
      "  ontology_id: 100.0%\n"
     ]
    }
   ],
   "source": [
    "# Summary by slot shows compliance for each recommended field\n",
    "print(\"Summary by Slot:\")\n",
    "for slot, percentage in report.summary_by_slot.items():\n",
    "    print(f\"  {slot}: {percentage}%\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5883c505",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.518490Z",
     "iopub.status.busy": "2025-12-07T04:10:19.518383Z",
     "iopub.status.idle": "2025-12-07T04:10:19.520289Z",
     "shell.execute_reply": "2025-12-07T04:10:19.520064Z"
    },
    "papermill": {
     "duration": 0.004237,
     "end_time": "2025-12-07T04:10:19.520934",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.516697",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Path Scores:\n",
      "  (root): 100.0%\n",
      "    description: 1/1 (100.0%)\n",
      "    synonyms: 1/1 (100.0%)\n",
      "    ontology_id: 1/1 (100.0%)\n"
     ]
    }
   ],
   "source": [
    "# Path scores show per-object compliance\n",
    "print(\"Path Scores:\")\n",
    "for ps in report.path_scores:\n",
    "    print(f\"  {ps.path}: {ps.overall_percentage}%\")\n",
    "    for ss in ps.slot_scores:\n",
    "        print(f\"    {ss.slot_name}: {ss.populated}/{ss.total} ({ss.percentage}%)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72eb2f2f",
   "metadata": {
    "papermill": {
     "duration": 0.00134,
     "end_time": "2025-12-07T04:10:19.523742",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.522402",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Schema Introspection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b3b055d6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.527126Z",
     "iopub.status.busy": "2025-12-07T04:10:19.526998Z",
     "iopub.status.idle": "2025-12-07T04:10:19.548419Z",
     "shell.execute_reply": "2025-12-07T04:10:19.548065Z"
    },
    "papermill": {
     "duration": 0.02405,
     "end_time": "2025-12-07T04:10:19.549221",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.525171",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recommended slots: {'ontology_id', 'synonyms', 'description'}\n",
      "\n",
      "Disease class recommended: ['description', 'synonyms', 'ontology_id']\n"
     ]
    }
   ],
   "source": [
    "from linkml_data_qc import SchemaIntrospector\n",
    "\n",
    "introspector = SchemaIntrospector(\"/tmp/disease_schema.yaml\")\n",
    "\n",
    "# Get all recommended slots in schema\n",
    "print(f\"Recommended slots: {introspector.recommended_slots}\")\n",
    "\n",
    "# Get class-specific info\n",
    "class_info = introspector.get_class_slots(\"Disease\")\n",
    "print(f\"\\nDisease class recommended: {class_info.recommended_slots}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44cc831c",
   "metadata": {
    "papermill": {
     "duration": 0.001255,
     "end_time": "2025-12-07T04:10:19.551977",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.550722",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Using Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "77abe599",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.555444Z",
     "iopub.status.busy": "2025-12-07T04:10:19.555326Z",
     "iopub.status.idle": "2025-12-07T04:10:19.561123Z",
     "shell.execute_reply": "2025-12-07T04:10:19.560897Z"
    },
    "papermill": {
     "duration": 0.008488,
     "end_time": "2025-12-07T04:10:19.561809",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.553321",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Global Compliance: 100.0%\n",
      "Weighted Compliance: 100.0%\n"
     ]
    }
   ],
   "source": [
    "from linkml_data_qc import QCConfig, SlotQCConfig\n",
    "\n",
    "# Create configuration with weights and thresholds\n",
    "config = QCConfig(\n",
    "    default_weight=1.0,\n",
    "    slots={\n",
    "        \"ontology_id\": SlotQCConfig(weight=2.0, min_compliance=80.0),\n",
    "        \"description\": SlotQCConfig(weight=0.5),\n",
    "    }\n",
    ")\n",
    "\n",
    "# Create analyzer with configuration\n",
    "analyzer = ComplianceAnalyzer(\"/tmp/disease_schema.yaml\", config)\n",
    "report = analyzer.analyze_file(\"/tmp/disease_good.yaml\", \"Disease\")\n",
    "\n",
    "print(f\"Global Compliance: {report.global_compliance}%\")\n",
    "print(f\"Weighted Compliance: {report.weighted_compliance}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26fb8747",
   "metadata": {
    "papermill": {
     "duration": 0.001311,
     "end_time": "2025-12-07T04:10:19.564700",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.563389",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Checking for Violations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "dd684d68",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.571812Z",
     "iopub.status.busy": "2025-12-07T04:10:19.571663Z",
     "iopub.status.idle": "2025-12-07T04:10:19.578285Z",
     "shell.execute_reply": "2025-12-07T04:10:19.578035Z"
    },
    "papermill": {
     "duration": 0.013065,
     "end_time": "2025-12-07T04:10:19.579036",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.565971",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No violations!\n"
     ]
    }
   ],
   "source": [
    "# Analyze poor compliance file with strict threshold\n",
    "config = QCConfig(\n",
    "    slots={\n",
    "        \"description\": SlotQCConfig(min_compliance=50.0),\n",
    "    }\n",
    ")\n",
    "\n",
    "analyzer = ComplianceAnalyzer(\"/tmp/disease_schema.yaml\", config)\n",
    "report = analyzer.analyze_file(\"/tmp/disease_poor.yaml\", \"Disease\")\n",
    "\n",
    "if report.threshold_violations:\n",
    "    print(\"Threshold Violations:\")\n",
    "    for v in report.threshold_violations:\n",
    "        print(f\"  {v.path}.{v.slot_name}: {v.actual_compliance}% < {v.min_required}%\")\n",
    "else:\n",
    "    print(\"No violations!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1841a4a2",
   "metadata": {
    "papermill": {
     "duration": 0.001492,
     "end_time": "2025-12-07T04:10:19.582216",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.580724",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Formatting Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "57effbdc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.585451Z",
     "iopub.status.busy": "2025-12-07T04:10:19.585321Z",
     "iopub.status.idle": "2025-12-07T04:10:19.591303Z",
     "shell.execute_reply": "2025-12-07T04:10:19.591061Z"
    },
    "papermill": {
     "duration": 0.008393,
     "end_time": "2025-12-07T04:10:19.591970",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.583577",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compliance Report: /tmp/disease_good.yaml\n",
      "Target Class: Disease\n",
      "Global Compliance: 100.0% (3/3)\n",
      "Weighted Compliance: 100.0%\n",
      "\n",
      "Summary by Slot:\n",
      "  description: 100.0%\n",
      "  ontology_id: 100.0%\n",
      "  synonyms: 100.0%\n",
      "\n",
      "Detailed Path Scores:\n",
      "  (root) (Disease): 100.0%\n",
      "    - description: OK\n",
      "    - synonyms: OK\n",
      "    - ontology_id: OK\n"
     ]
    }
   ],
   "source": [
    "from linkml_data_qc import JSONFormatter, TextFormatter\n",
    "\n",
    "analyzer = ComplianceAnalyzer(\"/tmp/disease_schema.yaml\")\n",
    "report = analyzer.analyze_file(\"/tmp/disease_good.yaml\", \"Disease\")\n",
    "\n",
    "# Text format\n",
    "print(TextFormatter.format(report))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ad6fb6c9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.595431Z",
     "iopub.status.busy": "2025-12-07T04:10:19.595319Z",
     "iopub.status.idle": "2025-12-07T04:10:19.598400Z",
     "shell.execute_reply": "2025-12-07T04:10:19.598134Z"
    },
    "papermill": {
     "duration": 0.00545,
     "end_time": "2025-12-07T04:10:19.598995",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.593545",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"file_path\": \"/tmp/disease_good.yaml\",\n",
      "  \"target_class\": \"Disease\",\n",
      "  \"schema_path\": \"/tmp/disease_schema.yaml\",\n",
      "  \"global_compliance\": 100.0,\n",
      "  \"weighted_compliance\": 100.0,\n",
      "  \"total_checks\": 3,\n",
      "  \"total_populated\": 3,\n",
      "  \"path_scores\": [\n",
      "    {\n",
      "      \"path\": \"(root)\",\n",
      "      \"parent_class\": \"Disease\",\n",
      "      \"item_count\": 1,\n",
      "      \"slot_scores\": [\n",
      "        {\n",
      "          \"path\": \"(root)\",\n",
      "          \"slot_name\": \"description\",\n",
      "          \"populated\": 1,\n",
      "          \"total\": 1,\n",
      "          \"percentage\": 100.0\n",
      "        },\n",
      "        {\n",
      "          \"path\": \"(root)\",\n",
      "          \"slot_name\": \"synonyms\",\n",
      "          \"populated\": 1,\n",
      "          \"total\": 1,\n",
      "          \"percentage\": 100.0\n",
      "        },\n",
      "        {\n",
      "          \"path\": \"(root)\",\n",
      "          \"slot_name\": \"ontology_id\",\n",
      "          \"populated\": 1,\n",
      "          \"total\": 1,\n",
      "          \"percentage\": 100.0\n",
      "        }\n",
      "      ],\n",
      "      \"overall_percentage\": 100.0\n",
      "    }\n",
      "  ],\n",
      "  \"aggregated_scores\": [],\n",
      "  \"threshold_violations\": [],\n",
      "  \"summary_by_slot\": {\n",
      "    \"description\": 100.0,\n",
      "    \"synonyms\": 100.0,\n",
      "    \"ontology_id\": 100.0\n",
      "  },\n",
      "  \"recommended_slots\": [\n",
      "    \"ontology_id\",\n",
      "    \"synonyms\",\n",
      "    \"description\"\n",
      "  ],\n",
      "  \"config_path\": null,\n",
      "  \"timestamp\": \"2025-12-06T20:10:19.589874\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# JSON format\n",
    "import json\n",
    "json_output = JSONFormatter.format(report)\n",
    "print(json.dumps(json.loads(json_output), indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f2695c0",
   "metadata": {
    "papermill": {
     "duration": 0.001524,
     "end_time": "2025-12-07T04:10:19.602315",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.600791",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Multi-File Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "54b8fd32",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-12-07T04:10:19.605674Z",
     "iopub.status.busy": "2025-12-07T04:10:19.605544Z",
     "iopub.status.idle": "2025-12-07T04:10:19.612417Z",
     "shell.execute_reply": "2025-12-07T04:10:19.612111Z"
    },
    "papermill": {
     "duration": 0.009494,
     "end_time": "2025-12-07T04:10:19.613249",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.603755",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files Analyzed: 3\n",
      "Overall Compliance: 33.33333333333333%\n",
      "\n",
      "Summary by Slot:\n",
      "  description: 33.33333333333333%\n",
      "  synonyms: 33.33333333333333%\n",
      "  ontology_id: 33.33333333333333%\n"
     ]
    }
   ],
   "source": [
    "from linkml_data_qc import analyze_directory, create_multi_file_report\n",
    "\n",
    "# Analyze all matching files in a directory\n",
    "reports = analyze_directory(\n",
    "    schema_path=\"/tmp/disease_schema.yaml\",\n",
    "    data_dir=\"/tmp\",\n",
    "    target_class=\"Disease\",\n",
    "    pattern=\"disease_*.yaml\"\n",
    ")\n",
    "\n",
    "# Create aggregated report\n",
    "multi_report = create_multi_file_report(reports)\n",
    "\n",
    "print(f\"Files Analyzed: {multi_report.files_analyzed}\")\n",
    "print(f\"Overall Compliance: {multi_report.global_compliance}%\")\n",
    "print(\"\\nSummary by Slot:\")\n",
    "for slot, pct in multi_report.summary_by_slot.items():\n",
    "    print(f\"  {slot}: {pct}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed4e6f8c",
   "metadata": {
    "papermill": {
     "duration": 0.001485,
     "end_time": "2025-12-07T04:10:19.616441",
     "exception": false,
     "start_time": "2025-12-07T04:10:19.614956",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## When to Use the CLI vs Python API\n",
    "\n",
    "**Use the CLI when:**\n",
    "- Running one-off compliance checks\n",
    "- Integrating with CI/CD pipelines\n",
    "- Generating reports for external tools\n",
    "\n",
    "**Use the Python API when:**\n",
    "- Building custom analysis pipelines\n",
    "- Integrating with other Python tools\n",
    "- Needing programmatic access to detailed results\n",
    "- Building dashboards or visualizations"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 1.530299,
   "end_time": "2025-12-07T04:10:19.834234",
   "environment_variables": {},
   "exception": null,
   "input_path": "docs/notebooks/02_python_api.ipynb",
   "output_path": "docs/notebooks/02_python_api.ipynb",
   "parameters": {},
   "start_time": "2025-12-07T04:10:18.303935",
   "version": "2.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}