{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **`AITutor-AssessmentKit:` An Open-Source Library to Measure Pedagogical Ability of AI Tutors in Educational Dialogues**\n",
    "\n",
    "---\n",
    "\n",
    "The `AITutor-AssessmentKit` is the *first* open-source library to enable the pedagogical abilities assessment of large language model (LLM)-powered AI tutors in educational dialogues. This unified framework:\n",
    "\n",
    "- **Evaluates AI tutor responses** across eight comprehensive dimensions in the context of student error remediation tasks in mathematics.\n",
    "- **Offers a pluggable and customizable interface** for integrating models and LLM releases from the community.\n",
    "\n",
    "By providing an efficient, scalable alternative to costly and subjective human evaluations, `AITutor-AssessmentKit` facilitates *on-the-fly* assessment of AI tutors.\n",
    "\n",
    "---\n",
    "\n",
    "## 📥 **Installation**\n",
    "\n",
    "To install `AITutor-AssessmentKit` with `pip`, type:\n",
    "\n",
    "```bash\n",
    "pip install aitutor-assessmentkit\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## 📚 **Overview of the `AITutor-AssessmentKit`**\n",
    "\n",
    "The library comprises three modular components:\n",
    "\n",
    "1. **`autoeval`**: For automated evaluation.\n",
    "2. **`llmeval`**: For LLM-based evaluation.\n",
    "3. **`visualizer`**: For visualization and interpretation of evaluation scores.\n",
    "\n",
    "![](/home/kaushal.maurya/AITutor_AssessmentKit/outputs/AITutor-assessmentKit-Main_vf3.png)\n",
    "\n",
    "---\n",
    "\n",
    "## 📖 **Tutorials**\n",
    "\n",
    "We provide several resources to help you get started:\n",
    "\n",
    "1. [Tutorial Notebook-I: Automated Evaluation with AITutor-AssessmentKit](#)\n",
    "2. [Tutorial Notebook-II: LLM-Based Evaluation with AITutor-AssessmentKit](#)\n",
    "3. [Tutorial Notebook-III: Visualizing AI Tutor Performance with AITutor-AssessmentKit](#)\n",
    "\n",
    "For a quick overview, check out our [Demo Notebook](#).\n",
    "\n",
    "---\n",
    "\n",
    "## 🚀 **Next Steps**\n",
    "\n",
    "Here are the upcoming milestones for the project:\n",
    "\n",
    "1. Create and release detailed documentation.\n",
    "2. Publish a longer tutorial video.\n",
    "3. Develop a GUI-friendly interaction mode.\n",
    "\n",
    "---\n",
    "\n",
    "## 📜 **Citation**\n",
    "\n",
    "If you use `AITutor-AssessmentKit` in your research, please cite us:\n",
    "\n",
    "```bibtex\n",
    "@inproceedings{maurya2024aitutorassessmentkit,\n",
    "  title={AITutor-AssessmentKit: An Open-Source Library to Measure Pedagogical Ability of AI Tutors in Educational Dialogues},\n",
    "  author={Kaushal Kumar Maurya and Ekaterina Kochmar},\n",
    "  year={2024}\n",
    "}\n",
    "```\n",
    "\n",
    "---\n",
    "\n",
    "## 📧 **Contact**\n",
    "\n",
    "For any questions or support, feel free to reach out:\n",
    "\n",
    "📧 **Kaushal Kumar Maurya**: Kaushal.Maurya@mbzuai.ac.ae\n",
    "\n",
    "---\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pds_py310",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
