{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Web Service Client Annotators\n",
    "\n",
    "Web service client annotators are annotators which use a web service to annotate documents: for each document that gets processed, data is sent to a HTTP endpoint, processed there and information is sent back that is then used to annotate the document. \n",
    "\n",
    "Currently two client annotators are implemented:\n",
    "\n",
    "* GateCloudAnnotator: this annotator connects to one of the many services on the GATE Cloud platform (https://cloud.gate.ac.uk/). Services include named entity recognition for tweets or standard texts in several  languages, entity disambiguation and linking to Wikipedia or MeSH or Snomed. NOTE: some services require input that is not just plain text, at the moment only those services are supported which can annotate plain text\n",
    "* TagMeAnnotator: this annotator connects to the TagMe mention disambiguation and linking service (https://sobigdata.d4science.org/group/tagme/tagme) to either perform the task \"tag\" (disambiguation and linking of mentions) or \"spot\" finding mentions only. \n",
    "* TextRazorTextAnnotator: this annotator connects to the TextRazor API endpoint (see https://www.textrazor.com/) to annotate the text of the document\n",
    "* ElgTextAnnotator: this annotator connects to one of the public endpoints from the European Language Grid project (see https://live.european-language-grid.eu) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gatenlp import Document\n",
    "from gatenlp.processing.client import GateCloudAnnotator\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets try annotating a document with the English Named Entity Recognizer on GATE\n",
    "cloud (https://cloud.gate.ac.uk/shopfront/displayItem/annie-named-entity-recognizer). \n",
    "\n",
    "The information page for that service shows that the following annotation types can be requested of which the first 5 are requested by default if no alternate list is specified:\n",
    "\n",
    "* Address  (included by default)\n",
    "* Date (included by default)\n",
    "* Location (included by default)\n",
    "* Organization (included by default)\n",
    "* Person (included by default)\n",
    "* Money \n",
    "* Percent \n",
    "* Token \n",
    "* SpaceToken \n",
    "* Sentence\n",
    "\n",
    "We create a GateCloudAnnotator an specify the full list of all supported annotation types. We also specify the URL of the service endpoint as provided on the info page and specify that the annotations should be put into the annotation set \"ANNIE\". Note that a limited number of documents can be annotated for free and without authentication, so we do not need to specify the `api_key` and `api_password` parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "annotator = GateCloudAnnotator(\n",
    "    url=\"https://cloud-api.gate.ac.uk/process-document/annie-named-entity-recognizer\", \n",
    "    out_annset=\"ANNIE\", \n",
    "    ann_types=\":Address,:Date,:Location,:Organization,:Person,:Money,:Percent,:Token,:SpaceToken,:Sentence\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# an example document to annotate\n",
    "doc = Document(\"Barack Obama visited Microsoft in New York last May.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><style>#MOUFKYLXSS-wrapper { color: black !important; }</style>\n",
       "<div id=\"MOUFKYLXSS-wrapper\">\n",
       "\n",
       "<div>\n",
       "<style>\n",
       "#MOUFKYLXSS-content {\n",
       "    width: 100%;\n",
       "    height: 100%;\n",
       "    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-row {\n",
       "    width: 100%;\n",
       "    display: flex;\n",
       "    flex-direction: row;\n",
       "    flex-wrap: nowrap;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-col {\n",
       "    border: 1px solid grey;\n",
       "    display: inline-block;\n",
       "    min-width: 200px;\n",
       "    padding: 5px;\n",
       "    /* white-space: normal; */\n",
       "    /* white-space: pre-wrap; */\n",
       "    overflow-y: auto;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-hdr {\n",
       "    font-size: 1.2rem;\n",
       "    font-weight: bold;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-label {\n",
       "    margin-bottom: -15px;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-input {\n",
       "    vertical-align: middle;\n",
       "    position: relative;\n",
       "    *overflow: hidden;\n",
       "}\n",
       "\n",
       "#MOUFKYLXSS-popup {\n",
       "    display: none;\n",
       "    color: black;\n",
       "    position: absolute;\n",
       "    margin-top: 10%;\n",
       "    margin-left: 10%;\n",
       "    background: #aaaaaa;\n",
       "    width: 60%;\n",
       "    height: 60%;\n",
       "    z-index: 50;\n",
       "    padding: 25px 25px 25px;\n",
       "    border: 1px solid black;\n",
       "    overflow: auto;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-selection {\n",
       "    margin-bottom: 5px;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-featuretable {\n",
       "    margin-top: 10px;\n",
       "}\n",
       "\n",
       ".MOUFKYLXSS-fname {\n",
       "    text-align: left !important;\n",
       "    font-weight: bold;\n",
       "    margin-right: 10px;\n",
       "}\n",
       ".MOUFKYLXSS-fvalue {\n",
       "    text-align: left !important;\n",
       "}\n",
       "</style>\n",
       "  <div id=\"MOUFKYLXSS-content\">\n",
       "        <div id=\"MOUFKYLXSS-popup\" style=\"display: none;\">\n",
       "        </div>\n",
       "        <div class=\"MOUFKYLXSS-row\" id=\"MOUFKYLXSS-row1\" style=\"height:67vh; min-height:100px;\">\n",
       "            <div id=\"MOUFKYLXSS-text-wrapper\" class=\"MOUFKYLXSS-col\" style=\"width:70%;\">\n",
       "                <div class=\"MOUFKYLXSS-hdr\" id=\"MOUFKYLXSS-dochdr\"></div>\n",
       "                <div id=\"MOUFKYLXSS-text\">\n",
       "                </div>\n",
       "            </div>\n",
       "            <div id=\"MOUFKYLXSS-chooser\" class=\"MOUFKYLXSS-col\" style=\"width:30%; border-left-width: 0px;\"></div>\n",
       "        </div>\n",
       "        <div class=\"MOUFKYLXSS-row\" id=\"MOUFKYLXSS-row2\" style=\"height:30vh; min-height: 100px;\">\n",
       "            <div id=\"MOUFKYLXSS-details\" class=\"MOUFKYLXSS-col\" style=\"width:100%; border-top-width: 0px;\">\n",
       "            </div>\n",
       "        </div>\n",
       "    </div>\n",
       "    <script src=\"https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js\"></script><script src=\"https://unpkg.com/gatenlp-ann-viewer@1.0.11/gatenlp-ann-viewer.js\"></script>\n",
       "    <script type=\"application/json\" id=\"MOUFKYLXSS-data\">\n",
       "    {\"annotation_sets\": {\"ANNIE\": {\"name\": \"detached-from:ANNIE\", \"annotations\": [{\"type\": \"Date\", \"start\": 43, \"end\": 51, \"id\": 0, \"features\": {\"rule\": \"ModifierNamedDate\", \"ruleFinal\": \"DateOnlyFinal\", \"kind\": \"date\"}}, {\"type\": \"Location\", \"start\": 34, \"end\": 42, \"id\": 1, \"features\": {\"kind\": \"locName\", \"rule\": \"InLoc1\", \"locType\": \"city\", \"ruleFinal\": \"LocFinal\"}}, {\"type\": \"Organization\", \"start\": 21, \"end\": 30, \"id\": 2, \"features\": {\"orgType\": \"company\", \"rule\": \"GazOrganization\", \"ruleFinal\": \"OrgFinal\"}}, {\"type\": \"Person\", \"start\": 0, \"end\": 12, \"id\": 3, \"features\": {\"firstName\": \"Barack\", \"surname\": \"Obama\", \"kind\": \"fullName\", \"rule\": \"GazPerson\", \"gender\": \"male\", \"ruleFinal\": \"PersonFinal\"}}, {\"type\": \"Token\", \"start\": 0, \"end\": 6, \"id\": 4, \"features\": {\"string\": \"Barack\", \"length\": \"6\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 7, \"end\": 12, \"id\": 5, \"features\": {\"string\": \"Obama\", \"length\": \"5\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 13, \"end\": 20, \"id\": 6, \"features\": {\"string\": \"visited\", \"length\": \"7\", \"orth\": \"lowercase\", \"kind\": \"word\", \"category\": \"VBD\"}}, {\"type\": \"Token\", \"start\": 21, \"end\": 30, \"id\": 7, \"features\": {\"string\": \"Microsoft\", \"length\": \"9\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 31, \"end\": 33, \"id\": 8, \"features\": {\"string\": \"in\", \"length\": \"2\", \"orth\": \"lowercase\", \"kind\": \"word\", \"category\": \"IN\"}}, {\"type\": \"Token\", \"start\": 34, \"end\": 37, \"id\": 9, \"features\": {\"string\": \"New\", \"length\": \"3\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 38, \"end\": 42, \"id\": 10, \"features\": {\"string\": \"York\", \"length\": \"4\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 43, \"end\": 47, \"id\": 11, \"features\": {\"string\": \"last\", \"length\": \"4\", \"orth\": \"lowercase\", \"kind\": \"word\", \"category\": \"JJ\"}}, {\"type\": \"Token\", \"start\": 48, \"end\": 51, \"id\": 12, \"features\": {\"string\": \"May\", \"length\": \"3\", \"orth\": \"upperInitial\", \"kind\": \"word\", \"category\": \"NNP\"}}, {\"type\": \"Token\", \"start\": 51, \"end\": 52, \"id\": 13, \"features\": {\"string\": \".\", \"length\": \"1\", \"kind\": \"punctuation\", \"category\": \".\"}}, {\"type\": \"SpaceToken\", \"start\": 6, \"end\": 7, \"id\": 14, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 12, \"end\": 13, \"id\": 15, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 20, \"end\": 21, \"id\": 16, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 30, \"end\": 31, \"id\": 17, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 33, \"end\": 34, \"id\": 18, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 37, \"end\": 38, \"id\": 19, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 42, \"end\": 43, \"id\": 20, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"SpaceToken\", \"start\": 47, \"end\": 48, \"id\": 21, \"features\": {\"string\": \" \", \"length\": \"1\", \"kind\": \"space\"}}, {\"type\": \"Sentence\", \"start\": 0, \"end\": 52, \"id\": 22, \"features\": {}}], \"next_annid\": 23}}, \"text\": \"Barack Obama visited Microsoft in New York last May.\", \"features\": {}, \"offset_type\": \"j\", \"name\": \"\"}\n",
       "    </script>\n",
       "    <script type=\"text/javascript\">\n",
       "        gatenlp_run(\"MOUFKYLXSS-\");\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "</div></div>"
      ],
      "text/plain": [
       "Document(Barack Obama visited Microsoft in New York last May.,features=Features({}),anns={'ANNIE': AnnotationSet([Annotation(0,12,Person,features=Features({'firstName': 'Barack', 'surname': 'Obama', 'kind': 'fullName', 'rule': 'GazPerson', 'gender': 'male', 'ruleFinal': 'PersonFinal'}),id=3), Annotation(0,6,Token,features=Features({'string': 'Barack', 'length': '6', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=4), Annotation(0,52,Sentence,features=Features({}),id=22), Annotation(6,7,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=14), Annotation(7,12,Token,features=Features({'string': 'Obama', 'length': '5', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=5), Annotation(12,13,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=15), Annotation(13,20,Token,features=Features({'string': 'visited', 'length': '7', 'orth': 'lowercase', 'kind': 'word', 'category': 'VBD'}),id=6), Annotation(20,21,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=16), Annotation(21,30,Organization,features=Features({'orgType': 'company', 'rule': 'GazOrganization', 'ruleFinal': 'OrgFinal'}),id=2), Annotation(21,30,Token,features=Features({'string': 'Microsoft', 'length': '9', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=7), Annotation(30,31,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=17), Annotation(31,33,Token,features=Features({'string': 'in', 'length': '2', 'orth': 'lowercase', 'kind': 'word', 'category': 'IN'}),id=8), Annotation(33,34,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=18), Annotation(34,42,Location,features=Features({'kind': 'locName', 'rule': 'InLoc1', 'locType': 'city', 'ruleFinal': 'LocFinal'}),id=1), Annotation(34,37,Token,features=Features({'string': 'New', 'length': '3', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=9), Annotation(37,38,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=19), Annotation(38,42,Token,features=Features({'string': 'York', 'length': '4', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=10), Annotation(42,43,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=20), Annotation(43,51,Date,features=Features({'rule': 'ModifierNamedDate', 'ruleFinal': 'DateOnlyFinal', 'kind': 'date'}),id=0), Annotation(43,47,Token,features=Features({'string': 'last', 'length': '4', 'orth': 'lowercase', 'kind': 'word', 'category': 'JJ'}),id=11), Annotation(47,48,SpaceToken,features=Features({'string': ' ', 'length': '1', 'kind': 'space'}),id=21), Annotation(48,51,Token,features=Features({'string': 'May', 'length': '3', 'orth': 'upperInitial', 'kind': 'word', 'category': 'NNP'}),id=12), Annotation(51,52,Token,features=Features({'string': '.', 'length': '1', 'kind': 'punctuation', 'category': '.'}),id=13)])})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Run the annotator and show the annotated document\n",
    "doc = annotator(doc)\n",
    "doc"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python gatenlp",
   "language": "python",
   "name": "gatenlp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
