{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM for alle - Introduksjonskurs til språkmodeller med Python og Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduksjon\n",
    "\n",
    "Etter prosjektet vi gjorde for Elkem fikk vi hands-on erfaring med å utvikle og bruke språkmodeller (LLMer) i praksis. Vi så hvor effektivt dette kan være for å spare tid, redusere kostnader og automatisere manuelle oppgaver – på en måte som enkelt kan gjenbrukes i andre prosjekter.\n",
    "\n",
    "I dette kurset bruker vi et konkret og gjenkjennbart case: analyse av resultatene fra en medarbeiderundersøkelse – en oppgave mange sliter med å strukturere. Vi skal lære dere å koble dere til en LLM via API, bruke LangChain Expression Language (LCEL) til å bygge AI-kjeder og hente ut strukturert innsikt med Pydantic – alt i én oversiktlig notebook.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Oppgave\n",
    "Du jobber i et IT-selskap og har fått i oppgave å analysere svarene fra en intern medarbeiderundersøkelse. Undersøkelsen er anonym, og du har fått tilsendt en CSV-fil med 50 tilbakemeldinger – én per ansatt. Målet er å finne ut hva folk er fornøyde eller misfornøyde med, og særlig se nærmere på temaene Nettverk, Opplæring og IT-support, som ledelsen er ekstra interessert i. Tilbakemeldinger som ikke passer i disse kategoriene skal også få sin plass. Til slutt skal du lage en oppsummering som kan sendes til ledelsen.\n",
    "\n",
    "For å jobbe effektivt bruker du en språkmodell til å hjelpe deg med både kategorisering og oppsummering. \n",
    "\n",
    "**Oppgaven blir dermed å bruke en språkmodell til å kategorisere samt oppsummere tilbakemeldingene fra undersøkelsen.** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasett"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Feedback\\t</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.\\t</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.\\t</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Security</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.\\t</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.\\t</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.\\t</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ID          Category  \\\n",
       "0   1           Network   \n",
       "1   2    Business needs   \n",
       "2   3          Security   \n",
       "3   4  Quality of tools   \n",
       "4   5        IT-support   \n",
       "\n",
       "                                                                                                                                 Feedback\\t  \n",
       "0     The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.\\t  \n",
       "1         Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.\\t  \n",
       "2       I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.\\t  \n",
       "3   Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.\\t  \n",
       "4  The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.\\t  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Importerer rådataen med tilbakemeldinger, en rad per ansatt. Alle ansatte har svart på undersøkelsen. \n",
    "\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_colwidth', None) # Ensure no truncated output of dataframe\n",
    "\n",
    "enr_path = \"../files/IT_survey.csv\"\n",
    "df = pd.read_csv(enr_path)\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bruk av språkmodeller gjennom en API (Using a language model through the API)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### En vanlig LLM-spørring\n",
    "Enkle LLM-spørringer er bygget opp av noen sentrale deler:\n",
    "\n",
    "1. Tilkobling til en API, som feks. Azure OpenAI\n",
    "\n",
    "2. En prompt, som vil si en tekstbasert forespørsel/instruks\n",
    "\n",
    "3. Sending av prompt til språkmodellen for å hente en respons\n",
    "\n",
    "\n",
    "Som en del av tilkoblingen er det vanlig å oppgi en temperaturparameter. Denne parametreren angir nivået av presisjon du ønsker å få i responsen fra språkmodellen, og kan enten måles på en skala fra \"low\" til \"high\" eller numerisk fra 0 til 1.\n",
    "Hvis denne parameteren settes til \"low\"/nærme 0 tillater du liten grad av variasjon og kreativitet i responsen, og du vil få tryggere og mer forutsigbare svar. Hvis den derimot settes til høy/nærme 1 tillater du større grad av kreativitet og detaljer, men vil følgelig også få en mer uforutsigbar respons. \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "///\n",
    "TODO: Intro til språkmodeller\n",
    "- Slik kaller man en språkmodell\n",
    "- Hva er temperaturparameteren"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/IPython/core/interactiveshell.py:3377: UserWarning: WARNING! reasoning_effort is not default parameter.\n",
      "                reasoning_effort was transferred to model_kwargs.\n",
      "                Please confirm that reasoning_effort is what you intended.\n",
      "  if (await self.run_code(code, result,  async_=asy)):\n"
     ]
    }
   ],
   "source": [
    "# Connect through the API\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from dotenv import find_dotenv, load_dotenv\n",
    "import os\n",
    "from IPython.display import Markdown # Pretty output\n",
    "\n",
    "\n",
    "# Get environment variables\n",
    "load_dotenv(find_dotenv(), override=True)\n",
    "\n",
    "\n",
    "llm = AzureChatOpenAI(\n",
    "    azure_deployment=\"gpt-4o-mini\",\n",
    "    model=os.environ.get(\"OPENAI_MODEL_GPT_4O-MINI\", default=\"gpt-4o-mini\"),\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "reasoning_llm = AzureChatOpenAI(\n",
    "    azure_deployment=\"o3-mini\",\n",
    "    model=\"o3-mini\",\n",
    "    reasoning_effort=\"medium\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "AuthenticationError",
     "evalue": "Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}}",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAuthenticationError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_1007/868937501.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;31m# Send the prompt and recieve a response\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mllm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minvoke\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;31m# Show the response from the model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py\u001b[0m in \u001b[0;36minvoke\u001b[0;34m(self, input, config, stop, **kwargs)\u001b[0m\n\u001b[1;32m    284\u001b[0m         return cast(\n\u001b[1;32m    285\u001b[0m             \u001b[0mChatGeneration\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 286\u001b[0;31m             self.generate_prompt(\n\u001b[0m\u001b[1;32m    287\u001b[0m                 \u001b[0;34m[\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_convert_input\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    288\u001b[0m                 \u001b[0mstop\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstop\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py\u001b[0m in \u001b[0;36mgenerate_prompt\u001b[0;34m(self, prompts, stop, callbacks, **kwargs)\u001b[0m\n\u001b[1;32m    784\u001b[0m     ) -> LLMResult:\n\u001b[1;32m    785\u001b[0m         \u001b[0mprompt_messages\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_messages\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mprompts\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 786\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt_messages\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstop\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcallbacks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    787\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    788\u001b[0m     async def agenerate_prompt(\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)\u001b[0m\n\u001b[1;32m    641\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mrun_managers\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    642\u001b[0m                     \u001b[0mrun_managers\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_llm_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mLLMResult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgenerations\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 643\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    644\u001b[0m         flattened_outputs = [\n\u001b[1;32m    645\u001b[0m             \u001b[0mLLMResult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgenerations\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mres\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerations\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mllm_output\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mres\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mllm_output\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[list-item]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)\u001b[0m\n\u001b[1;32m    631\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    632\u001b[0m                 results.append(\n\u001b[0;32m--> 633\u001b[0;31m                     self._generate_with_cache(\n\u001b[0m\u001b[1;32m    634\u001b[0m                         \u001b[0mm\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    635\u001b[0m                         \u001b[0mstop\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstop\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py\u001b[0m in \u001b[0;36m_generate_with_cache\u001b[0;34m(self, messages, stop, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    849\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    850\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0minspect\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msignature\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_generate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparameters\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"run_manager\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 851\u001b[0;31m                 result = self._generate(\n\u001b[0m\u001b[1;32m    852\u001b[0m                     \u001b[0mmessages\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstop\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrun_manager\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    853\u001b[0m                 )\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/langchain_openai/chat_models/base.py\u001b[0m in \u001b[0;36m_generate\u001b[0;34m(self, messages, stop, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    703\u001b[0m             \u001b[0mgeneration_info\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m\"headers\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mdict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mraw_response\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    704\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 705\u001b[0;31m             \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcreate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mpayload\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    706\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_create_chat_result\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgeneration_info\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    707\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/openai/_utils/_utils.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    272\u001b[0m                         \u001b[0mmsg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34mf\"Missing required argument: {quote(missing[0])}\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    273\u001b[0m                 \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 274\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    275\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    276\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mwrapper\u001b[0m  \u001b[0;31m# type: ignore\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/openai/resources/chat/completions.py\u001b[0m in \u001b[0;36mcreate\u001b[0;34m(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, presence_penalty, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)\u001b[0m\n\u001b[1;32m    813\u001b[0m     ) -> ChatCompletion | Stream[ChatCompletionChunk]:\n\u001b[1;32m    814\u001b[0m         \u001b[0mvalidate_response_format\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresponse_format\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 815\u001b[0;31m         return self._post(\n\u001b[0m\u001b[1;32m    816\u001b[0m             \u001b[0;34m\"/chat/completions\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    817\u001b[0m             body=maybe_transform(\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/openai/_base_client.py\u001b[0m in \u001b[0;36mpost\u001b[0;34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1275\u001b[0m             \u001b[0mmethod\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"post\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mjson_data\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbody\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfiles\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mto_httpx_files\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfiles\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1276\u001b[0m         )\n\u001b[0;32m-> 1277\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mcast\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mResponseT\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcast_to\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mopts\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstream\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream_cls\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstream_cls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1278\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1279\u001b[0m     def patch(\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/openai/_base_client.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m    952\u001b[0m             \u001b[0mretries_taken\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    953\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 954\u001b[0;31m         return self._request(\n\u001b[0m\u001b[1;32m    955\u001b[0m             \u001b[0mcast_to\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcast_to\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    956\u001b[0m             \u001b[0moptions\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/openai/_base_client.py\u001b[0m in \u001b[0;36m_request\u001b[0;34m(self, cast_to, options, retries_taken, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1056\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1057\u001b[0m             \u001b[0mlog\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdebug\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Re-raising status error\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1058\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_status_error_from_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1059\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1060\u001b[0m         return self._process_response(\n",
      "\u001b[0;31mAuthenticationError\u001b[0m: Error code: 401 - {'error': {'code': '401', 'message': 'Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.'}}"
     ]
    }
   ],
   "source": [
    "# Generate the prompt\n",
    "prompt = 'Hei!'\n",
    "\n",
    "# Send the prompt and recieve a response\n",
    "response = llm.invoke(prompt)\n",
    "\n",
    "# Show the response from the model\n",
    "response\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hei! Hvordan kan jeg hjelpe deg i dag?'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show only the content of the response\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hvordan lage en enkel kjede med LangChain Expression Language (LCEL) // Creating a basic chain with Langchain Expression Language (LCEL)\n",
    "\n",
    "LCEL er en metode for å bygge og kjøre såkalte *kjeder* i LangChain på. Kjeder, eller chains, brukes for å koble sammen ulike AI-komponenter. F.eks. kan språkmodeller, datakilder og logikk kobles sammen til én sammenhengende prosess, dvs. en kjede. LCEL gir et standardisert språk for å definere disse kjedene, og er brukervennlig fordi man slipper å lage alt manuelt med kode. Med andre ord får du en \"oppskrift\" på hvordan AI-komponentene dine skal jobbe sammen på en rask og skalerbar måte.\n",
    "\n",
    "**Fordelene med LCEL**\n",
    "1. Støtter parallell og asynkron kjøring - Ulike deler av kjeden kan kjøre samtidig, og systemet kan behandle flere forespørsler på en gang. Dermed kan oppgaver behandles raskere. \n",
    "2. Strømming av resultater - Man kan begynne å se svar mens AI-en fremdeles jobber. (Passer spesielt godt for chatbaserte løsninger)\n",
    "3. Enkel feilsøking med LangSmith - Når kjedene blir komplekse er det viktig å kunne se hva som har blitt gjort underveis. LCEL logger automatisk alt til LangSmith, som gjør det enklere å feilsøke. \n",
    "4. Standardisert - Alle kjeder i LCEL bruker samme grensesnitt, som gjør dem enkle å kombinere og gjenbruke på tvers av prosjekter. \n",
    "\n",
    "LCEL bruker en pipe-operator (|) til å koble sammen ulike trinn i kjeden. Den tar ut data fra én komponent og sender den direkte som input til neste komponent. LCEL bruker også PromptTemplate, som kan tenkes på som en mal for teksten du sender til språkmodellen. PromptTemplate gjør det enkelt å lage dynamiske meldinger ved at man kan sette inn variabler i teksten, litt som en oppskrift der du fyller inn det som mangler før det sendes til AI-modellen. Fordelen med PromptTemplate er at man kan lage én mal, og bruke den med ulike data. Det hjelper deg også med å skille selve teksten fra logikken, og kan gjøre prosessen sikrere ved at man unngår feil som kan oppstå ved manuell string-manipulasjon. Vi skal nå se på noen eksempler med LCEL som bruker pipe-operator og PromptTemplate.\n",
    "\n",
    "Kilde: xxx\n",
    "\n",
    "//\n",
    "TODO:\n",
    "- Fordelene med LCEL\n",
    "- Forklare pipe-operatoren\n",
    "- Forklare PromptTemplate\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PromptTemplate(input_variables=['role'], input_types={}, partial_variables={}, template='\\n    Hi! Please talk like a {role}.\\n    ')"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "prompt = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Hi! Please talk like a {role}.\n",
    "    \"\"\"\n",
    ")\n",
    "prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"Ahoy, matey! What be ye wantin' to chat about on this fine day upon the high seas? Be it treasure maps, fearsome sea monsters, or the finest rum in the seven seas? Speak up, or I’ll be sendin’ ye to Davy Jones’ locker! Arrr! 🏴\\u200d☠️\", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 72, 'prompt_tokens': 18, 'total_tokens': 90, 'completion_tokens_details': {'audio_tokens': 0, 'reasoning_tokens': 0, 'accepted_prediction_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_7a53abb7a2', 'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'jailbreak': {'filtered': False, 'detected': False}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}, id='run-126011eb-1a28-4c6b-a9c2-9ea6c8cf401b-0', usage_metadata={'input_tokens': 18, 'output_tokens': 72, 'total_tokens': 90, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain = prompt | llm\n",
    "\n",
    "chain.invoke({\"role\": \"pirate\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Ahoy, matey! What be ye wantin' to chat about on this fine day? Be it treasure maps, sea shanties, or tales of the high seas? Speak up, and let’s set sail on a grand adventure! Arrr! 🏴\\u200d☠️\""
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "chain2 = chain | StrOutputParser()\n",
    "\n",
    "chain2.invoke({\"role\": \"pirate\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batching and streaming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"Ahoy, matey! What be ye wantin' to chat about on this fine day upon the high seas? Arrr!\",\n",
       " \"Well howdy there, partner! What brings ya 'round these parts? If yer lookin' for a good ol' yarn or some advice on ridin' the range, I’m all ears. Just remember, life’s a wild ride, so keep yer hat on tight and yer boots polished! What’s on yer mind, friend?\",\n",
       " 'Greetings, silent shadow. The night whispers secrets, and the wind carries the tales of the unseen. What knowledge do you seek, traveler of the hidden path? Stealth and wisdom await. Speak, and I shall share the way of the ninja. 🥷✨']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain2.batch(\n",
    "    [\n",
    "        {\"role\": \"pirate\"},\n",
    "        {\"role\": \"cowboy\"},\n",
    "        {\"role\": \"ninja\"},\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ahoy, matey! What be ye wantin' to chat about on this fine day upon the high seas? Arrr!"
     ]
    }
   ],
   "source": [
    "from time import sleep\n",
    "for chunk in chain2.stream({\"role\": \"pirate\"}):\n",
    "    print(chunk, end=\"\")\n",
    "    sleep(0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enkel kategorisering av hver enkelt tilbakemelding // Simple categorization of each survey reply"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bruk LLMen til kategorisering av tilbakemeldingene\n",
    "I første omgang av kategoriseringen er vi interesserte i å se hvor mange av tilbakemeldingene som passer innenfor de kategoriene ledelsen ønsket et ekstra fokus på, nemlig Network, Training og IT-support. Vi kan be LLMen om å utføre denne kategoriseringen ved å gi den tilgang på feedback-dataen. Husk på de sentrale delene av enkle LLM-kall, og benytt deg av LCEL som vist i forrige eksempel. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Category: IT-support'"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorize_prompt = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into one of the following categories:\n",
    "- Network\n",
    "- Training\n",
    "- IT-support\n",
    "- Other\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "categorize_chain = categorize_prompt | llm | StrOutputParser()\n",
    "\n",
    "categorize_chain.invoke({\"feedback\": \"I am very happy with the IT support I received last week.\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structured output\n",
    "\n",
    "TODO\n",
    "- Hvorfor trenger man strukturert output\n",
    "- Forklare hva Pydantic er\n",
    "- FOrklare hvordan Pydantic brukes for å få strukturert output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pydantic \n",
    "Pydantic er et Python-bibliotek for datavalidering og datastrukturering. Hovedklassen i Pydantic heter BaseModel og er klassen vi arver fra når vi lager våre egne datamodeller. Når vi arver fra BaseModel får vi automatisk funksjonalitet som kan:\n",
    "1. Validere innhold du sender inn\n",
    "    - Eks: Du definerer en liste med godkjente land, \"Norge, Sverige, Finland\", da vil ikke pydantic godkjenne \"Australia\".\n",
    "2. Konvertere data til riktig type\n",
    "    - Eks: Du definerer at output skal være en int og sender inn '1', pydantic vil da returnere 1 (som int)\n",
    "3. Påtvinge JSON-formatering\n",
    "    - Sørger for at responsen LLMen gir matcher JSOM-skjemaet til Pydantic. Dette gjør at vi kan være sikre på strukturen til outputen, som for eksempel er svært nyttig om vi ønsker å bruke outputen fra en modell som input i en annen. \n",
    "\n",
    "\n",
    "\n",
    "### Structured output \n",
    "I denne konteksten refererer structured output til strategien og verktøyene vi bruker for å forsikre oss om at dataen vår blir organisert på en måte vi definerer på forhånd. \n",
    "Funksjonalitet i Pydantic biblioteket lar oss bestemme strukturen på outputen gjennom å spesifisere felter, definere typer (eks. int, str, List[int]) og validere data.\n",
    "I dette kurset kommer vi til å bruke BaseModel til å lage våre egne klasser for å sikre at det vi mottar fra modellene når vi prompter de komme rpå akkurat den formen vi ønsker. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "\n",
    "class Categorize(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category : str = Field(description=\"The best fitting category. Only one.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I eksempelet over har vi laget vår egen klasse 'Categorize' som arver av BaseModel.\n",
    "For variabelen category har vi brukt type hinting ( x: type ) for å definere typen Pydantic skal forvente at category er lik. Dette for ekesempel blir da ulovlig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValidationError",
     "evalue": "1 validation error for Categorize\ncategory\n  Input should be a valid string [type=string_type, input_value=1, input_type=int]\n    For further information visit https://errors.pydantic.dev/2.9/v/string_type",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValidationError\u001b[0m                           Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_3761/2542644843.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mCategorize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcategory\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/.local/lib/python3.10/site-packages/pydantic/main.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, **data)\u001b[0m\n\u001b[1;32m    210\u001b[0m         \u001b[0;31m# `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    211\u001b[0m         \u001b[0m__tracebackhide__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 212\u001b[0;31m         \u001b[0mvalidated_self\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__pydantic_validator__\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalidate_python\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself_instance\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    213\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mvalidated_self\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    214\u001b[0m             warnings.warn(\n",
      "\u001b[0;31mValidationError\u001b[0m: 1 validation error for Categorize\ncategory\n  Input should be a valid string [type=string_type, input_value=1, input_type=int]\n    For further information visit https://errors.pydantic.dev/2.9/v/string_type"
     ]
    }
   ],
   "source": [
    "Categorize(category=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mens dette er helt ok"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Categorize(category='Network')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Categorize(category=\"Network\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vi bruker også funksjonen Field, denne kan du bruke til å sette standardverdier, valideringsregler og beskrivelser.\n",
    "I dette kurset bruker vi kun beskrivelse, men for de spesielt interesserte kan dere lese mer om funksjonaliteten [her](https://docs.pydantic.dev/latest/concepts/fields/).\n",
    "\n",
    "For å få LLMen til å skjønne at den må følge reglene vi har definert i klassen vår bruker vi wrapperen with_structured_output(...). \n",
    "Denne wrapper kallet vårt til språkmodellen med logikk som forsikrer at outputen følger strukturen vi har definert i klassen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Categorize(category='IT-support')"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorize_chain_structured_output = categorize_prompt | llm.with_structured_output(\n",
    "    Categorize,\n",
    "    method=\"json_schema\", # Påtvinger JSON-skjema for output\n",
    "    strict=True           # Modellen må følge skjema etter punkt og prikke, ingen ekstra felter, ingen manglende felter og alle typer må være en eksakt match.\n",
    ")\n",
    "\n",
    "categorize_chain_structured_output.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vi kan bruke Literal for å definere våre egne typer til å bruke for type hinting. Vi kan type hinte variabler med CATEGORIES under og dette vil da modellen toke på samme måte som at en int bare har lov til å være et heltall har denne \"typen\" bare lov til å være en av verdiene listet opp i Literal-objektet.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "CATEGORIES = Literal[\n",
    "    \"Network\",\n",
    "    \"Training\",\n",
    "    \"IT-support\",\n",
    "    'Other'\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La oss sette sammen det vi har lært. Legg merke til at vi ikke lenger lister kategoriene i prompten (som på ingen måte garanterer at vi kun får ut en av kategorierne vi øsnker), men definerer de som et krav til strukturen på outputen fra modellen. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "categorize_prompt2 = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into the provided categories.\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "\n",
    "\n",
    "class CategorizeFromOptions(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  ## noqa: E501\n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_structured_output2 = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeFromOptions,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CategorizeFromOptions(category='IT-support')"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = categorize_chain_structured_output2.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")\n",
    "\n",
    "result\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate performance across whole dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def categorize_single_feedback(feedback: str) -> str:\n",
    "    result = categorize_chain_structured_output2.invoke(\n",
    "        {\"feedback\": feedback}\n",
    "    )\n",
    "    return result.category\n",
    "\n",
    "df[\"AI Classification\"] = df[\"Feedback\"].apply(\n",
    "    lambda feedback: categorize_single_feedback(feedback)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Feedback</th>\n",
       "      <th>AI Classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Training</td>\n",
       "      <td>The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Training</td>\n",
       "      <td>The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Training</td>\n",
       "      <td>I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&amp;A sometimes leaves lingering doubts.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Training</td>\n",
       "      <td>The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Training</td>\n",
       "      <td>The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Training</td>\n",
       "      <td>Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Training</td>\n",
       "      <td>Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Training</td>\n",
       "      <td>I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Training</td>\n",
       "      <td>The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Training</td>\n",
       "      <td>While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>Network</td>\n",
       "      <td>The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>Network</td>\n",
       "      <td>I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Network</td>\n",
       "      <td>Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Network</td>\n",
       "      <td>Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>Network</td>\n",
       "      <td>While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>Security</td>\n",
       "      <td>The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>Security</td>\n",
       "      <td>I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>Security</td>\n",
       "      <td>The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>Security</td>\n",
       "      <td>Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>Security</td>\n",
       "      <td>Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>31</td>\n",
       "      <td>Security</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>Security</td>\n",
       "      <td>The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Security</td>\n",
       "      <td>Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>Security</td>\n",
       "      <td>Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>35</td>\n",
       "      <td>Security</td>\n",
       "      <td>I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>36</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>37</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>38</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>39</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>40</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>41</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>42</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>43</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>44</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>45</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>46</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>47</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>48</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>49</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    ID          Category  \\\n",
       "0    1          Training   \n",
       "1    2          Training   \n",
       "2    3          Training   \n",
       "3    4          Training   \n",
       "4    5          Training   \n",
       "5    6          Training   \n",
       "6    7          Training   \n",
       "7    8          Training   \n",
       "8    9          Training   \n",
       "9   10          Training   \n",
       "10  11           Network   \n",
       "11  12           Network   \n",
       "12  13           Network   \n",
       "13  14           Network   \n",
       "14  15           Network   \n",
       "15  16           Network   \n",
       "16  17           Network   \n",
       "17  18        IT-support   \n",
       "18  19        IT-support   \n",
       "19  20        IT-support   \n",
       "20  21        IT-support   \n",
       "21  22        IT-support   \n",
       "22  23        IT-support   \n",
       "23  24        IT-support   \n",
       "24  25        IT-support   \n",
       "25  26          Security   \n",
       "26  27          Security   \n",
       "27  28          Security   \n",
       "28  29          Security   \n",
       "29  30          Security   \n",
       "30  31          Security   \n",
       "31  32          Security   \n",
       "32  33          Security   \n",
       "33  34          Security   \n",
       "34  35          Security   \n",
       "35  36    Business needs   \n",
       "36  37    Business needs   \n",
       "37  38    Business needs   \n",
       "38  39    Business needs   \n",
       "39  40    Business needs   \n",
       "40  41    Business needs   \n",
       "41  42    Business needs   \n",
       "42  43  Quality of tools   \n",
       "43  44  Quality of tools   \n",
       "44  45  Quality of tools   \n",
       "45  46  Quality of tools   \n",
       "46  47  Quality of tools   \n",
       "47  48  Quality of tools   \n",
       "48  49  Quality of tools   \n",
       "49  50  Quality of tools   \n",
       "\n",
       "                                                                                                                                                              Feedback  \\\n",
       "0         The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.   \n",
       "1             The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.   \n",
       "2                           I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&A sometimes leaves lingering doubts.   \n",
       "3                                              The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.   \n",
       "4                                            The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.   \n",
       "5                                                      Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.   \n",
       "6                            Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.   \n",
       "7                                 I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.   \n",
       "8                                              The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.   \n",
       "9                       While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.   \n",
       "10                               The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.   \n",
       "11                                     I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.   \n",
       "12                                   Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.   \n",
       "13                                 The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.   \n",
       "14                                                     Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.   \n",
       "15                       The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.   \n",
       "16                                         While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.   \n",
       "17                              The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.   \n",
       "18                                    I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.   \n",
       "19                                    The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.   \n",
       "20                           IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.   \n",
       "21                           Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.   \n",
       "22                                  The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.   \n",
       "23                             I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.   \n",
       "24                                        While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.   \n",
       "25                           The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.   \n",
       "26                        I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.   \n",
       "27                                           The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.   \n",
       "28                                              Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.   \n",
       "29                                            Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.   \n",
       "30                                   I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.   \n",
       "31                                    The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.   \n",
       "32                                                 Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.   \n",
       "33                                    Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.   \n",
       "34                              I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.   \n",
       "35         The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.   \n",
       "36               Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.   \n",
       "37                          The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.   \n",
       "38                             I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.   \n",
       "39                                     Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.   \n",
       "40  The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.   \n",
       "41                                           I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.   \n",
       "42                                  The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.   \n",
       "43                                      I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.   \n",
       "44                                        The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.   \n",
       "45                               Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.   \n",
       "46                                 The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.   \n",
       "47                                     I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.   \n",
       "48                            Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.   \n",
       "49          The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.   \n",
       "\n",
       "   AI Classification  \n",
       "0           Training  \n",
       "1           Training  \n",
       "2           Training  \n",
       "3           Training  \n",
       "4           Training  \n",
       "5           Training  \n",
       "6         IT-support  \n",
       "7           Training  \n",
       "8           Training  \n",
       "9           Training  \n",
       "10           Network  \n",
       "11           Network  \n",
       "12           Network  \n",
       "13           Network  \n",
       "14           Network  \n",
       "15           Network  \n",
       "16           Network  \n",
       "17        IT-support  \n",
       "18        IT-support  \n",
       "19        IT-support  \n",
       "20        IT-support  \n",
       "21        IT-support  \n",
       "22        IT-support  \n",
       "23        IT-support  \n",
       "24        IT-support  \n",
       "25             Other  \n",
       "26           Network  \n",
       "27             Other  \n",
       "28             Other  \n",
       "29             Other  \n",
       "30             Other  \n",
       "31             Other  \n",
       "32             Other  \n",
       "33             Other  \n",
       "34             Other  \n",
       "35             Other  \n",
       "36             Other  \n",
       "37             Other  \n",
       "38        IT-support  \n",
       "39             Other  \n",
       "40             Other  \n",
       "41             Other  \n",
       "42             Other  \n",
       "43        IT-support  \n",
       "44             Other  \n",
       "45             Other  \n",
       "46             Other  \n",
       "47             Other  \n",
       "48           Network  \n",
       "49           Network  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "done\n"
     ]
    }
   ],
   "source": [
    "# Vil bruke resonerinsmodell her, men den nekter å akseptere \"reasoning_effort\"\n",
    "categorize_prompt_other = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback from an IT-survey into the category that best describes the feedback.\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "categorize_chain_structured_output_others = categorize_prompt_other | llm.with_structured_output(\n",
    "    Categorize,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "def categorize_single_feedback_other(feedback: str) -> str:\n",
    "    result = categorize_chain_structured_output_others.invoke(\n",
    "        {\"feedback\": feedback}\n",
    "    )\n",
    "    return result.category\n",
    "\n",
    "# Retrieve indexes for all rows where AI classification = \"Other\"\n",
    "other_indices = df[df['AI Classification'] == \"Other\"].index\n",
    "\n",
    "# Apply classification function to only these rows and assign back correctly\n",
    "df.loc[other_indices, \"AI Classification\"] = df.loc[other_indices, \"Feedback\"].apply(\n",
    "    lambda feedback: categorize_single_feedback_other(feedback)\n",
    ")\n",
    "\n",
    "print(\"done\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Feedback</th>\n",
       "      <th>AI Classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Training</td>\n",
       "      <td>The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Training</td>\n",
       "      <td>The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Training</td>\n",
       "      <td>I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&amp;A sometimes leaves lingering doubts.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Training</td>\n",
       "      <td>The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Training</td>\n",
       "      <td>The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Training</td>\n",
       "      <td>Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Training</td>\n",
       "      <td>Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Training</td>\n",
       "      <td>I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Training</td>\n",
       "      <td>The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Training</td>\n",
       "      <td>While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>Network</td>\n",
       "      <td>The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>Network</td>\n",
       "      <td>I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Network</td>\n",
       "      <td>Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Network</td>\n",
       "      <td>Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>Network</td>\n",
       "      <td>While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>Security</td>\n",
       "      <td>The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>Security</td>\n",
       "      <td>I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>Security</td>\n",
       "      <td>The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>Security</td>\n",
       "      <td>Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>Security</td>\n",
       "      <td>Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>31</td>\n",
       "      <td>Security</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>Security</td>\n",
       "      <td>The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Security</td>\n",
       "      <td>Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>Security</td>\n",
       "      <td>Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>35</td>\n",
       "      <td>Security</td>\n",
       "      <td>I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>36</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>37</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>38</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>39</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>40</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>41</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>42</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>43</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>44</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>45</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>46</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>47</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>48</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>49</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    ID          Category  \\\n",
       "0    1          Training   \n",
       "1    2          Training   \n",
       "2    3          Training   \n",
       "3    4          Training   \n",
       "4    5          Training   \n",
       "5    6          Training   \n",
       "6    7          Training   \n",
       "7    8          Training   \n",
       "8    9          Training   \n",
       "9   10          Training   \n",
       "10  11           Network   \n",
       "11  12           Network   \n",
       "12  13           Network   \n",
       "13  14           Network   \n",
       "14  15           Network   \n",
       "15  16           Network   \n",
       "16  17           Network   \n",
       "17  18        IT-support   \n",
       "18  19        IT-support   \n",
       "19  20        IT-support   \n",
       "20  21        IT-support   \n",
       "21  22        IT-support   \n",
       "22  23        IT-support   \n",
       "23  24        IT-support   \n",
       "24  25        IT-support   \n",
       "25  26          Security   \n",
       "26  27          Security   \n",
       "27  28          Security   \n",
       "28  29          Security   \n",
       "29  30          Security   \n",
       "30  31          Security   \n",
       "31  32          Security   \n",
       "32  33          Security   \n",
       "33  34          Security   \n",
       "34  35          Security   \n",
       "35  36    Business needs   \n",
       "36  37    Business needs   \n",
       "37  38    Business needs   \n",
       "38  39    Business needs   \n",
       "39  40    Business needs   \n",
       "40  41    Business needs   \n",
       "41  42    Business needs   \n",
       "42  43  Quality of tools   \n",
       "43  44  Quality of tools   \n",
       "44  45  Quality of tools   \n",
       "45  46  Quality of tools   \n",
       "46  47  Quality of tools   \n",
       "47  48  Quality of tools   \n",
       "48  49  Quality of tools   \n",
       "49  50  Quality of tools   \n",
       "\n",
       "                                                                                                                                                              Feedback  \\\n",
       "0         The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.   \n",
       "1             The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.   \n",
       "2                           I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&A sometimes leaves lingering doubts.   \n",
       "3                                              The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.   \n",
       "4                                            The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.   \n",
       "5                                                      Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.   \n",
       "6                            Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.   \n",
       "7                                 I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.   \n",
       "8                                              The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.   \n",
       "9                       While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.   \n",
       "10                               The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.   \n",
       "11                                     I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.   \n",
       "12                                   Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.   \n",
       "13                                 The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.   \n",
       "14                                                     Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.   \n",
       "15                       The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.   \n",
       "16                                         While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.   \n",
       "17                              The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.   \n",
       "18                                    I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.   \n",
       "19                                    The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.   \n",
       "20                           IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.   \n",
       "21                           Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.   \n",
       "22                                  The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.   \n",
       "23                             I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.   \n",
       "24                                        While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.   \n",
       "25                           The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.   \n",
       "26                        I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.   \n",
       "27                                           The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.   \n",
       "28                                              Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.   \n",
       "29                                            Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.   \n",
       "30                                   I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.   \n",
       "31                                    The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.   \n",
       "32                                                 Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.   \n",
       "33                                    Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.   \n",
       "34                              I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.   \n",
       "35         The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.   \n",
       "36               Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.   \n",
       "37                          The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.   \n",
       "38                             I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.   \n",
       "39                                     Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.   \n",
       "40  The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.   \n",
       "41                                           I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.   \n",
       "42                                  The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.   \n",
       "43                                      I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.   \n",
       "44                                        The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.   \n",
       "45                               Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.   \n",
       "46                                 The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.   \n",
       "47                                     I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.   \n",
       "48                            Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.   \n",
       "49          The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.   \n",
       "\n",
       "   AI Classification  \n",
       "0           Training  \n",
       "1           Training  \n",
       "2           Training  \n",
       "3           Training  \n",
       "4           Training  \n",
       "5           Training  \n",
       "6         IT-support  \n",
       "7           Training  \n",
       "8           Training  \n",
       "9           Training  \n",
       "10           Network  \n",
       "11           Network  \n",
       "12           Network  \n",
       "13           Network  \n",
       "14           Network  \n",
       "15           Network  \n",
       "16           Network  \n",
       "17        IT-support  \n",
       "18        IT-support  \n",
       "19        IT-support  \n",
       "20        IT-support  \n",
       "21        IT-support  \n",
       "22        IT-support  \n",
       "23        IT-support  \n",
       "24        IT-support  \n",
       "25             Other  \n",
       "26           Network  \n",
       "27             Other  \n",
       "28             Other  \n",
       "29             Other  \n",
       "30             Other  \n",
       "31             Other  \n",
       "32             Other  \n",
       "33             Other  \n",
       "34             Other  \n",
       "35             Other  \n",
       "36             Other  \n",
       "37             Other  \n",
       "38        IT-support  \n",
       "39             Other  \n",
       "40             Other  \n",
       "41             Other  \n",
       "42             Other  \n",
       "43        IT-support  \n",
       "44             Other  \n",
       "45             Other  \n",
       "46             Other  \n",
       "47             Other  \n",
       "48           Network  \n",
       "49           Network  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df\n",
    "\n",
    "#Per nå bruker den kategoriene vi fikk fra LLMen. Når den kategorieserer Other får vi mange nye og litt varierende kategorier. Det er også mange av de. Skulle vi bare brukt den som eksempel også bruke den ekte kategoriseringe her?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Feedback</th>\n",
       "      <th>AI Classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&amp;A sometimes leaves lingering doubts.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>31</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>35</td>\n",
       "      <td>I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>36</td>\n",
       "      <td>The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>37</td>\n",
       "      <td>Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>38</td>\n",
       "      <td>The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>39</td>\n",
       "      <td>I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>40</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>41</td>\n",
       "      <td>The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>42</td>\n",
       "      <td>I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>43</td>\n",
       "      <td>The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>44</td>\n",
       "      <td>I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>45</td>\n",
       "      <td>The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>46</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>47</td>\n",
       "      <td>The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>48</td>\n",
       "      <td>I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>49</td>\n",
       "      <td>Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    ID  \\\n",
       "0    1   \n",
       "1    2   \n",
       "2    3   \n",
       "3    4   \n",
       "4    5   \n",
       "5    6   \n",
       "6    7   \n",
       "7    8   \n",
       "8    9   \n",
       "9   10   \n",
       "10  11   \n",
       "11  12   \n",
       "12  13   \n",
       "13  14   \n",
       "14  15   \n",
       "15  16   \n",
       "16  17   \n",
       "17  18   \n",
       "18  19   \n",
       "19  20   \n",
       "20  21   \n",
       "21  22   \n",
       "22  23   \n",
       "23  24   \n",
       "24  25   \n",
       "25  26   \n",
       "26  27   \n",
       "27  28   \n",
       "28  29   \n",
       "29  30   \n",
       "30  31   \n",
       "31  32   \n",
       "32  33   \n",
       "33  34   \n",
       "34  35   \n",
       "35  36   \n",
       "36  37   \n",
       "37  38   \n",
       "38  39   \n",
       "39  40   \n",
       "40  41   \n",
       "41  42   \n",
       "42  43   \n",
       "43  44   \n",
       "44  45   \n",
       "45  46   \n",
       "46  47   \n",
       "47  48   \n",
       "48  49   \n",
       "49  50   \n",
       "\n",
       "                                                                                                                                                              Feedback  \\\n",
       "0         The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.   \n",
       "1             The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.   \n",
       "2                           I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&A sometimes leaves lingering doubts.   \n",
       "3                                              The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.   \n",
       "4                                            The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.   \n",
       "5                                                      Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.   \n",
       "6                            Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.   \n",
       "7                                 I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.   \n",
       "8                                              The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.   \n",
       "9                       While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.   \n",
       "10                               The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.   \n",
       "11                                     I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.   \n",
       "12                                   Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.   \n",
       "13                                 The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.   \n",
       "14                                                     Connectivity is strong and dependable; however, periodic drops sow seeds of concern regarding data consistency.   \n",
       "15                       The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.   \n",
       "16                                         While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.   \n",
       "17                              The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.   \n",
       "18                                    I value IT-support’s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.   \n",
       "19                                    The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.   \n",
       "20                           IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.   \n",
       "21                           Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.   \n",
       "22                                  The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.   \n",
       "23                             I appreciate IT-support’s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.   \n",
       "24                                        While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.   \n",
       "25                           The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.   \n",
       "26                        I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.   \n",
       "27                                           The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.   \n",
       "28                                              Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.   \n",
       "29                                            Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.   \n",
       "30                                   I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.   \n",
       "31                                    The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.   \n",
       "32                                                 Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.   \n",
       "33                                    Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.   \n",
       "34                              I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.   \n",
       "35         The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.   \n",
       "36               Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.   \n",
       "37                          The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.   \n",
       "38                             I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.   \n",
       "39                                     Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.   \n",
       "40  The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.   \n",
       "41                                           I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.   \n",
       "42                                  The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.   \n",
       "43                                      I’m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.   \n",
       "44                                        The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.   \n",
       "45                               Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.   \n",
       "46                                 The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.   \n",
       "47                                     I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.   \n",
       "48                            Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.   \n",
       "49          The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.   \n",
       "\n",
       "   AI Classification  \n",
       "0           Training  \n",
       "1           Training  \n",
       "2           Training  \n",
       "3           Training  \n",
       "4           Training  \n",
       "5           Training  \n",
       "6         IT-support  \n",
       "7           Training  \n",
       "8           Training  \n",
       "9           Training  \n",
       "10           Network  \n",
       "11           Network  \n",
       "12           Network  \n",
       "13           Network  \n",
       "14           Network  \n",
       "15           Network  \n",
       "16           Network  \n",
       "17        IT-support  \n",
       "18        IT-support  \n",
       "19        IT-support  \n",
       "20        IT-support  \n",
       "21        IT-support  \n",
       "22        IT-support  \n",
       "23        IT-support  \n",
       "24        IT-support  \n",
       "25             Other  \n",
       "26           Network  \n",
       "27             Other  \n",
       "28             Other  \n",
       "29             Other  \n",
       "30             Other  \n",
       "31             Other  \n",
       "32             Other  \n",
       "33             Other  \n",
       "34             Other  \n",
       "35             Other  \n",
       "36             Other  \n",
       "37             Other  \n",
       "38        IT-support  \n",
       "39             Other  \n",
       "40             Other  \n",
       "41             Other  \n",
       "42             Other  \n",
       "43        IT-support  \n",
       "44             Other  \n",
       "45             Other  \n",
       "46             Other  \n",
       "47             Other  \n",
       "48           Network  \n",
       "49           Network  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Legger resultatene fra kategoriseringen av \"Others\" i en egen variabel som ikke har med de faktiske kategoriene.\n",
    "# Denne kan da brukes i oppgavene lengere ned uten å forvirre LLMen på hva som er kategoriene den skal kjenne igjen.\n",
    "categorized_survey = df[['ID','Feedback','AI Classification']]\n",
    "categorized_survey"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chain-of-thought\n",
    "\n",
    "Chain of Thought (CoT) er en teknikk innen prompt engineering som hjelper språkmodeller med å løse oppgaver som krever flere tankesteg. I stedet for å hoppe rett til svaret, blir modellen ledet gjennom en logisk og trinnvis prosess, noe som gir mer presise og gjennomtenkte svar – spesielt på komplekse problemer [1]. \n",
    "\n",
    "Du kan altså be modellen om å \"tenke høyt\" under oppgaven og forklare stegene sine før den leverer et endelig svar. Dette ber du om i prompten som sendes inn. \n",
    "\n",
    "Eksempel på en prompt **uten** CoT: \n",
    "\n",
    "    Prompt: \"Hvor mange armer har Eline og Kaspara?\"\n",
    "\n",
    "    Svar: \"4\"\n",
    "\n",
    "Eksempel på en prompt **med** CoT:\n",
    "\n",
    "    Prompt: \"Hvor mange armer har ELine og Kaspara? Tenk trinn for trinn.\"\n",
    "\n",
    "    Svar: \"En person har to armer. To personer betyr 2x2 = 4 armer. Svaret er 4.\"\n",
    "\n",
    "Det kan være fordelaktig å bruke CoT når man jobber med komplekse oppgaver, da nøyaktigheten på outputet fra modellen øker når den \"får lov\" til å jobbe seg gjennom problemet. Dette gir ofte bedre resultater på logiske oppgaver, eller oppgaver med flere steg. \n",
    "\n",
    "I tillegg kan du se hvordan modellen tenker, som gjør det lettere for deg å evaluere svaret. Det blir også lettere å se hvor det gikk galt hvis modellen svarer feil.\n",
    "\n",
    "### *Kilder*\n",
    "[1] xxxx, Link: https://www.ibm.com/think/topics/chain-of-thoughts \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CategorizeCot(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    chain_of_thought: str = Field(\n",
    "        description=\"Use this space to think through the categorization.\"\n",
    "    )\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  \n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_cot = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeCot,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "The feedback discusses training sessions related to new software updates, highlighting both the clarity of guidance and the desire for more practical examples. Since the primary focus is on training, this feedback fits best into the 'Training' category."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Training"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "result = categorize_chain_cot.invoke(df[\"Feedback\"][0])\n",
    "display(Markdown(result.chain_of_thought))\n",
    "display(Markdown(result.category))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resonneringsmodell\n",
    "Resonneringsmodeller, som Azure Open AI sin O3-mini-modell, er språkmodeller som er spesielt trent på å tenke før de svarer. Slike modeller vil altså produsere en trinnvis tenkning før det endelige svaret leveres. Resonneringsmodeller er fordelaktige å bruke til oppgaver som krever kompleks problemløsning, logisk tenkning som koding eller matematikk eller til oppgaver med flere steg. De vil også være fordelaktige å bruke i situasjoner der nøyaktighet og forklarbarhet er viktig [2]. \n",
    "\n",
    "Dette kan minne om CoT, men det er en viktig forksjell her. CoT er en teknikk du kan bruke med språkmodeller for å gjøre dem bedre til å resonnere. Det er raskt og fleksibelt.\n",
    "Resonneringsmodeller er som nevnt en egen type modell som passer for oppgaver som krever presis og systematisk tenkning. Du vil få enda mer presise svar med en resonneringsmodell sammenlignet med CoT. \n",
    "\n",
    "#### Kilder\n",
    "[2] xxx, Link: https://platform.openai.com/docs/guides/reasoning?api-mode=chat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nå har vi definert passende kategorier for alle tilbakemeldingene. \n",
    "Videre vil vi få modellen til å oppsummere per kategori, slik at vi sitter igjen med en overordnet oversikt. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Din tur (forslag)\n",
    "'''Lag en LCEL-kjede som tar resultatet fra forrige oppgave (feedback med kategori) og lager en oppsummering\n",
    "per kategori ved hjelp av en LLM. Inkluder structured output.'''\n",
    "\n",
    "# Tips 1: Lag en god promt! Bruk dette til å reflektere over hva det endelige målet er. \n",
    "prompt = PromptTemplate.from_template(\"\"\"\n",
    "Ja, hvordan kan jeg formulere meg her da? Bør det ikke komme med noe data her også?\n",
    "\"\"\")\n",
    "\n",
    "# Tips 2: Kan du lage en klasse som arver fra BaseModel for å gjøre dette enklere?\n",
    "class SummarizeFeedback(BaseModel):\n",
    "    \"Beskrivelse...\"\n",
    "    summary : str = ...\n",
    "\n",
    "# Tips 3: På tide å lage kjeden\n",
    "summary_chain_structured_output = ...\n",
    "\n",
    "# Tips 4: Kall modellen med datasettet fra undersøkelsen\n",
    "summary = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Fasit\n",
    "# 1. Prompt Template\n",
    "summary_prompt = PromptTemplate.from_template(\"\"\"\n",
    "You are a domain expert in internal IT operations and organizational analysis. You will be provided with a dataset containing qualitative feedback from employees in an IT company. \n",
    "Each row in the dataset represents a feedback entry and is associated with a specific category.\n",
    "\n",
    "For each category, carefully:\n",
    "1. Read and interpret the feedback entries assigned to that category.\n",
    "2. Identify core themes, recurring patterns, and contrasting opinions within that category.\n",
    "3. Evaluate the feedback logically: What are the likely underlying causes of recurring issues or praises? Are there signs of systemic problems, isolated incidents, or misaligned expectations?\n",
    "4. Summarize each category in 3 to 6 bullet points, highlighting key sentiments (positive and negative), representative concerns or compliments, and any significant outliers\n",
    "\n",
    "Present your findings in a clean, professional way with one section per category. \n",
    "\n",
    "This is the employee feedback data: {survey_results}\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class SummarizeFeedback(BaseModel):\n",
    "    \"Summary of the different categorizes recognized in the feedback from an IT-survey.\"\n",
    "    summary : str = Field(\n",
    "        description=\"For each category: Category name and 3-6 bullet points summarizing the category.\"\n",
    "    )\n",
    "\n",
    "# 3. Summary-chain\n",
    "summary_chain_structured_output = summary_prompt | llm.with_structured_output(\n",
    "    SummarizeFeedback,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "# 4. Kall modellen med det kategoriserte datasettet fra undersøkelsen \n",
    "summary = summary_chain_structured_output.invoke({\"survey_results\": categorized_survey})\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "### Training\n",
       "- Positive feedback on the design and interactivity of training modules.\n",
       "- Hands-on exercises are appreciated for enhancing engagement and learning.\n",
       "- Some employees feel the breadth of topics covered could be improved.\n",
       "- Overall, training is seen as beneficial, but there are suggestions for more tailored content.\n",
       "\n",
       "### IT-support\n",
       "- IT-support is praised for promptness and effectiveness in resolving issues.\n",
       "- Clear communication and follow-ups are highlighted as strengths.\n",
       "- Some feedback indicates occasional delays in response times.\n",
       "- Overall, IT-support is viewed positively, with a few isolated concerns about consistency.\n",
       "\n",
       "### Network\n",
       "- The reliability and performance of the office network receive commendations.\n",
       "- Employees appreciate steady connectivity, though some mention occasional disruptions.\n",
       "- The network infrastructure is generally seen as robust and supportive of daily operations.\n",
       "- A few concerns about performance during peak usage times are noted.\n",
       "\n",
       "### Cybersecurity\n",
       "- Recent cybersecurity measures are viewed positively, enhancing employee confidence.\n",
       "- Employees appreciate proactive security updates and robust protocols.\n",
       "- There is a strong sentiment of trust in the security team's efforts.\n",
       "- Some feedback suggests a desire for more transparency regarding security measures.\n",
       "\n",
       "### Alignment of IT Solutions with Business Needs\n",
       "- IT solutions are generally seen as well-aligned with evolving business needs.\n",
       "- Employees appreciate efforts to tailor IT solutions to specific requirements.\n",
       "- There is recognition of the balance between technical capability and business strategy.\n",
       "- Some feedback indicates a need for ongoing adjustments to maintain alignment.\n",
       "\n",
       "### System Performance\n",
       "- Systems are reported to support business functions effectively.\n",
       "- Employees appreciate the integration of technology with business processes.\n",
       "- There are positive remarks about the overall performance of IT systems.\n",
       "- A few suggestions for improvements in system responsiveness are noted.\n",
       "\n",
       "### Process Improvement\n",
       "- Feedback indicates a recognition of ongoing efforts to improve processes.\n",
       "- Employees appreciate the focus on efficiency and effectiveness in IT operations.\n",
       "- Some suggestions for further enhancements in workflow are provided.\n",
       "\n",
       "### Tool Quality\n",
       "- The quality of software tools is highly regarded, with many praising their user-friendliness.\n",
       "- Employees appreciate the blend of innovation and stability in the tools provided.\n",
       "- There are positive remarks about the tools supporting day-to-day operations.\n",
       "- A few outliers express concerns about specific tool functionalities."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Nice måte å vise outputten fra modellen \n",
    "from IPython.display import Markdown\n",
    "\n",
    "display(Markdown(summary.summary))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rapport til ledelsen\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Oppgave \n",
    "report_prompt = PromptTemplate.from_template(\"\"\"\n",
    "Bla bla bla\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class ReportForLeadership(BaseModel):\n",
    "    \"Raport for the leadership of an IT-company on results of an internal IT-survey.\"\n",
    "    snappy_title : str = Field(\n",
    "        description=\"A fitting title for the report. Must begin with '# ' to ensure easy markdown formatting.\"\n",
    "    )\n",
    "    key_takeaway:str = ...\n",
    "\n",
    "# 3. Report-chain\n",
    "report_chain_structured_output = ...\n",
    "\n",
    "# 4. Kall modellen med oppsummeringen av kategoriene\n",
    "report = ...\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fasit\n",
    "report_prompt = PromptTemplate.from_template(\"\"\"\n",
    "You are an expert HR and technical operations analyst. I will provide you with a dataset of employee feedback collected from an IT company.\n",
    "\n",
    "Your task is to deeply analyze this feedback and generate a concise executive-level summary report in markdown format that includes:\n",
    "\n",
    "1. Key Takeaways\n",
    "Provide a short summary of the overall feedback in 3-5 bullet points. Focus only on the main issues or areas of satisfaction.\n",
    "Include both positive and negative themes, but prioritize the most important and impactful points.\n",
    "Limit each point to 1-2 sentences.\n",
    "Before finalizing each point, take a moment to reflect on why each issue might be present (e.g., systemic problems, temporary issues, resource constraints, etc.)\n",
    "\n",
    "2. Suggested Improvements\n",
    "Based on the overall feedback, propose 2-3 high-level, actionable measures that the company could take to address the most pressing issues and enhance overall performance or satisfaction.\n",
    "Each suggestion should be brief, directly tied to the feedback, and strategic in nature.\n",
    "Think about short-term vs long-term solutions and consider the feasibility of each suggestion.\n",
    "\n",
    "3. Output\n",
    "Present your findings in a structured way with clear section headings, bullet points for easy scanning, and a consise, direct and professional tone suitable for leadership review.\n",
    "\n",
    "This is the employee feedback data: {summary_text}\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class ReportForLeadership(BaseModel):\n",
    "    \"Raport for the leadership of an IT-company on results of an internal IT-survey.\"\n",
    "    snappy_title : str = Field(\n",
    "        description=\"A fitting title for the report. Must begin with '# ' to ensure easy markdown formatting.\"\n",
    "    )\n",
    "    intro : str = Field(\n",
    "        description=\"1 sentence describing thepurpose of the report.\" \n",
    "    )\n",
    "    key_takeaways : str = Field(\n",
    "        description=\"3-5 bulletpoints describing the key-takeaways. Limit each point to 1-2 sentences.\" \n",
    "    )\n",
    "    suggested_improvements : str = Field(\n",
    "        description=\"2-3 actionable measures for the company. Keep it brief.\"\n",
    "    )\n",
    "    outro: str = Field(\n",
    "        description=\"1 sentence ending for the report. Be creative.\" \n",
    "    )\n",
    "\n",
    "# 3. Report-chain\n",
    "report_chain_structured_output = report_prompt | llm.with_structured_output(\n",
    "    ReportForLeadership,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "# 4. Kall modellen med oppsummeringen av kategoriene\n",
    "report = report_chain_structured_output.invoke({\"summary_text\": summary.summary})\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RaportForLeadership(snappy_title='# Employee Feedback Analysis: Key Insights and Recommendations', intro='This report summarizes the findings from the recent employee feedback survey, highlighting key themes and actionable improvements.', key_takeaways='- **Training Effectiveness**: Employees appreciate the interactive design of training modules but desire a broader range of topics and more tailored content, indicating a potential gap in addressing diverse learning needs.\\n- **IT Support Performance**: While IT support is generally praised for its promptness and effectiveness, occasional delays suggest a need for improved consistency in response times.\\n- **Network Reliability**: The office network is largely reliable, though some disruptions during peak usage times point to potential infrastructure limitations that need addressing.\\n- **Cybersecurity Confidence**: Recent cybersecurity measures have bolstered employee trust, yet there is a call for greater transparency regarding these protocols to enhance confidence further.\\n- **Alignment with Business Needs**: IT solutions are well-aligned with business needs, but ongoing adjustments are necessary to maintain this alignment as the business evolves.', suggested_improvements='- **Enhance Training Programs**: Develop a more diverse and tailored training curriculum that addresses specific employee needs and interests, ensuring all staff feel adequately supported in their professional development.\\n- **Improve IT Support Consistency**: Implement a tracking system for IT support requests to identify patterns in delays and allocate resources more effectively, ensuring timely responses across the board.\\n- **Increase Transparency in Cybersecurity**: Regularly communicate updates and insights regarding cybersecurity measures to employees, fostering a culture of trust and awareness around security practices.', outro='By addressing these key areas, we can enhance employee satisfaction and operational efficiency, paving the way for a more engaged and productive workforce.')"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "# Employee Feedback Analysis: Key Insights and Recommendations\n",
       "\n",
       "This report summarizes the findings from the recent employee feedback survey, highlighting key themes and actionable improvements.\n",
       "\n",
       "## Key-takeaways\n",
       "\n",
       "- **Training Effectiveness**: Employees appreciate the interactive design of training modules but desire a broader range of topics and more tailored content, indicating a potential gap in addressing diverse learning needs.\n",
       "- **IT Support Performance**: While IT support is generally praised for its promptness and effectiveness, occasional delays suggest a need for improved consistency in response times.\n",
       "- **Network Reliability**: The office network is largely reliable, though some disruptions during peak usage times point to potential infrastructure limitations that need addressing.\n",
       "- **Cybersecurity Confidence**: Recent cybersecurity measures have bolstered employee trust, yet there is a call for greater transparency regarding these protocols to enhance confidence further.\n",
       "- **Alignment with Business Needs**: IT solutions are well-aligned with business needs, but ongoing adjustments are necessary to maintain this alignment as the business evolves.\n",
       "\n",
       "## Suggested improvements\n",
       "\n",
       "- **Enhance Training Programs**: Develop a more diverse and tailored training curriculum that addresses specific employee needs and interests, ensuring all staff feel adequately supported in their professional development.\n",
       "- **Improve IT Support Consistency**: Implement a tracking system for IT support requests to identify patterns in delays and allocate resources more effectively, ensuring timely responses across the board.\n",
       "- **Increase Transparency in Cybersecurity**: Regularly communicate updates and insights regarding cybersecurity measures to employees, fostering a culture of trust and awareness around security practices.\n",
       "\n",
       "By addressing these key areas, we can enhance employee satisfaction and operational efficiency, paving the way for a more engaged and productive workforce."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(\n",
    "    \"\\n\\n\".join([report.snappy_title,\n",
    "                 report.intro,\n",
    "                 '## Key-takeaways',report.key_takeaways,\n",
    "                 '## Suggested improvements',report.suggested_improvements,\n",
    "                 report.outro])\n",
    "                 \n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hvordan kunne vi gjort dette bedre?\n",
    "\n",
    "Anta at du har fått levert resultatene fra denne spørreundersøkelsen i fanget av en stressa mellomleder som ber deg levere en rapport han kan presentere ledelsen. \n",
    "Gitt verktøyene du har fått en innføring i gjennom dette kursene (og kanskje andre erfaringer?), hvordan ville du løst oppgaven?\n",
    "\n",
    "Ser du for eksempel noe som kunne vært forbedret i\n",
    "- Rekkefølgen på måten vi leter etter kategorier?\n",
    "- Legger vi får mye/lite vekt på inputen vi fikk om hva ledelsen \"tror\" kategoriene kommer til å være?\n",
    "- Promptingen?\n",
    "- Variablene eller type hintingen i pydantic-klassene?\n",
    "\n",
    "Ville du kanskje gjort det helt annerledes? \n",
    "Now's your chance to try!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prøv deg frem :))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
