{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM for alle - Introduksjonskurs til språkmodeller med Python og Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduksjon\n",
    "\n",
    "Etter prosjektet vi gjorde for Elkem fikk vi hands-on erfaring med å utvikle og bruke språkmodeller (LLMer) i praksis. Vi så hvor effektivt dette kan være for å spare tid, redusere kostnader og automatisere manuelle oppgaver – på en måte som enkelt kan gjenbrukes i andre prosjekter.\n",
    "\n",
    "I dette kurset bruker vi et konkret og gjenkjennbart case: analyse av resultatene fra en medarbeiderundersøkelse – en oppgave mange sliter med å strukturere. Vi skal lære dere å koble dere til en LLM via API, bruke LangChain Expression Language (LCEL) til å bygge AI-kjeder og hente ut strukturert innsikt med Pydantic – alt i én oversiktlig notebook.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Oppgave\n",
    "Du jobber i et IT-selskap og har fått i oppgave å analysere svarene fra en intern medarbeiderundersøkelse. Undersøkelsen er anonym, og du har fått tilsendt en CSV-fil med 50 tilbakemeldinger – én per ansatt. Målet er å finne ut hva folk er fornøyde eller misfornøyde med, og særlig se nærmere på temaene Nettverk, Opplæring og IT-support, som ledelsen er ekstra interessert i. Tilbakemeldinger som ikke passer i disse kategoriene skal også få sin plass. Til slutt skal du lage en oppsummering som kan sendes til ledelsen.\n",
    "\n",
    "For å jobbe effektivt bruker du en språkmodell til å hjelpe deg med både kategorisering og oppsummering. \n",
    "\n",
    "**Oppgaven blir dermed å bruke en språkmodell til å kategorisere samt oppsummere tilbakemeldingene fra undersøkelsen.** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasett"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importerer rådataen med tilbakemeldinger, en rad per ansatt. Alle ansatte har svart på undersøkelsen.\n",
    "\n",
    "import pandas as pd\n",
    "pd.set_option('display.max_colwidth', None) # Ensure no truncated output of dataframe\n",
    "\n",
    "enr_path = \"../files/IT_survey.csv\"\n",
    "df = pd.read_csv(enr_path)\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Før vi går videre: Hva slags informasjon ser du i tilbakemeldingene?\n",
    "Hvilke ting tror du språkmodellen vil slite med å forstå eller kategorisere riktig?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bruk av språkmodeller gjennom en API (Using a language model through the API)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### En vanlig LLM-spørring\n",
    "Enkle LLM-spørringer er bygget opp av noen sentrale deler:\n",
    "\n",
    "1. Tilkobling til en API, som feks. Azure OpenAI\n",
    "\n",
    "2. En prompt, som vil si en tekstbasert forespørsel/instruks\n",
    "\n",
    "3. Sending av prompt til språkmodellen for å hente en respons\n",
    "\n",
    "\n",
    "Som en del av tilkoblingen er det vanlig å oppgi en temperaturparameter. Denne parametreren angir nivået av presisjon du ønsker å få i responsen fra språkmodellen, og kan enten måles på en skala fra \"low\" til \"high\" eller numerisk fra 0 til 1.\n",
    "Hvis denne parameteren settes til \"low\"/nærme 0 tillater du liten grad av variasjon og kreativitet i responsen, og du vil få tryggere og mer forutsigbare svar. Hvis den derimot settes til høy/nærme 1 tillater du større grad av kreativitet og detaljer, men vil følgelig også få en mer uforutsigbar respons. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Connect through the API\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from dotenv import find_dotenv, load_dotenv\n",
    "import os\n",
    "from IPython.display import Markdown # Pretty output\n",
    "\n",
    "\n",
    "# Get environment variables\n",
    "load_dotenv(find_dotenv(), override=True)\n",
    "\n",
    "\n",
    "\n",
    "llm = AzureChatOpenAI(\n",
    "    azure_deployment=os.getenv(\"AZURE_OPENAI_DEPLOYMENT_GPT_4O_MINI\"),\n",
    "    model=os.getenv(\"OPENAI_MODEL_GPT_4O-MINI\", default=\"gpt-4o-mini\"),\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "reasoning_llm = AzureChatOpenAI(\n",
    "    azure_deployment=os.getenv(\"AZURE_OPENAI_DEPLOYMENT_O4_MINI\"),\n",
    "    model=os.getenv(\"OPENAI_MODEL_O4-MINI\", default=\"o4-mini\"),\n",
    "    reasoning_effort=\"medium\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate the prompt\n",
    "prompt = 'Hei!'\n",
    "\n",
    "# Send the prompt and recieve a response\n",
    "response = llm.invoke(prompt)\n",
    "response_res = reasoning_llm.invoke(prompt)\n",
    "\n",
    "# Show the response from the model\n",
    "print(response)\n",
    "print(response_res)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show only the content of the response\n",
    "response.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Forklar temperature også med kodeeksempel: La deltakerne teste temperature=0.9 og sammenligne responsen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hvordan lage en enkel kjede med LangChain Expression Language (LCEL) // Creating a basic chain with Langchain Expression Language (LCEL)\n",
    "\n",
    "LCEL er en metode for å bygge og kjøre såkalte *kjeder* i LangChain på. Kjeder, eller chains, brukes for å koble sammen ulike AI-komponenter. F.eks. kan språkmodeller, datakilder og logikk kobles sammen til én sammenhengende prosess, dvs. en kjede. LCEL gir et standardisert språk for å definere disse kjedene, og er brukervennlig fordi man slipper å lage alt manuelt med kode. Med andre ord får du en \"oppskrift\" på hvordan AI-komponentene dine skal jobbe sammen på en rask og skalerbar måte.\n",
    "\n",
    "**Fordelene med LCEL**\n",
    "1. Støtter parallell og asynkron kjøring - Ulike deler av kjeden kan kjøre samtidig, og systemet kan behandle flere forespørsler på en gang. Dermed kan oppgaver behandles raskere. \n",
    "2. Strømming av resultater - Man kan begynne å se svar mens AI-en fremdeles jobber. (Passer spesielt godt for chatbaserte løsninger)\n",
    "3. Enkel feilsøking med LangSmith - Når kjedene blir komplekse er det viktig å kunne se hva som har blitt gjort underveis. LCEL logger automatisk alt til LangSmith, som gjør det enklere å feilsøke. \n",
    "4. Standardisert - Alle kjeder i LCEL bruker samme grensesnitt, som gjør dem enkle å kombinere og gjenbruke på tvers av prosjekter. \n",
    "\n",
    "LCEL bruker en pipe-operator (|) til å koble sammen ulike trinn i kjeden. Den tar ut data fra én komponent og sender den direkte som input til neste komponent. LCEL bruker også PromptTemplate, som kan tenkes på som en mal for teksten du sender til språkmodellen. PromptTemplate gjør det enkelt å lage dynamiske meldinger ved at man kan sette inn variabler i teksten, litt som en oppskrift der du fyller inn det som mangler før det sendes til AI-modellen. Fordelen med PromptTemplate er at man kan lage én mal, og bruke den med ulike data. Det hjelper deg også med å skille selve teksten fra logikken, og kan gjøre prosessen sikrere ved at man unngår feil som kan oppstå ved manuell string-manipulasjon. Vi skal nå se på noen eksempler med LCEL som bruker pipe-operator og PromptTemplate.\n",
    "\n",
    "Kilde: xxx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "prompt = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Hi! Please talk like a {role}.\n",
    "    \"\"\"\n",
    ")\n",
    "prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm\n",
    "\n",
    "chain.invoke({\"role\": \"pirate\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Som dere ser returnerer modellen mye mer enn kun svaret på prompten vår, nemlig en hel haug med meta-data. Dette brys vi oss ikke så mye om i dette kurset, så vi bruker StrOutputParser for å kun få ut tekst-responsen fra modellen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "chain2 = chain | StrOutputParser()\n",
    "\n",
    "chain2.invoke({\"role\": \"pirate\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pRØV SElv oppgave. For eksempel: PromptTemplate er som en e-postmal: Du skriver \"Hei {navn}, du har fått {antall} nye meldinger\", og fyller inn variabler senere."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enkel kategorisering av hver enkelt tilbakemelding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bruk LLMen til kategorisering av tilbakemeldingene\n",
    "I første omgang av kategoriseringen er vi interesserte i å se hvor mange av tilbakemeldingene som passer innenfor de kategoriene ledelsen ønsket et ekstra fokus på, nemlig Network, Training og IT-support. Vi kan be LLMen om å utføre denne kategoriseringen ved å gi den tilgang på feedback-dataen. Husk på de sentrale delene av enkle LLM-kall, og benytt deg av LCEL som vist i forrige eksempel. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorize_prompt = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into one of the following categories:\n",
    "- Network\n",
    "- Training\n",
    "- IT-support\n",
    "- Other\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "categorize_chain = categorize_prompt | llm | StrOutputParser()\n",
    "\n",
    "categorize_chain.invoke({\"feedback\": \"I am very happy with the IT support I received last week.\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structured output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pydantic \n",
    "Pydantic er et Python-bibliotek for datavalidering og datastrukturering. Hovedklassen i Pydantic heter BaseModel og er klassen vi arver fra når vi lager våre egne datamodeller. Når vi arver fra BaseModel får vi automatisk funksjonalitet som kan:\n",
    "1. Validere innhold du sender inn\n",
    "    - Eks: Du definerer en liste med godkjente land, \"Norge, Sverige, Finland\", da vil ikke pydantic godkjenne \"Australia\".\n",
    "2. Konvertere data til riktig type\n",
    "    - Eks: Du definerer at output skal være en int og sender inn '1', pydantic vil da returnere 1 (som int)\n",
    "3. Påtvinge JSON-formatering\n",
    "    - Sørger for at responsen LLMen gir matcher JSON-kjemaet til Pydantic. Dette gjør at vi kan være sikre på strukturen til outputen, som for eksempel er svært nyttig om vi ønsker å bruke outputen fra en modell som input i en annen. \n",
    "\n",
    "\n",
    "\n",
    "### Structured output \n",
    "I denne konteksten refererer structured output til strategien og verktøyene vi bruker for å forsikre oss om at dataen vår blir organisert på en måte vi definerer på forhånd. \n",
    "Funksjonalitet i Pydantic biblioteket lar oss bestemme strukturen på outputen gjennom å spesifisere felter, definere typer (eks. int, str, List[int]) og validere data.\n",
    "I dette kurset kommer vi til å bruke BaseModel til å lage våre egne klasser for å sikre at det vi mottar fra modellene når vi prompter de kommer på akkurat den formen vi forventer.\n",
    "Dette gjør det også mye enklere å bruke resultater videre i pipelines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "\n",
    "class Categorize(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category : str = Field(description=\"The best fitting category. Only one.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I eksempelet over har vi laget vår egen klasse 'Categorize' som arver av BaseModel.\n",
    "For variabelen category har vi brukt type hinting ( x: type ) for å definere typen Pydantic skal forvente at category er lik. Dette for ekesempel blir da ulovlig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Categorize(category=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mens dette er helt ok"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Categorize(category=\"Network\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vi bruker også funksjonen Field, denne kan du bruke til å sette standardverdier, valideringsregler og beskrivelser.\n",
    "I dette kurset bruker vi kun beskrivelse, men for de spesielt interesserte kan dere lese mer om funksjonaliteten [her](https://docs.pydantic.dev/latest/concepts/fields/).\n",
    "\n",
    "For å få LLMen til å skjønne at den må følge reglene vi har definert i klassen vår bruker vi wrapperen with_structured_output(...). \n",
    "Denne wrapper kallet vårt til språkmodellen med logikk som forsikrer at outputen følger strukturen vi har definert i klassen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorize_chain_structured_output = categorize_prompt | llm.with_structured_output(\n",
    "    Categorize,\n",
    "    method=\"json_schema\", # Påtvinger JSON-skjema for output\n",
    "    strict=True           # Modellen må følge skjema etter punkt og prikke, ingen ekstra felter, ingen manglende felter og alle typer må være en eksakt match.\n",
    ")\n",
    "\n",
    "categorize_chain_structured_output.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vi kan bruke Literal for å definere våre egne typer til å bruke for type hinting. Vi kan type hinte variabler med CATEGORIES under og dette vil da modellen toke på samme måte som at en int bare har lov til å være et heltall har denne \"typen\" bare lov til å være en av verdiene listet opp i Literal-objektet.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "CATEGORIES = Literal[\n",
    "    \"Network\",\n",
    "    \"Training\",\n",
    "    \"IT-support\",\n",
    "    'Other'\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La oss sette sammen det vi har lært. Legg merke til at vi ikke lenger lister kategoriene i prompten (som på ingen måte garanterer at vi kun får ut en av kategorierne vi øsnker), men definerer de som et krav til strukturen på outputen fra modellen. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "categorize_prompt2 = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into the provided categories.\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "\n",
    "\n",
    "class CategorizeFromOptions(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  ## noqa: E501\n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_structured_output2 = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeFromOptions,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = categorize_chain_structured_output2.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")\n",
    "\n",
    "result\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vi lar modellen gå løs på hele datasettet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "def categorize_single_feedback(feedback: str) -> str:\n",
    "    result = categorize_chain_structured_output2.invoke(\n",
    "        {\"feedback\": feedback}\n",
    "    )\n",
    "    return result.category\n",
    "\n",
    "df[\"AI Classification\"] = df[\"Feedback\"].apply(\n",
    "    lambda feedback: categorize_single_feedback(feedback)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Feedback</th>\n",
       "      <th>AI Classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Security</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Security</td>\n",
       "      <td>I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Security</td>\n",
       "      <td>The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Security</td>\n",
       "      <td>Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>Network</td>\n",
       "      <td>Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>Training</td>\n",
       "      <td>The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>Training</td>\n",
       "      <td>The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Training</td>\n",
       "      <td>The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Training</td>\n",
       "      <td>Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>Training</td>\n",
       "      <td>While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>Network</td>\n",
       "      <td>While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>Security</td>\n",
       "      <td>I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>Security</td>\n",
       "      <td>The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>Training</td>\n",
       "      <td>The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>Security</td>\n",
       "      <td>The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>Security</td>\n",
       "      <td>Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>Training</td>\n",
       "      <td>Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>31</td>\n",
       "      <td>Security</td>\n",
       "      <td>Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>Network</td>\n",
       "      <td>I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>Training</td>\n",
       "      <td>The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>35</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>36</td>\n",
       "      <td>Training</td>\n",
       "      <td>I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&amp;A sometimes leaves lingering doubts.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>37</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Iâ€™m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>38</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>39</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I appreciate IT-supportâ€™s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>40</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>41</td>\n",
       "      <td>Network</td>\n",
       "      <td>The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>42</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>43</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I value IT-supportâ€™s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>44</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>45</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>46</td>\n",
       "      <td>Training</td>\n",
       "      <td>I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>47</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>48</td>\n",
       "      <td>Network</td>\n",
       "      <td>Connectivity is strong and dependable\\t however, periodic drops sow seeds of concern regarding data consistency.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>49</td>\n",
       "      <td>Security</td>\n",
       "      <td>Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    ID          Category  \\\n",
       "0    1           Network   \n",
       "1    2    Business needs   \n",
       "2    3          Security   \n",
       "3    4  Quality of tools   \n",
       "4    5        IT-support   \n",
       "5    6  Quality of tools   \n",
       "6    7          Security   \n",
       "7    8          Security   \n",
       "8    9          Security   \n",
       "9   10        IT-support   \n",
       "10  11           Network   \n",
       "11  12          Training   \n",
       "12  13    Business needs   \n",
       "13  14          Training   \n",
       "14  15          Training   \n",
       "15  16          Training   \n",
       "16  17    Business needs   \n",
       "17  18  Quality of tools   \n",
       "18  19  Quality of tools   \n",
       "19  20           Network   \n",
       "20  21          Training   \n",
       "21  22           Network   \n",
       "22  23        IT-support   \n",
       "23  24          Security   \n",
       "24  25          Security   \n",
       "25  26          Training   \n",
       "26  27  Quality of tools   \n",
       "27  28          Security   \n",
       "28  29          Security   \n",
       "29  30          Training   \n",
       "30  31          Security   \n",
       "31  32           Network   \n",
       "32  33    Business needs   \n",
       "33  34          Training   \n",
       "34  35        IT-support   \n",
       "35  36          Training   \n",
       "36  37  Quality of tools   \n",
       "37  38    Business needs   \n",
       "38  39        IT-support   \n",
       "39  40    Business needs   \n",
       "40  41           Network   \n",
       "41  42        IT-support   \n",
       "42  43        IT-support   \n",
       "43  44  Quality of tools   \n",
       "44  45        IT-support   \n",
       "45  46          Training   \n",
       "46  47  Quality of tools   \n",
       "47  48           Network   \n",
       "48  49          Security   \n",
       "49  50    Business needs   \n",
       "\n",
       "                                                                                                                                                              Feedback  \\\n",
       "0                                  The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.   \n",
       "1                                      Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.   \n",
       "2                                    I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.   \n",
       "3                                Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.   \n",
       "4                               The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.   \n",
       "5                             Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.   \n",
       "6                         I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.   \n",
       "7                            The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.   \n",
       "8                                                  Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.   \n",
       "9                                     The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.   \n",
       "10                                   Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.   \n",
       "11                                           The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.   \n",
       "12                          The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.   \n",
       "13                                             The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.   \n",
       "14                                             The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.   \n",
       "15                           Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.   \n",
       "16                                           I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.   \n",
       "17                                 The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.   \n",
       "18                                     I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.   \n",
       "19                       The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.   \n",
       "20                      While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.   \n",
       "21                                         While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.   \n",
       "22                                        While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.   \n",
       "23                              I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.   \n",
       "24                                    The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.   \n",
       "25        The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.   \n",
       "26                                        The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.   \n",
       "27                                           The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.   \n",
       "28                                    Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.   \n",
       "29                                                     Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.   \n",
       "30                                            Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.   \n",
       "31                                     I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.   \n",
       "32               Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.   \n",
       "33            The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.   \n",
       "34                           Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.   \n",
       "35                          I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&A sometimes leaves lingering doubts.   \n",
       "36                                    Iâ€™m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.   \n",
       "37         The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.   \n",
       "38                           I appreciate IT-supportâ€™s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.   \n",
       "39  The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.   \n",
       "40                               The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.   \n",
       "41                                  The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.   \n",
       "42                                  I value IT-supportâ€™s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.   \n",
       "43          The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.   \n",
       "44                           IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.   \n",
       "45                                I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.   \n",
       "46                                  The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.   \n",
       "47                                                    Connectivity is strong and dependable\\t however, periodic drops sow seeds of concern regarding data consistency.   \n",
       "48                                              Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.   \n",
       "49                             I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.   \n",
       "\n",
       "   AI Classification  \n",
       "0            Network  \n",
       "1              Other  \n",
       "2              Other  \n",
       "3              Other  \n",
       "4         IT-support  \n",
       "5            Network  \n",
       "6            Network  \n",
       "7              Other  \n",
       "8              Other  \n",
       "9         IT-support  \n",
       "10           Network  \n",
       "11          Training  \n",
       "12             Other  \n",
       "13          Training  \n",
       "14          Training  \n",
       "15        IT-support  \n",
       "16             Other  \n",
       "17             Other  \n",
       "18             Other  \n",
       "19           Network  \n",
       "20          Training  \n",
       "21           Network  \n",
       "22        IT-support  \n",
       "23             Other  \n",
       "24             Other  \n",
       "25          Training  \n",
       "26             Other  \n",
       "27             Other  \n",
       "28             Other  \n",
       "29          Training  \n",
       "30             Other  \n",
       "31           Network  \n",
       "32             Other  \n",
       "33          Training  \n",
       "34        IT-support  \n",
       "35          Training  \n",
       "36        IT-support  \n",
       "37             Other  \n",
       "38        IT-support  \n",
       "39             Other  \n",
       "40           Network  \n",
       "41        IT-support  \n",
       "42        IT-support  \n",
       "43           Network  \n",
       "44        IT-support  \n",
       "45          Training  \n",
       "46             Other  \n",
       "47           Network  \n",
       "48             Other  \n",
       "49        IT-support  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Categorize_others(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category_take1 : str = Field(\n",
    "        description=\"The best fitting category. Only one.\"\n",
    "    )\n",
    "    category_take2 : str = Field(\n",
    "        description = \"Do another take of the categorization. If the category is similar enough to any of the predetermined categories 'Network','Training', 'IT-support' then rather use one of those\"\n",
    "    )\n",
    "\n",
    "\n",
    "categorize_prompt_other = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback from an IT-survey into the category that best describes the feedback.\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "categorize_chain_structured_output_others = categorize_prompt_other | reasoning_llm.with_structured_output(\n",
    "    Categorize_others,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "# Retrieve indexes for all rows where AI classification = \"Other\"\n",
    "other_indices = df[df['AI Classification'] == \"Other\"].index\n",
    "\n",
    "\n",
    "result_batch = categorize_chain_structured_output_others.batch(\n",
    "    [\n",
    "    {\"feedback\": feedback} for feedback in df.loc[other_indices, \"Feedback\"]\n",
    "    ]\n",
    ")\n",
    "\n",
    "df.loc[other_indices, \"AI Classification\"] = [result.category_take2 for result in result_batch]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Feedback</th>\n",
       "      <th>AI Classification</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.</td>\n",
       "      <td>Infrastructure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Security</td>\n",
       "      <td>I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.</td>\n",
       "      <td>Security</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.</td>\n",
       "      <td>Performance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>Security</td>\n",
       "      <td>I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Security</td>\n",
       "      <td>The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Security</td>\n",
       "      <td>Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>Network</td>\n",
       "      <td>Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>Training</td>\n",
       "      <td>The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.</td>\n",
       "      <td>Technology-Business Alignment</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>Training</td>\n",
       "      <td>The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Training</td>\n",
       "      <td>The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Training</td>\n",
       "      <td>Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.</td>\n",
       "      <td>Strategy Alignment</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.</td>\n",
       "      <td>Usability</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>Network</td>\n",
       "      <td>The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>Training</td>\n",
       "      <td>While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>Network</td>\n",
       "      <td>While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>Security</td>\n",
       "      <td>I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>Security</td>\n",
       "      <td>The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.</td>\n",
       "      <td>Security</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>Training</td>\n",
       "      <td>The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>Security</td>\n",
       "      <td>The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>Security</td>\n",
       "      <td>Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.</td>\n",
       "      <td>Security</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>Training</td>\n",
       "      <td>Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>31</td>\n",
       "      <td>Security</td>\n",
       "      <td>Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.</td>\n",
       "      <td>Security</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>Network</td>\n",
       "      <td>I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>33</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>34</td>\n",
       "      <td>Training</td>\n",
       "      <td>The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>35</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>36</td>\n",
       "      <td>Training</td>\n",
       "      <td>I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&amp;A sometimes leaves lingering doubts.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>37</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>Iâ€™m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>38</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.</td>\n",
       "      <td>Strategy and Technology Alignment</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>39</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I appreciate IT-supportâ€™s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>40</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.</td>\n",
       "      <td>Other</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>41</td>\n",
       "      <td>Network</td>\n",
       "      <td>The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>42</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>43</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>I value IT-supportâ€™s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>44</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>45</td>\n",
       "      <td>IT-support</td>\n",
       "      <td>IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>46</td>\n",
       "      <td>Training</td>\n",
       "      <td>I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.</td>\n",
       "      <td>Training</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>47</td>\n",
       "      <td>Quality of tools</td>\n",
       "      <td>The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.</td>\n",
       "      <td>Others</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>48</td>\n",
       "      <td>Network</td>\n",
       "      <td>Connectivity is strong and dependable\\t however, periodic drops sow seeds of concern regarding data consistency.</td>\n",
       "      <td>Network</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>49</td>\n",
       "      <td>Security</td>\n",
       "      <td>Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.</td>\n",
       "      <td>Security</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>50</td>\n",
       "      <td>Business needs</td>\n",
       "      <td>I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.</td>\n",
       "      <td>IT-support</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    ID          Category  \\\n",
       "0    1           Network   \n",
       "1    2    Business needs   \n",
       "2    3          Security   \n",
       "3    4  Quality of tools   \n",
       "4    5        IT-support   \n",
       "5    6  Quality of tools   \n",
       "6    7          Security   \n",
       "7    8          Security   \n",
       "8    9          Security   \n",
       "9   10        IT-support   \n",
       "10  11           Network   \n",
       "11  12          Training   \n",
       "12  13    Business needs   \n",
       "13  14          Training   \n",
       "14  15          Training   \n",
       "15  16          Training   \n",
       "16  17    Business needs   \n",
       "17  18  Quality of tools   \n",
       "18  19  Quality of tools   \n",
       "19  20           Network   \n",
       "20  21          Training   \n",
       "21  22           Network   \n",
       "22  23        IT-support   \n",
       "23  24          Security   \n",
       "24  25          Security   \n",
       "25  26          Training   \n",
       "26  27  Quality of tools   \n",
       "27  28          Security   \n",
       "28  29          Security   \n",
       "29  30          Training   \n",
       "30  31          Security   \n",
       "31  32           Network   \n",
       "32  33    Business needs   \n",
       "33  34          Training   \n",
       "34  35        IT-support   \n",
       "35  36          Training   \n",
       "36  37  Quality of tools   \n",
       "37  38    Business needs   \n",
       "38  39        IT-support   \n",
       "39  40    Business needs   \n",
       "40  41           Network   \n",
       "41  42        IT-support   \n",
       "42  43        IT-support   \n",
       "43  44  Quality of tools   \n",
       "44  45        IT-support   \n",
       "45  46          Training   \n",
       "46  47  Quality of tools   \n",
       "47  48           Network   \n",
       "48  49          Security   \n",
       "49  50    Business needs   \n",
       "\n",
       "                                                                                                                                                              Feedback  \\\n",
       "0                                  The network infrastructure is robust and usually efficient, though intermittent glitches sometimes hamper productivity and clarity.   \n",
       "1                                      Our current infrastructure aligns with business needs, yet occasional rigidity in legacy systems hinders innovative approaches.   \n",
       "2                                    I appreciate the proactive approach to security, though the multi-step authentication process sometimes seems overly complicated.   \n",
       "3                                Our tools maintain high quality with intuitive design and regular updates, even though rare performance lags can hinder productivity.   \n",
       "4                               The IT-support team is consistently prompt and courteous, though occasional miscommunications lead to fleeting moments of uncertainty.   \n",
       "5                             Our advanced software tools offer a mix of efficiency and creativity, but the occasional downtime introduces a fleeting sense of dismay.   \n",
       "6                         I feel more secure with the recent network defense upgrades, yet the complexity of the security software occasionally overwhelms my routine.   \n",
       "7                            The new cybersecurity measures give me confidence in our data protection, though frequently changing protocols sometimes cause confusion.   \n",
       "8                                                  Our enhanced firewall settings significantly lower threats, though frequent reminders to update can feel intrusive.   \n",
       "9                                     The assistance from IT-support is generally effective, though sporadic technical mishaps occasionally undermine steady progress.   \n",
       "10                                   Network performance is generally satisfactory, but unexpected downtime during peak hours creates anxiety about meeting deadlines.   \n",
       "11                                           The training materials are succinct and creatively presented, but rapid changes in content sometimes create a disconnect.   \n",
       "12                          The integration of technology with our business goals is promising, though outdated processes occasionally clash with modern expectations.   \n",
       "13                                             The curriculum is robust and adapts to emerging trends, though it sometimes lacks tailored details specific to my role.   \n",
       "14                                             The hands-on exercises are engaging and boost confidence, though I occasionally struggle to keep up with the fast pace.   \n",
       "15                           Although the sessions are structured to be interactive, unexpected glitches disrupt the flow and leave me with some unanswered questions.   \n",
       "16                                           I recognize the efforts to merge IT and business strategies, though sporadic mismatches sometimes lead to project delays.   \n",
       "17                                 The digital tools consistently support day-to-day work, yet occasional interface inconsistencies open up space for minor critiques.   \n",
       "18                                     I appreciate the blend of innovation and stability in our tools, though sporadic integration issues sometimes test my patience.   \n",
       "19                       The network supports daily operations well, yet intermittent signal drops leave me with a mix of irritation and hope for prompt improvements.   \n",
       "20                      While the training sessions aim to cover a wide range of topics, their fast pacing occasionally results in uneven clarity and minor confusion.   \n",
       "21                                         While the network operates effectively most times, sporadic slow transmissions during heavy usage create brief instability.   \n",
       "22                                        While IT-support usually offers strong resolution strategies, intermittent communication gaps occasionally disrupt workflow.   \n",
       "23                              I value our firm stance on cybersecurity, even as a continuous stream of alerts occasionally turns reassurance into slight irritation.   \n",
       "24                                    The security team usually implements robust measures successfully, yet sporadic warnings create moments of heightened alertness.   \n",
       "25        The recent training sessions on new software updates provided clear guidance, though sometimes their rapid pace left me wishing for more practical examples.   \n",
       "26                                        The suite of tools is modern and user-friendly, though a few lagging behind expectations stir brief episodes of frustration.   \n",
       "27                                           The stringent security protocols inspire trust, even if the regular system prompts interrupt workflow more than I'd like.   \n",
       "28                                    Robust security protocols foster a safe system environment, but occasional delays in updates sometimes spark minor apprehension.   \n",
       "29                                                     Live workshops are full of energy and support, though scheduling conflicts now and then lead to mixed messages.   \n",
       "30                                            Timely security updates fortify our systems, even though brief periods of isolation during updates can be disconcerting.   \n",
       "31                                     I appreciate the steady connectivity, although occasional latency disrupts workflow and adds a layer of controlled frustration.   \n",
       "32               Our systems generally support business functions effectively, yet sporadic delays in adapting to new trends sometimes create operational bottlenecks.   \n",
       "33            The interactive training modules are well-designed but occasionally overwhelm with too many complex details, balancing excitement with mild frustration.   \n",
       "34                           Customer service from IT-support is engaging and resourceful, but the occasional oversight leaves tasks unfinished and briefly worrisome.   \n",
       "35                          I appreciate how the sessions cover both the basics and advanced features, yet the limited time for Q&A sometimes leaves lingering doubts.   \n",
       "36                                    Iâ€™m impressed by the robustness of our software tools, yet occasional glitches during critical operations disrupt my workflow.   \n",
       "37         The IT solutions seem aligned with our evolving business needs, though occasional mismatches in technology and strategy leave me seeking clearer direction.   \n",
       "38                           I appreciate IT-supportâ€™s readiness to tackle challenges, even though periodic delays in follow-up communications cloud the experience.   \n",
       "39  The balance between technical capability and business strategy is well-considered, though occasional oversights in market adaptability create subtle frustrations.   \n",
       "40                               The office network's reliability is commendable, yet sporadic slowdowns introduce brief moments of uncertainty during critical tasks.   \n",
       "41                                  The comprehensive responses from IT-support are quite reassuring, though backlog delays sometimes counteract the positive support.   \n",
       "42                                  I value IT-supportâ€™s clear follow-ups, yet delays in resolving complex issues sometimes leave a lingering sense of apprehension.   \n",
       "43          The high quality of our system tools fosters productivity, though minor and persistent connectivity challenges occasionally dampen the overall experience.   \n",
       "44                           IT-support often exceeds expectations with timely responses and practical solutions, though recurring issues still stir mild frustration.   \n",
       "45                                I find the training generally beneficial, yet at times the depth of content exceeds my comfort level and leaves parts underexplored.   \n",
       "46                                  The quality of our primary tools is commendable, though sporadic bugs and outdated interfaces sometimes temper initial enthusiasm.   \n",
       "47                                                    Connectivity is strong and dependable\\t however, periodic drops sow seeds of concern regarding data consistency.   \n",
       "48                                              Enhanced monitoring tools fortify our security practices, though occasional false alarms trigger unnecessary concerns.   \n",
       "49                             I appreciate the efforts to tailor IT solutions to our business model, even if delays in responsiveness sometimes generate uncertainty.   \n",
       "\n",
       "                    AI Classification  \n",
       "0                             Network  \n",
       "1                      Infrastructure  \n",
       "2                            Security  \n",
       "3                         Performance  \n",
       "4                          IT-support  \n",
       "5                             Network  \n",
       "6                             Network  \n",
       "7                            Training  \n",
       "8                             Network  \n",
       "9                          IT-support  \n",
       "10                            Network  \n",
       "11                           Training  \n",
       "12      Technology-Business Alignment  \n",
       "13                           Training  \n",
       "14                           Training  \n",
       "15                         IT-support  \n",
       "16                 Strategy Alignment  \n",
       "17                          Usability  \n",
       "18                         IT-support  \n",
       "19                            Network  \n",
       "20                           Training  \n",
       "21                            Network  \n",
       "22                         IT-support  \n",
       "23                         IT-support  \n",
       "24                           Security  \n",
       "25                           Training  \n",
       "26                         IT-support  \n",
       "27                         IT-support  \n",
       "28                           Security  \n",
       "29                           Training  \n",
       "30                           Security  \n",
       "31                            Network  \n",
       "32                         IT-support  \n",
       "33                           Training  \n",
       "34                         IT-support  \n",
       "35                           Training  \n",
       "36                         IT-support  \n",
       "37  Strategy and Technology Alignment  \n",
       "38                         IT-support  \n",
       "39                              Other  \n",
       "40                            Network  \n",
       "41                         IT-support  \n",
       "42                         IT-support  \n",
       "43                            Network  \n",
       "44                         IT-support  \n",
       "45                           Training  \n",
       "46                             Others  \n",
       "47                            Network  \n",
       "48                           Security  \n",
       "49                         IT-support  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df\n",
    "\n",
    "#Per nå bruker den kategoriene vi fikk fra LLMen. Når den kategorieserer Other får vi mange nye og litt varierende kategorier. Det er også mange av de. Skulle vi bare brukt den som eksempel også bruke den ekte kategoriseringe her?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ser du noen feilkategoriserte tilbakemeldiger? Eller ser det ut som vi kunne definert en til kategori som \"eksempel\" for modellen for å unngå så mange kategorier med små variasjoner. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Legger resultatene fra kategoriseringen av \"Others\" i en egen variabel som ikke har med de faktiske kategoriene.\n",
    "# Denne kan da brukes i oppgavene lengere ned uten å forvirre LLMen på hva som er kategoriene den skal kjenne igjen.\n",
    "categorized_survey = df[['ID','Feedback','AI Classification']]\n",
    "categorized_survey"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chain-of-thought\n",
    "\n",
    "Chain of Thought (CoT) er en teknikk innen prompt engineering som hjelper språkmodeller med å løse oppgaver som krever flere tankesteg. I stedet for å hoppe rett til svaret, blir modellen ledet gjennom en logisk og trinnvis prosess, noe som gir mer presise og gjennomtenkte svar – spesielt på komplekse problemer [1]. \n",
    "\n",
    "Du kan altså be modellen om å \"tenke høyt\" under oppgaven og forklare stegene sine før den leverer et endelig svar. Dette ber du om i prompten som sendes inn. \n",
    "\n",
    "Eksempel på en prompt **uten** CoT: \n",
    "\n",
    "    Prompt: \"Hvor mange armer har Eline og Kaspara?\"\n",
    "\n",
    "    Svar: \"4\"\n",
    "\n",
    "Eksempel på en prompt **med** CoT:\n",
    "\n",
    "    Prompt: \"Hvor mange armer har ELine og Kaspara? Tenk trinn for trinn.\"\n",
    "\n",
    "    Svar: \"En person har to armer. To personer betyr 2x2 = 4 armer. Svaret er 4.\"\n",
    "\n",
    "Det kan være fordelaktig å bruke CoT når man jobber med komplekse oppgaver, da nøyaktigheten på outputet fra modellen øker når den \"får lov\" til å jobbe seg gjennom problemet. Dette gir ofte bedre resultater på logiske oppgaver, eller oppgaver med flere steg. \n",
    "\n",
    "I tillegg kan du se hvordan modellen tenker, som gjør det lettere for deg å evaluere svaret. Det blir også lettere å se hvor det gikk galt hvis modellen svarer feil.\n",
    "\n",
    "### *Kilder*\n",
    "[1] xxxx, Link: https://www.ibm.com/think/topics/chain-of-thoughts \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CategorizeCot(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    chain_of_thought: str = Field(\n",
    "        description=\"Use this space to think through the categorization.\"\n",
    "    )\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  \n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_cot = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeCot,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = categorize_chain_cot.invoke(df[\"Feedback\"][0])\n",
    "result_no_cot = categorize_chain_structured_output2.invoke(df[\"Feedback\"][0])\n",
    "result.chain_of_thought\n",
    "result.category"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resonneringsmodell\n",
    "Resonneringsmodeller, som Azure Open AI sin O3-mini-modell, er språkmodeller som er spesielt trent på å tenke før de svarer. Slike modeller vil altså produsere en trinnvis tenkning før det endelige svaret leveres. Resonneringsmodeller er fordelaktige å bruke til oppgaver som krever kompleks problemløsning, logisk tenkning som koding eller matematikk eller til oppgaver med flere steg. De vil også være fordelaktige å bruke i situasjoner der nøyaktighet og forklarbarhet er viktig [2]. \n",
    "\n",
    "Dette kan minne om CoT, men det er en viktig forksjell her. CoT er en teknikk du kan bruke med språkmodeller for å gjøre dem bedre til å resonnere. Det er raskt og fleksibelt.\n",
    "Resonneringsmodeller er som nevnt en egen type modell som passer for oppgaver som krever presis og systematisk tenkning. Du vil få enda mer presise svar med en resonneringsmodell sammenlignet med CoT. \n",
    "\n",
    "#### Kilder\n",
    "[2] xxx, Link: https://platform.openai.com/docs/guides/reasoning?api-mode=chat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nå har vi definert passende kategorier for alle tilbakemeldingene. \n",
    "Videre vil vi få modellen til å oppsummere per kategori, slik at vi sitter igjen med en overordnet oversikt. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Din tur (forslag)\n",
    "'''Lag en LCEL-kjede som tar resultatet fra forrige oppgave (feedback med kategori) og lager en oppsummering\n",
    "per kategori ved hjelp av en LLM. Inkluder structured output.'''\n",
    "\n",
    "# Tips 1: Lag en god promt! Bruk dette til å reflektere over hva det endelige målet er. \n",
    "prompt = PromptTemplate.from_template(\"\"\"\n",
    "Ja, hvordan kan jeg formulere meg her da? Bør det ikke komme med noe data her også?\n",
    "\"\"\")\n",
    "\n",
    "# Tips 2: Kan du lage en klasse som arver fra BaseModel for å gjøre dette enklere?\n",
    "class SummarizeFeedback(BaseModel):\n",
    "    \"Beskrivelse...\"\n",
    "    summary : str = ...\n",
    "\n",
    "# Tips 3: På tide å lage kjeden\n",
    "summary_chain_structured_output = ...\n",
    "\n",
    "# Tips 4: Kall modellen med datasettet fra undersøkelsen\n",
    "summary = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Fasit\n",
    "# 1. Prompt Template\n",
    "summary_prompt = PromptTemplate.from_template(\"\"\"\n",
    "You are a domain expert in internal IT operations and organizational analysis. You will be provided with a dataset containing qualitative feedback from employees in an IT company. \n",
    "Each row in the dataset represents a feedback entry and is associated with a specific category.\n",
    "\n",
    "For each category, carefully:\n",
    "1. Read and interpret the feedback entries assigned to that category.\n",
    "2. Identify core themes, recurring patterns, and contrasting opinions within that category.\n",
    "3. Evaluate the feedback logically: What are the likely underlying causes of recurring issues or praises? Are there signs of systemic problems, isolated incidents, or misaligned expectations?\n",
    "4. Summarize each category in 3 to 6 bullet points, highlighting key sentiments (positive and negative), representative concerns or compliments, and any significant outliers\n",
    "\n",
    "Present your findings in a clean, professional way with one section per category. \n",
    "\n",
    "This is the employee feedback data: {survey_results}\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class SummarizeFeedback(BaseModel):\n",
    "    \"Summary of the different categorizes recognized in the feedback from an IT-survey.\"\n",
    "    summary : str = Field(\n",
    "        description=\"For each category: Category name and 3-6 bullet points summarizing the category.\"\n",
    "    )\n",
    "\n",
    "# 3. Summary-chain\n",
    "summary_chain_structured_output = summary_prompt | llm.with_structured_output(\n",
    "    SummarizeFeedback,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "# 4. Kall modellen med det kategoriserte datasettet fra undersøkelsen \n",
    "summary = summary_chain_structured_output.invoke({\"survey_results\": categorized_survey})\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Nice måte å vise outputten fra modellen \n",
    "from IPython.display import Markdown\n",
    "\n",
    "display(Markdown(summary.summary))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rapport til ledelsen\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Oppgave \n",
    "report_prompt = PromptTemplate.from_template(\"\"\"\n",
    "Bla bla bla\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class ReportForLeadership(BaseModel):\n",
    "    \"Raport for the leadership of an IT-company on results of an internal IT-survey.\"\n",
    "    snappy_title : str = Field(\n",
    "        description=\"A fitting title for the report. Must begin with '# ' to ensure easy markdown formatting.\"\n",
    "    )\n",
    "    key_takeaway:str = ...\n",
    "\n",
    "# 3. Report-chain\n",
    "report_chain_structured_output = ...\n",
    "\n",
    "# 4. Kall modellen med oppsummeringen av kategoriene\n",
    "report = ...\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fasit\n",
    "report_prompt = PromptTemplate.from_template(\"\"\"\n",
    "You are an expert HR and technical operations analyst. I will provide you with a dataset of employee feedback collected from an IT company.\n",
    "\n",
    "Your task is to deeply analyze this feedback and generate a concise executive-level summary report in markdown format that includes:\n",
    "\n",
    "1. Key Takeaways\n",
    "Provide a short summary of the overall feedback in 3-5 bullet points. Focus only on the main issues or areas of satisfaction.\n",
    "Include both positive and negative themes, but prioritize the most important and impactful points.\n",
    "Limit each point to 1-2 sentences.\n",
    "Before finalizing each point, take a moment to reflect on why each issue might be present (e.g., systemic problems, temporary issues, resource constraints, etc.)\n",
    "\n",
    "2. Suggested Improvements\n",
    "Based on the overall feedback, propose 2-3 high-level, actionable measures that the company could take to address the most pressing issues and enhance overall performance or satisfaction.\n",
    "Each suggestion should be brief, directly tied to the feedback, and strategic in nature.\n",
    "Think about short-term vs long-term solutions and consider the feasibility of each suggestion.\n",
    "\n",
    "3. Output\n",
    "Present your findings in a structured way with clear section headings, bullet points for easy scanning, and a consise, direct and professional tone suitable for leadership review.\n",
    "\n",
    "This is the employee feedback data: {summary_text}\n",
    "\"\"\")\n",
    "\n",
    "# 2. Class for structured output\n",
    "class ReportForLeadership(BaseModel):\n",
    "    \"Raport for the leadership of an IT-company on results of an internal IT-survey.\"\n",
    "    snappy_title : str = Field(\n",
    "        description=\"A fitting title for the report. Must begin with '# ' to ensure easy markdown formatting.\"\n",
    "    )\n",
    "    intro : str = Field(\n",
    "        description=\"1 sentence describing thepurpose of the report.\" \n",
    "    )\n",
    "    key_takeaways : str = Field(\n",
    "        description=\"3-5 bulletpoints describing the key-takeaways. Limit each point to 1-2 sentences.\" \n",
    "    )\n",
    "    suggested_improvements : str = Field(\n",
    "        description=\"2-3 actionable measures for the company. Keep it brief.\"\n",
    "    )\n",
    "    outro: str = Field(\n",
    "        description=\"1 sentence ending for the report. Be creative.\" \n",
    "    )\n",
    "\n",
    "# 3. Report-chain\n",
    "report_chain_structured_output = report_prompt | llm.with_structured_output(\n",
    "    ReportForLeadership,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")\n",
    "\n",
    "# 4. Kall modellen med oppsummeringen av kategoriene\n",
    "report = report_chain_structured_output.invoke({\"summary_text\": summary.summary})\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(\n",
    "    \"\\n\\n\".join([report.snappy_title,\n",
    "                 report.intro,\n",
    "                 '## Key-takeaways',report.key_takeaways,\n",
    "                 '## Suggested improvements',report.suggested_improvements,\n",
    "                 report.outro])\n",
    "                 \n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hvordan kunne vi gjort dette bedre?\n",
    "\n",
    "Anta at du har fått levert resultatene fra denne spørreundersøkelsen i fanget av en stressa mellomleder som ber deg levere en rapport han kan presentere ledelsen. \n",
    "Gitt verktøyene du har fått en innføring i gjennom dette kursene (og kanskje andre erfaringer?), hvordan ville du løst oppgaven?\n",
    "\n",
    "Ser du for eksempel noe som kunne vært forbedret i\n",
    "- Rekkefølgen på måten vi leter etter kategorier?\n",
    "- Legger vi får mye/lite vekt på inputen vi fikk om hva ledelsen \"tror\" kategoriene kommer til å være?\n",
    "- Promptingen?\n",
    "- Variablene eller type hintingen i pydantic-klassene?\n",
    "\n",
    "Ville du kanskje gjort det helt annerledes? \n",
    "Now's your chance to try!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prøv deg frem :))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hva er risikoene ved bruk av LLM til denne typen analyse?\n",
    "Gi et lite blikk på bias, hallucination og behov for kvalitetssikring."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hva er risikoene ved bruk av LLM til denne typen analyse?\n",
    "Ved bruk av store språkmodeller (LLM) til analyse av åpne tekstsvar i spørreundersøkelser er risikoen generelt lavere enn i mer kritiske anvendelser som faktagenerering eller beslutningsstøtte. Likevel finnes det enkelte utfordringer som bør tas på alvor for å sikre kvalitet og transparens i analysen.\n",
    "\n",
    "Skjevheter og bias\n",
    "LLM-er kan tolke og vekte visse formuleringer eller temaer ulikt, basert på mønstre fra treningsdataene. Dette kan føre til at noen svar over- eller underrepresenteres i visse kategorier, avhengig av hvordan modellen \"forstår\" teksten. For eksempel kan emosjonelt ladede eller språklig avanserte svar få uforholdsmessig stor plass, mens mer nøkterne eller utypiske svar blir feilklassifisert. Det er viktig å være oppmerksom på slike tendenser, særlig ved tolkning av følsomme temaer.\n",
    "\n",
    "Hallusinasjon – mindre relevant, men ikke fraværende\n",
    "I denne typen oppgave, hvor modellen hovedsakelig skal kategorisere eller oppsummere eksisterende tekstsvar, er risikoen for såkalte hallusinasjoner (altså at modellen «finner opp» informasjon) mindre, men ikke helt ubetydelig. Det kan forekomme at modellen overfortolker et kort eller tvetydig svar og tillegger det en mening som ikke eksplisitt finnes i teksten. Dette kan føre til at enkelte svar havner i feil kategori eller påvirker tolkningen av sentiment eller tematikk.\n",
    "\n",
    "Behov for kvalitetssikring\n",
    "Selv med høy presisjon og effektivitet, bør modellens resultater ikke tas for gitt. Det anbefales å gjennomføre stikkprøver, sammenligne med manuell koding, og evaluere klassifiseringsnøyaktigheten for å sikre at resultatene er konsistente og representative. Det kan også være nyttig å kombinere automatisk kategorisering med menneskelig vurdering, spesielt i pilotfasen."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
