{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM for alle - Introduksjonskurs til språkmodeller med Python og Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spørsmål\n",
    "- Vi trenger at dte går fortere å kalle på modellen :)\n",
    "- Hvordan gjør vi det med nøkler? Per nå leser denne fortsatt fra .env-filen da jeg ikke ville pushe de til github.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tanker\n",
    "\n",
    "- Gi mer info i prompt om situasjonen rundt spørreundersøkelsen og hva det har blitt spurt om. Spesielt i situasjonen hvor vi øsnker å oppsummere den generelle viben av feedbacken i hver kategori. Var folk fornøyde? Ønsker de tiltak for forbedring?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Oppgave\n",
    "Du jobber i et IT-selskap og har fått i oppgave å analysere svarene fra en intern medarbeiderundersøkelse. Undersøkelsen er anonym, og du har fått tilsendt en CSV-fil med 50 tilbakemeldinger – én per ansatt. Målet er å finne ut hva folk er fornøyde eller misfornøyde med, og særlig se nærmere på temaene Nettverk, Opplæring og IT-support, som ledelsen er ekstra interessert i. Tilbakemeldinger som ikke passer i disse kategoriene skal også få sin plass. Til slutt skal du lage en oppsummering som kan sendes til ledelsen.\n",
    "\n",
    "For å jobbe effektivt bruker du en språkmodell til å hjelpe deg med både kategorisering og oppsummering. \n",
    "\n",
    "**Oppgaven blir dermed å bruke en språkmodell til å kategorisere samt oppsummere tilbakemeldingene fra undersøkelsen.** \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datasett"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importerer rådataen med tilbakemeldinger, en rad per ansatt. Alle ansatte har svart på undersøkelsen. \n",
    "\n",
    "import pandas as pd\n",
    "enr_path = \"../files/random_out.csv\"\n",
    "df = pd.read_csv(enr_path)\n",
    "df.head()\n",
    "\n",
    "## Using a language model through the API ???"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a language model through the API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO: Intro til språkmodeller\n",
    "- Slik kaller man en språkmodell\n",
    "- Hva er temperaturparameteren"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import AzureChatOpenAI\n",
    "from dotenv import find_dotenv, load_dotenv\n",
    "import os\n",
    "\n",
    "\n",
    "# Get environment variables\n",
    "load_dotenv(find_dotenv(), override=True)\n",
    "\n",
    "\n",
    "llm = AzureChatOpenAI(\n",
    "    azure_deployment=\"gpt-4o-mini\",\n",
    "    model=os.environ.get(\"OPENAI_MODEL_GPT_4O-MINI\", default=\"gpt-4o-mini\"),\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "reasoning_llm = AzureChatOpenAI(\n",
    "    azure_deployment=\"o3-mini\",\n",
    "    model=\"o3-mini\",\n",
    "    reasoning_effort=\"medium\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = 'Hei!'\n",
    "response = llm.invoke(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a basic chain with Langchain Expression Language (LCEL)\n",
    "\n",
    "TODO:\n",
    "- Fordelene med LCEL\n",
    "- Forklare pipe-operatoren\n",
    "- Forklare PromptTemplate\n",
    "\n",
    "\n",
    "Vurdere å droppe?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "prompt = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Hi! Please talk like a {role}.\n",
    "    \"\"\"\n",
    ")\n",
    "prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm\n",
    "\n",
    "chain.invoke({\"role\": \"pirate\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "chain2 = chain | StrOutputParser()\n",
    "\n",
    "chain2.invoke({\"role\": \"pirate\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batching and streaming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain2.batch(\n",
    "    [\n",
    "        {\"role\": \"pirate\"},\n",
    "        {\"role\": \"cowboy\"},\n",
    "        {\"role\": \"ninja\"},\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import sleep\n",
    "for chunk in chain2.stream({\"role\": \"pirate\"}):\n",
    "    print(chunk, end=\"\")\n",
    "    sleep(0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple categorization of each survey reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorize_prompt = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into one of the following categories:\n",
    "- Network\n",
    "- Training\n",
    "- IT-support\n",
    "- Other\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "categorize_chain = categorize_prompt | llm | StrOutputParser()\n",
    "\n",
    "categorize_chain.invoke({\"feedback\": \"I am very happy with the IT support I received last week.\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Structured output\n",
    "\n",
    "TODO\n",
    "- Hvorfor trenger man strukturert output\n",
    "- Forklare hva Pydantic er\n",
    "- FOrklare hvordan Pydantic brukes for å få strukturert output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "\n",
    "class Categorize(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category : str = Field(description=\"The best fitting general category. Only one.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Categorize(category=\"Network\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorize_chain_structured_output = categorize_prompt | llm.with_structured_output(\n",
    "    Categorize,\n",
    "    method=\"json_schema\",\n",
    "    strict=True\n",
    ")\n",
    "\n",
    "categorize_chain_structured_output.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO:\n",
    "- Forklare hva Literal er"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "CATEGORIES = Literal[\n",
    "    \"Network\",\n",
    "    \"Training\",\n",
    "    \"IT-support\",\n",
    "    'Other'\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Literal\n",
    "\n",
    "categorize_prompt2 = PromptTemplate.from_template(\n",
    "\"\"\"\n",
    "Categorize the following feedback into the provided categories.\n",
    "\n",
    "Feedback:\n",
    "<feedback>\n",
    "{feedback}\n",
    "</feedback>\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "\n",
    "\n",
    "class CategorizeFromOptions(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  ## noqa: E501\n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_structured_output2 = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeFromOptions,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = categorize_chain_structured_output2.invoke(\n",
    "    {\"feedback\": \"I am very happy with the IT support I received last week.\"}\n",
    ")\n",
    "\n",
    "result\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate performance across whole dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def categorize_single_feedback(feedback: str) -> str:\n",
    "    result = categorize_chain_structured_output2.invoke(\n",
    "        {\"feedback\": feedback}\n",
    "    )\n",
    "    return result.category\n",
    "\n",
    "df[\"AI Classification\"] = df[\"Feedback\"].apply(\n",
    "    lambda feedback: categorize_single_feedback(feedback)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(df[\"AI Classification\"] == df[\"Category\"]).value_counts()\n",
    "\n",
    "#TODO: Replace this be an evaluation function which is imported"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chain-of-thought\n",
    "\n",
    "Chain of Thought (CoT) er en teknikk innen prompt engineering som hjelper språkmodeller med å løse oppgaver som krever flere tankesteg. I stedet for å hoppe rett til svaret, blir modellen ledet gjennom en logisk og trinnvis prosess, noe som gir mer presise og gjennomtenkte svar – spesielt på komplekse problemer [1]. \n",
    "\n",
    "Du kan altså be modellen om å \"tenke høyt\" under oppgaven og forklare stegene sine før den leverer et endelig svar. Dette ber du om i prompten som sendes inn. \n",
    "\n",
    "Eksempel på en prompt **uten** CoT: \n",
    "\n",
    "    Prompt: \"Hvor mange armer har Eline og Kaspara?\"\n",
    "\n",
    "    Svar: \"4\"\n",
    "\n",
    "Eksempel på en prompt **med** CoT:\n",
    "\n",
    "    Prompt: \"Hvor mange armer har ELine og Kaspara? Tenk trinn for trinn.\"\n",
    "\n",
    "    Svar: \"En person har to armer. To personer betyr 2x2 = 4 armer. Svaret er 4.\"\n",
    "\n",
    "Det kan være fordelaktig å bruke CoT når man jobber med komplekse oppgaver, da nøyaktigheten på outputet fra modellen øker når den \"får lov\" til å jobbe seg gjennom problemet. Dette gir ofte bedre resultater på logiske oppgaver, eller oppgaver med flere steg. \n",
    "\n",
    "I tillegg kan du se hvordan modellen tenker, som gjør det lettere for deg å evaluere svaret. Det blir også lettere å se hvor det gikk galt hvis modellen svarer feil.\n",
    "\n",
    "### *Kilder*\n",
    "[1] xxxx, Link: https://www.ibm.com/think/topics/chain-of-thoughts \n",
    "\n",
    "---\n",
    "TODO:\n",
    "- Forklare hva chain-of-thought er og hvorfor det kan være nyttig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CategorizeCot(BaseModel):\n",
    "    \"Categorization of a single feedback entry from an IT survey.\"\n",
    "    chain_of_thought: str = Field(\n",
    "        description=\"Use this space to think through the categorization.\"\n",
    "    )\n",
    "    category: CATEGORIES = Field(\n",
    "        description=\"Chosen category for the feedback. Choose 'Other' if the other categories provided are not a good fit.\"  ## noqa: E501\n",
    "    )\n",
    "\n",
    "\n",
    "categorize_chain_cot = categorize_prompt2 | llm.with_structured_output(\n",
    "    CategorizeCot,\n",
    "    method=\"json_schema\",\n",
    "    strict=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorize_chain_cot.invoke(df[\"Feedback\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result.category"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resonneringsmodell\n",
    "Resonneringsmodeller, som Azure Open AI sin O3-mini-modell, er språkmodeller som er spesielt trent på å tenke før de svarer. Slike modeller vil altså produsere en trinnvis tenkning før det endelige svaret leveres. Resonneringsmodeller er fordelaktige å bruke til oppgaver som krever kompleks problemløsning, logisk tenkning som koding eller matematikk eller til oppgaver med flere steg. De vil også være fordelaktige å bruke i situasjoner der nøyaktighet og forklarbarhet er viktig [2]. \n",
    "\n",
    "Dette kan minne om CoT, men det er en viktig forksjell her. CoT er en teknikk du kan bruke med språkmodeller for å gjøre dem bedre til å resonnere. Det er raskt og fleksibelt.\n",
    "Resonneringsmodeller er som nevnt en egen type modell som passer for oppgaver som krever presis og systematisk tenkning. Du vil få enda mer presise svar med en resonneringsmodell sammenlignet med CoT. \n",
    "\n",
    "#### Kilder\n",
    "[2] xxx, Link: https://platform.openai.com/docs/guides/reasoning?api-mode=chat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Kategoriser radene som ligger i *Other*\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nå har vi definert passende kategorier for alle tilbakemeldinhgene. \n",
    "#Videre vil vi få modellen til å oppsummere per kategori, slik at vi sitter igjen med en overordnet oversikt. \n",
    "\n",
    "from langchain_core.prompts import PromptTemplate\n",
    "from langchain_core.output_parsers import JsonOutputKeyParser\n",
    "\n",
    "# Din tur (forslag)\n",
    "'''Lag en LCEL-kjede som tar resultatet fra forrige oppgave (feedback med kategori) og lager en oppsummering\n",
    "## per kategori ved hjelp av en LLM. Inkluder structured output.'''\n",
    "\n",
    "    # Tips: Begynn med en funksjon som lager LCEL-kjeden\n",
    "def summary_chain(x, y):\n",
    "    prompt = ___\n",
    "\n",
    "    # Tips 2: Bruk structured output med method = \"function_calling\"\n",
    "\n",
    "\n",
    "# Fasit\n",
    "## OBS: bytt ut feedback_txt med resultat fra forrige oppgave\n",
    "def build_summary_chain(struktur, llm_model):\n",
    "    # 1. Prompt Template\n",
    "    prompt = PromptTemplate.from_template(\"\"\"\n",
    "You are a domain expert in internal IT operations and organizational analysis. You will be provided with a dataset containing qualitative feedback from employees in an IT company. \n",
    "Each row in the dataset represents a feedback entry and is associated with a specific category.\n",
    "\n",
    "For each category, carefully:\n",
    "1. Read and interpret the feedback entries assigned to that category.\n",
    "2. Identify core themes, recurring patterns, and contrasting opinions within that category.\n",
    "3. Evaluate the feedback logically: What are the likely underlying causes of recurring issues or praises? Are there signs of systemic problems, isolated incidents, or misaligned expectations?\n",
    "4. Summarize each category in 3 to 6 bullet points, highlighting key sentiments (positive and negative), representative concerns or compliments, and any significant outliers\n",
    "\n",
    "Present your findings in a clean, professional way with one section per category. \n",
    "\n",
    "This is the employee feedback data: {feedback_txt}\n",
    "\"\"\")\n",
    "\n",
    "    # 2. Structured output LLM\n",
    "    structured_llm = llm_model.with_structured_output(struktur, method=\"function_calling\")\n",
    "\n",
    "    # 3. LCEL Chain\n",
    "    chain = prompt | structured_llm\n",
    "\n",
    "    return chain\n",
    "\n",
    "chain = build_summary_chain(struktur, llm_model)\n",
    "response = chain.invoke({\"feedback_txt\": feedback_txt})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ekstraoppgave\n",
    "'''Bruk oppsummeringen til å lage en rapport som kan sendes til ledelsen med forslag til endringer for å forbedre\n",
    "resultatene på neste års undersøkelse'''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Herfra og ned er gammel kode. kan trolig slettes men tørr ikke helt enda. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "structured_llm = Llm.llm.with_structured_output(   #Gå igjennom denne, hvordan bruker man structured output?\n",
    "        Categorize, method=\"function_calling\"\n",
    "    )\n",
    "\n",
    "response = structured_llm.invoke(\"Send inn prompt + data\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Find potential new categories based on feedback categorized as others"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CategorizeFromOptions2(BaseModel):\n",
    "    \"\"\"Categorization of IT-questionaire feedback.\"\"\"\n",
    "\n",
    "    category: Literal[\n",
    "       # Insert the categories you found while exploring the feedback initially categorized as 'Other'.\n",
    "    ] = Field(\n",
    "        description=\"Categorize the feedback into the most fitting category. If the categories provided are not a perfect fit, default to 'Other'.\"\n",
    "    )\n",
    "    reason : str = Field(description=\"Why do you believe this category is the most fitting one? Explain.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Oppgave??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "from typing import Literal, Optional\n",
    "\n",
    "# Kan de mekke på disse selv? Hvordan kan vi gjøre dette steget mer interaktivt\n",
    "\n",
    "\n",
    "class Categorize_search(BaseModel):\n",
    "    \"\"\"Categorization of IT-questionaire feedback.\"\"\"\n",
    "\n",
    "    categories: Literal[\n",
    "        \"Network\",\n",
    "        \"IT Training\",\n",
    "        \"IT Support\",\n",
    "        'Other'\n",
    "    ] = Field(\n",
    "        description=\"Categorize the feedback into the most fitting category. If the categories provided are not a perfect fit, default to 'Other'.\"\n",
    "    )\n",
    "    #if_other: str = Field(\n",
    "        #description=\"If you chose to categorize the feedback as 'Other', return an explanation as to why and what category you think would be the best fit for the feedback.\"\n",
    "    #)\n",
    "\n",
    "\n",
    "class Categorize_bound(BaseModel):\n",
    "    # Begrenser kategoriene modellen kan velge mellom. Den får kun lov til å putte feedbacken i en av de forhåndsbestemte kategoriene.\n",
    "    \"\"\"Categorization of IT-questionaire feedback.\"\"\"\n",
    "\n",
    "    categories: Literal[\n",
    "        \"Cyber security\",\n",
    "        \"IT training\",\n",
    "        \"IT support\",\n",
    "        \"Quality of technology\",\n",
    "        \"Data quality\",\n",
    "        \"Network\",\n",
    "    ] = Field(description=\"Categorize the feedback into the most fitting category.\")\n",
    "   # rating: Optional[int] = Field(\n",
    "       # description=\"Your certainty of the corectness of the best category on a scale from 1-10\"\n",
    "  #  )\n",
    "    #reason: str = Field(\n",
    "    #    description=\"Give a short scentence as to why you think this category is the best fit.\"\n",
    "   # )\n",
    "\n",
    "\n",
    "\n",
    "class Categorize(BaseModel):\n",
    "    \"Categorization of feedback on IT-services.\"\n",
    "\n",
    "    # Thoughts: Forklar tankegangen din.\n",
    "    cat_1: str = Field(description=\"The best fitting general category. Only one.\")\n",
    "    \n",
    "class Summarize(BaseModel):\n",
    "\n",
    "    # 1. Oppsummer kategori, tar inn alle feedbacks under 1 kategori\n",
    "    # 2. Likely underlying causes of recurring issues or praises\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Case"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kategorisering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lag en funksjon som tar inn enten full prompt eller data fra undersøkelsen + pydantic objekt (struct) + modell og returnerer output fra modell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## KURS\n",
    "def categorize_feedback(feedback_txt : str, struktur, llm_model) -> dict:\n",
    "    # Prompt\n",
    "    task = f\"\"\"\n",
    "    PROMT\n",
    "    Catgorize the feedback: {feedback_txt}.\n",
    "    \"\"\"\n",
    "\n",
    "    # Defining structured output for the model\n",
    "    structured_llm = llm_model.with_structured_output(   #Gå igjennom denne, hvordan bruker man structured output?\n",
    "        struktur, method=\"function_calling\"\n",
    "    )\n",
    "\n",
    "    # Prompt model\n",
    "    response = structured_llm.invoke(task)\n",
    "    # Return the response\n",
    "    return response.__dict__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def categorize_feedback(feedback_txt : str, struktur, llm_model) -> dict:\n",
    "    # Prompt\n",
    "    task = f\"\"\"\n",
    "    The following feedback is from an internal survey at 'IT and Things Company' where they asked their employees for feedback on their IT-services in general.\n",
    "    Catgorize the feedback: {feedback_txt}.\n",
    "    \"\"\"\n",
    "\n",
    "    # Defining structured output for the model\n",
    "    structured_llm = llm_model.with_structured_output(\n",
    "        struktur, method=\"function_calling\"\n",
    "    )\n",
    "\n",
    "    # Prompt model\n",
    "    response = structured_llm.invoke(task)\n",
    "    # Return the response\n",
    "    return response.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Din tur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Returner en dataframe med kategoriserte feedback \n",
    "# 1. Be den kategorisere uten begrensninger\n",
    "# 2. Gi den et sett med forhåndsdefinerte kategorier\n",
    "# 3. La 'other' være et alternativ. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## returnerer kategorisert feedback\n",
    "\n",
    "responses = []\n",
    "\n",
    "for f in FeedbackData.df[\"Feedback\"]:\n",
    "    feedback_4O = categorize_feedback(feedback_txt = f, struktur = Categorize_search, llm_model = Llm.llm)\n",
    "    feedback_3O = categorize_feedback(feedback_txt = f, struktur = Categorize_search, llm_model = LlmRes.llm)\n",
    "    responses.append({'Feedback': f, 'Category_4O': feedback_4O['categories'], 'Category_3O': feedback_3O['categories']})\n",
    "    #print(f)\n",
    "\n",
    "    \n",
    "cat_df = pd.DataFrame(responses)\n",
    "print(cat_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Identifiser de som ble identifisert som 'other' og finn kategori for de. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "others_df = cat_df[(cat_df['Category'] == 'Other') | (cat_df['Category2'] == 'Other')]\n",
    "print(others_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## returnerer kategorisert feedback\n",
    "\n",
    "# Heller send inn alle others-feedback samtidig og be den finne X nye kategorier. \n",
    "responses_new_cat = []\n",
    "\n",
    "for f in others_df[\"Feedback\"]:\n",
    "    output_i = categorize_feedback(feedback_txt = f, struktur = Categorize, llm_model = Llm.llm)\n",
    "    responses_new_cat.append({'Feedback': f,'Category': output_i[\"cat_1\"]})\n",
    "\n",
    "new_cat_df = pd.DataFrame(responses_new_cat)\n",
    "print(new_cat_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ny kategorisering hvor vi gir den grunnkategorier + de modellen har gjenkjent under other "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reasoning models - when we want the LLM to return logic analyzation across rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nå vil vi teste resoneringsmodellen, sammenlikne?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summarization per category"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Få modellen til å oppsummere per kategori."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now that we have defined fitting categories for the feedback, we can use a reasoning model to create a summary of the rows per category to provide an overall overview\n",
    "def summarize(feedback_txt : str, struktur, llm_model) -> dict:\n",
    "\n",
    "    # Prompt\n",
    "    task = f\"\"\"\n",
    "You are a domain expert in internal IT operations and organizational analysis. You will be provided with a dataset containing qualitative feedback from employees in an IT company. \n",
    "Each row in the dataset represents a feedback entry and is associated with a specific category.\n",
    "\n",
    "For each category, carefully:\n",
    "1. Read and interpret the feedback entries assigned to that category.\n",
    "2. Identify core themes, recurring patterns, and contrasting opinions within that category.\n",
    "3. Evaluate the feedback logically: What are the likely underlying causes of recurring issues or praises? Are there signs of systemic problems, isolated incidents, or misaligned expectations?\n",
    "4. Summarize each category in 3 to 6 bullet points, highlighting key sentiments (positive and negative), representative concerns or compliments, and any significant outliers\n",
    "\n",
    "Present your findings in a clean, professional way with one section per category. \n",
    "\n",
    "This is the employee feedback data: {feedback_txt}\n",
    "    \"\"\"\n",
    "\n",
    "    # Defining structured output for the model\n",
    "    structured_llm = llm_model.with_structured_output(\n",
    "        struktur, method=\"function_calling\"\n",
    "    )\n",
    "\n",
    "    # Prompt model\n",
    "    response_sum = structured_llm.invoke(task)\n",
    "    # Return the response\n",
    "    return response_sum.__dict__\n",
    "\n",
    "# Print head of response\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating a report for the leadership"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Bruk oppsummeringen til å generere en en rapport til ledelsen med forlag til forbedringspotensiale i IT. Hvilke tiltak bør bedriften gjøre?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def report(feedback_txt : str, struktur, llm_model) -> dict:\n",
    "\n",
    "    # Prompt\n",
    "    task = f\"\"\"\n",
    "You are an expert HR and technical operations analyst. I will provide you with a dataset of employee feedback collected from an IT company.\n",
    "\n",
    "Your task is to deeply analyze this feedback and generate a concise executive-level summary report in markdown format that includes:\n",
    "\n",
    "1. Key Takeaways\n",
    "Provide a short summary of the overall feedback in 3-5 bullet points. Focus only on the main issues or areas of satisfaction.\n",
    "Include both positive and negative themes, but prioritize the most important and impactful points.\n",
    "Limit each point to 1-2 sentences.\n",
    "Before finalizing each point, take a moment to reflect on why each issue might be present (e.g., systemic problems, temporary issues, resource constraints, etc.)\n",
    "\n",
    "2. Suggested Improvements\n",
    "Based on the overall feedback, propose 2-3 high-level, actionable measures that the company could take to address the most pressing issues and enhance overall performance or satisfaction.\n",
    "Each suggestion should be brief, directly tied to the feedback, and strategic in nature.\n",
    "Think about short-term vs long-term solutions and consider the feasibility of each suggestion.\n",
    "\n",
    "3. Output\n",
    "Present your findings in a structured way with clear section headings, bullet points for easy scanning, and a consise, direct and professional tone suitable for leadership review.\n",
    "\n",
    "This is the employee feedback data: {feedback_txt}\n",
    "    \"\"\"\n",
    "\n",
    "    # Defining structured output for the model\n",
    "    structured_llm = llm_model.with_structured_output(\n",
    "        struktur, method=\"function_calling\"\n",
    "    )\n",
    "\n",
    "    # Prompt model\n",
    "    response = structured_llm.invoke(task)\n",
    "    # Return the response\n",
    "    return response.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Skulle man hatt en oppgave til slutt for de som er fort ferdig hvor de sender inn hele undersøkelsen og ber modellen velge beste kategorier før vi igjen bruker de kategoriene i structured output for å kategorisere. kjør så hele på nytt og se om du synes resultatene ble bedre. \n",
    "\n",
    "2. Hvordan kunne dette vært gjort bedre? Be LLMen si hva som kunne vært gjort annerledes for å skape mer verdi. \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
