{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NeSy4PPM: Suffix prediction Tutorial\n",
    "This notebook demonstrates how to use the NeSy4PPM framework for suffix prediction, specifically focused on activity and resource or activity-only prediction using neural architectures like LSTM and Transformer models. NeSy4PPM combines neural learning with symbolic background knowledge (BK) to produce accurate and compliant predictions under various conditions, including concept drift.\n",
    "\n",
    "This notebook walks through the entire NeSy4PPM pipeline, including:\n",
    "\n",
    "    1. Data preprocessing\n",
    "    2. Training\n",
    "    3. Prediction\n",
    "    4. Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 Data preprocessing\n",
    "The __Data preprocessing__ is responsible for loading and transforming event log into neural-compatible inputs. This phase involves:\n",
    "\n",
    "   - `Data loading`, which involves reading the event log from the input file,\n",
    "   - `Prefixes extraction`,which refers to extracting prefixes (i.e., partial trace executions) from the training log.\n",
    "   - `Prefixes encoding`,which handles the conversion of input prefixes and their corresponding labels into numerical representations suitable for neural models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Data loading\n",
    "The first step in the __data preprocessing__ pipeline is to load the event log (in a .xes, .csv or .xes.gz files) using the `LogData` class. The input of this step can be:\n",
    "- A __single event log__, which will be automatically split into training and test sets based on the chronological order of case start timestamps and a defined train ratio.\n",
    "- A pair of __separate training and test logs__."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Single event log:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T11:03:28.422201Z",
     "start_time": "2025-06-28T11:03:24.919846Z"
    }
   },
   "source": [
    "from pathlib import Path\n",
    "from NeSy4PPM.Data_preprocessing import log_utils\n",
    "\n",
    "log_path = Path.cwd().parent/'data'/'input'/'logs'\n",
    "log_name = \"helpdesk.xes\"\n",
    "train_ratio = 0.8\n",
    "case_name_key = 'case:concept:name'\n",
    "act_name_key = 'concept:name'\n",
    "res_name_key = 'org:resource'\n",
    "timestamp_key = 'time:timestamp'\n",
    "\n",
    "log_data = log_utils.LogData(log_path=log_path,log_name=log_name,train_ratio=train_ratio,\n",
    "                             case_name_key=case_name_key,act_name_key=act_name_key,\n",
    "                             res_name_key=res_name_key,timestamp_key=timestamp_key)\n",
    "print(f\"Loaded log: {log_data.log_name}\")\n",
    "print(f\"Trace max size: {log_data.max_len}\")"
   ],
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "parsing log, completed traces :: 100%|██████████| 4580/4580 [00:01<00:00, 2715.77it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded log: helpdesk\n",
      "Trace max size: 15\n"
     ]
    }
   ],
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Separate training and test logs:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:03:21.614995Z",
     "start_time": "2025-06-28T13:03:20.414310Z"
    }
   },
   "source": [
    "from pathlib import Path\n",
    "from NeSy4PPM.Data_preprocessing import log_utils\n",
    "\n",
    "log_path = Path.cwd().parent/'data'/'input'/'logs'\n",
    "train_log = \"helpdesk_train.xes\"\n",
    "test_log = \"helpdesk_test.xes\"\n",
    "\n",
    "log_data = log_utils.LogData(log_path=log_path,train_log=train_log,test_log=test_log)\n",
    "print(f\"Loaded log: {log_data.log_name}\")\n",
    "print(f\"Trace max size: {log_data.max_len}\")"
   ],
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "parsing log, completed traces :: 100%|██████████| 3664/3664 [00:00<00:00, 5390.36it/s]\n",
      "parsing log, completed traces :: 100%|██████████| 820/820 [00:00<00:00, 10806.80it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded log: helpdesk_train\n",
      "Trace max size: 15\n"
     ]
    }
   ],
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Prefixes extraction\n",
    "The `extract_trace_prefixes` function extracts all possible prefixes from each trace in the training log, up to a predefined maximum length. These prefixes represent partial executions of cases and are used as inputs to the neural model."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:03:28.323294Z",
     "start_time": "2025-06-28T13:03:28.051152Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import extract_trace_prefixes\n",
    "\n",
    "extracted_prefixes = extract_trace_prefixes(log_data=log_data, resource=True)"
   ],
   "outputs": [],
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Prefixes encodings\n",
    "Before training a neural model, extracted prefixes must be converted into vectorized formats. NeSy4PPM supports four encoding techniques for multi-attribute: \n",
    "- `One-hot encoding`,\n",
    "- `Index-based encoding`,\n",
    "- `Shrinked index-based encoding`, \n",
    "- `Multi-encoders encoding`.\n",
    "\n",
    "Each encoding is implemented via the function `encode_prefixes` and prepares both input features (`x`) and targets labels: `y_a` for activity prediction and `y_g` for resource prediction when `resource = True`. Note that only `One-hot` and `Index-based` encodings are applicable when performing activity-only prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One-hot encoding\n",
    "In the __One-hot encoding__, sequences of events are converted into high-dimensional binary feature vectors. Each feature corresponds to a concatenation of one-hot encoded activity and resource values derived from the log. To apply index-based encoding, set the `encoder` parameter to `Encodings.One_hot` when calling the `encode_prefixes` function:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:04:00.430360Z",
     "start_time": "2025-06-28T13:04:00.217942Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import encode_prefixes\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "encoder=Encodings.One_hot\n",
    "x, y_a, y_g= encode_prefixes(log_data,prefixes=extracted_prefixes,encoder=encoder,resource=True)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total resources: 22 - Target resources: 23\n",
      "\t ['Value 2', 'Value 5', 'Value 16', 'Value 15', 'Value 21', 'Value 10', 'Value 11', 'Value 12', 'Value 6', 'Value 7', 'Value 9', 'Value 14', 'Value 19', 'Value 17', 'Value 8', 'Value 13', 'Value 22', 'Value 1', 'Value 4', 'Value 3', 'Value 18', 'Value 20']\n",
      "Total activities: 14 - Target activities: 15\n",
      "\t ['Assign seriousness', 'Take in charge ticket', 'Resolve ticket', 'Closed', 'Wait', 'Create SW anomaly', 'Insert ticket', 'Schedule intervention', 'INVALID', 'RESOLVED', 'VERIFIED', 'Resolve SW anomaly', 'Require upgrade', 'DUPLICATE']\n",
      "Num. of Training sequences: 16937\n",
      "Encoding...\n",
      "Num. of features: 36\n"
     ]
    }
   ],
   "execution_count": 5
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Index-based encoding\n",
    "In the __Index-based encoding__, sequences of events are transformed into numerical feature vectors, where each event is represented by a pair of indices: one for the activity and one for the resource. These indices correspond to the positions of the activity and resource in their respective predefined sets. To apply index-based encoding, set the `encoder` parameter to `Encodings.Index_based` when calling the `encode_prefixes` function:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:04:19.497561Z",
     "start_time": "2025-06-28T13:04:19.287455Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import encode_prefixes\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "encoder=Encodings.Index_based\n",
    "x, y_a, y_g = encode_prefixes(log_data,prefixes=extracted_prefixes, encoder=encoder,resource=True)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total resources: 22 - Target resources: 23\n",
      "\t ['Value 2', 'Value 5', 'Value 16', 'Value 15', 'Value 21', 'Value 10', 'Value 11', 'Value 12', 'Value 6', 'Value 7', 'Value 9', 'Value 14', 'Value 19', 'Value 17', 'Value 8', 'Value 13', 'Value 22', 'Value 1', 'Value 4', 'Value 3', 'Value 18', 'Value 20']\n",
      "Total activities: 14 - Target activities: 15\n",
      "\t ['Assign seriousness', 'Take in charge ticket', 'Resolve ticket', 'Closed', 'Wait', 'Create SW anomaly', 'Insert ticket', 'Schedule intervention', 'INVALID', 'RESOLVED', 'VERIFIED', 'Resolve SW anomaly', 'Require upgrade', 'DUPLICATE']\n",
      "Num. of Training sequences: 16937\n",
      "Encoding...\n",
      "Num. of features: 30\n"
     ]
    }
   ],
   "execution_count": 6
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Shrinked index-based encoding\n",
    "In the __Shrinked index-based encoding__, sequences of events are transformed into numerical feature vectors by assigning a unique integer index to each activity–resource pair. To apply shrinked index-based encoding, set the encoder parameter to `Encodings.Shrinked_based` when calling the `encode_prefixes` function:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:04:39.141789Z",
     "start_time": "2025-06-28T13:04:38.944126Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import encode_prefixes\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "\n",
    "encoder=Encodings.Shrinked_based\n",
    "x, y_a, y_g = encode_prefixes(log_data,prefixes=extracted_prefixes, encoder=encoder, resource=True)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total resources: 22 - Target resources: 23\n",
      "\t ['Value 2', 'Value 5', 'Value 16', 'Value 15', 'Value 21', 'Value 10', 'Value 11', 'Value 12', 'Value 6', 'Value 7', 'Value 9', 'Value 14', 'Value 19', 'Value 17', 'Value 8', 'Value 13', 'Value 22', 'Value 1', 'Value 4', 'Value 3', 'Value 18', 'Value 20']\n",
      "Total activities: 14 - Target activities: 15\n",
      "\t ['Assign seriousness', 'Take in charge ticket', 'Resolve ticket', 'Closed', 'Wait', 'Create SW anomaly', 'Insert ticket', 'Schedule intervention', 'INVALID', 'RESOLVED', 'VERIFIED', 'Resolve SW anomaly', 'Require upgrade', 'DUPLICATE']\n",
      "Num. of Training sequences: 16937\n",
      "Encoding...\n",
      "Num. of features: 15\n"
     ]
    }
   ],
   "execution_count": 7
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multi-encoders encoding\n",
    "In the __Multi-encoders encoding__, sequences of events are represented using separate embedding spaces for activities and resources. Each activity and resource is first embedded independently, and then enriched with cross-information using a modulation mechanism that captures their interactions. The final representation combines the modulated embeddings using learned alignment weights. To apply multi-encoders encoding, set the encoder parameter to `Encodings.Multi_encoders` when calling the `encode_prefixes` function:"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T13:05:12.433914Z",
     "start_time": "2025-06-28T13:05:12.234572Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import encode_prefixes\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "\n",
    "encoder=Encodings.Multi_encoders\n",
    "x, y_a, y_g = encode_prefixes(log_data,prefixes=extracted_prefixes, encoder=encoder, resource=True)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total resources: 22 - Target resources: 23\n",
      "\t ['Value 2', 'Value 5', 'Value 16', 'Value 15', 'Value 21', 'Value 10', 'Value 11', 'Value 12', 'Value 6', 'Value 7', 'Value 9', 'Value 14', 'Value 19', 'Value 17', 'Value 8', 'Value 13', 'Value 22', 'Value 1', 'Value 4', 'Value 3', 'Value 18', 'Value 20']\n",
      "Total activities: 14 - Target activities: 15\n",
      "\t ['Assign seriousness', 'Take in charge ticket', 'Resolve ticket', 'Closed', 'Wait', 'Create SW anomaly', 'Insert ticket', 'Schedule intervention', 'INVALID', 'RESOLVED', 'VERIFIED', 'Resolve SW anomaly', 'Require upgrade', 'DUPLICATE']\n",
      "Num. of Training sequences: 16937\n",
      "Encoding...\n",
      "Num. of features: 15\n"
     ]
    }
   ],
   "execution_count": 9
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## End-to-end data preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T12:45:09.132835Z",
     "start_time": "2025-06-28T12:45:07.718810Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.data_preprocessing import end_to_end_process\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "from pathlib import Path\n",
    "\n",
    "log_path = Path.cwd().parent/'data'/'input'/'logs'\n",
    "train_log = \"helpdesk_train.xes\"\n",
    "test_log = \"helpdesk_test.xes\"\n",
    "encoder = Encodings.Index_based\n",
    "log_data, x, y_a, y_g = end_to_end_process(log_path=log_path,train_log=train_log,test_log=test_log, encoder=encoder, resource=True)"
   ],
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "parsing log, completed traces :: 100%|██████████| 3664/3664 [00:00<00:00, 8690.64it/s]\n",
      "parsing log, completed traces :: 100%|██████████| 820/820 [00:00<00:00, 8723.79it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total resources: 22 - Target resources: 23\n",
      "\t ['Value 2', 'Value 5', 'Value 16', 'Value 15', 'Value 21', 'Value 10', 'Value 11', 'Value 12', 'Value 6', 'Value 7', 'Value 9', 'Value 14', 'Value 19', 'Value 17', 'Value 8', 'Value 13', 'Value 22', 'Value 1', 'Value 4', 'Value 3', 'Value 18', 'Value 20']\n",
      "Total activities: 14 - Target activities: 15\n",
      "\t ['Assign seriousness', 'Take in charge ticket', 'Resolve ticket', 'Closed', 'Wait', 'Create SW anomaly', 'Insert ticket', 'Schedule intervention', 'INVALID', 'RESOLVED', 'VERIFIED', 'Resolve SW anomaly', 'Require upgrade', 'DUPLICATE']\n",
      "Num. of Training sequences: 16937\n",
      "Encoding...\n",
      "Num. of features: 30\n"
     ]
    }
   ],
   "execution_count": 37
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 Training\n",
    "Once the prefixes are encoded, NeSy4PPM proceeds to train a neural network that learns to predict the next activity and resource given a partial trace. The training is handled via the `train` function, which takes the encoded prefix data (`x`, `y_a`, `y_g`) and builds a model according to the chosen architecture. NeSy4PPM supports two neural architectures:\n",
    "\n",
    "- __LSTM (Long Short-Term Memory)__ networks, which are recurrent neural networks designed to handle sequential data with long-range dependencies. To use LSTM, set the `model_arch` parameter to `NN_model.LSTM`.\n",
    "- __Transformer__ architectures, which use attention mechanisms to model relationships across all positions in the prefix sequence simultaneously. To use a Transformer, set the `model_arch` parameter to `NN_model.Transformer`."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "from NeSy4PPM.Training.train_model import train\n",
    "from NeSy4PPM.Data_preprocessing.utils import NN_model\n",
    "\n",
    "model = NN_model.Transformer\n",
    "model_folder= Path.cwd().parent/'data'/'output'\n",
    "train(log_data, encoder, model_arch=model, output_folder=model_folder, X=x, y_a=y_a, y_g=y_g)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 Prediction\n",
    "The __Prediction__ in NeSy4PPM is responsible for generating suffix predictions from a prefix (i.e., an incomplete trace) using a trained neural model. To improve both accuracy and compliance, particularly under concept drift, NeSy4PPM supports three main prediction modes:\n",
    "\n",
    "- `Purely Neural predictions`: The suffix is predicted mainly based on the output of the neural model, without considering background knowledge (BK).\n",
    "- `BK-Contextualized predictions`: BK is integrated during either the greedy or beam search process. It guides the search by pruning branches that violate compliance constraints, ensuring that only valid continuations are explored.\n",
    "- `BK-filtering`: BK is applied after generating candidate suffixes. Predicted suffixes that do not satisfy the compliance constraints are filtered out."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Set prediction parameters\n",
    "The prediction process begins by specifying the following parameters that control how the prediction algorithm operates:\n",
    "- `log_data.evaluation_prefix_start`: the minimum prefix length (in events) for prediction.\n",
    "- `log_data.evaluation_prefix_end`: the maximum prefix length for prediction.\n",
    "- `model_arch`: the trained model architecture (`NN_model.LSTM` or `NN_model.Transformer`).\n",
    "- `output_folder`: the path where the trained model and prediction results are saved.\n",
    "- `encoder`: the encoding method used during training (`Encodings.One_hot`, `Encodings.Index_based`, `Encodings.Shrinked_index_based` or `Encodings.Multi_Encoders` ).\n",
    "- `beam_size`: the number of alternative suffixes explored in parallel by the beam search. A `simple autoregressive prediction` can be performed by setting `beam_size` to `1` (greedy search).\n",
    "- `bk_file_path`: the path to the `BK` (background knowledge) file. `BK` can represent domain constraints or business rules, and can be encoded in various formats:\n",
    "    - __Procedural models__: `.bpmn` (Business Process Model and Notation), `.pnml` (Petri Nets),\n",
    "    - __Declare models__: `.decl` (Crisp or Multi-perspective Declare constraints)\n",
    "    - __Probabilistic Declare models__: `.txt` (Declare constraints annotated with probabilities).\n",
    "- `weight`: a float value in [0, 1] that balances the importance of neural predictions and BK compliance. A value of 0 uses only the neural model, while higher values increase the importance of BK during the search.\n",
    "- `BK_end`: a boolean parameter indicating whether BK is applied at the end (i.e., filtering) instead of during the search.\n",
    "- `resource`: a boolean parameter indicating whether the resource prediction is performed onlongside activty prediction.\n",
    "- `fitness_method`: the method used to compute compliance scores, i.e., the alignment or replay fitness between the predicted trace and the procedural model. This parameter is only applicable when the BK model is procedural, and must be set to one of the following: : `conformance_diagnostics_alignments`, `fitness_alignments`, `conformance_diagnostics_token_based_replay` or `fitness_token_based_replay`.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T11:10:56.147680Z",
     "start_time": "2025-06-28T11:10:56.131592Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.utils import NN_model\n",
    "from NeSy4PPM.Data_preprocessing.utils import Encodings\n",
    "\n",
    "(log_data.evaluation_prefix_start, log_data.evaluation_prefix_end) = (1,4)\n",
    "model_arch = NN_model.Transformer\n",
    "encoder = Encodings.Index_based\n",
    "output_folder= Path.cwd().parent/'data'/'output'\n",
    "bk_file_path = Path.cwd().parent/'data'/'input'/'declare_models'/'BK_helpdesk.decl'\n",
    "beam_size = 3\n",
    "weight = 0.9\n",
    "BK_end = False\n",
    "resource = True"
   ],
   "outputs": [],
   "execution_count": 11
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Load the Background Knowledge (BK)\n",
    "After setting the parameters, a background knowledge (BK) model must be loaded using the `load_bk` function."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T11:34:40.605733Z",
     "start_time": "2025-06-28T11:34:40.543425Z"
    }
   },
   "source": [
    "from NeSy4PPM.Data_preprocessing.utils import load_bk\n",
    "\n",
    "bk_model = load_bk(bk_file_path)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 Existence1[Closed] |A.org:resource is Value 3 |\n",
      "1 Chain Precedence[Resolve ticket, Closed] |A.org:resource is Value 3 | |\n"
     ]
    }
   ],
   "execution_count": 21
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Perform Prediction\n",
    "NeSy4PPM implements the `predict_evaluate` function, which generates activity-resource suffixes using the proposed neuro-symbolic beam search algorithm and computes two evaluation metrics:\n",
    "   - __Damerau-Levenshtein Similarity__, measuring the similarity between the predicted and actual suffixes based on edit distance,\n",
    "   - __Jaccard Similarity__, measuring the overlap between the sets of predicted and actual activities. suffix prediction using a trained neural model and loaded `BK` model.\n",
    "\n",
    "By default, this function operates on the __entire test log__, predicting suffixes for all traces defined in the test set."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "## Entire test log Prediction\n",
    "from NeSy4PPM.Prediction import predict_suffix\n",
    "prediction_filename = 'helpdesk_test_prediction.csv'\n",
    "predict_suffix.predict_evaluate(log_data, model_arch=model_arch, encoder=encoder,output_filename=prediction_filename,\n",
    "                            output_folder=output_folder, bk_model=bk_model, beam_size=beam_size, resource=resource, weight=weight, bk_end=BK_end)"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, `predict_evaluate` function can also be used to predict suffixes for a specific __subset of traces__ by providing a list of case IDs from the test log."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T11:36:55.384275Z",
     "start_time": "2025-06-28T11:36:14.374552Z"
    }
   },
   "source": [
    "## A subset of test log Prediction Beam search with KB-contextualized\n",
    "from NeSy4PPM.Prediction import predict_suffix\n",
    "traces_ids = ['Case 1327']\n",
    "prediction_filename = 'helpdesk_case1327_prediction.csv'\n",
    "predict_suffix.predict_evaluate(log_data, model_arch=model_arch, encoder=encoder,output_filename=prediction_filename,\n",
    "                                output_folder=output_folder, evaluation_trace_ids= traces_ids, bk_model=bk_model, beam_size=beam_size,\n",
    "                                resource=resource, weight=weight, bk_end=BK_end)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Activity & Resource Prediction ...\n",
      "Model filepath: C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\docs\\source\\data\\output\\keras_trans_index-based\\models\\CFR\\helpdesk_train\n",
      "Latest checkpoint file: C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\docs\\source\\data\\output\\keras_trans_index-based\\models\\CFR\\helpdesk_train\\model_015-1.198.keras\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n",
      "C:\\Users\\JOukharijane\\AppData\\Local\\Programs\\Python\\Python310\\lib\\random.py:370: DeprecationWarning: non-integer arguments to randrange() have been deprecated since Python 3.10 and will be removed in a subsequent version\n",
      "  return self.randrange(a, b+1)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Case ID', 'Prefix length', 'Trace Prefix Act', 'Ground truth', 'Predicted Acts', 'Damerau-Levenshtein Acts', 'Jaccard Acts', 'Trace Prefix Res', 'Ground Truth Resources', 'Predicted Resources', 'Damerau-Levenshtein Resources', 'Jaccard Resources', 'Weight', 'Time']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/1 [00:00<?, ?it/s]C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\venv\\lib\\site-packages\\tqdm\\std.py:917: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n",
      "  return getattr(df, df_function)(wrapper, **kwargs)\n",
      "100%|██████████| 1/1 [00:20<00:00, 20.76s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Case 1327', 1, 'Assign seriousness', 'Wait>>Resolve ticket>>Closed', 'Take in charge ticket>>Resolve ticket>>Closed', 0.6666666666666667, 0.5, 'Value 13', 'Value 1>>Value 13>>Value 3', 'Value 13>>Value 13>>Value 3', 0.6666666666666667, 0.6666666666666666, 0.9, 20.74420714378357]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/1 [00:00<?, ?it/s]C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\venv\\lib\\site-packages\\tqdm\\std.py:917: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n",
      "  return getattr(df, df_function)(wrapper, **kwargs)\n",
      "100%|██████████| 1/1 [00:12<00:00, 12.49s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Case 1327', 2, 'Assign seriousness>>Wait', 'Resolve ticket>>Closed', 'Resolve ticket>>Closed', 1.0, 1.0, 'Value 13>>Value 1', 'Value 13>>Value 3', 'Value 1>>Value 3', 0.5, 0.33333333333333326, 0.9, 12.488699197769165]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/1 [00:00<?, ?it/s]C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\venv\\lib\\site-packages\\tqdm\\std.py:917: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n",
      "  return getattr(df, df_function)(wrapper, **kwargs)\n",
      "100%|██████████| 1/1 [00:07<00:00,  7.09s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Case 1327', 3, 'Assign seriousness>>Wait>>Resolve ticket', 'Closed', 'Closed', 1.0, 1.0, 'Value 13>>Value 1>>Value 13', 'Value 3', 'Value 3', 1.0, 1.0, 0.9, 7.0855488777160645]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "  0%|          | 0/1 [00:00<?, ?it/s]C:\\Users\\JOukharijane\\Desktop\\PostDoc\\NeSy4PPM\\venv\\lib\\site-packages\\tqdm\\std.py:917: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n",
      "  return getattr(df, df_function)(wrapper, **kwargs)\n",
      "100%|██████████| 1/1 [00:00<?, ?it/s]\n"
     ]
    }
   ],
   "execution_count": 23
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4 Evaluation\n",
    "The __evaluation__ assesses the performance of the prediction by computing a variety of metrics, including:\n",
    "\n",
    "- `Time`: Computes the average and standard deviation of prediction times.\n",
    "- `Jaccard similarity`: Measures the jaccard similarity between the predicted and ground-truth sequences of activities, and resources if `resource = True`.\n",
    "- `Damerau-Levenshtein similarity`: Computes the Damerau-Levenshtein similarity between predicted and ground-truth sequences of activities, and resources if `resource = True`.\n",
    "- `Compliance`: Measures the proportion of predicted traces (i.e., the prefix concatenated with the predicted suffix) that satisfy the (MP)-Declare background knowledge.\n",
    "- `Fitness`: Evaluates how well the predicted traces can be replayed in the discovered Petri net, using the specified `fitness_method` (e.g., `token-based replay`).\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-06-28T11:37:24.543983Z",
     "start_time": "2025-06-28T11:37:22.298909Z"
    }
   },
   "source": [
    "from NeSy4PPM.Evaluation import evaluate_all, discover_petri_net\n",
    "\n",
    "# discover a petri nets model from test log:\n",
    "petri_net_model = discover_petri_net(log_path/test_log)\n",
    "\n",
    "metrics = [\"Time\", \"Jaccard similarity\", \"Damerau-Levenshtien similarity\", \"Compliance\", \"Fitness\"]\n",
    "prediction_filename = 'helpdesk_case1327_prediction.csv'\n",
    "\n",
    "metrics_results = evaluate_all(log_data=log_data, model_arch=model_arch, encoder=encoder, output_folder=output_folder,\n",
    "                               filename=prediction_filename, metrics=metrics, resource=resource, declare_model=bk_model,petri_net_model=petri_net_model, fitness_method=\"fitness_token_based_replay\")\n",
    "for metric in metrics:\n",
    "    print(metric,\":\",metrics_results[metric],\"\\t\")"
   ],
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "parsing log, completed traces :: 100%|██████████| 820/820 [00:00<00:00, 2480.57it/s]\n",
      "DEBUG:graphviz._tools:os.makedirs('C:\\\\Users\\\\JOUKHA~1\\\\AppData\\\\Local\\\\Temp')\n",
      "DEBUG:graphviz.saving:write lines to 'C:\\\\Users\\\\JOUKHA~1\\\\AppData\\\\Local\\\\Temp\\\\tmpyzof20bn.gv'\n",
      "DEBUG:graphviz.backend.execute:run [WindowsPath('dot'), '-Kdot', '-Tpng', '-O', 'tmpyzof20bn.gv']\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 403880 characters]",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "replaying log with TBR, completed traces :: 100%|██████████| 2/2 [00:00<00:00, 249.25it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time : {'Average time': 13.439, 'Standard deviation time': 6.879} \t\n",
      "Jaccard similarity : {'Activities': 0.833, 'Resources': 0.667} \t\n",
      "Damerau-Levenshtien similarity : {'Activities': 0.889, 'Resources': 0.722} \t\n",
      "Compliance : 1.0 \t\n",
      "Fitness : 1.0 \t\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "execution_count": 24
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "9b13726099ff4a9270d97cd5a303046c40236cea9d4b3d3acf7f22861afad882"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
