python/lecture2-discovery.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is part of a course on Applied Process Mining. The collection of notebooks is a *living document* and subject to change. \n",
    "\n",
    "# Lecture 2 - 'Process Discovery with the Heuristics Miner' (Python / PM4Py)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "<img src=\"https://pm4py.fit.fraunhofer.de/static/assets/images/pm4py-site-logo-padded.png\" alt=\"PM4Py\" style=\"width: 200px;\"/>\n",
    "\n",
    "In this notebook, we are using several libraries:\n",
    "\n",
    "* [PM4Py](https://pm4py.fit.fraunhofer.de/)\n",
    "* [pandas](https://pandas.pydata.org/)\n",
    "* [plotnine](https://plotnine.readthedocs.io/en/stable/)\n",
    "\n",
    "Often used dependencies are imported:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import pm4py\n",
    "import plotnine\n",
    "from plotnine import ggplot, geom_point, aes, theme_bw, coord_flip, scale_y_discrete, theme, element_text, geom_bin2d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Process Discovery"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Process Models\n",
    "\n",
    "A common industry standard for Process Model is [BPMN](https://www.bpmn.org/), which can be created by web-based tools like [BPMN.io](https://bpmn.io/) and loaded and used by PM4Py."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pm4py\n",
    "import os\n",
    "\n",
    "bpmn_graph = pm4py.read_bpmn(os.path.join(\"..\", \"data\", \"sepsis.bpmn\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm4py.vis.view_bpmn(bpmn_graph)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quality Dimensions\n",
    "\n",
    "The quality dimensions `fitness`, `precision`, `simplicity`, and `generalisation` are best illustrated by using a small example event log.\n",
    "We are using an example event log in XES format that is used in the book `Process Mining - Data Science in Action` by Wil van der Aalst, which is downloaded and stored in the `../data` directory with the code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pm4py.objects.log.importer.xes import importer as xes_importer\n",
    "example_log = xes_importer.apply('../data/Lfull.xes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us have a look at the event log in tabular form. The mapping of the activity labels to actual activities is:\n",
    "\n",
    "* a = register request, \n",
    "* b = examine thoroughly, \n",
    "* c = examine casually, \n",
    "* d = check ticket, \n",
    "* e = decide, \n",
    "* f = reinitiate request, \n",
    "* g = pay compensation, and \n",
    "* h = reject request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm4py.convert_to_dataframe(example_log)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us discover a process map as we have seen in Lecture 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm4py.view_dfg(*pm4py.discover_directly_follows_graph(example_log))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not really very insightful the directly-follows based process map visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Heuristics Miner 🚧"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "L_heur_1 = pd.read_csv(\"../data/L_heur_1.csv\", sep=';')\n",
    "num_rows = len(L_heur_1)\n",
    "print(\"Number of rows: {}\".format(num_rows))\n",
    "L_heur_1_log = pm4py.format_dataframe(L_heur_1, case_id='CASE_concept_name', activity_key='activity_id', timestamp_key='timestamp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfg, sa, ea = pm4py.discover_directly_follows_graph(L_heur_1_log)\n",
    "pm4py.view_dfg(dfg, sa, ea)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dependency Graph 🚧"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In PM4Py, there is no option to review the intermediate data structures used for the Heuristics Miner such as the dependency graph. This would need to be implemented on top of Numpy. Feel free to provide a pull request :-)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Causal net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Causal net formalism is called `Heuristics Net` in PM4Py along with the definitions used in the original paper that proposed the Heuristics Miner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pm4py.algo.discovery.heuristics import algorithm as heuristics_miner\n",
    "\n",
    "# due to bug here: https://github.com/pm4py/pm4py-core/issues/220\n",
    "# we need to convert this to a event log object\n",
    "\n",
    "L_heur_1_log = pm4py.convert_to_event_log(L_heur_1_log)\n",
    "L_heur_1_map = pm4py.discover_heuristics_net(L_heur_1_log, dependency_threshold = 0.8, and_threshold = 0.65, loop_two_threshold = 0.5)\n",
    "pm4py.view_heuristics_net(L_heur_1_map)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Convert to BPMN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pm4py.objects.conversion.wf_net.variants import to_bpmn\n",
    "\n",
    "# we need to have an intermediate step to get a Petri net\n",
    "L_heur_1_petrinet, im, fm = pm4py.convert_to_petri_net(L_heur_1_map)\n",
    "L_heur_1_bpmn = to_bpmn.apply(L_heur_1_petrinet, im, fm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm4py.view_bpmn(L_heur_1_bpmn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pm4py.objects.petri_net.exporter import exporter as pnml_exporter\n",
    "pnml_exporter.apply(L_heur_1_petrinet, im, \"petri_final.pnml\", final_marking=fm)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "28aff1567d8aae5536826c1be921f2ff2e204808293d43dc67bdcb73bd29110e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}