What is SmartRPA

Robotic Process Automation (RPA) is a technology which automates mouse and keyboard interactions by means of a software (SW) robot to remove intensive routines. The current generation of RPA tools is driven by predefined rules and manual configurations made by expert users rather than automated techniques.

SmartRPA is a cross-platform tool that tackles such issues. It allows to easily record event logs and to automatically generating executable RPA scripts that will drive a SW robots in emulating an observed user behavior (previously recorded in dedicated UI logs) during the enactment of a routine of interest.

Authors

SmartRPA has been developed at DIAG, Department of Computer, Control, and Management Engineering Antonio Ruberti in Sapienza University of Rome by:

• Simone Agostinelli
• Marco Lupia
• Andrea Marrella

Many thanks also to Tom Hohenadl (Technische Hochschule Ingolstadt) and Antonio Martínez-Rojas (Universidad de Sevilla) for the ongoing updates of the tool.

A Screencast of the tool is available on [Vimeo].

Architecture

The architecture of SmartRPA integrates five main SW components.

Key features include:

• Action Logger, log user behaviour, take screenshots, tag actions, supports wide range of applications, cross-platform;
• Log Processing, generates both CSV and XES event log;
• Event abstraction, abstracts events to a higher level;
• Process Discovery, selects the most suitable routine variant to automate and generates high-level flowchart diagram, thus skipping completely the manual modeling activity;
• Decision Points, discover differencies between multiple traces in a process and build a new routine based on user decisions;
• RPA, implements and enacts a SW robot emulating a routine reflecting the observed behavior (either the most frequent one or the one based on decision points). Available both as a cross-platform Python script and as a UiPath project.

A list of events supported by the Action Logger is available in SmartRPA_events.pdf.

The full documentation of the tool is available here. A user manual for SmartRPA is available in this repository here.

Installation and execution:

The following dependencies are required to enable all functionalities within SmartRPA. Check all boxes and you are able to log user actions, discover process models, and emulate RPA bots in UiPath and Automagica automatically:

• Prerequisites
  • Visual Studio C/C++ Build Tools (Windows Only)
  • Graphviz Windows/Mac
  • Automagica
  • Chromedriver for Edge/Chrome
  • UiPath Studio
• Action Logger dependencies
  • Python Requirements
  • Browser extension
• Excel AddIn (MacOS Only)

Furthermore, there are possible installation issues that can arise with the installation of the prerequisites or libraries. You can find fixes to the most common issues in the section Installation Errors .

1. Install Prerequisites

1.1 Visual Studio (Windows Only)

• On Windows, Visual Studio C/C++ Build Tools must be installed. It is vital to install all C++ related development tools listed here:

  • Windows 10/11 SDK
  • Visual C++ tools for CMake
  • C++ x64/x86 build tools

1.2a GraphViz Windows

Install the latest version of graphviz. Make sure to add it to system PATH. Detailed instructions here.

1.2b GraphViz Mac

• On MacOS:

  1. Make sure you installed brew package manager
  2. Check [brew] installation for graphviz with brew install graphviz
  3. Check pm4py installation and run pip3 install pm4py==1.5.0.1 if necessary

2. Python Installation

2.1 Requirements.txt

Python ≥ 3.7 (64bit) is required. Python 3.12 is recommended.

pip3 install -r src/requirements.txt

We have specified three requirements files for distinct Python versions. We recommend using the specified requirements file. The requirements.txt is for Python 3.12.

CMD installation lines for Python 3.7 and Python 3.10

pip3 install -r src/requirements308.txt

pip3 install -r src/requirements310.txt

2.2 Automagica Library

To run the generated RPA scripts you must install automagica module available in the libraries directory.

pip3 install src/libraries/smartRPA-automagica-2-1-12.zip

CMD installation lines Automagica with Python <= 3.7

pip3 install src/libraries/Automagica-2.0.25-py3-none-any.whl`

2.3 Chromedriver (Required for Chrome/Edge users)

Required for executing the emulated Automagica Python bots in Chrome and Edge. To select the correct version of the pip installation for the Chromedriver go to the Chrome or Edge settings/help. Get the version number, e.g., Version 124.0.2478.97. Navigate to PyPi Chromedriver History. Select the release with a same main number, e.g., chromedriver-binary 124.0.6367.207.0, and install this version using pip, i.e.,

pip install chromedriver-binary==124.0.6367.207.0

2.4 UiPath (Windows Only) (Optional)

The generated UiPath project requires UiPath Studio, available at https://www.uipath.com/product/studio.
Without UiPath Studio you cannot import and execute the generated UiPath bots from SmartRPA.

3. Install browser extension

The browser extension is required to log browser events. It is available in src/extensions/browserlogger and supports 4 major browsers:

• Google Chrome: load unpacked browserlogger directory in chrome://extensions/

• Mozilla Firefox: install browserlogger.xpi in about:addons

• Microsoft Edge (chromium): load unpacked browserlogger directory in edge://extensions/

• Opera: load unpacked browserlogger directory in opera:extensions

Currently the browser extension requires the developer mode to be active in your browser. Please check the browser documentation on how to enable the developer mode.

Enabling the Browser Logger

Once the main logger is running, you must click on the browser extension to enable it.

Inactive Browser Logging	Active Browser Logging

4. Install Excel Addin (MacOS Only)

The excel addin is required to log Excel events only on MacOS.

Node.js must be installed to run this addin.

cd src/extensions/excelAddinMac
npm install # install dependencies
npm start   # sideload Add-in
npm stop    # stop server

Click to show how to activate the Add-in in Excel

1. Start the Action Logger selecting Excel module
2. Go to Insert tab
3. Click on the small down-arrow to the right of My Add-ins > OfficeLogger

4. Go to Home tab
5. Click the Show Taskpane button in the ribbon
6. Enable the checkbox

If you don't find OfficeLogger under My Add-ins, copy extensions/excelAddinMac/manifest.xml into ~/Library/Containers/com.microsoft.Excel/Data/Documents/wef, as described here.

5. Run main logger

python3 main.py

A detailed screencast of the tool is available on [Vimeo] including the core functions of recording, analyzing, and emulating RPA bots. The resulting event log will be saved in logs/ directory. If the screenshot feature is active the screenshots are stored in the Screenshots directory.
SmartRPA generates two types of SW Robots in the RPA/ directory:

1. a cross-platform executable Python script, available on both Windows and MacOS
2. a UiPath project, available only on Windows

The advantages of the UiPath integration is that the generated SW Robot can be easily customized by the end user.

NOTE: In the Action Logger, when selecting a Microsoft Office program to log, it will automatically be opened. This is required to correctly handle events. The opened window should not be closed until logging is completed.

Action Logger Only

If you only want the action logger to record event logs without further analysis, you can find it here . Please note that this logger is associated with version 1.0 of SmartRPA and has not been updated with the new functionalities (Supervision Feature, Screenshot Feature, Updated Python Libraries) of SmartRPA 2.0.

Error Handling for possible installation errors

1. Visual Studio Errors

Error: Microsoft Visual C++ 14.0 is required

If you encounter errors like `Microsoft Visual C++ 14.0 is required`, [check here](https://bobbyhadz.com/blog/error-microsoft-visual-c-14-0-or-greater-is-required) .

2. Graphviz Errors

Windows: Fix graphviz error: Command '[WindowsPath('dot'), '-Kdot', '-Tpdf', '-O', 'tmp24z1pppy.gv']' returned non-zero exit status 1.

Could not save image: Command '[WindowsPath('dot'), '-Kdot', '-Tpdf', '-O', 'tmp24z1pppy.gv']' returned non-zero exit status 1. [stderr: b'There is no layout engine support for "dot"\r\nPerhaps "dot -c" needs to be run (with installer\'s privileges) to register the plugins?\r\n'] 

If you recieve the shown error and you use Anaconda or conda, you have to use another version of the graphviz python library. With conda run the following comand in your environment:

conda install conda-forge::python-graphviz conda-forge::graphviz=2.46.1

Click to show how to fix ModuleNotFoundError error on Windows

If you have installed all the requirements but you still get ModuleNotFoundError, run SmartRPA with:

py main.py

MacOS: Click to show how to fix installation errors on MacOS

If you get an error during installation:

• If you're on MacOS 10.14 Mojave, run the following command as suggested here, and then, in the same terminal window, install pm4py again

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

• If you're on MacOS 10.15 Catalina, run the following command as suggested here, and then, in the same terminal window, install pm4py again

export CPATH=`xcrun --show-sdk-path`/usr/include

• If you're on MacOS 11.0 Big Sur, install the latest nightly build version of scikit-learn running the following command and then install pm4py again

pip3 install --pre --extra-index https://pypi.anaconda.org/scipy-wheels-nightly/simple scikit-learn

3. Automagica Errors

Click to show how to fix installation errors on Windows

1. Make sure you are using 64bit version of Python3
2. Install Win64 OpenSSL v1.1.1 from this website. When prompted select "Copy OpenSSL DLLs to: the Windows system directory"
3. Open CMD as admin and type (one command per line):

set LIB=C:\Program Files\OpenSSL-Win64\lib;%LIB%
set INCLUDE=C:\Program Files\OpenSSL-Win64\include;%INCLUDE%
pip3 install libraries/Automagica-2.0.25-py3-none-any.whl

4. Chromedriver Errors

Click to show how to fix installation errors on MacOS

If you get the error RuntimeError: Failed to download chromedriver archive

1. Navigate to /Applications/Python 3.x/ folder
2. Run Install Certificates.command file
3. Install the package again

Reference

If you don't find the Python 3.8 folder under /Applications, make sure you installed Python using the official installer and not from a package manager like brew.

Message: session not created: This version of ChromeDriver only supports Chrome version 123

If you get the error Message: session not created: This version of ChromeDriver only supports Chrome version 123 when running a smartRPA created bot:

1. Navigate Pypi Chromedriver-binary
2. Select the chromedriver-binary matching your current Chrome version. E.g. 121.0.6134.0.0 for Chrome 121
3. Install the package again using the pip command from Pypi, e.g. pip install chromedriver-binary==121.0.6134.0.0

Publications

• Agostinelli S., Lupia M., Marrella A., Mecella M.: Reactive Synthesis of Software Robots in RPA from User Interface Logs. Accepted at Computers in Industry (Elsevier), 2022
• Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: SmartRPA: A Tool to Reactively Synthesize Software Robots from User Interface Logs. In: 33rd Int. Conf. on Advanced Information Systems Engineering (CAiSE Forum). pp. 137-145 (2021)
  • The synthetic UI logs generated for the test are available at: https://tinyurl.com/yyk68psx.
  • The complete results can be analyzed at: https://tinyurl.com/y55v56qa.
• Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: Automated Generation of Executable RPA Scripts from User Interface Logs. In: 18th Int. Conf. on Business Process Management (RPA Forum). pp. 116-131 (2020)

Acknowledgments

This work has been partially supported by the PNRR MUR project PE0000013-FAIR