ToolkitICL is an open source tool for automated OpenCL kernel execution with minimal overhead. It can be used as an easy to use cross platform tool to execute a set of kernels for example on compute clusters, run automated OpenCL benchmarks or test and validate kernels. HDF5 files are used for the entire configuration and data handling. The list of kernels to be executed, all variables and the workgroup size is defined in the input HDF5 file. After execution, the output data, memory copy- and runtime is written to the output HDF5 file. The power consumption and temperature for supported Nvidia GPUs or Intel and AMD CPUs and GPUs can also be logged. Which housekeeping data is logged during execution can be selected by individual command line options.
To build ToolkitICL the following needs to be installed:
- OpenCL (headers and drivers)
- HDF5
- CMake
For optional power and temperature logging, the following is needed:
- CUDA Toolkit (only for NVidia GPU power/temperature logging)
- Intel Power Gadget (only for Intel CPU/GPU power/temperature logging on Windows and Mac OS X systems)
msr-tools
(only for Intel CPU/GPU power/temperature logging on Linux systems)- AMD µProf (only for AMD CPU/GPU power/temperature logging on Windows and Linux systems)
This project uses the common CMake build system. Thus, the following commands can be used on Linux.
mkdir build && cd build
cmake .. # if you want to build only the main executable toolkitICL
make
If you want to build the tests, you can run
mkdir build && cd build
cmake -DTESTS=ON .. # if you want to build also the tests
make
make test
If you want to change the default compiler, the usual CMake workflow is supported, i.e. you can use something like
mkdir build && cd build
cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ..
make
Recent versions of Visual Studio should automatically detect the cmake configuration files and build
ToolkitICL automatically. Otherwise cmake-gui
can be used on Windows systems to create Visul Studio
project files.
In case some of the OpenCL headers are missing, they can be obtained directly from Khronos:
https://github.com/KhronosGroup. To automate this process,
you can add the command line switch -DGET_CL2HPP=ON
to CMake.
On some compute cluster systems, it might be necessary to explicitly define the library path in the cmake configuration files. If no ICD loader is available, the path to the OpenCL vendor library has to be defined manually.
If the CUDA toolkit is detected, CMake will enable CUDA support for power and temperature logging automatically.
It can also be controlled manually (in the source code) using the USENVML
define.
On Linux based systems, Intel CPU power consumption can be read directly from the model-specific registers (MSR).
To enable MSR support the msr-tools
package is needed on Debian based systems.
It might be necesarry to set the permissions sudo chmod o+rw /dev/cpu/0/msr
.
Depending on the architecture, the power consumption is determined by the system based on a heuristic algorithm and not measured directly.
If the Intel Power Gadget API is detected (Windows only), CMake will enable power and temperature logging for supported Intel CPUs and GPUs automatically.
During execution, power and temperture loggin is only available, if the EnergyLib64.dll
is found.
Depending on the architecture, the power consumption is determined by the system based on a heuristic algorithm and not measured directly.
The necessary drivers and additional information are available at Intel.
The default path will be used to search for the necessary libraries (HDF5, Intel Power Gadget, AMD µProf, CUDA). It is also possible to place the libraries together with the executable in the same folder.
Examples on how to create the input HDF5 files are provided in the directory
notebooks
.
ToolkitICL can be controlled by the following command line options:
-d device_id
: Use the device specified bydevice_id
.-b
: Activate benchmark mode (minimal console logs, additional delay before & after runs).-c config.h5
: Specify the URLconfig.h5
of the HDF5 configuration file.-nvidia_power sample_rate
: Log Nvidia GPU power consumption withsample_rate
(ms).-nvidia_temp sample_rate
: Log Nvidia GPU temperature withsample_rate
(ms).-intel_power sample_rate
: Log Intel system power consumption withsample_rate
(ms).-intel_temp sample_rate
: Log Intel CPU temperature withsample_rate
(ms).-amd_cpu_power sample_rate
: Log AMD CPU power consumption withsample_rate
(ms).-amd_cpu_temp sample_rate
: Log AMD CPU temperature withsample_rate
(ms).
Additional domumentation will be provided in the directory doc
.
A set of example testcases is available as part of the release files.
A useful tool to view and edit HDF5 files is HDFView.
This software can be cited as:
@misc{heinisch2019ToolkitICL,
title={{ToolkitICL}. {A}n open source tool for automated {OpenCL} kernel execution.},
author={Heinisch, Philip and Ostaszewski, Katharina and Ranocha, Hendrik},
month={03},
year={2019},
howpublished={\url{https://github.com/IANW-Projects/ToolkitICL}},
doi={10.5281/zenodo.2597653}
}
ToolkitICL has been used in the publication
@online{heinisch2020towards,
title={Towards Green Computing: {A} Survey of Performance and Energy
Efficiency of Different Platforms using {OpenCL}},
author={Heinisch, Philip and Ostaszewski, Katharina and Ranocha, Hendrik},
year={2020},
month={03},
eprint={2003.03794},
eprinttype={arxiv},
eprintclass={CS.PF}
}
This project is licensed under the terms of the Creative Commons CC BY-NC-ND 4.0 license.
Product and company names may be trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them or their affiliates. Everything is provided as is and without warranty. Use at your own risk!