BACCA is an open source C++ chain code benchmarking framework, developed using YACCLAB v2.0 as a starting point. BACCA provides correctness and average run-time (average) tests for chain code (also known as boundary tracing) algorithms over a collection of real world datasets.
| OS | Build | Compiler | OpenCV | CMake | GitHub Actions |
|---|---|---|---|---|---|
| Ubuntu 16.04 LTS |
x32 | gcc 5.4.0 | 3.0.0 | 3.13.5 |  |
| Ubuntu 18.04 LTS |
x64 | gcc 9.3.0 | 4.1.2 | 3.13.5 |  |
| MacOS (Darwin 19.6.0) |
x64 | AppleClang 12.0.0 (Xcode-12) |
3.1.0 | 3.13.0 |  |
To correctly install and run BACCA the following packages, libraries and utilities are needed:
- CMake 3.13 or higher (https://cmake.org),
- OpenCV 3.0 or higher (http://opencv.org),
- Gnuplot (http://www.gnuplot.info/),
- One of your favourite IDE/compiler: Visual Studio 2013 or higher, Xcode 5.0.1, gcc 4.7 or higher, .. (with C++14 support)
Notes for gnuplot:
- on Windows system: be sure of adding gnuplot to the system path to allow the automatic charts generation.
- on MacOS system: 'pdf terminal' seems to be not available, 'postscript' is used instead.
Clone the GitHub repository (HTTPS clone URL: https://github.com/prittt/BACCA.git) or simply download the full master branch zip file and extract it (e.g BACCA folder).
Install software in BACCA/bin subfolder (suggested) or wherever you want using CMake. Note that CMake should automatically find the OpenCV path if correctly installed on your OS, download the BACCA Dataset (be sure to check the box if you want to download it or to select the correct path if the dataset is already on your file system, and create a C++ project for the selected IDE/compiler.
Set the configuration file (config.yaml) placed in the installation folder (bin in this example) in order to select desired tests.
Open the project, compile and run it: the work is done!
| Name | Meaning | Default |
|---|---|---|
BACCA_DOWNLOAD_DATASET |
whether to automatically download the BACCA dataset or not | OFF |
BACCA_INPUT_DATASET_PATH |
path to the input dataset folder, where to find test datasets |
${CMAKE_INSTALL_PREFIX}/input |
BACCA_OUTPUT_RESULTS_PATH |
path to the output folder, where to save output results |
${CMAKE_INSTALL_PREFIX}/output |
OpenCV_DIR |
OpenCV installation path | - |
A YAML configuration file placed in the installation folder lets you to specify which kind of tests should be performed, on which datasets and on which algorithms. A complete description of all configuration parameters is reported below.
- perform - dictionary which specifies the kind of tests to perform:
perform:
correctness: false
average: true
average_with_steps: false- correctness_tests - dictionary indicating the kind of correctness tests to perform:
correctness_tests:
standard: true
steps: true- tests_number - dictionary which sets the number of runs for each test available:
tests_number:
average: 10
average_with_steps: 10- algorithms - list of algorithms on which to apply the chosen tests, along with display name and reference for correctness check:
algorithms:
- Suzuki , Suzuki85(OpenCV) ; Suzuki
- Cederberg_LUT , Cederberg_LUT ; Suzuki
- Cederberg_LUT_PRED , Cederberg_LUT_PRED ; Suzuki
- Cederberg_Tree , Cederberg_Tree ; Suzuki
- Cederberg_Spaghetti , Cederberg_Spaghetti ; Suzuki
- Cederberg_Spaghetti_FREQ_All , Cederberg_SpaghettiF ; Suzuki- check_datasets, average_datasets, average_ws_datasets - lists of datasets on which, respectively, correctness, average, and average_ws tests should be run:
...
average_datasets: ["3dpes", "fingerprints", "hamlet", "medical", "mirflickr", "tobacco800", "xdocs"]
...- paths - dictionary with both input (datasets) and output (results) paths. It is automatically filled by CMake during the creation of the project:
paths: {input: "<datasets_path>", output: "<output_results_path>"}- save_middle_tests - dictionary specifying, separately for every test, whether to save the output of single runs, or only a summary of the whole test:
save_middle_tests: {average: false, average_with_steps: false}