Project for benchmarking high-performance open source CSV parsers using JMH.
Bugs, feature suggestions and help requests can be filed with the issue-tracker.
The project is based on Gradle.
Modify the build version to your current snapshot, and potentially change the include expression in jmh-benchmark/build.gradle and/or the CsvParam to scope your tests.
Then run
./gradlew clean jmhClasses jmh --info
If the JMH plugin seems to have trouble refreshing the project, restart the Gradle deamon before running:
./gradlew --stop && ./gradlew clean jmhClasses jmh --info
And also optionally refresh the dependencies using
./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info
Open the file ./jmh-benchmark/build/reports/jmh/index.html to view a visualization of the results.
Inputs:
- World cities
- GTFS Trips
- GTFS StopTimes
Modes:
- plain or quoted
- rows in intervals from 1 to 1 million
- single-shot and sample time
- databinding or raw String parsing
From using JDK 8 build 232.
What is measured is the cold start, which is how long the parsing take the first time. As much as possible, this includes things like class-loading and inital configuration / creation from a Reader.
The results must be taken with a grain of salt. Most of these will differ quite a lot from run to run, but the current results give a ballpark picture of the performance.
The results show that the SimpleFlatMapper beats univocity-parsers when file size increases. However sesseltjonna-csv performs better than both.
What is measured here is the warm start, which is how long the parsing takes after an initial warmup (i.e. for repeated use). The warmup here referes to the Just-in-time-compilator to do its optimizations, which takes place after something like a second of back to back invocation of the code. Basically these optimizations usually mean inlining methods and compiling the bytecode to platform-specific native code.
Below 1000 rows takes at most 1.1 milliseconds for all libraries, 1 million rows takes less than a second: 10000-1000000.
SimpleFlatMapper beats sesseltjonna-csv in one of the raw String benchmarks, otherwise is it quite consistenly sesseltjonna-csv fastest, SimpleFlatMapper second and univocity-parsers third.
When parsing a lot of different CSV file formats, libraries using a static parser implementation will have an advantage on the JIT side as there will be fewer files to compile and thus more of the overall time will be spent running JIT-optimized code paths. In practice though, the number of CSV files (and ordering of the columns) is usually limited.
Just a note on using this library to minimize startup time - your milage will vary in relation to the above benchmark because you also have to take into account the time loading the dependency libraries - indeed this is true for all the CSV parser libraries.
For sesseltjonna-csv, when using explicit setter, it loads the byte-buddy library to identify the setter method, whereas the default is to use reflection.
If however the CSV format is written in stone, it is possible to add the dynamically generated parser as regular class during build-time.
So as with all optimization, make sure to measure the startup time before and after modifications.
Alternative benchmarks
- 1.0.2: Added String-array benchmarks, made single-shot benchmarks do most of the classloading.
- 1.0.1: Bumped versions, refreshed results.
- 1.0.0: Initial version