Test Framework: Support CSV to Parquet conversion

[rfdavid]

Introduction

This commit adds the capability of converting CSV datasets to parquet datasets on the fly for the tests by simply using the following command in the test header:

-DATASET PARQUET CSV_TO_PARQUET(dataset)

To load dataset without any conversion:

-DATASET CSV tinysnb
-DATASET NPY npy-1d
-DATASET PARQUET demo-db/parquet

How it works

1. Create a directory dataset/parquet_temp/tinysnb
2. Copy schema.cypher to the created directory
3. Read and parse copy.cypher, extract the csv file names, header/no header information, csv delimiter
4. Create a new copy.cypher with the new COPY commands and paths
5. Convert .csv files to .parquet files to the parquet temp directory
6. Set dataset path to parquet temp directory
7. Remove parquet temp directory after all tests run.

Related to #1521

[ray6080]

Is this script still useful? Should we consider removing it?

[rfdavid]

It might be useful if you want to generate a dataset to store in our codebase. Other than that, I can't think of anything. We can also remove demo-db/parquet too.

[ray6080]

My understanding is that you are relying on arrow's auto csv reader to figure out data types for each csv column. I'm not sure if it's always as expected, especially when it comes to dates/timestamps/nested data types. I believe arrow converts some nested data types into strings. Can you double check the metadata of generated parquet files?
Ideally, we need to figure out data types for each column, and let the arrow reader be aware of that.
I'm fine with this auto reader for now if it works mostly as expected.

Actually I think the best way is we should add native support of exporting tables to parquet files in Cypher.
Then the conversion would be much easier.

Besides this, I'm curious what's the default row group size when we dump data into parquet?

[rfdavid]

That's right, I'm relying on arrow's auto csv reader and it is not much assertive. Dates and timestamps are being converted into strings.

[codecov]

Codecov Report

Patch coverage: 90.90% and project coverage change: +0.01 🎉

Comparison is base (dbb2552) 91.49% compared to head (19e8077) 91.50%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1611      +/-   ##
==========================================
+ Coverage   91.49%   91.50%   +0.01%     
==========================================
  Files         725      725              
  Lines       26326    26334       +8     
==========================================
+ Hits        24086    24098      +12     
+ Misses       2240     2236       -4     

Impacted Files	Coverage Δ
src/include/common/file_utils.h	100.00% <ø> (ø)
src/common/file_utils.cpp	73.03% <90.90%> (+1.42%)	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.