Add Spark connector option for passing Pinot query options

[cbalci]

Adding Spark connector option for passing Pinot query options.

Example:

val data = spark.read
  .format("pinot")
  .option("table", "myTable")
  .option("tableType", "OFFLINE")
  .option("queryOptions", "enableNullHandling=true,maxExecutionThreads=1") // <- new
  .load()

Also includes a bugfix which allows the Spark connector to return null values instead of their placeholders.

Testing

• Unit tests are updated
• Ran the suite of integration tests (ExampleSparkPinotConnectorTest) against a Pinot cluster successfully

feature bugfix
release-notes (Spark connector now accepts Pinot query options)

[codecov-commenter]

Codecov Report

Merging #10443 (522e852) into master (d9c4315) will increase coverage by 6.07%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master   #10443      +/-   ##
============================================
+ Coverage     64.21%   70.29%   +6.07%     
- Complexity     6089     6107      +18     
============================================
  Files          2007     2061      +54     
  Lines        109281   111752    +2471     
  Branches      16692    16985     +293     
============================================
+ Hits          70177    78553    +8376     
+ Misses        33993    27672    -6321     
- Partials       5111     5527     +416     

Flag	Coverage Δ
integration1	24.37% <ø> (?)
integration2	24.27% <ø> (?)
unittests1	67.95% <ø> (-0.05%)	⬇️
unittests2	13.91% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ctor/spark/common/PinotDataSourceReadOptions.scala	85.48% <100.00%> (+1.55%)	⬆️
...nector/spark/common/query/ScanQueryGenerator.scala	96.87% <100.00%> (+0.32%)	⬆️

... and 446 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more