HllSketch Merge Aggregator optimizations

[adarshsanjeev]

Description

This PR attempts to improve the performance of HllSketchMergeAggregators.

Improvements

• Serialize to null instead of an empty sketch when converting to byte array. This will allow future ingestions to write null, which is faster to read and is slightly cheaper than storing an empty sketch of 8 bytes.
• Directly check memory to see if the sketch is empty, based on certain flags. This will provide the benefits of the above without reingesting the data.
• Add cache for hll union objects in HllSketchMergeAggregators

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[gianm]

FYI I benchmarked this patch on a dataset with 11 groups and a few million sketches going into those groups (so, each Union would get hundreds-of-thousands of calls). It's about 37% faster.

Query                   Status    Iter     Min     Avg  Median     Max   OK?
watch_byuser_hll_12_4      200      25     454     463     461     511     Y [patch]
watch_byuser_hll_12_4      200      25     622     636     636     665     Y [master]

I also tried the case where the number of groups and number of sketches is similar. In this case the patch is slightly slower. I think it's worth it on balance though. The speedup in the speedup case is a lot bigger than the slowdown in the slowdown case.

[adarshsanjeev]

@gianm Thanks for running perf tests on this PR! I've tried running a perf test on a sketch column with ~20M empty sketches and observed somewhat comparable results.

Regarding the PR, while testing, I've noticed one issue with MSQ and estimation on an empty sketch, returning null instead of 0 after this change. I'll look into resolving it.

[adarshsanjeev]

Resolved the above

[imply-cheddar]

This LGTM. Fwiw, the check we are doing to make sure that things are null should be safe, but it's also sad that we are needing to do it. It would probably be nice to file a ticket against the apache datasketches project pointing them to what we are doing to identify that the sketch is effectively empty and ask if they can give us a less invasive (and maybe even faster) way of doing the same thing.

[adarshsanjeev]

@imply-cheddar Sure, I'll follow up on that