Benchmark Zarr vs WKW

[normanrz]

It would be interesting to see how the backend implementation of Zarr3 + Sharding compares to WKW.
We should run a benchmark. We should use a dataset that is available both in WKW and Zarr (e.g. l4dense_motta_et_al_demo). Both versions should be on the same local filesystem. As a test, I would recommend random reads to color and segmentation layers.

[fm3]

Currently blocked by #7241

[fm3]

@normanrz could you paste the relevant results here and then close this issue? 🙏

[normanrz]

Results on Hetzner cloud CCX53 (32 cores, 128G RAM)

Benchmarking with wrk (32 threads, 500 conenctions) and a custom lua script

COLOR WKW
Running 1m test @ http://localhost:9001
  32 threads and 500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    72.34ms  206.46ms   4.49s    97.65%
    Req/Sec   337.55     75.59   780.00     77.81%
  630421 requests in 1.00m, 115.67GB read
Requests/sec:  10490.76
Transfer/sec:      1.92GB

SEGMENTATION WKW
Running 1m test @ http://localhost:9001
  32 threads and 500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    63.89ms  196.38ms   3.23s    98.18%
    Req/Sec   379.16     80.62     0.89k    81.07%
  699823 requests in 1.00m, 512.83GB read
Requests/sec:  11644.98
Transfer/sec:      8.53GB

COLOR ZARR3
Running 1m test @ http://localhost:9001
  32 threads and 500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    23.92ms   35.13ms   1.22s    96.30%
    Req/Sec   760.45    165.42     3.69k    86.93%
  1430832 requests in 1.00m, 262.53GB read
Requests/sec:  23808.88
Transfer/sec:      4.37GB

SEGMENTATION ZARR3
Running 1m test @ http://localhost:9001
  32 threads and 500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    47.03ms  136.51ms   3.37s    98.52%
    Req/Sec   438.95     97.88     1.40k    78.19%
  819432 requests in 1.00m, 600.48GB read
  Non-2xx or 3xx responses: 1
Requests/sec:  13635.31
Transfer/sec:      9.99GB

Zarr3 implementation is faster. For color data 2.2x faster and for segmentations 1.2x faster.