-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read WKW as DatasetArray, no more Memory Mapping #7528
Conversation
webknossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/Compressor.scala
Outdated
Show resolved
Hide resolved
webknossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/DatasetArray.scala
Outdated
Show resolved
Hide resolved
...ossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/MultiArrayUtils.scala
Outdated
Show resolved
Hide resolved
webknossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/wkw/WKWArray.scala
Outdated
Show resolved
Hide resolved
webknossos-datastore/app/com/scalableminds/webknossos/datastore/dataformats/wkw/WKWFile.scala
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you also do some performance comparisons?
webknossos-datastore/app/com/scalableminds/webknossos/datastore/dataformats/wkw/WKWFile.scala
Show resolved
Hide resolved
Benchmark results on Hetzner cloud CCX53 (32 cores, 128G RAM) Benchmarking with wrk (32 threads, 500 conenctions) and a custom lua script
WKW is now faster than before, 2.5x for color data and 1.15x for segmentation data. Zarr3 and WKW are basically on par. Compare with #7245 (comment) |
URL of deployed dev instance (used for testing):
Steps to test:
TODOs:
expectedUncompressedSizeBytes
?Issues: