-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement index_location setting for Zarr3 Datasets #7553
Conversation
...ssos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/zarr3/Zarr3Array.scala
Outdated
Show resolved
Hide resolved
af8a201
to
8eda514
Compare
@@ -122,8 +119,22 @@ class Zarr3Array(vaultPath: VaultPath, | |||
parsed = parseShardIndex(shardIndexRaw) | |||
} yield parsed | |||
|
|||
private lazy val shardIndexChecksumLength = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this works for most datasets, but not for all. Although, it doesn't make a lot of sense, it is spec-compliant to have multiple crc32c checksums. In the future, there might be other codecs that change the size of the shard index.
A more correct way would be to compute the size of the index based on the entire index_codecs pipeline. zarrita uses for that the compute_encoded_size
methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At least, there should be some assertions that only supported index configurations are usable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
webknossos-datastore/app/com/scalableminds/webknossos/datastore/datareaders/zarr3/Codecs.scala
Outdated
Show resolved
Hide resolved
Approving this now based on normans previous feedback. @normanrz if you have other wishes, let’s add them as a follow-up issue :) that way this PR isn’t blocked |
URL of deployed dev instance (used for testing):
Steps to test:
TODOs:
Issues:
(Please delete unneeded items, merge only when none are left open)