Update tutorial.rst to include section about accessing Zip Files on S3 #1615
Conversation
|
Thanks, @jeffpeck10x! I've triggered the checks. |
|
Beautiful!
Would you like to add yourself to docs/release.txt |
|
Thank you @joshmoore . I have updated the release notes. |
Per discussion here, add information about about accessing zip files on s3: zarr-developers#1613
|
Thanks for the review, @d-v-b, and @jeffpeck10x, for getting this started! 🎉 If anyone else would like to further improve these (or other 😄) docs, please feel free. |
|
Sorry for being a pedant, but could we revert? I'm not done with my suggestions :) For example, the new docs still contain references to |
| Accessing Zip Files on S3 | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| The built-in `ZipStore` will only work with paths on the local file-system, however |
There was a problem hiding this comment.
I'm not sure how links work in sphinx, but could we make this a link to the API docs for ZipStore?
|
|
||
| The built-in `ZipStore` will only work with paths on the local file-system, however | ||
| it is also possible to access ``.zarr.zip`` data on the cloud. Here is an example of | ||
| accessing a zipped Zarr file on s3: |
There was a problem hiding this comment.
I think I prefer "zipped Zarr hierarchy" to "zipped Zarr file"
|
|
||
| >>> store = zarr.storage.FSStore(url=f"zip::{s3_path}", mode="r") | ||
|
|
||
| This can be especially useful if you have a very large ``.zarr.zip`` file on s3 |
There was a problem hiding this comment.
same as above -- let's replace zarr.zip with "zipped Zarr hierarchy"
| >>> fs = ZipFileSystem(f, mode="r") | ||
| >>> store = FSMap("", fs, check=False) | ||
| >>> | ||
| >>> # cache is optional, but may be a good idea depending on the situation |
There was a problem hiding this comment.
Which situations benefit from the cache?
There was a problem hiding this comment.
If you are going to access the same chunks of data multiple times
|
I'm also happy to make the changes myself, if people broadly agree with what I'm proposing |
|
@joshmoore any objections if I revert the merge and make a few tweaks, then re-merge? |
|
Let's just start with a new PR from your side. Then @jeffpeck10x can decide if he's going to jump back in (rather than definitely getting the notifications, etc. 🙂) |
|
@d-v-b I appreciate the careful review. Please go ahead and make those changes. |
zarr-developers#1615) * Update tutorial.rst to include section about accessing Zip Files on S3 Per discussion here, add information about about accessing zip files on s3: zarr-developers#1613 * Update release.rst * Implement d-v-b's suggestions --------- Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com> Co-authored-by: Josh Moore <josh@openmicroscopy.org>
Per discussion here, add information about about accessing zip files on s3: #1613
[Description of PR]
TODO: