Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test standalone Zarr streaming library (do not review, do not merge) #289

Closed
wants to merge 90 commits into from
Closed

test standalone Zarr streaming library (do not review, do not merge) #289

wants to merge 90 commits into from

Commits on Aug 21, 2024

  1. Move driver code/tests to driver-specific folders.

    @aliddell
    aliddell committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    c5ea2ae View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. ZarrStreamSettings

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    e92de52 View commit details
    Browse the repository at this point in the history

  2. Some reorg

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    8099252 View commit details
    Browse the repository at this point in the history

  3. Some more reorg

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    8c9952e View commit details
    Browse the repository at this point in the history

  4. More C++

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    86a21dd View commit details
    Browse the repository at this point in the history

  5. Add thread pool

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    cfe8808 View commit details
    Browse the repository at this point in the history

  6. Create filesystem store on stream creation.

    @aliddell
    aliddell committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    a7ab8ef View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Add a neat logger.

    @aliddell
    aliddell committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    968c735 View commit details
    Browse the repository at this point in the history

  2. Add in the blosc compressor.

    @aliddell
    aliddell committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    00375b9 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. wip

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    a79b3e9 View commit details
    Browse the repository at this point in the history

  2. update acquire-common to main

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    6f69c46 View commit details
    Browse the repository at this point in the history

  3. Break out S3 unit tests. Some other tests.

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    04d639d View commit details
    Browse the repository at this point in the history

  4. Install miniocpp 0.3.0 from vcpkg.

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    8b4efcf View commit details
    Browse the repository at this point in the history

  5. Integrate S3Sink.

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    30a7a2f View commit details
    Browse the repository at this point in the history

  6. wip

    @aliddell
    aliddell committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    92888e5 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. wip

    @aliddell
    aliddell committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b545922 View commit details
    Browse the repository at this point in the history

  2. Integrate SinkCreator.

    @aliddell
    aliddell committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    d88ebdc View commit details
    Browse the repository at this point in the history

  3. Add ZarrDataType.

    @aliddell
    aliddell committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    414c978 View commit details
    Browse the repository at this point in the history

  4. wip: moving functions from array.writer.cpp to zarr.common.cpp

    @aliddell
    aliddell committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    1e2ef2d View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Fix name collisions in tests.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a2e46df View commit details
    Browse the repository at this point in the history

  2. Fix bug in chunk_lattice_index caused by dimension reversal.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    fdb14d7 View commit details
    Browse the repository at this point in the history

  3. Break out common functions from array.writer.cpp into zarr.common.cpp.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    5baf4f2 View commit details
    Browse the repository at this point in the history

  4. Integrate ArrayWriter class.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    164aa32 View commit details
    Browse the repository at this point in the history

  5. Embrace the brace.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    b370f69 View commit details
    Browse the repository at this point in the history

  6. Undefine min macro in array.writer.cpp.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    165df23 View commit details
    Browse the repository at this point in the history

  7. ZarrV2ArrayWriter

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    b1ac3fd View commit details
    Browse the repository at this point in the history

  8. ZarrV3ArrayWriter

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    f2ca319 View commit details
    Browse the repository at this point in the history

  9. wip: cleanup; check S3 connection. Start to allocate writers.

    @aliddell
    aliddell committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    81b3392 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Moar compression settings.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    31dc6db View commit details
    Browse the repository at this point in the history

  2. We have stream create and destroy.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    691faa0 View commit details
    Browse the repository at this point in the history

  3. wip: implement ZarrStream_s::append (no tests)

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    4b4f4eb View commit details
    Browse the repository at this point in the history

  4. Test ZarrV2 stream to disk.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    a144cb6 View commit details
    Browse the repository at this point in the history

  5. Fix flush/rollover bug.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    5ff7729 View commit details
    Browse the repository at this point in the history

  6. Test ZarrV3 stream to disk.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    7145770 View commit details
    Browse the repository at this point in the history

  7. Update ZarrV2 streaming test to 5d

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    4f30363 View commit details
    Browse the repository at this point in the history

  8. Compression tests.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    4c7fa47 View commit details
    Browse the repository at this point in the history

  9. Install headers.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    93e7b6c View commit details
    Browse the repository at this point in the history

  10. wip: fix S3 storage paths

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    ee5a8b5 View commit details
    Browse the repository at this point in the history

  11. Write and validate raw Zarr V2 to S3.

    @aliddell
    aliddell committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    4e0f960 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Some cleanup.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c40a0c4 View commit details
    Browse the repository at this point in the history

  2. Remove ArrayWriter::finalize().

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    0fc4799 View commit details
    Browse the repository at this point in the history

  3. Run S3 tests on Ubuntu.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    9dde5c3 View commit details
    Browse the repository at this point in the history

  4. derp

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    cca74a0 View commit details
    Browse the repository at this point in the history

  5. remove miniocpp binaries

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    30560f3 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    9251785 View commit details
    Browse the repository at this point in the history

  7. Remove zarr_errors.h.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    dd47b96 View commit details
    Browse the repository at this point in the history

  8. Define API

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    a612210 View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'upstream/main' into standalone-sequence

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    72cfe47 View commit details
    Browse the repository at this point in the history

  10. Docstrings.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    e57493d View commit details
    Browse the repository at this point in the history

  11. More documentation

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    e5afc9d View commit details
    Browse the repository at this point in the history

  12. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    ac651fa View commit details
    Browse the repository at this point in the history

  13. Some rearranging.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    74bd626 View commit details
    Browse the repository at this point in the history

  14. Move driver tests to tests/driver. Add logger and stream settings code.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    4ef1c86 View commit details
    Browse the repository at this point in the history

  15. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    a906a11 View commit details
    Browse the repository at this point in the history

  16. extern "C" some things

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    dbd1e35 View commit details
    Browse the repository at this point in the history

  17. Get it building.

    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    18ac6d4 View commit details
    Browse the repository at this point in the history

  18. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    04576eb View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Swap order of ZarrDimensionType_Channel and ZarrDimensionType_Time.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    4397b75 View commit details
    Browse the repository at this point in the history

  2. Add external metadata handling.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    ed8be14 View commit details
    Browse the repository at this point in the history

  3. Get the driver building again.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    c7a4827 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    0d88249 View commit details
    Browse the repository at this point in the history

  5. Do JSON validation when setting external metadata.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    6843827 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    fb2b000 View commit details
    Browse the repository at this point in the history

  7. Write external metadata.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    3d12b1d View commit details
    Browse the repository at this point in the history

  8. Get some more tests passing.

    @aliddell
    aliddell committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    4937bea View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Set -fPIC.

    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    9b397c6 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    69cb9b1 View commit details
    Browse the repository at this point in the history

  3. Get tests passing again.

    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e1e6ab0 View commit details
    Browse the repository at this point in the history

  4. Allow setting empty external metadata.

    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6a4aa86 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    e3807a6 View commit details
    Browse the repository at this point in the history

  6. Copy settings.

    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6cae047 View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    93230d3 View commit details
    Browse the repository at this point in the history

  8. Get all non-multiscale driver tests passing.

    @aliddell
    aliddell committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    44fcba2 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Get multiscale tests passing.

    @aliddell
    aliddell committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    2b870c1 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into standalone

    @aliddell
    aliddell committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    453eb9d View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into standalone-sequence

    @aliddell
    aliddell committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    3fa201c View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/standalone-sequence' into stan… 

    …dalone
    @aliddell
    aliddell committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    dcd8ab0 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2024

  1. Fix format warnings

    @aliddell
    aliddell committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    e4f1c1a View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Rearrange some files

    @aliddell
    aliddell committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    b313658 View commit details
    Browse the repository at this point in the history

  2. Consolidate set_store_path + S3 settings.

    @aliddell
    aliddell committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    2fb6d73 View commit details
    Browse the repository at this point in the history

  3. Consolidate set_compression_settings.

    @aliddell
    aliddell committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    1c6298e View commit details
    Browse the repository at this point in the history

  4. Consolidate set_dimension.

    @aliddell
    aliddell committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    005e6b3 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Fix up getters, remove dead code.

    @aliddell
    aliddell committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    1777924 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. Build on Windows

    @aliddell
    aliddell committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    43860d9 View commit details
    Browse the repository at this point in the history

  2. Temporarily disable mac build. Shared libs? y/n

    @aliddell
    aliddell committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    e328940 View commit details
    Browse the repository at this point in the history

  3. Temporarily disable mac build. Shared libs? y/n

    @aliddell
    aliddell committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    a2ab3ba View commit details
    Browse the repository at this point in the history

  4. Just build shared

    @aliddell
    aliddell committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d3bb9fc View commit details
    Browse the repository at this point in the history

  5. Implement Zarr_get_api_version

    @aliddell
    aliddell committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    ee97d3e View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Don't std::move dimensions when setting up a stream.

    @aliddell
    aliddell committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    afa9ace View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. export all the things

    @aliddell
    aliddell committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    e652c90 View commit details
    Browse the repository at this point in the history