Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add DELTA_LENGTH_BYTE_ARRAY encoder and decoder for Parquet #14590

Merged
merged 37 commits into from
Dec 20, 2023
Merged

Add DELTA_LENGTH_BYTE_ARRAY encoder and decoder for Parquet #14590

merged 37 commits into from
Dec 20, 2023

Commits on Dec 6, 2023

  1. add delta length byte array encoder/decoder

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    a95259f View commit details
    Browse the repository at this point in the history

  2. change encoding in file

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    27dfcdd View commit details
    Browse the repository at this point in the history

  3. rename some things

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    73573f3 View commit details
    Browse the repository at this point in the history

  4. a few cleanups

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    b454b76 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4821128 View commit details
    Browse the repository at this point in the history

  6. finish merge of size statistics

    @etseidl
    etseidl committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    fb50fcb View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Merge branch 'rapidsai:branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 8, 2023
    Configuration menu
    Copy the full SHA
    394fe81 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2023

  1. Merge remote-tracking branch 'origin/branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    a2181bc View commit details
    Browse the repository at this point in the history

  2. add syncthreads

    @etseidl
    etseidl committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    07af187 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Merge branch 'rapidsai:branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    fae7388 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    aa94791 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    f806a48 View commit details
    Browse the repository at this point in the history

  4. change unsupported encoding to 15

    @etseidl
    etseidl committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    080e89d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'dlba_enc_dec' of github.com:etseidl/cudf into dlba_enc_dec

    @etseidl
    etseidl committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    0827fd3 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 13, 2023
    Configuration menu
    Copy the full SHA
    7e9d752 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. pass random number genertor into data gen functions

    @etseidl
    etseidl committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    8b0f277 View commit details
    Browse the repository at this point in the history

  2. implement suggestion from review 

    Co-authored-by: Nghia Truong <7416935+ttnghia@users.noreply.github.com>
    @etseidl @ttnghia
    etseidl and ttnghia authored Dec 14, 2023
    Configuration menu
    Copy the full SHA
    4bbbf63 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. implement suggestion from review 

    Co-authored-by: Vukasin Milovanovic <vmilovanovic@nvidia.com>
    @etseidl @vuule
    etseidl and vuule authored Dec 15, 2023
    Configuration menu
    Copy the full SHA
    901644d View commit details
    Browse the repository at this point in the history

  2. remove __host__ decorator

    @etseidl
    etseidl committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    0934708 View commit details
    Browse the repository at this point in the history

  3. change skip_values_and_sum to run on a single warp

    @etseidl
    etseidl committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    ad42470 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 15, 2023
    Configuration menu
    Copy the full SHA
    c552ffb View commit details
    Browse the repository at this point in the history

  5. implement suggestion from review 

    Co-authored-by: Vukasin Milovanovic <vmilovanovic@nvidia.com>
    @etseidl @vuule
    etseidl and vuule authored Dec 15, 2023
    Configuration menu
    Copy the full SHA
    ace5be3 View commit details
    Browse the repository at this point in the history

  6. move delta char len calculation

    @etseidl
    etseidl committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    3523a36 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2023

  1. a few cleanups

    @etseidl
    etseidl committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    c028f2a View commit details
    Browse the repository at this point in the history

  2. remove some outdated TODOs and superfluous threadfences

    @etseidl
    etseidl committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    4c2dc56 View commit details
    Browse the repository at this point in the history

  3. handle non-string byte arrays

    @etseidl
    etseidl committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    c07bea2 View commit details
    Browse the repository at this point in the history

  4. parquet-mr does not like duplicate column names

    @etseidl
    etseidl committed Dec 16, 2023
    Configuration menu
    Copy the full SHA
    62f8b4a View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 18, 2023
    Configuration menu
    Copy the full SHA
    36478bf View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2023

  1. fix for writing all-null column

    @etseidl
    etseidl committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    b4e6999 View commit details
    Browse the repository at this point in the history

  2. fix for reading single null row

    @etseidl
    etseidl committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    385dce1 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    edd3c13 View commit details
    Browse the repository at this point in the history

  4. finish merge

    @etseidl
    etseidl committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    6a25a6a View commit details
    Browse the repository at this point in the history

  5. make sure header is written if all values are null

    @etseidl
    etseidl committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    0d0c95f View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2023

  1. add extra delta tests

    @etseidl
    etseidl committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    e0d8cf1 View commit details
    Browse the repository at this point in the history

  2. change param name to match use

    @etseidl
    etseidl committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    a65f512 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-24.02' into dlba_enc_dec

    @etseidl
    etseidl authored Dec 20, 2023
    Configuration menu
    Copy the full SHA
    b77b9ee View commit details
    Browse the repository at this point in the history

  4. try 2 at consistent naming

    @etseidl
    etseidl committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    fc282fe View commit details
    Browse the repository at this point in the history