Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Delta housekeeping initial version #101

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

Delta housekeeping initial version #101

wants to merge 26 commits into from

Commits on Dec 18, 2023

  1. delta housekeeping initial commit

    @lorenzorubi-db
    lorenzorubi-db committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    5d7889e View commit details
    Browse the repository at this point in the history

  2. debugging initial version

    lorenzorubi-db committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    90bab27 View commit details
    Browse the repository at this point in the history

  3. convert output to pandas

    @lorenzorubi-db
    lorenzorubi-db committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    94629e0 View commit details
    Browse the repository at this point in the history

  4. debugging -convert output to pandas

    @lorenzorubi-db
    lorenzorubi-db committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    543f852 View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2023

  1. DeltaHousekeepingActions object and tests

    @lorenzorubi-db
    lorenzorubi-db committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    567b303 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2023

  1. added more insights to housekeeping and refactored tests

    @lorenzorubi-db
    lorenzorubi-db committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    bded305 View commit details
    Browse the repository at this point in the history

  2. regression and cleanup

    lorenzorubi-db committed Dec 21, 2023
    Configuration menu
    Copy the full SHA
    cf4ef07 View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. move implementation of map_chunked to a separated branch 

    + improved unit tests
    @lorenzorubi-db
    lorenzorubi-db committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    bc303cd View commit details
    Browse the repository at this point in the history

  2. readability, cleanup, follow discoverx patterns

    @lorenzorubi-db
    lorenzorubi-db committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    feeafaf View commit details
    Browse the repository at this point in the history

  3. debugging on cluster + adding spark session to `DeltaHousekeepingActi… 

    …ons`
    lorenzorubi-db committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    e8a1b66 View commit details
    Browse the repository at this point in the history

  4. simplify scan implementation & remove dependency to BeautifulSoup

    @lorenzorubi-db
    lorenzorubi-db committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    e177ef4 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. faster implementation + unit tests

    @lorenzorubi-db
    lorenzorubi-db committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    023b02f View commit details
    Browse the repository at this point in the history

  2. cleanup

    @lorenzorubi-db
    lorenzorubi-db committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    c2b028f View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. cleanup and PR comments

    @lorenzorubi-db
    lorenzorubi-db committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    0e4c8e5 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. proper use of dbwidgets

    @lorenzorubi-db
    lorenzorubi-db committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    9a9fe6b View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2024

  1. refactoring apply to return a single dataframe

    @lorenzorubi-db
    lorenzorubi-db committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    6c5ecf2 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2024

  1. add test datasets for all housekeeping checks + bug fixes

    @lorenzorubi-db
    lorenzorubi-db committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    5359876 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into delta-housekeeping-notebooks

    @lorenzorubi-db
    lorenzorubi-db committed Feb 4, 2024
    Configuration menu
    Copy the full SHA
    613a290 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2024

  1. fix explain / apply methods

    lorenzorubi-db committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    9758a00 View commit details
    Browse the repository at this point in the history

  2. refactoring to control output column names

    lorenzorubi-db committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    59760f9 View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2024

  1. refactoring to spark API -intermediate commit

    lorenzorubi-db committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    a0d434e View commit details
    Browse the repository at this point in the history

  2. tests with DBR -nan's & timestamps

    lorenzorubi-db committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    0abe9a2 View commit details
    Browse the repository at this point in the history

  3. failing test + cleanup

    lorenzorubi-db committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    16e7ec6 View commit details
    Browse the repository at this point in the history

  4. cleanup

    lorenzorubi-db committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    24edacb View commit details
    Browse the repository at this point in the history

  5. cleanup

    lorenzorubi-db committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    1b1de40 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. remove 'reason' column from the output dfs

    @lorenzorubi-db
    lorenzorubi-db committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    aa671a2 View commit details
    Browse the repository at this point in the history