Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Call set_epoch on Dataloader.batch_sampler if defined #3124

Merged
merged 31 commits into from
Apr 2, 2024
Merged

Call set_epoch on Dataloader.batch_sampler if defined #3124

merged 31 commits into from
Apr 2, 2024

Commits on Mar 21, 2024

  1. adding _get_dist_sampler function

    @Ghelfi
    Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    408179e View commit details
    Browse the repository at this point in the history

  2. apply _get_dist_sampler across all trainer.py

    @Ghelfi
    Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    514a8b1 View commit details
    Browse the repository at this point in the history

  3. Switch condition to check if batch_sampler if filled first since samp… 

    …ler is always defined
    @Ghelfi
    Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    ef5540d View commit details
    Browse the repository at this point in the history

  4. adding docstring

    @Ghelfi
    Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    413c4c3 View commit details
    Browse the repository at this point in the history

  5. removing antipattenr from _get_sampler and linting

    @Ghelfi
    Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    ed8535f View commit details
    Browse the repository at this point in the history

  6. removed default attrs from exception in the attrs dict (mosaicml#3126)

    @jjanezhang @Ghelfi
    jjanezhang authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    ca7873f View commit details
    Browse the repository at this point in the history

  7. Bump coverage[toml] from 7.4.3 to 7.4.4 (mosaicml#3121) 

    Bumps [coverage[toml]](https://github.com/nedbat/coveragepy) from 7.4.3 to 7.4.4.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@7.4.3...7.4.4)

---
updated-dependencies:
- dependency-name: coverage[toml]
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    @dependabot @Ghelfi
    dependabot[bot] authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    027156e View commit details
    Browse the repository at this point in the history

  8. Refactor initialization (mosaicml#3127) 

    Co-authored-by: practicinginhell <oneforall1412@gmail.com>
    @Practicinginhell @Ghelfi
    2 people authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    9f768a4 View commit details
    Browse the repository at this point in the history

  9. Bump databricks sdk version (mosaicml#3128)

    @dakinggg @Ghelfi
    dakinggg authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    06b8afd View commit details
    Browse the repository at this point in the history

  10. Update packaging requirement from <23.3,>=21.3.0 to >=21.3.0,<24.1 (m… 

    …osaicml#3122)

Updates the requirements on [packaging](https://github.com/pypa/packaging) to permit the latest version.
- [Release notes](https://github.com/pypa/packaging/releases)
- [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst)
- [Commits](pypa/packaging@21.3...24.0)

---
updated-dependencies:
- dependency-name: packaging
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mihir Patel <mihir.v.patel7@gmail.com>
    @dependabot @mvpatel2000 @Ghelfi
    2 people authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    c9b41f7 View commit details
    Browse the repository at this point in the history

  11. remove rng from save weights only ckpt (mosaicml#3129)

    @eracah @Ghelfi
    eracah authored and Ghelfi committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    0c42c51 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. More compression options (mosaicml#3118) 

    * fix documentation about which file extensions perform compression

* refactor tar handling when writing checkpoint files

* add CLI compressor options to checkpoint saving

* update documentation

* restructure CLI compressor handling. Added more tests

* fix skipping test when compressor not installed

* add documentation and fix some code style issues

* use correct file extension for zstd

* add missing full stop

* fix failing tests

* remove type ignore

Co-authored-by: Mihir Patel <mihir.v.patel7@gmail.com>

* combine imports into one

* fix capitalisation

Co-authored-by: Mihir Patel <mihir.v.patel7@gmail.com>

---------

Co-authored-by: Mihir Patel <mihir.v.patel7@gmail.com>
    @mbway @mvpatel2000 @Ghelfi
    2 people authored and Ghelfi committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ccaeec5 View commit details
    Browse the repository at this point in the history

  2. Only broadcast distcp files (mosaicml#3130) 

    * filter

* remove metadata skip
    @mvpatel2000 @Ghelfi
    mvpatel2000 authored and Ghelfi committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    2fdbf45 View commit details
    Browse the repository at this point in the history

  3. Bump version to 0.21 (mosaicml#3132) 

    * bump version to 021

* fix lint

* fix docstring

* remove old ones
    @mvpatel2000 @Ghelfi
    mvpatel2000 authored and Ghelfi committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    88677c3 View commit details
    Browse the repository at this point in the history

  4. Adding a test for checking the update of epoch on batch_sampler

    @Ghelfi
    Ghelfi committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    5161f59 View commit details
    Browse the repository at this point in the history

  5. proper formating

    @Ghelfi
    Ghelfi committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    c731f67 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. adding _get_dist_sampler function

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    570558d View commit details
    Browse the repository at this point in the history

  2. apply _get_dist_sampler across all trainer.py

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    ab09d43 View commit details
    Browse the repository at this point in the history

  3. Switch condition to check if batch_sampler if filled first since samp… 

    …ler is always defined
    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    df73325 View commit details
    Browse the repository at this point in the history

  4. adding docstring

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    61da1b8 View commit details
    Browse the repository at this point in the history

  5. removing antipattenr from _get_sampler and linting

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    12245cc View commit details
    Browse the repository at this point in the history

  6. Adding a test for checking the update of epoch on batch_sampler

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b2c147a View commit details
    Browse the repository at this point in the history

  7. proper formating

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    68e6d9d View commit details
    Browse the repository at this point in the history

  8. fixing test

    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    98c4e51 View commit details
    Browse the repository at this point in the history

  9. Chang _get_sampler to _get_distributed_sampler to bypass torch defaul… 

    …t batch_sampler
    @Ghelfi
    Ghelfi committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b3e6551 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. Move batch_sampler instanciation for test

    @Ghelfi
    Ghelfi committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    76b888b View commit details
    Browse the repository at this point in the history

  2. fix batch_sampler setting in eval_loop

    @Ghelfi
    Ghelfi committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    dd45d3e View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. Merge branch 'dev' into set-epoch-on-batch-sampler

    @mvpatel2000
    mvpatel2000 authored Apr 1, 2024
    Configuration menu
    Copy the full SHA
    14eb4f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. trainer

    @mvpatel2000
    mvpatel2000 committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    448d531 View commit details
    Browse the repository at this point in the history

  2. fix

    @mvpatel2000
    mvpatel2000 committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    522afe4 View commit details
    Browse the repository at this point in the history

  3. lint

    @mvpatel2000
    mvpatel2000 committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    fdb2efa View commit details
    Browse the repository at this point in the history