Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Raise an error when lightning replaces an existing sampler #2020

Merged
merged 6 commits into from
Jun 2, 2020
Merged

Raise an error when lightning replaces an existing sampler #2020

merged 6 commits into from
Jun 2, 2020

Commits on May 30, 2020

  1. Raise an error when lightning replaces an existing sampler 

    Currently, Trainer replaces the existing sampler with DistributedSampler
if running distributing training and `replace_sampler_ddp=True` (default
behaviour). If a user has configured an existing sampler, this would
lead to widely different results if running a distributed vs
non-distributed training.

This PR fixes this by raising an Error if user has configured a sampler
and uses `replace_sampler_ddp=True`. The recommended behavior from now
on is to either remove the sampler or set `replace_sampler_ddp=False`
    @devashishshankar
    devashishshankar committed May 30, 2020
    Configuration menu
    Copy the full SHA
    ae2d935 View commit details
    Browse the repository at this point in the history

  2. Fix tests

    @devashishshankar
    devashishshankar committed May 30, 2020
    Configuration menu
    Copy the full SHA
    cffee08 View commit details
    Browse the repository at this point in the history

  3. Simpler fix

    @devashishshankar
    devashishshankar committed May 30, 2020
    Configuration menu
    Copy the full SHA
    98aa87b View commit details
    Browse the repository at this point in the history

  4. Fix tests

    @devashishshankar
    devashishshankar committed May 30, 2020
    Configuration menu
    Copy the full SHA
    2a849c3 View commit details
    Browse the repository at this point in the history

  5. Make inner method protected

    @devashishshankar
    devashishshankar committed May 30, 2020
    Configuration menu
    Copy the full SHA
    4b62609 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2020

  1. Apply suggestions from code review

    @Borda
    Borda committed Jun 1, 2020
    Configuration menu
    Copy the full SHA
    a434593 View commit details
    Browse the repository at this point in the history