Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix][FSDP] fix weight init when using apply() (fixes #490 and #444) #543

Merged
merged 6 commits into from
Mar 20, 2021
Merged

[fix][FSDP] fix weight init when using apply() (fixes #490 and #444) #543

merged 6 commits into from
Mar 20, 2021

Commits on Mar 20, 2021

  1. Add new test for weight init (fails)

    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    d184867 View commit details
    Browse the repository at this point in the history

  2. Set FSDP.compute_device so summon_full_params works before module mov… 

    …es to CUDA
    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    293e151 View commit details
    Browse the repository at this point in the history

  3. Override FSDP.apply to enable custom weight init

    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    442f3dd View commit details
    Browse the repository at this point in the history

  4. Fix SyncBN

    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    7ba4542 View commit details
    Browse the repository at this point in the history

  5. Very strange fix for PyTorch 1.6 -- remove torch.no_grad...

    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    7445fc3 View commit details
    Browse the repository at this point in the history

  6. CR

    @myleott
    myleott committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    51b61f9 View commit details
    Browse the repository at this point in the history