Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove FSDP restriction from PyTorch 1.13 #3395

Merged
merged 2 commits into from
Jun 12, 2024

Conversation

mvpatel2000
Copy link
Contributor

What does this PR do?

Remove FSDP restriction from PyTorch 1.13. We added this to give a friendlier error for users when PyTorch had a bug, but the corresponding issue is closed and fixed and has been patched for a while since PyTorch 2.

Copy link
Contributor

@snarayan21 snarayan21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but can you include a run name that successfully runs with torch 1.3 and fp32 param_dtype?

@mvpatel2000
Copy link
Contributor Author

lgtm, but can you include a run name that successfully runs with torch 1.3 and fp32 param_dtype?

It does not run with 1.13, but we've moved off from that and upgraded

Copy link
Contributor

@snarayan21 snarayan21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah i misunderstood. lgtm

Copy link
Contributor

@b-chu b-chu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mvpatel2000 mvpatel2000 merged commit ba82cc9 into mosaicml:dev Jun 12, 2024
17 checks passed
@mvpatel2000 mvpatel2000 deleted the mvpatel2000/remove-113-code branch June 12, 2024 15:16
mvpatel2000 added a commit to mvpatel2000/composer that referenced this pull request Jul 21, 2024
mvpatel2000 added a commit that referenced this pull request Jul 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants