Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite state_dict in a more pytorch idiomatic way #60

Closed
blefaudeux opened this issue Sep 2, 2020 · 1 comment · Fixed by #61
Closed

Rewrite state_dict in a more pytorch idiomatic way #60

blefaudeux opened this issue Sep 2, 2020 · 1 comment · Fixed by #61
Assignees

Comments

@blefaudeux
Copy link
Contributor

🚀 Feature

Change the param_groups handling in the state dict, in order to follow more closely the default PyTorch assumptions
https://pytorch.org/docs/stable/optim.html#torch.optim.Optimizer.state_dict

Motivation

  • Some users may assume that the default pytorch optimizer interface with respect to the state dict is respected by fairscale/oss
  • People familiar with pytorch optimizers would have an easier learning curve when peeking into OSS

Pitch

Rewrite the exposed state dict in order to return "state" and "param_groups" in accordance to pytorch expectations, without duplications

Alternatives

  • rely on the python/pytorch memory model to remove duplicates in memory and while serializing
  • add wrappers on the user side

Additional context

@blefaudeux blefaudeux self-assigned this Sep 2, 2020
@blefaudeux
Copy link
Contributor Author

cc @msbaines @mannatsingh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant