Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LAMB Optimizer - rename weight_decay_rate to weight_decay #2610

Closed
ksachdeva opened this issue Nov 19, 2021 · 2 comments · Fixed by #2611
Closed

LAMB Optimizer - rename weight_decay_rate to weight_decay #2610

ksachdeva opened this issue Nov 19, 2021 · 2 comments · Fixed by #2611

Comments

@ksachdeva
Copy link
Contributor

Describe the bug

Unlike the rest of the optimizers API, in LAMB the name of the argument for weight_decay is weight_decay_rate

In my experiments where I try various different optimizers, this makes me add a special condition for LAMB because of this different name of the argument.

Code to reproduce the issue

weight_decay_rate: FloatTensorLike = 0.0,

@bhack
Copy link
Contributor

bhack commented Nov 19, 2021

I think that you can introduce a deprecation and add the new argument name with a PR.
You can handle the deprecation like in #2048

@ksachdeva
Copy link
Contributor Author

Thanks. I will send the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants