Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p.add_(torch.sign(update), alpha=-group['lr']) has been made more eff… #1193

Merged
merged 1 commit into from
Aug 9, 2023

Conversation

sachinspanicker
Copy link
Contributor

…icient by using the torch.sign function's inplace mode. This will prevent the need to create a new tensor for the updated parameter, which can save a significant amount of time for large models.

…icient by using the torch.sign function's inplace mode. This will prevent the need to create a new tensor for the updated parameter, which can save a significant amount of time for large models.
@google-cla
Copy link

google-cla bot commented May 13, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@crazydonkey200
Copy link
Collaborator

Sorry that I have missed this pull request earlier. Thanks for the contribution!

@crazydonkey200 crazydonkey200 merged commit c7392f2 into google:master Aug 9, 2023
@liuslnlp
Copy link

Error

add_() received an invalid combination of arguments - got unrecognized keyword arguments: inplace

My torch version

>>> import torch
>>> torch.__version__
'2.1.0.dev20230829+cu121'

@fbotp
Copy link

fbotp commented Oct 25, 2023

@liuslnlp add_ is already an inplace method, so I remove this parameter in method, and it works, you can have a try

@yongchanghao
Copy link
Contributor

yongchanghao commented Nov 2, 2023

As the comments above suggested, there isn't such an argument as inplace=True in torch.Tensor.add_. for both torch versions above or under 2.0. I believe this code hasn't been tested. A rollback might be needed? @crazydonkey200

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants