Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug in named_tensor_tutorial #821

Closed
stas00 opened this issue Jan 13, 2020 · 2 comments
Closed

bug in named_tensor_tutorial #821

stas00 opened this issue Jan 13, 2020 · 2 comments
Labels
docathon-h1-2023 A label for the docathon in H1 2023 medium Tensors Issues relating to tensors (generic issues/questions or specific tensor tutorials)

Comments

@stas00
Copy link
Contributor

stas00 commented Jan 13, 2020

In https://pytorch.org/tutorials/intermediate/named_tensor_tutorial.html I think there is a bug:

dot_prod = q.div_(scale).matmul(k.align_to(..., 'D_head', 'T_key'))
[...]
attn_weights = self.attn_dropout(F.softmax(dot_prod / scale,
                                                   dim='T_key'))

the scaling is done twice, and I think it should be done only once.

Thanks.

@holly1238 holly1238 added the Tensors Issues relating to tensors (generic issues/questions or specific tensor tutorials) label Jul 30, 2021
@svekars svekars added medium docathon-h1-2023 A label for the docathon in H1 2023 labels May 31, 2023
@arunppsg
Copy link
Contributor

arunppsg commented Jun 1, 2023

I think the tutorial has been removed in #1952 since namedtensor has been deprecated.

@stas00
Copy link
Contributor Author

stas00 commented Jun 1, 2023

Thank you for discovering that, @arunppsg - I guess I should close it then.

@stas00 stas00 closed this as completed Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docathon-h1-2023 A label for the docathon in H1 2023 medium Tensors Issues relating to tensors (generic issues/questions or specific tensor tutorials)
Projects
None yet
Development

No branches or pull requests

4 participants