Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StaticEmbedding 中的 dropout 问题 #417

Open
XieDelong opened this issue May 24, 2022 · 1 comment
Open

StaticEmbedding 中的 dropout 问题 #417

XieDelong opened this issue May 24, 2022 · 1 comment

Comments

@XieDelong
Copy link

在 StaticEmbedding forward 方法中,
words = self.drop_word(words) words = self.embedding(words) words = self.dropout(words)

drop_word 函数有 self.training 的判断,但是 dropout 没有,导致在推断的时候,同样的 query 预测结果不一致问题

@yhcc
Copy link
Member

yhcc commented May 24, 2022

由于 self.dropout 是nn.Dropout,它本身是受到module的training状态管控的,所以如果调用了 StaticEmbedding.eval() 的话,这个 dropout 就是自动关闭的了。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants