New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could I arise one PR to fix MRR metric statics method ??? #119

Open

Yazooliu opened this issue Dec 1, 2023 · 1 comment

Yazooliu commented Dec 1, 2023 •

edited

Loading

I saw
https://github.com/PaddlePaddle/RocketQA/blob/main/research/DuReader-Retrieval-Baseline/metric/evaluation.py

line 135 is wrong.

for MRR 定义：

计算分母时，分母表示的是问题对应的推理答案长度。

而您的代码在计算时分母是ground_truth的长度。实际应用中ground_truth的长度和推理结果的长度不一定相同，所以您的代码有问题，请求提交PR修改，请批准。

祝好
Yazooliu

Author

Yazooliu commented Dec 19, 2023

Any respond？Could I try?

Yazooliu mentioned this issue

Update evaluation.py MRR metrics as previous metrics is Wrong #120

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment