-
Notifications
You must be signed in to change notification settings - Fork 237
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add QuestionAnsweringEvaluator #179
Add QuestionAnsweringEvaluator #179
Conversation
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot @fxmarty for adding this - it is in really good shape already. Just a few minor comments.
After adding these evaluators we should really work on adding a bit of documentation for them! |
1096dc7
to
f0e11ff
Compare
540956f
to
6aa0a2f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks pretty good. Left two comments/questions.
Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
fb7fffc
to
56d74bb
Compare
I refactored a bit according to #185 . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! If we can resolve the few open comments plus the few new ones I left we can merge :)
96556c2
to
665df6f
Compare
This PR adds a subclass for
Evaluator
to handle question-answering. I keptlabel_column
for the answers, but I wonder if we should not rename it everywherereference_column
in all evaluators, not sure.