-
Notifications
You must be signed in to change notification settings - Fork 237
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make rouge support multi-ref #229
Conversation
The documentation is not available anymore as the PR was closed or merged. |
metrics/rouge/README.md
Outdated
### Inputs | ||
- **predictions** (`list`): list of predictions to score. Each prediction | ||
should be a string with tokens separated by spaces. | ||
- **references** (`list`): list of reference for each prediction. Each | ||
- **references** (`list` or `list[list]`): list of reference for each prediction or a list of several predictions. Each |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Should this be "a list of several references for this prediction" as opposed to "a list of several predictions"? I assume the change is meant to surface that each prediction can be correctly mapped to several references.
Otherwise I tested this and it LGTM :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Absolutely, thanks for noticing this!
* make rouge support multi-ref * update rouge dependency * clean style * fix rouge dependency * fix readme
* make rouge support multi-ref * update rouge dependency * clean style * fix rouge dependency * fix readme
This PR makes use of the
score_multi
method added inrouge_score==0.12.0
and allows to pass multiple references per prediction. This makes this metric now compatible withbleu
andmeteor
.Closes #118