Add regression task and fix stsb to use it #505
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #505 +/- ##
=======================================
Coverage 94.13% 94.14%
=======================================
Files 194 197 +3
Lines 7803 7816 +13
=======================================
+ Hits 7345 7358 +13
Misses 458 458 ☔ View full report in Codecov by Sentry. |
| add_to_catalog( | ||
| OutputQuantizingTemplate( | ||
| input_format=""" | ||
| Given this text: '{text1}' and this text: '{text2}', on a scale of {min_value} to {max_value}, what is the {type_of_value} of this texts? |
There was a problem hiding this comment.
Why not keep to existing format?
Given this sentence: '{text1}', on a scale of {min_value} to {max_value}, what is the {type_of_relation} to this text {text2}
prepare/cards/stsb.py
Outdated
| } | ||
| ), | ||
| AddFields( | ||
| fields={"type_of_value": "similarity", "min_value": "1", "max_value": "5"} |
There was a problem hiding this comment.
type_of_value -> type_of_relation
| "templates.regression.bounded.pair.simple", | ||
| ] | ||
| ), | ||
| "templates.regression.bounded.pair.all", |
There was a problem hiding this comment.
I think it should be more clear:
templates.regression.bounded.pair -> templates.regression.two_texts
Is there a need for an unbounded regression between text?
.
| add_to_catalog( | ||
| OutputQuantizingTemplate( | ||
| input_format=""" | ||
| Given this text: '{text1}', on a scale of {min_value} to {max_value}, what is the {type_of_value} of this text? |
There was a problem hiding this comment.
type_of_value -> attribute
[ Maybe a better name could be found? ]
elronbandel
force-pushed
the
add-regression-tasks-and-fix-stsb-to-use-it
branch
2 times, most recently
from
c590956
to
6a90b07
Compare
Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
elronbandel
force-pushed
the
add-regression-tasks-and-fix-stsb-to-use-it
branch
from
6a90b07
to
d23ee23
Compare
No description provided.