v0.2.0

What's New

evaluator

The evaluator has been extended to three new tasks:

• "image-classification"
• "token-classification"
• "question-answering"

combine

With combine one can bundle several metrics into a single object that can be evaluated in one call and also used in combination with the evalutor.

What's Changed

• Fix typo in WER docs by @pn11 in #147
• Fix rouge outputs by @lvwerra in #158
• add tutorial for custom pipeline by @lvwerra in #154
• refactor evaluator tests by @lvwerra in #155
• rename input_texts to predictions in perplexity by @lvwerra in #157
• Add link to GitHub author by @lewtun in #166
• Add combine to compose multiple evaluations by @lvwerra in #150
• test string casting only on first element by @lvwerra in #159
• remove unused fixtures from unittests by @lvwerra in #170
• Add a test to check that Evaluator evaluations match transformers examples by @fxmarty in #163
• Add smaller model for TextClassificationEvaluator test by @fxmarty in #172
• Add tags to spaces by @lvwerra in #162
• Rename evaluation modules by @lvwerra in #160
• Update push_evaluations_to_hub.py by @lvwerra in #174
• update evaluate dependency for spaces by @lvwerra in #175
• Add ImageClassificationEvaluator by @fxmarty in #173
• attempting to let meteor handle multiple references per prediction by @sashavor in #164
• fixed duplicate calculation of spearmanr function in metrics wrapper. by @benlipkin in #176
• forbid hyphens in template for module names by @lvwerra in #177
• switch from Github to Hub module factory for canonical modules by @lvwerra in #180
• Fix bertscore idf by @lvwerra in #183
• refactor evaluator base and task classes by @lvwerra in #185
• Avoid importing tensorflow when importing evaluate by @NouamaneTazi in #135
• Add QuestionAnsweringEvaluator by @fxmarty in #179
• Evaluator perf by @ola13 in #178
• Fix QuestionAnsweringEvaluator for squad v2, fix examples by @fxmarty in #190
• Rename perf metric evaluator by @lvwerra in #191
• Fix typos in QA Evaluator by @lewtun in #192
• Evaluator device placement by @lvwerra in #193
• Change test command in installation.mdx to use exact_match by @mathemakitten in #194
• Add TokenClassificationEvaluator by @fxmarty in #167
• Pin rouge_score by @albertvillanova in #197
• add poseval by @lvwerra in #195
• Combine docs by @lvwerra in #201
• Evaluator column loading by @lvwerra in #200
• Evaluator documentation by @lvwerra in #199

New Contributors

• @pn11 made their first contribution in #147
• @fxmarty made their first contribution in #163
• @benlipkin made their first contribution in #176
• @NouamaneTazi made their first contribution in #135
• @mathemakitten made their first contribution in #194

Full Changelog: v0.1.2...v0.2.0