Skip to content

Latest commit

 

History

History
35 lines (26 loc) · 1.35 KB

CHANGELOG.md

File metadata and controls

35 lines (26 loc) · 1.35 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

## [Unreleased]

Changed

  • Now requires Python 3.11 and above.

Fixed

  • Now catches a new HFValidationError caused when attempting to check if a model exists on the Hugging Face Hub.

Security

  • Updated urllib3 to 2.0.7 due to a security update.

[v0.1.0] - 2023-02-22

Added

  • Support for evaluation of local Hugging Face models.
  • Tests for the question_answering-task.
  • The automatic_speech_recognition-task.
  • Util functions, leaderboard_utils, for interacting with the associated REST-api which interacts with the leaderboard holding the evaluation results.
  • A new function in the evaluator module, called _send_results_to_leaderboard which sends evaluation results to the leaderboard using the util functions from leaderboard_utils, and tests for this function and leaderboard_utils.
  • The discourse-coherence-task.
  • Support for integer labels.

[v0.0.1] - 2022-08-29

Added

  • First release, which includes evaluation of sentiment models from the Hugging Face Hub. This can be run with the CLI using the evaluate command, or via a script using the Evaluator class.