Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BREAKING] 003 depreciation: changing autoannotators to AlpacaEval 1. #88

Merged
merged 5 commits into from
Feb 24, 2024
Merged

[BREAKING] 003 depreciation: changing autoannotators to AlpacaEval 1. #88

merged 5 commits into from
Feb 24, 2024

Conversation

YannDubs
Copy link
Collaborator

@YannDubs YannDubs commented Feb 24, 2024 •
edited
Loading

text-davinci-003 is depreciated. To ensure that the codebase is still usable I'm switching to AlapcaEval annotations.

Changes:

  • merge instruction and input (to be able to use AE 1). Note that we only do that for preferences, I did not modify the code for generations/inference of the analyzed models.
  • switch to AlpacaEval 1
  • test
  • update annotation notebook
  • update AE dependencies
  • update readme

FYI @lxuechen

closes #87
closes #85
closes #68

@YannDubs YannDubs merged commit d47689b into main Feb 24, 2024
@YannDubs YannDubs deleted the yann/af_use_ae branch February 24, 2024 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@YannDubs