Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FT] To keep target_for_fewshot_sorting for another purpose #240

Open
sadra-barikbin opened this issue Jul 28, 2024 · 0 comments · May be fixed by #241
Open

[FT] To keep target_for_fewshot_sorting for another purpose #240

sadra-barikbin opened this issue Jul 28, 2024 · 0 comments · May be fixed by #241
Labels
feature request New feature/request

Comments

@sadra-barikbin
Copy link
Contributor

Hi there!

In the code it's stated that target_for_fewshot_sorting might be removed in the future, maybe because line["__few_shots"] is introduced.

target_for_fewshot_sorting: Optional[str] = None # will probably have to be removed in the future

I suggest keeping target_for_fewshot_sorting, but for another purpose. To let the user determine the criteria by which fewshot examples are sorted and picked in fewshot manager.

My concrete example is in fewshot structured generation in which model is to generate function call json representations. I want to pick the examples, in a balanced setting, according to the name of the functions but currently they're picked according to the whole representation of the function call which doesn't make sense as all of them are different.

@sadra-barikbin sadra-barikbin added the feature request New feature/request label Jul 28, 2024
@sadra-barikbin sadra-barikbin changed the title [FT] To keep or to remove target_for_fewshot_sorting [FT] To keep target_for_fewshot_sorting for another purpose Jul 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature/request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant