Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add is_in and stop_at generation constraints to transformers integration #108

Merged
merged 4 commits into from
May 16, 2023

Conversation

rlouf
Copy link
Member

@rlouf rlouf commented May 15, 2023

We add two constraints to the transformers integration:

  • The is_in keyword argument allows to restrict the generations within a particular set. This is achieved by dynamically computing a mask and stopping when one of the sequences has been generated;
  • The stop_at keyword allows to stop the generation when a given sequence has been generated. This sequence is not returned.

Closes #107

TODO

  • Generation can be stopped sooner than it currently is for is_in: we don't need to generate the entire sequence, but rather exclude all the other sequences
  • The whole "constrained generation" part of the file could do with some refactoring. Code could be better structured and documented.

@rlouf rlouf force-pushed the is-in-hf branch 2 times, most recently from 9dca663 to a6429ec Compare May 15, 2023 14:28
@rlouf rlouf merged commit 9cc575f into main May 16, 2023
@rlouf rlouf deleted the is-in-hf branch May 16, 2023 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow stop_at and is_in with the transformers integration
1 participant