Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type constraint to generation #91

Merged
merged 5 commits into from
May 12, 2023
Merged

Add type constraint to generation #91

merged 5 commits into from
May 12, 2023

Conversation

rlouf
Copy link
Member

@rlouf rlouf commented May 3, 2023

This PR allows to limit the generation to ints or floats. Closes #90.

The implementation with the OpenAI API is very limited since OpenAI doesn't allow to send more than 300 logit_bias values when ~900 tokens in their vocabulary correspond to ints. API:

import outlines.models as models

complete = models.text_completion.hf("sshleifer/tiny-gpt2", max_tokens=128)
question = "When I was 6 my sister was half my age. Now I’m 88 how old is my sister?"
answers = complete(question, type="int")

Also closes #96

@rlouf rlouf marked this pull request as draft May 3, 2023 10:03
@rlouf rlouf force-pushed the main branch 2 times, most recently from 7bd0f4d to a2e0b68 Compare May 4, 2023 12:39
@rlouf rlouf added text Linked to text generation enhancement labels May 6, 2023
@rlouf rlouf force-pushed the type-constraint branch 5 times, most recently from 593be16 to 51c2e94 Compare May 12, 2023 09:04
@rlouf rlouf marked this pull request as ready for review May 12, 2023 11:25
@rlouf rlouf merged commit 9d166d5 into main May 12, 2023
@rlouf rlouf deleted the type-constraint branch May 12, 2023 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement text Linked to text generation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

is_in is not working with samples != 1 Add controlled type generation
1 participant