Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More HF Llama tokenizer space fixes #282

Merged

Conversation

brandonwillard
Copy link
Contributor

Closes #273 (again)

@brandonwillard brandonwillard self-assigned this Sep 16, 2023
@brandonwillard brandonwillard added bug transformers Linked to the `transformers` integration tokenization llama labels Sep 16, 2023
@brandonwillard brandonwillard merged commit 9229a5d into outlines-dev:main Sep 16, 2023
4 checks passed
@brandonwillard brandonwillard deleted the more-hf-llama-space-fixes branch September 16, 2023 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug tokenization transformers Linked to the `transformers` integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Llama Models decoding produces white spaces between characters
1 participant