Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bugfix sentpiece #2198

Merged
merged 1 commit into from
May 4, 2023
Merged

bugfix sentpiece #2198

merged 1 commit into from
May 4, 2023

Conversation

bmosaicml
Copy link
Contributor

What does this PR do?

Fixes bug with spacing on SentPiece tokenizer

What issue(s) does this change relate to?

Before submitting

  • Have you read the contributor guidelines?
  • Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
  • Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
  • Did you update any related docs and document your change?
  • Did you update any related tests and add any new tests related to your change? (see testing)
  • Did you run the tests locally to make sure they pass?
  • Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

@bmosaicml bmosaicml requested a review from dakinggg May 4, 2023 20:02
@bmosaicml bmosaicml marked this pull request as ready for review May 4, 2023 20:02
@bmosaicml bmosaicml requested a review from a team as a code owner May 4, 2023 20:02
@mvpatel2000 mvpatel2000 merged commit f5ebb59 into mosaicml:dev May 4, 2023
bmosaicml added a commit to bmosaicml/composer that referenced this pull request May 4, 2023
@bmosaicml
bugfix sentpiece (mosaicml#2198)
1df3b45
mvpatel2000 pushed a commit that referenced this pull request May 4, 2023
@bmosaicml @dakinggg
bugfix sentpiece (#2198) (#2200)
e589652 
Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
mvpatel2000 pushed a commit that referenced this pull request May 4, 2023
@bmosaicml @mvpatel2000
bugfix sentpiece (#2198)
be18b56
mvpatel2000 pushed a commit that referenced this pull request May 4, 2023
@bmosaicml @dakinggg @mvpatel2000
bugfix sentpiece (#2198) (#2200)
8da757e 
Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dakinggg dakinggg dakinggg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bmosaicml @dakinggg @mvpatel2000