Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds configurable stop tokens #293

Merged
merged 3 commits into from
Feb 15, 2024

Conversation

zmackie
Copy link
Contributor

@zmackie zmackie commented Jan 31, 2024

This is an initial start of #224
I wanted to open this PR to make sure the maintainers were okay with the direction.
stop is sort of an odd parameter that doesn't have an obvious place to live (from what I saw...very well could be wrong).

Thanks!

Signed-off-by: Zander Mackie <zmackie@gmail.com>
Signed-off-by: Zander Mackie <zmackie@gmail.com>
Copy link
Collaborator

@drazvan drazvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍 . This is the right direction.

nemoguardrails/llm/taskmanager.py Outdated Show resolved Hide resolved
@zmackie zmackie changed the title [WIP] Adds stop config Adds configurable stop tokens Feb 2, 2024
@zmackie zmackie requested a review from drazvan February 2, 2024 19:48
@zmackie
Copy link
Contributor Author

zmackie commented Feb 5, 2024

@drazvan I believe this should do it. I wasn't totally sure about the use of stops in generation tasks, but I guessed this wasn't useful. LMK what you think. Happy to update documentation, etc. if that's important.

@drazvan
Copy link
Collaborator

drazvan commented Feb 15, 2024

Let's merge this. I'll add the documentation in a separate commit.

@drazvan drazvan merged commit 32f902c into NVIDIA:develop Feb 15, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants