New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance ExllamaV2Sampler with Temperature Parameter and Update EOS Token for Llama3 Compatibility #2

Merged

isamu-isozaki merged 1 commit into isamu-isozaki:dynamic_outlines from ManilShrestha:feature/pass_temperature_and_enhance_stop_condition

Jul 4, 2024

ManilShrestha commented Jul 4, 2024

This pull request includes the following updates:

Passing Temperature Parameter:

Updated the code to allow passing the temperature parameter through ExllamaV2Sampler.Settings(). This enhancement provides better control over the sampling process by enabling dynamic adjustment of the temperature setting.

EOS Token Compatibility:

Modified the EOS_token handling to ensure compatibility with the Llama3 model. The previous implementation did not correctly recognize the tokenizer.eos_token_id as the EOS token in Llama3.
Integrated the stop_at values from the outline's extra body into the preferred_eos variable. This ensures that the end-of-sequence tokens are properly managed, accommodating both the model-specific and outline-specific stop conditions.


          Pass temperature with ExLlamaV2Sampler.Settings() and also made the E…

c4b4c8f

…OS compatible with Llama3 and 'stop_at' string from outlines

isamu-isozaki merged commit f302126 into isamu-isozaki:dynamic_outlines

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet