New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make sure that indices are available on all relevant devices #328

Merged

brandonwillard merged 1 commit into outlines-dev:main from brandonwillard:transfer-indices-to-devices

Oct 29, 2023

Contributor

brandonwillard commented Oct 29, 2023

This PR transfers all is-sequence-finished indices to the relevant devices during token generation.


          Make sure that indices are available on all relevant devices

d43ae0a

brandonwillard added the bug label

namin commented Oct 29, 2023

👍
I confirm that this PR fixes my issue with running a BitsAndBytes-configured model on a multi-GPU system.

brandonwillard merged commit c3421aa into outlines-dev:main

5 checks passed

brandonwillard deleted the transfer-indices-to-devices branch

October 29, 2023 23:30

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment