Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OBCQ Bug Fix] Fallback to CPU if no CUDA #1828

Merged
merged 4 commits into from
Nov 14, 2023
Merged

[OBCQ Bug Fix] Fallback to CPU if no CUDA #1828

merged 4 commits into from
Nov 14, 2023

Conversation

Satrat
Copy link
Contributor

@Satrat Satrat commented Nov 14, 2023

tests.sparseml.transformers.obcq.test_obcq was failing on Jenkins due to CUDA not being available, this PR updates the OBCQ script to fall back to CPU if CUDA is unavailable. The model used in this test is very small, so it still runs quickly enough on CPU.

Also fixes a small bug in the perplexity calculations in the OBCQ script caused by the recent target_ids change

Testing

export CUDA_VISIBLE_DEVICES=[]
pytest tests/sparseml/transformers/obcq/test_obcq.py

Output:

WARNING    sparseml.transformers.sparsification.obcq.obcq:obcq.py:156 Requested cuda:0 but CUDA is not available, falling back to CPU
.....
tests/sparseml/transformers/obcq/test_obcq.py::test_obcq_tinystories[tests/sparseml/transformers/obcq/test_tiny.yaml] PASSED                                                                     [ 50%]
tests/sparseml/transformers/obcq/test_obcq.py::test_obcq_tinystories[tests/sparseml/transformers/obcq/test_tiny2.yaml] PASSED                                                                    [100%]

rahul-tuli
rahul-tuli previously approved these changes Nov 14, 2023
@Satrat Satrat merged commit d4a9ce9 into main Nov 14, 2023
11 checks passed
@Satrat Satrat deleted the obcq_fallback_cpu branch November 14, 2023 18:23
bfineran pushed a commit that referenced this pull request Nov 16, 2023
* fallback to cpu if no cuda:

* warning instead of info

* add log for device
bfineran pushed a commit that referenced this pull request Nov 16, 2023
* fallback to cpu if no cuda:

* warning instead of info

* add log for device
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants