New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed an issue where the Qwen model had too much error after quantization #243

Merged

casper-hansen merged 1 commit into casper-hansen:main from jundolc:main

Dec 8, 2023

Contributor

jundolc commented Dec 8, 2023

Fixed an issue where the Qwen model had too much error after quantization


          Update qwen.py

efd2203

Fixed an issue where the Qwen model had too much error after quantization

jundolc changed the title ~~Update qwen.py~~ Fixed an issue where the Qwen model had too much error after quantization

Owner

casper-hansen commented Dec 8, 2023

I tested perplexity before and after and found that this works!

FP16: 8.672
Main INT4 (duo_scaling=True): 10.224
Main INT4 (duo_scaling=False): 9.949
PR INT4 (duo_scaling=True): 8.846
PR INT4 (duo_scaling=False): 8.860

casper-hansen merged commit 5eb1d2f into casper-hansen:main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet