-
Notifications
You must be signed in to change notification settings - Fork 540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CPU and M1/M2 GPU platform support #71
Conversation
I tried.
Seems something wrong.
|
@HIRANO-Satoshi Did you just run the code on your Mac machine? If so, you should add |
This comment was marked as outdated.
This comment was marked as outdated.
I don't have NVIDIA. With --platform cpu, it start working. Thanks much! Maybe apps/completion.py needs the --platform option.
ppa-hirano:FlexGen hirano-s$ python3 -m flexgen.apps.completion --model facebook/opt-1.3b --platform cpu
|
A proper default without an explicit option would be better. I'm curious how Apple Neural Engine is fast. |
Minimal modification to extend FlexGen to CPU and M1/M2 GPU platforms.
Not fully tested with various offloading settings.
@Ying1123 @merrymercy