New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

batch inference support #372

Open

cswry opened this issue Jul 12, 2024 · 1 comment

Assignees

cswry commented Jul 12, 2024

Awesome work!

Currently, the provided inference code example seems to not support batch inference. Can you add support for it?

mm-assistant bot assigned myownskyW7

YerongLi commented Jul 31, 2024 •

edited

Loading

It supports batch inference, could you post your code?
Which model are you using?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment