New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MPT-7B model conversion? #596

Open

SinanAkkoyun opened this issue May 9, 2023 · 2 comments

SinanAkkoyun commented May 9, 2023 •

edited

Loading

Hello!
I'd like to know how to convert the standard MPT-7b model weights to the right format to run inference with?

SinanAkkoyun changed the title ~~MPT-7B model conversion? And H100~~ MPT-7B model conversion?

ankit-db commented May 27, 2023

mosaicml/llm-foundry#169

Collaborator

byshiue commented Oct 20, 2023

FasterTransformer development has transitioned to TensorRT-LLM.

MPT is supported in TensorRT-LLM. Please take a try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment