Add MPT onnx and ORT support

[jiqing-feng]

Hi @fxmarty @echarlaix

Relate to 1101. This PR enables MPT models onnx config to support generating dummy inputs.

The shape arrangement of past_key_values may be different across models. We can use the member variable sequence_length of DummyPastKeyValuesGenerator to identify the sequence length of past_key_values.

Would you please help me review it? Thanks

cc @changwangss

[fxmarty]

Hi @jiqing-feng , thank you for the PR! I'm personally not in favor of supporting custom models in Optimum that otherwise use trust_remote_code=True. I think it would make more sense to host custom ONNX configs in the model repos directly, and allow the main_export function to use a custom ONNX config.

What is your opinion @echarlaix @JingyaHuang @michaelbenayoun ?

I propose #1166 #1143 in this regard, see the API here: https://moon-ci-docs.huggingface.co/docs/optimum/pr_1166/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models

[jiqing-feng]

Hi @fxmarty . Thanks for your comment.

The MPT model is going to upstream on transformers, see 24629. We do not need trust_remote_code=True after the PR is merged.

We can wait for the 24629 merge and then we can discuss if we can move forward with my PR.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[JingyaHuang]

@fxmarty @jiqing-feng For the ease of maintenance, Optimum shall only support configs of transformers native models. For those withtrust_remote_code(chatglm and mpt for now), hosting directly on the hub sounds very nice to me.

btw, mpt sounds like a good example to test a bit with what you suggest @fxmarty

[fxmarty]

@jiqing-feng Yes, once mpt is merged & released in transformers for sure we can merge this PR in Optimum! My suggestion is more for models with custom modeling code (as mpt is currently).

[jiqing-feng]

Hi @fxmarty . Since MPT has been merged to HF models, see 24629.

Could we merge this PR for supporting MPT onnx config?

cc @JingyaHuang

[fxmarty]

Hi @jiqing-feng, for sure we can merge once there is a transformers release! Is the KV cache layout fine?

[jiqing-feng]

Hi @jiqing-feng, for sure we can merge once there is a transformers release! Is the KV cache layout fine?

Yes! The KV cache layout is the same as most CLM models like llama. We can wait for the release.

[jiqing-feng]

Hi @fxmarty @JingyaHuang . I think we can move forward on this PR since the recently released version of transformers contained MPT model. I see that some checks have failed. Would you please tell me where and what kind of tests should I add? Thx!

[echarlaix]

Would you please tell me where and what kind of tests should I add? Thx!

Could you add ONNX Runtime tests as well by adding the model name here and here using a tiny random model like this one?

[jiqing-feng]

Would you please tell me where and what kind of tests should I add? Thx!

Could you add ONNX Runtime tests as well by adding the model name here and here using a tiny random model like this one?

Done. Thx!

[jiqing-feng]

Hi @echarlaix @fxmarty @JingyaHuang

Could we merge this PR? I see that the failed checks are not related to my changes and the mpt model has been released in the latest released version of transformers. Thx!

[fxmarty]

@jiqing-feng Thank you for the addition and apologize for the delay!