Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge_decoders mixes up decoder_model and decoder_model_with_past #1913

Open
4 tasks
xenova opened this issue Jun 19, 2024 · 0 comments
Open
4 tasks

merge_decoders mixes up decoder_model and decoder_model_with_past #1913

xenova opened this issue Jun 19, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@xenova
Copy link
Contributor

xenova commented Jun 19, 2024

System Info

Optimum: main
Platform: Linux (Ubuntu)
Python version: 3.10.12

Who can help?

@JingyaHuang @michaelbenayoun @fxmarty

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

  1. Following this tutorial, export a model with these two components

  2. According to the docs and source code, the past model should be placed second:

def merge_decoders(
decoder: Union[ModelProto, Path, str],
decoder_with_past: Union[ModelProto, Path, str],
graph_name: str = "merged",
producer_name: str = "optimum-onnx",
save_path: Optional[Union[str, Path]] = None,
strict: bool = True,
) -> ModelProto:

So, run

from optimum.onnx.graph_transformations import merge_decoders
merge_decoders('./decoder_model.onnx', './decoder_with_past_model.onnx', strict=False)
# merge_decoders('./decoder_with_past_model.onnx', './decoder_model.onnx', strict=False) works

This results in this error:

ValueError: The second ModelProto should not have more outputs than the first.
  1. Swap the arguments and watch it work.

This also affects the other tutorial here, which breaks if you set post-process to true.

Expected behavior

The docs/source code should be updated to be consistent, and/or the example code in the tutorial should work with post-process=True

@xenova xenova added the bug Something isn't working label Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant