Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix output error occurring due to check if it is a dict or not #1742

Merged
merged 3 commits into from
Dec 29, 2022

Conversation

rnyak
Copy link
Contributor

@rnyak rnyak commented Dec 29, 2022

The Transformers4Rrec end-to-end example gives an error Failed to process the request(s) for model 'mymodel_pt', message: ValueError: output of the forward function should be a dict when we send request to TIS. the reason for that is due to recent changes in the prediction tasks' outputs in TF4Rec repo (see NVIDIA-Merlin/Transformers4Rec#546). This PR provides a quick fix to avoid such error. However, for long term solution Oliver's draft PR (see NVIDIA-Merlin/systems#252) might be preferable.

@rnyak rnyak added this to the 23.01 milestone Dec 29, 2022
@rnyak rnyak added the P0 label Dec 29, 2022
Copy link
Contributor

@karlhigley karlhigley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine but could be cleaned up a bit. Left suggestions that can be merged directly from the comments.

nvtabular/inference/triton/model/model_pt.py Outdated Show resolved Hide resolved
nvtabular/inference/triton/model/model_pt.py Outdated Show resolved Hide resolved
nvtabular/inference/triton/model/model_pt.py Outdated Show resolved Hide resolved
@karlhigley
Copy link
Contributor

karlhigley commented Dec 29, 2022

The code modified by NVIDIA-Merlin/systems#252 was never intended to be used directly—it was added to Systems as a starting point for refactoring toward the operator API, and some parts of the old code still exist there. If something is using the legacy serving code (e.g. export_pytorch_ensemble) from Systems instead of from NVTabular, it should be reverted to using the NVT version, which is mostly frozen in order avoid regressions where we have insufficient test coverage.

Making minor fixes to the NVT code in order to keep it working with the rest of the Merlin ecosystem is fine if needed, but we've been avoiding making any major changes there. The corresponding code in Systems is open to major changes and therefore unstable; it's also highly likely to be removed in a future Systems version.

@github-actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1742

@karlhigley karlhigley added the bug Something isn't working label Dec 29, 2022
@karlhigley
Copy link
Contributor

See NVIDIA-Merlin/systems#265 for links to the open PRs related to legacy (non-Torchscript Python) serving for PyTorch. Possible that the fixes in NVIDIA-Merlin/systems#252 should be applied to NVTabular instead?

@sararb
Copy link
Contributor

sararb commented Dec 29, 2022

See NVIDIA-Merlin/systems#265 for links to the open PRs related to legacy (non-Torchscript Python) serving for PyTorch. Possible that the fixes in NVIDIA-Merlin/systems#252 should be applied to NVTabular instead?

Thank you for the context @karlhigley, with Ronay we weren't sure if the legacy function will be included in systems or not. So It makes sense to me to include the changes proposed by Oliver in NVIDIA-Merlin/systems#265 in NVTabular instead (as a separate PR from this quick fix).

Co-authored-by: Karl Higley <kmhigley@gmail.com>
@jperez999 jperez999 merged commit c5bc409 into main Dec 29, 2022
sararb added a commit that referenced this pull request Dec 29, 2022
* fix pred output type

* remove commented out lines

* Apply suggestions from code review

Co-authored-by: Karl Higley <kmhigley@gmail.com>

Co-authored-by: Sara Rabhi <sara.rabhi@gmail.com>
Co-authored-by: Karl Higley <kmhigley@gmail.com>
@rnyak rnyak deleted the fix_pyt_inf_model branch January 3, 2023 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Inference P0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants