Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

torch.nn.functional.grid_sample for volumetric input/ Rank 5 input and grid #2284

Open
nighting0le01 opened this issue Jul 23, 2024 · 3 comments
Open

torch.nn.functional.grid_sample for volumetric input/ Rank 5 input and grid #2284

nighting0le01 opened this issue Jul 23, 2024 · 3 comments
Labels
Core ML Framework An issue related to the Core ML Framework feature request Functionality does not currently exist, would need to be created as a new feature (type)

Comments

@nighting0le01
Copy link

nighting0le01 commented Jul 23, 2024 •
edited
Loading

🌱 Describe your Feature Request

support torch.nn.function.grid_sample for 5d Volumetric input by supporting 5d input for mb.affine and mb.resample for rank >=4 :https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html

How can this feature be used?

Please provide some examples where this feature can be used.
For video generation models features on device

Describe alternatives you've considered

currently grid_sampler is supported for rank 4 tensors. it erros out in resample op mb.resample
input "x" to the "resample" op must be a rank 4 tensor. Got rank 5 tensor of shape (22, 4, c,h,w)

please support to get mb.affine and mb.resample for rank 5 inputs
@nighting0le01 nighting0le01 added the feature request Functionality does not currently exist, would need to be created as a new feature (type) label Jul 23, 2024
@nighting0le01
Copy link
Author

nighting0le01 commented Jul 23, 2024 •
edited
Loading

@junpeiz @TobyRoseman @aseemw this feature is needed for video gen application

@nighting0le01 nighting0le01 changed the title torch.nn.functional.grid_sample for volumetric input torch.nn.functional.grid_sample for volumetric input/ Rank 5 input and grid Jul 25, 2024
@jakesabathia2
Copy link
Collaborator

This sounds like a limitation in the Framework :)

@jakesabathia2 jakesabathia2 added the Core ML Framework An issue related to the Core ML Framework label Jul 26, 2024
@nighting0le01
Copy link
Author

nighting0le01 commented Jul 26, 2024 •
edited
Loading

hello @jakesabathia2 what are the steps to support such a custom layer? do we have a up to date example. So i can implement and land the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Core ML Framework An issue related to the Core ML Framework feature request Functionality does not currently exist, would need to be created as a new feature (type)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jakesabathia2 @nighting0le01