Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow fp16 input tensor to apply with fp16 saved model #36

Closed
deepkyu opened this issue Sep 1, 2023 · 1 comment
Closed

Allow fp16 input tensor to apply with fp16 saved model #36

deepkyu opened this issue Sep 1, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@deepkyu
Copy link

deepkyu commented Sep 1, 2023

Problem

From @illian01

Compressor

  • Current API gets input tensor as input_shape
  • Some models can be saved with half() (i.e. fp16), which cannot infer with float32 input tensor .
    • This model should give the example input tensor whose dtype is same with the model (float16)
  • Need additional feature for users to select the data type of example input tensor
  • Our back-end has already supported different input types of tensors, and it doesn't affect to compress the model

Related Link

Slack: https://nota-workspace.slack.com/archives/C040F65LSAJ/p1693546735327899

@deepkyu deepkyu added the enhancement New feature or request label Sep 1, 2023
@deepkyu deepkyu assigned deepkyu and Only-bottle and unassigned deepkyu Sep 1, 2023
@Only-bottle
Copy link
Member

@deepkyu @illian01님, 해당 이슈는 Compressor Backend에서 처리하였습니다.

해결한 방식은 아래와 같습니다.

  • half의 경우 cpu에서는 사용할 수 없다는 것을 확인했고(관련 issue), float으로 변환해서 압축 후 half로 변환해서 제공하도록 처리하였습니다.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants