-
-
Notifications
You must be signed in to change notification settings - Fork 15.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The accuracy of the .pt model will decrease after being converted to .engine model. #12996
Comments
@arkerman hello! It's quite common to observe slight discrepancies in model performance when converting from a If the performance difference is significant and these adjustments don't help, consider reviewing the conversion logs for any warnings or error messages that could indicate what might be going wrong during the process. Happy coding! 😊 |
A warning will be reported during the conversion process: |
Hi @arkerman! The warning you're seeing indicates a mismatch in data types during the conversion process from ONNX to TensorRT, where TensorRT does not support INT64 weights. To help ensure better precision and compatibility, you could manually cast the weights from INT64 to FP32 before converting to TensorRT. This is generally more aligned with TensorRT's capabilities than INT32 and helps minimize potential loss of information. Here's a quick code snippet to adjust the data types in the ONNX model: import onnx
from onnx import numpy_helper
# Load your ONNX model
model = onnx.load('model.onnx')
# Iterate through each initializer (weight)
for initializer in model.graph.initializer:
data = numpy_helper.to_array(initializer)
if data.dtype == np.int64:
# Cast INT64 to FP32
data = data.astype(np.float32)
# Replace the initializer with the new data
new_initializer = numpy_helper.from_array(data, initializer.name)
model.graph.initializer.remove(initializer)
model.graph.initializer.append(new_initializer)
# Save the modified model
onnx.save(model, 'modified_model.onnx') This snippet converts INT64 weights to FP32, which might help with your conversion process! 😊 Happy coding! |
@glenn-jocher Thanks for your help! |
Hi @arkerman! It looks like the model is expecting a different data type for certain operations. Instead of converting all weights to FP32 indiscriminately, you might need to specifically target only those tensors that are compatible with such a conversion. Here's a refined approach to ensure you only adjust the necessary tensors: import onnx
from onnx import numpy_helper
# Load your ONNX model
model = onnx.load('model.onnx')
# Iterate through each initializer (weight)
for initializer in model.graph.initializer:
data = numpy_helper.to_array(initializer)
if data.dtype == np.int64 and data.ndim == 0: # Target scalar int64 weights
data = data.astype(np.float32)
new_initializer = numpy_helper.from_array(data, initializer.name)
model.graph.initializer.remove(initializer)
model.graph.initializer.append(new_initializer)
# Save the modified model
onnx.save(model, 'modified_model.onnx') This code now checks if the tensor is a scalar (0-dimensional) before converting, which might help avoid the type error you encountered. Give it a try and let us know how it goes! 😊 |
👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help. For additional resources and information, please see the links below:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed! Thank you for your contributions to YOLO 🚀 and Vision AI ⭐ |
Search before asking
YOLOv5 Component
Detection
Bug
The results obtained by my inference using the .pt model and the .engine model are different.
Environment
No response
Minimal Reproducible Example
No response
Additional
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: