export Tensorrt model error

[liquored]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I want to export my model as tensort. But when I use export.py, I was been told as:

TensorRT: starting export with TensorRT 8.6.0...
[04/28/2023-16:38:59] [TRT] [W] Unable to determine GPU memory usage
[04/28/2023-16:38:59] [TRT] [W] Unable to determine GPU memory usage
[04/28/2023-16:38:59] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 3351, GPU 0 (MiB)
[04/28/2023-16:38:59] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
TensorRT: export failure ❌ 7.7s: pybind11::init(): factory function returned nullptr

Additional

I am sure that I got the driver of Nvidia, because I could finish my train, test and detect.
If you could help me to deal with it, I would appreciate you a lot!
Looking forward to your reply!
Thank you!
Yours YangBo Zhou!

[glenn-jocher]

@liquored hello YangBo Zhou,

It seems like TensorRT is having trouble initializing CUDA on your machine. This could be due to a variety of reasons, such as an incompatible version of CUDA or not having enough memory available.

Can you please try the following steps:

1. Verify that you have a compatible version of CUDA installed on your machine. TensorRT requires version 10.2, 11.1, or 11.3.
2. Check that there is enough memory available on your GPU. You can do this by running nvidia-smi.
3. Try running the export script with sudo privileges.
4. Make sure that you have the latest version of TensorRT installed.
5. Restart your machine and try again.

If these steps do not resolve the issue, please let me know and I'll be happy to help you further.

Best regards,
Glenn Jocher

[xiezhangxiang]

@liquored I encountered the same problem, can you tell me the solution? Looking forward to your reply!

[glenn-jocher]

@xiezhangxiang hi there,

I understand that you're facing the same issue with exporting your model using TensorRT. To resolve this problem, you can try the following steps:

1. Make sure you have a compatible version of CUDA installed on your machine (version 10.2, 11.1, or 11.3).
2. Verify that you have enough GPU memory available by running nvidia-smi.
3. Try running the export script with sudo privileges.
4. Ensure that you have the latest version of TensorRT installed.
5. Restart your machine and attempt the export again.

Please give these steps a try and let me know if you encounter any further difficulties.

Thank you,
Glenn Jocher

[Capitolhill]

Hi, I followed the instructions (5 steps) above. I downgraded CUDA 11.4 to 11.3 but I still can't get past the following error.

`Namespace(calib_batch_size=8, calib_cache='./calibration.cache', calib_input=None, calib_num_images=5000, conf_thres=0.4, end2end=False, engine='models/object-detector/y7_b1.trt', iou_thres=0.5, max_det=100, onnx='models/object-detector/y7_b1.onnx', precision='fp16', v8=False, verbose=False, workspace=1)

[09/11/2023-16:37:21] [TRT] [W] Unable to determine GPU memory usage
[09/11/2023-16:37:21] [TRT] [W] Unable to determine GPU memory usage
[09/11/2023-16:37:21] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 21, GPU 0 (MiB)
[09/11/2023-16:37:21] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
Traceback (most recent call last):
File "TensorRT-For-YOLO-Series/export.py", line 308, in
main(args)
File "TensorRT-For-YOLO-Series/export.py", line 266, in main
builder = EngineBuilder(args.verbose, args.workspace)
File "TensorRT-For-YOLO-Series/export.py", line 107, in init
self.builder = trt.Builder(self.trt_logger)
TypeError: pybind11::init(): factory function returned nullptr
`

Kindly advise.

[glenn-jocher]

@Capitolhill hi,

I'm sorry to hear that you're still encountering the error even after following the steps mentioned earlier.
The error message you shared indicates a CUDA initialization failure, which suggests that there may be an issue with your CUDA installation.

Here are a few additional troubleshooting steps you can try:

1. Ensure that you have completely uninstalled the previous version of CUDA and have reinstalled CUDA 11.3.
2. Double-check that your GPU driver is compatible with CUDA 11.3. You may need to update the GPU driver if necessary.
3. Verify the correct installation of CUDA and check if the necessary environment variables (PATH, CUDA_HOME) are correctly set.
4. Make sure that you have sufficient GPU memory available for TensorRT. You can verify this by running nvidia-smi.

If the issue persists, please provide more details about your system configuration (GPU model, OS version, etc.) so we can further assist you.

Regards,
Glenn Jocher

[liquored]

Hi, I followed the instructions (5 steps) above. I downgraded CUDA 11.4 to 11.3 but I still can't get past the following error.

`Namespace(calib_batch_size=8, calib_cache='./calibration.cache', calib_input=None, calib_num_images=5000, conf_thres=0.4, end2end=False, engine='models/object-detector/y7_b1.trt', iou_thres=0.5, max_det=100, onnx='models/object-detector/y7_b1.onnx', precision='fp16', v8=False, verbose=False, workspace=1)

[09/11/2023-16:37:21] [TRT] [W] Unable to determine GPU memory usage [09/11/2023-16:37:21] [TRT] [W] Unable to determine GPU memory usage [09/11/2023-16:37:21] [TRT] [I] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 21, GPU 0 (MiB) [09/11/2023-16:37:21] [TRT] [W] CUDA initialization failure with error: 35. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html Traceback (most recent call last): File "TensorRT-For-YOLO-Series/export.py", line 308, in main(args) File "TensorRT-For-YOLO-Series/export.py", line 266, in main builder = EngineBuilder(args.verbose, args.workspace) File "TensorRT-For-YOLO-Series/export.py", line 107, in init self.builder = trt.Builder(self.trt_logger) TypeError: pybind11::init(): factory function returned nullptr `

Kindly advise.

@Capitolhill hi,
I am sorry about lately reply. In that case, I try to use another computer, and get success, and my teammate try to reinstall the system and CUDA which finally success. So in my opinion, it is because your CUDA went wrong.
Wish it helpful!

[glenn-jocher]

@liquored hi,

I apologize for the delayed response. It appears that the issue you're facing is related to a problem with your CUDA installation. Based on the experiences of others, reinstalling the system and CUDA has resolved similar issues. It's worth checking if your CUDA installation is functioning correctly.

I hope this information is helpful to you. If you have any further questions or concerns, please don't hesitate to ask.

Regards,
Glenn Jocher

[Capitolhill]

My CUDA driver incompatibility was indeed the problem. Thanks @glenn-jocher and @liquored for your kind help!

[glenn-jocher]

@Capitolhill glad to hear that the issue has been resolved! You're welcome, and I'm glad I could assist you. If you have any more questions or need further assistance, feel free to ask. The YOLOv5 community and the Ultralytics team are always here to help. Have a great day!