可以使用onnx模型转换为tensorrt模型，再进行推理吗

[poppinjie]

Env

• GPU：RTX2070
• OS：Win10
• Cuda version：11.8
• TensorRT version：8.6

About this repo

• which branch/tag/commit are you using?
• which model? yolov5, retinaface?

Your problem

我想问下博主大大，是否可以先将原始模型（pt）转为onnx模型，然后再转为engine模型，加载进去进行推理？我尝试这样去做了，但是报错了，具体错误信息如下：

./tensorrt_yolov5.exe -d D:\JCDZ\project\DLInference\tensorrt_yolov5\models\yolov5s.engine D:\JCDZ\project\DLInference\tensorrt_yolov5\images
[05/23/2024-20:27:21] [W] [TRT] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
[05/23/2024-20:27:21] [E] [TRT] 3: Cannot find binding of given name: data
[05/23/2024-20:27:21] [E] [TRT] 3: Cannot find binding of given name: prob
[05/23/2024-20:27:21] [W] [TRT] The enqueue() method has been deprecated when used with engines built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. Please use enqueueV2() instead.
[05/23/2024-20:27:21] [W] [TRT] Also, the batchSize argument passed into this function has no effect on changing the input shapes. Please use setBindingDimensions() function to change input shapes instead.
CUDA error 700 at d:\jcdz\project\dlinference\tensorrt_yolov5\src\yolov5_det.cpp:72inference time: 203ms
CUDA error 700 at D:\JCDZ\project\DLInference\tensorrt_yolov5\src\preprocess.cu:105CUDA error 700 at D:\JCDZ\project\DLInference\tensorrt_yolov5\src\preprocess.cu:139[05/23/2024-20:27:21] [W] [TRT] The enqueue() method has been deprecated when used with engines built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. Please use enqueueV2() instead.
[05/23/2024-20:27:21] [W] [TRT] Also, the batchSize argument passed into this function has no effect on changing the input shapes. Please use setBindingDimensions() function to change input shapes instead.
[05/23/2024-20:27:21] [E] [TRT] 1: [convBaseRunner.cpp::nvinfer1::rt::task::CaskConvBaseRunner::execute::295] Error Code 1: Cask (Cask convolution execution)
[05/23/2024-20:27:21] [E] [TRT] 1: [checkMacros.cpp::nvinfer1::catchCudaError::203] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
CUDA error 700 at d:\jcdz\project\dlinference\tensorrt_yolov5\src\yolov5_det.cpp:72inference time: 1ms
CUDA error 700 at d:\jcdz\project\dlinference\tensorrt_yolov5\src\yolov5_det.cpp:213CUDA error 700 at d:\jcdz\project\dlinference\tensorrt_yolov5\src\yolov5_det.cpp:214CUDA error 700 at D:\JCDZ\project\DLInference\tensorrt_yolov5\src\preprocess.cu:151CUDA error 700 at D:\JCDZ\project\DLInference\tensorrt_yolov5\src\preprocess.cu:152[05/23/2024-20:27:21] [E] [TRT] 1: [defaultAllocator.cpp::nvinfer1::internal::DefaultAllocator::deallocate::61] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [defaultAllocator.cpp::nvinfer1::internal::DefaultAllocator::deallocate::61] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaStream::~ScopedCudaStream::47] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda
......
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda Runtime (an illegal memory access was encountered)
[05/23/2024-20:27:21] [E] [TRT] 1: [cudaResources.cpp::nvinfer1::ScopedCudaEvent::~ScopedCudaEvent::24] Error Code 1: Cuda
......

[wang-xinyu]

这个repo不支持onnx，可以直接用yolov5官方代码

[poppinjie]

好的 发自我的iPhone

…

------------------ 原始邮件 ------------------ 发件人: Wang Xinyu ***@***.***> 发送时间: 2024年5月30日 11:12 收件人: wang-xinyu/tensorrtx ***@***.***> 抄送: poppinjie ***@***.***>, Author ***@***.***> 主题: Re: [wang-xinyu/tensorrtx] 可以使用onnx模型转换为tensorrt模型，再进行推理吗 (Issue #1531) 这个repo不支持onnx，可以直接用yolov5官方代码 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>