Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Autoinstall TensorRT if missing #7537

Merged
merged 7 commits into from
Apr 22, 2022
Merged

Autoinstall TensorRT if missing #7537

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
59a3503
Autoinstall TensorRT if missing
glenn-jocher Apr 22, 2022
f0af29c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 22, 2022
95ebe90
Update export.py
glenn-jocher Apr 22, 2022
20f5efe
Update export.py
glenn-jocher Apr 22, 2022
df7da68
Update export.py
glenn-jocher Apr 22, 2022
c5777b9
Update export.py
glenn-jocher Apr 22, 2022
6a0173f
Merge branch 'master' into auto/tensorrt
glenn-jocher Apr 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions export.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,15 @@ def export_coreml(model, im, file, int8, half, prefix=colorstr('CoreML:')):
def export_engine(model, im, file, train, half, simplify, workspace=4, verbose=False, prefix=colorstr('TensorRT:')):
# YOLOv5 TensorRT export https://developer.nvidia.com/tensorrt
try:
import tensorrt as trt # pip install -U nvidia-tensorrt --index-url https://pypi.ngc.nvidia.com
assert im.device.type != 'cpu', 'export running on CPU but must be on GPU, i.e. `python export.py --device 0`'
try:
import tensorrt as trt
except Exception:
s = f"\n{prefix} tensorrt not found and is required by YOLOv5"
LOGGER.info(f"{s}, attempting auto-update...")
r = '-U nvidia-tensorrt --index-url https://pypi.ngc.nvidia.com'
Copy link
Contributor

@zhiqwang zhiqwang Apr 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @glenn-jocher , These wheels only works on Linux operating system and x86_64 CPU architecture with Python versions 3.6 to 3.9 and CUDA 11.x. See https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-pip for more details. Many users use TensorRT on Windows, perhaps some explanations should be added here to prevent user annoyance when using it, such as #7009 (comment) .

Copy link
Member Author

@glenn-jocher glenn-jocher Apr 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhiqwang good point! I can't programmatically check the CUDA version, but we can check OS before install. I'll make a PR. The Python version requirement is already met as YOLOv5 requires 3.7 (may also work with 3.6, but no longer supported).

Copy link
Member Author

@glenn-jocher glenn-jocher Apr 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhiqwang all done! PR #7549 is merged that implements OS checks prior to install. This should resolve most of the problem, and migration over time to CUDA 11 should resolve the rest of the issue. I guess Python 3.10 could be an edge case though.

LOGGER.info(subprocess.check_output(f"pip install {r}", shell=True).decode())
import tensorrt as trt

if trt.__version__[0] == '7': # TensorRT 7 handling https://github.com/ultralytics/yolov5/issues/6012
grid = model.model[-1].anchor_grid
Expand All @@ -230,7 +238,6 @@ def export_engine(model, im, file, train, half, simplify, workspace=4, verbose=F
onnx = file.with_suffix('.onnx')

LOGGER.info(f'\n{prefix} starting export with TensorRT {trt.__version__}...')
assert im.device.type != 'cpu', 'export running on CPU but must be on GPU, i.e. `python export.py --device 0`'
assert onnx.exists(), f'failed to export ONNX file: {onnx}'
f = file.with_suffix('.engine') # TensorRT engine file
logger = trt.Logger(trt.Logger.INFO)
Expand Down