Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revert FP16 test.py and detect.py inference to FP32 default #3423

Merged
merged 8 commits into from
Jun 4, 2021
Merged

Revert FP16 test.py and detect.py inference to FP32 default #3423

merged 8 commits into from
Jun 4, 2021

Conversation

PresageBoat
Copy link
Contributor

@PresageBoat PresageBoat commented Jun 2, 2021 •
edited by UltralyticsAssistant
Loading

We discussed the problem of half-precision inference failure in #3280
I think we can use this method to fix the half-precision inference failure when we run detect.py and test.py on some GPUs.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Implementation of a user-controllable flag to enable FP16 half-precision inference in detection and testing.

📊 Key Changes

  • Added an option --half to enable FP16 half-precision inference both in detect.py and test.py.
  • Modified the half variable initialization to respect the new --half option, ensuring it's active only when requested by the user.

🎯 Purpose & Impact

  • Purpose: This change provides users with the option to perform inference in half-precision mode, which can offer a significant speedup on compatible hardware (mostly GPUs) without substantially sacrificing accuracy.
  • Impact: This enhancement empowers users, especially those with compatible CUDA-enabled GPUs, to opt into performance optimizations that were previously not available. This can be particularly useful for real-time applications or situations where compute resources are limited. 🚀

@glenn-jocher glenn-jocher mentioned this pull request Jun 4, 2021
Closed
@glenn-jocher glenn-jocher changed the title fixed inference bug ,while use half precision Revert FP16 detect.py inference to FP32 default Jun 4, 2021
@glenn-jocher
Copy link
Member

@edificewang I cleaned this up a bit, and renamed --use-half to --half.

It looks like your test.py modifications have no effect. Can you remove them or fix them please? Thanks.

@glenn-jocher
Copy link
Member

/rebase

@glenn-jocher glenn-jocher linked an issue Jun 4, 2021 that may be closed by this pull request
half inference bug while using detect.py #3280
Closed
@glenn-jocher
Copy link
Member

@edificewang ok I've tested the PR and everything seems to work correctly. Unfortunately FP32 speeds with test.py are about 2x slower for me on a Colab V100

This PR

Speed: 11.2/1.5/12.7 ms inference/NMS/total per 640x640 image at batch-size 32

Default code

Speed: 5.2/1.5/6.8 ms inference/NMS/total per 640x640 image at batch-size 32

@glenn-jocher
Copy link
Member

@edificewang I guess we'll try this way out for a bit. Merging PR now.

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

@glenn-jocher glenn-jocher changed the title Revert FP16 detect.py inference to FP32 default Revert FP16 test.py and detect.py inference to FP32 default Jun 4, 2021
@glenn-jocher glenn-jocher merged commit b31229a into ultralytics:develop Jun 4, 2021
@glenn-jocher glenn-jocher mentioned this pull request Jun 4, 2021
Closed
glenn-jocher added a commit that referenced this pull request Jun 8, 2021
@glenn-jocher @SkalskiP
@pizzaz93 @tudoulei @chocosaj @developer0hye @SamSamhuns @PresageBoat
Merge develop branch into master (#3518)
f3c3d2c 
* update ci-testing.yml (#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* update ci-testing.yml (#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* Enable direct `--weights URL` definition (#3373)

* Enable direct `--weights URL` definition

@kalenmike this PR will enable direct --weights URL definition. Example use case:
```
python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt
```

* cleanup

* bug fixes

* weights = attempt_download(weights)

* Update experimental.py

* Update hubconf.py

* return bug fix

* comment mirror

* min_bytes

* Update tutorial.ipynb (#3368)

add Open in Kaggle badge

* `cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)

* Update datasets.py

* comment

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* COCO evolution fix (#3388)

* COCO evolution fix

* cleanup

* update print

* print fix

* Create `is_pip()` function (#3391)

Returns `True` if file is part of pip package. Useful for contextual behavior modification.

```python
def is_pip():
    # Is file in a pip package?
    return 'site-packages' in Path(__file__).absolute().parts
```

* Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (#3379)" (#3395)

This reverts commit 21a9607.

* Update FLOPs description (#3422)

* Update README.md

* Changing FLOPS to FLOPs.

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>

* Parse URL authentication (#3424)

* Parse URL authentication

* urllib.parse.unquote()

* improved error handling

* improved error handling

* remove %3F

* update check_file()

* Add FLOPs title to table (#3453)

* Suppress jit trace warning + graph once (#3454)

* Suppress jit trace warning + graph once

Suppress harmless jit trace warning on TensorBoard add_graph call. Also fix multiple add_graph() calls bug, now only on batch 0.

* Update train.py

* Update MixUp augmentation `alpha=beta=32.0` (#3455)

Per VOC empirical results #3380 (comment) by @developer0hye

* Add `timeout()` class (#3460)

* Add `timeout()` class

* rearrange order

* Faster HSV augmentation (#3462)

remove datatype conversion process that can be skipped

* Add `check_git_status()` 5 second timeout (#3464)

* Add check_git_status() 5 second timeout

This should prevent the SSH Git bug that we were discussing @kalenmike

* cleanup

* replace timeout with check_output built-in timeout

* Improved `check_requirements()` offline-handling (#3466)

Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline).

* Add `output_names` argument for ONNX export with dynamic axes (#3456)

* Add output names & dynamic axes for onnx export

Add output_names and dynamic_axes names for all outputs in torch.onnx.export. The first four outputs of the model will have names output0, output1, output2, output3

* use first output only + cleanup

Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Revert FP16 `test.py` and `detect.py` inference to FP32 default (#3423)

* fixed inference bug ,while use half precision

* replace --use-half with --half

* replace space and PEP8 in detect.py

* PEP8 detect.py

* update --half help comment

* Update test.py

* revert space

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Add additional links/resources to stale.yml message (#3467)

* Update stale.yml

* cleanup

* Update stale.yml

* reformat

* Update stale.yml HUB URL (#3468)

* Stale `github.actor` bug fix (#3483)

* Explicit `model.eval()` call `if opt.train=False` (#3475)

* call model.eval() when opt.train is False

call model.eval() when opt.train is False

* single-line if statement

* cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* check_requirements() exclude `opencv-python` (#3495)

Fix for 3rd party or contrib versions of installed OpenCV as in #3494.

* Earlier `assert` for cpu and half option (#3508)

* early assert for cpu and half option

early assert for cpu and half option

* Modified comment

Modified comment

* Update tutorial.ipynb (#3510)

* Reduce test.py results spacing (#3511)

* Update README.md (#3512)

* Update README.md

Minor modifications

* 850 width

* Update greetings.yml

revert greeting change as PRs will now merge to master.

Co-authored-by: Piotr Skalski <SkalskiP@users.noreply.github.com>
Co-authored-by: SkalskiP <piotr.skalski92@gmail.com>
Co-authored-by: Peretz Cohen <pizzaz93@users.noreply.github.com>
Co-authored-by: tudoulei <34886368+tudoulei@users.noreply.github.com>
Co-authored-by: chocosaj <chocosaj@users.noreply.github.com>
Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com>
Co-authored-by: Sam_S <SamSamhuns@users.noreply.github.com>
Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: edificewang <609552430@qq.com>
Lechtr pushed a commit to Lechtr/yolov5 that referenced this pull request Jul 20, 2021
@glenn-jocher @Lechtr
Merge develop branch into master (ultralytics#3518)
9969677 
* update ci-testing.yml (ultralytics#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* update ci-testing.yml (ultralytics#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* Enable direct `--weights URL` definition (ultralytics#3373)

* Enable direct `--weights URL` definition

@kalenmike this PR will enable direct --weights URL definition. Example use case:
```
python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt
```

* cleanup

* bug fixes

* weights = attempt_download(weights)

* Update experimental.py

* Update hubconf.py

* return bug fix

* comment mirror

* min_bytes

* Update tutorial.ipynb (ultralytics#3368)

add Open in Kaggle badge

* `cv2.imread(img, -1)` for IMREAD_UNCHANGED (ultralytics#3379)

* Update datasets.py

* comment

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* COCO evolution fix (ultralytics#3388)

* COCO evolution fix

* cleanup

* update print

* print fix

* Create `is_pip()` function (ultralytics#3391)

Returns `True` if file is part of pip package. Useful for contextual behavior modification.

```python
def is_pip():
    # Is file in a pip package?
    return 'site-packages' in Path(__file__).absolute().parts
```

* Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (ultralytics#3379)" (ultralytics#3395)

This reverts commit 21a9607.

* Update FLOPs description (ultralytics#3422)

* Update README.md

* Changing FLOPS to FLOPs.

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>

* Parse URL authentication (ultralytics#3424)

* Parse URL authentication

* urllib.parse.unquote()

* improved error handling

* improved error handling

* remove %3F

* update check_file()

* Add FLOPs title to table (ultralytics#3453)

* Suppress jit trace warning + graph once (ultralytics#3454)

* Suppress jit trace warning + graph once

Suppress harmless jit trace warning on TensorBoard add_graph call. Also fix multiple add_graph() calls bug, now only on batch 0.

* Update train.py

* Update MixUp augmentation `alpha=beta=32.0` (ultralytics#3455)

Per VOC empirical results ultralytics#3380 (comment) by @developer0hye

* Add `timeout()` class (ultralytics#3460)

* Add `timeout()` class

* rearrange order

* Faster HSV augmentation (ultralytics#3462)

remove datatype conversion process that can be skipped

* Add `check_git_status()` 5 second timeout (ultralytics#3464)

* Add check_git_status() 5 second timeout

This should prevent the SSH Git bug that we were discussing @kalenmike

* cleanup

* replace timeout with check_output built-in timeout

* Improved `check_requirements()` offline-handling (ultralytics#3466)

Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline).

* Add `output_names` argument for ONNX export with dynamic axes (ultralytics#3456)

* Add output names & dynamic axes for onnx export

Add output_names and dynamic_axes names for all outputs in torch.onnx.export. The first four outputs of the model will have names output0, output1, output2, output3

* use first output only + cleanup

Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Revert FP16 `test.py` and `detect.py` inference to FP32 default (ultralytics#3423)

* fixed inference bug ,while use half precision

* replace --use-half with --half

* replace space and PEP8 in detect.py

* PEP8 detect.py

* update --half help comment

* Update test.py

* revert space

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Add additional links/resources to stale.yml message (ultralytics#3467)

* Update stale.yml

* cleanup

* Update stale.yml

* reformat

* Update stale.yml HUB URL (ultralytics#3468)

* Stale `github.actor` bug fix (ultralytics#3483)

* Explicit `model.eval()` call `if opt.train=False` (ultralytics#3475)

* call model.eval() when opt.train is False

call model.eval() when opt.train is False

* single-line if statement

* cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* check_requirements() exclude `opencv-python` (ultralytics#3495)

Fix for 3rd party or contrib versions of installed OpenCV as in ultralytics#3494.

* Earlier `assert` for cpu and half option (ultralytics#3508)

* early assert for cpu and half option

early assert for cpu and half option

* Modified comment

Modified comment

* Update tutorial.ipynb (ultralytics#3510)

* Reduce test.py results spacing (ultralytics#3511)

* Update README.md (ultralytics#3512)

* Update README.md

Minor modifications

* 850 width

* Update greetings.yml

revert greeting change as PRs will now merge to master.

Co-authored-by: Piotr Skalski <SkalskiP@users.noreply.github.com>
Co-authored-by: SkalskiP <piotr.skalski92@gmail.com>
Co-authored-by: Peretz Cohen <pizzaz93@users.noreply.github.com>
Co-authored-by: tudoulei <34886368+tudoulei@users.noreply.github.com>
Co-authored-by: chocosaj <chocosaj@users.noreply.github.com>
Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com>
Co-authored-by: Sam_S <SamSamhuns@users.noreply.github.com>
Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: edificewang <609552430@qq.com>

(cherry picked from commit f3c3d2c)
@glenn-jocher glenn-jocher mentioned this pull request Oct 12, 2021
Merged
@glenn-jocher glenn-jocher mentioned this pull request Nov 7, 2021
Closed
@glenn-jocher glenn-jocher mentioned this pull request Nov 14, 2021
Merged
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
@PresageBoat @glenn-jocher
Revert FP16 test.py and detect.py inference to FP32 default (ultr…
9a4ef3f 
…alytics#3423)

* fixed inference bug ,while use half precision

* replace --use-half with --half

* replace space and PEP8 in detect.py

* PEP8 detect.py

* update --half help comment

* Update test.py

* revert space

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
@glenn-jocher @SkalskiP
@pizzaz93 @tudoulei @chocosaj @developer0hye @SamSamhuns @PresageBoat
Merge develop branch into master (ultralytics#3518)
8be5168 
* update ci-testing.yml (ultralytics#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* update ci-testing.yml (ultralytics#3322)

* update ci-testing.yml

* update greetings.yml

* bring back os matrix

* Enable direct `--weights URL` definition (ultralytics#3373)

* Enable direct `--weights URL` definition

@kalenmike this PR will enable direct --weights URL definition. Example use case:
```
python train.py --weights https://storage.googleapis.com/bucket/dir/model.pt
```

* cleanup

* bug fixes

* weights = attempt_download(weights)

* Update experimental.py

* Update hubconf.py

* return bug fix

* comment mirror

* min_bytes

* Update tutorial.ipynb (ultralytics#3368)

add Open in Kaggle badge

* `cv2.imread(img, -1)` for IMREAD_UNCHANGED (ultralytics#3379)

* Update datasets.py

* comment

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* COCO evolution fix (ultralytics#3388)

* COCO evolution fix

* cleanup

* update print

* print fix

* Create `is_pip()` function (ultralytics#3391)

Returns `True` if file is part of pip package. Useful for contextual behavior modification.

```python
def is_pip():
    # Is file in a pip package?
    return 'site-packages' in Path(__file__).absolute().parts
```

* Revert "`cv2.imread(img, -1)` for IMREAD_UNCHANGED (ultralytics#3379)" (ultralytics#3395)

This reverts commit 67db4b6.

* Update FLOPs description (ultralytics#3422)

* Update README.md

* Changing FLOPS to FLOPs.

Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>

* Parse URL authentication (ultralytics#3424)

* Parse URL authentication

* urllib.parse.unquote()

* improved error handling

* improved error handling

* remove %3F

* update check_file()

* Add FLOPs title to table (ultralytics#3453)

* Suppress jit trace warning + graph once (ultralytics#3454)

* Suppress jit trace warning + graph once

Suppress harmless jit trace warning on TensorBoard add_graph call. Also fix multiple add_graph() calls bug, now only on batch 0.

* Update train.py

* Update MixUp augmentation `alpha=beta=32.0` (ultralytics#3455)

Per VOC empirical results ultralytics#3380 (comment) by @developer0hye

* Add `timeout()` class (ultralytics#3460)

* Add `timeout()` class

* rearrange order

* Faster HSV augmentation (ultralytics#3462)

remove datatype conversion process that can be skipped

* Add `check_git_status()` 5 second timeout (ultralytics#3464)

* Add check_git_status() 5 second timeout

This should prevent the SSH Git bug that we were discussing @kalenmike

* cleanup

* replace timeout with check_output built-in timeout

* Improved `check_requirements()` offline-handling (ultralytics#3466)

Improve robustness of `check_requirements()` function to offline environments (do not attempt pip installs when offline).

* Add `output_names` argument for ONNX export with dynamic axes (ultralytics#3456)

* Add output names & dynamic axes for onnx export

Add output_names and dynamic_axes names for all outputs in torch.onnx.export. The first four outputs of the model will have names output0, output1, output2, output3

* use first output only + cleanup

Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Revert FP16 `test.py` and `detect.py` inference to FP32 default (ultralytics#3423)

* fixed inference bug ,while use half precision

* replace --use-half with --half

* replace space and PEP8 in detect.py

* PEP8 detect.py

* update --half help comment

* Update test.py

* revert space

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* Add additional links/resources to stale.yml message (ultralytics#3467)

* Update stale.yml

* cleanup

* Update stale.yml

* reformat

* Update stale.yml HUB URL (ultralytics#3468)

* Stale `github.actor` bug fix (ultralytics#3483)

* Explicit `model.eval()` call `if opt.train=False` (ultralytics#3475)

* call model.eval() when opt.train is False

call model.eval() when opt.train is False

* single-line if statement

* cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>

* check_requirements() exclude `opencv-python` (ultralytics#3495)

Fix for 3rd party or contrib versions of installed OpenCV as in ultralytics#3494.

* Earlier `assert` for cpu and half option (ultralytics#3508)

* early assert for cpu and half option

early assert for cpu and half option

* Modified comment

Modified comment

* Update tutorial.ipynb (ultralytics#3510)

* Reduce test.py results spacing (ultralytics#3511)

* Update README.md (ultralytics#3512)

* Update README.md

Minor modifications

* 850 width

* Update greetings.yml

revert greeting change as PRs will now merge to master.

Co-authored-by: Piotr Skalski <SkalskiP@users.noreply.github.com>
Co-authored-by: SkalskiP <piotr.skalski92@gmail.com>
Co-authored-by: Peretz Cohen <pizzaz93@users.noreply.github.com>
Co-authored-by: tudoulei <34886368+tudoulei@users.noreply.github.com>
Co-authored-by: chocosaj <chocosaj@users.noreply.github.com>
Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com>
Co-authored-by: Sam_S <SamSamhuns@users.noreply.github.com>
Co-authored-by: Samridha Shrestha <samridha.shrestha@g42.ai>
Co-authored-by: edificewang <609552430@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@PresageBoat PresageBoat

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

half inference bug while using detect.py
2 participants
@PresageBoat @glenn-jocher