YOLO Segment sigmoid() fix
#13939
Merged
YOLO Segment sigmoid() fix
#13939
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…chieve better segmentation edges. (#13912) Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #13939 +/- ##
=======================================
Coverage 70.25% 70.25%
=======================================
Files 124 124
Lines 15891 15891
=======================================
Hits 11164 11164
Misses 4727 4727
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
glenn-jocher
changed the title
ultralytics 8.2.42 Segment Sigmoid() fix
glenn-jocher
changed the title
ultralytics 8.2.42 Segment Sigmoid() fixultralytics 8.2.42 YOLO Segment sigmoid() fix
Main branch results: |
PR results |
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
glenn-jocher
changed the title
ultralytics 8.2.42 YOLO Segment sigmoid() fixsigmoid() fix
|
@Laughing-q this fix PR seems to make a lot of sense, and the edges are much smoother (see #13912), but segment mAP drops very slightly after implementing it. What do you think? |
|
@glenn-jocher I also tested yolov8s-seg and yolov8m-seg, both of them shown slightly mAP drop as well. # main
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.405
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.633
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.432
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.216
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.456
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.582
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.327
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.521
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.556
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.351
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.621
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.741
# segment-fix
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.404
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.634
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.431
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.213
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.456
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.581
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.326
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.520
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.555
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.347
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.620
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.740yolov8s-seg: # main
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.366
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.580
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.387
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.168
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.410
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.537
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.306
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.488
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.521
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.301
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.585
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.715
# segment-fix
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.365
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.580
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.385
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.168
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.409
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.537
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.306
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.487
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.520
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.300
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.583
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.714But the update does show better visualization, meanwhile eliminating |
|
@glenn-jocher Here are some visualized samples on main branch and this branch, and it looks like on the latter in some level it's fixing the "cut" issue we are having when BTW I think we are having too many |
|
@Laughing-q yes I think on a mathematical level this PR does make sense, it's taking all the values that range from -inf to +inf and placing the threshold at 0, so there will be a better dynamic range on the op than if we sigmoid from 0 to 1 and then do the threshold at 0.5. The mAP drops are in the Ultralytics mAP but the COCO mAP is mostly the same, and the visual results do look better, less discretization, so let's merge this. |
|
@zxDeepDiver PR merged! Thank you for your contributions. |
|
@Laughing-q yes the multiple mask processing functions are too many, I agree, I think there are at least two because of the order of processing the upsample, either before or after the mask processing and depending on if it's |
|
@glenn-jocher Maybe this fix can be made optional? My previously trained neural network, focused on working with ultra-small objects, shows a lower quality... |
|
pip3 install -U ultralytics==8.2.42 predict_t2_brain_512 predict_t2_ischemia pip3 install -U ultralytics==8.2.43 predict_t2_brain_512 predict_t2_ischemia All F1 values worse on the new version |
|
Hi @Alarmod, Thank you for sharing your detailed results. It appears that the recent update has impacted the performance metrics of your model, particularly the F1 scores. We understand how crucial these metrics are for your application, especially when dealing with ultra-small objects. To address this, we can consider making the recent fix optional. This would allow users to choose whether to apply the sigmoid operation based on their specific use case. In the meantime, you can revert to the previous version (8.2.42) to maintain your model's performance. If you could provide a minimum reproducible example of your code, it would help us investigate the issue further and ensure that any changes we make will not negatively impact your use case. You can find more details on how to create a reproducible example here. Additionally, please ensure that you are using the latest versions of all dependencies to rule out any compatibility issues. Thank you for your patience and understanding. We are committed to improving the performance and flexibility of our models and appreciate your feedback. |
Alarmod
added a commit
to Alarmod/MRI_MedicalAnalysis
that referenced
this pull request
Jul 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…chieve better segmentation edges. (#13912)
🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
This PR simplifies mask processing logic and enhances testing in the Ultralytics repository.
📊 Key Changes
pytest-covfor test coverage reporting.process_mask_*functions inultralytics/utils/ops.pyto remove unnecessary sigmoid operations and adjusted the mask threshold condition.🎯 Purpose & Impact
pytest-covhelps in measuring test coverage, leading to higher code quality and reliability.