Fix detect unit tests to test false negatives as well

[vinnamkim]

Summary

• Ticket no. 104511
• To test false positive, change detect() test from assertIn() to assertEqual().
• In addition, add false positive test code as well to some datasets which must have multiple candidates from
  detect() in nature (e.g. kitti).
• Refactor Importer.detect() confidence return logic.
• Some addition fixes for dataset format plugins during this work.

How to test

This changes includes tests as well.

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added the description of my changes into CHANGELOG.​
• I have updated the documentation accordingly

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

[codecov-commenter]

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.02 🎉

Comparison is base (fc7f607) 78.49% compared to head (12302f8) 78.52%.

Additional details and impacted files

@@                Coverage Diff                 @@
##           releases/1.1.0     #868      +/-   ##
==================================================
+ Coverage           78.49%   78.52%   +0.02%     
==================================================
  Files                 206      206              
  Lines               24939    24961      +22     
  Branches             5007     5011       +4     
==================================================
+ Hits                19577    19601      +24     
+ Misses               4236     4235       -1     
+ Partials             1126     1125       -1     

Flag	Coverage Δ
macos-11_Python-3.8	77.50% <100.00%> (+0.02%)	⬆️
ubuntu-20.04_Python-3.8	78.51% <100.00%> (+0.02%)	⬆️
windows-2019_Python-3.8	78.45% <100.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
datumaro/components/format_detection.py	93.58% <100.00%> (+0.03%)	⬆️
datumaro/components/importer.py	90.00% <100.00%> (+0.31%)	⬆️
datumaro/plugins/data_formats/image_dir.py	80.55% <100.00%> (-1.50%)	⬇️
datumaro/plugins/data_formats/imagenet.py	91.25% <100.00%> (+1.10%)	⬆️
datumaro/plugins/data_formats/kitti/importer.py	85.00% <100.00%> (+5.93%)	⬆️
datumaro/plugins/data_formats/market1501.py	91.22% <100.00%> (+0.15%)	⬆️
datumaro/plugins/data_formats/mvtec/importer.py	100.00% <100.00%> (ø)
datumaro/util/definitions.py	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[sooahleex]

Looks good to me. I have one question, is there any reason to check whether images is under blacklisted folder only for imagenet dataset?

[vinnamkim]

Looks good to me. I have one question, is there any reason to check whether images is under blacklisted folder only for imagenet dataset?

Because imagenet format has very loose condition to be detected (sub-directories filled with images), we need to add the blacklist names to prevent it from being falsely detected.

[wonjuleee]

Thank you for your great work!