Add with_subset_dirs decorator

[vinnamkim]

Summary

• Ticket no. 104802
• Please see (TEMP) Datumaro Dataset CLI Guide by @sungmanc. It describes how to input the data path when using OTX CLI. However, the current state is that there are different rules according to task types, which can cause confusion to users. For example, Multi-class Classification task requires otx train [${template}] --train-data-roots ${root}/train --val-data-roots ${root}/val which needs the directory path at the subset level but Multi-label Classification task requires only the root level directory path such as otx train [${template}] --train-data-roots ${root} --val-data-roots ${root}. This should be resolved from Datumaro side.
• In this PR, we introduce @with_subset_dirs decorator to make the existing importer to handle the following directory structure including subsets:

root
  - train
     - {dataset}
  - val
     - {dataset}
  - test
     - {dataset}

• Using @with_subset_dirs, ImagenetWithSubsetDirsImporter will cover the Multi-class Classification task case.

How to test

I added unit tests to cover this feature.

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

[codecov-commenter]

Codecov Report

Base: 78.26% // Head: 78.26% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (ba8cb54) compared to base (fd4d695).
Patch coverage: 84.74% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop     #816   +/-   ##
========================================
  Coverage    78.26%   78.26%           
========================================
  Files          184      184           
  Lines        23563    23614   +51     
  Branches      4880     4892   +12     
========================================
+ Hits         18442    18482   +40     
- Misses        4028     4033    +5     
- Partials      1093     1099    +6     

Flag	Coverage Δ
macos-11_Python-3.8	?
ubuntu-20.04_Python-3.8	78.25% <84.74%> (+<0.01%)	⬆️
windows-2019_Python-3.8	78.19% <84.74%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
datumaro/components/importer.py	89.43% <80.43%> (-5.51%)	⬇️
datumaro/components/dataset.py	82.42% <100.00%> (ø)
datumaro/components/environment.py	90.90% <100.00%> (+0.14%)	⬆️
datumaro/plugins/data_formats/imagenet.py	90.14% <100.00%> (+0.58%)	⬆️
datumaro/plugins/synthetic_data/utils.py	83.33% <0.00%> (-1.67%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[vinnamkim]

Please see #817 (comment), this PR should be blocked until the backport of v1.0.0.

[sungmanc]

What a nice feature, it must improve the user experience. Thanks

[vinnamkim]

Please see #817 (comment), this PR should be blocked until the backport of v1.0.0.

Done. It's ready for review now.

[wonjuleee]

Looks good, but could you provide some examples for using with_subset_dirs more than ImageNet cases?

[vinnamkim]

Looks good, but could you provide some examples for using with_subset_dirs more than ImageNet cases?

After this PR. I'll do a next work to plugin with_subset_dirs to Common Semantic Segmentation format.

More works need to investigate another format which we can plugin with_subset_dirs.

[wonjuleee]

Looks good, but could you provide some examples for using with_subset_dirs more than ImageNet cases?

After this PR. I'll do a next work to plugin with_subset_dirs to Common Semantic Segmentation format.

More works need to investigate another format which we can plugin with_subset_dirs.

It must be good!