Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a bug in the previous behavior when importing nested datasets in the project #1243

Merged
merged 3 commits into from
Jan 16, 2024

Conversation

vinnamkim
Copy link
Contributor

@vinnamkim vinnamkim commented Jan 15, 2024

Summary

  • Ticket no. 127589
  • Fix a bug found in Import only import one source #1214
  • In the previous Datumaro version, we could import the nested datasets in the given path. For example, if we Dataset.import_from(./some_project) for the following directory structure
./some_project/
├── dataset_1
└── dataset_2

, the imported dataset include dataset_1 and dataset_2.

  • This is common pattern for the Datumaro project, so that we have to fix this.

How to test

Added an integration test for this scenario.

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

…the project

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim added this to the 1.6.0 milestone Jan 15, 2024
@vinnamkim vinnamkim added the BUG Something isn't working label Jan 15, 2024
@vinnamkim vinnamkim marked this pull request as ready for review January 15, 2024 05:43
@vinnamkim vinnamkim requested review from a team as code owners January 15, 2024 05:44
@vinnamkim vinnamkim requested review from sooahleex and removed request for a team January 15, 2024 05:44
Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Copy link

codecov bot commented Jan 15, 2024

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (eb44355) 80.53% compared to head (bf9e6b7) 80.53%.

❗ Current head bf9e6b7 differs from pull request most recent head e0d266b. Consider uploading reports for the commit e0d266b to get more accurate results

Files Patch % Lines
src/datumaro/components/merge/extractor_merger.py 69.56% 4 Missing and 3 partials ⚠️
src/datumaro/cli/commands/convert.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1243   +/-   ##
========================================
  Coverage    80.53%   80.53%           
========================================
  Files          270      270           
  Lines        30232    30238    +6     
  Branches      5898     5903    +5     
========================================
+ Hits         24348    24353    +5     
  Misses        4506     4506           
- Partials      1378     1379    +1     
Flag Coverage Δ
ubuntu-20.04_Python-3.8 80.52% <81.39%> (+<0.01%) ⬆️
windows-2022_Python-3.8 80.50% <81.39%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Copy link
Contributor

@wonjuleee wonjuleee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In overall, it looks good to me, but I have a one quick question. Is this fix available only for coco and yolo formats?

@vinnamkim vinnamkim merged commit 5f8ddef into openvinotoolkit:develop Jan 16, 2024
3 checks passed
@vinnamkim
Copy link
Contributor Author

In overall, it looks good to me, but I have a one quick question. Is this fix available only for coco and yolo formats?

As we discussed, it is available only for the yolo formats. This is because importing datasets from sub-directories at once is not commonly available for all data formats.

@yunchu yunchu modified the milestones: 1.6.0, 2.0.0 Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BUG Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants