Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for format-agnostic storage and loading of hashkey #1003

Merged

Conversation

sooahleex
Copy link
Contributor

@sooahleex sooahleex commented May 18, 2023

Summary

  • Ticket no.110555
  • In previous version, saving and loading hash_key is only supported for datumaro format. To enhance this, this PR support to format-agnostic save and load HashKey of annotations
    • Support format which could exported : cifar, camvid, cityscapes, coco, datumaro, icdar, image_dir, image_zip, kitti, labelme, market1501, mnist, mot, mots, open_images, vgg_face2, widerface, yolo
  • Load hash_key from hash_meta file during import
  • Add HashKey annotation in existed annotation

How to test

Added unittest to check save_and_load for Dataset which contains HashKey in annotations.

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

@sooahleex sooahleex added the ENHANCE Enhancement of existing features label May 18, 2023
@sooahleex sooahleex added this to the 1.3.0 milestone May 18, 2023
datumaro/plugins/data_formats/cifar.py Outdated Show resolved Hide resolved
datumaro/util/meta_file_util.py Outdated Show resolved Hide resolved
datumaro/util/meta_file_util.py Outdated Show resolved Hide resolved
tests/unit/test_camvid_format.py Outdated Show resolved Hide resolved
@sooahleex sooahleex marked this pull request as ready for review May 19, 2023 08:50
@sooahleex sooahleex requested review from a team as code owners May 19, 2023 08:50
@codecov-commenter
Copy link

codecov-commenter commented May 19, 2023

Codecov Report

Patch coverage: 77.14% and project coverage change: +0.18 🎉

Comparison is base (f01a905) 78.58% compared to head (1748fe6) 78.76%.

Additional details and impacted files
@@                Coverage Diff                 @@
##           releases/1.3.0    #1003      +/-   ##
==================================================
+ Coverage           78.58%   78.76%   +0.18%     
==================================================
  Files                 237      237              
  Lines               27074    27085      +11     
  Branches             5401     5404       +3     
==================================================
+ Hits                21275    21333      +58     
+ Misses               4534     4492      -42     
+ Partials             1265     1260       -5     
Flag Coverage Δ
macos-11_Python-3.8 77.77% <77.14%> (+0.17%) ⬆️
ubuntu-20.04_Python-3.8 78.74% <77.14%> (+0.18%) ⬆️
windows-2019_Python-3.8 78.65% <77.14%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
datumaro/cli/commands/explore.py 19.48% <0.00%> (ø)
datumaro/plugins/data_formats/cifar.py 88.77% <ø> (+1.53%) ⬆️
datumaro/plugins/data_formats/datumaro/exporter.py 95.25% <ø> (+4.04%) ⬆️
datumaro/plugins/data_formats/vgg_face2.py 83.71% <ø> (ø)
datumaro/components/annotation.py 91.30% <42.85%> (-0.29%) ⬇️
datumaro/components/exporter.py 86.69% <70.00%> (-0.33%) ⬇️
datumaro/components/dataset.py 82.80% <100.00%> (+0.04%) ⬆️
datumaro/plugins/data_formats/datumaro/base.py 92.36% <100.00%> (+3.32%) ⬆️
datumaro/util/meta_file_util.py 93.54% <100.00%> (+28.91%) ⬆️

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@sooahleex sooahleex changed the base branch from develop to releases/1.3.0 May 22, 2023 06:28
Copy link
Contributor

@vinnamkim vinnamkim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vinnamkim added a commit that referenced this pull request May 24, 2023
- This is required for #1003 to reserve a place to put in code snippets
for data-format agnostic hash exportation.

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
Copy link
Contributor

@vinnamkim vinnamkim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@sooahleex sooahleex merged commit 1d40e78 into openvinotoolkit:releases/1.3.0 May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ENHANCE Enhancement of existing features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants