Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ExtractedMask and update importers who can use it to use it #1480

Conversation

vinnamkim
Copy link
Contributor

@vinnamkim vinnamkim commented Apr 26, 2024

Summary

  • Ticket no. 138711
  • Add ExtractedMask which is a dedicated mask annotation class to use a single index mask source. The index mask is a integer 2D array and its pixel can indicate a label id (class) or instance id.
  • The advantage of introducing this class is that we can minimize a memory burden by sharing a single source explicitly. In addition, we can use a computational burden for the following scenario:
Create a semantic segmentation map by combining Datumaro Mask annotations
Setup Assume there is a Datumaro dataset item and it has multiple Datumaro Mask. We can merge them into one index mask to construct a semantic segmentation map.
Before 1) For each Datumaro Mask annotation, create a binary masks, 2) Merge binary masks into one index mask
After 1) Check all Mask annotations are ExtractedMask and it share the same index mask, 2) Just extract the source index mask

It can save huge computations required for 2D array manipulations: creating a binary mask and merging binary masks.

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2024 Intel Corporation
#
# SPDX-License-Identifier: MIT

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim requested review from a team as code owners April 26, 2024 10:51
@vinnamkim vinnamkim requested review from sooahleex and removed request for a team April 26, 2024 10:51
Copy link

codecov bot commented Apr 26, 2024

Codecov Report

Attention: Patch coverage is 98.33333% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 80.84%. Comparing base (44cc56a) to head (af03f29).
Report is 47 commits behind head on develop.

Files Patch % Lines
src/datumaro/plugins/data_formats/voc/base.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1480      +/-   ##
===========================================
- Coverage    80.85%   80.84%   -0.02%     
===========================================
  Files          271      273       +2     
  Lines        30689    31402     +713     
  Branches      6197     6316     +119     
===========================================
+ Hits         24815    25386     +571     
- Misses        4489     4592     +103     
- Partials      1385     1424      +39     
Flag Coverage Δ
ubuntu-20.04_Python-3.10 80.82% <98.33%> (-0.03%) ⬇️
windows-2022_Python-3.10 80.81% <98.33%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

>>>
>>> index_mask = np.random.randint(low=1, high=3, size=(10, 10), dtype=np.uint8)
>>> mask1 = ExtractedMask(index_mask=index_mask, index=1, label=1) #
>>> mask2 = ExtractedMask(index_mask=index_mask, index=2, label=2) #
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between index and label here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As described here

Attributes:
index_mask: Integer 2D Numpy array. Its pixel can indicate a label id (class) or an instance id.
index: Integer value to extract a binary mask from the given index mask.

index can denote an instance id or a label id. It depends on the dataset. Anyway, I enhanced the example docstring in af03f29 to make it more descriptive.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still confused. The updated example have the same index and label. In my understanding, mask1 is a binary mask with 0 and 1, where all 1s indicate the index indeed. This looks fine. But index might be label for all cases. Is there any case that index is not equal to label?

Copy link
Contributor Author

@vinnamkim vinnamkim Apr 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mask is a binary mask and its 1s actually indicate label=1 (not index=1) in this example. index is just used to extract a binary mask from index_mask. It is possible that index can be different with label by dataset transformation. For example, if the label remapping transform merges label=1, index=1 (1: apple) and label=2, index=2 (2: orange) to label=3 (3: fruits). Then, apple masks will have index=1, label=3 and orange masks will have index=2, label=3.

Signed-off-by: Kim, Vinnam <vinnam.kim@intel.com>
@vinnamkim vinnamkim merged commit d196fc8 into openvinotoolkit:develop Apr 29, 2024
8 checks passed
@wonjuleee wonjuleee added this to the 1.7.0 milestone May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants