Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open Images: implement bounding box support #352

Merged
merged 8 commits into from
Jul 16, 2021
Merged

Open Images: implement bounding box support #352

merged 8 commits into from
Jul 16, 2021

Conversation

IRDonch
Copy link

@IRDonch IRDonch commented Jul 13, 2021

Summary

This continues the implementation of #274.

How to test

You can download some of the bounding boxes for OID, create images.meta as described in the documentation and try to import the dataset.

Checklist

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below)
# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

@zhiltsov-max
Copy link
Contributor

Also add a test like in #346

@IRDonch IRDonch marked this pull request as ready for review July 13, 2021 14:58
@IRDonch
Copy link
Author

IRDonch commented Jul 13, 2021

Also add a test like in #346

I have examined the implementation in #346, and it would be tricky to implement something like that for OID, because the annotation files often don't have fixed names - just possible suffixes. So it wouldn't be possible to specifically remove the annotation files for a particular subset that was removed. Would it work if I just removed all existing annotation files at the beginning of apply?

@zhiltsov-max
Copy link
Contributor

Honestly, it is tricky every time. We know from the patch which items were added, removed or modified and which subsets were modified. During writing it is also easy to find out modified subsets that became empty now. Will this information be enough to remove unnecessary files in this format?

@IRDonch
Copy link
Author

IRDonch commented Jul 14, 2021

We know from the patch which items were added, removed or modified and which subsets were modified. During writing it is also easy to find out modified subsets that became empty now. Will this information be enough to remove unnecessary files in this format?

The problem is that we don't know which files which annotations/metadata came from. For the training subset, it could be train-images-with-labels-with-rotation.csv, oidv6-train-images-with-labels-with-rotation.csv, image_ids_and_rotation.csv, or (if the user tries to really exercise the flexibility of the loader) even foobar-images-with-rotation.csv. So I don't see a better solution than just nuking the annotation directory.

Although I guess for safety I could do the nuking after the new files are written, instead of before.

@IRDonch
Copy link
Author

IRDonch commented Jul 14, 2021

Anyway, I think patching support is a distinct issue from bounding box support, so I think I'm going to work on it separately.

if label_index is None:
raise UndefinedLabel(
item_id=item.id, subset=item.subset,
label_name=label_name, severity=Severity.error)
Copy link
Contributor

@zhiltsov-max zhiltsov-max Jul 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, the severity argument should be separate from the error class. Probably, I see reporting a dataset problem as an Extractor (Converter) - base class - method, which could have such severity parameter. Then, it could be handled according to the current error control policy - e.g. a warning could be promoted to an error or ignored. #142. It is a big new feature, though.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand - do you want me to change something in the code? I didn't implement the UndefinedLabel class, I'm just using it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, this error class is not supposed to be used outside dataset validators. But we can leave it.

@zhiltsov-max zhiltsov-max merged commit cd12ec7 into openvinotoolkit:develop Jul 16, 2021
@IRDonch IRDonch deleted the open-images-boxes branch July 16, 2021 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants