Fix misimplements for stream importer found while testing on the real-life COCO2017 object detection dataset

[vinnamkim]

Summary

1. Fix broken is_stream test
2. Fix StreamDataset.import_from(path, format="coco_instances") actually create Dataset not StreamDataset.
3. To speed up, let the initial length of _CocoBase(stream=True) can be obtained from COCOPageMapper.
4. Speed up parsing the "categories" section when it is at the end of JSON file.
5. To speed up, change caching logic and size slightly.
6. Fix _CocoBase(stream=False) abruptly raising an error when the progress reporter is given.
7. Add COCOExtractorMerger to handle "coco" import which should merge extractors across the tasks.

How to test

I manually tested the following code on the real-life COCO2017 object detection dataset.

from datumaro.components.dataset_base import DatasetItem
from datumaro.components.dataset import StreamDataset, Dataset
from time import time
from datumaro.components.progress_reporting import TQDMProgressReporter
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--stream", action="store_true", help="Use stream importer")


def upload_to_geti_db(item: DatasetItem) -> None:
    # Hi, I'm mock!
    pass


if __name__ == "__main__":
    args = parser.parse_args()
    start = time()

    dataset = (
        StreamDataset.import_from(
            "coco_json", format="coco_instances", progress_reporter=TQDMProgressReporter()
        )
        if args.stream
        else Dataset.import_from(
            "coco_json", format="coco_instances", progress_reporter=TQDMProgressReporter()
        )
    )

    for item in dataset:
        upload_to_geti_db(item)

    print(f"Done. Elapsed time: {time() - start:.2f}s")

Results:

• No stream

(datumaro-basic) vinnamki@vinnamki:~/datumaro/ws_datum/coco$ mprof run --python python test_perf.py
mprof: Sampling memory every 0.1s
running new process
running as a Python program...
WARNING:root:File 'coco_json/annotations/person_keypoints_val2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/person_keypoints_train2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/captions_train2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/captions_val2017.json' was skipped, could't match this file with any of these tasks: coco_instances
Parsing image info in 'instances_val2017.json': 100%|██████████████████████████████████████████████████| 5000/5000 [00:00<00:00, 77041.13it/s]
Parsing annotations in 'instances_val2017.json': 100%|███████████████████████████████████████████████| 36781/36781 [00:01<00:00, 32391.89it/s]
Parsing image info in 'instances_train2017.json': 100%|████████████████████████████████████████████| 118287/118287 [00:01<00:00, 83969.43it/s]
Parsing annotations in 'instances_train2017.json': 100%|███████████████████████████████████████████| 860001/860001 [00:27<00:00, 31262.75it/s]
Done. Elapsed time: 38.17s

• Stream

(datumaro-basic) vinnamki@vinnamki:~/datumaro/ws_datum/coco$ mprof run --python python test_perf.py --stream
mprof: Sampling memory every 0.1s
running new process
running as a Python program...
WARNING:root:File 'coco_json/annotations/person_keypoints_val2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/person_keypoints_train2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/captions_train2017.json' was skipped, could't match this file with any of these tasks: coco_instances
WARNING:root:File 'coco_json/annotations/captions_val2017.json' was skipped, could't match this file with any of these tasks: coco_instances
Parsing image info in 'instances_val2017.json': 100%|███████████████████████████████████████████████████| 5000/5000 [00:01<00:00, 3650.78it/s]
Parsing image info in 'instances_train2017.json': 100%|█████████████████████████████████████████████| 118287/118287 [00:39<00:00, 3027.10it/s]
Done. Elapsed time: 238.61s

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added the description of my changes into CHANGELOG.​
• I have updated the documentation accordingly

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

[vinnamkim]

Change all method to classmethod to be used in COCOExtractorMerger

[vinnamkim]

Move some classes here to break the circular dependency

[wonjuleee]

I am not sure how COCOExtractorMerger is operated across tasks and what difference from the original ExtractorMerger. Could you clarify this?

[vinnamkim]

I am not sure how COCOExtractorMerger is operated across tasks and what difference from the original ExtractorMerger. Could you clarify this?

Let's assume we import tests/assets/coco_dataset/coco as "coco" format.
This will have two extractors for the val subset at least:

1. annotations/captions_val.json
2. annotations/instances_val.json

Both extractors have a dataset item (id=40), but different annotations (captions or polygons).
Therefore, COCOExtractorMerger should merge the two dataset item into one by aggregating all annotations.