Add openvino interpreter samples #159

chuneuny-emily · 2021-03-12T02:18:09Z

Summary

This PR includes ,

README.md, which explains what models are supported in the samples, and how to download OpenVINO models.
Keyword in sampler was modified to match the format of the interpreter. ('score' -> 'confidences')
Modified the words of test_sampler.py accordingly.
The example of the user manual has been modified to fit the refactored sampler(Refactor sampler #149).

How to test

Checklist

I submit my changes into the develop branch
I have added description of my changes into CHANGELOG
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

zhiltsov-max · 2021-03-12T11:43:11Z

I suggest moving openvino_interpreter_samples into openvino/samples and renaming openvino_launcher.py into openvino/launcher.py.

…openvino_launcher.py into openvino/launcher.py.

chuneuny-emily · 2021-03-16T00:13:21Z

I updated the requested changes. Can you review it again?

zhiltsov-max · 2021-03-16T07:12:21Z

We've discussed the problem with importing openvino launcher after these changes, but I don't see any changes here regarding this. Have you tested the patch?

chuneuny-emily · 2021-03-16T07:17:21Z

Yes. I tested. It works well. The reason previously openvino was not found, was because I had used python datum.py in repo instead of datum installed(v0.1.6.1)

datumaro/plugins/openvino/README.md

zhiltsov-max · 2021-03-16T09:01:29Z

datumaro/plugins/openvino/README.md

+    - vehicle-detection-0202 (https://docs.openvinotoolkit.org/latest/omz_models_intel_vehicle_detection_0202_description_vehicle_detection_0202.html)
+
+- Public Pre-Trained Models(OMZ)
+  - Classification


Probably, it would be great, if you gave a notion about classes of supported models, instead of specific models. This would also cover custom, manually-converted models.

can you give an example? The model link contains model input/output info. Do you want to change,
before

Classification

mobilenet-v2-pytorch

to

Classification

mobilenet-v2-pytorch: imagenet 1000 classes

No, I'm speaking about a line "or any other model with the last layer of type DetectionOutput" or any other similar kind of advice for searching other classes of models, which can be supported, but not tested (including custom user models). Or any instruction on how to get a (potentially, custom) model to work.

datumaro/plugins/openvino/README.md

zhiltsov-max · 2021-03-16T09:06:56Z

datumaro/plugins/openvino/README.md

+
+## Model inference
+- Prerequisites: 
+    - Put the models downloaded in <datumaro_root_dir>/datumaro/plugins/openvino/models


This approach doesn't look right. Do your examples only work with manually downloaded Datumaro repository? For a typical user, <datumaro_root_dir> is somewhere in the python libraries.

It's only the part that performs inference under the premise that the Datumaro repo is received. Do yo want to insert Datumaro ?

datumaro/plugins/openvino/README.md

datumaro/plugins/openvino/samples/mobilenet_v2_pytorch_interp.py

datumaro/plugins/openvino/samples/ssd_face_detection_interp.py

datumaro/plugins/openvino/samples/ssd_person_vehicle_bike_detection_interp.py

zhiltsov-max · 2021-03-16T09:27:04Z

Add a link to the plugin docs in the user manual.

chuneuny-emily · 2021-03-16T13:09:42Z

I uploaded PR again based on your comments. Please review it.

zhiltsov-max · 2021-03-16T14:09:05Z

datumaro/plugins/openvino/README.md

-    - OpenVINO models (To download OpenVINO models, please go https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
-    - VOCdevkit dataset (To download VOC2012 dataset, please go http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
+    - OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)


Missing )

Suggested change

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html))

zhiltsov-max · 2021-03-16T14:09:27Z

datumaro/plugins/openvino/README.md

-    - VOCdevkit dataset (To download VOC2012 dataset, please go http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
+    - OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
+    - PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)


Missing )

Suggested change

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit))

zhiltsov-max · 2021-03-16T14:10:15Z

datumaro/plugins/openvino/README.md

@@ -43,30 +50,31 @@ Interpreter samples to parse OpenVINO inference outputs.

 ## Model inference
 - Prerequisites: 
-    - Put the models downloaded in <datumaro_root_dir>/datumaro/plugins/openvino/models
-    - Put the VOCdevkit dataset downloaded in <datumaro_root_dir>
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)


Missing ) (+3 next)

Suggested change

- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)

- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html))

zhiltsov-max · 2021-03-16T14:11:20Z

datumaro/plugins/openvino/README.md

  #	datum create -o <proj_dir>
  #	datum model add -l <launcher> -p <proj_dir> --copy -- -d <path_to_xml> -w <path_to_bin> -i <path_to_interpreter_script>
  #	datum add path -p <proj_dir> -f <format> <path_to_dataset>
  #	datum model run -p <proj_dir> -m model-0
  #
  # Examples
  # Detection> ssd_mobilenet_v2_coco
-  . /opt/intel/openvino_2021/bin/setupvars.sh
+  /opt/intel/openvino_2021/bin/setupvars.sh


. or source in the beginning is required

Suggested change

/opt/intel/openvino_2021/bin/setupvars.sh

. /opt/intel/openvino/bin/setupvars.sh

If we install openvino 2021.r1, the default path is /opt/intel/openvino_2021, not /opt/intel/openvino. The default path like openvino_2021 had been using since openvino 2021.r1. Do you still want me to change it?

I changed it to openvino because we don't know user which Openvino version will install. openvino is more general. So I changed it.

zhiltsov-max · 2021-03-16T14:29:28Z

datumaro/plugins/openvino/samples/ssd_vehicle_detection_interp.py

+            w = min(int(det[5] * input_width - x), input_width)
+            h = min(int(det[6] * input_height - y), input_height)
+
+            image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))


Suggested change

image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))

image_results.append(Bbox(x, y, w, h, label=label,

attributes={ 'score': conf, 'scores': list(map(float, det_confs)) }

))

zhiltsov-max · 2021-03-16T14:30:18Z

datumaro/plugins/openvino/samples/ssd_person_vehicle_bike_detection_interp.py

+            w = min(int(det[5] * input_width - x), input_width)
+            h = min(int(det[6] * input_height - y), input_height)
+
+            image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))


Suggested change

image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))

image_results.append(Bbox(x, y, w, h, label=label,

attributes={ 'score': conf, 'scores': list(map(float, det_confs)) }

))

zhiltsov-max · 2021-03-16T14:30:58Z

datumaro/plugins/openvino/samples/ssd_person_detection_interp.py

+            w = min(int(det[5] * input_width - x), input_width)
+            h = min(int(det[6] * input_height - y), input_height)
+
+            image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))


Suggested change

image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))

image_results.append(Bbox(x, y, w, h, label=label,

attributes={ 'score': conf, 'scores': list(map(float, det_confs)) }

))

zhiltsov-max · 2021-03-16T14:31:33Z

datumaro/plugins/openvino/samples/ssd_mobilenet_coco_detection_interp.py

+            w = min(int(det[5] * input_width - x), input_width)
+            h = min(int(det[6] * input_height - y), input_height)
+
+            image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))


Suggested change

image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))

image_results.append(Bbox(x, y, w, h, label=label,

attributes={ 'score': conf, 'scores': list(map(float, det_confs)) }

))

zhiltsov-max · 2021-03-16T14:32:06Z

datumaro/plugins/openvino/samples/ssd_face_detection_interp.py

+            w = min(int(det[5] * input_width - x), input_width)
+            h = min(int(det[6] * input_height - y), input_height)
+
+            image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))


Suggested change

image_results.append(Bbox(x, y, w, h, label=label, attributes={"score": conf, "scores": list(map(float, det_confs)),},))

image_results.append(Bbox(x, y, w, h, label=label,

attributes={ 'score': conf, 'scores': list(map(float, det_confs)) }

))

zhiltsov-max · 2021-03-16T14:33:14Z

datumaro/plugins/openvino/README.md

+  # Examples
+  # Detection> ssd_mobilenet_v2_coco
+  /opt/intel/openvino_2021/bin/setupvars.sh
+  cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/


Suggested change

cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/

cd datumaro/plugins/openvino/

zhiltsov-max · 2021-03-16T14:36:09Z

datumaro/plugins/openvino/README.md

+  datum create -o proj_ssd_mobilenet_v2_coco_detection
+  datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- --output-layers=do_ExpandDims_conf/sigmoid -d model/ssd_mobilenet_v2_coco.xml -w model/ssd_mobilenet_v2_coco.bin -i samples/ssd_mobilenet_coco_detection_interp.py
+  datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc /home/cvalgo/workspace/datumaro/VOCdevkit
+  datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0


Suggested change

datum create -o proj_ssd_mobilenet_v2_coco_detection

datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- --output-layers=do_ExpandDims_conf/sigmoid -d model/ssd_mobilenet_v2_coco.xml -w model/ssd_mobilenet_v2_coco.bin -i samples/ssd_mobilenet_coco_detection_interp.py

datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc /home/cvalgo/workspace/datumaro/VOCdevkit

datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0

datum create -o proj_ssd_mobilenet_v2_coco_detection

datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- \

--output-layers=do_ExpandDims_conf/sigmoid \

-d model/ssd_mobilenet_v2_coco.xml \

-w model/ssd_mobilenet_v2_coco.bin \

-i samples/ssd_mobilenet_coco_detection_interp.py

datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc VOCdevkit/

datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0

zhiltsov-max · 2021-03-16T14:36:47Z

datumaro/plugins/openvino/README.md

+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
+    - Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)
+    - OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
+    - PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)


Suggested change

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit))

zhiltsov-max · 2021-03-16T14:37:04Z

datumaro/plugins/openvino/README.md

+- Prerequisites: 
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
+    - Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)
+    - OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)


Suggested change

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] (https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html))

zhiltsov-max · 2021-03-16T14:37:16Z

datumaro/plugins/openvino/README.md

+## Model inference
+- Prerequisites: 
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
+    - Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)


Suggested change

- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)

- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md))

zhiltsov-max · 2021-03-16T14:38:25Z

datumaro/plugins/openvino/README.md

+
+## Model download
+- Prerequisites
+    - OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)


Suggested change

- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)

- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html))

zhiltsov-max · 2021-03-16T14:39:08Z

datumaro/plugins/openvino/README.md

+Interpreter samples to parse OpenVINO™ inference outputs.
+
+## Models supported from interpreter samples
+There are 5 SSD and 1 image classification interpreter samples.  


Suggested change

There are 5 SSD and 1 image classification interpreter samples.

There are detection and image classification examples.

zhiltsov-max · 2021-03-16T14:39:54Z

datumaro/plugins/openvino/README.md

+## Models supported from interpreter samples
+There are 5 SSD and 1 image classification interpreter samples.  
+
+- 5 SSD interpreter samples support the following models.


Suggested change

- 5 SSD interpreter samples support the following models.

- Detection (SSD-based)

chuneuny-emily · 2021-03-17T01:35:57Z

I uploaded PR again based on your comments. Can you review it?

chuneuny-emily added 3 commits March 12, 2021 10:16

Add OpenVINO interpreter samples

bd11660

Change variable name from score to confidences

adf7490

Fix wrong path

ec0a405

chuneuny-emily requested a review from zhiltsov-max March 12, 2021 02:18

Move openvino_interpreter_samples into openvino/samples and renaming …

1021724

…openvino_launcher.py into openvino/launcher.py.

zhiltsov-max suggested changes Mar 16, 2021

View reviewed changes

Update Maxim's comments

6cefbfd

zhiltsov-max reviewed Mar 16, 2021

View reviewed changes

chuneuny-emily added 2 commits March 17, 2021 10:02

Update Maxim's comments

5dc3744

Update Maxim's comments

fab3782

zhiltsov-max approved these changes Mar 17, 2021

View reviewed changes

zhiltsov-max merged commit a698cac into openvinotoolkit:develop Mar 17, 2021

	- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
	- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html))

	- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
	- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit))

	- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
	- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html))

	/opt/intel/openvino_2021/bin/setupvars.sh
	. /opt/intel/openvino/bin/setupvars.sh

	cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/
	cd datumaro/plugins/openvino/

	- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)
	- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md))

	There are 5 SSD and 1 image classification interpreter samples.
	There are detection and image classification examples.

	- 5 SSD interpreter samples support the following models.
	- Detection (SSD-based)

Add openvino interpreter samples #159

Add openvino interpreter samples #159

Conversation

chuneuny-emily commented Mar 12, 2021

Summary

How to test

Checklist

License

zhiltsov-max commented Mar 12, 2021

chuneuny-emily commented Mar 16, 2021

zhiltsov-max commented Mar 16, 2021

chuneuny-emily commented Mar 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiltsov-max Mar 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiltsov-max commented Mar 16, 2021

chuneuny-emily commented Mar 16, 2021

zhiltsov-max Mar 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiltsov-max Mar 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiltsov-max Mar 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiltsov-max Mar 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chuneuny-emily commented Mar 17, 2021

zhiltsov-max Mar 16, 2021 •

edited

Loading

zhiltsov-max Mar 16, 2021 •

edited

Loading

zhiltsov-max Mar 16, 2021 •

edited

Loading

zhiltsov-max Mar 16, 2021 •

edited

Loading

zhiltsov-max Mar 16, 2021 •

edited

Loading