Yolo

[moz4r]

Place for yolo gurus to talk about integration Inside mrl !

What I understood

• Need to publish data
• Need linux support
• Need to call yolo from runtime instead of a .bat
• Need a filter for opencv
• Need code somewhere on github!

What else ?

[bretzel59]

For Linux support I think we only need to add the Linux build and the required libraries.

[supertick]

Ahoy !
It looks like as kwatters has mentioned that Yolo is supported in OpenCV,
I'm sure JavaCV is currently or shortly will be updated to support this as well ..

Here's the first link I found searching for OpenCV & Yolo - https://docs.opencv.org/3.4.0/da/d9d/tutorial_dnn_yolo.html

If we can add this as a filter to our OpenCV service, potentially it will solve all 5 line items by moz4r

[moz4r]

5 lines saved * x code lines , great ! that will be a filter that rock.
I think bretzel59 is afraid about performances and want to use the gpu for compute.

[kwatters]

heh. i feel honored.
so.. javacv 1.4 (which includes OpenCV 3.4) still has some dependency issues in maven central. (missing dependencies for ffmpeg and other transient dependencies.)
In theory similar support existed in OpenCV 3.3.1 .. I'll see what I can find.

[bretzel59]

Does yolo in JavaCV be CUDA Ready ? In my opinion, the support of Cuda is mandatory. (it seems to be but could you confirm that)

Will it be possible to publish data like :

• Name of detected objects
• Window coordinates (x-y) in the picture
• Numbers of identical objects in order to enumerate them

Will it be possible to use yolo 9000 model or any homemade pre-trained model ? Can I use my own file.names to inclued French or English translation ?

Could I switch pre-trained model on the fly ?

Hehe a lot of questions :D

[kwatters]

Mrl is upgraded to javacv 1.4. a Yolo filter has been added to OpenCV. You will need to have the yolo.cfg, yolo.weights and the coco.names files downloaded manually now for it to work.

I took a similar approach to the dl4j filter so the frame rate isn't affected by adding the Yolo filter.

[ghost]

Hi bretzel many questions, yes, I can only shine a partial light on it I have been able to make yolo2 run for me with the help of mark jay's videos (youtube). I run it as a python task but I assume the options for java are at least equivalent. The use of CUDA in my case is controlled in an "option" section for TFNet (from darkflow.net.build import TFNet) I was able to make it run with 'gpu' : 0.8 on my gforce 1070 card. using 'gpu':1.0 failed because it wanted to allocate too much memory from the card. using the YOLOv2 608x608 (as of this site https://pjreddie.com/darknet/yolo/) I get about 25 fps for running it on a video. without the gpu it's around 1 fps. If you only want to analyze a single frame it could well be done without the gpu. I was able to find yolo9000 data but it was slower and in my opinion worse on recognition (almost everything is a "whole"). But maybe there are different data sets around and I had no luck with mine. The result from return_predict(frame) is "label", "confidence" (0..1), "topleft" and "bottomright" for each recognized object in the frame. From this it should be easy to create an ordered list of the found labels. Mark Jay shows also how to add new objects to be recognized to an existing cfg (a spinner in his case) but that is a hard and tedious way to go as you need tons of example pictures to make it useful. I think we have to wait for algorithms that can do picture analysis by themself and build picture databases based on similarities / context with an example or by entering the objects label for such sets. Loading a model takes quite some time, about 70 seconds on my machine. With fat hardware you might be able to load 2 different models and then do the swiching by calling the return_predict from your wanted model. Yolo2 as of now does not include text recognition (surprised me a bit) and I am missing a bit a context relation - so many times it recognises a chair/tie/smartphone where it does not make sense from a human point of view. So there is room for improvement. It is however very good at identifying persons and cars - but e.g. can not identify a nose, eye, mouth when shown in close up. Juerg 2018-03-01 20:18 GMT+01:00 bretzel59 <notifications@github.com>:

…

Does yolo in JavaCV be CUDA Ready ? In my opinion, the support of Cuda is mandatory. (it seems to be but could you confirm that) Will it be possible to publish data like : - Name of detected objects - Window coordinates (x-y) in the picture - Numbers of identical objects in order to enumerate them Will it be possible to use yolo 9000 model or any homemade pre-trained model ? Can I use my own file.names to inclued French or English translation ? Could I switch pre-trained model on the fly ? Hehe a lot of questions :D — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#165 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AD0_CHepe3YXxFHXP6alKe2E7Wg-Ig85ks5taElsgaJpZM4SW8Ac> .

-- Jürg Maier Trungerstrasse 33 9543 St. Margarethen Schweiz juerg.maier@pobox.ch

[bretzel59]

Hi Juerg,

I'm already using cuda with darknet Under Windows, I've compiled a port made by Alexey. https://github.com/AlexeyAB/darknet The question is if it's possible tu use CUDA under JAVACV 1.4 in MRL.

I've already using yolo with MRL by a "yolo service" programmed by Dom14, The idea was to launch a single pic detection on demand. I can use any weigth files and switch on the by switching between 3 batch files (vos, coco 9000). Loading weigth files like yolo 9000 takes less than 5 seconds on my robot, (I3 7300T with 16Go RAM, SSD and GTX 1060). Yes you can use cpu for single frame detection but my idea is to release CPU for other tasks (motion control).

I' m thinking like you about Yolo 9000, it seems less accurate but works for me.

[hairygael]

Hello, Thanks to Bretzel59 and Dom14, I have been using Yolo via OpenCV, since two weeks. I am very surprised to be able to run yolo9000 on my tablet Lenovo with a analyse time of about 70 seconds, same as juerg. I just added some AIML strings in English version to run the three yolo.bat via voice command. Indeed the 9000 version has strange behaviors from a human point of view. I had a few times that a simple plastic bag laying on the floor can be analysed as a placenta... And even sometimes it doesn't see a person standing on the front of the camera... For demonstrations, I have been using the coco version (80 objects) which turns to be more accurate. In the data files there seems to be a version with a 1000 object included in the yolo directory, did anyone get that one working? Gael Langevin Creator of InMoov InMoov Robot <http://www.inmoov.fr> @inmoov <http://twitter.com/inmoov> 2018-03-02 10:44 GMT+01:00 bretzel59 <notifications@github.com>:

…

Hi Juerg, I'm already using cuda with darknet Under Windows, I've compiled a port made by Alexey. https://github.com/AlexeyAB/darknet The question is if it's possible tu use CUDA under JAVACV 1.4 in MRL. I've already using yolo with MRL by a "yolo service" programmed by Dom14, The idea was to launch a single pic detection on demand. I can use any weigth files and switch on the by switching between 3 batch files (vos, coco 9000). Loading weigth files like yolo 9000 takes less than 5 seconds on my robot, (I3 7300T with 16Go RAM, SSD and GTX 1060). Yes you can use cpu for single frame detection but my idea is to release CPU for other tasks (motion control). I' m thinking like you about Yolo 9000, it seems less accurate but works for me. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#165 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIF2x9lperrqaGXCKagnB5xLSyB8czMkks5taRRqgaJpZM4SW8Ac> .

[moz4r]

Will look forward to test last Yolo integration tks ! I just saw it : bytedeco/javacv#481
Kwatters you uptaded javaCv to 1.4 I assume we have now some kind of cuda support for people want to use it ?

[kwatters]

I've update MRL to JavaCV 1.4. Also, I added a new opencvfilter for "Yolo". This uses the coco names (80 classes of objects recognized.). It finds things like person, chair, and tv monitor pretty well. I haven't tested extensively.
The new OpenCV filter for Yolo has some configuration options on them for the "darknetHome" (this is the directory where darknet is installed. (including the yolo.weights cfg/yolo.cfg and data/coco.names )
The filter samples images from the frame grabber as fast as it can, processes the image on a separate thread, and then updates the current frame with the bounding boxes. This approach means it does not slow down the frame rate when running it. (I was seeing 15 fps with or without the yolo filter running.)
I definitely still need to clean up the code a bit and provide some python examples for using it.. Also, when installing opencv it should know to download the yolo models files, so perhaps we should add those to the myrobotlab artifactory instance and define them as a proper dependency.

As far as I can tell, I'm almost 85% a person.
https://twitter.com/kwatters76/status/969335209698131968

@moz4r I'm not sure about cuda support.. it all depends on how it was compiled and I'm not sure how to verify that from javacv land.

[bretzel59]

Ok manage to get it worky !

GREAT ! 👯‍♂️

But seems it takes a lot of CPU Power (60% for my i7). As I thought, 22fps for opencv video Stream, but 1fps detection. Cuda Cuda Cuda :)

You can see GPU was working but I think it was just for opencv stream because GPU is working exactly the same without yolo filter (34%) and opencv without filters takes 4% of my i7

[bretzel59]

It seems to be not compatible with Yolo 9000, this is the warning when configuring Yolo 9000 in the YoloFilter : "OpenCV Error: The function/feature is not implemented (Yolo9000 is not implemented) "

[bretzel59]

Ok, below, confirmation about Yolo 9000 (maybe others ?) :

Not supported under Opencv 3.4 at this time (maybe later).

[kwatters]

I just checked in changes that will automatically download and cache the models. Also a lot of other code cleanup. I think it's pretty good now for initial support.

Opencv should now publishYoloClassifications that you can subscribe to

[kwatters]

we should deploy the model as a zip file dependency in the mrl repo. Once that's done and I test that it's all working , we'll close this issue.. as we will have some yolo support. we can always add more models later under a new issue

[kwatters]

ok. now the latest will deploy the yolo models as part of the build. I still can't get it working on the raspi.. i think it's due to overheating or something.. the raspi becomes unresponsive when i try to start the yolo filter. otherwise, it seems to be working just fine on win x86 64 bit. I'd like to expose yolo via dl4j, however that's still work in progress and we can track that as a different issue.

closing ticket.