Recommendations for running Yolo5 on Android

[jeggers-88]

❔Question

Does anybody has recommendations, how to run yolo5 with nice performance on android ?

Additional context

First I tried to export my custom trained yolo5s NN to onnx to import it with opencv. But I was not able to load the onnx File successfully with the newest opencv 4.4.0:

 Slice layer only supports steps = 1 (expected: 'countNonZero(step_blob != 1) == 0'), where 'countNonZero(step_blob != 1)' is 1
 must be equal to  '0' is 0

Actually I was successfully able to run the torchscript export with pytorch in my android app. But with very poor performance (nearly 2 seconds inference time on galaxy tab a sm-t510).
Running on my laptop the detect.py shows only ~0.015 seconds (yolo5s with NVIDIA Quadro T2000).

Thanks very much for any help!

[github-actions]

Hello @jeggers-88, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[zldrobit]

@jeggers-88 you could try TFLite Android demo with GPU support, since your galaxy tab a sm-t510 has a GPU. I guess the inference time is about 1.5s like Snapdragon 820 GPU.

[jeggers-88]

Thanks for your answer, great that yolo5 now has inbuilt tflite export possibilities!
However I switched the last week to tensorflow lite with a simple mobilenet network v1 (CPU execution time ~100ms for 320x320). The ~1 second execution time (ok there 640x640) of the exported yolo tflite NN is too slow for my use case.