-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial support for rockchip boards #8382
Conversation
✅ Deploy Preview for frigate-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
handful of things to look at but this is a good start, a couple notes / questions:
- how much testing has been done?
- is ffmpeg going to be added in a separate PR?
- be sure to add yourself as a code owner in CODEOWNERS file for
docker/rockchip
docker/rockchip/rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl
Outdated
Show resolved
Hide resolved
Is there a docker container availible for this already? I would really like to test it. |
apply requested changes Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
Not yet, but you can build it yourself. Clone my fork https://github.com/MarcA711/frigate-rockchip then cd into the cloned directory and build it with |
Thank you very much :) |
By the way what inference times are y'all seeing? |
Comfortable under 100 for me (around 80), but i only have one 2.560 x 1.440 camera running as of now at 5 FPS detection speed |
I guess I did all requested changes besides removing the hide-warnings package. To your questions:
|
this would be a major undertaking especially given all the ways that ffmpeg is embedded into the configuration and is only moving more in that direction, I don't think using gstreamer in frigate makes sense |
Yes, I already imagined that this wouldn't be that easy. Too bad because I like to work with gstreamer because it is so versatile. You asked for inference speeds: |
Nice, that is not too bad at all |
How does the EdgeTpu perform? |
It is generally faster but of course it depends on the model, the COCO edge tpu model that comes with frigate has an inference speed of about 9 ms. My frigate+ model has an inference speed of ~ 4.9 ms |
@MarcA711 I let the container run for the night but it does not pick up any objects/persons this far where my coral would definitely have. I used your minimal config example without changes. Do you have any extra ffmpeg args? |
Did you enable recording for your camera stream? Try this config:
|
those input args are not input args, that's not right. I would just let frigate use the default input args |
You are right, I updated my comment. |
I have a question. At the moment, I rewrite my dockerfile. I try to build the wheels for the rknn toolkit2. I do it this way:
requirements-wheels-rk.txt has the following contents:
rknn-toolkit-lite2 depends on numpy for example. However, numpy was already installed in the main frigate image before. How do I make sure, that I don't overwrite the numpy version in the main image? |
This is my config:
But no objects/persons still.
|
@MarcA711 by default I believe pip will not overwrite the dependencies, numpy is already installed so as long as the installed version satisfies rknn requirement it won't be overwritten. if it does not satisfy the requirement then pip will throw an error and the build will fail |
@MrPlatnum you are also missing some important input args, I'd highly suggest just using one of the presets. The easiest way to debug this behavior would be with a recording of the debug view with bounding boxes, regions, and motion boxes enabled |
@NickM-27 the problem is, there are no packages installed in the wheel image. It just builds the wheels and saves them to a .whl-file. When I install these .whl-files in the main image, it will override numpy. |
I don't believe that is correct if you are pulling from the correct layer. Tensorrt pulls from the Then this is installed from in the frigate layer frigate/docker/tensorrt/Dockerfile.amd64 Lines 7 to 19 in d1620b4
they are then installed later on in the final image frigate/docker/tensorrt/Dockerfile.amd64 Lines 31 to 32 in d1620b4
|
actually nevermind I believe you are correct, looking at the requirements file for amd64 nvidia and it does define numpy https://github.com/blakeblackshear/frigate/blob/dev/docker/tensorrt/requirements-amd64.txt |
Well, I found a solution. I can use the requirements-wheels.txt from the main image as a constraints file: However, links are not allowed in constraints files, so these lines have to be removed somehow:
Any idea how to do that? |
I have now started playing with different setup. I installed Ubuntu 20.04 with kernel 5.10.0 that according to vendor of the device has most of the features included.
here is my docker compose setup:
FFPROBE: would appreciate more knowledgable people help how to setup my linux and docker setup as I assume I am missing something? @MarcA711 if anytime you would be able to assist would be helpfull. |
Hey, Could you please run this command and post the output:
Can you provide a link to the distro that you run on your device? Moreover, could you try adding the bold lines to your docker-compose.yml:
|
cat: /sys/kernel/debug/rknpu/version: No such file or directory
this is under kernel directory /sys/kernel update: update: p.s. I have mostly free time until new year so will be mostly available also |
Could you then try: Does rknn detector work as well now? If not, could you run this: |
It seems that the rknpu is not enabled on your distro. From the odroid os install guide (https://wiki.odroid.com/getting_started/os_installation_guide#tab__odroid-m1) I can see that there are multiple versions of Ubuntu. Some explicitly mention that the NPU is enabled and you should disable Petitboot (described here: https://wiki.odroid.com/odroid-m1/software/boot_sequence#bypassing_petitboot). First, I would disable Petitboot on your current installation and see, if the NPU works. If not, install Ubuntu Gnome with NPU support. To check if the NPU works, you should type |
ohhh well that lovely linux.... i made love for all day today. 3 OS reinstalls 😄 I would appreciate some more help with few other issues:
|
I am glad that it works! To the other issues you mentioned:
|
Ok, this means fun only begins here... What about hardware acceleration, how can I test that? If I understand right rkmpp is key here? |
hardware video decoding should work, since you specified this options:
Hardware transcoding is also possible using go2rtc, but since your cam already sends a h264 streams this is not necessary. Hardware video scaling is not possible in the current version but will be included in the next one. |
sad story for this device I am not suitable :) at least I lack knowledge thats needed to make it run. looking around now to order sbc with AlderLake N100 CPU I found few options with cost of 130$ and those devices have also reasonable performance and power consumption. |
Sorry for the late reply, i had a cold. It's a pity that the board didn't work out for you. Unfortunately, the rk3568 platform is not as well supported as the rk3588 platform. In the future this will hopefully get better. However, didn't the board work for you using the 4.19 kernel? The only issue was a higher CPU usage. |
Yes it worked but with very low performance. Seemed even worse than with frigate as HAOS addon running only on cpu. Already ordered new device but if vendor will bring some upgrades will test it someday Thanks anyway for your contribution to the community . P.s. tried armbian and other options with no luck |
@MaBeniu I have seen, that Rockchip plans to release a new Kernel (version 6.1) for rk3568 socs. Maybe this will resolve your issues. |
Maybe. Thanks for heads up! Once released I will test it. soon I this device will become secondary. |
hi @MarcA711 , could you pass me your dockercompose and the basic frigate configuration file so I can run it on my op5+ please? |
Hey @JMNRA, Have a look at the docs, they cover everything: If you have trouble or need further assistance feel free to contact me. |
@MarcA711 Is there any news about the HA Add-On with the hardware decoder for FFMPEG enabled? Not needed anymore after I read everything in this ticket here: Wow, I thought it might be easier, but I have to be pacient... |
Hey @Bascht74, As you may have seen, I found a new approach and posted instructions. Maybe you can help testing? |
@MarcA711 Could you point me to the right direction? Where can I find it :-) ? I hope it is not using supervised HA. As I am doing it for a friend I cannot play with his stuff too much… |
Coudn‘t we try something different? We could patch that file so that the right folder would be mounted within the docker container. From that perspective we could see, if there are any side effects and if it is a solution. If yes, we could kindly ask for implementation because it is a solution for our issue/demand. |
like was described that was already tried and it does not work due to limitations around addons |
@Bascht74 But yes, it uses supervised HA. HAOS will not work with frigate rockchip in under a year or so. The only alternative is HA Container. This will work right now (I use this setup). But it will not support HA add-ons. |
As I don't want to patch the OS for my friend, I cannot follow that path. ... but if I understand everything correctly that would only help for HA supervised. |
There are two ways to run frigate and Home Assistant together: Home Assistant Supervised and Home Assistant Container If you want to go with Home Assistant Supervised, you don't need to patch the os. You just need to install Debian on your board and then install home assistant supervised on it. You can still use the system (e.g. run other docker containers not related to Home Assistant). However, if you say you don't want to change your friends system too much, you should look into Home Assistant Container. This basically installs Home Assistant in Docker and leaves the rest of the system as is. This method lacks some features compared to Home Assistant Supervised (see this overview), most notably you can't install Home Assistant Add-Ons. However, most Add-Ons are available as standalone docker images, so this is no deal breaker. If you use Home Assistant Container, you can just run frigate in another docker container. Instructions to install Home Assistant Container are here Edit: added links |
What do you mean by this? The discussion actually led to a solution (using Home Assistant Supervised). Unfortunately, Home Assistant OS will not work with frigate-rockchip. But since you said you don't want to change your friends system, Home Assistant OS is even worse for you. You would need to install a new OS to his system and Home Assistant actually manages his board then, so his access to the system would be restricted. |
... could lead to a solution for HAOS with the add-on. But I think then he has to wait... |
Well, it will at least take one year until the missing drives are developed, tested, merged into the Linux Kernel and then they will have to arrive in HAOS... Once setup, HA supervised is actually not that different. Maybe you should give it a try once it works. |
This allows users with certain rockchip socs to use the integrated NPU to accelerate inference.