This project demonstrates the combined power of the Zynq UltraScale+ and the Hailo-8 AI accelerator when used in multi-camera vision applications. The repo contains designs for several Zynq UltraScale+ development boards and connects to 4x Raspberry Pi cameras via the Opsero RPi Camera FMC. The Hailo-8 AI accelerator connects to the development board via the FPGA Drive FMC Gen4 or the M.2 M-key Stack FMC depending on the target design (see list of target designs below).
A detailed description of this design and how to use it was written up in this blog post: Multi-camera YOLOv5 on Zynq UltraScale+ with Hailo-8 AI Acceleration
Important links:
- The user guide for these reference designs
- The RPi Camera FMC datasheet
- The FPGA Drive FMC Gen4 datasheet (for the
zcu106
target design only) - The M.2 M-key Stack FMC datasheet
- To report an issue
- For technical support: Contact Opsero
This project is designed for version 2022.1 of the Xilinx tools (Vivado/Vitis/PetaLinux). If you are using an older version of the Xilinx tools, then refer to the release tags to find the version of this repository that matches your version of the tools.
In order to test this design on hardware, you will need the following:
- Vivado 2022.1
- Vitis 2022.1
- PetaLinux Tools 2022.1
- 1x Hailo-8 M.2 AI Acceleration Module
- 4x Raspberry Pi Camera Module 2
- 1x RPi Camera FMC
- 1x FPGA Drive FMC Gen4 (for
zcu106
target design) - 1x M.2 M-key Stack FMC (for all other designs)
- 1x DisplayPort monitor that supports 2560 x 1440 resolution video
- Alternatively, 1x HDMI monitor that supports 2560 x 1440 resolution and DP-to-HDMI adapter
- 1x of the supported target boards listed below
Note that there are two target designs for the ZCU106 board: zcu106
and zcu106_hpc0
, and the
differences are explained in the table below.
All target designs except zcu106
require the M.2 M-key Stack FMC as the M.2 adapter for the Hailo-8, with the
RPi Camera FMC stacked on top of it.
Target board | Target design | FMC slots used | Cameras | M.2 adapter for Hailo | M.2 active slots |
---|---|---|---|---|---|
ZCU104 | zcu104 |
LPC | 4 | M.2 M-key Stack FMC | 1 |
ZCU106 | zcu106 |
HPC0+HPC1 (note 1) | 4 | FPGA Drive FMC Gen4 | 1 |
ZCU106 | zcu106_hpc0 |
HPC0 | 4 | M.2 M-key Stack FMC | 2 (note 3) |
PYNQ-ZU | pynqzu |
LPC | 2 (note 2) | M.2 M-key Stack FMC | 1 |
Genesys-ZU | genesyszu |
LPC | 2 (note 2) | M.2 M-key Stack FMC | 1 |
UltraZed EV carrier | uzev |
HPC | 4 | M.2 M-key Stack FMC | 2 (note 3) |
- The
zcu106
target design uses the FPGA Drive FMC Gen4 as the M.2 adapter for the Hailo-8. In that design, the FPGA Drive FMC Gen4 connects to HPC1 while the RPi Camera FMC connects to the HPC0 connector. - The
pynqzu
andgenesyszu
target designs have video pipelines for only 2 cameras (CAM1 and CAM2 as labelled on the RPi Camera FMC). This is due to the resource limitations of the devices on these boards. - The
zcu106_hpc0
anduzev
target designs have support for 2x M.2 modules. To use the Hailo demo scripts, at least one of these modules must be the Hailo-8 M.2 AI Acceleration Module. The second slot can be used for a second Hailo module, or an NVMe SSD for storage.
This repo contains submodules. To clone this repo, run:
git clone --recursive https://github.com/fpgadeveloper/zynqmp-hailo-ai.git
Source Vivado and PetaLinux tools:
source <path-to-petalinux>/2022.1/settings.sh
source <path-to-vivado>/2022.1/settings64.sh
Build all (Vivado project, accelerator kernel and PetaLinux):
cd zynqmp-hailo-ai/PetaLinux
make petalinux TARGET=uzev
We strongly encourage community contribution to these projects. Please make a pull request if you would like to share your work:
- if you've spotted and fixed any issues
- if you've added designs for other target platforms
- if you've added software support for other cameras
Thank you to everyone who supports us!
- Test all M.2 M-key Stack FMC based designs on hardware
- Add 2x camera script for PYNQ-ZU board
- Add some demo scripts for VVAS and VCU
Opsero Inc. is a team of FPGA developers delivering FPGA products and design services to start-ups and tech companies. Follow our blog, FPGA Developer, for news, tutorials and updates on the awesome projects we work on.