Peize Liu, Chen Feng†, Yang Xu, Yan Ning, Hao Xu†, and Shaojie Shen
HKUST Aerial Robotics Group
†Corresponding Authors
- [30/06/2024]: OmniNxt is accepted to IROS 2024.
- [15/08/2024] V0.1 Release
You can access our latest design by using Fusion360.
Projest link: https://a360.co/3vK6dJd
Access code: hkustUAV
All our components can be purchased at Taobao
Component | Specification | Link |
---|---|---|
Quad-Fisheye Camera Set | ||
OAK-FFC-4P | Camera control board | Official / Taobao |
B0335 (OV9782) | Camera modules (210-degree fisheye lens) | Official / Taobao |
Camera cables | Customized | Camera cable SCH |
Onboard computer | ||
Jetson Orin Nx | ||
Jetson Orin Nx carrier board | DM | |
Jetson Orin Nx radiator | ||
Jeston Orin Network card | ||
2230 SSD | ||
Intel AX200 | ||
Flight platforms (6S) | ||
Nxt-FC | Project Page | MicoAir / Taobao |
Flight frame | Customized | Fusion360 |
Protector | Customized / Oddity RC XI35 protector | Fusion360 / Oddity RC |
Motors | 2204 1750KV/ 1804 2450KV | |
6S Battery |
Camera hardware preparation and ROS wrap driver see oak_ffc_4p
Calibrate this camera set see quater-calib
Our Jetson Info:
We suggest you follow these settings to avoid conflict (especially CUDA and tensorRT).
We open-source a coin-size PX4-based flight controller see NXT-FC
Follow the Setting up Nxt-FC part to configure your flight controller.
Our Omni-VINS and Omni-Depth are developed from D2SLAM. Since building the docker images on the local host (Jetson Orin) usually takes a long time, we suggest you pull the images from the docker-hub.
the docker images structure is as follows:
git clone --branch pr_fix_main https://github.com/HKUST-Aerial-Robotics/D2SLAM.git
cd ./D2SLAM/config/quadcam_drone_nxt_tmp
if you have already calibrated your quad fisheye camera set, you can simply replace the files with the same name, which includes:
- stereo_calib_n_m_240_320.yaml (n and m is the camera number; used in Omni-Depth).
- fisheye_cams.yaml. (used in Omni-VINS)
./start_docker.sh is under the ./D2SLAM directory.
Modify the following parameters with the absolute path under your local host environment.
- DATA_SET (where the data set is) (Optional)
Then run with the following command under the D2SLAM directory(very important, this will map your D2SLAM dir into the container):
./start_docker.sh 1
run all algorithm modules together (Omni-VINS & Omni-Depth)
## you should under /root/swarm_ws/
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh
Only launch Omni-VINS, please remove the nodes in the red box
and then
source ./devel/setup.bash
roslaunch d2vins quadcam.laucnh
Only launch Omni-Depth
source ./devel/setup.bash
roslaunch quadcam_depth_est depth-node.launch
[Notice] If you are running Omni-VINS and Omni-Depth for the first time, the initialization time for these two modules would be a little bit long as these two modules would generate the inference engine.
If everything works well, you will see
which means Omni-VINS(D2VINS) initialized normally.
And
for Omni-Depth.
OmniNxt px4 parameter:
We highly suggest you refer to Fast-Lab's PX4 controller PX4-Control
We also provide our yaw-rotation-free version of Fast-Lab's PX4 controller PX4-Control.
Please cite ZJU-Fast-Lab if these modules are useful for your research and project.
We modified some of the strategies in the ego-planner. Our version is here ego-planner-omni-modify
Please cite ZJU-Fast-Lab's ego-planner if these modules are useful for your research and project.