Software for visualizing drone mapping environments using HoloLens2. I am using this software for my Master's research, please cite the paper when using the program.
It is developed using ROS Melodic, ROS-SHARP, and Unity.
The source code is released under GPLv3 license.
refer to how using ros command.
Brief steps on installing the Require packages.
If you need any help, I suggest you refer to it.
A brief description of how to set up to HoloLens2 (Unity).
Note: if u use a virtual-box, you must connect TELLO-XXXXX before launch Ubuntu.
roslaunch flock_driver orbslam2_with_cloud_map.launch
sudo apt-get install ros-melodic-rosbridge-suite
roslaunch rosbridge_server rosbridge_websocket.launch
roslaunch pcl_ros_processing pcl_downsampling.launch
We have tested the library in **Ubuntu 18.04.
basically have all the necessary libraries, so move to the folder and install each package
cd ~/tello-orbslam2-ros-hololens2/Requirements/
Install what you need
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros-latest.list'
wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
sudo apt-get update
sudo apt-get install python-catkin-tools
sudo apt install ffmpeg
sudo apt-get install python-imaging-tk
sudo apt-get install python-pygame
We use Pangolin for visualization and user interface. Dowload and install instructions can be found at: https://github.com/stevenlovegrove/Pangolin.
sudo apt install libgl1-mesa-dev
sudo apt install libglew-dev
sudo apt-get install libxkbcommon-dev
cd Pangolin
mkdir build
cd build
cmake ..
cmake --build .
sudo make install
pybind11 is a lightweight header-only library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code.
cd pybind
mkdir build
cmake ..
make -j6
sudo make install
Required by g2o (see below). Download and install instructions can be found at: http://eigen.tuxfamily.org. Required at least 3.1.0.
cd eigen-3.3.7
mkdir build
cd build
cmake ..
# make check (not definitely)
sudo make install
We use modified versions of the DBoW2 library to perform place recognition and g2o library to perform non-linear optimizations. Both modified libraries (which are BSD) are included in the Thirdparty folder.
cd DBoW2
mkdir build
cd build
cmake ..
make
sudo make install
cd h264decoder
mkdir build
cd build
cmake ..
make
now copy it to python path
sudo cp ~/ROS/h264decoder/libh264decoder.so /usr/local/lib/python2.7/dist-packages
We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org. Required at OpenCV 3.4.0.
# root authority
sudo su -
apt-get update
apt-get upgrade
# Add repository to include libjasper-dev
add-apt-repository "deb http://security.ubuntu.com/ubuntu bionic-security main"
# install dependent packages
apt-get -y install build-essential checkinstall cmake unzip pkg-config yasm
apt-get -y install git gfortran python3-dev
apt-get -y install libjpeg8-dev libjasper-dev libpng12-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine2-dev libv4l-dev
apt-get -y install libjpeg-dev libpng-dev libtiff-dev libtbb-dev
apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libatlas-base-dev libxvidcore-dev libx264-dev libgtk-3-dev
# Specify the latest version from the tag
cd /usr/local/src
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
cd opencv_contrib
# change the version
cd opencv_contrib
git checkout -b 3.4.0 refs/tags/3.2.0
cd ../opencv/
git checkout -b 3.4.0 refs/tags/3.2.0
# Build and Install
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_PRECOMPILED_HEADERS=OFF -DWITH_FFMPEG=ON -D WITH_CUDA=ON -DWITH_NVCUVID=ON -DBUILD_opencv_cudacodec=OFF -DCUDA_GENERATION=Pascal ..
// cmake -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_PRECOMPILED_HEADERS=OFF -DWITH_FFMPEG=ON ..
make
make install
echo /usr/local/lib > /etc/ld.so.conf.d/opencv.conf
ldconfig -v
# Check the OpenCV Version
opencv_version
exit
based on https://github.com/dji-sdk/Tello-Python and https://github.com/hanyazou/TelloPy
cd ~/drone-slam-octomap/Tello_ROS_ORBSLAM/TelloPy
sudo python setup.py install
based on https://github.com/appliedAI-Initiative/orb_slam_2_ros and https://github.com/rayvburn/ORB-SLAM2_ROS and https://github.com/tau-adl/Tello_ROS_ORBSLAM
cd ~/tello-orbslam2-ros-hololens2/Tello_ROS_ORBSLAM/ROS/tello_catkin_ws/
catkin init
catkin clean
catkin build --mem-limit 70% -j1
echo "source $PWD/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
rosbridge allows dynamic socket and web-socket based access to the full capabilities of ROS.
sudo apt-get install ros-melodic-rosbridge-server
roslaunch rosbridge_server rosbridge_websocket.launch
pcl_ros_processing allows downsampling pointcloud from ros-orb-slam2.
cd pc_ws
catkin init
catkin build
echo "source $PWD/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
Unzipping below files
tello-orbslam2-ros-hololens2/Unity Projects/ar-ros-package.zip
tello-orbslam2-ros-hololens2/Unity Projects/mrtk-ros-droneImage-package.zip
and please import these package files.
You must change IP address to yours IP address in below file.
tello-orbslam2-ros-hololens2/Unity Projects/ar-ros/Assets/RosSharp/Scripts/RosBridgeClient/RosCommuncation/RosConnector.cs
Note: If you want to change the connection point, check the second pc checkbox (see figure).
wget https://cmake.org/files/v3.14/cmake-3.14.0.tar.gz
tar xvf cmake-3.14.0.tar.gz
cd ~/Downloads/cmake-3.14.0 # or wherever you downloaded cmake
./bootstrap --prefix=$HOME/cmake-install
make
make install
export PATH=$HOME/cmake-install/bin:$PATH
export CMAKE_PREFIX_PATH=$HOME/cmake-install:$CMAKE_PREFIX_PATH
sudo rm /var/lib/apt/lists/lock
sudo rm /var/lib/dpkg/lock
sudo rm /var/lib/dpkg/lock-frontend
cd ~
mkdir eigen_ws
wget https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.tar.gz
tar xvf eigen-3.3.7.tar.gz
cd eigen-3.3.7
mkdir build
cd build
cmake ..
# make check (not definitely)
sudo make install
sudo apt install libsuitesparse-dev - qtdeclarative5-dev - qt5-qmake - libqglviewer-dev
mkdir ~/g2o_ws
cd ~/g2o_ws
git clone https://github.com/RainerKuemmerle/g2o.git
cd g2o
mkdir build
cd build
cmake ..
make
sudo make install
sudo apt-get install libboost-dev
mkdir ~/DBoW2_ws
cd ~/DBoW2_ws
git clone https://github.com/dorian3d/DBoW2.git
cd DBoW2
mkdir build
cd build
cmake ..
make
sudo make install
cd ~/ROS/
git clone --recursive https://github.com/stevenlovegrove/Pangolin.git -b v0.5
sudo apt install libgl1-mesa-dev
sudo apt install libglew-dev
sudo apt-get install libxkbcommon-dev
cd Pangolin
mkdir build
cd build
cmake ..
cmake --build .
sudo make install
git clone https://github.com/pybind/pybind11.git
cd pybind
mkdir build
cmake ..
make -j6
sudo make install
This is research code, it is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of merchantability or fitness for a particular purpose.