This implements SuperPoint: Self-Supervised Interest Point Detection and Description in C++ using LibTorch and TorchScript.
matches | tracks |
---|---|
![]() |
![]() |
This C++ implementation of SuperPoint requires LibTorch. The following installation instructions will use vcstool, rosdep and colcon for managing source and binary dependencies and to create a relocatable workspace.
-
CUDA and cuDNN:
- add the official CUDA Toolkit repo for Ubuntu
- install CUDA and cuDNN runtime
sudo apt install cuda-runtime-11-3 libcudnn8
-
vcstool, rosdep and colcon:
sudo pip3 install -U vcstool rosdep colcon-common-extensions sudo rosdep init rosdep update
-
create and compile the colcon workspace:
# create workspace folder mkdir ~/super_point_ws/ cd ~/super_point_ws/ # download sources vcs import << EOF - git: {local-name: src/torch_cpp, uri: "https://github.com/christian-rauch/torch_cpp.git"} - git: {local-name: src/super_point_inference, uri: "https://github.com/christian-rauch/super_point_inference.git"} EOF # resolve binary dependencies rosdep install --from-paths src --ignore-src -y # build workspace colcon build --cmake-args -D CMAKE_BUILD_TYPE=Release -D CMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
The torch_cpp
and super_point_inference
packages will be installed to ~/super_point_ws/install/
. After sourcing the workspace via source ~/super_point_ws/install/setup.bash
those packages will be discoverable by CMake.
The super_point_inference
contains the original model and weights converted via TorchScript and an example program to extract keypoints and find matches in a sequence of images. The converted model and weights are located at ~/super_point_ws/install/super_point_inference/share/weights/SuperPointNet.pt
. The example program superpoint_match
takes the path to the converted model file and a list of image paths as argument:
# download example data
wget https://github.com/raw/magicleap/SuperPointPretrainedNetwork/master/assets/icl_snippet/{250,254,258}.png -P /tmp/images/
# source the workspace
source ~/super_point_ws/install/setup.bash
# extract keypoints and matches
superpoint_match ~/super_point_ws/install/super_point_inference/share/weights/SuperPointNet.pt /tmp/images/{250,254,258}.png
The matches between consecutive image pairs will be shown in separate windows (press any key to close) and exported as /tmp/matches-$PAIR.png
and /tmp/tracks-$PAIR.png
.
If you use the original work by Daniel DeTone et al., please cite their work:
@InProceedings{DeTone2018,
author = {DeTone, Daniel and Malisiewicz, Tomasz and Rabinovich, Andrew},
title = {SuperPoint: Self-Supervised Interest Point Detection and Description},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
year = {2018}
}
If you use this implementation, please cite our work:
@ARTICLE{Rauch2022,
author={Rauch, Christian and Long, Ran and Ivan, Vladimir and Vijayakumar, Sethu},
journal={IEEE Robotics and Automation Letters},
title={Sparse-Dense Motion Modelling and Tracking for Manipulation Without Prior Object Models},
year={2022},
volume={7},
number={4},
pages={11394-11401},
doi={10.1109/LRA.2022.3200177}
}