This tool performs 6DoF Pose estimation for shelf and table-top environments using multi-view RGB-D images. You get the option to use Faster-RCNN or FCN for object segmentation. It also gives option to use PCA and Super4PCS for computing pose estimates. Finally as a post processing one could chose from it performs ICP and physical reasoning (optional).
-
install
Matlab Robotics toolbox
-
execute in matlab
path-to-repo/ros-packages/src/pose_estimation/src/make.m
-
setup
caffe
for Faster-RCNN -
setup
caffe
for FCN -
install Blender
-
realsense camera setup
-
run
cd path-to-repo/ros-packages/src
-
run
catkin_init_workspace
-
run
cd ../
-
run
catkin_make
-
run
cd src/super4pcs
-
run
mkdir build && cd build
-
run
cmake -DCMAKE_BUILD_TYPE=Release -DANN_DIR=$PWD/../ann_1.1.2/ ..
-
run
make
in case dependecies are not installed refer to Super4PCS installation
-
run
cd path-to-repo/ros-packages/src/detection_package/lib
-
run
make
-
Add the following to
~/.bashrc
:-export PHYSIM_6DPose_PATH=path to PHYSIM_6DPose repository
export BLENDER_PATH=path to blender
-
download rcnn model from this webpage and store it in
$PHYSIM_6DPose_PATH/ros-packages/src/detection_package/data/faster_rcnn_models/
-
run
cd $PHYSIM_6DPose_PATH
-
run
./runMaster.sh
-
execute in matlab
ros-packages/src/pose_estimation/src/poseServiceStart.m
-
run
rosservice call /pose_estimation "path-to-tmp-directory" "path-to-calibration-folder"
-
run
robot.launch
(specific to Rutgers) which publishes the realsense camera pose. -
run
rosrun marvin_convnet save_images _write_directory:="path-to-some-tmp-directory" _camera_service_name:="/realsense_camera"
-
run
rosservice call /save_images ["expo_dry_erase_board_eraser","other-object-names"] binId frameId
(for table top you can use 13 as thebin id
and for shelf from1-12
) -
run
rosservice call /pose_estimation "path-to-tmp-directory" "path-to-calibration-folder"
-
Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge : Andy Zeng, Kuan-Ting Yu, Shuran Song, Daniel Suo, Ed Walker Jr., Alberto Rodriguez and Jianxiong Xiao
-
Super4PCS: Fast Global Pointcloud Registration via Smart Indexing : Mellado, Nicolas and Aiger, Dror and Mitra, Niloy J.
-
Fast Global registration: Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun