Ganlin Zhang1*
·
Erik Sandström1*
·
Youmin Zhang2,3
·
Manthan Patel1
Luc Van Gool1,4,5
·
Martin R. Oswald1,6
1 ETH Zurich, 2 University of Bologna, 3 Rock Universe, 4 KU Leuven, 5 INSAIT, 6 University of Amsterdam
(* Equal Contribution)
GlORIE-SLAM produces accurate dense geometry and camera tracking on large-scale indoor scenes thanks to our deformable scene representation and DSPO layer.
GlORIE-SLAM uses a deformable point cloud as the scene representation and achieves lower trajectory error and higher rendering accuracy compared to competitive approaches.
GlORIE-SLAM Architecture.
Table of Contents
- Clone the repo using the
--recursive
flag
git clone --recursive https://github.com/zhangganlin/GlORIE-SLAM.git
cd GlOIRE-SLAM
- Creating a new conda environment using the provided
glorie_env.yaml
file.
Notice that if you are running GlORIE-SLAM on a cluster GPU without a display, we recommend installing the headless version of Open3D which is required to evaluate the depth L1 metric of the reconstructed mesh. This requires compiling Open3D from scratch. If that, remove the Open3D dependency from the
glorie_env.yaml
file. The code is tested with Open3D 0.16.0.
conda env create -f glorie_env.yaml
conda activate glorie-slam
- Compile and install thirdparty libraries.
pip install thirdparty/evaluate_3d_reconstruction_lib
python setup.py install
- Download pretrained model.
Download the pretained models from Google Drive, unzip them inside the
pretrained
folder.
[Directory structure of pretrianed (click to expand)]
.
└── pretrained
├── .gitkeep
├── droid.pth
├── middle_fine.pt
└── omnidata_dpt_depth_v2.ckpt
Download the data as below and the data is saved into the ./datasets/Replica
folder. Note that the Replica data is generated by the authors of iMAP (but hosted by the authors of NICE-SLAM). Please cite iMAP if you use the data.
bash scripts/download_replica.sh
To be able to evaluate the reconstruction error, download the ground truth Replica meshes where unseen region have been culled.
bash scripts/download_cull_replica_mesh.sh
bash scripts/download_tum.sh
DATAROOT is ./datasets
by default. Please change the input_folder
path in the scene specific config files if stored somewhere else on your machine.
Please follow the data downloading procedure on the ScanNet website, and extract color/depth frames from the .sens
file using this code.
[Directory structure of ScanNet (click to expand)]
DATAROOT is ./datasets
by default. If a sequence (sceneXXXX_XX
) is stored in other places, please change the input_folder
path in the config file or in the command line.
DATAROOT
└── scannet
└── scene0000_00
└── frames
├── color
│ ├── 0.jpg
│ ├── 1.jpg
│ ├── ...
│ └── ...
├── depth
│ ├── 0.png
│ ├── 1.png
│ ├── ...
│ └── ...
├── intrinsic
└── pose
├── 0.txt
├── 1.txt
├── ...
└── ...
We use the following sequences:
scene0000_00
scene0054_00
scene0059_00
scene0106_00
scene0169_00
scene0181_00
scene0207_00
scene0233_00
For running GlORIE-SLAM, we recommend using weights and biases for the logging. This can be turned on by setting the wandb
flag to True in the configs/*.yaml
file. Also make sure to specify the path wandb_folder
. If you don't have a wandb account, first create one. Each scene has a config folder, where the input_folder
,output
paths need to be specified. Below, we show some example run commands for one tiny scene from each dataset.
To run GlORIE-SLAM on the office0
scene, run the following command.
python run.py configs/Replica/demo_office0.yaml
After reconstruction, the trajectory error will be evaluated and so will the mesh accuracy along with the rendering metrics.
To run GlORIE-SLAM on the freiburg3_office
scene, run the following command.
python run.py configs/TUM_RGBD/demo_freiburg3_office.yaml
After reconstruction, the trajectory error will be evaluated automatically.
To run GlORIE-SLAM on the scene0000_00
scene, run the following command.
python run.py configs/Scannet/demo_scene0000.yaml
After reconstruction, the trajectory error will be evaluated automatically.
Our GlORIE-SLAM pipeline uses two processes for tracking and mapping, and it is possible to run tracking only without mapping/rendering. In case you only want to test the DSPO
part of our paper, add --only_tracking
in each of the above commands.
python run.py configs/Replica/demo_office0.yaml --only_tracking
python run.py configs/TUM_RGBD/demo_freiburg3_office.yaml --only_tracking
python run.py configs/Scannet/demo_scene0000.yaml --only_tracking
Our codebase is partially based on Point-SLAM, GO-SLAM and DROID-SLAM, we thank the authors for making these codebases publicly available. Our work would not have been possible without your great efforts!
There may be minor differences between the released codebase and the results reported in the paper. Further, we note that the GPU hardware has an influence, despite running the same seed and conda environment.
If you find our code or paper useful, please cite
@article{zhang2024glorie,
title={Glorie-slam: Globally optimized rgb-only implicit encoding point cloud slam},
author={Zhang, Ganlin and Sandstr{\"o}m, Erik and Zhang, Youmin and Patel, Manthan and Van Gool, Luc and Oswald, Martin R},
journal={arXiv preprint arXiv:2403.19549},
year={2024}
}
Contact Ganlin Zhang and Erik Sandström for questions, comments and reporting bugs.