In IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR), 2022
For more details: Project page / Arxiv(to be updated)
We present a method for generating large amounts of color/depth training data from abundant internet 360 videos. After creating a large-scale general omnidirectional dataset, Depth360, we propose an end-to-end two-branch multitasking network, SegFuse to learn single-view depth estimation from it. Our method shows dense, consistent and detailed predictions.
Qualitative results of the proposed method. Our method generates globally consistent estimation and sharper results at local regions. Detailed comparisons with state-of-the-art can be found in the paper.
- Python (tested on 3.7.4)
- PyTorch (tested on 1.6.0)
- Other dependencies
pip install -r requirements.txt
First clone our repo:
git clone https://github.com/HAL-lucination/segfuse.git
cd segfuse
The Depth360 dataset includes 30000 pairs of color and depth images generated with the test-time training method described in the paper. Depth360 dataset link
Download the pretrained model and put in the save folder:
mkdir save
This work is licensed under MIT License. See LICENSE for details.
@InProceedings{
}
This will be updated after the conference.
We appreciate the anonymous reviewers for their valuable feedback. This research was supported by JST-Mirai Program (JPMJMI19B2), JSPS KAKENHI (19H01129, 19H04137, 21H0504) and the Royal Society (IES\ R2\ 181024).