Skip to content
/ MOVER Public

The official repository for [CVPR2022] MOVER: Human-Aware Object Placement for Visual Environment Reconstruction.

License

Notifications You must be signed in to change notification settings

yhw-yhw/MOVER

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Human-Aware Object Placement for Visual Environment Reconstruction. (CVPR2022)

[Project Page] [Paper] [MPI Project Page] [Youtube Video]

drawing

drawing
3D Scene and Humans Reconstruction Results from a single RGB video

What Can You Learn from MOVER?

  • 3D Scene Initialization with water-tight mesh (benefit from Total3DUnderstand and OccupancyNet).
  • Single Video Batch-wise SMPLify-X for single person.
  • Ground-Plane & Camera Orientation Optimization with human contacted feet.
  • Three HSIs Constraints: the ordering depth, collision and contact.

Installation

Please follow the Installation Instruction to setup all the required packages.


Data

Please register SMPL-X at first, and then download smpl-x_model.tar.gz from our webpage, put it under ${MOVER_REPORSITORY}/data/.

We provide demo sequences and MOVER reconstructed humans and 3D scenes of PROX qualitative and quantitative in our webpage.

See more details in data document


Get Started

We provide the core part of MOVER, use three different kinds of HSI constraints (depth, collision, and contact) to help understand the 3D scene.

Scene optimization with 2D cues and HSIs:

cd ./demo
bash run.sh

We also provide the visualization for the final reconstructed 3D scenes and 3D humans.

Visualize reconstructed 3D humans and 3D scene results:

cd ./demo
bash run_rendering.sh

Scene Initialization

See more details in scene initialization document

Human Pose ans Shape (HPS) Initialization

See more details in HPS initialization document


Citation

@inproceedings{yi2022mover,
title = {Human-Aware Object Placement for Visual Environment Reconstruction},
author = {Yi, Hongwei and Huang, Chun-Hao P. and Tzionas, Dimitrios and Kocabas, Muhammed and 
  Hassan, Mohamed and Tang, Siyu and Thies, Justus and Black, Michael J.},
booktitle = {Computer Vision and Pattern Recognition (CVPR)},
month = jun,
year = {2022},
month_numeric = {6}}

Acknowledgments

We thank Yixin Chen, Yuliang Xiu for the insightful discussions, Yao Feng, Partha Ghosh and
Maria Paola Forte for proof-reading, and Benjamin Pellkofer for IT support. This work was supported by the German Federal Ministry of Education and Research (BMBF): Tubingen AI Center, FKZ: 01IS18039B.

License

This code and model are available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using the code and model you agree to the terms in the LICENSE.

Contact

For more questions, please contact mover@tue.mpg.de

For commercial licensing, please contact ps-licensing@tue.mpg.de

About

The official repository for [CVPR2022] MOVER: Human-Aware Object Placement for Visual Environment Reconstruction.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published