Squeezing Multiple Cues from a Single Image for Clothed 3D Human Reconstruction

Leyuan Liu . Jianchi Sun . Yunqi Gao . Jingying Chen

National Engineering Research Center for E-Learning, Central China Normal University

Submitted to The Visual Computer

This repository contains a PyTorch implementation of "Squeezing Multiple Cues from a Single Image for Clothed 3D Human Reconstruction".

NOTE: The current model is trained in a low-resolution dataset called THuman1.0, which contains simple poses and clothes. Fortunately, our method achieves state-of-the-art performance on human images with complex poses and loose clothes.

The code is released, but not complete. We are still updating it.

NOTE 2022/11/25: Our paper has been submitted to IEEE Transaction on Multimedia. Good luck. This page shows more results generated by our method on in-the-wild images collected from the Internet.

NOTE 2022/12/17: The core code and pre-trained models have been uploaded. They can be downloaded from here. Install instructions and commands for training the network (directly or incrementally) will be updated and uploaded soon.

License

Code and pre-trained models are available for non-commercial research purposes as defined in the LICENSE file. By downloading and using the code and model you agree to the terms in the LICENSE.

Citation

@article{MCHuman,
  title     = {Squeezing Multiple Cues from a Single Image for Clothed 3D Human Reconstruction},
  author    = {Liu, Leyuan and Sun, Jianchi and Gao, Yunqi and Chen, Jingying.},
}

Environments

This code has been tested with PyTorch 1.4.0 and CUDA 10.1. on Ubuntu 18.04.

Installation

Please run the following commands to download the necessary assets (including the pre-trained models):

Coming Soon！

You can download our provided pre-trained models from Download link and put them into the ./checkpoints directory.

Run Demo

(1) Testing on a single image without SMPL model

python Demo.py -- input_dir img_path

e.g., python Demo.py -- input_dir ./img.png

(2) Testing on a single image with the ground-truth SMPL model

python Demo.py -- input_dir img_path -- SMPL_dir gtSMPL_path

e.g., python Demo.py -- input_dir ./img.png -- SMPL_dir ./gtSMPL.pkl

Train the network directly using all cues

Please run the following command to train the network: Coming Soon！

bash 

Train the network incrementally by plugging and unplugging cues

Please run the following command to train the network: Coming Soon！

bash 

Acknowledgments

Note that some code of this repo is based on GeoPIFu, PyMAF, pix2pixHD, and OpenPose. We thank the authors for their great work!

Contact

• If you have any trouble when using this repo, please do not hesitate to send an E-mail to Jianchi Sun (sunjc0306@qq.com).