Skip to content

aashishrai3799/Towards-Realistic-Generative-3D-Face-Models

Repository files navigation

Towards Realistic Generative 3D Face Models (AlbedoGAN)

Aashish Rai, Hiresh Gupta*, Ayush Pandey*, Francisco Vicente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, Fernando de la Torre

Carnegie Mellon University, Meta Reality Labs

WACV 2024

[Project Page] [Arxiv] [Demo Results]

We propose a 3D face generative model that generates high-quality albedo and precise 3D shape by leveraging StyleGAN2, resulting in a photo-realistic rendered image.

Inference

Conda environment: Refer environment.yml

Download pre-trained models and put in the respective folders.

Follow [MICA] to download insightface and MICA pre-trained models. Put the weights in 'insightface' and 'data/mica_pretrained' folders, respectively. Follow [DECA] to download DECA pre-trained weights. Put them in the 'data' folder.

Download AlbedoGAN modified weights from the following [LINK]. Put these modified ArcFace backbone and DECA weights to generate better reconstruction results.

  • Generate Random 3D Faces (mesh and texture)

    python demos/demo_generate.py
    
  • Reconstruct 3D Faces from 2D Images

    python demos/demo_reconstruct.py
    
  • Generate multi-pose videos

    python video.py
    

Training code will be released at the earliest convenience.

Acknowledgements

For functions or scripts that are based on external sources, we acknowledge the origin individually in each file. Our code is inspired by the following code repositories. Please make sure to refer to their and our license terms before downloading the pre-trained weights.

  1. [DECA]
  2. [MICA]
  3. [FLAME]

License Terms

The code is available under X11 License. Please read the license terms available at [Link]. Quick summary available at [Link].

Citation

If you find this paper/code useful, please consider citing:

@article{rai2023towards,
  		title={Towards Realistic Generative 3D Face Models},
  		author={Rai, Aashish and Gupta, Hiresh and Pandey, Ayush and Carrasco, Francisco Vicente and Takagi, Shingo Jason and Aubel, Amaury and Kim, Daeil and Prakash, Aayush and De la Torre, Fernando},
  		journal={arXiv preprint arXiv:2304.12483},
  		year={2023}
 		}

About

3D face model that can generate high-quality mesh and texture

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published